2024-05-13T22:26:10.281 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-13T22:26:10.281 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi --description orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028 --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028/orig.config.yaml 2024-05-13T22:26:10.391 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-13T22:26:10.518 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028 branch: squid description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7705028' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 7273 sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d sleep_before_teardown: 0 subset: 41/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d targets: smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFHHykhW0QJ1+PifbqX+AGbMudNyzBH9SP5TnsP/vfMbeIdaBUKSUoXTD/3T8est43rde8+dlA8rxIGSrWzST3I= smithi130.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOMHgsGBASIxa/qjEYlPJwBIJ6EfSpSn/WcPJ3g1i3lk9XMcKXpOsjyBYt4f8UobW6tsqTC55zpw+MljJwAtL5M= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-13_21:08:02 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-05-13T22:26:10.518 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa; will attempt to use it 2024-05-13T22:26:10.519 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa/tasks 2024-05-13T22:26:10.519 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-13T22:26:10.522 INFO:teuthology.task.internal:Checking packages... 2024-05-13T22:26:10.554 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c3e55145f4f4b3a500bb66ee08c938a215bc231d' 2024-05-13T22:26:10.554 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-13T22:26:10.554 INFO:teuthology.packaging:ref: None 2024-05-13T22:26:10.554 INFO:teuthology.packaging:tag: None 2024-05-13T22:26:10.554 INFO:teuthology.packaging:branch: squid 2024-05-13T22:26:10.554 INFO:teuthology.packaging:sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:26:10.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-13T22:26:10.792 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2565.gc3e55145 2024-05-13T22:26:10.794 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-13T22:26:10.799 INFO:teuthology.task.internal:no buildpackages task found 2024-05-13T22:26:10.800 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-13T22:26:10.806 INFO:teuthology.task.internal:Saving configuration 2024-05-13T22:26:10.908 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-13T22:26:10.923 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-13T22:26:10.961 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-13 22:21:08.553861', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFHHykhW0QJ1+PifbqX+AGbMudNyzBH9SP5TnsP/vfMbeIdaBUKSUoXTD/3T8est43rde8+dlA8rxIGSrWzST3I='} 2024-05-13T22:26:10.985 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-13 22:21:08.555620', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOMHgsGBASIxa/qjEYlPJwBIJ6EfSpSn/WcPJ3g1i3lk9XMcKXpOsjyBYt4f8UobW6tsqTC55zpw+MljJwAtL5M='} 2024-05-13T22:26:10.986 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-13T22:26:10.991 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-05-13T22:26:10.992 INFO:teuthology.task.internal:roles: ubuntu@smithi130.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-05-13T22:26:10.992 INFO:teuthology.run_tasks:Running task console_log... 2024-05-13T22:26:11.162 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1522e290d0>, signals=[15]) 2024-05-13T22:26:11.162 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-13T22:26:11.169 INFO:teuthology.task.internal:Opening connections... 2024-05-13T22:26:11.169 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:26:11.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-13T22:26:11.260 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi130.front.sepia.ceph.com 2024-05-13T22:26:11.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-13T22:26:11.405 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-13T22:26:11.413 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-05-13T22:26:11.430 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-05-13T22:26:11.431 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-05-13T22:26:11.489 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-13T22:26:11.490 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-13T22:26:11.491 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-13T22:26:11.491 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-05-13T22:26:11.520 DEBUG:teuthology.orchestra.run.smithi130:> uname -m 2024-05-13T22:26:11.541 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64 2024-05-13T22:26:11.541 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2024-05-13T22:26:11.599 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="CentOS Stream" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="9" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:ID="centos" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="rhel fedora" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="9" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el9" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:LOGO="fedora-logo-icon" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://centos.org/" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-13T22:26:11.600 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-13T22:26:11.601 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-13T22:26:11.601 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2024-05-13T22:26:11.628 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-13T22:26:11.642 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-13T22:26:11.656 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-13T22:26:11.657 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-05-13T22:26:11.660 DEBUG:teuthology.orchestra.run.smithi130:> test '!' -e /home/ubuntu/cephtest 2024-05-13T22:26:11.679 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-13T22:26:11.686 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-13T22:26:11.686 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-05-13T22:26:11.717 DEBUG:teuthology.orchestra.run.smithi130:> test -z $(ls -A /var/lib/ceph) 2024-05-13T22:26:11.754 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-13T22:26:11.802 INFO:teuthology.run_tasks:Running task kernel... 2024-05-13T22:26:11.854 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-13T22:26:11.855 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-13T22:26:11.855 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-13T22:26:11.862 DEBUG:teuthology.orchestra.run.smithi130:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-13T22:26:11.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-13T22:26:11.879 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-05-13T22:26:11.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-13T22:26:11.881 DEBUG:teuthology.orchestra.run.smithi130:> uname -r 2024-05-13T22:26:11.936 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-437.el9.x86_64 2024-05-13T22:26:11.937 INFO:teuthology.task.kernel:Running kernel on smithi102: 5.14.0-437.el9.x86_64 2024-05-13T22:26:11.937 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum install -y kernel 2024-05-13T22:26:11.939 INFO:teuthology.orchestra.run.smithi130.stdout:5.14.0-437.el9.x86_64 2024-05-13T22:26:11.939 INFO:teuthology.task.kernel:Running kernel on smithi130: 5.14.0-437.el9.x86_64 2024-05-13T22:26:11.939 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum install -y kernel 2024-05-13T22:26:15.655 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-05-13T22:26:17.175 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.1 MB 00:03 2024-05-13T22:26:21.647 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 19 MB 00:04 2024-05-13T22:26:26.883 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.2 MB 00:01 2024-05-13T22:26:28.515 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-05-13T22:26:29.650 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-05-13T22:26:34.858 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-05-13T22:26:35.684 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-05-13T22:26:36.835 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-13T22:26:36.835 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-13T22:26:36.944 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:26:36.951 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:26:36.952 INFO:teuthology.orchestra.run.smithi102.stdout:Install 4 Packages 2024-05-13T22:26:36.952 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:26:36.952 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 78 M 2024-05-13T22:26:36.952 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 123 M 2024-05-13T22:26:36.952 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:26:37.885 INFO:teuthology.orchestra.run.smithi102.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 675 kB/s | 550 kB 00:00 2024-05-13T22:26:41.475 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-05-13T22:26:43.225 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-05-13T22:26:45.048 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-13T22:26:46.076 INFO:teuthology.orchestra.run.smithi102.stdout:(2/4): kernel-modules-core-5.14.0-446.el9.x86_6 3.4 MB/s | 28 MB 00:08 2024-05-13T22:26:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:(3/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 3.5 MB/s | 34 MB 00:09 2024-05-13T22:26:50.251 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-05-13T22:26:52.226 INFO:teuthology.orchestra.run.smithi102.stdout:(4/4): kernel-core-5.14.0-446.el9.x86_64.rpm 1.0 MB/s | 15 MB 00:15 2024-05-13T22:26:52.226 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:26:52.226 INFO:teuthology.orchestra.run.smithi102.stdout:Total 5.1 MB/s | 78 MB 00:15 2024-05-13T22:26:52.262 INFO:teuthology.orchestra.run.smithi130.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-13T22:26:52.263 INFO:teuthology.orchestra.run.smithi130.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-13T22:26:52.350 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:26:52.362 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:26:52.362 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2024-05-13T22:26:52.362 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:26:52.362 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:26:52.362 INFO:teuthology.orchestra.run.smithi130.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout:Install 4 Packages 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 78 M 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 123 M 2024-05-13T22:26:52.363 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:26:52.479 INFO:teuthology.orchestra.run.smithi130.stdout:[MIRROR] kernel-5.14.0-446.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-446.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-13T22:26:52.479 INFO:teuthology.orchestra.run.smithi130.stdout:[MIRROR] kernel-core-5.14.0-446.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-446.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-13T22:26:52.479 INFO:teuthology.orchestra.run.smithi130.stdout:[MIRROR] kernel-modules-5.14.0-446.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-446.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-13T22:26:52.907 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:26:53.137 INFO:teuthology.orchestra.run.smithi130.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 776 kB/s | 550 kB 00:00 2024-05-13T22:26:53.268 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:26:53.269 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:26:54.200 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:26:54.200 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:26:56.602 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:26:57.208 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-13T22:26:57.285 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-13T22:26:59.761 INFO:teuthology.orchestra.run.smithi130.stdout:(2/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 4.6 MB/s | 34 MB 00:07 2024-05-13T22:26:59.954 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-13T22:27:00.098 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-13T22:27:04.820 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-13T22:27:04.926 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:27:08.519 INFO:teuthology.orchestra.run.smithi130.stdout:(3/4): kernel-core-5.14.0-446.el9.x86_64.rpm 986 kB/s | 15 MB 00:16 2024-05-13T22:27:09.670 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:27:25.509 INFO:teuthology.orchestra.run.smithi130.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 886 kB/s | 28 MB 00:32 2024-05-13T22:27:25.509 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:27:25.509 INFO:teuthology.orchestra.run.smithi130.stdout:Total 2.4 MB/s | 78 MB 00:33 2024-05-13T22:27:26.186 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:27:26.530 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:27:26.531 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:27:27.476 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:27:27.476 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:27:29.858 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:27:30.650 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-13T22:27:30.737 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-13T22:27:33.378 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-13T22:27:33.510 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-13T22:27:38.251 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-13T22:27:38.360 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:27:39.045 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:27:39.084 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:27:39.450 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:27:39.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-13T22:27:39.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-13T22:27:39.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-13T22:27:40.291 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:27:40.291 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:27:40.291 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:27:40.291 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-13T22:27:40.292 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-13T22:27:40.292 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-13T22:27:40.292 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-13T22:27:40.292 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:27:40.292 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:27:40.631 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-05-13T22:27:40.710 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-13T22:27:40.711 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-13T22:27:40.711 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-13T22:27:40.711 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-13T22:27:40.711 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-13T22:27:40.711 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-05-13T22:27:40.711 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi102.front.sepia.ceph.com, path=None, version=distro) 2024-05-13T22:27:40.712 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-05-13T22:27:40.763 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-13T22:27:40.763 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-13T22:27:40.763 DEBUG:teuthology.orchestra.run.smithi102:> sudo rpm -qi grub2-tools 2024-05-13T22:27:40.794 INFO:teuthology.orchestra.run.smithi102.stdout:Name : grub2-tools 2024-05-13T22:27:40.794 INFO:teuthology.orchestra.run.smithi102.stdout:Epoch : 1 2024-05-13T22:27:40.794 INFO:teuthology.orchestra.run.smithi102.stdout:Version : 2.06 2024-05-13T22:27:40.794 INFO:teuthology.orchestra.run.smithi102.stdout:Release : 68.el9 2024-05-13T22:27:40.794 INFO:teuthology.orchestra.run.smithi102.stdout:Architecture: x86_64 2024-05-13T22:27:40.794 INFO:teuthology.orchestra.run.smithi102.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-13T22:27:40.794 INFO:teuthology.orchestra.run.smithi102.stdout:Group : Unspecified 2024-05-13T22:27:40.794 INFO:teuthology.orchestra.run.smithi102.stdout:Size : 8264254 2024-05-13T22:27:40.794 INFO:teuthology.orchestra.run.smithi102.stdout:License : GPLv3+ 2024-05-13T22:27:40.794 INFO:teuthology.orchestra.run.smithi102.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-13T22:27:40.794 INFO:teuthology.orchestra.run.smithi102.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:Packager : builder@centos.org 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:Vendor : CentOS 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:Summary : Support tools for GRUB. 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:Description : 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:hardware devices. 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:27:40.795 INFO:teuthology.orchestra.run.smithi102.stdout:This subpackage provides tools for support of all platforms. 2024-05-13T22:27:40.796 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-13T22:27:40.796 INFO:teuthology.task.kernel:Updating grub on smithi102 to boot 5.14.0-446.el9.x86_64 2024-05-13T22:27:40.797 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-13T22:27:41.411 INFO:teuthology.orchestra.run.smithi102.stderr:Generating grub configuration file ... 2024-05-13T22:27:42.819 INFO:teuthology.orchestra.run.smithi102.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-13T22:27:42.850 INFO:teuthology.orchestra.run.smithi102.stderr:done 2024-05-13T22:27:42.856 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-05-13T22:27:42.872 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.75R7oJv9mm 2024-05-13T22:27:42.873 DEBUG:teuthology.orchestra.run.smithi102:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.75R7oJv9mm 2024-05-13T22:27:42.940 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.75R7oJv9mm 2024-05-13T22:27:43.044 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:27:43.082 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.75R7oJv9mm is 6KB 2024-05-13T22:27:43.131 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.75R7oJv9mm 2024-05-13T22:27:43.146 DEBUG:teuthology.orchestra.run.smithi102:> sudo /bin/ls /boot/loader/entries || true 2024-05-13T22:27:43.207 INFO:teuthology.orchestra.run.smithi102.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-13T22:27:43.208 INFO:teuthology.orchestra.run.smithi102.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-13T22:27:43.208 INFO:teuthology.orchestra.run.smithi102.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-13T22:27:43.208 INFO:teuthology.orchestra.run.smithi102.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-13T22:27:43.209 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-13T22:27:43.357 DEBUG:teuthology.orchestra.run.smithi102:> sudo shutdown -r now 2024-05-13T22:28:12.050 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:28:12.091 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:28:12.401 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:28:12.401 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-13T22:28:12.401 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-13T22:28:12.401 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-13T22:28:13.273 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-13T22:28:13.273 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:28:13.273 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:28:13.273 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-13T22:28:13.273 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-13T22:28:13.274 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-13T22:28:13.274 INFO:teuthology.orchestra.run.smithi130.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-13T22:28:13.274 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:28:13.274 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:28:13.364 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-05-13T22:28:13.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-05-13T22:28:13.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-13T22:28:13.625 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q kernel | sort -rV | head -n 1 2024-05-13T22:28:13.700 INFO:teuthology.orchestra.run.smithi130.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-13T22:28:13.701 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-13T22:28:13.701 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-13T22:28:13.701 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-13T22:28:13.701 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-13T22:28:13.701 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-05-13T22:28:13.701 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi130.front.sepia.ceph.com, path=None, version=distro) 2024-05-13T22:28:13.701 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q kernel | sort -rV | head -n 1 2024-05-13T22:28:13.751 INFO:teuthology.orchestra.run.smithi130.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-13T22:28:13.751 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-13T22:28:13.751 DEBUG:teuthology.orchestra.run.smithi130:> sudo rpm -qi grub2-tools 2024-05-13T22:28:13.782 INFO:teuthology.orchestra.run.smithi130.stdout:Name : grub2-tools 2024-05-13T22:28:13.782 INFO:teuthology.orchestra.run.smithi130.stdout:Epoch : 1 2024-05-13T22:28:13.782 INFO:teuthology.orchestra.run.smithi130.stdout:Version : 2.06 2024-05-13T22:28:13.782 INFO:teuthology.orchestra.run.smithi130.stdout:Release : 68.el9 2024-05-13T22:28:13.782 INFO:teuthology.orchestra.run.smithi130.stdout:Architecture: x86_64 2024-05-13T22:28:13.782 INFO:teuthology.orchestra.run.smithi130.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:Group : Unspecified 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:Size : 8264254 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:License : GPLv3+ 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:Packager : builder@centos.org 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:Vendor : CentOS 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:Summary : Support tools for GRUB. 2024-05-13T22:28:13.783 INFO:teuthology.orchestra.run.smithi130.stdout:Description : 2024-05-13T22:28:13.784 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:28:13.784 INFO:teuthology.orchestra.run.smithi130.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-13T22:28:13.784 INFO:teuthology.orchestra.run.smithi130.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-13T22:28:13.784 INFO:teuthology.orchestra.run.smithi130.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-13T22:28:13.784 INFO:teuthology.orchestra.run.smithi130.stdout:hardware devices. 2024-05-13T22:28:13.784 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:28:13.784 INFO:teuthology.orchestra.run.smithi130.stdout:This subpackage provides tools for support of all platforms. 2024-05-13T22:28:13.785 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-13T22:28:13.785 INFO:teuthology.task.kernel:Updating grub on smithi130 to boot 5.14.0-446.el9.x86_64 2024-05-13T22:28:13.785 DEBUG:teuthology.orchestra.run.smithi130:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-13T22:28:14.355 INFO:teuthology.orchestra.run.smithi130.stderr:Generating grub configuration file ... 2024-05-13T22:28:15.735 INFO:teuthology.orchestra.run.smithi130.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-13T22:28:15.765 INFO:teuthology.orchestra.run.smithi130.stderr:done 2024-05-13T22:28:15.768 DEBUG:teuthology.orchestra.run.smithi130:> mktemp 2024-05-13T22:28:15.783 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/tmp.ybDolGhVDp 2024-05-13T22:28:15.783 DEBUG:teuthology.orchestra.run.smithi130:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ybDolGhVDp 2024-05-13T22:28:15.857 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0666 /tmp/tmp.ybDolGhVDp 2024-05-13T22:28:15.990 DEBUG:teuthology.orchestra.remote:smithi130:/tmp/tmp.ybDolGhVDp is 6KB 2024-05-13T22:28:16.049 DEBUG:teuthology.orchestra.run.smithi130:> rm -fr /tmp/tmp.ybDolGhVDp 2024-05-13T22:28:16.064 DEBUG:teuthology.orchestra.run.smithi130:> sudo /bin/ls /boot/loader/entries || true 2024-05-13T22:28:16.125 INFO:teuthology.orchestra.run.smithi130.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-13T22:28:16.125 INFO:teuthology.orchestra.run.smithi130.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-13T22:28:16.125 INFO:teuthology.orchestra.run.smithi130.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-13T22:28:16.125 INFO:teuthology.orchestra.run.smithi130.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-13T22:28:16.127 DEBUG:teuthology.orchestra.run.smithi130:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-13T22:28:16.274 DEBUG:teuthology.orchestra.run.smithi130:> sudo shutdown -r now 2024-05-13T22:28:46.317 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-05-13T22:28:46.317 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-05-13T22:28:46.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-13T22:28:48.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-05-13T22:28:57.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-05-13T22:28:57.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-13T22:29:15.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-05-13T22:29:20.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.130 2024-05-13T22:29:27.430 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-05-13T22:29:27.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-13T22:29:27.733 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-05-13T22:29:28.190 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-05-13T22:29:28.190 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-446.el9.x86_64"... 2024-05-13T22:29:28.191 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-05-13T22:29:28.205 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-446.el9.x86_64 2024-05-13T22:29:28.206 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-13T22:29:28.206 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-13T22:29:28.206 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-13T22:29:29.208 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-05-13T22:29:29.208 DEBUG:teuthology.orchestra.run.smithi102:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-13T22:29:29.282 INFO:teuthology.orchestra.run.smithi102.stdout:ttyS1 2024-05-13T22:29:29.314 DEBUG:teuthology.parallel:result is None 2024-05-13T22:29:29.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-05-13T22:29:29.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-13T22:29:48.191 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.130 2024-05-13T22:30:00.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-05-13T22:30:00.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-13T22:30:00.621 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-05-13T22:30:01.135 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-05-13T22:30:01.135 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-446.el9.x86_64"... 2024-05-13T22:30:01.136 DEBUG:teuthology.orchestra.run.smithi130:> uname -r 2024-05-13T22:30:01.152 INFO:teuthology.orchestra.run.smithi130.stdout:5.14.0-446.el9.x86_64 2024-05-13T22:30:01.152 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-13T22:30:01.152 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-13T22:30:01.152 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-13T22:30:02.153 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-05-13T22:30:02.153 DEBUG:teuthology.orchestra.run.smithi130:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-13T22:30:02.252 INFO:teuthology.orchestra.run.smithi130.stdout:ttyS1 2024-05-13T22:30:02.295 DEBUG:teuthology.parallel:result is None 2024-05-13T22:30:02.295 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-13T22:30:02.306 INFO:teuthology.task.internal:Creating test directory... 2024-05-13T22:30:02.306 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-13T22:30:02.309 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-13T22:30:02.326 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-13T22:30:02.336 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-13T22:30:02.397 INFO:teuthology.task.internal:Creating archive directory... 2024-05-13T22:30:02.398 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-13T22:30:02.402 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-13T22:30:02.433 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-13T22:30:02.440 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-13T22:30:02.441 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-13T22:30:02.464 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-13T22:30:02.494 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-13T22:30:02.499 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-13T22:30:02.507 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-13T22:30:02.509 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-13T22:30:02.511 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-13T22:30:02.519 INFO:teuthology.task.internal:Configuring sudo... 2024-05-13T22:30:02.519 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-13T22:30:02.555 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-13T22:30:02.583 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-13T22:30:02.596 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-13T22:30:02.596 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-13T22:30:02.626 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-13T22:30:02.642 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-13T22:30:02.731 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-13T22:30:02.779 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-13T22:30:02.862 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-13T22:30:02.908 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:30:02.909 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-13T22:30:02.974 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-13T22:30:03.019 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-13T22:30:03.061 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-13T22:30:03.143 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-13T22:30:03.184 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:30:03.184 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-13T22:30:03.249 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-05-13T22:30:03.252 DEBUG:teuthology.orchestra.run.smithi130:> sudo service rsyslog restart 2024-05-13T22:30:03.317 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-13T22:30:03.335 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-13T22:30:03.716 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-13T22:30:03.724 INFO:teuthology.task.internal:Starting timer... 2024-05-13T22:30:03.724 INFO:teuthology.run_tasks:Running task pcp... 2024-05-13T22:30:03.733 INFO:teuthology.run_tasks:Running task selinux... 2024-05-13T22:30:03.745 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-13T22:30:03.746 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-05-13T22:30:03.833 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-05-13T22:30:03.835 DEBUG:teuthology.orchestra.run.smithi130:> sudo service auditd rotate 2024-05-13T22:30:03.901 INFO:teuthology.orchestra.run.smithi130.stdout:Rotating logs: 2024-05-13T22:30:03.903 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-13T22:30:03.903 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-05-13T22:30:03.932 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-05-13T22:30:03.933 DEBUG:teuthology.orchestra.run.smithi130:> /usr/sbin/getenforce 2024-05-13T22:30:03.953 INFO:teuthology.orchestra.run.smithi130.stdout:Permissive 2024-05-13T22:30:03.953 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi102.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi130.front.sepia.ceph.com': 'permissive'} 2024-05-13T22:30:03.953 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-13T22:30:04.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-13T22:30:04.001 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-13T22:30:04.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-13T22:30:04.028 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-13T22:30:04.028 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-05-13T22:30:04.085 DEBUG:teuthology.orchestra.run.smithi130:> sudo /usr/sbin/setenforce permissive 2024-05-13T22:30:04.111 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-13T22:30:04.124 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-13T22:30:04.154 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-13T22:30:04.155 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi102.front.sepia.ceph.com,smithi130.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-13T22:34:51.343 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi102.front.sepia.ceph.com'), Remote(name='ubuntu@smithi130.front.sepia.ceph.com')] 2024-05-13T22:34:51.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-05-13T22:34:51.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-13T22:34:51.425 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-05-13T22:34:51.511 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-05-13T22:34:51.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-05-13T22:34:51.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-13T22:34:51.589 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-05-13T22:34:51.673 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-05-13T22:34:51.674 INFO:teuthology.run_tasks:Running task clock... 2024-05-13T22:34:51.691 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-13T22:34:51.691 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-13T22:34:51.692 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-13T22:34:51.694 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-13T22:34:51.694 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-13T22:34:51.722 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-13T22:34:51.737 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-13T22:34:51.743 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-13T22:34:51.758 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-13T22:34:51.762 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-05-13T22:34:51.773 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-05-13T22:34:51.782 INFO:teuthology.orchestra.run.smithi130.stderr:sudo: ntpd: command not found 2024-05-13T22:34:51.787 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-13T22:34:51.793 INFO:teuthology.orchestra.run.smithi130.stdout:506 Cannot talk to daemon 2024-05-13T22:34:51.801 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-13T22:34:51.807 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-13T22:34:51.821 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-13T22:34:51.857 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-05-13T22:34:51.860 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-13T22:34:51.860 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-05-13T22:34:51.860 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-13T22:34:51.860 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-13T22:34:51.860 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-13T22:34:51.860 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-13T22:34:51.874 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-05-13T22:34:51.877 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-13T22:34:51.877 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-05-13T22:34:51.877 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-13T22:34:51.877 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-13T22:34:51.877 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-13T22:34:51.877 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-13T22:34:51.879 INFO:teuthology.run_tasks:Running task install... 2024-05-13T22:34:51.887 DEBUG:teuthology.task.install:project ceph 2024-05-13T22:34:51.887 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c3e55145f4f4b3a500bb66ee08c938a215bc231d'}} 2024-05-13T22:34:51.887 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': 'c3e55145f4f4b3a500bb66ee08c938a215bc231d'} 2024-05-13T22:34:51.887 INFO:teuthology.task.install:Using flavor: default 2024-05-13T22:34:51.893 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-13T22:34:51.893 INFO:teuthology.task.install:extra packages: [] 2024-05-13T22:34:51.893 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c3e55145f4f4b3a500bb66ee08c938a215bc231d', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-05-13T22:34:51.894 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-13T22:34:51.894 INFO:teuthology.packaging:ref: None 2024-05-13T22:34:51.894 INFO:teuthology.packaging:tag: v18.2.1 2024-05-13T22:34:51.894 INFO:teuthology.packaging:branch: None 2024-05-13T22:34:51.894 INFO:teuthology.packaging:sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:34:54.845 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-05-13T22:34:54.845 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-05-13T22:34:54.851 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c3e55145f4f4b3a500bb66ee08c938a215bc231d', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-05-13T22:34:54.851 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-13T22:34:54.851 INFO:teuthology.packaging:ref: None 2024-05-13T22:34:54.851 INFO:teuthology.packaging:tag: v18.2.1 2024-05-13T22:34:54.851 INFO:teuthology.packaging:branch: None 2024-05-13T22:34:54.852 INFO:teuthology.packaging:sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:34:54.852 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-05-13T22:34:54.999 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-05-13T22:34:54.999 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-05-13T22:34:55.033 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-13T22:34:55.033 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:34:55.033 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-13T22:34:55.065 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-13T22:34:55.065 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-13T22:34:55.065 INFO:teuthology.packaging:ref: None 2024-05-13T22:34:55.065 INFO:teuthology.packaging:tag: v18.2.1 2024-05-13T22:34:55.065 INFO:teuthology.packaging:branch: None 2024-05-13T22:34:55.065 INFO:teuthology.packaging:sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:34:55.066 DEBUG:teuthology.orchestra.run.smithi130:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-13T22:34:55.067 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-05-13T22:34:55.067 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-05-13T22:34:55.092 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-13T22:34:55.092 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:34:55.092 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-13T22:34:55.122 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-13T22:34:55.122 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-13T22:34:55.122 INFO:teuthology.packaging:ref: None 2024-05-13T22:34:55.122 INFO:teuthology.packaging:tag: v18.2.1 2024-05-13T22:34:55.122 INFO:teuthology.packaging:branch: None 2024-05-13T22:34:55.122 INFO:teuthology.packaging:sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:34:55.122 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-13T22:34:55.134 DEBUG:teuthology.orchestra.run.smithi130:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-13T22:34:55.192 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-13T22:34:55.213 DEBUG:teuthology.orchestra.run.smithi130:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-13T22:34:55.270 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-13T22:34:55.279 INFO:teuthology.orchestra.run.smithi130.stdout:check_obsoletes = 1 2024-05-13T22:34:55.280 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum clean all 2024-05-13T22:34:55.336 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-05-13T22:34:55.337 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-05-13T22:34:55.522 INFO:teuthology.orchestra.run.smithi130.stdout:57 files removed 2024-05-13T22:34:55.549 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-radosgw 2024-05-13T22:34:55.570 INFO:teuthology.orchestra.run.smithi102.stdout:57 files removed 2024-05-13T22:34:55.597 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw 2024-05-13T22:34:55.964 INFO:teuthology.orchestra.run.smithi130.stdout:ceph packages for x86_64 543 kB/s | 76 kB 00:00 2024-05-13T22:34:56.009 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 543 kB/s | 76 kB 00:00 2024-05-13T22:34:56.234 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 54 kB/s | 9.4 kB 00:00 2024-05-13T22:34:56.239 INFO:teuthology.orchestra.run.smithi130.stdout:ceph noarch packages 42 kB/s | 9.4 kB 00:00 2024-05-13T22:34:56.415 INFO:teuthology.orchestra.run.smithi130.stdout:ceph source packages 16 kB/s | 2.2 kB 00:00 2024-05-13T22:34:56.418 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 15 kB/s | 2.2 kB 00:00 2024-05-13T22:34:58.727 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 8.1 MB 00:02 2024-05-13T22:34:59.973 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.1 MB 00:03 2024-05-13T22:35:16.719 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-05-13T22:35:18.031 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-05-13T22:35:21.969 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-05-13T22:35:24.270 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-05-13T22:35:25.515 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.2 MB 00:03 2024-05-13T22:35:26.949 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 16 kB 00:00 2024-05-13T22:35:28.578 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-13T22:35:33.718 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-05-13T22:35:36.059 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:35:36.070 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:35:36.070 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-05-13T22:35:36.070 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:35:36.070 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:35:36.070 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrading: 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-13T22:35:36.071 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-05-13T22:35:36.072 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout:Install 28 Packages 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrade 2 Packages 2024-05-13T22:35:36.073 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:35:36.074 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 53 M 2024-05-13T22:35:36.074 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:35:36.394 INFO:teuthology.orchestra.run.smithi130.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 31 MB/s | 5.2 MB 00:00 2024-05-13T22:35:36.410 INFO:teuthology.orchestra.run.smithi130.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 1.4 MB/s | 24 kB 00:00 2024-05-13T22:35:36.444 INFO:teuthology.orchestra.run.smithi130.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 19 MB/s | 658 kB 00:00 2024-05-13T22:35:36.478 INFO:teuthology.orchestra.run.smithi130.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-05-13T22:35:36.595 INFO:teuthology.orchestra.run.smithi130.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 21 MB/s | 7.7 MB 00:00 2024-05-13T22:35:36.620 INFO:teuthology.orchestra.run.smithi130.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-05-13T22:35:36.645 INFO:teuthology.orchestra.run.smithi130.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-05-13T22:35:36.671 INFO:teuthology.orchestra.run.smithi130.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.3 MB/s | 161 kB 00:00 2024-05-13T22:35:36.730 INFO:teuthology.orchestra.run.smithi130.stdout:(9/30): librgw2-18.2.1-0.el9.x86_64.rpm 18 MB/s | 4.5 MB 00:00 2024-05-13T22:35:36.754 INFO:teuthology.orchestra.run.smithi130.stdout:(10/30): python3-rados-18.2.1-0.el9.x86_64.rpm 3.7 MB/s | 321 kB 00:00 2024-05-13T22:35:36.780 INFO:teuthology.orchestra.run.smithi130.stdout:(11/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 5.8 MB/s | 297 kB 00:00 2024-05-13T22:35:36.805 INFO:teuthology.orchestra.run.smithi130.stdout:(12/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2024-05-13T22:35:37.006 INFO:teuthology.orchestra.run.smithi130.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 23 MB/s | 18 MB 00:00 2024-05-13T22:35:37.348 INFO:teuthology.orchestra.run.smithi130.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 188 kB/s | 107 kB 00:00 2024-05-13T22:35:37.373 INFO:teuthology.orchestra.run.smithi130.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 123 kB/s | 45 kB 00:00 2024-05-13T22:35:37.423 INFO:teuthology.orchestra.run.smithi130.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 259 kB/s | 160 kB 00:00 2024-05-13T22:35:37.532 INFO:teuthology.orchestra.run.smithi130.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 389 kB/s | 42 kB 00:00 2024-05-13T22:35:37.715 INFO:teuthology.orchestra.run.smithi130.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 854 kB/s | 292 kB 00:00 2024-05-13T22:35:37.849 INFO:teuthology.orchestra.run.smithi130.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.3 MB/s | 662 kB 00:00 2024-05-13T22:35:38.091 INFO:teuthology.orchestra.run.smithi130.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 198 kB/s | 111 kB 00:00 2024-05-13T22:35:38.291 INFO:teuthology.orchestra.run.smithi130.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 130 kB/s | 26 kB 00:00 2024-05-13T22:35:38.316 INFO:teuthology.orchestra.run.smithi130.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 514 kB/s | 309 kB 00:00 2024-05-13T22:35:38.450 INFO:teuthology.orchestra.run.smithi130.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 307 kB/s | 49 kB 00:00 2024-05-13T22:35:38.567 INFO:teuthology.orchestra.run.smithi130.stdout:(24/30): libarrow-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-05-13T22:35:38.592 INFO:teuthology.orchestra.run.smithi130.stdout:(25/30): libunwind-1.6.2-1.el9.x86_64.rpm 244 kB/s | 67 kB 00:00 2024-05-13T22:35:38.626 INFO:teuthology.orchestra.run.smithi130.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-05-13T22:35:38.701 INFO:teuthology.orchestra.run.smithi130.stdout:(27/30): librados2-18.2.1-0.el9.x86_64.rpm 44 MB/s | 3.3 MB 00:00 2024-05-13T22:35:38.727 INFO:teuthology.orchestra.run.smithi130.stdout:(28/30): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-05-13T22:35:38.785 INFO:teuthology.orchestra.run.smithi130.stdout:(29/30): librbd1-18.2.1-0.el9.x86_64.rpm 36 MB/s | 3.0 MB 00:00 2024-05-13T22:35:38.827 INFO:teuthology.orchestra.run.smithi130.stdout:(30/30): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-05-13T22:35:38.832 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:35:38.832 INFO:teuthology.orchestra.run.smithi130.stdout:Total 19 MB/s | 53 MB 00:02 2024-05-13T22:35:38.908 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:35:38.944 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:35:38.944 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:35:39.511 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:35:39.511 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:35:40.814 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:35:40.878 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-05-13T22:35:40.908 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-05-13T22:35:40.962 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-05-13T22:35:41.325 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-05-13T22:35:41.373 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-05-13T22:35:41.456 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-05-13T22:35:41.545 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-05-13T22:35:41.575 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-05-13T22:35:41.638 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-05-13T22:35:41.674 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-05-13T22:35:41.718 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-05-13T22:35:41.811 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-05-13T22:35:41.835 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-05-13T22:35:41.901 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-05-13T22:35:42.007 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-05-13T22:35:42.043 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-05-13T22:35:42.084 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-05-13T22:35:42.116 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-05-13T22:35:42.143 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-05-13T22:35:42.179 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-05-13T22:35:42.212 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-05-13T22:35:42.518 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-05-13T22:35:42.558 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-05-13T22:35:42.649 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-05-13T22:35:42.680 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-05-13T22:35:42.716 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-05-13T22:35:43.115 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-05-13T22:35:43.153 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-05-13T22:35:43.207 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-05-13T22:35:43.275 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-05-13T22:35:43.305 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-05-13T22:35:43.448 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-05-13T22:35:45.414 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-05-13T22:35:45.454 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-05-13T22:35:46.015 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-05-13T22:35:46.051 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-05-13T22:35:46.150 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-05-13T22:35:46.151 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-13T22:35:46.151 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-13T22:35:46.151 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:35:46.186 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-05-13T22:35:48.629 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-05-13T22:35:50.991 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:35:51.001 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:35:51.001 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-05-13T22:35:51.001 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:35:51.001 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:35:51.001 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-05-13T22:35:51.001 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-05-13T22:35:51.002 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-13T22:35:51.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-05-13T22:35:51.004 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-05-13T22:35:51.004 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-05-13T22:35:51.004 INFO:teuthology.orchestra.run.smithi102.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-13T22:35:51.004 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-13T22:35:51.004 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:35:51.004 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:35:51.004 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:35:51.004 INFO:teuthology.orchestra.run.smithi102.stdout:Install 28 Packages 2024-05-13T22:35:51.004 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 2 Packages 2024-05-13T22:35:51.004 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:35:51.005 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 53 M 2024-05-13T22:35:51.005 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:35:51.530 INFO:teuthology.orchestra.run.smithi102.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 24 MB/s | 5.2 MB 00:00 2024-05-13T22:35:51.547 INFO:teuthology.orchestra.run.smithi102.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 1.4 MB/s | 24 kB 00:00 2024-05-13T22:35:51.589 INFO:teuthology.orchestra.run.smithi102.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 15 MB/s | 658 kB 00:00 2024-05-13T22:35:51.623 INFO:teuthology.orchestra.run.smithi102.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-05-13T22:35:51.731 INFO:teuthology.orchestra.run.smithi102.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 19 MB/s | 7.7 MB 00:00 2024-05-13T22:35:51.748 INFO:teuthology.orchestra.run.smithi102.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 2.6 MB/s | 45 kB 00:00 2024-05-13T22:35:51.773 INFO:teuthology.orchestra.run.smithi102.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-05-13T22:35:51.799 INFO:teuthology.orchestra.run.smithi102.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.3 MB/s | 161 kB 00:00 2024-05-13T22:35:51.824 INFO:teuthology.orchestra.run.smithi102.stdout:(9/30): python3-rados-18.2.1-0.el9.x86_64.rpm 13 MB/s | 321 kB 00:00 2024-05-13T22:35:51.849 INFO:teuthology.orchestra.run.smithi102.stdout:(10/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 12 MB/s | 297 kB 00:00 2024-05-13T22:35:51.916 INFO:teuthology.orchestra.run.smithi102.stdout:(11/30): librgw2-18.2.1-0.el9.x86_64.rpm 15 MB/s | 4.5 MB 00:00 2024-05-13T22:35:51.942 INFO:teuthology.orchestra.run.smithi102.stdout:(12/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 1.1 MB/s | 99 kB 00:00 2024-05-13T22:35:52.142 INFO:teuthology.orchestra.run.smithi102.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 22 MB/s | 18 MB 00:00 2024-05-13T22:35:52.475 INFO:teuthology.orchestra.run.smithi102.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 136 kB/s | 45 kB 00:00 2024-05-13T22:35:52.501 INFO:teuthology.orchestra.run.smithi102.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 183 kB/s | 107 kB 00:00 2024-05-13T22:35:52.559 INFO:teuthology.orchestra.run.smithi102.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 259 kB/s | 160 kB 00:00 2024-05-13T22:35:52.660 INFO:teuthology.orchestra.run.smithi102.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 422 kB/s | 42 kB 00:00 2024-05-13T22:35:52.768 INFO:teuthology.orchestra.run.smithi102.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 2024-05-13T22:35:52.910 INFO:teuthology.orchestra.run.smithi102.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.5 MB/s | 662 kB 00:00 2024-05-13T22:35:52.994 INFO:teuthology.orchestra.run.smithi102.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 332 kB/s | 111 kB 00:00 2024-05-13T22:35:53.119 INFO:teuthology.orchestra.run.smithi102.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 881 kB/s | 309 kB 00:00 2024-05-13T22:35:53.144 INFO:teuthology.orchestra.run.smithi102.stdout:(22/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 173 kB/s | 26 kB 00:00 2024-05-13T22:35:53.195 INFO:teuthology.orchestra.run.smithi102.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 644 kB/s | 49 kB 00:00 2024-05-13T22:35:53.220 INFO:teuthology.orchestra.run.smithi102.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 896 kB/s | 67 kB 00:00 2024-05-13T22:35:53.337 INFO:teuthology.orchestra.run.smithi102.stdout:(25/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 839 kB 00:00 2024-05-13T22:35:53.362 INFO:teuthology.orchestra.run.smithi102.stdout:(26/30): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-05-13T22:35:53.438 INFO:teuthology.orchestra.run.smithi102.stdout:(27/30): librados2-18.2.1-0.el9.x86_64.rpm 44 MB/s | 3.3 MB 00:00 2024-05-13T22:35:53.496 INFO:teuthology.orchestra.run.smithi102.stdout:(28/30): librbd1-18.2.1-0.el9.x86_64.rpm 52 MB/s | 3.0 MB 00:00 2024-05-13T22:35:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:(29/30): libarrow-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-05-13T22:35:53.755 INFO:teuthology.orchestra.run.smithi102.stdout:(30/30): thrift-0.15.0-2.el9.x86_64.rpm 3.8 MB/s | 1.6 MB 00:00 2024-05-13T22:35:53.757 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:35:53.757 INFO:teuthology.orchestra.run.smithi102.stdout:Total 19 MB/s | 53 MB 00:02 2024-05-13T22:35:53.830 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:35:53.864 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:35:53.864 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:35:54.438 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:35:54.438 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:35:55.750 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:35:55.785 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-05-13T22:35:55.785 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /sys 2024-05-13T22:35:55.785 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /proc 2024-05-13T22:35:55.785 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /mnt 2024-05-13T22:35:55.785 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /var/tmp 2024-05-13T22:35:55.785 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /home 2024-05-13T22:35:55.786 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /root 2024-05-13T22:35:55.786 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /tmp 2024-05-13T22:35:55.786 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:35:55.833 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-05-13T22:35:55.860 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-05-13T22:35:55.917 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-05-13T22:35:56.296 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-05-13T22:35:56.343 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-05-13T22:35:56.436 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-05-13T22:35:56.531 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-05-13T22:35:56.557 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-05-13T22:35:56.569 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-05-13T22:35:56.582 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-05-13T22:35:56.582 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:35:56.583 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-13T22:35:56.583 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-13T22:35:56.583 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-13T22:35:56.583 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:35:56.584 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-13T22:35:56.633 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-05-13T22:35:56.669 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-05-13T22:35:56.676 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-13T22:35:56.676 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-13T22:35:56.713 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-05-13T22:35:56.778 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-05-13T22:35:56.805 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-05-13T22:35:56.871 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-05-13T22:35:56.977 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-05-13T22:35:57.021 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-05-13T22:35:57.063 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-05-13T22:35:57.102 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-05-13T22:35:57.129 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-05-13T22:35:57.166 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-05-13T22:35:57.198 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-05-13T22:35:57.498 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-05-13T22:35:57.561 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-05-13T22:35:57.685 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-05-13T22:35:57.716 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-05-13T22:35:57.761 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-05-13T22:35:57.768 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-05-13T22:35:57.769 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-05-13T22:35:57.770 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-05-13T22:35:57.770 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-05-13T22:35:57.770 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-05-13T22:35:57.770 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-05-13T22:35:57.770 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-05-13T22:35:57.770 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-05-13T22:35:58.162 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-05-13T22:35:58.189 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-05-13T22:35:58.261 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-05-13T22:35:58.328 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-05-13T22:35:58.359 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-05-13T22:35:58.661 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-05-13T22:35:58.661 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:35:58.661 INFO:teuthology.orchestra.run.smithi130.stdout:Upgraded: 2024-05-13T22:35:58.661 INFO:teuthology.orchestra.run.smithi130.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.661 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:35:58.661 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-13T22:35:58.661 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: fmt-8.1.1-5.el9.x86_64 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-13T22:35:58.662 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: re2-1:20211101-3.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-13T22:35:58.663 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:35:58.664 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:35:58.994 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-test 2024-05-13T22:35:59.574 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:00:26 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:35:59.694 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:35:59.694 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:35:59.694 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout:Install 4 Packages 2024-05-13T22:35:59.695 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:35:59.696 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 40 M 2024-05-13T22:35:59.696 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 166 M 2024-05-13T22:35:59.696 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:35:59.922 INFO:teuthology.orchestra.run.smithi130.stdout:(1/4): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.5 MB/s | 165 kB 00:00 2024-05-13T22:35:59.998 INFO:teuthology.orchestra.run.smithi130.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-05-13T22:36:00.023 INFO:teuthology.orchestra.run.smithi130.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 634 kB/s | 64 kB 00:00 2024-05-13T22:36:00.440 INFO:teuthology.orchestra.run.smithi130.stdout:(4/4): ceph-test-18.2.1-0.el9.x86_64.rpm 63 MB/s | 40 MB 00:00 2024-05-13T22:36:00.442 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:00.442 INFO:teuthology.orchestra.run.smithi130.stdout:Total 54 MB/s | 40 MB 00:00 2024-05-13T22:36:00.470 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:36:00.490 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:36:00.490 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:36:00.617 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-05-13T22:36:00.647 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-05-13T22:36:00.806 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:36:00.806 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:36:01.201 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-05-13T22:36:01.234 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-05-13T22:36:01.269 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:36:01.331 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-05-13T22:36:01.331 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-13T22:36:01.331 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-13T22:36:01.331 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:01.355 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-13T22:36:01.361 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-05-13T22:36:01.397 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-13T22:36:01.425 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-05-13T22:36:05.151 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-05-13T22:36:05.223 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-05-13T22:36:07.375 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-05-13T22:36:07.375 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/4 2024-05-13T22:36:07.375 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/4 2024-05-13T22:36:07.375 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-13T22:36:07.660 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-13T22:36:07.660 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:07.660 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:36:07.661 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test-2:18.2.1-0.el9.x86_64 libcephsqlite-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:07.661 INFO:teuthology.orchestra.run.smithi130.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-05-13T22:36:07.661 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:07.661 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:36:07.974 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph 2024-05-13T22:36:08.551 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:00:35 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:36:08.675 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:36:08.678 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:08.678 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-05-13T22:36:08.678 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:08.678 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:36:08.678 INFO:teuthology.orchestra.run.smithi130.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-05-13T22:36:08.678 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-05-13T22:36:08.678 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-05-13T22:36:08.678 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-05-13T22:36:08.678 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-05-13T22:36:08.678 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-05-13T22:36:08.678 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-13T22:36:08.679 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-13T22:36:08.680 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-13T22:36:08.681 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:08.681 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:36:08.681 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:08.681 INFO:teuthology.orchestra.run.smithi130.stdout:Install 33 Packages 2024-05-13T22:36:08.681 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:08.681 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 30 M 2024-05-13T22:36:08.681 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 110 M 2024-05-13T22:36:08.682 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:36:09.250 INFO:teuthology.orchestra.run.smithi130.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 60 kB/s | 6.4 kB 00:00 2024-05-13T22:36:09.292 INFO:teuthology.orchestra.run.smithi130.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 9.7 MB/s | 1.4 MB 00:00 2024-05-13T22:36:09.343 INFO:teuthology.orchestra.run.smithi130.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 11 MB/s | 2.1 MB 00:00 2024-05-13T22:36:09.368 INFO:teuthology.orchestra.run.smithi130.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.4 MB/s | 242 kB 00:00 2024-05-13T22:36:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-05-13T22:36:09.677 INFO:teuthology.orchestra.run.smithi130.stdout:(6/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 46 MB/s | 18 MB 00:00 2024-05-13T22:36:09.702 INFO:teuthology.orchestra.run.smithi130.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 758 kB/s | 253 kB 00:00 2024-05-13T22:36:09.752 INFO:teuthology.orchestra.run.smithi130.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-05-13T22:36:09.795 INFO:teuthology.orchestra.run.smithi130.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.9 MB/s | 125 kB 00:00 2024-05-13T22:36:09.820 INFO:teuthology.orchestra.run.smithi130.stdout:(10/33): python3-ply-3.11-14.el9.noarch.rpm 748 kB/s | 106 kB 00:00 2024-05-13T22:36:09.853 INFO:teuthology.orchestra.run.smithi130.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 3.0 MB/s | 1.2 MB 00:00 2024-05-13T22:36:09.879 INFO:teuthology.orchestra.run.smithi130.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-05-13T22:36:10.137 INFO:teuthology.orchestra.run.smithi130.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 122 kB/s | 35 kB 00:00 2024-05-13T22:36:10.238 INFO:teuthology.orchestra.run.smithi130.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 142 kB/s | 51 kB 00:00 2024-05-13T22:36:10.329 INFO:teuthology.orchestra.run.smithi130.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 217 kB/s | 42 kB 00:00 2024-05-13T22:36:10.446 INFO:teuthology.orchestra.run.smithi130.stdout:(16/33): python3-mako-1.1.4-6.el9.noarch.rpm 275 kB/s | 172 kB 00:00 2024-05-13T22:36:10.756 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-05-13T22:36:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-05-13T22:36:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-05-13T22:36:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-05-13T22:36:10.756 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-05-13T22:36:10.757 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-05-13T22:36:10.757 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-05-13T22:36:10.757 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-05-13T22:36:10.757 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:10.814 INFO:teuthology.orchestra.run.smithi130.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 75 kB/s | 43 kB 00:00 2024-05-13T22:36:10.913 INFO:teuthology.orchestra.run.smithi130.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-05-13T22:36:11.032 INFO:teuthology.orchestra.run.smithi130.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 151 kB/s | 18 kB 00:00 2024-05-13T22:36:11.073 INFO:teuthology.orchestra.run.smithi130.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 240 kB/s | 172 kB 00:00 2024-05-13T22:36:11.131 INFO:teuthology.orchestra.run.smithi130.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 231 kB/s | 23 kB 00:00 2024-05-13T22:36:11.173 INFO:teuthology.orchestra.run.smithi130.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2024-05-13T22:36:11.232 INFO:teuthology.orchestra.run.smithi130.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 196 kB/s | 20 kB 00:00 2024-05-13T22:36:11.274 INFO:teuthology.orchestra.run.smithi130.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 462 kB/s | 46 kB 00:00 2024-05-13T22:36:11.416 INFO:teuthology.orchestra.run.smithi130.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 429 kB/s | 79 kB 00:00 2024-05-13T22:36:11.464 INFO:teuthology.orchestra.run.smithi130.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 363 kB/s | 358 kB 00:00 2024-05-13T22:36:11.505 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-05-13T22:36:11.525 INFO:teuthology.orchestra.run.smithi130.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 151 kB/s | 16 kB 00:00 2024-05-13T22:36:11.531 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-05-13T22:36:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:36:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-13T22:36:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-13T22:36:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-13T22:36:11.531 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:11.532 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-13T22:36:11.633 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-13T22:36:11.633 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-13T22:36:11.633 INFO:teuthology.orchestra.run.smithi130.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 330 kB/s | 36 kB 00:00 2024-05-13T22:36:11.658 INFO:teuthology.orchestra.run.smithi130.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 462 kB/s | 90 kB 00:00 2024-05-13T22:36:11.734 INFO:teuthology.orchestra.run.smithi130.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 592 kB/s | 272 kB 00:00 2024-05-13T22:36:11.834 INFO:teuthology.orchestra.run.smithi130.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 200 kB/s | 20 kB 00:00 2024-05-13T22:36:11.976 INFO:teuthology.orchestra.run.smithi130.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 671 kB/s | 230 kB 00:00 2024-05-13T22:36:12.259 INFO:teuthology.orchestra.run.smithi130.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 711 kB/s | 427 kB 00:00 2024-05-13T22:36:12.266 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:12.266 INFO:teuthology.orchestra.run.smithi130.stdout:Total 8.4 MB/s | 30 MB 00:03 2024-05-13T22:36:12.385 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:36:12.422 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:36:12.422 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:36:12.709 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-13T22:36:12.710 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-05-13T22:36:12.710 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-05-13T22:36:12.710 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-05-13T22:36:12.710 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-05-13T22:36:12.710 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-05-13T22:36:12.710 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-05-13T22:36:12.710 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-05-13T22:36:12.710 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-05-13T22:36:12.710 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-05-13T22:36:12.710 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-05-13T22:36:12.710 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-05-13T22:36:12.711 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-05-13T22:36:12.712 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-05-13T22:36:12.712 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-05-13T22:36:12.712 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-05-13T22:36:12.712 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-05-13T22:36:12.712 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-05-13T22:36:12.712 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-05-13T22:36:12.712 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-05-13T22:36:12.823 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:36:12.823 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:36:13.422 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-05-13T22:36:13.422 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: fmt-8.1.1-5.el9.x86_64 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-13T22:36:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:13.425 INFO:teuthology.orchestra.run.smithi102.stdout: re2-1:20211101-3.el9.x86_64 2024-05-13T22:36:13.425 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-13T22:36:13.425 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:13.425 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:36:13.748 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-test 2024-05-13T22:36:13.995 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:36:14.054 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-13T22:36:14.086 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-13T22:36:14.119 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-13T22:36:14.211 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-13T22:36:14.289 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-13T22:36:14.324 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:26 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:36:14.330 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-13T22:36:14.373 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-13T22:36:14.403 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-13T22:36:14.438 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-13T22:36:14.444 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:36:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-05-13T22:36:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:36:14.445 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-05-13T22:36:14.445 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-05-13T22:36:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-05-13T22:36:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-13T22:36:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-13T22:36:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:14.446 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:36:14.446 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:14.446 INFO:teuthology.orchestra.run.smithi102.stdout:Install 4 Packages 2024-05-13T22:36:14.446 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:14.447 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 40 M 2024-05-13T22:36:14.447 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 166 M 2024-05-13T22:36:14.447 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:36:14.471 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-13T22:36:14.512 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-13T22:36:14.546 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-13T22:36:14.583 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-13T22:36:14.606 INFO:teuthology.orchestra.run.smithi102.stdout:(1/4): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.6 MB/s | 165 kB 00:00 2024-05-13T22:36:14.630 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-13T22:36:14.689 INFO:teuthology.orchestra.run.smithi102.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-05-13T22:36:14.706 INFO:teuthology.orchestra.run.smithi102.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 637 kB/s | 64 kB 00:00 2024-05-13T22:36:14.720 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-13T22:36:14.891 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-13T22:36:14.979 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-13T22:36:15.032 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-13T22:36:15.071 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-13T22:36:15.128 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-13T22:36:15.131 INFO:teuthology.orchestra.run.smithi102.stdout:(4/4): ceph-test-18.2.1-0.el9.x86_64.rpm 63 MB/s | 40 MB 00:00 2024-05-13T22:36:15.133 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:15.134 INFO:teuthology.orchestra.run.smithi102.stdout:Total 58 MB/s | 40 MB 00:00 2024-05-13T22:36:15.161 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:36:15.179 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:36:15.179 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:36:15.190 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-13T22:36:15.381 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-13T22:36:15.442 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-13T22:36:15.495 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:36:15.495 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:36:15.520 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-13T22:36:15.608 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-13T22:36:15.641 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-13T22:36:15.758 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-13T22:36:15.912 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-05-13T22:36:15.943 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-05-13T22:36:15.966 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:36:15.967 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-05-13T22:36:15.967 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:36:15.968 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-13T22:36:15.968 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-13T22:36:15.968 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-13T22:36:15.968 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:16.059 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-13T22:36:16.105 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-13T22:36:16.133 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-05-13T22:36:17.505 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-05-13T22:36:17.542 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-05-13T22:36:17.542 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:36:17.542 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-13T22:36:17.543 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-13T22:36:17.543 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-13T22:36:17.543 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:18.020 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-05-13T22:36:18.051 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-05-13T22:36:18.051 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:36:18.052 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-13T22:36:18.052 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-13T22:36:18.052 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-13T22:36:18.052 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:18.271 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-05-13T22:36:18.298 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-05-13T22:36:18.298 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:36:18.299 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-13T22:36:18.299 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-13T22:36:18.299 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-13T22:36:18.299 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:18.368 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-05-13T22:36:19.893 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-05-13T22:36:19.974 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-05-13T22:36:19.983 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-05-13T22:36:19.983 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-05-13T22:36:19.983 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-05-13T22:36:19.983 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-05-13T22:36:19.983 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-05-13T22:36:19.983 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-05-13T22:36:19.983 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-05-13T22:36:19.983 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-13T22:36:19.983 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-13T22:36:19.984 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-13T22:36:19.984 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-13T22:36:19.984 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-13T22:36:19.984 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-13T22:36:19.984 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-13T22:36:19.984 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-13T22:36:19.984 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-13T22:36:19.985 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-13T22:36:19.985 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-13T22:36:19.985 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-13T22:36:19.985 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-13T22:36:19.985 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-13T22:36:19.985 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-13T22:36:19.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-13T22:36:19.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-13T22:36:19.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-13T22:36:19.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-13T22:36:19.986 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-13T22:36:19.987 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-13T22:36:19.987 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-13T22:36:19.987 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-13T22:36:19.987 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-13T22:36:19.987 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-13T22:36:19.987 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-13T22:36:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-13T22:36:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:20.913 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:36:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-05-13T22:36:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-13T22:36:20.913 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-13T22:36:20.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply-3.11-14.el9.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:20.915 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:36:21.258 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-base 2024-05-13T22:36:21.831 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:00:48 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:36:21.912 INFO:teuthology.orchestra.run.smithi130.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:36:21.953 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:36:21.954 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-05-13T22:36:21.954 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:36:21.984 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-05-13T22:36:21.984 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/4 2024-05-13T22:36:21.985 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/4 2024-05-13T22:36:21.985 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-13T22:36:22.022 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install cephadm 2024-05-13T22:36:22.247 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-13T22:36:22.247 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:22.247 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:36:22.247 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:18.2.1-0.el9.x86_64 libcephsqlite-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:22.247 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-05-13T22:36:22.247 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:22.248 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:36:22.512 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph 2024-05-13T22:36:22.584 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:00:49 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:36:22.707 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:36:22.708 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:22.708 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2024-05-13T22:36:22.708 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:22.708 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:36:22.708 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-05-13T22:36:22.708 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:22.709 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:36:22.709 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:22.709 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-05-13T22:36:22.709 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:22.709 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 221 k 2024-05-13T22:36:22.709 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 213 k 2024-05-13T22:36:22.709 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:36:22.770 INFO:teuthology.orchestra.run.smithi130.stdout:cephadm-18.2.1-0.el9.noarch.rpm 3.5 MB/s | 221 kB 00:00 2024-05-13T22:36:22.771 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:22.771 INFO:teuthology.orchestra.run.smithi130.stdout:Total 3.4 MB/s | 221 kB 00:00 2024-05-13T22:36:22.771 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:36:22.776 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:36:22.776 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:36:22.779 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:36:22.779 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:36:22.847 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:36:23.089 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:35 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:36:23.212 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:36:23.214 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:23.214 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-05-13T22:36:23.214 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:23.214 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:36:23.214 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-05-13T22:36:23.214 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-05-13T22:36:23.214 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-05-13T22:36:23.214 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-05-13T22:36:23.214 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-13T22:36:23.215 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-13T22:36:23.216 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-13T22:36:23.217 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-13T22:36:23.217 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:23.217 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:36:23.217 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:23.217 INFO:teuthology.orchestra.run.smithi102.stdout:Install 33 Packages 2024-05-13T22:36:23.217 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:23.217 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 30 M 2024-05-13T22:36:23.217 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 110 M 2024-05-13T22:36:23.218 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:36:23.233 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-05-13T22:36:23.323 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-05-13T22:36:23.660 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-05-13T22:36:23.667 INFO:teuthology.orchestra.run.smithi102.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 60 kB/s | 6.4 kB 00:00 2024-05-13T22:36:23.709 INFO:teuthology.orchestra.run.smithi102.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 9.8 MB/s | 1.4 MB 00:00 2024-05-13T22:36:23.767 INFO:teuthology.orchestra.run.smithi102.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 10 MB/s | 2.1 MB 00:00 2024-05-13T22:36:23.792 INFO:teuthology.orchestra.run.smithi102.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.5 MB/s | 242 kB 00:00 2024-05-13T22:36:23.860 INFO:teuthology.orchestra.run.smithi102.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-05-13T22:36:23.889 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-05-13T22:36:23.889 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:23.889 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:36:23.889 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-05-13T22:36:23.889 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:23.889 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:36:24.093 INFO:teuthology.orchestra.run.smithi102.stdout:(6/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 46 MB/s | 18 MB 00:00 2024-05-13T22:36:24.118 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-immutable-object-cache 2024-05-13T22:36:24.119 INFO:teuthology.orchestra.run.smithi102.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 778 kB/s | 253 kB 00:00 2024-05-13T22:36:24.169 INFO:teuthology.orchestra.run.smithi102.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-05-13T22:36:24.227 INFO:teuthology.orchestra.run.smithi102.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 3.3 MB/s | 1.2 MB 00:00 2024-05-13T22:36:24.252 INFO:teuthology.orchestra.run.smithi102.stdout:(10/33): python3-ply-3.11-14.el9.noarch.rpm 670 kB/s | 106 kB 00:00 2024-05-13T22:36:24.278 INFO:teuthology.orchestra.run.smithi102.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 1.1 MB/s | 125 kB 00:00 2024-05-13T22:36:24.303 INFO:teuthology.orchestra.run.smithi102.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-05-13T22:36:24.328 INFO:teuthology.orchestra.run.smithi102.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 695 kB/s | 35 kB 00:00 2024-05-13T22:36:24.354 INFO:teuthology.orchestra.run.smithi102.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-13T22:36:24.379 INFO:teuthology.orchestra.run.smithi102.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 825 kB/s | 42 kB 00:00 2024-05-13T22:36:24.405 INFO:teuthology.orchestra.run.smithi102.stdout:(16/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-05-13T22:36:24.696 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:00:51 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:36:24.822 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:36:24.823 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:24.823 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repo Size 2024-05-13T22:36:24.823 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:24.823 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:36:24.823 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-05-13T22:36:24.823 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:24.823 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:36:24.823 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:24.824 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-05-13T22:36:24.824 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:24.824 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 142 k 2024-05-13T22:36:24.824 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 395 k 2024-05-13T22:36:24.824 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:36:24.909 INFO:teuthology.orchestra.run.smithi130.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.6 MB/s | 142 kB 00:00 2024-05-13T22:36:24.910 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:24.910 INFO:teuthology.orchestra.run.smithi130.stdout:Total 1.6 MB/s | 142 kB 00:00 2024-05-13T22:36:24.910 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:36:24.918 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:36:24.918 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:36:24.930 INFO:teuthology.orchestra.run.smithi102.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 75 kB/s | 43 kB 00:00 2024-05-13T22:36:24.977 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:36:24.977 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:36:25.124 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:36:25.202 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:36:25.228 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:36:25.228 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:36:25.228 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-13T22:36:25.228 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:25.263 INFO:teuthology.orchestra.run.smithi102.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 194 kB/s | 172 kB 00:00 2024-05-13T22:36:25.382 INFO:teuthology.orchestra.run.smithi102.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 367 kB/s | 358 kB 00:00 2024-05-13T22:36:25.414 INFO:teuthology.orchestra.run.smithi102.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 118 kB/s | 18 kB 00:00 2024-05-13T22:36:25.522 INFO:teuthology.orchestra.run.smithi102.stdout:(21/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 18 kB/s | 11 kB 00:00 2024-05-13T22:36:25.547 INFO:teuthology.orchestra.run.smithi102.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 138 kB/s | 23 kB 00:00 2024-05-13T22:36:25.595 INFO:teuthology.orchestra.run.smithi102.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 107 kB/s | 19 kB 00:00 2024-05-13T22:36:25.665 INFO:teuthology.orchestra.run.smithi102.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 138 kB/s | 20 kB 00:00 2024-05-13T22:36:25.690 INFO:teuthology.orchestra.run.smithi102.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 326 kB/s | 46 kB 00:00 2024-05-13T22:36:25.740 INFO:teuthology.orchestra.run.smithi102.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 545 kB/s | 79 kB 00:00 2024-05-13T22:36:25.799 INFO:teuthology.orchestra.run.smithi102.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 151 kB/s | 16 kB 00:00 2024-05-13T22:36:25.899 INFO:teuthology.orchestra.run.smithi102.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 567 kB/s | 90 kB 00:00 2024-05-13T22:36:25.924 INFO:teuthology.orchestra.run.smithi102.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 285 kB/s | 36 kB 00:00 2024-05-13T22:36:26.075 INFO:teuthology.orchestra.run.smithi102.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 665 kB/s | 272 kB 00:00 2024-05-13T22:36:26.080 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:36:26.080 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:26.080 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:36:26.080 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:26.080 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:26.080 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:36:26.133 INFO:teuthology.orchestra.run.smithi102.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 983 kB/s | 230 kB 00:00 2024-05-13T22:36:26.158 INFO:teuthology.orchestra.run.smithi102.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.8 MB/s | 427 kB 00:00 2024-05-13T22:36:26.183 INFO:teuthology.orchestra.run.smithi102.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 184 kB/s | 20 kB 00:00 2024-05-13T22:36:26.187 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:26.188 INFO:teuthology.orchestra.run.smithi102.stdout:Total 10 MB/s | 30 MB 00:02 2024-05-13T22:36:26.295 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-mgr 2024-05-13T22:36:26.307 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:36:26.343 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:36:26.343 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:36:26.720 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:36:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:36:26.868 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:00:53 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:36:26.954 INFO:teuthology.orchestra.run.smithi130.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:36:26.997 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:36:26.998 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-05-13T22:36:26.998 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:36:27.064 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-mgr-dashboard 2024-05-13T22:36:27.631 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:00:54 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:36:27.755 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:36:27.756 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:27.756 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-05-13T22:36:27.756 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:27.756 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout:Installing weak dependencies: 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout:Install 7 Packages 2024-05-13T22:36:27.757 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:27.758 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 2.1 M 2024-05-13T22:36:27.758 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 8.5 M 2024-05-13T22:36:27.758 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:36:27.906 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:36:27.978 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-13T22:36:28.010 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-13T22:36:28.043 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-13T22:36:28.135 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-13T22:36:28.204 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-13T22:36:28.222 INFO:teuthology.orchestra.run.smithi130.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 219 kB/s | 23 kB 00:00 2024-05-13T22:36:28.239 INFO:teuthology.orchestra.run.smithi130.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 119 kB/s | 15 kB 00:00 2024-05-13T22:36:28.246 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-13T22:36:28.281 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-13T22:36:28.289 INFO:teuthology.orchestra.run.smithi130.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 10 MB/s | 1.8 MB 00:00 2024-05-13T22:36:28.311 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-13T22:36:28.335 INFO:teuthology.orchestra.run.smithi130.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 80 kB/s | 9.0 kB 00:00 2024-05-13T22:36:28.345 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-13T22:36:28.378 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-13T22:36:28.398 INFO:teuthology.orchestra.run.smithi130.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 257 kB/s | 41 kB 00:00 2024-05-13T22:36:28.419 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-13T22:36:28.423 INFO:teuthology.orchestra.run.smithi130.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 230 kB/s | 31 kB 00:00 2024-05-13T22:36:28.454 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-13T22:36:28.488 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-13T22:36:28.498 INFO:teuthology.orchestra.run.smithi130.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.1 MB/s | 188 kB 00:00 2024-05-13T22:36:28.500 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:28.500 INFO:teuthology.orchestra.run.smithi130.stdout:Total 2.8 MB/s | 2.1 MB 00:00 2024-05-13T22:36:28.500 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:36:28.508 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:36:28.508 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:36:28.525 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-13T22:36:28.560 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:36:28.560 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:36:28.600 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-13T22:36:28.737 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-13T22:36:28.820 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-13T22:36:28.861 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:36:28.873 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-13T22:36:28.912 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-13T22:36:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-05-13T22:36:28.959 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-05-13T22:36:28.960 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-13T22:36:29.005 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-05-13T22:36:29.016 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-13T22:36:29.037 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-05-13T22:36:29.066 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-05-13T22:36:29.210 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-13T22:36:29.266 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-13T22:36:29.301 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-05-13T22:36:29.335 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-13T22:36:29.336 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-05-13T22:36:29.424 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-13T22:36:29.468 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-13T22:36:29.589 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-13T22:36:29.751 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-05-13T22:36:29.776 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-05-13T22:36:29.801 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-05-13T22:36:29.801 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:36:29.801 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-13T22:36:29.801 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-13T22:36:29.801 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-13T22:36:29.801 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:29.806 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-05-13T22:36:29.806 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-05-13T22:36:29.806 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-05-13T22:36:29.806 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-05-13T22:36:29.806 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-05-13T22:36:29.807 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-05-13T22:36:29.807 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-05-13T22:36:30.096 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-05-13T22:36:30.097 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:30.097 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:36:30.097 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-05-13T22:36:30.097 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-05-13T22:36:30.097 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-05-13T22:36:30.097 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-05-13T22:36:30.097 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-05-13T22:36:30.097 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-13T22:36:30.097 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-13T22:36:30.098 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:30.098 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:36:30.292 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-13T22:36:30.869 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:00:57 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:36:30.993 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:36:30.994 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-13T22:36:30.995 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-13T22:36:30.996 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-13T22:36:30.996 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-13T22:36:30.996 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-13T22:36:30.996 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-13T22:36:30.996 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:30.996 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:36:30.996 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:36:30.996 INFO:teuthology.orchestra.run.smithi130.stdout:Install 12 Packages 2024-05-13T22:36:30.996 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:30.997 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 38 M 2024-05-13T22:36:30.997 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 209 M 2024-05-13T22:36:30.997 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:36:31.247 INFO:teuthology.orchestra.run.smithi130.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.1-0. 51 MB/s | 7.4 MB 00:00 2024-05-13T22:36:31.377 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-05-13T22:36:31.406 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-05-13T22:36:31.406 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:36:31.406 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-13T22:36:31.406 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-13T22:36:31.406 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-13T22:36:31.407 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:31.505 INFO:teuthology.orchestra.run.smithi130.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 127 kB/s | 33 kB 00:00 2024-05-13T22:36:31.689 INFO:teuthology.orchestra.run.smithi130.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 332 kB/s | 195 kB 00:00 2024-05-13T22:36:31.814 INFO:teuthology.orchestra.run.smithi130.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 145 kB/s | 18 kB 00:00 2024-05-13T22:36:31.853 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-05-13T22:36:31.877 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-05-13T22:36:31.877 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:36:31.878 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-13T22:36:31.878 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-13T22:36:31.878 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-13T22:36:31.878 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:31.915 INFO:teuthology.orchestra.run.smithi130.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 352 kB/s | 35 kB 00:00 2024-05-13T22:36:31.990 INFO:teuthology.orchestra.run.smithi130.stdout:(6/12): libgfortran-11.4.1-3.el9.x86_64.rpm 903 kB/s | 803 kB 00:00 2024-05-13T22:36:32.087 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-05-13T22:36:32.090 INFO:teuthology.orchestra.run.smithi130.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-13T22:36:32.110 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-05-13T22:36:32.110 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:36:32.111 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-13T22:36:32.111 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-13T22:36:32.111 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-13T22:36:32.111 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:32.176 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-05-13T22:36:32.440 INFO:teuthology.orchestra.run.smithi130.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.2 MB/s | 3.0 MB 00:00 2024-05-13T22:36:32.540 INFO:teuthology.orchestra.run.smithi130.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 250 kB 00:00 2024-05-13T22:36:33.024 INFO:teuthology.orchestra.run.smithi130.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.5 MB/s | 5.1 MB 00:00 2024-05-13T22:36:33.886 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-05-13T22:36:33.886 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-05-13T22:36:33.886 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-05-13T22:36:33.886 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-13T22:36:33.887 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-13T22:36:33.888 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-13T22:36:33.889 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-13T22:36:34.199 INFO:teuthology.orchestra.run.smithi130.stdout:(11/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 2.1 MB/s | 4.7 MB 00:02 2024-05-13T22:36:34.563 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-13T22:36:34.563 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:34.563 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:36:34.563 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:34.563 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:34.563 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:34.563 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-05-13T22:36:34.563 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-13T22:36:34.564 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-13T22:36:34.565 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:34.566 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:36:34.921 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-base 2024-05-13T22:36:35.495 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:47 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:36:35.576 INFO:teuthology.orchestra.run.smithi102.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:36:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:36:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-05-13T22:36:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:36:35.681 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install cephadm 2024-05-13T22:36:36.245 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:48 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:36:36.366 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:36:36.367 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:36.367 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-05-13T22:36:36.367 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:36.367 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:36:36.367 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-05-13T22:36:36.368 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:36.368 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:36:36.368 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:36.368 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-05-13T22:36:36.368 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:36.368 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 221 k 2024-05-13T22:36:36.368 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 213 k 2024-05-13T22:36:36.368 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:36:36.435 INFO:teuthology.orchestra.run.smithi102.stdout:cephadm-18.2.1-0.el9.noarch.rpm 3.2 MB/s | 221 kB 00:00 2024-05-13T22:36:36.436 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:36.436 INFO:teuthology.orchestra.run.smithi102.stdout:Total 3.1 MB/s | 221 kB 00:00 2024-05-13T22:36:36.436 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:36:36.441 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:36:36.441 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:36:36.444 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:36:36.444 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:36:36.511 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:36:36.941 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-05-13T22:36:37.032 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-05-13T22:36:37.396 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-05-13T22:36:37.661 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-05-13T22:36:37.661 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:37.661 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:36:37.661 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-05-13T22:36:37.661 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:37.661 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:36:37.832 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-immutable-object-cache 2024-05-13T22:36:38.420 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:50 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:36:38.543 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:36:38.544 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:38.544 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-05-13T22:36:38.544 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:38.545 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:36:38.545 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-05-13T22:36:38.545 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:38.545 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:36:38.545 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:38.545 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-05-13T22:36:38.545 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:38.545 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 142 k 2024-05-13T22:36:38.545 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 395 k 2024-05-13T22:36:38.545 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:36:38.617 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 2.0 MB/s | 142 kB 00:00 2024-05-13T22:36:38.617 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:38.618 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.9 MB/s | 142 kB 00:00 2024-05-13T22:36:38.618 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:36:38.625 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:36:38.625 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:36:38.681 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:36:38.682 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:36:38.830 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:36:38.901 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:36:38.927 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:36:38.927 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:36:38.927 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-13T22:36:38.927 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:39.839 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:36:39.839 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:39.839 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:36:39.839 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-05-13T22:36:39.839 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:39.840 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:36:40.070 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr 2024-05-13T22:36:40.646 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:52 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:36:40.729 INFO:teuthology.orchestra.run.smithi102.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:36:40.772 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:36:40.773 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-05-13T22:36:40.773 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:36:40.839 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-dashboard 2024-05-13T22:36:41.407 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:53 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:36:41.534 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:36:41.535 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:41.535 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-05-13T22:36:41.535 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:41.535 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:36:41.535 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-05-13T22:36:41.535 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-05-13T22:36:41.535 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-05-13T22:36:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-05-13T22:36:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-05-13T22:36:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-13T22:36:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-13T22:36:41.536 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-05-13T22:36:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-05-13T22:36:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:41.536 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:36:41.536 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:41.536 INFO:teuthology.orchestra.run.smithi102.stdout:Install 7 Packages 2024-05-13T22:36:41.536 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:41.537 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 2.1 M 2024-05-13T22:36:41.537 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 8.5 M 2024-05-13T22:36:41.537 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:36:41.698 INFO:teuthology.orchestra.run.smithi102.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 198 kB/s | 23 kB 00:00 2024-05-13T22:36:41.712 INFO:teuthology.orchestra.run.smithi102.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 109 kB/s | 15 kB 00:00 2024-05-13T22:36:41.763 INFO:teuthology.orchestra.run.smithi102.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 9.6 MB/s | 1.8 MB 00:00 2024-05-13T22:36:42.029 INFO:teuthology.orchestra.run.smithi102.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 27 kB/s | 9.0 kB 00:00 2024-05-13T22:36:42.163 INFO:teuthology.orchestra.run.smithi102.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 77 kB/s | 31 kB 00:00 2024-05-13T22:36:42.188 INFO:teuthology.orchestra.run.smithi102.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 86 kB/s | 41 kB 00:00 2024-05-13T22:36:42.389 INFO:teuthology.orchestra.run.smithi102.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 525 kB/s | 188 kB 00:00 2024-05-13T22:36:42.392 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:42.393 INFO:teuthology.orchestra.run.smithi102.stdout:Total 2.4 MB/s | 2.1 MB 00:00 2024-05-13T22:36:42.393 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:36:42.401 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:36:42.401 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:36:42.453 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:36:42.453 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:36:42.740 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:36:42.807 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-05-13T22:36:42.841 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-05-13T22:36:42.887 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-05-13T22:36:42.927 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-05-13T22:36:42.956 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-05-13T22:36:43.191 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-05-13T22:36:43.226 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-05-13T22:36:43.733 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-05-13T22:36:43.733 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-05-13T22:36:43.733 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-05-13T22:36:43.733 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-05-13T22:36:43.733 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-05-13T22:36:43.733 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-05-13T22:36:43.733 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-05-13T22:36:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-05-13T22:36:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:44.019 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:36:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-05-13T22:36:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-05-13T22:36:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-05-13T22:36:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-05-13T22:36:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-05-13T22:36:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-13T22:36:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-13T22:36:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:44.019 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:36:44.215 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-13T22:36:44.784 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:56 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:36:44.906 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-13T22:36:44.908 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-13T22:36:44.909 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-13T22:36:44.909 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-13T22:36:44.909 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-13T22:36:44.909 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-13T22:36:44.909 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-13T22:36:44.909 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:44.909 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:36:44.909 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:44.909 INFO:teuthology.orchestra.run.smithi102.stdout:Install 12 Packages 2024-05-13T22:36:44.909 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:44.910 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 38 M 2024-05-13T22:36:44.910 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 209 M 2024-05-13T22:36:44.910 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:36:45.454 INFO:teuthology.orchestra.run.smithi102.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.1-0. 49 MB/s | 7.4 MB 00:00 2024-05-13T22:36:45.520 INFO:teuthology.orchestra.run.smithi102.stdout:(2/12): libquadmath-11.4.1-3.el9.x86_64.rpm 890 kB/s | 195 kB 00:00 2024-05-13T22:36:45.546 INFO:teuthology.orchestra.run.smithi102.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 357 kB/s | 33 kB 00:00 2024-05-13T22:36:45.579 INFO:teuthology.orchestra.run.smithi102.stdout:(4/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-05-13T22:36:45.605 INFO:teuthology.orchestra.run.smithi102.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 310 kB/s | 18 kB 00:00 2024-05-13T22:36:45.630 INFO:teuthology.orchestra.run.smithi102.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-05-13T22:36:45.672 INFO:teuthology.orchestra.run.smithi102.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-13T22:36:45.730 INFO:teuthology.orchestra.run.smithi102.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-05-13T22:36:45.781 INFO:teuthology.orchestra.run.smithi102.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-05-13T22:36:45.856 INFO:teuthology.orchestra.run.smithi102.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-05-13T22:36:45.923 INFO:teuthology.orchestra.run.smithi102.stdout:(11/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 15 MB/s | 4.7 MB 00:00 2024-05-13T22:36:46.148 INFO:teuthology.orchestra.run.smithi102.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-05-13T22:36:46.149 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:46.149 INFO:teuthology.orchestra.run.smithi102.stdout:Total 31 MB/s | 38 MB 00:01 2024-05-13T22:36:46.503 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:36:46.529 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:36:46.529 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:36:47.015 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:36:47.015 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:36:47.765 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:36:47.846 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-13T22:36:47.896 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-13T22:36:48.004 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-13T22:36:48.047 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-13T22:36:48.216 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-13T22:36:48.255 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-13T22:36:48.338 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-13T22:36:48.992 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-13T22:36:49.108 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-13T22:36:50.350 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-13T22:36:51.386 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-13T22:36:51.413 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-05-13T22:36:51.514 INFO:teuthology.orchestra.run.smithi130.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 877 kB/s | 16 MB 00:18 2024-05-13T22:36:51.515 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:51.515 INFO:teuthology.orchestra.run.smithi130.stdout:Total 1.9 MB/s | 38 MB 00:20 2024-05-13T22:36:51.873 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:36:51.899 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:36:51.899 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:36:52.404 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:36:52.404 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:36:53.163 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:36:53.235 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-13T22:36:53.278 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-13T22:36:53.376 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-13T22:36:53.413 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-13T22:36:53.592 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-13T22:36:53.621 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-13T22:36:53.713 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-13T22:36:53.974 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-05-13T22:36:53.974 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-05-13T22:36:53.974 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-13T22:36:53.974 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-13T22:36:53.974 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-13T22:36:53.974 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-13T22:36:53.974 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-13T22:36:53.974 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-13T22:36:53.974 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-13T22:36:53.975 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-13T22:36:53.975 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-13T22:36:53.975 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-13T22:36:54.362 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-13T22:36:54.362 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:54.362 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:36:54.362 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-05-13T22:36:54.362 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-13T22:36:54.362 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-13T22:36:54.362 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-13T22:36:54.362 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-13T22:36:54.362 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-13T22:36:54.363 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-13T22:36:54.363 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-13T22:36:54.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-13T22:36:54.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-13T22:36:54.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-13T22:36:54.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-13T22:36:54.363 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:54.363 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:36:54.370 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-13T22:36:54.483 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-13T22:36:54.732 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-rook 2024-05-13T22:36:55.304 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:07 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:36:55.425 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:36:55.426 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-13T22:36:55.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-13T22:36:55.428 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-13T22:36:55.428 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-13T22:36:55.428 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-13T22:36:55.428 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-13T22:36:55.428 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:55.428 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:36:55.428 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:36:55.428 INFO:teuthology.orchestra.run.smithi102.stdout:Install 13 Packages 2024-05-13T22:36:55.428 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:36:55.429 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 2.2 M 2024-05-13T22:36:55.429 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 26 M 2024-05-13T22:36:55.429 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:36:55.705 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-13T22:36:55.888 INFO:teuthology.orchestra.run.smithi102.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 830 kB/s | 50 kB 00:00 2024-05-13T22:36:55.914 INFO:teuthology.orchestra.run.smithi102.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 223 kB/s | 19 kB 00:00 2024-05-13T22:36:55.939 INFO:teuthology.orchestra.run.smithi102.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 231 kB/s | 26 kB 00:00 2024-05-13T22:36:55.998 INFO:teuthology.orchestra.run.smithi102.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-05-13T22:36:56.023 INFO:teuthology.orchestra.run.smithi102.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-05-13T22:36:56.048 INFO:teuthology.orchestra.run.smithi102.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-05-13T22:36:56.074 INFO:teuthology.orchestra.run.smithi102.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.0 MB/s | 279 kB 00:00 2024-05-13T22:36:56.676 INFO:teuthology.orchestra.run.smithi102.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 44 kB/s | 15 kB 00:00 2024-05-13T22:36:56.676 INFO:teuthology.orchestra.run.smithi102.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 77 kB/s | 32 kB 00:00 2024-05-13T22:36:56.676 INFO:teuthology.orchestra.run.smithi102.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 337 kB/s | 59 kB 00:00 2024-05-13T22:36:56.676 INFO:teuthology.orchestra.run.smithi102.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 357 kB/s | 206 kB 00:00 2024-05-13T22:36:56.750 INFO:teuthology.orchestra.run.smithi102.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 672 kB/s | 90 kB 00:00 2024-05-13T22:36:56.751 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-13T22:36:56.787 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-05-13T22:36:56.900 INFO:teuthology.orchestra.run.smithi102.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.0 MB/s | 1.0 MB 00:00 2024-05-13T22:36:56.902 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:36:56.902 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.5 MB/s | 2.2 MB 00:01 2024-05-13T22:36:56.964 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:36:56.979 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:36:56.979 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:36:57.062 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:36:57.062 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:36:57.589 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:36:57.753 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-13T22:36:57.809 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-13T22:36:57.864 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-13T22:36:57.894 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-13T22:36:57.935 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-13T22:36:58.028 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-13T22:36:58.138 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-13T22:36:58.196 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-13T22:36:58.923 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-13T22:36:58.978 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-13T22:36:59.012 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-13T22:36:59.059 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-13T22:36:59.082 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-05-13T22:36:59.416 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-05-13T22:36:59.416 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-05-13T22:36:59.416 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-13T22:36:59.417 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-13T22:36:59.417 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-13T22:36:59.417 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-13T22:36:59.417 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-13T22:36:59.417 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-13T22:36:59.417 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-13T22:36:59.417 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-13T22:36:59.417 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-13T22:36:59.418 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-13T22:36:59.779 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-05-13T22:36:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-05-13T22:36:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-13T22:36:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-13T22:36:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-13T22:36:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-13T22:36:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-13T22:36:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-13T22:36:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-13T22:36:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-13T22:36:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-13T22:36:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-13T22:36:59.781 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-13T22:36:59.809 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-13T22:36:59.809 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-13T22:36:59.810 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-13T22:36:59.811 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:36:59.811 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:00.099 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-13T22:37:00.099 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:00.099 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:37:00.099 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-05-13T22:37:00.099 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-13T22:37:00.099 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:00.100 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:00.208 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-mgr-rook 2024-05-13T22:37:00.340 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-cephadm 2024-05-13T22:37:00.789 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:27 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:00.909 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:12 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:00.911 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:00.913 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:00.913 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-05-13T22:37:00.913 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:00.913 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:37:00.913 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-05-13T22:37:00.913 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-05-13T22:37:00.913 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-13T22:37:00.913 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-13T22:37:00.913 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:00.914 INFO:teuthology.orchestra.run.smithi130.stdout:Install 13 Packages 2024-05-13T22:37:00.915 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:00.915 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 2.2 M 2024-05-13T22:37:00.915 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 26 M 2024-05-13T22:37:00.915 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:37:01.031 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:01.031 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:01.031 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-05-13T22:37:01.031 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:01.032 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:37:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-05-13T22:37:01.032 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-05-13T22:37:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-13T22:37:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-13T22:37:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-13T22:37:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-13T22:37:01.032 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-13T22:37:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:01.033 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:37:01.033 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:01.033 INFO:teuthology.orchestra.run.smithi102.stdout:Install 6 Packages 2024-05-13T22:37:01.033 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:01.034 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 7.0 M 2024-05-13T22:37:01.034 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 33 M 2024-05-13T22:37:01.034 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:37:01.062 INFO:teuthology.orchestra.run.smithi130.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 829 kB/s | 50 kB 00:00 2024-05-13T22:37:01.096 INFO:teuthology.orchestra.run.smithi130.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 204 kB/s | 19 kB 00:00 2024-05-13T22:37:01.121 INFO:teuthology.orchestra.run.smithi130.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 214 kB/s | 26 kB 00:00 2024-05-13T22:37:01.188 INFO:teuthology.orchestra.run.smithi130.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-13T22:37:01.213 INFO:teuthology.orchestra.run.smithi130.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-05-13T22:37:01.238 INFO:teuthology.orchestra.run.smithi130.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.3 MB/s | 279 kB 00:00 2024-05-13T22:37:01.264 INFO:teuthology.orchestra.run.smithi130.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 710 kB/s | 54 kB 00:00 2024-05-13T22:37:01.363 INFO:teuthology.orchestra.run.smithi102.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-05-13T22:37:01.472 INFO:teuthology.orchestra.run.smithi130.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 124 kB/s | 32 kB 00:00 2024-05-13T22:37:01.498 INFO:teuthology.orchestra.run.smithi130.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 58 kB/s | 15 kB 00:00 2024-05-13T22:37:01.581 INFO:teuthology.orchestra.run.smithi130.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 648 kB/s | 206 kB 00:00 2024-05-13T22:37:01.607 INFO:teuthology.orchestra.run.smithi130.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 544 kB/s | 59 kB 00:00 2024-05-13T22:37:01.665 INFO:teuthology.orchestra.run.smithi130.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-05-13T22:37:01.897 INFO:teuthology.orchestra.run.smithi102.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 403 kB/s | 248 kB 00:00 2024-05-13T22:37:01.965 INFO:teuthology.orchestra.run.smithi130.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.1 MB/s | 1.0 MB 00:00 2024-05-13T22:37:01.967 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:01.968 INFO:teuthology.orchestra.run.smithi130.stdout:Total 2.1 MB/s | 2.2 MB 00:01 2024-05-13T22:37:02.023 INFO:teuthology.orchestra.run.smithi102.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 831 kB/s | 548 kB 00:00 2024-05-13T22:37:02.033 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:37:02.048 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:37:02.048 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:37:02.123 INFO:teuthology.orchestra.run.smithi102.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 511 kB/s | 51 kB 00:00 2024-05-13T22:37:02.130 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:37:02.132 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:37:02.331 INFO:teuthology.orchestra.run.smithi102.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 133 kB/s | 58 kB 00:00 2024-05-13T22:37:02.640 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:37:02.681 INFO:teuthology.orchestra.run.smithi102.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.3 MB/s | 6.0 MB 00:01 2024-05-13T22:37:02.683 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:02.683 INFO:teuthology.orchestra.run.smithi102.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-05-13T22:37:02.756 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:37:02.767 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:37:02.768 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:37:02.802 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-13T22:37:02.858 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-13T22:37:02.865 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:37:02.865 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:37:02.913 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-13T22:37:02.951 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-13T22:37:02.992 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-13T22:37:03.085 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-13T22:37:03.171 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:37:03.205 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-13T22:37:03.260 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-13T22:37:03.336 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-13T22:37:03.391 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-13T22:37:03.908 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-13T22:37:03.968 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-13T22:37:03.993 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-13T22:37:04.026 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-13T22:37:04.051 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-13T22:37:04.054 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-13T22:37:04.079 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-05-13T22:37:04.095 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-13T22:37:04.118 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-05-13T22:37:04.801 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-05-13T22:37:04.802 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-05-13T22:37:04.802 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-13T22:37:04.802 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-13T22:37:04.802 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-13T22:37:04.802 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-13T22:37:04.890 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-05-13T22:37:04.890 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-05-13T22:37:04.890 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-13T22:37:04.890 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-13T22:37:04.890 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-13T22:37:04.890 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-13T22:37:04.890 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-13T22:37:04.890 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-13T22:37:04.890 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-13T22:37:04.890 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-13T22:37:04.891 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-13T22:37:04.891 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-13T22:37:04.891 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-13T22:37:05.048 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-13T22:37:05.049 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:05.049 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:37:05.049 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-05-13T22:37:05.049 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-13T22:37:05.049 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-13T22:37:05.049 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-13T22:37:05.049 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-13T22:37:05.049 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-13T22:37:05.049 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:05.049 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:05.225 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-13T22:37:05.225 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:05.225 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:37:05.225 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-13T22:37:05.226 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:05.227 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:05.269 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-fuse 2024-05-13T22:37:05.484 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-mgr-cephadm 2024-05-13T22:37:05.846 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:17 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:05.969 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:05.970 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:05.970 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout:Install 2 Packages 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 920 k 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 2.7 M 2024-05-13T22:37:05.971 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:37:06.056 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:33 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:06.177 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:06.178 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:06.178 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-05-13T22:37:06.178 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:06.178 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:37:06.178 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-05-13T22:37:06.178 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-05-13T22:37:06.179 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-13T22:37:06.179 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-13T22:37:06.179 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-13T22:37:06.179 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-13T22:37:06.179 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-13T22:37:06.179 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:06.179 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:37:06.179 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:06.179 INFO:teuthology.orchestra.run.smithi130.stdout:Install 6 Packages 2024-05-13T22:37:06.179 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:06.180 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 7.0 M 2024-05-13T22:37:06.180 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 33 M 2024-05-13T22:37:06.180 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:37:06.325 INFO:teuthology.orchestra.run.smithi102.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 10 MB/s | 839 kB 00:00 2024-05-13T22:37:06.709 INFO:teuthology.orchestra.run.smithi102.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 173 kB/s | 80 kB 00:00 2024-05-13T22:37:06.709 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:06.709 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.2 MB/s | 920 kB 00:00 2024-05-13T22:37:06.722 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:37:06.740 INFO:teuthology.orchestra.run.smithi130.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 2.1 MB/s | 132 kB 00:00 2024-05-13T22:37:06.744 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:37:06.744 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:37:06.822 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:37:06.822 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:37:06.840 INFO:teuthology.orchestra.run.smithi130.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.5 MB/s | 248 kB 00:00 2024-05-13T22:37:07.006 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:37:07.007 INFO:teuthology.orchestra.run.smithi130.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-05-13T22:37:07.131 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-13T22:37:07.207 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-05-13T22:37:07.491 INFO:teuthology.orchestra.run.smithi130.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 92 kB/s | 58 kB 00:00 2024-05-13T22:37:07.558 INFO:teuthology.orchestra.run.smithi130.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 97 kB/s | 51 kB 00:00 2024-05-13T22:37:07.831 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-05-13T22:37:07.831 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-05-13T22:37:08.008 INFO:teuthology.orchestra.run.smithi130.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 432 kB/s | 548 kB 00:01 2024-05-13T22:37:08.011 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:08.011 INFO:teuthology.orchestra.run.smithi130.stdout:Total 3.8 MB/s | 7.0 MB 00:01 2024-05-13T22:37:08.086 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:37:08.089 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-13T22:37:08.090 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:08.090 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:37:08.090 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-13T22:37:08.090 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:08.090 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:08.098 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:37:08.098 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:37:08.194 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:37:08.194 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:37:08.353 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados-devel 2024-05-13T22:37:08.495 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:37:08.672 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-13T22:37:08.726 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-13T22:37:08.930 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:20 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:09.057 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:37:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-05-13T22:37:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:37:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:09.058 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-05-13T22:37:09.058 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 127 k 2024-05-13T22:37:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 456 k 2024-05-13T22:37:09.059 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:37:09.122 INFO:teuthology.orchestra.run.smithi102.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 2.0 MB/s | 127 kB 00:00 2024-05-13T22:37:09.123 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:09.123 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.9 MB/s | 127 kB 00:00 2024-05-13T22:37:09.123 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:37:09.128 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:37:09.128 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:37:09.191 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:37:09.191 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:37:09.256 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-13T22:37:09.338 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:37:09.345 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-13T22:37:09.406 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-13T22:37:09.427 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:09.438 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-05-13T22:37:09.827 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:10.073 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:10.073 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:10.073 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:37:10.073 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-05-13T22:37:10.073 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:10.074 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:10.215 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-05-13T22:37:10.215 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-05-13T22:37:10.215 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-13T22:37:10.215 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-13T22:37:10.215 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-13T22:37:10.215 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-13T22:37:10.324 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install libcephfs2 2024-05-13T22:37:10.481 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-13T22:37:10.481 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:10.481 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:37:10.481 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-05-13T22:37:10.481 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-13T22:37:10.481 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-13T22:37:10.481 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-13T22:37:10.482 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-13T22:37:10.482 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-13T22:37:10.482 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:10.482 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:10.739 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-fuse 2024-05-13T22:37:10.907 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:22 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:10.991 INFO:teuthology.orchestra.run.smithi102.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:11.034 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:11.035 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-05-13T22:37:11.036 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:11.107 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install libcephfs-devel 2024-05-13T22:37:11.312 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:38 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:11.434 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:11.435 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:11.435 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:11.435 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout:Install 2 Packages 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 920 k 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 2.7 M 2024-05-13T22:37:11.436 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:37:11.642 INFO:teuthology.orchestra.run.smithi130.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 10 MB/s | 839 kB 00:00 2024-05-13T22:37:11.679 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:23 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:11.801 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:11.802 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:37:11.802 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-05-13T22:37:11.802 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:37:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:11.802 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-05-13T22:37:11.802 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:11.803 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 31 k 2024-05-13T22:37:11.803 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 139 k 2024-05-13T22:37:11.803 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:37:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 424 kB/s | 31 kB 00:00 2024-05-13T22:37:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:11.876 INFO:teuthology.orchestra.run.smithi102.stdout:Total 416 kB/s | 31 kB 00:00 2024-05-13T22:37:11.877 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:37:11.880 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:37:11.880 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:37:11.905 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:37:11.905 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:37:11.984 INFO:teuthology.orchestra.run.smithi130.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 191 kB/s | 80 kB 00:00 2024-05-13T22:37:11.984 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:11.985 INFO:teuthology.orchestra.run.smithi130.stdout:Total 1.6 MB/s | 920 kB 00:00 2024-05-13T22:37:11.987 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:37:11.997 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:37:12.019 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:37:12.020 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:37:12.090 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:12.097 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:37:12.097 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:37:12.280 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:37:12.415 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-13T22:37:12.488 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:12.499 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-05-13T22:37:12.707 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:12.707 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:37:12.707 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-05-13T22:37:12.707 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:12.908 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados2 2024-05-13T22:37:13.193 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-05-13T22:37:13.194 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-05-13T22:37:13.467 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-13T22:37:13.467 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:13.467 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:37:13.467 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-13T22:37:13.468 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:13.468 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:13.484 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:25 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:13.565 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:13.606 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-05-13T22:37:13.607 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:13.671 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librbd1 2024-05-13T22:37:13.763 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install librados-devel 2024-05-13T22:37:14.245 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:26 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:14.326 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:14.345 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:41 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:14.366 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:14.368 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-05-13T22:37:14.368 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:14.431 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rados 2024-05-13T22:37:14.466 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:14.467 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:14.467 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:14.467 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:14.467 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:37:14.467 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-05-13T22:37:14.467 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:14.467 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:37:14.467 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:14.468 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-05-13T22:37:14.468 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:14.468 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 127 k 2024-05-13T22:37:14.468 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 456 k 2024-05-13T22:37:14.468 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:37:14.529 INFO:teuthology.orchestra.run.smithi130.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 2.0 MB/s | 127 kB 00:00 2024-05-13T22:37:14.530 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:14.530 INFO:teuthology.orchestra.run.smithi130.stdout:Total 2.0 MB/s | 127 kB 00:00 2024-05-13T22:37:14.530 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:37:14.535 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:37:14.535 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:37:14.598 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:37:14.598 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:37:14.748 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:37:14.842 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:15.004 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:27 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:15.085 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:15.127 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:15.128 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-05-13T22:37:15.128 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:15.188 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rgw 2024-05-13T22:37:15.267 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:15.510 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:15.510 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:15.510 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:37:15.510 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-05-13T22:37:15.510 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:15.510 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:15.722 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install libcephfs2 2024-05-13T22:37:15.758 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:27 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:15.840 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:15.881 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-05-13T22:37:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:15.946 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-cephfs 2024-05-13T22:37:16.307 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:43 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:16.389 INFO:teuthology.orchestra.run.smithi130.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:16.432 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:16.433 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-05-13T22:37:16.433 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:16.495 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install libcephfs-devel 2024-05-13T22:37:16.510 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:28 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:16.589 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:16.630 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:16.631 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-05-13T22:37:16.631 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:16.700 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rbd 2024-05-13T22:37:17.068 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:44 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:17.189 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:17.190 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:17.190 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:17.190 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:17.190 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:37:17.191 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-05-13T22:37:17.191 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:17.191 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:37:17.191 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:17.191 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-05-13T22:37:17.191 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:17.191 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 31 k 2024-05-13T22:37:17.191 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 139 k 2024-05-13T22:37:17.191 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:37:17.265 INFO:teuthology.orchestra.run.smithi130.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 417 kB/s | 31 kB 00:00 2024-05-13T22:37:17.266 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:17.266 INFO:teuthology.orchestra.run.smithi130.stdout:Total 409 kB/s | 31 kB 00:00 2024-05-13T22:37:17.266 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:37:17.270 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:37:17.270 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:37:17.270 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:29 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:17.294 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:37:17.294 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:37:17.351 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:17.370 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:37:17.393 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:17.394 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-05-13T22:37:17.394 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:17.463 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-fuse 2024-05-13T22:37:17.479 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:17.843 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:18.036 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:30 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:18.055 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:18.056 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:18.056 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:37:18.056 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-05-13T22:37:18.056 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:18.056 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:18.158 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:18.158 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 86 k 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 226 k 2024-05-13T22:37:18.159 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:37:18.230 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.2 MB/s | 86 kB 00:00 2024-05-13T22:37:18.230 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:18.230 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.2 MB/s | 86 kB 00:00 2024-05-13T22:37:18.230 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:37:18.237 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:37:18.237 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:37:18.256 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install librados2 2024-05-13T22:37:18.303 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:37:18.303 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:37:18.450 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:37:18.543 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:18.832 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:45 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:18.912 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:18.914 INFO:teuthology.orchestra.run.smithi130.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:18.954 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:18.955 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-05-13T22:37:18.956 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:19.020 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install librbd1 2024-05-13T22:37:19.147 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:19.148 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:19.148 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:37:19.148 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-05-13T22:37:19.148 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:19.148 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:19.384 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-mirror 2024-05-13T22:37:19.594 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:46 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:19.676 INFO:teuthology.orchestra.run.smithi130.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:19.717 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:19.718 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-05-13T22:37:19.718 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:19.780 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install python3-rados 2024-05-13T22:37:19.970 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:31 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:20.095 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:20.096 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:20.096 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:20.096 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:37:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-05-13T22:37:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:20.096 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:37:20.096 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:20.096 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-05-13T22:37:20.096 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:20.097 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 3.0 M 2024-05-13T22:37:20.097 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 12 M 2024-05-13T22:37:20.097 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:37:20.200 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 29 MB/s | 3.0 MB 00:00 2024-05-13T22:37:20.200 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:20.201 INFO:teuthology.orchestra.run.smithi102.stdout:Total 29 MB/s | 3.0 MB 00:00 2024-05-13T22:37:20.201 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:37:20.210 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:37:20.210 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:37:20.289 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:37:20.289 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:37:20.354 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:47 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:20.435 INFO:teuthology.orchestra.run.smithi130.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:20.477 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:20.478 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-05-13T22:37:20.478 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:20.546 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install python3-rgw 2024-05-13T22:37:20.705 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:37:20.767 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:20.792 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:20.792 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:37:20.792 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-13T22:37:20.792 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-13T22:37:20.792 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-13T22:37:20.792 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:21.122 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:48 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:21.202 INFO:teuthology.orchestra.run.smithi130.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:21.244 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:21.245 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-05-13T22:37:21.245 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:21.310 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install python3-cephfs 2024-05-13T22:37:21.756 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:21.756 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:21.756 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:37:21.756 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-05-13T22:37:21.756 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:21.756 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:21.882 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:48 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:21.962 INFO:teuthology.orchestra.run.smithi130.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:21.986 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-nbd 2024-05-13T22:37:22.003 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:22.004 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-05-13T22:37:22.004 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:22.068 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install python3-rbd 2024-05-13T22:37:22.570 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:34 ago on Mon 13 May 2024 10:35:48 PM UTC. 2024-05-13T22:37:22.645 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:49 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:22.694 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-05-13T22:37:22.695 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:22.695 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:22.695 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-05-13T22:37:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-05-13T22:37:22.695 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:22.695 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-05-13T22:37:22.696 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-05-13T22:37:22.696 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-05-13T22:37:22.696 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:22.696 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 171 k 2024-05-13T22:37:22.696 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 494 k 2024-05-13T22:37:22.696 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-05-13T22:37:22.726 INFO:teuthology.orchestra.run.smithi130.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-05-13T22:37:22.762 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-05-13T22:37:22.762 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:22.763 INFO:teuthology.orchestra.run.smithi102.stdout:Total 2.5 MB/s | 171 kB 00:00 2024-05-13T22:37:22.763 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-05-13T22:37:22.770 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-05-13T22:37:22.770 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-05-13T22:37:22.771 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:22.773 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-05-13T22:37:22.773 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:22.832 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-05-13T22:37:22.832 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-05-13T22:37:22.833 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install rbd-fuse 2024-05-13T22:37:22.992 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-05-13T22:37:23.084 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:23.414 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:50 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:23.481 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:23.537 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:23.537 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:23.538 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:23.538 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:23.538 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:37:23.538 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-05-13T22:37:23.538 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:23.538 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:37:23.538 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:23.538 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-05-13T22:37:23.538 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:23.539 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 86 k 2024-05-13T22:37:23.539 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 226 k 2024-05-13T22:37:23.539 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:37:23.601 INFO:teuthology.orchestra.run.smithi130.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.3 MB/s | 86 kB 00:00 2024-05-13T22:37:23.602 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:23.602 INFO:teuthology.orchestra.run.smithi130.stdout:Total 1.3 MB/s | 86 kB 00:00 2024-05-13T22:37:23.602 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:37:23.609 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:37:23.609 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:37:23.671 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:37:23.671 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:37:23.729 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:23.729 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:23.730 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-05-13T22:37:23.730 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-05-13T22:37:23.730 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:37:23.730 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-05-13T22:37:23.815 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:37:23.907 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:23.974 DEBUG:teuthology.parallel:result is None 2024-05-13T22:37:24.255 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:24.497 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:24.497 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:24.497 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:37:24.497 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-05-13T22:37:24.497 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:24.497 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:24.745 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install rbd-mirror 2024-05-13T22:37:25.330 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:52 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:25.456 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 3.0 M 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 12 M 2024-05-13T22:37:25.457 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:37:25.562 INFO:teuthology.orchestra.run.smithi130.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 29 MB/s | 3.0 MB 00:00 2024-05-13T22:37:25.563 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:25.563 INFO:teuthology.orchestra.run.smithi130.stdout:Total 28 MB/s | 3.0 MB 00:00 2024-05-13T22:37:25.563 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:37:25.572 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:37:25.573 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:37:25.652 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:37:25.652 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:37:26.080 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:37:26.238 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:26.265 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:26.265 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-13T22:37:26.265 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-13T22:37:26.265 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-13T22:37:26.265 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-13T22:37:26.265 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:27.228 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:27.229 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:27.229 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:37:27.229 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-05-13T22:37:27.229 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:27.229 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:27.456 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install rbd-nbd 2024-05-13T22:37:28.048 INFO:teuthology.orchestra.run.smithi130.stdout:Last metadata expiration check: 0:01:55 ago on Mon 13 May 2024 10:35:33 PM UTC. 2024-05-13T22:37:28.172 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-05-13T22:37:28.173 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:28.173 INFO:teuthology.orchestra.run.smithi130.stdout: Package Architecture Version Repository Size 2024-05-13T22:37:28.173 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:28.173 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-05-13T22:37:28.173 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-05-13T22:37:28.173 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:28.174 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-05-13T22:37:28.174 INFO:teuthology.orchestra.run.smithi130.stdout:================================================================================ 2024-05-13T22:37:28.174 INFO:teuthology.orchestra.run.smithi130.stdout:Install 1 Package 2024-05-13T22:37:28.174 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:28.174 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 171 k 2024-05-13T22:37:28.174 INFO:teuthology.orchestra.run.smithi130.stdout:Installed size: 494 k 2024-05-13T22:37:28.174 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-05-13T22:37:28.231 INFO:teuthology.orchestra.run.smithi130.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.9 MB/s | 171 kB 00:00 2024-05-13T22:37:28.232 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:28.232 INFO:teuthology.orchestra.run.smithi130.stdout:Total 2.8 MB/s | 171 kB 00:00 2024-05-13T22:37:28.232 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-05-13T22:37:28.240 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-05-13T22:37:28.240 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-05-13T22:37:28.302 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-05-13T22:37:28.302 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-05-13T22:37:28.465 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-05-13T22:37:28.553 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:28.995 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:29.232 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-05-13T22:37:29.232 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:29.232 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-05-13T22:37:29.232 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-05-13T22:37:29.232 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:37:29.232 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-05-13T22:37:29.521 DEBUG:teuthology.parallel:result is None 2024-05-13T22:37:29.521 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-13T22:37:29.521 INFO:teuthology.packaging:ref: None 2024-05-13T22:37:29.521 INFO:teuthology.packaging:tag: v18.2.1 2024-05-13T22:37:29.522 INFO:teuthology.packaging:branch: None 2024-05-13T22:37:29.522 INFO:teuthology.packaging:sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:37:29.522 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-05-13T22:37:29.739 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-13T22:37:29.764 INFO:teuthology.orchestra.run.smithi102.stdout:18.2.1-0.el9 2024-05-13T22:37:29.765 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-05-13T22:37:29.765 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-05-13T22:37:29.767 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-13T22:37:29.767 INFO:teuthology.packaging:ref: None 2024-05-13T22:37:29.767 INFO:teuthology.packaging:tag: v18.2.1 2024-05-13T22:37:29.767 INFO:teuthology.packaging:branch: None 2024-05-13T22:37:29.767 INFO:teuthology.packaging:sha1: c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:37:29.768 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-05-13T22:37:29.908 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-13T22:37:29.929 INFO:teuthology.orchestra.run.smithi130.stdout:18.2.1-0.el9 2024-05-13T22:37:29.929 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-05-13T22:37:29.930 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-05-13T22:37:29.932 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-13T22:37:29.932 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:37:29.932 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-13T22:37:29.962 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:37:29.962 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-13T22:37:29.995 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-13T22:37:29.995 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:37:29.995 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-05-13T22:37:30.030 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-13T22:37:30.097 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:37:30.097 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/daemon-helper 2024-05-13T22:37:30.122 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-13T22:37:30.187 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-13T22:37:30.187 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:37:30.187 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-13T22:37:30.214 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-13T22:37:30.280 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:37:30.280 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-13T22:37:30.304 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-13T22:37:30.369 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-13T22:37:30.369 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:37:30.369 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-05-13T22:37:30.395 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-13T22:37:30.460 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:37:30.460 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/stdin-killer 2024-05-13T22:37:30.484 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-13T22:37:30.549 INFO:teuthology.run_tasks:Running task print... 2024-05-13T22:37:30.559 INFO:teuthology.task.print:**** done install task... 2024-05-13T22:37:30.560 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-13T22:37:30.723 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': 'c3e55145f4f4b3a500bb66ee08c938a215bc231d'} 2024-05-13T22:37:30.723 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-05-13T22:37:30.723 INFO:tasks.cephadm:Cluster fsid is 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:37:30.723 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-13T22:37:30.724 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-05-13T22:37:30.724 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi102': '172.21.15.102', 'mon.smithi130': '172.21.15.130'} 2024-05-13T22:37:30.724 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-13T22:37:30.724 DEBUG:teuthology.orchestra.run.smithi102:> sudo hostname $(hostname -s) 2024-05-13T22:37:30.751 DEBUG:teuthology.orchestra.run.smithi130:> sudo hostname $(hostname -s) 2024-05-13T22:37:30.776 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-05-13T22:37:30.777 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:37:31.001 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c3e55145f4f4b3a500bb66ee08c938a215bc231d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55140/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2565-gc3e55145', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2565.gc3e55145'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/c3e55145f4f4b3a500bb66ee08c938a215bc231d/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-13 17:39:34.721604', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/c3e55145f4f4b3a500bb66ee08c938a215bc231d/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-13T22:37:31.335 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef, sha1 2965081fb1a343a09e00ceeede998525f5c6cb5e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-05-13T22:37:31.337 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/x86_64/flavors/default/cephadm 2024-05-13T22:37:31.337 DEBUG:teuthology.orchestra.run.smithi102:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-13T22:37:31.854 INFO:teuthology.orchestra.run.smithi102.stdout:-rw-r--r--. 1 ubuntu ubuntu 215331 May 13 22:37 /home/ubuntu/cephtest/cephadm 2024-05-13T22:37:31.860 DEBUG:teuthology.orchestra.run.smithi130:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/2965081fb1a343a09e00ceeede998525f5c6cb5e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-13T22:37:32.379 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-r--r--. 1 ubuntu ubuntu 215331 May 13 22:37 /home/ubuntu/cephtest/cephadm 2024-05-13T22:37:32.380 DEBUG:teuthology.orchestra.run.smithi102:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-13T22:37:32.399 DEBUG:teuthology.orchestra.run.smithi130:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-13T22:37:32.429 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-05-13T22:37:32.430 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-05-13T22:37:32.441 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-05-13T22:37:33.311 INFO:teuthology.orchestra.run.smithi102.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-05-13T22:37:33.375 INFO:teuthology.orchestra.run.smithi130.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-05-13T22:37:52.970 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:37:52.971 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-05-13T22:37:52.971 INFO:teuthology.orchestra.run.smithi102.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-05-13T22:37:52.971 INFO:teuthology.orchestra.run.smithi102.stdout: "repo_digests": [ 2024-05-13T22:37:52.971 INFO:teuthology.orchestra.run.smithi102.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-05-13T22:37:52.971 INFO:teuthology.orchestra.run.smithi102.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-05-13T22:37:52.971 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-05-13T22:37:52.971 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:37:54.253 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-05-13T22:37:54.253 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-05-13T22:37:54.253 INFO:teuthology.orchestra.run.smithi130.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-05-13T22:37:54.253 INFO:teuthology.orchestra.run.smithi130.stdout: "repo_digests": [ 2024-05-13T22:37:54.254 INFO:teuthology.orchestra.run.smithi130.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-05-13T22:37:54.254 INFO:teuthology.orchestra.run.smithi130.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-05-13T22:37:54.254 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2024-05-13T22:37:54.254 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-05-13T22:37:54.271 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph 2024-05-13T22:37:54.304 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph 2024-05-13T22:37:54.343 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /etc/ceph 2024-05-13T22:37:54.372 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 777 /etc/ceph 2024-05-13T22:37:54.411 INFO:tasks.cephadm:Writing seed config... 2024-05-13T22:37:54.412 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-05-13T22:37:54.412 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-05-13T22:37:54.412 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-13T22:37:54.412 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-13T22:37:54.413 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-13T22:37:54.414 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-13T22:37:54.414 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-13T22:37:54.414 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-13T22:37:54.415 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-13T22:37:54.415 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-13T22:37:54.415 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-05-13T22:37:54.415 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-13T22:37:54.415 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-13T22:37:54.415 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-13T22:37:54.415 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-13T22:37:54.415 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-13T22:37:54.415 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-13T22:37:54.415 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-13T22:37:54.416 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-13T22:37:54.416 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-13T22:37:54.416 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-13T22:37:54.416 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-13T22:37:54.416 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-13T22:37:54.416 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-13T22:37:54.416 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-13T22:37:54.416 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-13T22:37:54.416 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-13T22:37:54.416 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-13T22:37:54.416 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-13T22:37:54.417 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:37:54.417 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-13T22:37:54.435 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 62275d5c-1179-11ef-bc98-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-13T22:37:54.435 DEBUG:teuthology.orchestra.run.smithi102:mon.smithi102> sudo journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi102.service 2024-05-13T22:37:54.478 INFO:tasks.cephadm:Bootstrapping... 2024-05-13T22:37:54.478 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid 62275d5c-1179-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.102 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-13T22:37:54.637 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-05-13T22:37:54.637 INFO:teuthology.orchestra.run.smithi102.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', '62275d5c-1179-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.102', '--skip-admin-label'] 2024-05-13T22:37:54.662 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stdout 5.0.2 2024-05-13T22:37:54.662 INFO:teuthology.orchestra.run.smithi102.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-13T22:37:54.662 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying podman|docker is present... 2024-05-13T22:37:54.687 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stdout 5.0.2 2024-05-13T22:37:54.687 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying lvm2 is present... 2024-05-13T22:37:54.687 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying time synchronization is in place... 2024-05-13T22:37:54.695 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-13T22:37:54.695 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-13T22:37:54.701 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-13T22:37:54.701 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-05-13T22:37:54.708 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout enabled 2024-05-13T22:37:54.714 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout active 2024-05-13T22:37:54.714 INFO:teuthology.orchestra.run.smithi102.stdout:Unit chronyd.service is enabled and running 2024-05-13T22:37:54.714 INFO:teuthology.orchestra.run.smithi102.stdout:Repeating the final host check... 2024-05-13T22:37:54.737 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stdout 5.0.2 2024-05-13T22:37:54.738 INFO:teuthology.orchestra.run.smithi102.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-13T22:37:54.738 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl is present 2024-05-13T22:37:54.738 INFO:teuthology.orchestra.run.smithi102.stdout:lvcreate is present 2024-05-13T22:37:54.744 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-13T22:37:54.744 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-13T22:37:54.750 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-13T22:37:54.750 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-05-13T22:37:54.757 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout enabled 2024-05-13T22:37:54.763 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout active 2024-05-13T22:37:54.763 INFO:teuthology.orchestra.run.smithi102.stdout:Unit chronyd.service is enabled and running 2024-05-13T22:37:54.763 INFO:teuthology.orchestra.run.smithi102.stdout:Host looks OK 2024-05-13T22:37:54.763 INFO:teuthology.orchestra.run.smithi102.stdout:Cluster fsid: 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:37:54.763 INFO:teuthology.orchestra.run.smithi102.stdout:Acquiring lock 140373701280048 on /run/cephadm/62275d5c-1179-11ef-bc98-c7b262605968.lock 2024-05-13T22:37:54.763 INFO:teuthology.orchestra.run.smithi102.stdout:Lock 140373701280048 acquired on /run/cephadm/62275d5c-1179-11ef-bc98-c7b262605968.lock 2024-05-13T22:37:54.764 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying IP 172.21.15.102 port 3300 ... 2024-05-13T22:37:54.764 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying IP 172.21.15.102 port 6789 ... 2024-05-13T22:37:54.764 INFO:teuthology.orchestra.run.smithi102.stdout:Base mon IP(s) is [172.21.15.102:3300, 172.21.15.102:6789], mon addrv is [v2:172.21.15.102:3300,v1:172.21.15.102:6789] 2024-05-13T22:37:54.767 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.102 metric 100 2024-05-13T22:37:54.767 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.102 metric 100 2024-05-13T22:37:54.770 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-13T22:37:54.770 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-13T22:37:54.770 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-05-13T22:37:54.772 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-13T22:37:54.772 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-13T22:37:54.772 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-13T22:37:54.772 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-13T22:37:54.773 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d769/64 scope link 2024-05-13T22:37:54.773 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-13T22:37:54.773 INFO:teuthology.orchestra.run.smithi102.stdout:Mon IP `172.21.15.102` is in CIDR network `172.21.0.0/20` 2024-05-13T22:37:54.773 INFO:teuthology.orchestra.run.smithi102.stdout:Mon IP `172.21.15.102` is in CIDR network `172.21.0.0/20` 2024-05-13T22:37:54.773 INFO:teuthology.orchestra.run.smithi102.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-13T22:37:54.774 INFO:teuthology.orchestra.run.smithi102.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-13T22:37:54.774 INFO:teuthology.orchestra.run.smithi102.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-05-13T22:37:55.508 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stdout 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-05-13T22:37:55.508 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-05-13T22:37:55.508 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Getting image source signatures 2024-05-13T22:37:55.508 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Copying blob sha256:a733d3c618b71f19c168ebecd1953429dce2c1631835ca182e9551c36dce5989 2024-05-13T22:37:55.509 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-05-13T22:37:55.509 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Copying config sha256:5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-05-13T22:37:55.509 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-13T22:37:56.407 INFO:teuthology.orchestra.run.smithi102.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-05-13T22:37:56.407 INFO:teuthology.orchestra.run.smithi102.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-05-13T22:37:56.407 INFO:teuthology.orchestra.run.smithi102.stdout:Extracting ceph user uid/gid from container image... 2024-05-13T22:37:57.451 INFO:teuthology.orchestra.run.smithi102.stdout:stat: stdout 167 167 2024-05-13T22:37:57.451 INFO:teuthology.orchestra.run.smithi102.stdout:Creating initial keys... 2024-05-13T22:37:58.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph-authtool: stdout AQBFlkJmxV2HLxAAHCzWfPtTJgAHXo3mqh1THQ== 2024-05-13T22:38:04.336 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph-authtool: stdout AQBGlkJmZE/8KxAA78TQNr/rENbgZiazzy/xmQ== 2024-05-13T22:38:10.796 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph-authtool: stdout AQBPlkJmkxraCBAA5ZYv9rAVmpOapxVt4Bljaw== 2024-05-13T22:38:10.797 INFO:teuthology.orchestra.run.smithi102.stdout:Creating initial monmap... 2024-05-13T22:38:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-13T22:38:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-13T22:38:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:38:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-13T22:38:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool for smithi102 [v2:172.21.15.102:3300,v1:172.21.15.102:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-13T22:38:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:setting min_mon_release = pacific 2024-05-13T22:38:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: set fsid to 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:38:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-13T22:38:11.777 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:38:11.777 INFO:teuthology.orchestra.run.smithi102.stdout:Creating mon... 2024-05-13T22:38:12.699 INFO:teuthology.orchestra.run.smithi102.stdout:create mon.smithi102 on 2024-05-13T22:38:12.901 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-13T22:38:13.039 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-13T22:38:13.220 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-62275d5c-1179-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-62275d5c-1179-11ef-bc98-c7b262605968.target. 2024-05-13T22:38:13.221 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-62275d5c-1179-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-62275d5c-1179-11ef-bc98-c7b262605968.target. 2024-05-13T22:38:13.466 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi102 2024-05-13T22:38:13.466 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to reset failed state of unit ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi102.service: Unit ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi102.service not loaded. 2024-05-13T22:38:13.621 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-62275d5c-1179-11ef-bc98-c7b262605968.target.wants/ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi102.service → /etc/systemd/system/ceph-62275d5c-1179-11ef-bc98-c7b262605968@.service. 2024-05-13T22:38:14.122 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:14 smithi102 podman[28213]: 2024-05-13 22:38:14.100380963 +0000 UTC m=+0.333990140 container start 2e78a131d86f2ebfd7f974288bce605019ad6e4f90133042a46485d4224c55d3 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD) 2024-05-13T22:38:14.122 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:14 smithi102 ceph-mon[28231]: mkfs 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:38:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-13T22:38:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-05-13T22:38:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-13T22:38:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-05-13T22:38:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-05-13T22:38:14.172 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to enable service . firewalld.service is not available 2024-05-13T22:38:14.173 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mon to start... 2024-05-13T22:38:14.173 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mon... 2024-05-13T22:38:14.399 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:14 smithi102 ceph-mon[28231]: mon.smithi102 is new leader, mons smithi102 in quorum (ranks 0) 2024-05-13T22:38:14.399 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:14 smithi102 bash[28213]: 2e78a131d86f2ebfd7f974288bce605019ad6e4f90133042a46485d4224c55d3 2024-05-13T22:38:14.399 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:14 smithi102 systemd[1]: Started Ceph mon.smithi102 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:38:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout cluster: 2024-05-13T22:38:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout id: 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:38:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-13T22:38:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout services: 2024-05-13T22:38:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi102 (age 0.550553s) 2024-05-13T22:38:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-13T22:38:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-13T22:38:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout data: 2024-05-13T22:38:15.167 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout pgs: 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.614+0000 7f78ffee4700 1 Processor -- start 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.614+0000 7f78ffee4700 1 -- start start 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.614+0000 7f78ffee4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8109420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.614+0000 7f78ffee4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f8079e30 con 0x7f78f8107030 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.615+0000 7f78fdc80700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8109420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.615+0000 7f78fdc80700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8109420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43110/0 (socket says 172.21.15.102:43110) 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.615+0000 7f78fdc80700 1 -- 172.21.15.102:0/3567061743 learned_addr learned my addr 172.21.15.102:0/3567061743 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.615+0000 7f78fdc80700 1 -- 172.21.15.102:0/3567061743 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f8109960 con 0x7f78f8107030 2024-05-13T22:38:15.168 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.615+0000 7f78fdc80700 1 --2- 172.21.15.102:0/3567061743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8109420 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f78e80098a0 tx=0x7f78e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f891e59a02adfd9f server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.615+0000 7f78fcc7e700 1 -- 172.21.15.102:0/3567061743 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78e8004180 con 0x7f78f8107030 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.615+0000 7f78fcc7e700 1 -- 172.21.15.102:0/3567061743 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e80042e0 con 0x7f78f8107030 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.615+0000 7f78fcc7e700 1 -- 172.21.15.102:0/3567061743 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78e80036a0 con 0x7f78f8107030 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.616+0000 7f78ffee4700 1 -- 172.21.15.102:0/3567061743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 msgr2=0x7f78f8109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.616+0000 7f78ffee4700 1 --2- 172.21.15.102:0/3567061743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8109420 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f78e80098a0 tx=0x7f78e8009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.616+0000 7f78ffee4700 1 -- 172.21.15.102:0/3567061743 shutdown_connections 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.616+0000 7f78ffee4700 1 --2- 172.21.15.102:0/3567061743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8109420 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.616+0000 7f78ffee4700 1 -- 172.21.15.102:0/3567061743 >> 172.21.15.102:0/3567061743 conn(0x7f78f8100a20 msgr2=0x7f78f8102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.616+0000 7f78ffee4700 1 -- 172.21.15.102:0/3567061743 shutdown_connections 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.616+0000 7f78ffee4700 1 -- 172.21.15.102:0/3567061743 wait complete. 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.617+0000 7f78ffee4700 1 Processor -- start 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.617+0000 7f78ffee4700 1 -- start start 2024-05-13T22:38:15.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.617+0000 7f78ffee4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.617+0000 7f78ffee4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f8199e80 con 0x7f78f8107030 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.617+0000 7f78fdc80700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.617+0000 7f78fdc80700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43114/0 (socket says 172.21.15.102:43114) 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.617+0000 7f78fdc80700 1 -- 172.21.15.102:0/638733079 learned_addr learned my addr 172.21.15.102:0/638733079 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.618+0000 7f78fdc80700 1 -- 172.21.15.102:0/638733079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8009550 con 0x7f78f8107030 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.618+0000 7f78fdc80700 1 --2- 172.21.15.102:0/638733079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8199940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f78f8079270 tx=0x7f78e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.618+0000 7f78eeffd700 1 -- 172.21.15.102:0/638733079 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78e8003ce0 con 0x7f78f8107030 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.618+0000 7f78eeffd700 1 -- 172.21.15.102:0/638733079 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e8020070 con 0x7f78f8107030 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.618+0000 7f78ffee4700 1 -- 172.21.15.102:0/638733079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f819a080 con 0x7f78f8107030 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.618+0000 7f78eeffd700 1 -- 172.21.15.102:0/638733079 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78e8011630 con 0x7f78f8107030 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.618+0000 7f78ffee4700 1 -- 172.21.15.102:0/638733079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f819a520 con 0x7f78f8107030 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.618+0000 7f78eeffd700 1 -- 172.21.15.102:0/638733079 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f78e8022030 con 0x7f78f8107030 2024-05-13T22:38:15.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.618+0000 7f78eeffd700 1 -- 172.21.15.102:0/638733079 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f78e8025d00 con 0x7f78f8107030 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.619+0000 7f78ffee4700 1 -- 172.21.15.102:0/638733079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f8193860 con 0x7f78f8107030 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.621+0000 7f78eeffd700 1 -- 172.21.15.102:0/638733079 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f78e8025480 con 0x7f78f8107030 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.689+0000 7f78ffee4700 1 -- 172.21.15.102:0/638733079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f78f802cc70 con 0x7f78f8107030 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.690+0000 7f78eeffd700 1 -- 172.21.15.102:0/638733079 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f78e8011c20 con 0x7f78f8107030 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.690+0000 7f78ffee4700 1 -- 172.21.15.102:0/638733079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 msgr2=0x7f78f8199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.690+0000 7f78ffee4700 1 --2- 172.21.15.102:0/638733079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8199940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f78f8079270 tx=0x7f78e8004750 comp rx=0 tx=0).stop 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.690+0000 7f78ffee4700 1 -- 172.21.15.102:0/638733079 shutdown_connections 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.690+0000 7f78ffee4700 1 --2- 172.21.15.102:0/638733079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78f8107030 0x7f78f8199940 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.691+0000 7f78ffee4700 1 -- 172.21.15.102:0/638733079 >> 172.21.15.102:0/638733079 conn(0x7f78f8100a20 msgr2=0x7f78f8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.691+0000 7f78ffee4700 1 -- 172.21.15.102:0/638733079 shutdown_connections 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:14.691+0000 7f78ffee4700 1 -- 172.21.15.102:0/638733079 wait complete. 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:mon is available 2024-05-13T22:38:15.171 INFO:teuthology.orchestra.run.smithi102.stdout:Assimilating anything we can from ceph.conf... 2024-05-13T22:38:15.259 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:15 smithi102 ceph-mon[28231]: mon.smithi102 is new leader, mons smithi102 in quorum (ranks 0) 2024-05-13T22:38:15.259 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:15 smithi102 ceph-mon[28231]: monmap e1: 1 mons at {smithi102=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-13T22:38:15.259 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:15 smithi102 ceph-mon[28231]: fsmap 2024-05-13T22:38:15.259 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:15 smithi102 ceph-mon[28231]: osdmap e1: 0 total, 0 up, 0 in 2024-05-13T22:38:15.259 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:15 smithi102 ceph-mon[28231]: mgrmap e1: no daemons active 2024-05-13T22:38:15.259 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:15 smithi102 ceph-mon[28231]: from='client.? 172.21.15.102:0/638733079' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-13T22:38:16.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:16 smithi102 ceph-mon[28231]: from='client.? 172.21.15.102:0/2338205208' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-13T22:38:16.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:16 smithi102 ceph-mon[28231]: from='client.? 172.21.15.102:0/2338205208' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-13T22:38:17.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:17.217 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [global] 2024-05-13T22:38:17.217 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout fsid = 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:38:17.217 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.102:3300,v1:172.21.15.102:6789] 2024-05-13T22:38:17.217 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-13T22:38:17.217 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-13T22:38:17.217 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-13T22:38:17.217 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-13T22:38:17.217 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:17.217 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [osd] 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.580+0000 7f5029c16700 1 Processor -- start 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.580+0000 7f5029c16700 1 -- start start 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.580+0000 7f5029c16700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.580+0000 7f5029c16700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5024106f70 con 0x7f5024106580 2024-05-13T22:38:17.218 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.580+0000 7f50237fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.580+0000 7f50237fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43124/0 (socket says 172.21.15.102:43124) 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.580+0000 7f50237fe700 1 -- 172.21.15.102:0/1162137273 learned_addr learned my addr 172.21.15.102:0/1162137273 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.580+0000 7f50237fe700 1 -- 172.21.15.102:0/1162137273 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5024107780 con 0x7f5024106580 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.581+0000 7f50237fe700 1 --2- 172.21.15.102:0/1162137273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241069a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50180098a0 tx=0x7f5018009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e908c536d9eea4 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.581+0000 7f50227fc700 1 -- 172.21.15.102:0/1162137273 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5018004180 con 0x7f5024106580 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.581+0000 7f50227fc700 1 -- 172.21.15.102:0/1162137273 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50180042e0 con 0x7f5024106580 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.581+0000 7f50227fc700 1 -- 172.21.15.102:0/1162137273 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50180036a0 con 0x7f5024106580 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.581+0000 7f5029c16700 1 -- 172.21.15.102:0/1162137273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 msgr2=0x7f50241069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.581+0000 7f5029c16700 1 --2- 172.21.15.102:0/1162137273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241069a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50180098a0 tx=0x7f5018009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.581+0000 7f5029c16700 1 -- 172.21.15.102:0/1162137273 shutdown_connections 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.581+0000 7f5029c16700 1 --2- 172.21.15.102:0/1162137273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241069a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.581+0000 7f5029c16700 1 -- 172.21.15.102:0/1162137273 >> 172.21.15.102:0/1162137273 conn(0x7f5024101b00 msgr2=0x7f5024103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:17.219 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.581+0000 7f5029c16700 1 -- 172.21.15.102:0/1162137273 shutdown_connections 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.581+0000 7f5029c16700 1 -- 172.21.15.102:0/1162137273 wait complete. 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.582+0000 7f5029c16700 1 Processor -- start 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.582+0000 7f5029c16700 1 -- start start 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.582+0000 7f5029c16700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.582+0000 7f5029c16700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5024199f20 con 0x7f5024106580 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.582+0000 7f50237fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.582+0000 7f50237fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43140/0 (socket says 172.21.15.102:43140) 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.582+0000 7f50237fe700 1 -- 172.21.15.102:0/2338205208 learned_addr learned my addr 172.21.15.102:0/2338205208 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.583+0000 7f50237fe700 1 -- 172.21.15.102:0/2338205208 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5018009550 con 0x7f5024106580 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.583+0000 7f50237fe700 1 --2- 172.21.15.102:0/2338205208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241999e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5018004000 tx=0x7f5018004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.583+0000 7f5020ff9700 1 -- 172.21.15.102:0/2338205208 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5018003ce0 con 0x7f5024106580 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.583+0000 7f5020ff9700 1 -- 172.21.15.102:0/2338205208 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5018020070 con 0x7f5024106580 2024-05-13T22:38:17.220 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.583+0000 7f5029c16700 1 -- 172.21.15.102:0/2338205208 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502419a120 con 0x7f5024106580 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.583+0000 7f5020ff9700 1 -- 172.21.15.102:0/2338205208 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5018011630 con 0x7f5024106580 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.583+0000 7f5029c16700 1 -- 172.21.15.102:0/2338205208 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502419a5c0 con 0x7f5024106580 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.583+0000 7f5020ff9700 1 -- 172.21.15.102:0/2338205208 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5018022030 con 0x7f5024106580 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.583+0000 7f5020ff9700 1 -- 172.21.15.102:0/2338205208 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5018025d00 con 0x7f5024106580 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.584+0000 7f5029c16700 1 -- 172.21.15.102:0/2338205208 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50241932c0 con 0x7f5024106580 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.586+0000 7f5020ff9700 1 -- 172.21.15.102:0/2338205208 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f5018025480 con 0x7f5024106580 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.652+0000 7f5029c16700 1 -- 172.21.15.102:0/2338205208 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f502402cf20 con 0x7f5024106580 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.655+0000 7f5020ff9700 1 -- 172.21.15.102:0/2338205208 <== mon.0 v2:172.21.15.102:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5018011c20 con 0x7f5024106580 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.655+0000 7f5020ff9700 1 -- 172.21.15.102:0/2338205208 <== mon.0 v2:172.21.15.102:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f501801c500 con 0x7f5024106580 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.656+0000 7f5029c16700 1 -- 172.21.15.102:0/2338205208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 msgr2=0x7f50241999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.656+0000 7f5029c16700 1 --2- 172.21.15.102:0/2338205208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241999e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5018004000 tx=0x7f5018004750 comp rx=0 tx=0).stop 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.656+0000 7f5029c16700 1 -- 172.21.15.102:0/2338205208 shutdown_connections 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.656+0000 7f5029c16700 1 --2- 172.21.15.102:0/2338205208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5024106580 0x7f50241999e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:17.221 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.656+0000 7f5029c16700 1 -- 172.21.15.102:0/2338205208 >> 172.21.15.102:0/2338205208 conn(0x7f5024101b00 msgr2=0x7f502418df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:17.222 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.656+0000 7f5029c16700 1 -- 172.21.15.102:0/2338205208 shutdown_connections 2024-05-13T22:38:17.222 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:15.656+0000 7f5029c16700 1 -- 172.21.15.102:0/2338205208 wait complete. 2024-05-13T22:38:17.222 INFO:teuthology.orchestra.run.smithi102.stdout:Generating new minimal ceph.conf... 2024-05-13T22:38:17.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:17 smithi102 ceph-mon[28231]: from='client.? 172.21.15.102:0/503310812' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:38:18.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.687+0000 7f338147f700 1 Processor -- start 2024-05-13T22:38:18.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.687+0000 7f338147f700 1 -- start start 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.687+0000 7f338147f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c106960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.687+0000 7f338147f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c106f30 con 0x7f337c106540 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.688+0000 7f337affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c106960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.688+0000 7f337affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c106960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43144/0 (socket says 172.21.15.102:43144) 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.688+0000 7f337affd700 1 -- 172.21.15.102:0/67885983 learned_addr learned my addr 172.21.15.102:0/67885983 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.688+0000 7f337affd700 1 -- 172.21.15.102:0/67885983 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337c107740 con 0x7f337c106540 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.688+0000 7f337affd700 1 --2- 172.21.15.102:0/67885983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c106960 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f336c0098a0 tx=0x7f336c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddb19c35f852f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.688+0000 7f337a7fc700 1 -- 172.21.15.102:0/67885983 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f336c004180 con 0x7f337c106540 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.688+0000 7f337a7fc700 1 -- 172.21.15.102:0/67885983 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f336c0042e0 con 0x7f337c106540 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.688+0000 7f337a7fc700 1 -- 172.21.15.102:0/67885983 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f336c011a50 con 0x7f337c106540 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.689+0000 7f338147f700 1 -- 172.21.15.102:0/67885983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 msgr2=0x7f337c106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.689+0000 7f338147f700 1 --2- 172.21.15.102:0/67885983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c106960 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f336c0098a0 tx=0x7f336c009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:18.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.689+0000 7f338147f700 1 -- 172.21.15.102:0/67885983 shutdown_connections 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.689+0000 7f338147f700 1 --2- 172.21.15.102:0/67885983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c106960 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.689+0000 7f338147f700 1 -- 172.21.15.102:0/67885983 >> 172.21.15.102:0/67885983 conn(0x7f337c101b00 msgr2=0x7f337c103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.689+0000 7f338147f700 1 -- 172.21.15.102:0/67885983 shutdown_connections 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.689+0000 7f338147f700 1 -- 172.21.15.102:0/67885983 wait complete. 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.689+0000 7f338147f700 1 Processor -- start 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.690+0000 7f338147f700 1 -- start start 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.690+0000 7f338147f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c07bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.690+0000 7f338147f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c07c170 con 0x7f337c106540 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.690+0000 7f337affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c07bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.690+0000 7f337affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c07bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43156/0 (socket says 172.21.15.102:43156) 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.690+0000 7f337affd700 1 -- 172.21.15.102:0/503310812 learned_addr learned my addr 172.21.15.102:0/503310812 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.690+0000 7f337affd700 1 -- 172.21.15.102:0/503310812 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c009550 con 0x7f337c106540 2024-05-13T22:38:18.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.690+0000 7f337affd700 1 --2- 172.21.15.102:0/503310812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c07bc30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f336c003de0 tx=0x7f336c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.691+0000 7f3378ff9700 1 -- 172.21.15.102:0/503310812 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f336c003930 con 0x7f337c106540 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.691+0000 7f3378ff9700 1 -- 172.21.15.102:0/503310812 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f336c003a90 con 0x7f337c106540 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.691+0000 7f338147f700 1 -- 172.21.15.102:0/503310812 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337c07a280 con 0x7f337c106540 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.691+0000 7f3378ff9700 1 -- 172.21.15.102:0/503310812 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f336c0196c0 con 0x7f337c106540 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.691+0000 7f338147f700 1 -- 172.21.15.102:0/503310812 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337c07a720 con 0x7f337c106540 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.691+0000 7f3378ff9700 1 -- 172.21.15.102:0/503310812 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f336c019c30 con 0x7f337c106540 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.691+0000 7f3378ff9700 1 -- 172.21.15.102:0/503310812 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f336c02c3a0 con 0x7f337c106540 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.692+0000 7f338147f700 1 -- 172.21.15.102:0/503310812 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337c0623c0 con 0x7f337c106540 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.694+0000 7f3378ff9700 1 -- 172.21.15.102:0/503310812 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f336c01abc0 con 0x7f337c106540 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.760+0000 7f338147f700 1 -- 172.21.15.102:0/503310812 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f337c10a940 con 0x7f337c106540 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.760+0000 7f3378ff9700 1 -- 172.21.15.102:0/503310812 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f336c01a430 con 0x7f337c106540 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.761+0000 7f338147f700 1 -- 172.21.15.102:0/503310812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 msgr2=0x7f337c07bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.761+0000 7f338147f700 1 --2- 172.21.15.102:0/503310812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c07bc30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f336c003de0 tx=0x7f336c011ab0 comp rx=0 tx=0).stop 2024-05-13T22:38:18.323 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.761+0000 7f338147f700 1 -- 172.21.15.102:0/503310812 shutdown_connections 2024-05-13T22:38:18.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.761+0000 7f338147f700 1 --2- 172.21.15.102:0/503310812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f337c106540 0x7f337c07bc30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:18.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.761+0000 7f338147f700 1 -- 172.21.15.102:0/503310812 >> 172.21.15.102:0/503310812 conn(0x7f337c101b00 msgr2=0x7f337c1031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:18.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.761+0000 7f338147f700 1 -- 172.21.15.102:0/503310812 shutdown_connections 2024-05-13T22:38:18.324 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:17.761+0000 7f338147f700 1 -- 172.21.15.102:0/503310812 wait complete. 2024-05-13T22:38:18.324 INFO:teuthology.orchestra.run.smithi102.stdout:Restarting the monitor... 2024-05-13T22:38:18.593 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:18 smithi102 systemd[1]: Stopping Ceph mon.smithi102 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:38:18.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:18 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102[28227]: 2024-05-13T22:38:18.612+0000 7f94b3a39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi102 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T22:38:18.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:18 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102[28227]: 2024-05-13T22:38:18.612+0000 7f94b3a39700 -1 mon.smithi102@0(leader) e1 *** Got Signal Terminated *** 2024-05-13T22:38:18.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:18 smithi102 podman[28477]: 2024-05-13 22:38:18.776507165 +0000 UTC m=+0.268496908 container died 2e78a131d86f2ebfd7f974288bce605019ad6e4f90133042a46485d4224c55d3 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD) 2024-05-13T22:38:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:19 smithi102 podman[28477]: 2024-05-13 22:38:19.137218796 +0000 UTC m=+0.629208534 container cleanup 2e78a131d86f2ebfd7f974288bce605019ad6e4f90133042a46485d4224c55d3 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T22:38:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:19 smithi102 bash[28477]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102 2024-05-13T22:38:19.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:19 smithi102 podman[28489]: 2024-05-13 22:38:19.446698669 +0000 UTC m=+0.672718665 container remove 2e78a131d86f2ebfd7f974288bce605019ad6e4f90133042a46485d4224c55d3 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, RELEASE=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T22:38:19.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:19 smithi102 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi102.service: Deactivated successfully. 2024-05-13T22:38:19.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:19 smithi102 systemd[1]: Stopped Ceph mon.smithi102 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:38:19.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:19 smithi102 systemd[1]: Starting Ceph mon.smithi102 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:38:20.008 INFO:teuthology.orchestra.run.smithi102.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-13T22:38:20.050 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:19 smithi102 podman[28581]: 2024-05-13 22:38:19.805778433 +0000 UTC m=+0.095698429 container create 73e113d0d1b38bc6b4bcdf7197137859284183a31e8c14b26f4f658b6b07b284 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, RELEASE=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T22:38:20.050 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:19 smithi102 podman[28581]: 2024-05-13 22:38:19.726956074 +0000 UTC m=+0.016876096 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph:v18.2.1 2024-05-13T22:38:20.050 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:19 smithi102 podman[28581]: 2024-05-13 22:38:19.975281312 +0000 UTC m=+0.265201313 container init 73e113d0d1b38bc6b4bcdf7197137859284183a31e8c14b26f4f658b6b07b284 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, org.label-schema.license=GPLv2, ceph=True, RELEASE=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:19 smithi102 podman[28581]: 2024-05-13 22:38:19.978387691 +0000 UTC m=+0.268307693 container start 73e113d0d1b38bc6b4bcdf7197137859284183a31e8c14b26f4f658b6b07b284 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: set uid:gid to 167:167 (ceph:ceph) 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: pidfile_write: ignore empty --pid-file 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: load: jerasure load: lrc 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: RocksDB version: 7.9.2 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Git sha 0 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Compile date 2023-12-11 22:07:34 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: DB SUMMARY 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: DB Session ID: IXMVARTHCYZ4H18ZNAIF 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: CURRENT file: CURRENT 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: IDENTITY file: IDENTITY 2024-05-13T22:38:20.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi102/store.db dir, Total Num: 1, files: 000008.sst 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi102/store.db: 000009.log size: 98381 ; 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.error_if_exists: 0 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.create_if_missing: 0 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.paranoid_checks: 1 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.env: 0x55bef64be720 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.fs: PosixFileSystem 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.info_log: 0x55bef808b2e0 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_file_opening_threads: 16 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.statistics: (nil) 2024-05-13T22:38:20.052 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.use_fsync: 0 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_log_file_size: 0 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.keep_log_file_num: 1000 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.recycle_log_file_num: 0 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.allow_fallocate: 1 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.allow_mmap_reads: 0 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.allow_mmap_writes: 0 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.use_direct_reads: 0 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.create_missing_column_families: 0 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.db_log_dir: 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.wal_dir: 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-13T22:38:20.053 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.advise_random_on_open: 1 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.db_write_buffer_size: 0 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.write_buffer_manager: 0x55bef72fc140 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.rate_limiter: (nil) 2024-05-13T22:38:20.054 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-13T22:38:20.055 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.wal_recovery_mode: 2 2024-05-13T22:38:20.055 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.enable_thread_tracking: 0 2024-05-13T22:38:20.055 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.enable_pipelined_write: 0 2024-05-13T22:38:20.056 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.unordered_write: 0 2024-05-13T22:38:20.056 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-13T22:38:20.056 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-13T22:38:20.056 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-13T22:38:20.056 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-13T22:38:20.056 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.row_cache: None 2024-05-13T22:38:20.056 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.wal_filter: None 2024-05-13T22:38:20.056 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.allow_ingest_behind: 0 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.two_write_queues: 0 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.manual_wal_flush: 0 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.wal_compression: 0 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.atomic_flush: 0 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.log_readahead_size: 0 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.best_efforts_recovery: 0 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-13T22:38:20.057 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.allow_data_in_errors: 0 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.db_host_id: __hostname__ 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_background_jobs: 2 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_background_compactions: -1 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_subcompactions: 1 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_total_wal_size: 0 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-13T22:38:20.058 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_open_files: -1 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bytes_per_sync: 0 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_readahead_size: 0 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_background_flushes: -1 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Compression algorithms supported: 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: kZSTD supported: 0 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: kXpressCompression supported: 0 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: kLZ4HCCompression supported: 1 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: kZlibCompression supported: 1 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: kSnappyCompression supported: 1 2024-05-13T22:38:20.059 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: kLZ4Compression supported: 1 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: kBZip2Compression supported: 0 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi102/store.db/MANIFEST-000010 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.merge_operator: 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_filter: None 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_filter_factory: None 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.sst_partitioner_factory: None 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bef808b3e0) 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: cache_index_and_filter_blocks: 1 2024-05-13T22:38:20.060 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: pin_top_level_index_and_filter: 1 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: index_type: 0 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: data_block_index_type: 0 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: index_shortening: 1 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: checksum: 4 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: no_block_cache: 0 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_cache: 0x55bef739d1f0 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_cache_name: BinnedLRUCache 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_cache_options: 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: capacity : 536870912 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: num_shard_bits : 4 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: strict_capacity_limit : 0 2024-05-13T22:38:20.061 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: high_pri_pool_ratio: 0.000 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_cache_compressed: (nil) 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: persistent_cache: (nil) 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_size: 4096 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_size_deviation: 10 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_restart_interval: 16 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: index_block_restart_interval: 1 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: metadata_block_size: 4096 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: partition_filters: 0 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: use_delta_encoding: 1 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: filter_policy: bloomfilter 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: whole_key_filtering: 1 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: verify_compression: 0 2024-05-13T22:38:20.062 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: read_amp_bytes_per_bit: 0 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: format_version: 5 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: enable_index_compression: 1 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_align: 0 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: max_auto_readahead_size: 262144 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: prepopulate_block_cache: 0 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: initial_auto_readahead_size: 8192 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: num_file_reads_for_auto_readahead: 2 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.write_buffer_size: 33554432 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_write_buffer_number: 2 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compression: NoCompression 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bottommost_compression: Disabled 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.prefix_extractor: nullptr 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-13T22:38:20.063 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.num_levels: 7 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compression_opts.level: 32767 2024-05-13T22:38:20.064 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compression_opts.strategy: 0 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compression_opts.enabled: false 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.target_file_size_base: 67108864 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-13T22:38:20.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.arena_block_size: 1048576 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-13T22:38:20.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.disable_auto_compactions: 0 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.table_properties_collectors: 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.inplace_update_support: 0 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-13T22:38:20.067 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.bloom_locality: 0 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.max_successive_merges: 0 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.paranoid_file_checks: 0 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.force_consistency_checks: 1 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.report_bg_io_stats: 0 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.ttl: 2592000 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.enable_blob_files: false 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.min_blob_size: 0 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.blob_file_size: 268435456 2024-05-13T22:38:20.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.blob_file_starting_level: 0 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi102/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bf00ac29-5921-412f-8cd0-088117acaf84 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715639900007802, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715639900008868, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93625, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91647, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84256, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715639900, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bf00ac29-5921-412f-8cd0-088117acaf84", "db_session_id": "IXMVARTHCYZ4H18ZNAIF", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715639900008927, "job": 1, "event": "recovery_finished"} 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-13T22:38:20.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi102/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bef743a000 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: DB pointer 0x55bef7426000 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: ** DB Stats ** 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:38:20.070 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: ** Compaction Stats [default] ** 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: L0 2/0 93.30 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Sum 2/0 93.30 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: ** Compaction Stats [default] ** 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 127.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-13T22:38:20.071 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Cumulative compaction: 0.00 GB write, 29.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Interval compaction: 0.00 GB write, 29.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Block cache BinnedLRUCache@0x55bef739d1f0#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: starting mon.smithi102 rank 0 at public addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] at bind addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi102 fsid 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: mon.smithi102@-1(???) e1 preinit fsid 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:38:20.072 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: mon.smithi102@-1(???).mds e1 new map 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: mon.smithi102@-1(???).mds e1 print_map 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: e1 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: legacy client fscid: -1 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: No filesystems configured 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: mon.smithi102@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: mon.smithi102@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: mon.smithi102@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: mon.smithi102@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: mon.smithi102@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta expand map: {default=false} 2024-05-13T22:38:20.073 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta from 'false' to 'false' 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta expanded map: {default=false} 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta expand map: {default=info} 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta from 'info' to 'info' 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta expanded map: {default=info} 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta expand map: {default=daemon} 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta from 'daemon' to 'daemon' 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta expanded map: {default=daemon} 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta expand map: {default=debug} 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta from 'debug' to 'debug' 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: expand_channel_meta expanded map: {default=debug} 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 bash[28581]: 73e113d0d1b38bc6b4bcdf7197137859284183a31e8c14b26f4f658b6b07b284 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 systemd[1]: Started Ceph mon.smithi102 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: mon.smithi102 is new leader, mons smithi102 in quorum (ranks 0) 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: monmap e1: 1 mons at {smithi102=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-13T22:38:20.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: fsmap 2024-05-13T22:38:20.075 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: osdmap e1: 0 total, 0 up, 0 in 2024-05-13T22:38:20.075 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:20 smithi102 ceph-mon[28599]: mgrmap e1: no daemons active 2024-05-13T22:38:21.188 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.565+0000 7f449c978700 1 Processor -- start 2024-05-13T22:38:21.188 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.565+0000 7f449c978700 1 -- start start 2024-05-13T22:38:21.188 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.565+0000 7f449c978700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f4498109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:21.188 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.565+0000 7f449c978700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4498079e30 con 0x7f4498107040 2024-05-13T22:38:21.188 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.566+0000 7f449659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f4498109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:21.188 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.566+0000 7f449659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f4498109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48782/0 (socket says 172.21.15.102:48782) 2024-05-13T22:38:21.188 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.566+0000 7f449659c700 1 -- 172.21.15.102:0/3885967552 learned_addr learned my addr 172.21.15.102:0/3885967552 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:21.188 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.566+0000 7f449659c700 1 -- 172.21.15.102:0/3885967552 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4498109970 con 0x7f4498107040 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.566+0000 7f449659c700 1 --2- 172.21.15.102:0/3885967552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f4498109430 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44880098a0 tx=0x7f4488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba6f5592dd078dc server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.566+0000 7f4495d9b700 1 -- 172.21.15.102:0/3885967552 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4488004180 con 0x7f4498107040 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.566+0000 7f4495d9b700 1 -- 172.21.15.102:0/3885967552 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44880042e0 con 0x7f4498107040 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.566+0000 7f4495d9b700 1 -- 172.21.15.102:0/3885967552 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4488011a50 con 0x7f4498107040 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.567+0000 7f449c978700 1 -- 172.21.15.102:0/3885967552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 msgr2=0x7f4498109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.567+0000 7f449c978700 1 --2- 172.21.15.102:0/3885967552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f4498109430 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44880098a0 tx=0x7f4488009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.567+0000 7f449c978700 1 -- 172.21.15.102:0/3885967552 shutdown_connections 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.567+0000 7f449c978700 1 --2- 172.21.15.102:0/3885967552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f4498109430 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.567+0000 7f449c978700 1 -- 172.21.15.102:0/3885967552 >> 172.21.15.102:0/3885967552 conn(0x7f44981009f0 msgr2=0x7f4498102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.567+0000 7f449c978700 1 -- 172.21.15.102:0/3885967552 shutdown_connections 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.567+0000 7f449c978700 1 -- 172.21.15.102:0/3885967552 wait complete. 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.568+0000 7f449c978700 1 Processor -- start 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.568+0000 7f449c978700 1 -- start start 2024-05-13T22:38:21.189 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.568+0000 7f449c978700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f449819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.568+0000 7f449c978700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449819a6a0 con 0x7f4498107040 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.568+0000 7f449659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f449819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.568+0000 7f449659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f449819a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48784/0 (socket says 172.21.15.102:48784) 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.568+0000 7f449659c700 1 -- 172.21.15.102:0/1887997373 learned_addr learned my addr 172.21.15.102:0/1887997373 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.568+0000 7f449659c700 1 -- 172.21.15.102:0/1887997373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488009550 con 0x7f4498107040 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.569+0000 7f449659c700 1 --2- 172.21.15.102:0/1887997373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f449819a160 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4488000c00 tx=0x7f4488011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.569+0000 7f448ffff700 1 -- 172.21.15.102:0/1887997373 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4488003b90 con 0x7f4498107040 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.569+0000 7f448ffff700 1 -- 172.21.15.102:0/1887997373 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44880248b0 con 0x7f4498107040 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.569+0000 7f449c978700 1 -- 172.21.15.102:0/1887997373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449819a8a0 con 0x7f4498107040 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.569+0000 7f448ffff700 1 -- 172.21.15.102:0/1887997373 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f448801a8e0 con 0x7f4498107040 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.569+0000 7f449c978700 1 -- 172.21.15.102:0/1887997373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449819ad40 con 0x7f4498107040 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.569+0000 7f448ffff700 1 -- 172.21.15.102:0/1887997373 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f448802d3f0 con 0x7f4498107040 2024-05-13T22:38:21.190 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.569+0000 7f448ffff700 1 -- 172.21.15.102:0/1887997373 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f44880194a0 con 0x7f4498107040 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.570+0000 7f449c978700 1 -- 172.21.15.102:0/1887997373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44980623c0 con 0x7f4498107040 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.572+0000 7f448ffff700 1 -- 172.21.15.102:0/1887997373 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f4488024db0 con 0x7f4498107040 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.637+0000 7f449c978700 1 -- 172.21.15.102:0/1887997373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f449819d8d0 con 0x7f4498107040 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.640+0000 7f448ffff700 1 -- 172.21.15.102:0/1887997373 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4488024a20 con 0x7f4498107040 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.641+0000 7f449c978700 1 -- 172.21.15.102:0/1887997373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 msgr2=0x7f449819a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.641+0000 7f449c978700 1 --2- 172.21.15.102:0/1887997373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f449819a160 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4488000c00 tx=0x7f4488011ab0 comp rx=0 tx=0).stop 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.641+0000 7f449c978700 1 -- 172.21.15.102:0/1887997373 shutdown_connections 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.641+0000 7f449c978700 1 --2- 172.21.15.102:0/1887997373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4498107040 0x7f449819a160 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.641+0000 7f449c978700 1 -- 172.21.15.102:0/1887997373 >> 172.21.15.102:0/1887997373 conn(0x7f44981009f0 msgr2=0x7f4498101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.641+0000 7f449c978700 1 -- 172.21.15.102:0/1887997373 shutdown_connections 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:20.641+0000 7f449c978700 1 -- 172.21.15.102:0/1887997373 wait complete. 2024-05-13T22:38:21.191 INFO:teuthology.orchestra.run.smithi102.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-13T22:38:21.211 INFO:teuthology.orchestra.run.smithi102.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-13T22:38:21.211 INFO:teuthology.orchestra.run.smithi102.stdout:Creating mgr... 2024-05-13T22:38:21.211 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-13T22:38:21.212 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-13T22:38:21.212 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying port 0.0.0.0:8443 ... 2024-05-13T22:38:21.461 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-62275d5c-1179-11ef-bc98-c7b262605968@mgr.smithi102.xcijrj 2024-05-13T22:38:21.462 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to reset failed state of unit ceph-62275d5c-1179-11ef-bc98-c7b262605968@mgr.smithi102.xcijrj.service: Unit ceph-62275d5c-1179-11ef-bc98-c7b262605968@mgr.smithi102.xcijrj.service not loaded. 2024-05-13T22:38:21.607 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-62275d5c-1179-11ef-bc98-c7b262605968.target.wants/ceph-62275d5c-1179-11ef-bc98-c7b262605968@mgr.smithi102.xcijrj.service → /etc/systemd/system/ceph-62275d5c-1179-11ef-bc98-c7b262605968@.service. 2024-05-13T22:38:21.839 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:21 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1887997373' entity='client.admin' 2024-05-13T22:38:22.438 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-13T22:38:22.438 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-05-13T22:38:22.445 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-13T22:38:22.446 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-05-13T22:38:22.446 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-05-13T22:38:22.446 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to enable service . firewalld.service is not available 2024-05-13T22:38:22.454 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-13T22:38:22.454 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-05-13T22:38:22.461 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-13T22:38:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-05-13T22:38:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-05-13T22:38:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-05-13T22:38:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr to start... 2024-05-13T22:38:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr... 2024-05-13T22:38:23.315 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:23 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2939105057' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-13T22:38:23.578 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:23.578 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-05-13T22:38:23.578 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsid": "62275d5c-1179-11ef-bc98-c7b262605968", 2024-05-13T22:38:23.578 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "health": { 2024-05-13T22:38:23.578 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-13T22:38:23.578 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 0 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "smithi102" 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-13T22:38:23.579 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-13T22:38:23.580 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-13T22:38:23.581 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "restful" 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modified": "2024-05-13T22:38:14.141378+0000", 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-13T22:38:23.582 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.923+0000 7f4b67d79700 1 Processor -- start 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.923+0000 7f4b67d79700 1 -- start start 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.923+0000 7f4b67d79700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.923+0000 7f4b67d79700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6007a890 con 0x7f4b6007be60 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.924+0000 7f4b65b15700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.924+0000 7f4b65b15700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48796/0 (socket says 172.21.15.102:48796) 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.924+0000 7f4b65b15700 1 -- 172.21.15.102:0/1190356798 learned_addr learned my addr 172.21.15.102:0/1190356798 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.924+0000 7f4b65b15700 1 -- 172.21.15.102:0/1190356798 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6007a9d0 con 0x7f4b6007be60 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.924+0000 7f4b65b15700 1 --2- 172.21.15.102:0/1190356798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6007a2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0098a0 tx=0x7f4b5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06db59a43a7a263 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.924+0000 7f4b64b13700 1 -- 172.21.15.102:0/1190356798 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b5c004180 con 0x7f4b6007be60 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.925+0000 7f4b64b13700 1 -- 172.21.15.102:0/1190356798 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b5c0042e0 con 0x7f4b6007be60 2024-05-13T22:38:23.583 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.925+0000 7f4b64b13700 1 -- 172.21.15.102:0/1190356798 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b5c011a50 con 0x7f4b6007be60 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.925+0000 7f4b67d79700 1 -- 172.21.15.102:0/1190356798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 msgr2=0x7f4b6007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.925+0000 7f4b67d79700 1 --2- 172.21.15.102:0/1190356798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6007a2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0098a0 tx=0x7f4b5c009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.925+0000 7f4b67d79700 1 -- 172.21.15.102:0/1190356798 shutdown_connections 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.925+0000 7f4b67d79700 1 --2- 172.21.15.102:0/1190356798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6007a2c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.925+0000 7f4b67d79700 1 -- 172.21.15.102:0/1190356798 >> 172.21.15.102:0/1190356798 conn(0x7f4b60101ad0 msgr2=0x7f4b60103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.925+0000 7f4b67d79700 1 -- 172.21.15.102:0/1190356798 shutdown_connections 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.925+0000 7f4b67d79700 1 -- 172.21.15.102:0/1190356798 wait complete. 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.926+0000 7f4b67d79700 1 Processor -- start 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.926+0000 7f4b67d79700 1 -- start start 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.926+0000 7f4b67d79700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.926+0000 7f4b67d79700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6019c8e0 con 0x7f4b6007be60 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.926+0000 7f4b65b15700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:23.584 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.926+0000 7f4b65b15700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6019c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48808/0 (socket says 172.21.15.102:48808) 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.926+0000 7f4b65b15700 1 -- 172.21.15.102:0/2939105057 learned_addr learned my addr 172.21.15.102:0/2939105057 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.926+0000 7f4b65b15700 1 -- 172.21.15.102:0/2939105057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c009550 con 0x7f4b6007be60 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.927+0000 7f4b65b15700 1 --2- 172.21.15.102:0/2939105057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6019c3a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c003f80 tx=0x7f4b5c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.927+0000 7f4b56ffd700 1 -- 172.21.15.102:0/2939105057 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b5c003a80 con 0x7f4b6007be60 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.927+0000 7f4b56ffd700 1 -- 172.21.15.102:0/2939105057 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4b5c003be0 con 0x7f4b6007be60 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.927+0000 7f4b67d79700 1 -- 172.21.15.102:0/2939105057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6019cae0 con 0x7f4b6007be60 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.927+0000 7f4b56ffd700 1 -- 172.21.15.102:0/2939105057 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b5c019820 con 0x7f4b6007be60 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.927+0000 7f4b67d79700 1 -- 172.21.15.102:0/2939105057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6019cf80 con 0x7f4b6007be60 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.927+0000 7f4b56ffd700 1 -- 172.21.15.102:0/2939105057 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4b5c02d440 con 0x7f4b6007be60 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.927+0000 7f4b56ffd700 1 -- 172.21.15.102:0/2939105057 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4b5c0247c0 con 0x7f4b6007be60 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.928+0000 7f4b67d79700 1 -- 172.21.15.102:0/2939105057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b600623c0 con 0x7f4b6007be60 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:22.930+0000 7f4b56ffd700 1 -- 172.21.15.102:0/2939105057 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f4b5c01acc0 con 0x7f4b6007be60 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:23.001+0000 7f4b67d79700 1 -- 172.21.15.102:0/2939105057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4b6019f940 con 0x7f4b6007be60 2024-05-13T22:38:23.585 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:23.001+0000 7f4b56ffd700 1 -- 172.21.15.102:0/2939105057 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f4b5c035090 con 0x7f4b6007be60 2024-05-13T22:38:23.586 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:23.003+0000 7f4b67d79700 1 -- 172.21.15.102:0/2939105057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 msgr2=0x7f4b6019c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:23.586 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:23.003+0000 7f4b67d79700 1 --2- 172.21.15.102:0/2939105057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6019c3a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c003f80 tx=0x7f4b5c011ab0 comp rx=0 tx=0).stop 2024-05-13T22:38:23.586 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:23.003+0000 7f4b67d79700 1 -- 172.21.15.102:0/2939105057 shutdown_connections 2024-05-13T22:38:23.586 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:23.003+0000 7f4b67d79700 1 --2- 172.21.15.102:0/2939105057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b6007be60 0x7f4b6019c3a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:23.586 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:23.003+0000 7f4b67d79700 1 -- 172.21.15.102:0/2939105057 >> 172.21.15.102:0/2939105057 conn(0x7f4b60101ad0 msgr2=0x7f4b60103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:23.586 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:23.003+0000 7f4b67d79700 1 -- 172.21.15.102:0/2939105057 shutdown_connections 2024-05-13T22:38:23.586 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:23.003+0000 7f4b67d79700 1 -- 172.21.15.102:0/2939105057 wait complete. 2024-05-13T22:38:23.586 INFO:teuthology.orchestra.run.smithi102.stdout:mgr not available, waiting (1/15)... 2024-05-13T22:38:26.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:26 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2317451674' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-13T22:38:26.720 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsid": "62275d5c-1179-11ef-bc98-c7b262605968", 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "health": { 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 0 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "smithi102" 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:26.721 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-05-13T22:38:26.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-13T22:38:26.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:26.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-13T22:38:26.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-13T22:38:26.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:26.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-13T22:38:26.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:26.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-13T22:38:26.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-13T22:38:26.722 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-13T22:38:26.731 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "restful" 2024-05-13T22:38:26.732 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modified": "2024-05-13T22:38:14.141378+0000", 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.988+0000 7fd11b3b0700 1 Processor -- start 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.988+0000 7fd11b3b0700 1 -- start start 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.988+0000 7fd11b3b0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114106e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.988+0000 7fd11b3b0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd114107440 con 0x7fd114106a50 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.988+0000 7fd11914c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114106e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:26.733 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.988+0000 7fd11914c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114106e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48822/0 (socket says 172.21.15.102:48822) 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.988+0000 7fd11914c700 1 -- 172.21.15.102:0/1106980105 learned_addr learned my addr 172.21.15.102:0/1106980105 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.989+0000 7fd11914c700 1 -- 172.21.15.102:0/1106980105 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd114105660 con 0x7fd114106a50 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.989+0000 7fd11914c700 1 --2- 172.21.15.102:0/1106980105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114106e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd11000bb40 tx=0x7fd11000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3028b844c21147a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.989+0000 7fd10bfff700 1 -- 172.21.15.102:0/1106980105 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd11000dc00 con 0x7fd114106a50 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.989+0000 7fd10bfff700 1 -- 172.21.15.102:0/1106980105 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1100044a0 con 0x7fd114106a50 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.989+0000 7fd10bfff700 1 -- 172.21.15.102:0/1106980105 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd110010470 con 0x7fd114106a50 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.990+0000 7fd11b3b0700 1 -- 172.21.15.102:0/1106980105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 msgr2=0x7fd114106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.990+0000 7fd11b3b0700 1 --2- 172.21.15.102:0/1106980105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114106e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd11000bb40 tx=0x7fd11000be50 comp rx=0 tx=0).stop 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.990+0000 7fd11b3b0700 1 -- 172.21.15.102:0/1106980105 shutdown_connections 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.990+0000 7fd11b3b0700 1 --2- 172.21.15.102:0/1106980105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114106e70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.990+0000 7fd11b3b0700 1 -- 172.21.15.102:0/1106980105 >> 172.21.15.102:0/1106980105 conn(0x7fd114100190 msgr2=0x7fd1141025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:26.734 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.990+0000 7fd11b3b0700 1 -- 172.21.15.102:0/1106980105 shutdown_connections 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.990+0000 7fd11b3b0700 1 -- 172.21.15.102:0/1106980105 wait complete. 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.991+0000 7fd11b3b0700 1 Processor -- start 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.991+0000 7fd11b3b0700 1 -- start start 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.991+0000 7fd11b3b0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.991+0000 7fd11b3b0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd114198500 con 0x7fd114106a50 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.991+0000 7fd11914c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.991+0000 7fd11914c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48830/0 (socket says 172.21.15.102:48830) 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.991+0000 7fd11914c700 1 -- 172.21.15.102:0/2317451674 learned_addr learned my addr 172.21.15.102:0/2317451674 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.992+0000 7fd11914c700 1 -- 172.21.15.102:0/2317451674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11000b7f0 con 0x7fd114106a50 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.992+0000 7fd11914c700 1 --2- 172.21.15.102:0/2317451674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114197fc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd11000b340 tx=0x7fd110003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.992+0000 7fd10a7fc700 1 -- 172.21.15.102:0/2317451674 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd11001e070 con 0x7fd114106a50 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.992+0000 7fd10a7fc700 1 -- 172.21.15.102:0/2317451674 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd110004260 con 0x7fd114106a50 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.992+0000 7fd11b3b0700 1 -- 172.21.15.102:0/2317451674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd114198700 con 0x7fd114106a50 2024-05-13T22:38:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.992+0000 7fd10a7fc700 1 -- 172.21.15.102:0/2317451674 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1100228b0 con 0x7fd114106a50 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.992+0000 7fd11b3b0700 1 -- 172.21.15.102:0/2317451674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd114198ba0 con 0x7fd114106a50 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.992+0000 7fd10a7fc700 1 -- 172.21.15.102:0/2317451674 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd11002c440 con 0x7fd114106a50 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.992+0000 7fd10a7fc700 1 -- 172.21.15.102:0/2317451674 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd110019570 con 0x7fd114106a50 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.993+0000 7fd11b3b0700 1 -- 172.21.15.102:0/2317451674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd114191d00 con 0x7fd114106a50 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:25.995+0000 7fd10a7fc700 1 -- 172.21.15.102:0/2317451674 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fd110027070 con 0x7fd114106a50 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:26.067+0000 7fd11b3b0700 1 -- 172.21.15.102:0/2317451674 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd1140623c0 con 0x7fd114106a50 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:26.068+0000 7fd10a7fc700 1 -- 172.21.15.102:0/2317451674 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fd1140623c0 con 0x7fd114106a50 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:26.068+0000 7fd11b3b0700 1 -- 172.21.15.102:0/2317451674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 msgr2=0x7fd114197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:26.068+0000 7fd11b3b0700 1 --2- 172.21.15.102:0/2317451674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114197fc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd11000b340 tx=0x7fd110003d70 comp rx=0 tx=0).stop 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:26.068+0000 7fd11b3b0700 1 -- 172.21.15.102:0/2317451674 shutdown_connections 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:26.068+0000 7fd11b3b0700 1 --2- 172.21.15.102:0/2317451674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd114106a50 0x7fd114197fc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:26.068+0000 7fd11b3b0700 1 -- 172.21.15.102:0/2317451674 >> 172.21.15.102:0/2317451674 conn(0x7fd114100190 msgr2=0x7fd11418ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:26.069+0000 7fd11b3b0700 1 -- 172.21.15.102:0/2317451674 shutdown_connections 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:26.069+0000 7fd11b3b0700 1 -- 172.21.15.102:0/2317451674 wait complete. 2024-05-13T22:38:26.736 INFO:teuthology.orchestra.run.smithi102.stdout:mgr not available, waiting (2/15)... 2024-05-13T22:38:29.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: Activating manager daemon smithi102.xcijrj 2024-05-13T22:38:29.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: mgrmap e2: smithi102.xcijrj(active, starting, since 0.00292326s) 2024-05-13T22:38:29.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: from='mgr.14100 172.21.15.102:0/1228402220' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-13T22:38:29.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: from='mgr.14100 172.21.15.102:0/1228402220' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-13T22:38:29.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: from='mgr.14100 172.21.15.102:0/1228402220' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-13T22:38:29.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: from='mgr.14100 172.21.15.102:0/1228402220' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:38:29.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: from='mgr.14100 172.21.15.102:0/1228402220' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi102.xcijrj", "id": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:38:29.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: Manager daemon smithi102.xcijrj is now available 2024-05-13T22:38:29.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: from='mgr.14100 172.21.15.102:0/1228402220' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:38:29.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: from='mgr.14100 172.21.15.102:0/1228402220' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/trash_purge_schedule"}]: dispatch 2024-05-13T22:38:29.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: from='mgr.14100 172.21.15.102:0/1228402220' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:29.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: from='mgr.14100 172.21.15.102:0/1228402220' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:29.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:29 smithi102 ceph-mon[28599]: from='mgr.14100 172.21.15.102:0/1228402220' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:29.810 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsid": "62275d5c-1179-11ef-bc98-c7b262605968", 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "health": { 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 0 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "smithi102" 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:29.811 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-13T22:38:29.812 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-13T22:38:29.812 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:29.812 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-13T22:38:29.812 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-13T22:38:29.812 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:29.812 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-13T22:38:29.812 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:29.812 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-13T22:38:29.812 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-13T22:38:29.812 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-13T22:38:29.820 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-13T22:38:29.820 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-13T22:38:29.820 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-13T22:38:29.820 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:29.820 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-13T22:38:29.820 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-13T22:38:29.820 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-13T22:38:29.820 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-13T22:38:29.820 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-13T22:38:29.821 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "restful" 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modified": "2024-05-13T22:38:14.141378+0000", 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-05-13T22:38:29.822 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.158+0000 7f2ea1e7b700 1 Processor -- start 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.158+0000 7f2ea1e7b700 1 -- start start 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.159+0000 7f2ea1e7b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c106960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.159+0000 7f2ea1e7b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c106f30 con 0x7f2e9c106540 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.159+0000 7f2e9b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c106960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.159+0000 7f2e9b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c106960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57250/0 (socket says 172.21.15.102:57250) 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.159+0000 7f2e9b7fe700 1 -- 172.21.15.102:0/1484274041 learned_addr learned my addr 172.21.15.102:0/1484274041 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.159+0000 7f2e9b7fe700 1 -- 172.21.15.102:0/1484274041 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c107740 con 0x7f2e9c106540 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.159+0000 7f2e9b7fe700 1 --2- 172.21.15.102:0/1484274041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c106960 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0098a0 tx=0x7f2e8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebcabd44e79c6e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.159+0000 7f2e9affd700 1 -- 172.21.15.102:0/1484274041 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e8c004180 con 0x7f2e9c106540 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.159+0000 7f2e9affd700 1 -- 172.21.15.102:0/1484274041 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e8c0042e0 con 0x7f2e9c106540 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.159+0000 7f2e9affd700 1 -- 172.21.15.102:0/1484274041 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e8c011a50 con 0x7f2e9c106540 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.160+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/1484274041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 msgr2=0x7f2e9c106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.160+0000 7f2ea1e7b700 1 --2- 172.21.15.102:0/1484274041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c106960 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0098a0 tx=0x7f2e8c009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.160+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/1484274041 shutdown_connections 2024-05-13T22:38:29.823 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.160+0000 7f2ea1e7b700 1 --2- 172.21.15.102:0/1484274041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c106960 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.160+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/1484274041 >> 172.21.15.102:0/1484274041 conn(0x7f2e9c101b40 msgr2=0x7f2e9c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.160+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/1484274041 shutdown_connections 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.160+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/1484274041 wait complete. 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.161+0000 7f2ea1e7b700 1 Processor -- start 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.161+0000 7f2ea1e7b700 1 -- start start 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.161+0000 7f2ea1e7b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.161+0000 7f2ea1e7b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c19a770 con 0x7f2e9c106540 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.161+0000 7f2e9b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.161+0000 7f2e9b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57254/0 (socket says 172.21.15.102:57254) 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.161+0000 7f2e9b7fe700 1 -- 172.21.15.102:0/3669433222 learned_addr learned my addr 172.21.15.102:0/3669433222 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.161+0000 7f2e9b7fe700 1 -- 172.21.15.102:0/3669433222 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c009550 con 0x7f2e9c106540 2024-05-13T22:38:29.824 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.161+0000 7f2e9b7fe700 1 --2- 172.21.15.102:0/3669433222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c19a230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009520 tx=0x7f2e8c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.162+0000 7f2e997fa700 1 -- 172.21.15.102:0/3669433222 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e8c003a80 con 0x7f2e9c106540 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.162+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/3669433222 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c19a970 con 0x7f2e9c106540 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.162+0000 7f2e997fa700 1 -- 172.21.15.102:0/3669433222 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2e8c003be0 con 0x7f2e9c106540 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.162+0000 7f2e997fa700 1 -- 172.21.15.102:0/3669433222 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e8c0197b0 con 0x7f2e9c106540 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.162+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/3669433222 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c19ae10 con 0x7f2e9c106540 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.162+0000 7f2e997fa700 1 -- 172.21.15.102:0/3669433222 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 2) v1 ==== 45039+0+0 (secure 0 0 0) 0x7f2e8c019a70 con 0x7f2e9c106540 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.163+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/3669433222 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c0623c0 con 0x7f2e9c106540 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.163+0000 7f2e997fa700 1 -- 172.21.15.102:0/3669433222 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2e8c04c0e0 con 0x7f2e9c106540 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.165+0000 7f2e997fa700 1 -- 172.21.15.102:0/3669433222 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f2e8c019d20 con 0x7f2e9c106540 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.233+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/3669433222 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2e9c10a970 con 0x7f2e9c106540 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.234+0000 7f2e997fa700 1 -- 172.21.15.102:0/3669433222 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f2e8c0172b0 con 0x7f2e9c106540 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.234+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/3669433222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 msgr2=0x7f2e9c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.234+0000 7f2ea1e7b700 1 --2- 172.21.15.102:0/3669433222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c19a230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009520 tx=0x7f2e8c011ab0 comp rx=0 tx=0).stop 2024-05-13T22:38:29.825 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.234+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/3669433222 shutdown_connections 2024-05-13T22:38:29.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.234+0000 7f2ea1e7b700 1 --2- 172.21.15.102:0/3669433222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e9c106540 0x7f2e9c19a230 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:29.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.234+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/3669433222 >> 172.21.15.102:0/3669433222 conn(0x7f2e9c101b40 msgr2=0x7f2e9c1031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:29.826 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.235+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/3669433222 shutdown_connections 2024-05-13T22:38:29.827 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:29.235+0000 7f2ea1e7b700 1 -- 172.21.15.102:0/3669433222 wait complete. 2024-05-13T22:38:29.827 INFO:teuthology.orchestra.run.smithi102.stdout:mgr not available, waiting (3/15)... 2024-05-13T22:38:30.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:30 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3669433222' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-13T22:38:30.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:30 smithi102 ceph-mon[28599]: mgrmap e3: smithi102.xcijrj(active, since 1.00694s) 2024-05-13T22:38:32.208 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:32 smithi102 ceph-mon[28599]: mgrmap e4: smithi102.xcijrj(active, since 2s) 2024-05-13T22:38:33.092 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:33.092 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-05-13T22:38:33.092 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsid": "62275d5c-1179-11ef-bc98-c7b262605968", 2024-05-13T22:38:33.092 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "health": { 2024-05-13T22:38:33.092 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-13T22:38:33.092 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-13T22:38:33.092 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-13T22:38:33.092 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:33.092 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-13T22:38:33.092 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 0 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "smithi102" 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-13T22:38:33.093 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-13T22:38:33.094 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-13T22:38:33.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-13T22:38:33.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-13T22:38:33.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-13T22:38:33.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:33.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-13T22:38:33.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-13T22:38:33.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-13T22:38:33.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-13T22:38:33.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-13T22:38:33.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-13T22:38:33.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "restful" 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modified": "2024-05-13T22:38:14.141378+0000", 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.254+0000 7f6289885700 1 Processor -- start 2024-05-13T22:38:33.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.254+0000 7f6289885700 1 -- start start 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.254+0000 7f6289885700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f6284104c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.254+0000 7f6289885700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284105250 con 0x7f6284104860 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.254+0000 7f6282ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f6284104c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.254+0000 7f6282ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f6284104c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57258/0 (socket says 172.21.15.102:57258) 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.254+0000 7f6282ffd700 1 -- 172.21.15.102:0/2336491248 learned_addr learned my addr 172.21.15.102:0/2336491248 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.255+0000 7f6282ffd700 1 -- 172.21.15.102:0/2336491248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62841077f0 con 0x7f6284104860 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.255+0000 7f6282ffd700 1 --2- 172.21.15.102:0/2336491248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f6284104c80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62740098a0 tx=0x7f6274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fa9ecdf1b8b96c server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.255+0000 7f6281ffb700 1 -- 172.21.15.102:0/2336491248 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6274004180 con 0x7f6284104860 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.255+0000 7f6281ffb700 1 -- 172.21.15.102:0/2336491248 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f62740042e0 con 0x7f6284104860 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.255+0000 7f6281ffb700 1 -- 172.21.15.102:0/2336491248 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6274011a50 con 0x7f6284104860 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.256+0000 7f6289885700 1 -- 172.21.15.102:0/2336491248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 msgr2=0x7f6284104c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.256+0000 7f6289885700 1 --2- 172.21.15.102:0/2336491248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f6284104c80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62740098a0 tx=0x7f6274009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.256+0000 7f6289885700 1 -- 172.21.15.102:0/2336491248 shutdown_connections 2024-05-13T22:38:33.105 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.256+0000 7f6289885700 1 --2- 172.21.15.102:0/2336491248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f6284104c80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.256+0000 7f6289885700 1 -- 172.21.15.102:0/2336491248 >> 172.21.15.102:0/2336491248 conn(0x7f6284100190 msgr2=0x7f62841025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.256+0000 7f6289885700 1 -- 172.21.15.102:0/2336491248 shutdown_connections 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.256+0000 7f6289885700 1 -- 172.21.15.102:0/2336491248 wait complete. 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.256+0000 7f6289885700 1 Processor -- start 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6289885700 1 -- start start 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6289885700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f628419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6289885700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628419a720 con 0x7f6284104860 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6282ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f628419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6282ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f628419a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57266/0 (socket says 172.21.15.102:57266) 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6282ffd700 1 -- 172.21.15.102:0/3687837929 learned_addr learned my addr 172.21.15.102:0/3687837929 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6282ffd700 1 -- 172.21.15.102:0/3687837929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274009550 con 0x7f6284104860 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6282ffd700 1 --2- 172.21.15.102:0/3687837929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f628419a1e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6274000c00 tx=0x7f6274011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6288883700 1 -- 172.21.15.102:0/3687837929 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6274003b90 con 0x7f6284104860 2024-05-13T22:38:33.106 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6289885700 1 -- 172.21.15.102:0/3687837929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628419a920 con 0x7f6284104860 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6288883700 1 -- 172.21.15.102:0/3687837929 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f62740248b0 con 0x7f6284104860 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6288883700 1 -- 172.21.15.102:0/3687837929 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f627401a8e0 con 0x7f6284104860 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.257+0000 7f6289885700 1 -- 172.21.15.102:0/3687837929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628419adc0 con 0x7f6284104860 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.258+0000 7f6288883700 1 -- 172.21.15.102:0/3687837929 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f627401aba0 con 0x7f6284104860 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.258+0000 7f6289885700 1 -- 172.21.15.102:0/3687837929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62840623c0 con 0x7f6284104860 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.259+0000 7f6288883700 1 --2- 172.21.15.102:0/3687837929 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f626c0382d0 0x7f626c03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.259+0000 7f6288883700 1 -- 172.21.15.102:0/3687837929 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f627404c230 con 0x7f6284104860 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.259+0000 7f62827fc700 1 --2- 172.21.15.102:0/3687837929 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f626c0382d0 0x7f626c03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.259+0000 7f62827fc700 1 --2- 172.21.15.102:0/3687837929 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f626c0382d0 0x7f626c03a790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6278006fd0 tx=0x7f6278006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.264+0000 7f6288883700 1 -- 172.21.15.102:0/3687837929 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6274023790 con 0x7f6284104860 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.538+0000 7f6289885700 1 -- 172.21.15.102:0/3687837929 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f628410aa20 con 0x7f6284104860 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.538+0000 7f6288883700 1 -- 172.21.15.102:0/3687837929 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f62740233a0 con 0x7f6284104860 2024-05-13T22:38:33.107 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.541+0000 7f6289885700 1 -- 172.21.15.102:0/3687837929 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f626c0382d0 msgr2=0x7f626c03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:33.108 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.541+0000 7f6289885700 1 --2- 172.21.15.102:0/3687837929 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f626c0382d0 0x7f626c03a790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6278006fd0 tx=0x7f6278006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:33.108 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.541+0000 7f6289885700 1 -- 172.21.15.102:0/3687837929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 msgr2=0x7f628419a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:33.108 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.541+0000 7f6289885700 1 --2- 172.21.15.102:0/3687837929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f628419a1e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6274000c00 tx=0x7f6274011ab0 comp rx=0 tx=0).stop 2024-05-13T22:38:33.108 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.541+0000 7f6289885700 1 -- 172.21.15.102:0/3687837929 shutdown_connections 2024-05-13T22:38:33.108 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.541+0000 7f6289885700 1 --2- 172.21.15.102:0/3687837929 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f626c0382d0 0x7f626c03a790 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:33.108 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.541+0000 7f6289885700 1 --2- 172.21.15.102:0/3687837929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6284104860 0x7f628419a1e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:33.108 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.541+0000 7f6289885700 1 -- 172.21.15.102:0/3687837929 >> 172.21.15.102:0/3687837929 conn(0x7f6284100190 msgr2=0x7f6284100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:33.108 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.542+0000 7f6289885700 1 -- 172.21.15.102:0/3687837929 shutdown_connections 2024-05-13T22:38:33.108 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:32.542+0000 7f6289885700 1 -- 172.21.15.102:0/3687837929 wait complete. 2024-05-13T22:38:33.108 INFO:teuthology.orchestra.run.smithi102.stdout:mgr is available 2024-05-13T22:38:33.217 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:33 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3687837929' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-13T22:38:34.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:34.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [global] 2024-05-13T22:38:34.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout fsid = 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:38:34.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [osd] 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.527+0000 7fda36455700 1 Processor -- start 2024-05-13T22:38:34.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.527+0000 7fda36455700 1 -- start start 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.527+0000 7fda36455700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda30104c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.527+0000 7fda36455700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda30105230 con 0x7fda30104840 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.528+0000 7fda2ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda30104c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.528+0000 7fda2ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda30104c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57278/0 (socket says 172.21.15.102:57278) 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.528+0000 7fda2ffff700 1 -- 172.21.15.102:0/824035640 learned_addr learned my addr 172.21.15.102:0/824035640 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.528+0000 7fda2ffff700 1 -- 172.21.15.102:0/824035640 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda301077d0 con 0x7fda30104840 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.528+0000 7fda2ffff700 1 --2- 172.21.15.102:0/824035640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda30104c60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda200098a0 tx=0x7fda20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214d98f3d5b37658 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.528+0000 7fda2f7fe700 1 -- 172.21.15.102:0/824035640 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda20004180 con 0x7fda30104840 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.528+0000 7fda2f7fe700 1 -- 172.21.15.102:0/824035640 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fda200042e0 con 0x7fda30104840 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.528+0000 7fda2f7fe700 1 -- 172.21.15.102:0/824035640 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda20011a50 con 0x7fda30104840 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.529+0000 7fda36455700 1 -- 172.21.15.102:0/824035640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 msgr2=0x7fda30104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.529+0000 7fda36455700 1 --2- 172.21.15.102:0/824035640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda30104c60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda200098a0 tx=0x7fda20009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.529+0000 7fda36455700 1 -- 172.21.15.102:0/824035640 shutdown_connections 2024-05-13T22:38:34.306 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.529+0000 7fda36455700 1 --2- 172.21.15.102:0/824035640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda30104c60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.529+0000 7fda36455700 1 -- 172.21.15.102:0/824035640 >> 172.21.15.102:0/824035640 conn(0x7fda30100190 msgr2=0x7fda301025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.529+0000 7fda36455700 1 -- 172.21.15.102:0/824035640 shutdown_connections 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.529+0000 7fda36455700 1 -- 172.21.15.102:0/824035640 wait complete. 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda36455700 1 Processor -- start 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda36455700 1 -- start start 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda36455700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda3019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda36455700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3019c950 con 0x7fda30104840 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda2ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda3019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda2ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda3019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57282/0 (socket says 172.21.15.102:57282) 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda2ffff700 1 -- 172.21.15.102:0/4121453998 learned_addr learned my addr 172.21.15.102:0/4121453998 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda2ffff700 1 -- 172.21.15.102:0/4121453998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda20009550 con 0x7fda30104840 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda2ffff700 1 --2- 172.21.15.102:0/4121453998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda3019c410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fda20009520 tx=0x7fda20011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:34.307 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda2dffb700 1 -- 172.21.15.102:0/4121453998 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda20003a80 con 0x7fda30104840 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda2dffb700 1 -- 172.21.15.102:0/4121453998 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fda20003be0 con 0x7fda30104840 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda36455700 1 -- 172.21.15.102:0/4121453998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3019cb50 con 0x7fda30104840 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.530+0000 7fda2dffb700 1 -- 172.21.15.102:0/4121453998 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fda200197b0 con 0x7fda30104840 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.531+0000 7fda36455700 1 -- 172.21.15.102:0/4121453998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3019cff0 con 0x7fda30104840 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.531+0000 7fda2dffb700 1 -- 172.21.15.102:0/4121453998 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fda20019a20 con 0x7fda30104840 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.531+0000 7fda2dffb700 1 --2- 172.21.15.102:0/4121453998 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7fda1c0382d0 0x7fda1c03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.531+0000 7fda36455700 1 -- 172.21.15.102:0/4121453998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda300623c0 con 0x7fda30104840 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.532+0000 7fda2dffb700 1 -- 172.21.15.102:0/4121453998 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fda2004c4a0 con 0x7fda30104840 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.532+0000 7fda27fff700 1 --2- 172.21.15.102:0/4121453998 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7fda1c0382d0 0x7fda1c03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.532+0000 7fda27fff700 1 --2- 172.21.15.102:0/4121453998 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7fda1c0382d0 0x7fda1c03a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fda18006fd0 tx=0x7fda18006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.536+0000 7fda2dffb700 1 -- 172.21.15.102:0/4121453998 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fda2002c3a0 con 0x7fda30104840 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.729+0000 7fda36455700 1 -- 172.21.15.102:0/4121453998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fda3019fef0 con 0x7fda30104840 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.730+0000 7fda2dffb700 1 -- 172.21.15.102:0/4121453998 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fda200172b0 con 0x7fda30104840 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.732+0000 7fda36455700 1 -- 172.21.15.102:0/4121453998 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7fda1c0382d0 msgr2=0x7fda1c03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:34.308 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.732+0000 7fda36455700 1 --2- 172.21.15.102:0/4121453998 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7fda1c0382d0 0x7fda1c03a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fda18006fd0 tx=0x7fda18006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:34.309 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.732+0000 7fda36455700 1 -- 172.21.15.102:0/4121453998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 msgr2=0x7fda3019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:34.309 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.732+0000 7fda36455700 1 --2- 172.21.15.102:0/4121453998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda3019c410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fda20009520 tx=0x7fda20011ab0 comp rx=0 tx=0).stop 2024-05-13T22:38:34.309 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.732+0000 7fda36455700 1 -- 172.21.15.102:0/4121453998 shutdown_connections 2024-05-13T22:38:34.309 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.732+0000 7fda36455700 1 --2- 172.21.15.102:0/4121453998 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7fda1c0382d0 0x7fda1c03a790 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:34.309 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.732+0000 7fda36455700 1 --2- 172.21.15.102:0/4121453998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda30104840 0x7fda3019c410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:34.309 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.732+0000 7fda36455700 1 -- 172.21.15.102:0/4121453998 >> 172.21.15.102:0/4121453998 conn(0x7fda30100190 msgr2=0x7fda30101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:34.309 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.732+0000 7fda36455700 1 -- 172.21.15.102:0/4121453998 shutdown_connections 2024-05-13T22:38:34.309 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:33.732+0000 7fda36455700 1 -- 172.21.15.102:0/4121453998 wait complete. 2024-05-13T22:38:34.309 INFO:teuthology.orchestra.run.smithi102.stdout:Enabling cephadm module... 2024-05-13T22:38:34.423 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:34 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/4121453998' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-13T22:38:35.358 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:35 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/168385148' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-13T22:38:35.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.719+0000 7f5627567700 1 Processor -- start 2024-05-13T22:38:35.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.719+0000 7f5627567700 1 -- start start 2024-05-13T22:38:35.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.719+0000 7f5627567700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f5620106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:35.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.719+0000 7f5627567700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5620106f50 con 0x7f5620106560 2024-05-13T22:38:35.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.719+0000 7f5625303700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f5620106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.720+0000 7f5625303700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f5620106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57298/0 (socket says 172.21.15.102:57298) 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.720+0000 7f5625303700 1 -- 172.21.15.102:0/703662158 learned_addr learned my addr 172.21.15.102:0/703662158 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.720+0000 7f5625303700 1 -- 172.21.15.102:0/703662158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5620107760 con 0x7f5620106560 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.720+0000 7f5625303700 1 --2- 172.21.15.102:0/703662158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f5620106980 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f561c0098a0 tx=0x7f561c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2decddd17cd126 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.720+0000 7f5617fff700 1 -- 172.21.15.102:0/703662158 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f561c004180 con 0x7f5620106560 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.720+0000 7f5617fff700 1 -- 172.21.15.102:0/703662158 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f561c0042e0 con 0x7f5620106560 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.720+0000 7f5617fff700 1 -- 172.21.15.102:0/703662158 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f561c011a50 con 0x7f5620106560 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.721+0000 7f5627567700 1 -- 172.21.15.102:0/703662158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 msgr2=0x7f5620106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.721+0000 7f5627567700 1 --2- 172.21.15.102:0/703662158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f5620106980 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f561c0098a0 tx=0x7f561c009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.721+0000 7f5627567700 1 -- 172.21.15.102:0/703662158 shutdown_connections 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.721+0000 7f5627567700 1 --2- 172.21.15.102:0/703662158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f5620106980 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.721+0000 7f5627567700 1 -- 172.21.15.102:0/703662158 >> 172.21.15.102:0/703662158 conn(0x7f5620101b40 msgr2=0x7f5620103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.721+0000 7f5627567700 1 -- 172.21.15.102:0/703662158 shutdown_connections 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.721+0000 7f5627567700 1 -- 172.21.15.102:0/703662158 wait complete. 2024-05-13T22:38:35.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.721+0000 7f5627567700 1 Processor -- start 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.721+0000 7f5627567700 1 -- start start 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f5627567700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f562019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f5627567700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562019a730 con 0x7f5620106560 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f5625303700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f562019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f5625303700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f562019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57308/0 (socket says 172.21.15.102:57308) 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f5625303700 1 -- 172.21.15.102:0/168385148 learned_addr learned my addr 172.21.15.102:0/168385148 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f5625303700 1 -- 172.21.15.102:0/168385148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c009550 con 0x7f5620106560 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f5625303700 1 --2- 172.21.15.102:0/168385148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f562019a1f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f561c009520 tx=0x7f561c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f56167fc700 1 -- 172.21.15.102:0/168385148 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f561c003a80 con 0x7f5620106560 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f56167fc700 1 -- 172.21.15.102:0/168385148 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f561c003be0 con 0x7f5620106560 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f5627567700 1 -- 172.21.15.102:0/168385148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562019a930 con 0x7f5620106560 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f56167fc700 1 -- 172.21.15.102:0/168385148 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f561c0197b0 con 0x7f5620106560 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.722+0000 7f5627567700 1 -- 172.21.15.102:0/168385148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562019add0 con 0x7f5620106560 2024-05-13T22:38:35.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.723+0000 7f56167fc700 1 -- 172.21.15.102:0/168385148 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f561c019a20 con 0x7f5620106560 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.723+0000 7f5627567700 1 -- 172.21.15.102:0/168385148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56200623c0 con 0x7f5620106560 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.723+0000 7f56167fc700 1 --2- 172.21.15.102:0/168385148 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f560c0382d0 0x7f560c03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.723+0000 7f56167fc700 1 -- 172.21.15.102:0/168385148 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f561c04c5f0 con 0x7f5620106560 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.723+0000 7f5624b02700 1 --2- 172.21.15.102:0/168385148 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f560c0382d0 0x7f560c03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.724+0000 7f5624b02700 1 --2- 172.21.15.102:0/168385148 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f560c0382d0 0x7f560c03a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5610006fd0 tx=0x7f5610006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.728+0000 7f56167fc700 1 -- 172.21.15.102:0/168385148 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f561c0189a0 con 0x7f5620106560 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:34.967+0000 7f5627567700 1 -- 172.21.15.102:0/168385148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f562019df60 con 0x7f5620106560 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.074+0000 7f56167fc700 1 -- 172.21.15.102:0/168385148 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f561c019cd0 con 0x7f5620106560 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.076+0000 7f56167fc700 1 -- 172.21.15.102:0/168385148 <== mon.0 v2:172.21.15.102:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f561c04c010 con 0x7f5620106560 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.078+0000 7f5627567700 1 -- 172.21.15.102:0/168385148 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f560c0382d0 msgr2=0x7f560c03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.078+0000 7f5627567700 1 --2- 172.21.15.102:0/168385148 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f560c0382d0 0x7f560c03a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5610006fd0 tx=0x7f5610006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.078+0000 7f5627567700 1 -- 172.21.15.102:0/168385148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 msgr2=0x7f562019a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.078+0000 7f5627567700 1 --2- 172.21.15.102:0/168385148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f562019a1f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f561c009520 tx=0x7f561c011ab0 comp rx=0 tx=0).stop 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.078+0000 7f5627567700 1 -- 172.21.15.102:0/168385148 shutdown_connections 2024-05-13T22:38:35.619 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.078+0000 7f5627567700 1 --2- 172.21.15.102:0/168385148 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f560c0382d0 0x7f560c03a790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:35.620 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.078+0000 7f5627567700 1 --2- 172.21.15.102:0/168385148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5620106560 0x7f562019a1f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:35.620 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.078+0000 7f5627567700 1 -- 172.21.15.102:0/168385148 >> 172.21.15.102:0/168385148 conn(0x7f5620101b40 msgr2=0x7f5620102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:35.620 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.078+0000 7f5627567700 1 -- 172.21.15.102:0/168385148 shutdown_connections 2024-05-13T22:38:35.620 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:35.078+0000 7f5627567700 1 -- 172.21.15.102:0/168385148 wait complete. 2024-05-13T22:38:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:36 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/168385148' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-13T22:38:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:36 smithi102 ceph-mon[28599]: mgrmap e5: smithi102.xcijrj(active, since 6s) 2024-05-13T22:38:36.891 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "active_name": "smithi102.xcijrj", 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.057+0000 7f2dbbbcc700 1 Processor -- start 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.057+0000 7f2dbbbcc700 1 -- start start 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.057+0000 7f2dbbbcc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db4105890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.057+0000 7f2dbbbcc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4105e60 con 0x7f2db4105470 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.057+0000 7f2db9968700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db4105890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.057+0000 7f2db9968700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db4105890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57334/0 (socket says 172.21.15.102:57334) 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.057+0000 7f2db9968700 1 -- 172.21.15.102:0/3932965498 learned_addr learned my addr 172.21.15.102:0/3932965498 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.058+0000 7f2db9968700 1 -- 172.21.15.102:0/3932965498 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db4106670 con 0x7f2db4105470 2024-05-13T22:38:36.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.058+0000 7f2db9968700 1 --2- 172.21.15.102:0/3932965498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db4105890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2db00098a0 tx=0x7f2db0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0a971e6465b267 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.058+0000 7f2db8966700 1 -- 172.21.15.102:0/3932965498 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2db0004180 con 0x7f2db4105470 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.058+0000 7f2db8966700 1 -- 172.21.15.102:0/3932965498 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2db00042e0 con 0x7f2db4105470 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.058+0000 7f2db8966700 1 -- 172.21.15.102:0/3932965498 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2db0011a50 con 0x7f2db4105470 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.059+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/3932965498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 msgr2=0x7f2db4105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.059+0000 7f2dbbbcc700 1 --2- 172.21.15.102:0/3932965498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db4105890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2db00098a0 tx=0x7f2db0009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.059+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/3932965498 shutdown_connections 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.059+0000 7f2dbbbcc700 1 --2- 172.21.15.102:0/3932965498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db4105890 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.059+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/3932965498 >> 172.21.15.102:0/3932965498 conn(0x7f2db41009f0 msgr2=0x7f2db4102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.059+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/3932965498 shutdown_connections 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.059+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/3932965498 wait complete. 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.059+0000 7f2dbbbcc700 1 Processor -- start 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.060+0000 7f2dbbbcc700 1 -- start start 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.060+0000 7f2dbbbcc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.060+0000 7f2dbbbcc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db419a6e0 con 0x7f2db4105470 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.060+0000 7f2db9968700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.060+0000 7f2db9968700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db419a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57340/0 (socket says 172.21.15.102:57340) 2024-05-13T22:38:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.060+0000 7f2db9968700 1 -- 172.21.15.102:0/2214964772 learned_addr learned my addr 172.21.15.102:0/2214964772 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.060+0000 7f2db9968700 1 -- 172.21.15.102:0/2214964772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db0009550 con 0x7f2db4105470 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.060+0000 7f2db9968700 1 --2- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db419a1a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2db0006af0 tx=0x7f2db0011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.060+0000 7f2daaffd700 1 -- 172.21.15.102:0/2214964772 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2db0003b90 con 0x7f2db4105470 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.060+0000 7f2daaffd700 1 -- 172.21.15.102:0/2214964772 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2db001a8a0 con 0x7f2db4105470 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.060+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/2214964772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db419a8e0 con 0x7f2db4105470 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.061+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/2214964772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db419ad80 con 0x7f2db4105470 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.061+0000 7f2daaffd700 1 -- 172.21.15.102:0/2214964772 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2db0019880 con 0x7f2db4105470 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.061+0000 7f2daaffd700 1 -- 172.21.15.102:0/2214964772 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f2db0019b10 con 0x7f2db4105470 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.061+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/2214964772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db40623c0 con 0x7f2db4105470 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.062+0000 7f2daaffd700 1 --2- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f2da0038330 0x7f2da003a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.062+0000 7f2db9167700 1 -- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f2da0038330 msgr2=0x7f2da003a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/4002480395 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.062+0000 7f2daaffd700 1 -- 172.21.15.102:0/2214964772 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2db004d140 con 0x7f2db4105470 2024-05-13T22:38:36.894 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.062+0000 7f2db9167700 1 --2- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f2da0038330 0x7f2da003a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.066+0000 7f2daaffd700 1 -- 172.21.15.102:0/2214964772 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2db002c3a0 con 0x7f2db4105470 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.263+0000 7f2db9167700 1 -- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f2da0038330 msgr2=0x7f2da003a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/4002480395 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.263+0000 7f2db9167700 1 --2- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f2da0038330 0x7f2da003a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.352+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/2214964772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f2db4109860 con 0x7f2db4105470 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.353+0000 7f2daaffd700 1 -- 172.21.15.102:0/2214964772 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f2db00172b0 con 0x7f2db4105470 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.355+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f2da0038330 msgr2=0x7f2da003a7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.355+0000 7f2dbbbcc700 1 --2- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f2da0038330 0x7f2da003a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.355+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 msgr2=0x7f2db419a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.355+0000 7f2dbbbcc700 1 --2- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db419a1a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2db0006af0 tx=0x7f2db0011a90 comp rx=0 tx=0).stop 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.355+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/2214964772 shutdown_connections 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.355+0000 7f2dbbbcc700 1 --2- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f2da0038330 0x7f2da003a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.355+0000 7f2dbbbcc700 1 --2- 172.21.15.102:0/2214964772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2db4105470 0x7f2db419a1a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.355+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/2214964772 >> 172.21.15.102:0/2214964772 conn(0x7f2db41009f0 msgr2=0x7f2db40798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:36.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.355+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/2214964772 shutdown_connections 2024-05-13T22:38:36.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:36.355+0000 7f2dbbbcc700 1 -- 172.21.15.102:0/2214964772 wait complete. 2024-05-13T22:38:36.896 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for the mgr to restart... 2024-05-13T22:38:36.896 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr epoch 5... 2024-05-13T22:38:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:37 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2214964772' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-13T22:38:41.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: Active manager daemon smithi102.xcijrj restarted 2024-05-13T22:38:41.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: Activating manager daemon smithi102.xcijrj 2024-05-13T22:38:41.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: osdmap e2: 0 total, 0 up, 0 in 2024-05-13T22:38:41.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: mgrmap e6: smithi102.xcijrj(active, starting, since 0.00358349s) 2024-05-13T22:38:41.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi102.xcijrj", "id": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: Manager daemon smithi102.xcijrj is now available 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:38:41.930 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:41 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/trash_purge_schedule"}]: dispatch 2024-05-13T22:38:43.208 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-05-13T22:38:43.208 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-13T22:38:43.208 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-13T22:38:43.208 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-05-13T22:38:43.208 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.341+0000 7f16b3b36700 1 Processor -- start 2024-05-13T22:38:43.208 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.341+0000 7f16b3b36700 1 -- start start 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.342+0000 7f16b3b36700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.342+0000 7f16b3b36700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac079e50 con 0x7f16ac107020 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.342+0000 7f16b18d2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.342+0000 7f16b18d2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57354/0 (socket says 172.21.15.102:57354) 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.342+0000 7f16b18d2700 1 -- 172.21.15.102:0/3301403226 learned_addr learned my addr 172.21.15.102:0/3301403226 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.342+0000 7f16b18d2700 1 -- 172.21.15.102:0/3301403226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac109950 con 0x7f16ac107020 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.342+0000 7f16b18d2700 1 --2- 172.21.15.102:0/3301403226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac109410 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f169c0098a0 tx=0x7f169c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3894dfe7899cecc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.342+0000 7f16b08d0700 1 -- 172.21.15.102:0/3301403226 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f169c004180 con 0x7f16ac107020 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.343+0000 7f16b08d0700 1 -- 172.21.15.102:0/3301403226 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f169c0042e0 con 0x7f16ac107020 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.343+0000 7f16b08d0700 1 -- 172.21.15.102:0/3301403226 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f169c011a50 con 0x7f16ac107020 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.343+0000 7f16b3b36700 1 -- 172.21.15.102:0/3301403226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 msgr2=0x7f16ac109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:43.209 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.343+0000 7f16b3b36700 1 --2- 172.21.15.102:0/3301403226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac109410 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f169c0098a0 tx=0x7f169c009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.343+0000 7f16b3b36700 1 -- 172.21.15.102:0/3301403226 shutdown_connections 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.343+0000 7f16b3b36700 1 --2- 172.21.15.102:0/3301403226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac109410 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.343+0000 7f16b3b36700 1 -- 172.21.15.102:0/3301403226 >> 172.21.15.102:0/3301403226 conn(0x7f16ac1009f0 msgr2=0x7f16ac102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.343+0000 7f16b3b36700 1 -- 172.21.15.102:0/3301403226 shutdown_connections 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.343+0000 7f16b3b36700 1 -- 172.21.15.102:0/3301403226 wait complete. 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.344+0000 7f16b3b36700 1 Processor -- start 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.344+0000 7f16b3b36700 1 -- start start 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.344+0000 7f16b3b36700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.344+0000 7f16b3b36700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac19a6b0 con 0x7f16ac107020 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.344+0000 7f16b18d2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.344+0000 7f16b18d2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac19a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57364/0 (socket says 172.21.15.102:57364) 2024-05-13T22:38:43.210 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.344+0000 7f16b18d2700 1 -- 172.21.15.102:0/2170324578 learned_addr learned my addr 172.21.15.102:0/2170324578 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:43.211 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.345+0000 7f16b18d2700 1 -- 172.21.15.102:0/2170324578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c009550 con 0x7f16ac107020 2024-05-13T22:38:43.211 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.345+0000 7f16b18d2700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac19a170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f169c0119e0 tx=0x7f169c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:43.211 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.345+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f169c003b90 con 0x7f16ac107020 2024-05-13T22:38:43.211 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.345+0000 7f16b3b36700 1 -- 172.21.15.102:0/2170324578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ac19a8b0 con 0x7f16ac107020 2024-05-13T22:38:43.211 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.345+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f169c01a8a0 con 0x7f16ac107020 2024-05-13T22:38:43.211 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.345+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f169c019930 con 0x7f16ac107020 2024-05-13T22:38:43.211 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.345+0000 7f16b3b36700 1 -- 172.21.15.102:0/2170324578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac19ad50 con 0x7f16ac107020 2024-05-13T22:38:43.211 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.346+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f169c019ba0 con 0x7f16ac107020 2024-05-13T22:38:43.211 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.346+0000 7f16a2ffd700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 0x7f169803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:43.211 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.346+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 --> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f169803af00 con 0x7f1698038330 2024-05-13T22:38:43.211 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.346+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f169c04c670 con 0x7f16ac107020 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.346+0000 7f16b10d1700 1 -- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 msgr2=0x7f169803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/4002480395 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.346+0000 7f16b10d1700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 0x7f169803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.547+0000 7f16b10d1700 1 -- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 msgr2=0x7f169803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/4002480395 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.547+0000 7f16b10d1700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 0x7f169803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.948+0000 7f16b10d1700 1 -- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 msgr2=0x7f169803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/4002480395 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:37.948+0000 7f16b10d1700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 0x7f169803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:38.749+0000 7f16b10d1700 1 -- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 msgr2=0x7f169803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/4002480395 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:38.749+0000 7f16b10d1700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 0x7f169803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:40.350+0000 7f16b10d1700 1 -- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 msgr2=0x7f169803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/4002480395 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:40.350+0000 7f16b10d1700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 0x7f169803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:41.660+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7f169c02c3a0 con 0x7f16ac107020 2024-05-13T22:38:43.212 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:41.661+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 msgr2=0x7f169803a7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:41.661+0000 7f16a2ffd700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 0x7f169803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.661+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f169c029d60 con 0x7f16ac107020 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.661+0000 7f16a2ffd700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f169803bc60 0x7f169803e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.661+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f169803af00 con 0x7f169803bc60 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.662+0000 7f16b10d1700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f169803bc60 0x7f169803e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.663+0000 7f16b10d1700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f169803bc60 0x7f169803e050 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f16a8003a10 tx=0x7f16a80092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.663+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f169803af00 con 0x7f169803bc60 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.669+0000 7f16b3b36700 1 -- 172.21.15.102:0/2170324578 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f16ac19b4b0 con 0x7f169803bc60 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.669+0000 7f16a2ffd700 1 -- 172.21.15.102:0/2170324578 <== mgr.14120 v2:172.21.15.102:6800/358342393 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f16ac19b4b0 con 0x7f169803bc60 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.670+0000 7f16b3b36700 1 -- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f169803bc60 msgr2=0x7f169803e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.670+0000 7f16b3b36700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f169803bc60 0x7f169803e050 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f16a8003a10 tx=0x7f16a80092b0 comp rx=0 tx=0).stop 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.670+0000 7f16b3b36700 1 -- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 msgr2=0x7f16ac19a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.670+0000 7f16b3b36700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac19a170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f169c0119e0 tx=0x7f169c011ac0 comp rx=0 tx=0).stop 2024-05-13T22:38:43.213 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.670+0000 7f16b3b36700 1 -- 172.21.15.102:0/2170324578 shutdown_connections 2024-05-13T22:38:43.214 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.670+0000 7f16b3b36700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f169803bc60 0x7f169803e050 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:43.214 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.670+0000 7f16b3b36700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16ac107020 0x7f16ac19a170 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:43.214 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.670+0000 7f16b3b36700 1 --2- 172.21.15.102:0/2170324578 >> [v2:172.21.15.102:6800/4002480395,v1:172.21.15.102:6801/4002480395] conn(0x7f1698038330 0x7f169803a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:43.214 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.670+0000 7f16b3b36700 1 -- 172.21.15.102:0/2170324578 >> 172.21.15.102:0/2170324578 conn(0x7f16ac1009f0 msgr2=0x7f16ac101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:43.214 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.670+0000 7f16b3b36700 1 -- 172.21.15.102:0/2170324578 shutdown_connections 2024-05-13T22:38:43.214 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:42.670+0000 7f16b3b36700 1 -- 172.21.15.102:0/2170324578 wait complete. 2024-05-13T22:38:43.214 INFO:teuthology.orchestra.run.smithi102.stdout:mgr epoch 5 is available 2024-05-13T22:38:43.214 INFO:teuthology.orchestra.run.smithi102.stdout:Setting orchestrator backend to cephadm... 2024-05-13T22:38:43.282 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:43 smithi102 ceph-mon[28599]: Found migration_current of "None". Setting to last migration. 2024-05-13T22:38:43.282 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:43 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:43.282 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:43 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:43.282 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:43 smithi102 ceph-mon[28599]: mgrmap e7: smithi102.xcijrj(active, since 1.00525s) 2024-05-13T22:38:44.111 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:44 smithi102 ceph-mon[28599]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-13T22:38:44.111 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:44 smithi102 ceph-mon[28599]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-13T22:38:44.111 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:44 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:38:44.111 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:44 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:44.111 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:44 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.630+0000 7f3aba967700 1 Processor -- start 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.630+0000 7f3aba967700 1 -- start start 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.630+0000 7f3aba967700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab4106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.630+0000 7f3aba967700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab4106f50 con 0x7f3ab4106560 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.630+0000 7f3ab3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab4106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.630+0000 7f3ab3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab4106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60650/0 (socket says 172.21.15.102:60650) 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.630+0000 7f3ab3fff700 1 -- 172.21.15.102:0/1077786570 learned_addr learned my addr 172.21.15.102:0/1077786570 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.631+0000 7f3ab3fff700 1 -- 172.21.15.102:0/1077786570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab4107760 con 0x7f3ab4106560 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.631+0000 7f3ab3fff700 1 --2- 172.21.15.102:0/1077786570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab4106980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40098a0 tx=0x7f3aa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8acc024687b2ef server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.631+0000 7f3ab37fe700 1 -- 172.21.15.102:0/1077786570 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3aa4004180 con 0x7f3ab4106560 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.631+0000 7f3ab37fe700 1 -- 172.21.15.102:0/1077786570 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3aa40042e0 con 0x7f3ab4106560 2024-05-13T22:38:44.387 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.631+0000 7f3ab37fe700 1 -- 172.21.15.102:0/1077786570 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3aa4011a50 con 0x7f3ab4106560 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3aba967700 1 -- 172.21.15.102:0/1077786570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 msgr2=0x7f3ab4106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3aba967700 1 --2- 172.21.15.102:0/1077786570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab4106980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40098a0 tx=0x7f3aa4009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3aba967700 1 -- 172.21.15.102:0/1077786570 shutdown_connections 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3aba967700 1 --2- 172.21.15.102:0/1077786570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab4106980 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3aba967700 1 -- 172.21.15.102:0/1077786570 >> 172.21.15.102:0/1077786570 conn(0x7f3ab4101b40 msgr2=0x7f3ab4103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3aba967700 1 -- 172.21.15.102:0/1077786570 shutdown_connections 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3aba967700 1 -- 172.21.15.102:0/1077786570 wait complete. 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3aba967700 1 Processor -- start 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3aba967700 1 -- start start 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3aba967700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3aba967700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab419a730 con 0x7f3ab4106560 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3ab3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:44.388 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.632+0000 7f3ab3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60652/0 (socket says 172.21.15.102:60652) 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.633+0000 7f3ab3fff700 1 -- 172.21.15.102:0/1941014319 learned_addr learned my addr 172.21.15.102:0/1941014319 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.633+0000 7f3ab3fff700 1 -- 172.21.15.102:0/1941014319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4009550 con 0x7f3ab4106560 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.633+0000 7f3ab3fff700 1 --2- 172.21.15.102:0/1941014319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab419a1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009520 tx=0x7f3aa4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.633+0000 7f3ab1ffb700 1 -- 172.21.15.102:0/1941014319 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3aa4003a80 con 0x7f3ab4106560 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.633+0000 7f3ab1ffb700 1 -- 172.21.15.102:0/1941014319 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3aa4003be0 con 0x7f3ab4106560 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.633+0000 7f3aba967700 1 -- 172.21.15.102:0/1941014319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab419a930 con 0x7f3ab4106560 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.633+0000 7f3ab1ffb700 1 -- 172.21.15.102:0/1941014319 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3aa40197b0 con 0x7f3ab4106560 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.633+0000 7f3aba967700 1 -- 172.21.15.102:0/1941014319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab419add0 con 0x7f3ab4106560 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.634+0000 7f3ab1ffb700 1 -- 172.21.15.102:0/1941014319 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f3aa4019a20 con 0x7f3ab4106560 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.634+0000 7f3ab1ffb700 1 --2- 172.21.15.102:0/1941014319 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f3aa00381f0 0x7f3aa003a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.634+0000 7f3aba967700 1 -- 172.21.15.102:0/1941014319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab40623c0 con 0x7f3ab4106560 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.634+0000 7f3ab1ffb700 1 -- 172.21.15.102:0/1941014319 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3aa404c340 con 0x7f3ab4106560 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.634+0000 7f3aabfff700 1 --2- 172.21.15.102:0/1941014319 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f3aa00381f0 0x7f3aa003a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:44.389 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.635+0000 7f3aabfff700 1 --2- 172.21.15.102:0/1941014319 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f3aa00381f0 0x7f3aa003a6b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c006fd0 tx=0x7f3a9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.639+0000 7f3ab1ffb700 1 -- 172.21.15.102:0/1941014319 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3aa402c3a0 con 0x7f3ab4106560 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.840+0000 7f3aba967700 1 -- 172.21.15.102:0/1941014319 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ab410a960 con 0x7f3aa00381f0 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.847+0000 7f3ab1ffb700 1 -- 172.21.15.102:0/1941014319 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3ab410a960 con 0x7f3aa00381f0 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.849+0000 7f3aba967700 1 -- 172.21.15.102:0/1941014319 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f3aa00381f0 msgr2=0x7f3aa003a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.849+0000 7f3aba967700 1 --2- 172.21.15.102:0/1941014319 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f3aa00381f0 0x7f3aa003a6b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c006fd0 tx=0x7f3a9c006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.849+0000 7f3aba967700 1 -- 172.21.15.102:0/1941014319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 msgr2=0x7f3ab419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.849+0000 7f3aba967700 1 --2- 172.21.15.102:0/1941014319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab419a1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009520 tx=0x7f3aa4011ab0 comp rx=0 tx=0).stop 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.849+0000 7f3aba967700 1 -- 172.21.15.102:0/1941014319 shutdown_connections 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.849+0000 7f3aba967700 1 --2- 172.21.15.102:0/1941014319 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f3aa00381f0 0x7f3aa003a6b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.849+0000 7f3aba967700 1 --2- 172.21.15.102:0/1941014319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ab4106560 0x7f3ab419a1f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.849+0000 7f3aba967700 1 -- 172.21.15.102:0/1941014319 >> 172.21.15.102:0/1941014319 conn(0x7f3ab4101b40 msgr2=0x7f3ab4102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.849+0000 7f3aba967700 1 -- 172.21.15.102:0/1941014319 shutdown_connections 2024-05-13T22:38:44.390 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:43.849+0000 7f3aba967700 1 -- 172.21.15.102:0/1941014319 wait complete. 2024-05-13T22:38:45.259 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:45 smithi102 ceph-mon[28599]: [13/May/2024:22:38:43] ENGINE Bus STARTING 2024-05-13T22:38:45.259 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:45 smithi102 ceph-mon[28599]: [13/May/2024:22:38:43] ENGINE Serving on https://172.21.15.102:7150 2024-05-13T22:38:45.260 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:45 smithi102 ceph-mon[28599]: [13/May/2024:22:38:43] ENGINE Serving on http://172.21.15.102:8765 2024-05-13T22:38:45.260 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:45 smithi102 ceph-mon[28599]: [13/May/2024:22:38:43] ENGINE Bus STARTED 2024-05-13T22:38:45.260 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:45 smithi102 ceph-mon[28599]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:38:45.260 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:45 smithi102 ceph-mon[28599]: mgrmap e8: smithi102.xcijrj(active, since 2s) 2024-05-13T22:38:45.552 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-13T22:38:45.552 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.801+0000 7fae86bf6700 1 Processor -- start 2024-05-13T22:38:45.552 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.801+0000 7fae86bf6700 1 -- start start 2024-05-13T22:38:45.552 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.801+0000 7fae86bf6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae80105890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:45.553 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.801+0000 7fae86bf6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae80105e60 con 0x7fae80105470 2024-05-13T22:38:45.553 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.802+0000 7fae84992700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae80105890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:45.553 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.802+0000 7fae84992700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae80105890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60656/0 (socket says 172.21.15.102:60656) 2024-05-13T22:38:45.553 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.802+0000 7fae84992700 1 -- 172.21.15.102:0/3223638660 learned_addr learned my addr 172.21.15.102:0/3223638660 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:45.553 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.802+0000 7fae84992700 1 -- 172.21.15.102:0/3223638660 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80106670 con 0x7fae80105470 2024-05-13T22:38:45.553 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.802+0000 7fae84992700 1 --2- 172.21.15.102:0/3223638660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae80105890 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009e80 tx=0x7fae7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdf41f5ad4c0c7b server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:45.553 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.802+0000 7fae777fe700 1 -- 172.21.15.102:0/3223638660 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae7c00e070 con 0x7fae80105470 2024-05-13T22:38:45.553 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.802+0000 7fae777fe700 1 -- 172.21.15.102:0/3223638660 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fae7c003e90 con 0x7fae80105470 2024-05-13T22:38:45.553 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.802+0000 7fae777fe700 1 -- 172.21.15.102:0/3223638660 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae7c004380 con 0x7fae80105470 2024-05-13T22:38:45.553 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.803+0000 7fae86bf6700 1 -- 172.21.15.102:0/3223638660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 msgr2=0x7fae80105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.803+0000 7fae86bf6700 1 --2- 172.21.15.102:0/3223638660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae80105890 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009e80 tx=0x7fae7c00b330 comp rx=0 tx=0).stop 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.803+0000 7fae86bf6700 1 -- 172.21.15.102:0/3223638660 shutdown_connections 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.803+0000 7fae86bf6700 1 --2- 172.21.15.102:0/3223638660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae80105890 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.803+0000 7fae86bf6700 1 -- 172.21.15.102:0/3223638660 >> 172.21.15.102:0/3223638660 conn(0x7fae801009f0 msgr2=0x7fae80102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.803+0000 7fae86bf6700 1 -- 172.21.15.102:0/3223638660 shutdown_connections 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.803+0000 7fae86bf6700 1 -- 172.21.15.102:0/3223638660 wait complete. 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.803+0000 7fae86bf6700 1 Processor -- start 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.803+0000 7fae86bf6700 1 -- start start 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae86bf6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae8019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae86bf6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8019a670 con 0x7fae80105470 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae84992700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae8019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae84992700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae8019a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60664/0 (socket says 172.21.15.102:60664) 2024-05-13T22:38:45.554 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae84992700 1 -- 172.21.15.102:0/2234301945 learned_addr learned my addr 172.21.15.102:0/2234301945 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae84992700 1 -- 172.21.15.102:0/2234301945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c009b30 con 0x7fae80105470 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae84992700 1 --2- 172.21.15.102:0/2234301945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae8019a130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b7b0 tx=0x7fae7c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae75ffb700 1 -- 172.21.15.102:0/2234301945 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae7c00e040 con 0x7fae80105470 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae75ffb700 1 -- 172.21.15.102:0/2234301945 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fae7c003c70 con 0x7fae80105470 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae86bf6700 1 -- 172.21.15.102:0/2234301945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8019a870 con 0x7fae80105470 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae75ffb700 1 -- 172.21.15.102:0/2234301945 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae7c01b730 con 0x7fae80105470 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.804+0000 7fae86bf6700 1 -- 172.21.15.102:0/2234301945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8019ad70 con 0x7fae80105470 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.805+0000 7fae75ffb700 1 -- 172.21.15.102:0/2234301945 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fae7c019040 con 0x7fae80105470 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.806+0000 7fae86bf6700 1 -- 172.21.15.102:0/2234301945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae800623c0 con 0x7fae80105470 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.806+0000 7fae75ffb700 1 --2- 172.21.15.102:0/2234301945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fae68038330 0x7fae6803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.806+0000 7fae75ffb700 1 -- 172.21.15.102:0/2234301945 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fae7c04bdc0 con 0x7fae80105470 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.806+0000 7fae77fff700 1 --2- 172.21.15.102:0/2234301945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fae68038330 0x7fae6803a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.806+0000 7fae77fff700 1 --2- 172.21.15.102:0/2234301945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fae68038330 0x7fae6803a7f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae70006fd0 tx=0x7fae70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:45.555 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.811+0000 7fae75ffb700 1 -- 172.21.15.102:0/2234301945 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fae7c023a00 con 0x7fae80105470 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.998+0000 7fae86bf6700 1 -- 172.21.15.102:0/2234301945 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fae80109870 con 0x7fae68038330 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:44.999+0000 7fae75ffb700 1 -- 172.21.15.102:0/2234301945 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fae80109870 con 0x7fae68038330 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:45.001+0000 7fae86bf6700 1 -- 172.21.15.102:0/2234301945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fae68038330 msgr2=0x7fae6803a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:45.001+0000 7fae86bf6700 1 --2- 172.21.15.102:0/2234301945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fae68038330 0x7fae6803a7f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae70006fd0 tx=0x7fae70006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:45.001+0000 7fae86bf6700 1 -- 172.21.15.102:0/2234301945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 msgr2=0x7fae8019a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:45.001+0000 7fae86bf6700 1 --2- 172.21.15.102:0/2234301945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae8019a130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b7b0 tx=0x7fae7c004430 comp rx=0 tx=0).stop 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:45.001+0000 7fae86bf6700 1 -- 172.21.15.102:0/2234301945 shutdown_connections 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:45.001+0000 7fae86bf6700 1 --2- 172.21.15.102:0/2234301945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fae68038330 0x7fae6803a7f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:45.001+0000 7fae86bf6700 1 --2- 172.21.15.102:0/2234301945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80105470 0x7fae8019a130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:45.001+0000 7fae86bf6700 1 -- 172.21.15.102:0/2234301945 >> 172.21.15.102:0/2234301945 conn(0x7fae801009f0 msgr2=0x7fae800740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:45.001+0000 7fae86bf6700 1 -- 172.21.15.102:0/2234301945 shutdown_connections 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:45.001+0000 7fae86bf6700 1 -- 172.21.15.102:0/2234301945 wait complete. 2024-05-13T22:38:45.556 INFO:teuthology.orchestra.run.smithi102.stdout:Generating ssh key... 2024-05-13T22:38:46.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:46 smithi102 ceph-mon[28599]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:38:47.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.010+0000 7fc78972f700 1 Processor -- start 2024-05-13T22:38:47.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.010+0000 7fc78972f700 1 -- start start 2024-05-13T22:38:47.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.010+0000 7fc78972f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc784109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:47.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.010+0000 7fc78972f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840745e0 con 0x7fc784107080 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.010+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc784109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.010+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc784109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60674/0 (socket says 172.21.15.102:60674) 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.010+0000 7fc782ffd700 1 -- 172.21.15.102:0/3794428377 learned_addr learned my addr 172.21.15.102:0/3794428377 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.010+0000 7fc782ffd700 1 -- 172.21.15.102:0/3794428377 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc784074720 con 0x7fc784107080 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.011+0000 7fc782ffd700 1 --2- 172.21.15.102:0/3794428377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc784109470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc7740098a0 tx=0x7fc774009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b9aa21c7bc05d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.011+0000 7fc781ffb700 1 -- 172.21.15.102:0/3794428377 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc774004180 con 0x7fc784107080 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.011+0000 7fc781ffb700 1 -- 172.21.15.102:0/3794428377 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7740042e0 con 0x7fc784107080 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.011+0000 7fc781ffb700 1 -- 172.21.15.102:0/3794428377 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc774011a50 con 0x7fc784107080 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.011+0000 7fc78972f700 1 -- 172.21.15.102:0/3794428377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 msgr2=0x7fc784109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.011+0000 7fc78972f700 1 --2- 172.21.15.102:0/3794428377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc784109470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc7740098a0 tx=0x7fc774009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.011+0000 7fc78972f700 1 -- 172.21.15.102:0/3794428377 shutdown_connections 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.011+0000 7fc78972f700 1 --2- 172.21.15.102:0/3794428377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc784109470 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.011+0000 7fc78972f700 1 -- 172.21.15.102:0/3794428377 >> 172.21.15.102:0/3794428377 conn(0x7fc7841009f0 msgr2=0x7fc784102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.011+0000 7fc78972f700 1 -- 172.21.15.102:0/3794428377 shutdown_connections 2024-05-13T22:38:47.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.011+0000 7fc78972f700 1 -- 172.21.15.102:0/3794428377 wait complete. 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.012+0000 7fc78972f700 1 Processor -- start 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.012+0000 7fc78972f700 1 -- start start 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.012+0000 7fc78972f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc7841069e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.012+0000 7fc78972f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc784106f20 con 0x7fc784107080 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.012+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc7841069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.012+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc7841069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60688/0 (socket says 172.21.15.102:60688) 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.012+0000 7fc782ffd700 1 -- 172.21.15.102:0/2227806984 learned_addr learned my addr 172.21.15.102:0/2227806984 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.013+0000 7fc782ffd700 1 -- 172.21.15.102:0/2227806984 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc774009550 con 0x7fc784107080 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.013+0000 7fc782ffd700 1 --2- 172.21.15.102:0/2227806984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc7841069e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc7740119e0 tx=0x7fc774011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.013+0000 7fc76bfff700 1 -- 172.21.15.102:0/2227806984 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc774003b90 con 0x7fc784107080 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.013+0000 7fc76bfff700 1 -- 172.21.15.102:0/2227806984 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc77401a8a0 con 0x7fc784107080 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.013+0000 7fc78972f700 1 -- 172.21.15.102:0/2227806984 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc784105030 con 0x7fc784107080 2024-05-13T22:38:47.102 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.013+0000 7fc76bfff700 1 -- 172.21.15.102:0/2227806984 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc774019930 con 0x7fc784107080 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.013+0000 7fc78972f700 1 -- 172.21.15.102:0/2227806984 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7841054d0 con 0x7fc784107080 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.014+0000 7fc76bfff700 1 -- 172.21.15.102:0/2227806984 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fc774019ba0 con 0x7fc784107080 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.014+0000 7fc78972f700 1 -- 172.21.15.102:0/2227806984 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc784062430 con 0x7fc784107080 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.014+0000 7fc76bfff700 1 --2- 172.21.15.102:0/2227806984 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc76c038330 0x7fc76c03a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.014+0000 7fc76bfff700 1 -- 172.21.15.102:0/2227806984 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc77404c6a0 con 0x7fc784107080 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.014+0000 7fc7827fc700 1 --2- 172.21.15.102:0/2227806984 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc76c038330 0x7fc76c03a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.015+0000 7fc7827fc700 1 --2- 172.21.15.102:0/2227806984 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc76c038330 0x7fc76c03a7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc778006fd0 tx=0x7fc778006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.019+0000 7fc76bfff700 1 -- 172.21.15.102:0/2227806984 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc784062430 con 0x7fc784107080 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.210+0000 7fc78972f700 1 -- 172.21.15.102:0/2227806984 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7840623c0 con 0x7fc76c038330 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.549+0000 7fc76bfff700 1 -- 172.21.15.102:0/2227806984 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc7840623c0 con 0x7fc76c038330 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.551+0000 7fc78972f700 1 -- 172.21.15.102:0/2227806984 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc76c038330 msgr2=0x7fc76c03a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.551+0000 7fc78972f700 1 --2- 172.21.15.102:0/2227806984 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc76c038330 0x7fc76c03a7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc778006fd0 tx=0x7fc778006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.551+0000 7fc78972f700 1 -- 172.21.15.102:0/2227806984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 msgr2=0x7fc7841069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:47.103 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.551+0000 7fc78972f700 1 --2- 172.21.15.102:0/2227806984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc7841069e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc7740119e0 tx=0x7fc774011ac0 comp rx=0 tx=0).stop 2024-05-13T22:38:47.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.551+0000 7fc78972f700 1 -- 172.21.15.102:0/2227806984 shutdown_connections 2024-05-13T22:38:47.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.551+0000 7fc78972f700 1 --2- 172.21.15.102:0/2227806984 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc76c038330 0x7fc76c03a7f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:47.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.551+0000 7fc78972f700 1 --2- 172.21.15.102:0/2227806984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc784107080 0x7fc7841069e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:47.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.551+0000 7fc78972f700 1 -- 172.21.15.102:0/2227806984 >> 172.21.15.102:0/2227806984 conn(0x7fc7841009f0 msgr2=0x7fc784101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:47.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.551+0000 7fc78972f700 1 -- 172.21.15.102:0/2227806984 shutdown_connections 2024-05-13T22:38:47.104 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:46.551+0000 7fc78972f700 1 -- 172.21.15.102:0/2227806984 wait complete. 2024-05-13T22:38:47.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:47 smithi102 ceph-mon[28599]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:38:47.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:47 smithi102 ceph-mon[28599]: Generating ssh key... 2024-05-13T22:38:47.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:47 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:47.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:47 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:48.241 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGf9rikECSrcy4I6yD+IeGLptnhpU/3IoIlfHjXRW5m+WoWYqndYUAeDbUBmqFKjdZd/IX3WM+5d8AffQV2T4o4Sao9YOqSMxyBAAyQsD86pzwtnWrA2LRAPJIPs/pSLb/KsprC9hJeNHLpnhTQNDpTvehouztrXQSZdXwfis7DmDi2SqU8u2EeX13+nPcYB+OO1miwlUso+KAVnVYliTfnWoGi0AC2FEzU3oKF5OsCf74PqCVgMDsZjXCWSz9rV2Y2qUcVQN+Y5KStpfUpizwxQgoGbTgyM34kBcDMrbqXtnbob1KDt/zoZy6FduQ88Y9UOQAo2xx4ZW2XQZ467D674vUi+QJqpOq6BZJJC/mkPkZNN5U1ixxUNLuOI/rDfKQehII8noRWfNil3n+ochLYXBh1CfnIpGkdetgTJfe1BPJEaQlUTV8HaZVnnsNIcr/3ldN7ibE1ldb2n3Eq47Kwi0syCDFe8yyHfaVEGp4akJlKMVbsJ7E2Wu5Cz454QE= ceph-62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:38:48.242 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.505+0000 7efe991c5700 1 Processor -- start 2024-05-13T22:38:48.242 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.505+0000 7efe991c5700 1 -- start start 2024-05-13T22:38:48.242 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.505+0000 7efe991c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe94104c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:48.242 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.505+0000 7efe991c5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe94105230 con 0x7efe94104840 2024-05-13T22:38:48.242 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.505+0000 7efe92d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe94104c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:48.242 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.505+0000 7efe92d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe94104c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60700/0 (socket says 172.21.15.102:60700) 2024-05-13T22:38:48.242 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.505+0000 7efe92d9d700 1 -- 172.21.15.102:0/3608974004 learned_addr learned my addr 172.21.15.102:0/3608974004 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:48.242 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.505+0000 7efe92d9d700 1 -- 172.21.15.102:0/3608974004 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe941077d0 con 0x7efe94104840 2024-05-13T22:38:48.242 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.506+0000 7efe92d9d700 1 --2- 172.21.15.102:0/3608974004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe94104c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efe840098a0 tx=0x7efe84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d46827c8b50d4f server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.506+0000 7efe91d9b700 1 -- 172.21.15.102:0/3608974004 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe84004180 con 0x7efe94104840 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.506+0000 7efe91d9b700 1 -- 172.21.15.102:0/3608974004 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe840042e0 con 0x7efe94104840 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.506+0000 7efe91d9b700 1 -- 172.21.15.102:0/3608974004 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe84011a50 con 0x7efe94104840 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.506+0000 7efe991c5700 1 -- 172.21.15.102:0/3608974004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 msgr2=0x7efe94104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.506+0000 7efe991c5700 1 --2- 172.21.15.102:0/3608974004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe94104c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efe840098a0 tx=0x7efe84009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.506+0000 7efe991c5700 1 -- 172.21.15.102:0/3608974004 shutdown_connections 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.506+0000 7efe991c5700 1 --2- 172.21.15.102:0/3608974004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe94104c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.506+0000 7efe991c5700 1 -- 172.21.15.102:0/3608974004 >> 172.21.15.102:0/3608974004 conn(0x7efe94100170 msgr2=0x7efe941025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.506+0000 7efe991c5700 1 -- 172.21.15.102:0/3608974004 shutdown_connections 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.506+0000 7efe991c5700 1 -- 172.21.15.102:0/3608974004 wait complete. 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.507+0000 7efe991c5700 1 Processor -- start 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.507+0000 7efe991c5700 1 -- start start 2024-05-13T22:38:48.243 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.507+0000 7efe991c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe9419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.507+0000 7efe991c5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9419a740 con 0x7efe94104840 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.508+0000 7efe92d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe9419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.508+0000 7efe92d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe9419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60710/0 (socket says 172.21.15.102:60710) 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.508+0000 7efe92d9d700 1 -- 172.21.15.102:0/3906494313 learned_addr learned my addr 172.21.15.102:0/3906494313 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.508+0000 7efe92d9d700 1 -- 172.21.15.102:0/3906494313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84009550 con 0x7efe94104840 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.508+0000 7efe92d9d700 1 --2- 172.21.15.102:0/3906494313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe9419a200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efe84009520 tx=0x7efe84011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.508+0000 7efe7bfff700 1 -- 172.21.15.102:0/3906494313 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe84003a80 con 0x7efe94104840 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.508+0000 7efe991c5700 1 -- 172.21.15.102:0/3906494313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9419a940 con 0x7efe94104840 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.508+0000 7efe7bfff700 1 -- 172.21.15.102:0/3906494313 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7efe84003be0 con 0x7efe94104840 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.508+0000 7efe7bfff700 1 -- 172.21.15.102:0/3906494313 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe840197b0 con 0x7efe94104840 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.508+0000 7efe991c5700 1 -- 172.21.15.102:0/3906494313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9419ade0 con 0x7efe94104840 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.509+0000 7efe7bfff700 1 -- 172.21.15.102:0/3906494313 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7efe84019a70 con 0x7efe94104840 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.509+0000 7efe991c5700 1 -- 172.21.15.102:0/3906494313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe940623c0 con 0x7efe94104840 2024-05-13T22:38:48.244 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.509+0000 7efe7bfff700 1 --2- 172.21.15.102:0/3906494313 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7efe7c0382d0 0x7efe7c03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.509+0000 7efe9259c700 1 --2- 172.21.15.102:0/3906494313 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7efe7c0382d0 0x7efe7c03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.509+0000 7efe7bfff700 1 -- 172.21.15.102:0/3906494313 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efe8404c560 con 0x7efe94104840 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.510+0000 7efe9259c700 1 --2- 172.21.15.102:0/3906494313 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7efe7c0382d0 0x7efe7c03a790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efe88006fd0 tx=0x7efe88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.514+0000 7efe7bfff700 1 -- 172.21.15.102:0/3906494313 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efe8402b750 con 0x7efe94104840 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.705+0000 7efe991c5700 1 -- 172.21.15.102:0/3906494313 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7efe9410a9d0 con 0x7efe7c0382d0 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.706+0000 7efe7bfff700 1 -- 172.21.15.102:0/3906494313 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7efe9410a9d0 con 0x7efe7c0382d0 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.708+0000 7efe991c5700 1 -- 172.21.15.102:0/3906494313 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7efe7c0382d0 msgr2=0x7efe7c03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.708+0000 7efe991c5700 1 --2- 172.21.15.102:0/3906494313 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7efe7c0382d0 0x7efe7c03a790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efe88006fd0 tx=0x7efe88006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.708+0000 7efe991c5700 1 -- 172.21.15.102:0/3906494313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 msgr2=0x7efe9419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.708+0000 7efe991c5700 1 --2- 172.21.15.102:0/3906494313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe9419a200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efe84009520 tx=0x7efe84011ab0 comp rx=0 tx=0).stop 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.708+0000 7efe991c5700 1 -- 172.21.15.102:0/3906494313 shutdown_connections 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.708+0000 7efe991c5700 1 --2- 172.21.15.102:0/3906494313 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7efe7c0382d0 0x7efe7c03a790 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.708+0000 7efe991c5700 1 --2- 172.21.15.102:0/3906494313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe94104840 0x7efe9419a200 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:48.245 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.708+0000 7efe991c5700 1 -- 172.21.15.102:0/3906494313 >> 172.21.15.102:0/3906494313 conn(0x7efe94100170 msgr2=0x7efe94101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:48.246 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.708+0000 7efe991c5700 1 -- 172.21.15.102:0/3906494313 shutdown_connections 2024-05-13T22:38:48.246 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:47.708+0000 7efe991c5700 1 -- 172.21.15.102:0/3906494313 wait complete. 2024-05-13T22:38:48.246 INFO:teuthology.orchestra.run.smithi102.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-13T22:38:48.246 INFO:teuthology.orchestra.run.smithi102.stdout:Adding key to root@localhost authorized_keys... 2024-05-13T22:38:48.246 INFO:teuthology.orchestra.run.smithi102.stdout:Adding host smithi102... 2024-05-13T22:38:48.871 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:48 smithi102 ceph-mon[28599]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:38:49.737 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:49 smithi102 ceph-mon[28599]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "addr": "172.21.15.102", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:38:51.268 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:51 smithi102 ceph-mon[28599]: Deploying cephadm binary to smithi102 2024-05-13T22:38:51.791 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Added host 'smithi102' with addr '172.21.15.102' 2024-05-13T22:38:51.791 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.690+0000 7f2d19ba0700 1 Processor -- start 2024-05-13T22:38:51.791 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.690+0000 7f2d19ba0700 1 -- start start 2024-05-13T22:38:51.791 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.690+0000 7f2d19ba0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d14105290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:51.791 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.690+0000 7f2d19ba0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d14079e10 con 0x7f2d14104e70 2024-05-13T22:38:51.791 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.690+0000 7f2d137fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d14105290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:51.791 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.690+0000 7f2d137fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d14105290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38394/0 (socket says 172.21.15.102:38394) 2024-05-13T22:38:51.791 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.690+0000 7f2d137fe700 1 -- 172.21.15.102:0/3886150805 learned_addr learned my addr 172.21.15.102:0/3886150805 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:51.791 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d137fe700 1 -- 172.21.15.102:0/3886150805 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d141057d0 con 0x7f2d14104e70 2024-05-13T22:38:51.791 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d137fe700 1 --2- 172.21.15.102:0/3886150805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d14105290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d080098a0 tx=0x7f2d08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434bbdd8360bb410 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d127fc700 1 -- 172.21.15.102:0/3886150805 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d08004180 con 0x7f2d14104e70 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d127fc700 1 -- 172.21.15.102:0/3886150805 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d080042e0 con 0x7f2d14104e70 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d127fc700 1 -- 172.21.15.102:0/3886150805 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d08011a50 con 0x7f2d14104e70 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d19ba0700 1 -- 172.21.15.102:0/3886150805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 msgr2=0x7f2d14105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d19ba0700 1 --2- 172.21.15.102:0/3886150805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d14105290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d080098a0 tx=0x7f2d08009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d19ba0700 1 -- 172.21.15.102:0/3886150805 shutdown_connections 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d19ba0700 1 --2- 172.21.15.102:0/3886150805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d14105290 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d19ba0700 1 -- 172.21.15.102:0/3886150805 >> 172.21.15.102:0/3886150805 conn(0x7f2d14100a50 msgr2=0x7f2d14102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d19ba0700 1 -- 172.21.15.102:0/3886150805 shutdown_connections 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.691+0000 7f2d19ba0700 1 -- 172.21.15.102:0/3886150805 wait complete. 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.692+0000 7f2d19ba0700 1 Processor -- start 2024-05-13T22:38:51.792 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.692+0000 7f2d19ba0700 1 -- start start 2024-05-13T22:38:51.793 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.692+0000 7f2d19ba0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d1419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:51.793 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.692+0000 7f2d19ba0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1419c980 con 0x7f2d14104e70 2024-05-13T22:38:51.793 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.692+0000 7f2d137fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d1419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:51.793 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.692+0000 7f2d137fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d1419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38406/0 (socket says 172.21.15.102:38406) 2024-05-13T22:38:51.793 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.692+0000 7f2d137fe700 1 -- 172.21.15.102:0/1667376991 learned_addr learned my addr 172.21.15.102:0/1667376991 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:51.793 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.693+0000 7f2d137fe700 1 -- 172.21.15.102:0/1667376991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08009550 con 0x7f2d14104e70 2024-05-13T22:38:51.793 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.693+0000 7f2d137fe700 1 --2- 172.21.15.102:0/1667376991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d1419c440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2d08003f80 tx=0x7f2d08011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:51.793 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.693+0000 7f2d10ff9700 1 -- 172.21.15.102:0/1667376991 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d08003a80 con 0x7f2d14104e70 2024-05-13T22:38:51.793 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.693+0000 7f2d10ff9700 1 -- 172.21.15.102:0/1667376991 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2d08003be0 con 0x7f2d14104e70 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.693+0000 7f2d19ba0700 1 -- 172.21.15.102:0/1667376991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1419cb80 con 0x7f2d14104e70 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.693+0000 7f2d10ff9700 1 -- 172.21.15.102:0/1667376991 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d08019820 con 0x7f2d14104e70 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.693+0000 7f2d19ba0700 1 -- 172.21.15.102:0/1667376991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1419d020 con 0x7f2d14104e70 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.694+0000 7f2d10ff9700 1 -- 172.21.15.102:0/1667376991 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f2d08019a70 con 0x7f2d14104e70 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.694+0000 7f2d19ba0700 1 -- 172.21.15.102:0/1667376991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d14195cf0 con 0x7f2d14104e70 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.694+0000 7f2d10ff9700 1 --2- 172.21.15.102:0/1667376991 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f2cfc038320 0x7f2cfc03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.694+0000 7f2d10ff9700 1 -- 172.21.15.102:0/1667376991 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2d0804c560 con 0x7f2d14104e70 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.694+0000 7f2d12ffd700 1 --2- 172.21.15.102:0/1667376991 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f2cfc038320 0x7f2cfc03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.695+0000 7f2d12ffd700 1 --2- 172.21.15.102:0/1667376991 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f2cfc038320 0x7f2cfc03a7e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d04006fd0 tx=0x7f2d04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.699+0000 7f2d10ff9700 1 -- 172.21.15.102:0/1667376991 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2d0802b750 con 0x7f2d14104e70 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:48.891+0000 7f2d19ba0700 1 -- 172.21.15.102:0/1667376991 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi102", "addr": "172.21.15.102", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d140611d0 con 0x7f2cfc038320 2024-05-13T22:38:51.794 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:51.245+0000 7f2d10ff9700 1 -- 172.21.15.102:0/1667376991 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f2d140611d0 con 0x7f2cfc038320 2024-05-13T22:38:51.795 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:51.247+0000 7f2d19ba0700 1 -- 172.21.15.102:0/1667376991 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f2cfc038320 msgr2=0x7f2cfc03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:51.795 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:51.247+0000 7f2d19ba0700 1 --2- 172.21.15.102:0/1667376991 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f2cfc038320 0x7f2cfc03a7e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d04006fd0 tx=0x7f2d04006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:51.795 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:51.247+0000 7f2d19ba0700 1 -- 172.21.15.102:0/1667376991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 msgr2=0x7f2d1419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:51.795 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:51.247+0000 7f2d19ba0700 1 --2- 172.21.15.102:0/1667376991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d1419c440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2d08003f80 tx=0x7f2d08011ab0 comp rx=0 tx=0).stop 2024-05-13T22:38:51.795 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:51.248+0000 7f2d19ba0700 1 -- 172.21.15.102:0/1667376991 shutdown_connections 2024-05-13T22:38:51.795 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:51.248+0000 7f2d19ba0700 1 --2- 172.21.15.102:0/1667376991 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f2cfc038320 0x7f2cfc03a7e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:51.795 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:51.248+0000 7f2d19ba0700 1 --2- 172.21.15.102:0/1667376991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d14104e70 0x7f2d1419c440 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:51.795 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:51.248+0000 7f2d19ba0700 1 -- 172.21.15.102:0/1667376991 >> 172.21.15.102:0/1667376991 conn(0x7f2d14100a50 msgr2=0x7f2d14190a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:51.795 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:51.248+0000 7f2d19ba0700 1 -- 172.21.15.102:0/1667376991 shutdown_connections 2024-05-13T22:38:51.795 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:51.248+0000 7f2d19ba0700 1 -- 172.21.15.102:0/1667376991 wait complete. 2024-05-13T22:38:51.795 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying mon service with default placement... 2024-05-13T22:38:52.323 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:52 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:52.324 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:52 smithi102 ceph-mon[28599]: Added host smithi102 2024-05-13T22:38:52.324 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:52 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:38:53.863 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:53 smithi102 ceph-mon[28599]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:38:53.863 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:53 smithi102 ceph-mon[28599]: Saving service mon spec with placement count:5 2024-05-13T22:38:53.863 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:53 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:53.863 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:53 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:53.870 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-13T22:38:53.870 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.557+0000 7f16f960d700 1 Processor -- start 2024-05-13T22:38:53.870 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.558+0000 7f16f960d700 1 -- start start 2024-05-13T22:38:53.870 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.558+0000 7f16f960d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f4104c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.558+0000 7f16f960d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f4105230 con 0x7f16f4104840 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.558+0000 7f16f2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f4104c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.558+0000 7f16f2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f4104c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38422/0 (socket says 172.21.15.102:38422) 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.558+0000 7f16f2ffd700 1 -- 172.21.15.102:0/3777948956 learned_addr learned my addr 172.21.15.102:0/3777948956 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.558+0000 7f16f2ffd700 1 -- 172.21.15.102:0/3777948956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f4105560 con 0x7f16f4104840 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.558+0000 7f16f2ffd700 1 --2- 172.21.15.102:0/3777948956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f4104c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f16e80098a0 tx=0x7f16e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db19303938f8ce5a server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.558+0000 7f16f1ffb700 1 -- 172.21.15.102:0/3777948956 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16e8004180 con 0x7f16f4104840 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.559+0000 7f16f1ffb700 1 -- 172.21.15.102:0/3777948956 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16e80042e0 con 0x7f16f4104840 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.559+0000 7f16f1ffb700 1 -- 172.21.15.102:0/3777948956 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16e8011a50 con 0x7f16f4104840 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.559+0000 7f16f960d700 1 -- 172.21.15.102:0/3777948956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 msgr2=0x7f16f4104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.559+0000 7f16f960d700 1 --2- 172.21.15.102:0/3777948956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f4104c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f16e80098a0 tx=0x7f16e8009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.559+0000 7f16f960d700 1 -- 172.21.15.102:0/3777948956 shutdown_connections 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.559+0000 7f16f960d700 1 --2- 172.21.15.102:0/3777948956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f4104c60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:53.871 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.559+0000 7f16f960d700 1 -- 172.21.15.102:0/3777948956 >> 172.21.15.102:0/3777948956 conn(0x7f16f4100190 msgr2=0x7f16f41025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.559+0000 7f16f960d700 1 -- 172.21.15.102:0/3777948956 shutdown_connections 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.559+0000 7f16f960d700 1 -- 172.21.15.102:0/3777948956 wait complete. 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.560+0000 7f16f960d700 1 Processor -- start 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.560+0000 7f16f960d700 1 -- start start 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.560+0000 7f16f960d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f41a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.560+0000 7f16f960d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f41a0d30 con 0x7f16f4104840 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.560+0000 7f16f2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f41a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.560+0000 7f16f2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f41a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38438/0 (socket says 172.21.15.102:38438) 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.560+0000 7f16f2ffd700 1 -- 172.21.15.102:0/2102605536 learned_addr learned my addr 172.21.15.102:0/2102605536 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.560+0000 7f16f2ffd700 1 -- 172.21.15.102:0/2102605536 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e8009550 con 0x7f16f4104840 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.560+0000 7f16f2ffd700 1 --2- 172.21.15.102:0/2102605536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f41a07f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009520 tx=0x7f16e8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.561+0000 7f16dbfff700 1 -- 172.21.15.102:0/2102605536 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16e8003a80 con 0x7f16f4104840 2024-05-13T22:38:53.872 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.561+0000 7f16dbfff700 1 -- 172.21.15.102:0/2102605536 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16e8003be0 con 0x7f16f4104840 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.561+0000 7f16f960d700 1 -- 172.21.15.102:0/2102605536 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f41a0f30 con 0x7f16f4104840 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.561+0000 7f16dbfff700 1 -- 172.21.15.102:0/2102605536 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16e80197b0 con 0x7f16f4104840 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.561+0000 7f16f960d700 1 -- 172.21.15.102:0/2102605536 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f41a13d0 con 0x7f16f4104840 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.562+0000 7f16dbfff700 1 -- 172.21.15.102:0/2102605536 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f16e8019a90 con 0x7f16f4104840 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.562+0000 7f16f960d700 1 -- 172.21.15.102:0/2102605536 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f419a8a0 con 0x7f16f4104840 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.562+0000 7f16dbfff700 1 --2- 172.21.15.102:0/2102605536 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f16dc0382d0 0x7f16dc03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.562+0000 7f16dbfff700 1 -- 172.21.15.102:0/2102605536 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f16e804c6d0 con 0x7f16f4104840 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.562+0000 7f16f27fc700 1 --2- 172.21.15.102:0/2102605536 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f16dc0382d0 0x7f16dc03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.562+0000 7f16f27fc700 1 --2- 172.21.15.102:0/2102605536 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f16dc0382d0 0x7f16dc03a790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006fd0 tx=0x7f16e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.567+0000 7f16dbfff700 1 -- 172.21.15.102:0/2102605536 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f16e8018e60 con 0x7f16f4104840 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.770+0000 7f16f960d700 1 -- 172.21.15.102:0/2102605536 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f16f40611d0 con 0x7f16dc0382d0 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.774+0000 7f16dbfff700 1 -- 172.21.15.102:0/2102605536 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f16f40611d0 con 0x7f16dc0382d0 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.776+0000 7f16f960d700 1 -- 172.21.15.102:0/2102605536 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f16dc0382d0 msgr2=0x7f16dc03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:53.873 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.776+0000 7f16f960d700 1 --2- 172.21.15.102:0/2102605536 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f16dc0382d0 0x7f16dc03a790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006fd0 tx=0x7f16e4006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:53.874 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.776+0000 7f16f960d700 1 -- 172.21.15.102:0/2102605536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 msgr2=0x7f16f41a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:53.874 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.776+0000 7f16f960d700 1 --2- 172.21.15.102:0/2102605536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f41a07f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009520 tx=0x7f16e8011ab0 comp rx=0 tx=0).stop 2024-05-13T22:38:53.874 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.776+0000 7f16f960d700 1 -- 172.21.15.102:0/2102605536 shutdown_connections 2024-05-13T22:38:53.874 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.776+0000 7f16f960d700 1 --2- 172.21.15.102:0/2102605536 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f16dc0382d0 0x7f16dc03a790 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:53.874 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.776+0000 7f16f960d700 1 --2- 172.21.15.102:0/2102605536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16f4104840 0x7f16f41a07f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:53.874 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.776+0000 7f16f960d700 1 -- 172.21.15.102:0/2102605536 >> 172.21.15.102:0/2102605536 conn(0x7f16f4100190 msgr2=0x7f16f410ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:53.874 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.776+0000 7f16f960d700 1 -- 172.21.15.102:0/2102605536 shutdown_connections 2024-05-13T22:38:53.874 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:52.776+0000 7f16f960d700 1 -- 172.21.15.102:0/2102605536 wait complete. 2024-05-13T22:38:53.874 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying mgr service with default placement... 2024-05-13T22:38:55.027 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-13T22:38:55.027 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.304+0000 7f634327c700 1 Processor -- start 2024-05-13T22:38:55.027 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.304+0000 7f634327c700 1 -- start start 2024-05-13T22:38:55.027 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.304+0000 7f634327c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c1093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:55.027 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.304+0000 7f634327c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c079e90 con 0x7f633c106fc0 2024-05-13T22:38:55.027 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.305+0000 7f6341018700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c1093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:55.027 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.305+0000 7f6341018700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c1093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38454/0 (socket says 172.21.15.102:38454) 2024-05-13T22:38:55.027 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.305+0000 7f6341018700 1 -- 172.21.15.102:0/2658350089 learned_addr learned my addr 172.21.15.102:0/2658350089 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.305+0000 7f6341018700 1 -- 172.21.15.102:0/2658350089 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c1098f0 con 0x7f633c106fc0 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.305+0000 7f6341018700 1 --2- 172.21.15.102:0/2658350089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c1093b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f632c0098a0 tx=0x7f632c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4633f53ce1abe72 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.305+0000 7f6333fff700 1 -- 172.21.15.102:0/2658350089 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f632c004180 con 0x7f633c106fc0 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.305+0000 7f6333fff700 1 -- 172.21.15.102:0/2658350089 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f632c0042e0 con 0x7f633c106fc0 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.305+0000 7f6333fff700 1 -- 172.21.15.102:0/2658350089 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f632c011af0 con 0x7f633c106fc0 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.306+0000 7f634327c700 1 -- 172.21.15.102:0/2658350089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 msgr2=0x7f633c1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.306+0000 7f634327c700 1 --2- 172.21.15.102:0/2658350089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c1093b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f632c0098a0 tx=0x7f632c009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.306+0000 7f634327c700 1 -- 172.21.15.102:0/2658350089 shutdown_connections 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.306+0000 7f634327c700 1 --2- 172.21.15.102:0/2658350089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c1093b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.306+0000 7f634327c700 1 -- 172.21.15.102:0/2658350089 >> 172.21.15.102:0/2658350089 conn(0x7f633c1009b0 msgr2=0x7f633c102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.306+0000 7f634327c700 1 -- 172.21.15.102:0/2658350089 shutdown_connections 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.306+0000 7f634327c700 1 -- 172.21.15.102:0/2658350089 wait complete. 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.307+0000 7f634327c700 1 Processor -- start 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.307+0000 7f634327c700 1 -- start start 2024-05-13T22:38:55.028 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.307+0000 7f634327c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.307+0000 7f634327c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c19a740 con 0x7f633c106fc0 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.307+0000 7f6341018700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.307+0000 7f6341018700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38466/0 (socket says 172.21.15.102:38466) 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.307+0000 7f6341018700 1 -- 172.21.15.102:0/2516765855 learned_addr learned my addr 172.21.15.102:0/2516765855 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.307+0000 7f6341018700 1 -- 172.21.15.102:0/2516765855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c009550 con 0x7f633c106fc0 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.307+0000 7f6341018700 1 --2- 172.21.15.102:0/2516765855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c19a200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f632c011670 tx=0x7f632c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.307+0000 7f63327fc700 1 -- 172.21.15.102:0/2516765855 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f632c003a40 con 0x7f633c106fc0 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.308+0000 7f634327c700 1 -- 172.21.15.102:0/2516765855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c19a940 con 0x7f633c106fc0 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.308+0000 7f63327fc700 1 -- 172.21.15.102:0/2516765855 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f632c003ba0 con 0x7f633c106fc0 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.308+0000 7f63327fc700 1 -- 172.21.15.102:0/2516765855 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f632c024820 con 0x7f633c106fc0 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.308+0000 7f634327c700 1 -- 172.21.15.102:0/2516765855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c19ade0 con 0x7f633c106fc0 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.308+0000 7f63327fc700 1 -- 172.21.15.102:0/2516765855 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f632c01a430 con 0x7f633c106fc0 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.309+0000 7f634327c700 1 -- 172.21.15.102:0/2516765855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c10a6e0 con 0x7f633c106fc0 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.309+0000 7f63327fc700 1 --2- 172.21.15.102:0/2516765855 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6328038330 0x7f632803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:55.029 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.309+0000 7f63327fc700 1 -- 172.21.15.102:0/2516765855 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f632c04c290 con 0x7f633c106fc0 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.309+0000 7f6340817700 1 --2- 172.21.15.102:0/2516765855 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6328038330 0x7f632803a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.309+0000 7f6340817700 1 --2- 172.21.15.102:0/2516765855 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6328038330 0x7f632803a7f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6338006fd0 tx=0x7f6338006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.313+0000 7f63327fc700 1 -- 172.21.15.102:0/2516765855 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f632c01a6e0 con 0x7f633c106fc0 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.524+0000 7f634327c700 1 -- 172.21.15.102:0/2516765855 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f633c101e90 con 0x7f6328038330 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.532+0000 7f63327fc700 1 -- 172.21.15.102:0/2516765855 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f633c101e90 con 0x7f6328038330 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.534+0000 7f634327c700 1 -- 172.21.15.102:0/2516765855 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6328038330 msgr2=0x7f632803a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.534+0000 7f634327c700 1 --2- 172.21.15.102:0/2516765855 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6328038330 0x7f632803a7f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6338006fd0 tx=0x7f6338006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.534+0000 7f634327c700 1 -- 172.21.15.102:0/2516765855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 msgr2=0x7f633c19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.534+0000 7f634327c700 1 --2- 172.21.15.102:0/2516765855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c19a200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f632c011670 tx=0x7f632c011aa0 comp rx=0 tx=0).stop 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.534+0000 7f634327c700 1 -- 172.21.15.102:0/2516765855 shutdown_connections 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.534+0000 7f634327c700 1 --2- 172.21.15.102:0/2516765855 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6328038330 0x7f632803a7f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.534+0000 7f634327c700 1 --2- 172.21.15.102:0/2516765855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f633c106fc0 0x7f633c19a200 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.534+0000 7f634327c700 1 -- 172.21.15.102:0/2516765855 >> 172.21.15.102:0/2516765855 conn(0x7f633c1009b0 msgr2=0x7f633c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:55.030 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.534+0000 7f634327c700 1 -- 172.21.15.102:0/2516765855 shutdown_connections 2024-05-13T22:38:55.031 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:54.534+0000 7f634327c700 1 -- 172.21.15.102:0/2516765855 wait complete. 2024-05-13T22:38:55.031 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying crash service with default placement... 2024-05-13T22:38:55.313 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:55 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:55.314 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:55 smithi102 ceph-mon[28599]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:38:55.314 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:55 smithi102 ceph-mon[28599]: Saving service mgr spec with placement count:2 2024-05-13T22:38:55.314 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:55 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:56.283 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-05-13T22:38:56.283 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.435+0000 7f4527f58700 1 Processor -- start 2024-05-13T22:38:56.283 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.435+0000 7f4527f58700 1 -- start start 2024-05-13T22:38:56.283 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.435+0000 7f4527f58700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f4520106950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:56.283 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.435+0000 7f4527f58700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4520106f20 con 0x7f4520106530 2024-05-13T22:38:56.283 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.435+0000 7f4525cf4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f4520106950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:56.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.435+0000 7f4525cf4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f4520106950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38474/0 (socket says 172.21.15.102:38474) 2024-05-13T22:38:56.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.435+0000 7f4525cf4700 1 -- 172.21.15.102:0/4218914915 learned_addr learned my addr 172.21.15.102:0/4218914915 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:56.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.435+0000 7f4525cf4700 1 -- 172.21.15.102:0/4218914915 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4520107730 con 0x7f4520106530 2024-05-13T22:38:56.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.436+0000 7f4525cf4700 1 --2- 172.21.15.102:0/4218914915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f4520106950 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ee220352401542 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:56.284 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.436+0000 7f4524cf2700 1 -- 172.21.15.102:0/4218914915 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f451c004180 con 0x7f4520106530 2024-05-13T22:38:56.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.436+0000 7f4524cf2700 1 -- 172.21.15.102:0/4218914915 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f451c0042e0 con 0x7f4520106530 2024-05-13T22:38:56.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.436+0000 7f4524cf2700 1 -- 172.21.15.102:0/4218914915 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f451c011af0 con 0x7f4520106530 2024-05-13T22:38:56.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.436+0000 7f4527f58700 1 -- 172.21.15.102:0/4218914915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 msgr2=0x7f4520106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:56.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.436+0000 7f4527f58700 1 --2- 172.21.15.102:0/4218914915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f4520106950 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:56.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.437+0000 7f4527f58700 1 -- 172.21.15.102:0/4218914915 shutdown_connections 2024-05-13T22:38:56.285 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.437+0000 7f4527f58700 1 --2- 172.21.15.102:0/4218914915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f4520106950 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:56.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.437+0000 7f4527f58700 1 -- 172.21.15.102:0/4218914915 >> 172.21.15.102:0/4218914915 conn(0x7f4520101ad0 msgr2=0x7f4520103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:56.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.437+0000 7f4527f58700 1 -- 172.21.15.102:0/4218914915 shutdown_connections 2024-05-13T22:38:56.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.437+0000 7f4527f58700 1 -- 172.21.15.102:0/4218914915 wait complete. 2024-05-13T22:38:56.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.437+0000 7f4527f58700 1 Processor -- start 2024-05-13T22:38:56.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.437+0000 7f4527f58700 1 -- start start 2024-05-13T22:38:56.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4527f58700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f452019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:56.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4527f58700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452019a790 con 0x7f4520106530 2024-05-13T22:38:56.286 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4525cf4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f452019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:56.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4525cf4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f452019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38488/0 (socket says 172.21.15.102:38488) 2024-05-13T22:38:56.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4525cf4700 1 -- 172.21.15.102:0/1325348834 learned_addr learned my addr 172.21.15.102:0/1325348834 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:56.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4525cf4700 1 -- 172.21.15.102:0/1325348834 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c009550 con 0x7f4520106530 2024-05-13T22:38:56.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4525cf4700 1 --2- 172.21.15.102:0/1325348834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f452019a250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f451c011670 tx=0x7f451c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:56.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4516ffd700 1 -- 172.21.15.102:0/1325348834 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f451c0039f0 con 0x7f4520106530 2024-05-13T22:38:56.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4516ffd700 1 -- 172.21.15.102:0/1325348834 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f451c003b50 con 0x7f4520106530 2024-05-13T22:38:56.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4527f58700 1 -- 172.21.15.102:0/1325348834 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452019a990 con 0x7f4520106530 2024-05-13T22:38:56.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4516ffd700 1 -- 172.21.15.102:0/1325348834 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f451c0247c0 con 0x7f4520106530 2024-05-13T22:38:56.287 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.438+0000 7f4527f58700 1 -- 172.21.15.102:0/1325348834 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452019ae30 con 0x7f4520106530 2024-05-13T22:38:56.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.439+0000 7f4527f58700 1 -- 172.21.15.102:0/1325348834 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45200623c0 con 0x7f4520106530 2024-05-13T22:38:56.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.439+0000 7f4516ffd700 1 -- 172.21.15.102:0/1325348834 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f451c02d3f0 con 0x7f4520106530 2024-05-13T22:38:56.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.440+0000 7f4516ffd700 1 --2- 172.21.15.102:0/1325348834 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f450c038370 0x7f450c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:56.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.440+0000 7f4516ffd700 1 -- 172.21.15.102:0/1325348834 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f451c04c340 con 0x7f4520106530 2024-05-13T22:38:56.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.440+0000 7f45254f3700 1 --2- 172.21.15.102:0/1325348834 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f450c038370 0x7f450c03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:56.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.440+0000 7f45254f3700 1 --2- 172.21.15.102:0/1325348834 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f450c038370 0x7f450c03a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4510006fd0 tx=0x7f4510006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:56.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.445+0000 7f4516ffd700 1 -- 172.21.15.102:0/1325348834 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f451c010970 con 0x7f4520106530 2024-05-13T22:38:56.288 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.650+0000 7f4527f58700 1 -- 172.21.15.102:0/1325348834 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f4520103dc0 con 0x7f450c038370 2024-05-13T22:38:56.289 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.657+0000 7f4516ffd700 1 -- 172.21.15.102:0/1325348834 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f4520103dc0 con 0x7f450c038370 2024-05-13T22:38:56.289 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.660+0000 7f4527f58700 1 -- 172.21.15.102:0/1325348834 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f450c038370 msgr2=0x7f450c03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:56.289 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.660+0000 7f4527f58700 1 --2- 172.21.15.102:0/1325348834 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f450c038370 0x7f450c03a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4510006fd0 tx=0x7f4510006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:56.289 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.660+0000 7f4527f58700 1 -- 172.21.15.102:0/1325348834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 msgr2=0x7f452019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:56.289 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.660+0000 7f4527f58700 1 --2- 172.21.15.102:0/1325348834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f452019a250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f451c011670 tx=0x7f451c011b50 comp rx=0 tx=0).stop 2024-05-13T22:38:56.289 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.660+0000 7f4527f58700 1 -- 172.21.15.102:0/1325348834 shutdown_connections 2024-05-13T22:38:56.289 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.660+0000 7f4527f58700 1 --2- 172.21.15.102:0/1325348834 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f450c038370 0x7f450c03a830 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:56.290 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.660+0000 7f4527f58700 1 --2- 172.21.15.102:0/1325348834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4520106530 0x7f452019a250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:56.290 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.660+0000 7f4527f58700 1 -- 172.21.15.102:0/1325348834 >> 172.21.15.102:0/1325348834 conn(0x7f4520101ad0 msgr2=0x7f4520103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:56.290 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.660+0000 7f4527f58700 1 -- 172.21.15.102:0/1325348834 shutdown_connections 2024-05-13T22:38:56.290 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:55.660+0000 7f4527f58700 1 -- 172.21.15.102:0/1325348834 wait complete. 2024-05-13T22:38:56.290 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying ceph-exporter service with default placement... 2024-05-13T22:38:56.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:56 smithi102 ceph-mon[28599]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:38:56.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:56 smithi102 ceph-mon[28599]: Saving service crash spec with placement * 2024-05-13T22:38:56.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:56 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:57.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-05-13T22:38:57.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.038+0000 7f2819448700 1 Processor -- start 2024-05-13T22:38:57.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.038+0000 7f2819448700 1 -- start start 2024-05-13T22:38:57.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.039+0000 7f2819448700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814105850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:57.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.039+0000 7f2819448700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814105e20 con 0x7f2814105430 2024-05-13T22:38:57.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.039+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814105850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.039+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814105850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38504/0 (socket says 172.21.15.102:38504) 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.039+0000 7f2812ffd700 1 -- 172.21.15.102:0/1389084708 learned_addr learned my addr 172.21.15.102:0/1389084708 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.039+0000 7f2812ffd700 1 -- 172.21.15.102:0/1389084708 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2814106630 con 0x7f2814105430 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.039+0000 7f2812ffd700 1 --2- 172.21.15.102:0/1389084708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814105850 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f28040098a0 tx=0x7f2804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a27a955db1f57ef server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.039+0000 7f2811ffb700 1 -- 172.21.15.102:0/1389084708 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2804004180 con 0x7f2814105430 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.039+0000 7f2811ffb700 1 -- 172.21.15.102:0/1389084708 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f28040042e0 con 0x7f2814105430 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.039+0000 7f2811ffb700 1 -- 172.21.15.102:0/1389084708 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2804011af0 con 0x7f2814105430 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.040+0000 7f2819448700 1 -- 172.21.15.102:0/1389084708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 msgr2=0x7f2814105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.040+0000 7f2819448700 1 --2- 172.21.15.102:0/1389084708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814105850 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f28040098a0 tx=0x7f2804009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.040+0000 7f2819448700 1 -- 172.21.15.102:0/1389084708 shutdown_connections 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.040+0000 7f2819448700 1 --2- 172.21.15.102:0/1389084708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814105850 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.040+0000 7f2819448700 1 -- 172.21.15.102:0/1389084708 >> 172.21.15.102:0/1389084708 conn(0x7f28141009b0 msgr2=0x7f2814102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.040+0000 7f2819448700 1 -- 172.21.15.102:0/1389084708 shutdown_connections 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.040+0000 7f2819448700 1 -- 172.21.15.102:0/1389084708 wait complete. 2024-05-13T22:38:57.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.041+0000 7f2819448700 1 Processor -- start 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.041+0000 7f2819448700 1 -- start start 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.041+0000 7f2819448700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814079ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.041+0000 7f2819448700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281419cf80 con 0x7f2814105430 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.041+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814079ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.041+0000 7f2812ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814079ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38516/0 (socket says 172.21.15.102:38516) 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.041+0000 7f2812ffd700 1 -- 172.21.15.102:0/3792583128 learned_addr learned my addr 172.21.15.102:0/3792583128 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.041+0000 7f2812ffd700 1 -- 172.21.15.102:0/3792583128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2804009550 con 0x7f2814105430 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.041+0000 7f2812ffd700 1 --2- 172.21.15.102:0/3792583128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814079ae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2804006ba0 tx=0x7f2804011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.042+0000 7f280bfff700 1 -- 172.21.15.102:0/3792583128 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2804003b00 con 0x7f2814105430 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.042+0000 7f2819448700 1 -- 172.21.15.102:0/3792583128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281419d0c0 con 0x7f2814105430 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.042+0000 7f280bfff700 1 -- 172.21.15.102:0/3792583128 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f280401a8e0 con 0x7f2814105430 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.042+0000 7f280bfff700 1 -- 172.21.15.102:0/3792583128 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2804019990 con 0x7f2814105430 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.042+0000 7f2819448700 1 -- 172.21.15.102:0/3792583128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281419d500 con 0x7f2814105430 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.042+0000 7f280bfff700 1 -- 172.21.15.102:0/3792583128 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f2804019c50 con 0x7f2814105430 2024-05-13T22:38:57.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.043+0000 7f280bfff700 1 --2- 172.21.15.102:0/3792583128 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f28000382d0 0x7f280003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.043+0000 7f280bfff700 1 -- 172.21.15.102:0/3792583128 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f280404c5a0 con 0x7f2814105430 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.043+0000 7f2819448700 1 -- 172.21.15.102:0/3792583128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28140623c0 con 0x7f2814105430 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.043+0000 7f28127fc700 1 --2- 172.21.15.102:0/3792583128 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f28000382d0 0x7f280003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.043+0000 7f28127fc700 1 --2- 172.21.15.102:0/3792583128 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f28000382d0 0x7f280003a790 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f27fc006fd0 tx=0x7f27fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.048+0000 7f280bfff700 1 -- 172.21.15.102:0/3792583128 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f280402d440 con 0x7f2814105430 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.265+0000 7f2819448700 1 -- 172.21.15.102:0/3792583128 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f2814109830 con 0x7f28000382d0 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.273+0000 7f280bfff700 1 -- 172.21.15.102:0/3792583128 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f2814109830 con 0x7f28000382d0 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.275+0000 7f2819448700 1 -- 172.21.15.102:0/3792583128 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f28000382d0 msgr2=0x7f280003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.275+0000 7f2819448700 1 --2- 172.21.15.102:0/3792583128 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f28000382d0 0x7f280003a790 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f27fc006fd0 tx=0x7f27fc006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.275+0000 7f2819448700 1 -- 172.21.15.102:0/3792583128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 msgr2=0x7f2814079ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.275+0000 7f2819448700 1 --2- 172.21.15.102:0/3792583128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814079ae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2804006ba0 tx=0x7f2804011b50 comp rx=0 tx=0).stop 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.275+0000 7f2819448700 1 -- 172.21.15.102:0/3792583128 shutdown_connections 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.275+0000 7f2819448700 1 --2- 172.21.15.102:0/3792583128 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f28000382d0 0x7f280003a790 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:57.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.275+0000 7f2819448700 1 --2- 172.21.15.102:0/3792583128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2814105430 0x7f2814079ae0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:57.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.275+0000 7f2819448700 1 -- 172.21.15.102:0/3792583128 >> 172.21.15.102:0/3792583128 conn(0x7f28141009b0 msgr2=0x7f2814102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:57.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.275+0000 7f2819448700 1 -- 172.21.15.102:0/3792583128 shutdown_connections 2024-05-13T22:38:57.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:57.275+0000 7f2819448700 1 -- 172.21.15.102:0/3792583128 wait complete. 2024-05-13T22:38:57.906 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying prometheus service with default placement... 2024-05-13T22:38:58.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:58 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:58.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:58 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:59.129 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:59 smithi102 ceph-mon[28599]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:38:59.129 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:59 smithi102 ceph-mon[28599]: Saving service ceph-exporter spec with placement * 2024-05-13T22:38:59.130 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:59 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:59.130 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:38:59 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:38:59.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-05-13T22:38:59.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.364+0000 7f56039dc700 1 Processor -- start 2024-05-13T22:38:59.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.364+0000 7f56039dc700 1 -- start start 2024-05-13T22:38:59.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.364+0000 7f56039dc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc109490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:59.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.364+0000 7f56039dc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc079db0 con 0x7f55fc1070a0 2024-05-13T22:38:59.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.365+0000 7f5601778700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc109490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:59.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.365+0000 7f5601778700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc109490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38530/0 (socket says 172.21.15.102:38530) 2024-05-13T22:38:59.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.365+0000 7f5601778700 1 -- 172.21.15.102:0/1593791569 learned_addr learned my addr 172.21.15.102:0/1593791569 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:59.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.365+0000 7f5601778700 1 -- 172.21.15.102:0/1593791569 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc079ef0 con 0x7f55fc1070a0 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.365+0000 7f5601778700 1 --2- 172.21.15.102:0/1593791569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc109490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55f80098a0 tx=0x7f55f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cd29195757475a server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.365+0000 7f55f3fff700 1 -- 172.21.15.102:0/1593791569 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55f8004180 con 0x7f55fc1070a0 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.365+0000 7f55f3fff700 1 -- 172.21.15.102:0/1593791569 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f55f80042e0 con 0x7f55fc1070a0 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.365+0000 7f55f3fff700 1 -- 172.21.15.102:0/1593791569 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55f8011af0 con 0x7f55fc1070a0 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.366+0000 7f56039dc700 1 -- 172.21.15.102:0/1593791569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 msgr2=0x7f55fc109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.366+0000 7f56039dc700 1 --2- 172.21.15.102:0/1593791569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc109490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55f80098a0 tx=0x7f55f8009bb0 comp rx=0 tx=0).stop 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.366+0000 7f56039dc700 1 -- 172.21.15.102:0/1593791569 shutdown_connections 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.366+0000 7f56039dc700 1 --2- 172.21.15.102:0/1593791569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc109490 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.366+0000 7f56039dc700 1 -- 172.21.15.102:0/1593791569 >> 172.21.15.102:0/1593791569 conn(0x7f55fc1009b0 msgr2=0x7f55fc102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.366+0000 7f56039dc700 1 -- 172.21.15.102:0/1593791569 shutdown_connections 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.366+0000 7f56039dc700 1 -- 172.21.15.102:0/1593791569 wait complete. 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f56039dc700 1 Processor -- start 2024-05-13T22:38:59.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f56039dc700 1 -- start start 2024-05-13T22:38:59.640 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f56039dc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:59.640 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f56039dc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc19a770 con 0x7f55fc1070a0 2024-05-13T22:38:59.641 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f5601778700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:59.642 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f5601778700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38532/0 (socket says 172.21.15.102:38532) 2024-05-13T22:38:59.642 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f5601778700 1 -- 172.21.15.102:0/3686660945 learned_addr learned my addr 172.21.15.102:0/3686660945 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:38:59.642 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f5601778700 1 -- 172.21.15.102:0/3686660945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f8009550 con 0x7f55fc1070a0 2024-05-13T22:38:59.642 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f5601778700 1 --2- 172.21.15.102:0/3686660945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc19a230 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55f8011670 tx=0x7f55f8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:59.642 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f55f27fc700 1 -- 172.21.15.102:0/3686660945 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55f8003a40 con 0x7f55fc1070a0 2024-05-13T22:38:59.642 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f55f27fc700 1 -- 172.21.15.102:0/3686660945 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f55f8003ba0 con 0x7f55fc1070a0 2024-05-13T22:38:59.642 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.367+0000 7f56039dc700 1 -- 172.21.15.102:0/3686660945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55fc19a970 con 0x7f55fc1070a0 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.368+0000 7f56039dc700 1 -- 172.21.15.102:0/3686660945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55fc19ae10 con 0x7f55fc1070a0 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.368+0000 7f55f27fc700 1 -- 172.21.15.102:0/3686660945 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55f8024820 con 0x7f55fc1070a0 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.368+0000 7f55f27fc700 1 -- 172.21.15.102:0/3686660945 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f55f8024ae0 con 0x7f55fc1070a0 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.368+0000 7f56039dc700 1 -- 172.21.15.102:0/3686660945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55fc10a6f0 con 0x7f55fc1070a0 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.369+0000 7f55f27fc700 1 --2- 172.21.15.102:0/3686660945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f55e80382e0 0x7f55e803a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.369+0000 7f55f27fc700 1 -- 172.21.15.102:0/3686660945 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f55f801a430 con 0x7f55fc1070a0 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.369+0000 7f5600f77700 1 --2- 172.21.15.102:0/3686660945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f55e80382e0 0x7f55e803a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.369+0000 7f5600f77700 1 --2- 172.21.15.102:0/3686660945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f55e80382e0 0x7f55e803a7a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f55ec006fd0 tx=0x7f55ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.373+0000 7f55f27fc700 1 -- 172.21.15.102:0/3686660945 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f55f8022310 con 0x7f55fc1070a0 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.602+0000 7f56039dc700 1 -- 172.21.15.102:0/3686660945 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f55fc101ec0 con 0x7f55e80382e0 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.607+0000 7f55f27fc700 1 -- 172.21.15.102:0/3686660945 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f55fc101ec0 con 0x7f55e80382e0 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.609+0000 7f56039dc700 1 -- 172.21.15.102:0/3686660945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f55e80382e0 msgr2=0x7f55e803a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.609+0000 7f56039dc700 1 --2- 172.21.15.102:0/3686660945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f55e80382e0 0x7f55e803a7a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f55ec006fd0 tx=0x7f55ec006e40 comp rx=0 tx=0).stop 2024-05-13T22:38:59.643 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.609+0000 7f56039dc700 1 -- 172.21.15.102:0/3686660945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 msgr2=0x7f55fc19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:38:59.644 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.609+0000 7f56039dc700 1 --2- 172.21.15.102:0/3686660945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc19a230 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55f8011670 tx=0x7f55f8011aa0 comp rx=0 tx=0).stop 2024-05-13T22:38:59.644 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.609+0000 7f56039dc700 1 -- 172.21.15.102:0/3686660945 shutdown_connections 2024-05-13T22:38:59.644 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.609+0000 7f56039dc700 1 --2- 172.21.15.102:0/3686660945 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f55e80382e0 0x7f55e803a7a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:59.644 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.609+0000 7f56039dc700 1 --2- 172.21.15.102:0/3686660945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55fc1070a0 0x7f55fc19a230 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:38:59.644 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.609+0000 7f56039dc700 1 -- 172.21.15.102:0/3686660945 >> 172.21.15.102:0/3686660945 conn(0x7f55fc1009b0 msgr2=0x7f55fc1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:38:59.644 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.609+0000 7f56039dc700 1 -- 172.21.15.102:0/3686660945 shutdown_connections 2024-05-13T22:38:59.644 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:38:58.609+0000 7f56039dc700 1 -- 172.21.15.102:0/3686660945 wait complete. 2024-05-13T22:38:59.644 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying grafana service with default placement... 2024-05-13T22:39:00.295 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:00 smithi102 ceph-mon[28599]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:39:00.296 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:00 smithi102 ceph-mon[28599]: Saving service prometheus spec with placement count:1 2024-05-13T22:39:00.913 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-05-13T22:39:00.913 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.099+0000 7f6a72274700 1 Processor -- start 2024-05-13T22:39:00.913 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.099+0000 7f6a72274700 1 -- start start 2024-05-13T22:39:00.913 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.099+0000 7f6a72274700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c105d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.099+0000 7f6a72274700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c106340 con 0x7f6a6c105950 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.099+0000 7f6a6bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c105d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.099+0000 7f6a6bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c105d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53330/0 (socket says 172.21.15.102:53330) 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.099+0000 7f6a6bfff700 1 -- 172.21.15.102:0/665958252 learned_addr learned my addr 172.21.15.102:0/665958252 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.099+0000 7f6a6bfff700 1 -- 172.21.15.102:0/665958252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c106670 con 0x7f6a6c105950 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.100+0000 7f6a6bfff700 1 --2- 172.21.15.102:0/665958252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c105d70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0098a0 tx=0x7f6a5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aeac2fbc9a5aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.100+0000 7f6a6affd700 1 -- 172.21.15.102:0/665958252 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a5c004180 con 0x7f6a6c105950 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.100+0000 7f6a6affd700 1 -- 172.21.15.102:0/665958252 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a5c0042e0 con 0x7f6a6c105950 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.100+0000 7f6a6affd700 1 -- 172.21.15.102:0/665958252 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a5c011af0 con 0x7f6a6c105950 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.100+0000 7f6a72274700 1 -- 172.21.15.102:0/665958252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 msgr2=0x7f6a6c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.100+0000 7f6a72274700 1 --2- 172.21.15.102:0/665958252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c105d70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0098a0 tx=0x7f6a5c009bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.100+0000 7f6a72274700 1 -- 172.21.15.102:0/665958252 shutdown_connections 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.100+0000 7f6a72274700 1 --2- 172.21.15.102:0/665958252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c105d70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.100+0000 7f6a72274700 1 -- 172.21.15.102:0/665958252 >> 172.21.15.102:0/665958252 conn(0x7f6a6c100190 msgr2=0x7f6a6c1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:00.914 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.100+0000 7f6a72274700 1 -- 172.21.15.102:0/665958252 shutdown_connections 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.100+0000 7f6a72274700 1 -- 172.21.15.102:0/665958252 wait complete. 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.101+0000 7f6a72274700 1 Processor -- start 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.101+0000 7f6a72274700 1 -- start start 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.101+0000 7f6a72274700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.101+0000 7f6a72274700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c19a760 con 0x7f6a6c105950 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.102+0000 7f6a6bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.102+0000 7f6a6bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c19a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53336/0 (socket says 172.21.15.102:53336) 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.102+0000 7f6a6bfff700 1 -- 172.21.15.102:0/31885651 learned_addr learned my addr 172.21.15.102:0/31885651 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.102+0000 7f6a6bfff700 1 -- 172.21.15.102:0/31885651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c009550 con 0x7f6a6c105950 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.102+0000 7f6a6bfff700 1 --2- 172.21.15.102:0/31885651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c19a220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c011670 tx=0x7f6a5c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.102+0000 7f6a697fa700 1 -- 172.21.15.102:0/31885651 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a5c003a40 con 0x7f6a6c105950 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.102+0000 7f6a697fa700 1 -- 172.21.15.102:0/31885651 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a5c003ba0 con 0x7f6a6c105950 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.102+0000 7f6a72274700 1 -- 172.21.15.102:0/31885651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c19a960 con 0x7f6a6c105950 2024-05-13T22:39:00.915 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.102+0000 7f6a697fa700 1 -- 172.21.15.102:0/31885651 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a5c024790 con 0x7f6a6c105950 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.102+0000 7f6a72274700 1 -- 172.21.15.102:0/31885651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c19ae00 con 0x7f6a6c105950 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.103+0000 7f6a697fa700 1 -- 172.21.15.102:0/31885651 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f6a5c02d3f0 con 0x7f6a6c105950 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.103+0000 7f6a72274700 1 -- 172.21.15.102:0/31885651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c0623c0 con 0x7f6a6c105950 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.103+0000 7f6a697fa700 1 --2- 172.21.15.102:0/31885651 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6a54038320 0x7f6a5403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.103+0000 7f6a697fa700 1 -- 172.21.15.102:0/31885651 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6a5c04c550 con 0x7f6a6c105950 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.103+0000 7f6a6b7fe700 1 --2- 172.21.15.102:0/31885651 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6a54038320 0x7f6a5403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.104+0000 7f6a6b7fe700 1 --2- 172.21.15.102:0/31885651 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6a54038320 0x7f6a5403a7e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6a60006fd0 tx=0x7f6a60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.108+0000 7f6a697fa700 1 -- 172.21.15.102:0/31885651 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6a5c010970 con 0x7f6a6c105950 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.308+0000 7f6a72274700 1 -- 172.21.15.102:0/31885651 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a6c1022c0 con 0x7f6a54038320 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.314+0000 7f6a697fa700 1 -- 172.21.15.102:0/31885651 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f6a6c1022c0 con 0x7f6a54038320 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.316+0000 7f6a72274700 1 -- 172.21.15.102:0/31885651 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6a54038320 msgr2=0x7f6a5403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.316+0000 7f6a72274700 1 --2- 172.21.15.102:0/31885651 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6a54038320 0x7f6a5403a7e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6a60006fd0 tx=0x7f6a60006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.316+0000 7f6a72274700 1 -- 172.21.15.102:0/31885651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 msgr2=0x7f6a6c19a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.316+0000 7f6a72274700 1 --2- 172.21.15.102:0/31885651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c19a220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c011670 tx=0x7f6a5c011b50 comp rx=0 tx=0).stop 2024-05-13T22:39:00.916 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.316+0000 7f6a72274700 1 -- 172.21.15.102:0/31885651 shutdown_connections 2024-05-13T22:39:00.917 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.316+0000 7f6a72274700 1 --2- 172.21.15.102:0/31885651 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f6a54038320 0x7f6a5403a7e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:00.917 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.316+0000 7f6a72274700 1 --2- 172.21.15.102:0/31885651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a6c105950 0x7f6a6c19a220 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:00.917 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.316+0000 7f6a72274700 1 -- 172.21.15.102:0/31885651 >> 172.21.15.102:0/31885651 conn(0x7f6a6c100190 msgr2=0x7f6a6c104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:00.917 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.316+0000 7f6a72274700 1 -- 172.21.15.102:0/31885651 shutdown_connections 2024-05-13T22:39:00.917 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:00.316+0000 7f6a72274700 1 -- 172.21.15.102:0/31885651 wait complete. 2024-05-13T22:39:00.917 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying node-exporter service with default placement... 2024-05-13T22:39:01.517 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:01 smithi102 ceph-mon[28599]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:39:01.517 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:01 smithi102 ceph-mon[28599]: Saving service grafana spec with placement count:1 2024-05-13T22:39:01.517 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:01 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:02.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-05-13T22:39:02.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.330+0000 7f759bf70700 1 Processor -- start 2024-05-13T22:39:02.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.330+0000 7f759bf70700 1 -- start start 2024-05-13T22:39:02.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.330+0000 7f759bf70700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594105880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:02.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.330+0000 7f759bf70700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7594105e50 con 0x7f7594105460 2024-05-13T22:39:02.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.330+0000 7f7599d0c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594105880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:02.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.330+0000 7f7599d0c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594105880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53352/0 (socket says 172.21.15.102:53352) 2024-05-13T22:39:02.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.330+0000 7f7599d0c700 1 -- 172.21.15.102:0/1476758989 learned_addr learned my addr 172.21.15.102:0/1476758989 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:02.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.331+0000 7f7599d0c700 1 -- 172.21.15.102:0/1476758989 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7594106660 con 0x7f7594105460 2024-05-13T22:39:02.079 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.331+0000 7f7599d0c700 1 --2- 172.21.15.102:0/1476758989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594105880 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7584009e80 tx=0x7f758400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcef8ca6723b4ebe server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.331+0000 7f7598d0a700 1 -- 172.21.15.102:0/1476758989 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f758400e070 con 0x7f7594105460 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.331+0000 7f7598d0a700 1 -- 172.21.15.102:0/1476758989 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f758400b8c0 con 0x7f7594105460 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.331+0000 7f7598d0a700 1 -- 172.21.15.102:0/1476758989 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f75840044d0 con 0x7f7594105460 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.332+0000 7f759bf70700 1 -- 172.21.15.102:0/1476758989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 msgr2=0x7f7594105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.332+0000 7f759bf70700 1 --2- 172.21.15.102:0/1476758989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594105880 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7584009e80 tx=0x7f758400b330 comp rx=0 tx=0).stop 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.332+0000 7f759bf70700 1 -- 172.21.15.102:0/1476758989 shutdown_connections 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.332+0000 7f759bf70700 1 --2- 172.21.15.102:0/1476758989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594105880 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.332+0000 7f759bf70700 1 -- 172.21.15.102:0/1476758989 >> 172.21.15.102:0/1476758989 conn(0x7f7594100a20 msgr2=0x7f7594102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.332+0000 7f759bf70700 1 -- 172.21.15.102:0/1476758989 shutdown_connections 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.332+0000 7f759bf70700 1 -- 172.21.15.102:0/1476758989 wait complete. 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.332+0000 7f759bf70700 1 Processor -- start 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.332+0000 7f759bf70700 1 -- start start 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.332+0000 7f759bf70700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:02.080 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.332+0000 7f759bf70700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7594199640 con 0x7f7594105460 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.333+0000 7f7599d0c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.333+0000 7f7599d0c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53360/0 (socket says 172.21.15.102:53360) 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.333+0000 7f7599d0c700 1 -- 172.21.15.102:0/379129411 learned_addr learned my addr 172.21.15.102:0/379129411 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.333+0000 7f7599d0c700 1 -- 172.21.15.102:0/379129411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584009b30 con 0x7f7594105460 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.333+0000 7f7599d0c700 1 --2- 172.21.15.102:0/379129411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594199100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7584004940 tx=0x7f7584004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.333+0000 7f7592ffd700 1 -- 172.21.15.102:0/379129411 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f758400e040 con 0x7f7594105460 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.333+0000 7f7592ffd700 1 -- 172.21.15.102:0/379129411 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f758400bc40 con 0x7f7594105460 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.333+0000 7f7592ffd700 1 -- 172.21.15.102:0/379129411 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7584019660 con 0x7f7594105460 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.333+0000 7f759bf70700 1 -- 172.21.15.102:0/379129411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7594199840 con 0x7f7594105460 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.333+0000 7f759bf70700 1 -- 172.21.15.102:0/379129411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7594199d40 con 0x7f7594105460 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.334+0000 7f7592ffd700 1 -- 172.21.15.102:0/379129411 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f7584012b00 con 0x7f7594105460 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.334+0000 7f759bf70700 1 -- 172.21.15.102:0/379129411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75940623c0 con 0x7f7594105460 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.334+0000 7f7592ffd700 1 --2- 172.21.15.102:0/379129411 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f757c038320 0x7f757c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:02.081 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.334+0000 7f7592ffd700 1 -- 172.21.15.102:0/379129411 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f758404d520 con 0x7f7594105460 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.334+0000 7f759950b700 1 --2- 172.21.15.102:0/379129411 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f757c038320 0x7f757c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.335+0000 7f759950b700 1 --2- 172.21.15.102:0/379129411 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f757c038320 0x7f757c03a7e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7588006fd0 tx=0x7f7588006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.339+0000 7f7592ffd700 1 -- 172.21.15.102:0/379129411 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7584020c90 con 0x7f7594105460 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.537+0000 7f759bf70700 1 -- 172.21.15.102:0/379129411 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f7594079900 con 0x7f757c038320 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.542+0000 7f7592ffd700 1 -- 172.21.15.102:0/379129411 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f7594079900 con 0x7f757c038320 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.544+0000 7f759bf70700 1 -- 172.21.15.102:0/379129411 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f757c038320 msgr2=0x7f757c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.544+0000 7f759bf70700 1 --2- 172.21.15.102:0/379129411 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f757c038320 0x7f757c03a7e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7588006fd0 tx=0x7f7588006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.544+0000 7f759bf70700 1 -- 172.21.15.102:0/379129411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 msgr2=0x7f7594199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.544+0000 7f759bf70700 1 --2- 172.21.15.102:0/379129411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594199100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7584004940 tx=0x7f7584004a20 comp rx=0 tx=0).stop 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.544+0000 7f759bf70700 1 -- 172.21.15.102:0/379129411 shutdown_connections 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.544+0000 7f759bf70700 1 --2- 172.21.15.102:0/379129411 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f757c038320 0x7f757c03a7e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.544+0000 7f759bf70700 1 --2- 172.21.15.102:0/379129411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7594105460 0x7f7594199100 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.544+0000 7f759bf70700 1 -- 172.21.15.102:0/379129411 >> 172.21.15.102:0/379129411 conn(0x7f7594100a20 msgr2=0x7f7594101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:02.082 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.544+0000 7f759bf70700 1 -- 172.21.15.102:0/379129411 shutdown_connections 2024-05-13T22:39:02.083 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:01.544+0000 7f759bf70700 1 -- 172.21.15.102:0/379129411 wait complete. 2024-05-13T22:39:02.083 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying alertmanager service with default placement... 2024-05-13T22:39:02.730 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:02 smithi102 ceph-mon[28599]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:39:02.730 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:02 smithi102 ceph-mon[28599]: Saving service node-exporter spec with placement * 2024-05-13T22:39:02.730 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:02 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:02.730 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:02 smithi102 ceph-mon[28599]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:03.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-05-13T22:39:03.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.520+0000 7fcd8b0b5700 1 Processor -- start 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.520+0000 7fcd8b0b5700 1 -- start start 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.520+0000 7fcd8b0b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.520+0000 7fcd8b0b5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd84106350 con 0x7fcd84105960 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.520+0000 7fcd88e51700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.520+0000 7fcd88e51700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53376/0 (socket says 172.21.15.102:53376) 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.520+0000 7fcd88e51700 1 -- 172.21.15.102:0/2658652171 learned_addr learned my addr 172.21.15.102:0/2658652171 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.520+0000 7fcd88e51700 1 -- 172.21.15.102:0/2658652171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd84106680 con 0x7fcd84105960 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.520+0000 7fcd88e51700 1 --2- 172.21.15.102:0/2658652171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd780098a0 tx=0x7fcd78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf44e8898ceca747 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.521+0000 7fcd837fe700 1 -- 172.21.15.102:0/2658652171 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd78004180 con 0x7fcd84105960 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.521+0000 7fcd837fe700 1 -- 172.21.15.102:0/2658652171 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcd780042e0 con 0x7fcd84105960 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.521+0000 7fcd837fe700 1 -- 172.21.15.102:0/2658652171 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd78011af0 con 0x7fcd84105960 2024-05-13T22:39:03.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.521+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/2658652171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 msgr2=0x7fcd84105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.521+0000 7fcd8b0b5700 1 --2- 172.21.15.102:0/2658652171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd780098a0 tx=0x7fcd78009bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.521+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/2658652171 shutdown_connections 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.521+0000 7fcd8b0b5700 1 --2- 172.21.15.102:0/2658652171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.521+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/2658652171 >> 172.21.15.102:0/2658652171 conn(0x7fcd841001a0 msgr2=0x7fcd841025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.521+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/2658652171 shutdown_connections 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.521+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/2658652171 wait complete. 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.522+0000 7fcd8b0b5700 1 Processor -- start 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.522+0000 7fcd8b0b5700 1 -- start start 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.522+0000 7fcd8b0b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.522+0000 7fcd8b0b5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8419cf30 con 0x7fcd84105960 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.522+0000 7fcd88e51700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.522+0000 7fcd88e51700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53392/0 (socket says 172.21.15.102:53392) 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.522+0000 7fcd88e51700 1 -- 172.21.15.102:0/1635886335 learned_addr learned my addr 172.21.15.102:0/1635886335 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:03.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.522+0000 7fcd88e51700 1 -- 172.21.15.102:0/1635886335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd78009550 con 0x7fcd84105960 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.523+0000 7fcd88e51700 1 --2- 172.21.15.102:0/1635886335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd78011670 tx=0x7fcd78011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.523+0000 7fcd81ffb700 1 -- 172.21.15.102:0/1635886335 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd78003a40 con 0x7fcd84105960 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.523+0000 7fcd81ffb700 1 -- 172.21.15.102:0/1635886335 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcd78003ba0 con 0x7fcd84105960 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.523+0000 7fcd81ffb700 1 -- 172.21.15.102:0/1635886335 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd78024790 con 0x7fcd84105960 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.523+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/1635886335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8419d070 con 0x7fcd84105960 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.523+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/1635886335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8419d510 con 0x7fcd84105960 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.524+0000 7fcd81ffb700 1 -- 172.21.15.102:0/1635886335 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fcd7801a430 con 0x7fcd84105960 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.524+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/1635886335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd840623c0 con 0x7fcd84105960 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.524+0000 7fcd81ffb700 1 --2- 172.21.15.102:0/1635886335 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fcd74038320 0x7fcd7403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.524+0000 7fcd81ffb700 1 -- 172.21.15.102:0/1635886335 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcd7804c540 con 0x7fcd84105960 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.524+0000 7fcd83fff700 1 --2- 172.21.15.102:0/1635886335 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fcd74038320 0x7fcd7403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.524+0000 7fcd83fff700 1 --2- 172.21.15.102:0/1635886335 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fcd74038320 0x7fcd7403a7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd70006fd0 tx=0x7fcd70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.529+0000 7fcd81ffb700 1 -- 172.21.15.102:0/1635886335 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcd78010380 con 0x7fcd84105960 2024-05-13T22:39:03.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.732+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/1635886335 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd841015a0 con 0x7fcd74038320 2024-05-13T22:39:03.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.737+0000 7fcd81ffb700 1 -- 172.21.15.102:0/1635886335 <== mgr.14120 v2:172.21.15.102:6800/358342393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fcd841015a0 con 0x7fcd74038320 2024-05-13T22:39:03.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.739+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/1635886335 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fcd74038320 msgr2=0x7fcd7403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:03.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.739+0000 7fcd8b0b5700 1 --2- 172.21.15.102:0/1635886335 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fcd74038320 0x7fcd7403a7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd70006fd0 tx=0x7fcd70006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:03.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.739+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/1635886335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 msgr2=0x7fcd84105020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:03.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.739+0000 7fcd8b0b5700 1 --2- 172.21.15.102:0/1635886335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd78011670 tx=0x7fcd78011b50 comp rx=0 tx=0).stop 2024-05-13T22:39:03.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.739+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/1635886335 shutdown_connections 2024-05-13T22:39:03.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.739+0000 7fcd8b0b5700 1 --2- 172.21.15.102:0/1635886335 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fcd74038320 0x7fcd7403a7e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:03.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.739+0000 7fcd8b0b5700 1 --2- 172.21.15.102:0/1635886335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd84105960 0x7fcd84105020 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:03.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.739+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/1635886335 >> 172.21.15.102:0/1635886335 conn(0x7fcd841001a0 msgr2=0x7fcd84100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:03.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.739+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/1635886335 shutdown_connections 2024-05-13T22:39:03.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:02.739+0000 7fcd8b0b5700 1 -- 172.21.15.102:0/1635886335 wait complete. 2024-05-13T22:39:03.939 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:03 smithi102 ceph-mon[28599]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:39:03.939 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:03 smithi102 ceph-mon[28599]: Saving service alertmanager spec with placement count:1 2024-05-13T22:39:03.939 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:03 smithi102 ceph-mon[28599]: from='mgr.14120 172.21.15.102:0/2911434992' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:04.474 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.738+0000 7fe53bff5700 1 Processor -- start 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.738+0000 7fe53bff5700 1 -- start start 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.738+0000 7fe53bff5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe534109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.738+0000 7fe53bff5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe534074620 con 0x7fe534107040 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.739+0000 7fe539d91700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe534109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.739+0000 7fe539d91700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe534109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53408/0 (socket says 172.21.15.102:53408) 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.739+0000 7fe539d91700 1 -- 172.21.15.102:0/1313375304 learned_addr learned my addr 172.21.15.102:0/1313375304 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.739+0000 7fe539d91700 1 -- 172.21.15.102:0/1313375304 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe534109970 con 0x7fe534107040 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.739+0000 7fe539d91700 1 --2- 172.21.15.102:0/1313375304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe534109430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe5240098a0 tx=0x7fe524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e44389a28b1da0c server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.741+0000 7fe538d8f700 1 -- 172.21.15.102:0/1313375304 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe524004180 con 0x7fe534107040 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.741+0000 7fe538d8f700 1 -- 172.21.15.102:0/1313375304 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe5240042e0 con 0x7fe534107040 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.741+0000 7fe538d8f700 1 -- 172.21.15.102:0/1313375304 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe524011af0 con 0x7fe534107040 2024-05-13T22:39:04.475 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.741+0000 7fe53bff5700 1 -- 172.21.15.102:0/1313375304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 msgr2=0x7fe534109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.741+0000 7fe53bff5700 1 --2- 172.21.15.102:0/1313375304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe534109430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe5240098a0 tx=0x7fe524009bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.741+0000 7fe53bff5700 1 -- 172.21.15.102:0/1313375304 shutdown_connections 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.741+0000 7fe53bff5700 1 --2- 172.21.15.102:0/1313375304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe534109430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.741+0000 7fe53bff5700 1 -- 172.21.15.102:0/1313375304 >> 172.21.15.102:0/1313375304 conn(0x7fe5341009f0 msgr2=0x7fe534102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.741+0000 7fe53bff5700 1 -- 172.21.15.102:0/1313375304 shutdown_connections 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.741+0000 7fe53bff5700 1 -- 172.21.15.102:0/1313375304 wait complete. 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.742+0000 7fe53bff5700 1 Processor -- start 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.742+0000 7fe53bff5700 1 -- start start 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.742+0000 7fe53bff5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe53419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.742+0000 7fe53bff5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53419a6a0 con 0x7fe534107040 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.742+0000 7fe539d91700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe53419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.742+0000 7fe539d91700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe53419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53414/0 (socket says 172.21.15.102:53414) 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.742+0000 7fe539d91700 1 -- 172.21.15.102:0/350995912 learned_addr learned my addr 172.21.15.102:0/350995912 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.743+0000 7fe539d91700 1 -- 172.21.15.102:0/350995912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe524009550 con 0x7fe534107040 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.743+0000 7fe539d91700 1 --2- 172.21.15.102:0/350995912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe53419a160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe524006ba0 tx=0x7fe524011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:04.476 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.743+0000 7fe52affd700 1 -- 172.21.15.102:0/350995912 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe524003b00 con 0x7fe534107040 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.743+0000 7fe52affd700 1 -- 172.21.15.102:0/350995912 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe52401a8e0 con 0x7fe534107040 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.743+0000 7fe53bff5700 1 -- 172.21.15.102:0/350995912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53419a8a0 con 0x7fe534107040 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.743+0000 7fe52affd700 1 -- 172.21.15.102:0/350995912 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5240199c0 con 0x7fe534107040 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.743+0000 7fe53bff5700 1 -- 172.21.15.102:0/350995912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53419ad40 con 0x7fe534107040 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.744+0000 7fe52affd700 1 -- 172.21.15.102:0/350995912 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fe524019c80 con 0x7fe534107040 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.744+0000 7fe52affd700 1 --2- 172.21.15.102:0/350995912 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fe5200382d0 0x7fe52003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.744+0000 7fe52affd700 1 -- 172.21.15.102:0/350995912 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe52404c9a0 con 0x7fe534107040 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.744+0000 7fe539590700 1 --2- 172.21.15.102:0/350995912 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fe5200382d0 0x7fe52003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.744+0000 7fe53bff5700 1 -- 172.21.15.102:0/350995912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5340623c0 con 0x7fe534107040 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.745+0000 7fe539590700 1 --2- 172.21.15.102:0/350995912 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fe5200382d0 0x7fe52003a790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe530006fd0 tx=0x7fe530006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.749+0000 7fe52affd700 1 -- 172.21.15.102:0/350995912 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe52402b930 con 0x7fe534107040 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.955+0000 7fe53bff5700 1 -- 172.21.15.102:0/350995912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fe53419d760 con 0x7fe534107040 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.959+0000 7fe52affd700 1 -- 172.21.15.102:0/350995912 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fe5240172b0 con 0x7fe534107040 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.961+0000 7fe53bff5700 1 -- 172.21.15.102:0/350995912 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fe5200382d0 msgr2=0x7fe52003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.961+0000 7fe53bff5700 1 --2- 172.21.15.102:0/350995912 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fe5200382d0 0x7fe52003a790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe530006fd0 tx=0x7fe530006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:04.477 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.961+0000 7fe53bff5700 1 -- 172.21.15.102:0/350995912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 msgr2=0x7fe53419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:04.478 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.961+0000 7fe53bff5700 1 --2- 172.21.15.102:0/350995912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe53419a160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe524006ba0 tx=0x7fe524011b50 comp rx=0 tx=0).stop 2024-05-13T22:39:04.478 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.961+0000 7fe53bff5700 1 -- 172.21.15.102:0/350995912 shutdown_connections 2024-05-13T22:39:04.478 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.961+0000 7fe53bff5700 1 --2- 172.21.15.102:0/350995912 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fe5200382d0 0x7fe52003a790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:04.478 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.961+0000 7fe53bff5700 1 --2- 172.21.15.102:0/350995912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe534107040 0x7fe53419a160 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:04.478 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.961+0000 7fe53bff5700 1 -- 172.21.15.102:0/350995912 >> 172.21.15.102:0/350995912 conn(0x7fe5341009f0 msgr2=0x7fe534101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:04.478 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.962+0000 7fe53bff5700 1 -- 172.21.15.102:0/350995912 shutdown_connections 2024-05-13T22:39:04.478 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:03.962+0000 7fe53bff5700 1 -- 172.21.15.102:0/350995912 wait complete. 2024-05-13T22:39:05.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:04 smithi102 ceph-mon[28599]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:05.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:04 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/350995912' entity='client.admin' 2024-05-13T22:39:05.665 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.877+0000 7f84c6487700 1 Processor -- start 2024-05-13T22:39:05.665 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.877+0000 7f84c6487700 1 -- start start 2024-05-13T22:39:05.665 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.877+0000 7f84c6487700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c0104c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:05.665 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.877+0000 7f84c6487700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c0105230 con 0x7f84c0104840 2024-05-13T22:39:05.665 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.877+0000 7f84bffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c0104c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:05.665 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.878+0000 7f84bffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c0104c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53422/0 (socket says 172.21.15.102:53422) 2024-05-13T22:39:05.665 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.878+0000 7f84bffff700 1 -- 172.21.15.102:0/4177527342 learned_addr learned my addr 172.21.15.102:0/4177527342 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:05.665 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.878+0000 7f84bffff700 1 -- 172.21.15.102:0/4177527342 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c01077d0 con 0x7f84c0104840 2024-05-13T22:39:05.665 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.878+0000 7f84bffff700 1 --2- 172.21.15.102:0/4177527342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c0104c60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f84b00098a0 tx=0x7f84b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18263c668bf6438d server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:05.665 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.878+0000 7f84bf7fe700 1 -- 172.21.15.102:0/4177527342 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84b0004180 con 0x7f84c0104840 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.878+0000 7f84bf7fe700 1 -- 172.21.15.102:0/4177527342 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f84b00042e0 con 0x7f84c0104840 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.878+0000 7f84bf7fe700 1 -- 172.21.15.102:0/4177527342 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84b0011af0 con 0x7f84c0104840 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.879+0000 7f84c6487700 1 -- 172.21.15.102:0/4177527342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 msgr2=0x7f84c0104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.879+0000 7f84c6487700 1 --2- 172.21.15.102:0/4177527342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c0104c60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f84b00098a0 tx=0x7f84b0009bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.879+0000 7f84c6487700 1 -- 172.21.15.102:0/4177527342 shutdown_connections 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.879+0000 7f84c6487700 1 --2- 172.21.15.102:0/4177527342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c0104c60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.879+0000 7f84c6487700 1 -- 172.21.15.102:0/4177527342 >> 172.21.15.102:0/4177527342 conn(0x7f84c0100190 msgr2=0x7f84c01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.879+0000 7f84c6487700 1 -- 172.21.15.102:0/4177527342 shutdown_connections 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.879+0000 7f84c6487700 1 -- 172.21.15.102:0/4177527342 wait complete. 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.879+0000 7f84c6487700 1 Processor -- start 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84c6487700 1 -- start start 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84c6487700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:05.666 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84c6487700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c019a7b0 con 0x7f84c0104840 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84bffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84bffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c019a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53436/0 (socket says 172.21.15.102:53436) 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84bffff700 1 -- 172.21.15.102:0/2069432931 learned_addr learned my addr 172.21.15.102:0/2069432931 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84bffff700 1 -- 172.21.15.102:0/2069432931 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b0009550 con 0x7f84c0104840 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84bffff700 1 --2- 172.21.15.102:0/2069432931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c019a270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84b0011670 tx=0x7f84b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84bdffb700 1 -- 172.21.15.102:0/2069432931 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84b00039c0 con 0x7f84c0104840 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84bdffb700 1 -- 172.21.15.102:0/2069432931 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f84b0003b20 con 0x7f84c0104840 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84c6487700 1 -- 172.21.15.102:0/2069432931 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c019a9b0 con 0x7f84c0104840 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84bdffb700 1 -- 172.21.15.102:0/2069432931 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84b00247f0 con 0x7f84c0104840 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.880+0000 7f84c6487700 1 -- 172.21.15.102:0/2069432931 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c019ae50 con 0x7f84c0104840 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.881+0000 7f84bdffb700 1 -- 172.21.15.102:0/2069432931 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f84b0024ab0 con 0x7f84c0104840 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.881+0000 7f84c6487700 1 -- 172.21.15.102:0/2069432931 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c00623c0 con 0x7f84c0104840 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.881+0000 7f84bdffb700 1 --2- 172.21.15.102:0/2069432931 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f84ac0382e0 0x7f84ac03a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.882+0000 7f84bdffb700 1 -- 172.21.15.102:0/2069432931 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f84b004c450 con 0x7f84c0104840 2024-05-13T22:39:05.667 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.882+0000 7f84b7fff700 1 --2- 172.21.15.102:0/2069432931 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f84ac0382e0 0x7f84ac03a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.882+0000 7f84b7fff700 1 --2- 172.21.15.102:0/2069432931 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f84ac0382e0 0x7f84ac03a7a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f84a8006fd0 tx=0x7f84a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:04.886+0000 7f84bdffb700 1 -- 172.21.15.102:0/2069432931 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f84b00187c0 con 0x7f84c0104840 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.077+0000 7f84c6487700 1 -- 172.21.15.102:0/2069432931 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f84c019dd50 con 0x7f84c0104840 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.080+0000 7f84bdffb700 1 -- 172.21.15.102:0/2069432931 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f84b00187c0 con 0x7f84c0104840 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.082+0000 7f84c6487700 1 -- 172.21.15.102:0/2069432931 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f84ac0382e0 msgr2=0x7f84ac03a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.082+0000 7f84c6487700 1 --2- 172.21.15.102:0/2069432931 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f84ac0382e0 0x7f84ac03a7a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f84a8006fd0 tx=0x7f84a8006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.082+0000 7f84c6487700 1 -- 172.21.15.102:0/2069432931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 msgr2=0x7f84c019a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.082+0000 7f84c6487700 1 --2- 172.21.15.102:0/2069432931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c019a270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84b0011670 tx=0x7f84b0011aa0 comp rx=0 tx=0).stop 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.082+0000 7f84c6487700 1 -- 172.21.15.102:0/2069432931 shutdown_connections 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.082+0000 7f84c6487700 1 --2- 172.21.15.102:0/2069432931 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f84ac0382e0 0x7f84ac03a7a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.082+0000 7f84c6487700 1 --2- 172.21.15.102:0/2069432931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84c0104840 0x7f84c019a270 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.082+0000 7f84c6487700 1 -- 172.21.15.102:0/2069432931 >> 172.21.15.102:0/2069432931 conn(0x7f84c0100190 msgr2=0x7f84c0101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.082+0000 7f84c6487700 1 -- 172.21.15.102:0/2069432931 shutdown_connections 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:05.082+0000 7f84c6487700 1 -- 172.21.15.102:0/2069432931 wait complete. 2024-05-13T22:39:05.668 INFO:teuthology.orchestra.run.smithi102.stdout:Enabling the dashboard module... 2024-05-13T22:39:06.182 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:06 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2069432931' entity='client.admin' 2024-05-13T22:39:07.372 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:07 smithi102 ceph-mon[28599]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:07.372 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:07 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3581905655' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-13T22:39:07.622 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.136+0000 7ff56eb41700 1 Processor -- start 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.136+0000 7ff56eb41700 1 -- start start 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.136+0000 7ff56eb41700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff568106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.136+0000 7ff56eb41700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff568106f50 con 0x7ff568106560 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.136+0000 7ff56c8dd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff568106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.136+0000 7ff56c8dd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff568106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53448/0 (socket says 172.21.15.102:53448) 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.136+0000 7ff56c8dd700 1 -- 172.21.15.102:0/2084409741 learned_addr learned my addr 172.21.15.102:0/2084409741 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.136+0000 7ff56c8dd700 1 -- 172.21.15.102:0/2084409741 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff568107760 con 0x7ff568106560 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.137+0000 7ff56c8dd700 1 --2- 172.21.15.102:0/2084409741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff568106980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0098a0 tx=0x7ff55c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e193616ba0cdfd server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.137+0000 7ff5677fe700 1 -- 172.21.15.102:0/2084409741 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff55c004180 con 0x7ff568106560 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.137+0000 7ff5677fe700 1 -- 172.21.15.102:0/2084409741 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff55c0042e0 con 0x7ff568106560 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.137+0000 7ff5677fe700 1 -- 172.21.15.102:0/2084409741 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff55c011af0 con 0x7ff568106560 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.137+0000 7ff56eb41700 1 -- 172.21.15.102:0/2084409741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 msgr2=0x7ff568106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.137+0000 7ff56eb41700 1 --2- 172.21.15.102:0/2084409741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff568106980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0098a0 tx=0x7ff55c009bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:07.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.137+0000 7ff56eb41700 1 -- 172.21.15.102:0/2084409741 shutdown_connections 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.137+0000 7ff56eb41700 1 --2- 172.21.15.102:0/2084409741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff568106980 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.137+0000 7ff56eb41700 1 -- 172.21.15.102:0/2084409741 >> 172.21.15.102:0/2084409741 conn(0x7ff568101b40 msgr2=0x7ff568103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.137+0000 7ff56eb41700 1 -- 172.21.15.102:0/2084409741 shutdown_connections 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.137+0000 7ff56eb41700 1 -- 172.21.15.102:0/2084409741 wait complete. 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.138+0000 7ff56eb41700 1 Processor -- start 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.138+0000 7ff56eb41700 1 -- start start 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.138+0000 7ff56eb41700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff56807bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.138+0000 7ff56eb41700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56807c170 con 0x7ff568106560 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.138+0000 7ff56c8dd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff56807bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.138+0000 7ff56c8dd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff56807bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53462/0 (socket says 172.21.15.102:53462) 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.138+0000 7ff56c8dd700 1 -- 172.21.15.102:0/3581905655 learned_addr learned my addr 172.21.15.102:0/3581905655 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.139+0000 7ff56c8dd700 1 -- 172.21.15.102:0/3581905655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c009550 con 0x7ff568106560 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.139+0000 7ff56c8dd700 1 --2- 172.21.15.102:0/3581905655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff56807bc30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff55c011670 tx=0x7ff55c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.139+0000 7ff565ffb700 1 -- 172.21.15.102:0/3581905655 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff55c0039f0 con 0x7ff568106560 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.139+0000 7ff56eb41700 1 -- 172.21.15.102:0/3581905655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56807a280 con 0x7ff568106560 2024-05-13T22:39:07.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.139+0000 7ff565ffb700 1 -- 172.21.15.102:0/3581905655 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff55c003b50 con 0x7ff568106560 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.139+0000 7ff565ffb700 1 -- 172.21.15.102:0/3581905655 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff55c0247c0 con 0x7ff568106560 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.139+0000 7ff56eb41700 1 -- 172.21.15.102:0/3581905655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56807a720 con 0x7ff568106560 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.140+0000 7ff565ffb700 1 -- 172.21.15.102:0/3581905655 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ff55c01ac00 con 0x7ff568106560 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.140+0000 7ff56eb41700 1 -- 172.21.15.102:0/3581905655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5680623c0 con 0x7ff568106560 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.140+0000 7ff565ffb700 1 --2- 172.21.15.102:0/3581905655 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7ff558038320 0x7ff55803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.140+0000 7ff565ffb700 1 -- 172.21.15.102:0/3581905655 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff55c04c420 con 0x7ff568106560 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.140+0000 7ff567fff700 1 --2- 172.21.15.102:0/3581905655 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7ff558038320 0x7ff55803a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.141+0000 7ff567fff700 1 --2- 172.21.15.102:0/3581905655 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7ff558038320 0x7ff55803a7e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff554006fd0 tx=0x7ff554006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.145+0000 7ff565ffb700 1 -- 172.21.15.102:0/3581905655 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff55c01a430 con 0x7ff568106560 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:06.386+0000 7ff56eb41700 1 -- 172.21.15.102:0/3581905655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7ff56810a960 con 0x7ff568106560 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.084+0000 7ff565ffb700 1 -- 172.21.15.102:0/3581905655 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7ff55c018350 con 0x7ff568106560 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.085+0000 7ff565ffb700 1 -- 172.21.15.102:0/3581905655 <== mon.0 v2:172.21.15.102:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7ff55c01a430 con 0x7ff568106560 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.087+0000 7ff56eb41700 1 -- 172.21.15.102:0/3581905655 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7ff558038320 msgr2=0x7ff55803a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.087+0000 7ff56eb41700 1 --2- 172.21.15.102:0/3581905655 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7ff558038320 0x7ff55803a7e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff554006fd0 tx=0x7ff554006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:07.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.087+0000 7ff56eb41700 1 -- 172.21.15.102:0/3581905655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 msgr2=0x7ff56807bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:07.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.087+0000 7ff56eb41700 1 --2- 172.21.15.102:0/3581905655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff56807bc30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff55c011670 tx=0x7ff55c011b50 comp rx=0 tx=0).stop 2024-05-13T22:39:07.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.088+0000 7ff56eb41700 1 -- 172.21.15.102:0/3581905655 shutdown_connections 2024-05-13T22:39:07.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.088+0000 7ff56eb41700 1 --2- 172.21.15.102:0/3581905655 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7ff558038320 0x7ff55803a7e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:07.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.088+0000 7ff56eb41700 1 --2- 172.21.15.102:0/3581905655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff568106560 0x7ff56807bc30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:07.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.088+0000 7ff56eb41700 1 -- 172.21.15.102:0/3581905655 >> 172.21.15.102:0/3581905655 conn(0x7ff568101b40 msgr2=0x7ff568102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:07.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.088+0000 7ff56eb41700 1 -- 172.21.15.102:0/3581905655 shutdown_connections 2024-05-13T22:39:07.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:07.088+0000 7ff56eb41700 1 -- 172.21.15.102:0/3581905655 wait complete. 2024-05-13T22:39:08.313 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:08 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3581905655' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-13T22:39:08.313 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:08 smithi102 ceph-mon[28599]: mgrmap e9: smithi102.xcijrj(active, since 25s) 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "active_name": "smithi102.xcijrj", 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.062+0000 7fc32aaa3700 1 Processor -- start 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.062+0000 7fc32aaa3700 1 -- start start 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.062+0000 7fc32aaa3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc3241093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.062+0000 7fc32aaa3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc324079e50 con 0x7fc324107000 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.063+0000 7fc32883f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc3241093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:08.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.063+0000 7fc32883f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc3241093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53474/0 (socket says 172.21.15.102:53474) 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.063+0000 7fc32883f700 1 -- 172.21.15.102:0/1176911370 learned_addr learned my addr 172.21.15.102:0/1176911370 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.063+0000 7fc32883f700 1 -- 172.21.15.102:0/1176911370 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc324109930 con 0x7fc324107000 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.063+0000 7fc32883f700 1 --2- 172.21.15.102:0/1176911370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc3241093f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3180098a0 tx=0x7fc318009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcfba138a46839d server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.063+0000 7fc3237fe700 1 -- 172.21.15.102:0/1176911370 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc318004180 con 0x7fc324107000 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.063+0000 7fc3237fe700 1 -- 172.21.15.102:0/1176911370 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc3180042e0 con 0x7fc324107000 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.063+0000 7fc3237fe700 1 -- 172.21.15.102:0/1176911370 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc318011af0 con 0x7fc324107000 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.064+0000 7fc32aaa3700 1 -- 172.21.15.102:0/1176911370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 msgr2=0x7fc3241093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.064+0000 7fc32aaa3700 1 --2- 172.21.15.102:0/1176911370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc3241093f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3180098a0 tx=0x7fc318009bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.064+0000 7fc32aaa3700 1 -- 172.21.15.102:0/1176911370 shutdown_connections 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.064+0000 7fc32aaa3700 1 --2- 172.21.15.102:0/1176911370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc3241093f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.064+0000 7fc32aaa3700 1 -- 172.21.15.102:0/1176911370 >> 172.21.15.102:0/1176911370 conn(0x7fc3241009b0 msgr2=0x7fc324102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.064+0000 7fc32aaa3700 1 -- 172.21.15.102:0/1176911370 shutdown_connections 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.064+0000 7fc32aaa3700 1 -- 172.21.15.102:0/1176911370 wait complete. 2024-05-13T22:39:08.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.065+0000 7fc32aaa3700 1 Processor -- start 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.065+0000 7fc32aaa3700 1 -- start start 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.065+0000 7fc32aaa3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc32419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.065+0000 7fc32aaa3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32419a750 con 0x7fc324107000 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.065+0000 7fc32883f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc32419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.065+0000 7fc32883f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc32419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53484/0 (socket says 172.21.15.102:53484) 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.065+0000 7fc32883f700 1 -- 172.21.15.102:0/31022201 learned_addr learned my addr 172.21.15.102:0/31022201 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.065+0000 7fc32883f700 1 -- 172.21.15.102:0/31022201 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc318009550 con 0x7fc324107000 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.065+0000 7fc32883f700 1 --2- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc32419a210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc318011670 tx=0x7fc318011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.065+0000 7fc321ffb700 1 -- 172.21.15.102:0/31022201 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc318003a40 con 0x7fc324107000 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.066+0000 7fc321ffb700 1 -- 172.21.15.102:0/31022201 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc318003ba0 con 0x7fc324107000 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.066+0000 7fc32aaa3700 1 -- 172.21.15.102:0/31022201 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32419a950 con 0x7fc324107000 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.066+0000 7fc321ffb700 1 -- 172.21.15.102:0/31022201 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc318024820 con 0x7fc324107000 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.066+0000 7fc32aaa3700 1 -- 172.21.15.102:0/31022201 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32419adf0 con 0x7fc324107000 2024-05-13T22:39:08.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.066+0000 7fc321ffb700 1 -- 172.21.15.102:0/31022201 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fc318024ae0 con 0x7fc324107000 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.067+0000 7fc321ffb700 1 --2- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc314038330 0x7fc31403a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.067+0000 7fc321ffb700 1 -- 172.21.15.102:0/31022201 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc31801ac70 con 0x7fc324107000 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.067+0000 7fc323fff700 1 -- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc314038330 msgr2=0x7fc31403a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/358342393 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.067+0000 7fc323fff700 1 --2- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc314038330 0x7fc31403a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.067+0000 7fc32aaa3700 1 -- 172.21.15.102:0/31022201 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3240623c0 con 0x7fc324107000 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.072+0000 7fc321ffb700 1 -- 172.21.15.102:0/31022201 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc318022310 con 0x7fc324107000 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.268+0000 7fc323fff700 1 -- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc314038330 msgr2=0x7fc31403a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/358342393 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.268+0000 7fc323fff700 1 --2- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc314038330 0x7fc31403a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.333+0000 7fc32aaa3700 1 -- 172.21.15.102:0/31022201 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc32419d900 con 0x7fc324107000 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.334+0000 7fc321ffb700 1 -- 172.21.15.102:0/31022201 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fc31802c350 con 0x7fc324107000 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.337+0000 7fc32aaa3700 1 -- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc314038330 msgr2=0x7fc31403a7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.337+0000 7fc32aaa3700 1 --2- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc314038330 0x7fc31403a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:08.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.337+0000 7fc32aaa3700 1 -- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 msgr2=0x7fc32419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.337+0000 7fc32aaa3700 1 --2- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc32419a210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc318011670 tx=0x7fc318011aa0 comp rx=0 tx=0).stop 2024-05-13T22:39:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.338+0000 7fc32aaa3700 1 -- 172.21.15.102:0/31022201 shutdown_connections 2024-05-13T22:39:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.338+0000 7fc32aaa3700 1 --2- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7fc314038330 0x7fc31403a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.338+0000 7fc32aaa3700 1 --2- 172.21.15.102:0/31022201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc324107000 0x7fc32419a210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.338+0000 7fc32aaa3700 1 -- 172.21.15.102:0/31022201 >> 172.21.15.102:0/31022201 conn(0x7fc3241009b0 msgr2=0x7fc324101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.339+0000 7fc32aaa3700 1 -- 172.21.15.102:0/31022201 shutdown_connections 2024-05-13T22:39:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:08.339+0000 7fc32aaa3700 1 -- 172.21.15.102:0/31022201 wait complete. 2024-05-13T22:39:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for the mgr to restart... 2024-05-13T22:39:08.909 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr epoch 9... 2024-05-13T22:39:09.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:09 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/31022201' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-13T22:39:13.717 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: Active manager daemon smithi102.xcijrj restarted 2024-05-13T22:39:13.717 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: Activating manager daemon smithi102.xcijrj 2024-05-13T22:39:13.717 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: osdmap e3: 0 total, 0 up, 0 in 2024-05-13T22:39:13.717 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: mgrmap e10: smithi102.xcijrj(active, starting, since 0.00374562s) 2024-05-13T22:39:13.718 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:39:13.718 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi102.xcijrj", "id": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:39:13.718 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-13T22:39:13.718 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-13T22:39:13.718 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-13T22:39:13.718 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: Manager daemon smithi102.xcijrj is now available 2024-05-13T22:39:13.718 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:39:13.718 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:39:13.718 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:39:13.718 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:13 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/trash_purge_schedule"}]: dispatch 2024-05-13T22:39:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-05-13T22:39:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-13T22:39:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-13T22:39:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-05-13T22:39:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.355+0000 7f5a22c25700 1 Processor -- start 2024-05-13T22:39:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.355+0000 7f5a22c25700 1 -- start start 2024-05-13T22:39:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.355+0000 7f5a22c25700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.355+0000 7f5a22c25700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c07a890 con 0x7f5a1c07be60 2024-05-13T22:39:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.355+0000 7f5a209c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:15.299 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.355+0000 7f5a209c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54608/0 (socket says 172.21.15.102:54608) 2024-05-13T22:39:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.355+0000 7f5a209c1700 1 -- 172.21.15.102:0/1072918133 learned_addr learned my addr 172.21.15.102:0/1072918133 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.355+0000 7f5a209c1700 1 -- 172.21.15.102:0/1072918133 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c07a9d0 con 0x7f5a1c07be60 2024-05-13T22:39:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.355+0000 7f5a209c1700 1 --2- 172.21.15.102:0/1072918133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c07a2c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a100098a0 tx=0x7f5a10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e2972a70ddce48 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.356+0000 7f5a1b7fe700 1 -- 172.21.15.102:0/1072918133 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a10004180 con 0x7f5a1c07be60 2024-05-13T22:39:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.356+0000 7f5a1b7fe700 1 -- 172.21.15.102:0/1072918133 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5a100042e0 con 0x7f5a1c07be60 2024-05-13T22:39:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.356+0000 7f5a1b7fe700 1 -- 172.21.15.102:0/1072918133 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a10011af0 con 0x7f5a1c07be60 2024-05-13T22:39:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.357+0000 7f5a22c25700 1 -- 172.21.15.102:0/1072918133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 msgr2=0x7f5a1c07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.357+0000 7f5a22c25700 1 --2- 172.21.15.102:0/1072918133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c07a2c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a100098a0 tx=0x7f5a10009bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.357+0000 7f5a22c25700 1 -- 172.21.15.102:0/1072918133 shutdown_connections 2024-05-13T22:39:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.357+0000 7f5a22c25700 1 --2- 172.21.15.102:0/1072918133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c07a2c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:15.300 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.357+0000 7f5a22c25700 1 -- 172.21.15.102:0/1072918133 >> 172.21.15.102:0/1072918133 conn(0x7f5a1c101b40 msgr2=0x7f5a1c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.357+0000 7f5a22c25700 1 -- 172.21.15.102:0/1072918133 shutdown_connections 2024-05-13T22:39:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.357+0000 7f5a22c25700 1 -- 172.21.15.102:0/1072918133 wait complete. 2024-05-13T22:39:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.357+0000 7f5a22c25700 1 Processor -- start 2024-05-13T22:39:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.357+0000 7f5a22c25700 1 -- start start 2024-05-13T22:39:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.357+0000 7f5a22c25700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.357+0000 7f5a22c25700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c19c8f0 con 0x7f5a1c07be60 2024-05-13T22:39:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.358+0000 7f5a209c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.358+0000 7f5a209c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c19c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54614/0 (socket says 172.21.15.102:54614) 2024-05-13T22:39:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.358+0000 7f5a209c1700 1 -- 172.21.15.102:0/39310324 learned_addr learned my addr 172.21.15.102:0/39310324 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.358+0000 7f5a209c1700 1 -- 172.21.15.102:0/39310324 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a10009550 con 0x7f5a1c07be60 2024-05-13T22:39:15.301 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.358+0000 7f5a209c1700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c19c3b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a10011ad0 tx=0x7f5a10011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.358+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a10003ab0 con 0x7f5a1c07be60 2024-05-13T22:39:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.358+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5a10003c10 con 0x7f5a1c07be60 2024-05-13T22:39:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.358+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a100198e0 con 0x7f5a1c07be60 2024-05-13T22:39:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.358+0000 7f5a22c25700 1 -- 172.21.15.102:0/39310324 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c19caf0 con 0x7f5a1c07be60 2024-05-13T22:39:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.358+0000 7f5a22c25700 1 -- 172.21.15.102:0/39310324 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c19cf90 con 0x7f5a1c07be60 2024-05-13T22:39:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.359+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f5a10019b50 con 0x7f5a1c07be60 2024-05-13T22:39:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.359+0000 7f5a19ffb700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 0x7f5a0c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.359+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 --> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a0c03af90 con 0x7f5a0c0383c0 2024-05-13T22:39:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.359+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5a1004c300 con 0x7f5a1c07be60 2024-05-13T22:39:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.359+0000 7f5a1bfff700 1 -- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 msgr2=0x7f5a0c03a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/358342393 2024-05-13T22:39:15.302 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.360+0000 7f5a1bfff700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 0x7f5a0c03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.561+0000 7f5a1bfff700 1 -- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 msgr2=0x7f5a0c03a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/358342393 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.561+0000 7f5a1bfff700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 0x7f5a0c03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.962+0000 7f5a1bfff700 1 -- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 msgr2=0x7f5a0c03a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/358342393 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:09.962+0000 7f5a1bfff700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 0x7f5a0c03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:10.762+0000 7f5a1bfff700 1 -- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 msgr2=0x7f5a0c03a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/358342393 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:10.762+0000 7f5a1bfff700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 0x7f5a0c03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:12.363+0000 7f5a1bfff700 1 -- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 msgr2=0x7f5a0c03a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/358342393 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:12.363+0000 7f5a1bfff700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 0x7f5a0c03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:13.682+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7f5a1001a8e0 con 0x7f5a1c07be60 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:13.682+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 msgr2=0x7f5a0c03a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:13.682+0000 7f5a19ffb700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 0x7f5a0c03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:15.303 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.683+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f5a1004db10 con 0x7f5a1c07be60 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.683+0000 7f5a19ffb700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5a0c03be30 0x7f5a0c03e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.683+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 --> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a0c03af90 con 0x7f5a0c03be30 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.684+0000 7f5a1bfff700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5a0c03be30 0x7f5a0c03e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.684+0000 7f5a1bfff700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5a0c03be30 0x7f5a0c03e220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5a08003a10 tx=0x7f5a08009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.685+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 <== mgr.14164 v2:172.21.15.102:6800/263535069 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f5a0c03af90 con 0x7f5a0c03be30 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.695+0000 7f5a22c25700 1 -- 172.21.15.102:0/39310324 --> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f5a1c19d6f0 con 0x7f5a0c03be30 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.695+0000 7f5a19ffb700 1 -- 172.21.15.102:0/39310324 <== mgr.14164 v2:172.21.15.102:6800/263535069 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f5a1c19d6f0 con 0x7f5a0c03be30 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.696+0000 7f5a22c25700 1 -- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5a0c03be30 msgr2=0x7f5a0c03e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.696+0000 7f5a22c25700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5a0c03be30 0x7f5a0c03e220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5a08003a10 tx=0x7f5a08009270 comp rx=0 tx=0).stop 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.696+0000 7f5a22c25700 1 -- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 msgr2=0x7f5a1c19c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.696+0000 7f5a22c25700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c19c3b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a10011ad0 tx=0x7f5a10011bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:15.304 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.696+0000 7f5a22c25700 1 -- 172.21.15.102:0/39310324 shutdown_connections 2024-05-13T22:39:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.696+0000 7f5a22c25700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5a0c03be30 0x7f5a0c03e220 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.696+0000 7f5a22c25700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a1c07be60 0x7f5a1c19c3b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.696+0000 7f5a22c25700 1 --2- 172.21.15.102:0/39310324 >> [v2:172.21.15.102:6800/358342393,v1:172.21.15.102:6801/358342393] conn(0x7f5a0c0383c0 0x7f5a0c03a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.696+0000 7f5a22c25700 1 -- 172.21.15.102:0/39310324 >> 172.21.15.102:0/39310324 conn(0x7f5a1c101b40 msgr2=0x7f5a1c1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.696+0000 7f5a22c25700 1 -- 172.21.15.102:0/39310324 shutdown_connections 2024-05-13T22:39:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:14.696+0000 7f5a22c25700 1 -- 172.21.15.102:0/39310324 wait complete. 2024-05-13T22:39:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:mgr epoch 9 is available 2024-05-13T22:39:15.305 INFO:teuthology.orchestra.run.smithi102.stdout:Generating a dashboard self-signed certificate... 2024-05-13T22:39:15.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:15 smithi102 ceph-mon[28599]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-13T22:39:15.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:15 smithi102 ceph-mon[28599]: mgrmap e11: smithi102.xcijrj(active, since 1.0056s) 2024-05-13T22:39:15.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:15 smithi102 ceph-mon[28599]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-13T22:39:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-13T22:39:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.761+0000 7fce39a27700 1 Processor -- start 2024-05-13T22:39:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.761+0000 7fce39a27700 1 -- start start 2024-05-13T22:39:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.761+0000 7fce39a27700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce34109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.761+0000 7fce39a27700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34079e10 con 0x7fce34107040 2024-05-13T22:39:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.762+0000 7fce32ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce34109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.762+0000 7fce32ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce34109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54696/0 (socket says 172.21.15.102:54696) 2024-05-13T22:39:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.762+0000 7fce32ffd700 1 -- 172.21.15.102:0/2433539033 learned_addr learned my addr 172.21.15.102:0/2433539033 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.762+0000 7fce32ffd700 1 -- 172.21.15.102:0/2433539033 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34109970 con 0x7fce34107040 2024-05-13T22:39:16.603 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.762+0000 7fce32ffd700 1 --2- 172.21.15.102:0/2433539033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce34109430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce240098a0 tx=0x7fce24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab582451a229c3c server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.762+0000 7fce31ffb700 1 -- 172.21.15.102:0/2433539033 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce24004180 con 0x7fce34107040 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.762+0000 7fce31ffb700 1 -- 172.21.15.102:0/2433539033 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fce240042e0 con 0x7fce34107040 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.762+0000 7fce31ffb700 1 -- 172.21.15.102:0/2433539033 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce24011af0 con 0x7fce34107040 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.763+0000 7fce39a27700 1 -- 172.21.15.102:0/2433539033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 msgr2=0x7fce34109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.763+0000 7fce39a27700 1 --2- 172.21.15.102:0/2433539033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce34109430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce240098a0 tx=0x7fce24009bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.763+0000 7fce39a27700 1 -- 172.21.15.102:0/2433539033 shutdown_connections 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.763+0000 7fce39a27700 1 --2- 172.21.15.102:0/2433539033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce34109430 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.763+0000 7fce39a27700 1 -- 172.21.15.102:0/2433539033 >> 172.21.15.102:0/2433539033 conn(0x7fce341009b0 msgr2=0x7fce34102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.763+0000 7fce39a27700 1 -- 172.21.15.102:0/2433539033 shutdown_connections 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.763+0000 7fce39a27700 1 -- 172.21.15.102:0/2433539033 wait complete. 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.764+0000 7fce39a27700 1 Processor -- start 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.764+0000 7fce39a27700 1 -- start start 2024-05-13T22:39:16.604 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.764+0000 7fce39a27700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce3419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.764+0000 7fce39a27700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3419a750 con 0x7fce34107040 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.764+0000 7fce32ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce3419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.764+0000 7fce32ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce3419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54706/0 (socket says 172.21.15.102:54706) 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.764+0000 7fce32ffd700 1 -- 172.21.15.102:0/1320751320 learned_addr learned my addr 172.21.15.102:0/1320751320 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.764+0000 7fce32ffd700 1 -- 172.21.15.102:0/1320751320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce24009550 con 0x7fce34107040 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.765+0000 7fce32ffd700 1 --2- 172.21.15.102:0/1320751320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce3419a210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce24011670 tx=0x7fce24011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.765+0000 7fce38a25700 1 -- 172.21.15.102:0/1320751320 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce24003a40 con 0x7fce34107040 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.765+0000 7fce38a25700 1 -- 172.21.15.102:0/1320751320 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fce24003ba0 con 0x7fce34107040 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.765+0000 7fce38a25700 1 -- 172.21.15.102:0/1320751320 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce24024820 con 0x7fce34107040 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.765+0000 7fce39a27700 1 -- 172.21.15.102:0/1320751320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3419a950 con 0x7fce34107040 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.765+0000 7fce39a27700 1 -- 172.21.15.102:0/1320751320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3419adf0 con 0x7fce34107040 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.766+0000 7fce38a25700 1 -- 172.21.15.102:0/1320751320 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7fce24024a50 con 0x7fce34107040 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.766+0000 7fce39a27700 1 -- 172.21.15.102:0/1320751320 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3410a6b0 con 0x7fce34107040 2024-05-13T22:39:16.605 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.766+0000 7fce38a25700 1 --2- 172.21.15.102:0/1320751320 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fce1c038210 0x7fce1c03a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.766+0000 7fce38a25700 1 -- 172.21.15.102:0/1320751320 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fce2404c330 con 0x7fce34107040 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.766+0000 7fce327fc700 1 --2- 172.21.15.102:0/1320751320 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fce1c038210 0x7fce1c03a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.767+0000 7fce327fc700 1 --2- 172.21.15.102:0/1320751320 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fce1c038210 0x7fce1c03a6d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce28006fd0 tx=0x7fce28006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.771+0000 7fce38a25700 1 -- 172.21.15.102:0/1320751320 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fce24022310 con 0x7fce34107040 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:15.976+0000 7fce39a27700 1 -- 172.21.15.102:0/1320751320 --> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fce34101eb0 con 0x7fce1c038210 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.022+0000 7fce38a25700 1 -- 172.21.15.102:0/1320751320 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fce2401ac70 con 0x7fce34107040 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.080+0000 7fce38a25700 1 -- 172.21.15.102:0/1320751320 <== mgr.14164 v2:172.21.15.102:6800/263535069 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fce34101eb0 con 0x7fce1c038210 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.082+0000 7fce39a27700 1 -- 172.21.15.102:0/1320751320 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fce1c038210 msgr2=0x7fce1c03a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.082+0000 7fce39a27700 1 --2- 172.21.15.102:0/1320751320 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fce1c038210 0x7fce1c03a6d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce28006fd0 tx=0x7fce28006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.082+0000 7fce39a27700 1 -- 172.21.15.102:0/1320751320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 msgr2=0x7fce3419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.082+0000 7fce39a27700 1 --2- 172.21.15.102:0/1320751320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce3419a210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce24011670 tx=0x7fce24011aa0 comp rx=0 tx=0).stop 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.082+0000 7fce39a27700 1 -- 172.21.15.102:0/1320751320 shutdown_connections 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.082+0000 7fce39a27700 1 --2- 172.21.15.102:0/1320751320 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fce1c038210 0x7fce1c03a6d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:16.606 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.082+0000 7fce39a27700 1 --2- 172.21.15.102:0/1320751320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce34107040 0x7fce3419a210 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:16.607 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.082+0000 7fce39a27700 1 -- 172.21.15.102:0/1320751320 >> 172.21.15.102:0/1320751320 conn(0x7fce341009b0 msgr2=0x7fce341017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:16.607 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.082+0000 7fce39a27700 1 -- 172.21.15.102:0/1320751320 shutdown_connections 2024-05-13T22:39:16.607 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:16.082+0000 7fce39a27700 1 -- 172.21.15.102:0/1320751320 wait complete. 2024-05-13T22:39:16.607 INFO:teuthology.orchestra.run.smithi102.stdout:Creating initial admin user... 2024-05-13T22:39:17.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:16 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:17.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:16 smithi102 ceph-mon[28599]: [13/May/2024:22:39:15] ENGINE Bus STARTING 2024-05-13T22:39:17.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:16 smithi102 ceph-mon[28599]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:39:17.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:16 smithi102 ceph-mon[28599]: mgrmap e12: smithi102.xcijrj(active, since 2s) 2024-05-13T22:39:17.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:16 smithi102 ceph-mon[28599]: [13/May/2024:22:39:16] ENGINE Serving on https://172.21.15.102:7150 2024-05-13T22:39:17.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:16 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:17.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:16 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:18.076 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:17 smithi102 ceph-mon[28599]: [13/May/2024:22:39:16] ENGINE Serving on http://172.21.15.102:8765 2024-05-13T22:39:18.077 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:17 smithi102 ceph-mon[28599]: [13/May/2024:22:39:16] ENGINE Bus STARTED 2024-05-13T22:39:18.077 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:17 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:18.077 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:17 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:18.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$3y52WxYIj9g4x6I5gh/cWu9wGEmtnxXSan7VHlAQeEMDJ4kU7mYxa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715639957, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-13T22:39:18.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.082+0000 7f10e05f5700 1 Processor -- start 2024-05-13T22:39:18.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.082+0000 7f10e05f5700 1 -- start start 2024-05-13T22:39:18.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.083+0000 7f10e05f5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d8109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:18.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.083+0000 7f10e05f5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d8079e30 con 0x7f10d8107040 2024-05-13T22:39:18.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.083+0000 7f10de391700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d8109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.083+0000 7f10de391700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d8109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54708/0 (socket says 172.21.15.102:54708) 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.083+0000 7f10de391700 1 -- 172.21.15.102:0/2583348412 learned_addr learned my addr 172.21.15.102:0/2583348412 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.083+0000 7f10de391700 1 -- 172.21.15.102:0/2583348412 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d8109970 con 0x7f10d8107040 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.083+0000 7f10de391700 1 --2- 172.21.15.102:0/2583348412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d8109430 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10d40098a0 tx=0x7f10d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a84ca477dfa213 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.084+0000 7f10dd38f700 1 -- 172.21.15.102:0/2583348412 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10d4004180 con 0x7f10d8107040 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.084+0000 7f10dd38f700 1 -- 172.21.15.102:0/2583348412 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f10d40042e0 con 0x7f10d8107040 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.084+0000 7f10dd38f700 1 -- 172.21.15.102:0/2583348412 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10d4011af0 con 0x7f10d8107040 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.084+0000 7f10e05f5700 1 -- 172.21.15.102:0/2583348412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 msgr2=0x7f10d8109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.084+0000 7f10e05f5700 1 --2- 172.21.15.102:0/2583348412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d8109430 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10d40098a0 tx=0x7f10d4009bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.084+0000 7f10e05f5700 1 -- 172.21.15.102:0/2583348412 shutdown_connections 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.084+0000 7f10e05f5700 1 --2- 172.21.15.102:0/2583348412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d8109430 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.084+0000 7f10e05f5700 1 -- 172.21.15.102:0/2583348412 >> 172.21.15.102:0/2583348412 conn(0x7f10d81009f0 msgr2=0x7f10d8102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:18.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.084+0000 7f10e05f5700 1 -- 172.21.15.102:0/2583348412 shutdown_connections 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.085+0000 7f10e05f5700 1 -- 172.21.15.102:0/2583348412 wait complete. 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.085+0000 7f10e05f5700 1 Processor -- start 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.085+0000 7f10e05f5700 1 -- start start 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.085+0000 7f10e05f5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.085+0000 7f10e05f5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d819a710 con 0x7f10d8107040 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.085+0000 7f10de391700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.086+0000 7f10de391700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54710/0 (socket says 172.21.15.102:54710) 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.086+0000 7f10de391700 1 -- 172.21.15.102:0/10702384 learned_addr learned my addr 172.21.15.102:0/10702384 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.086+0000 7f10de391700 1 -- 172.21.15.102:0/10702384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d4009550 con 0x7f10d8107040 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.086+0000 7f10de391700 1 --2- 172.21.15.102:0/10702384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d819a1d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006ba0 tx=0x7f10d4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.086+0000 7f10cf7fe700 1 -- 172.21.15.102:0/10702384 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10d4003b00 con 0x7f10d8107040 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.086+0000 7f10cf7fe700 1 -- 172.21.15.102:0/10702384 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f10d401a8e0 con 0x7f10d8107040 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.086+0000 7f10e05f5700 1 -- 172.21.15.102:0/10702384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d819a910 con 0x7f10d8107040 2024-05-13T22:39:18.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.086+0000 7f10cf7fe700 1 -- 172.21.15.102:0/10702384 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10d40199c0 con 0x7f10d8107040 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.086+0000 7f10e05f5700 1 -- 172.21.15.102:0/10702384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d819adb0 con 0x7f10d8107040 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.087+0000 7f10cf7fe700 1 -- 172.21.15.102:0/10702384 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f10d4019c30 con 0x7f10d8107040 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.087+0000 7f10e05f5700 1 -- 172.21.15.102:0/10702384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d810a700 con 0x7f10d8107040 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.088+0000 7f10cf7fe700 1 --2- 172.21.15.102:0/10702384 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f10c4038370 0x7f10c403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.088+0000 7f10cf7fe700 1 -- 172.21.15.102:0/10702384 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f10d404c6a0 con 0x7f10d8107040 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.088+0000 7f10ddb90700 1 --2- 172.21.15.102:0/10702384 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f10c4038370 0x7f10c403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.088+0000 7f10ddb90700 1 --2- 172.21.15.102:0/10702384 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f10c4038370 0x7f10c403a830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f10c8006fd0 tx=0x7f10c8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.092+0000 7f10cf7fe700 1 -- 172.21.15.102:0/10702384 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f10d402b930 con 0x7f10d8107040 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.303+0000 7f10e05f5700 1 -- 172.21.15.102:0/10702384 --> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f10d80621e0 con 0x7f10c4038370 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.529+0000 7f10cf7fe700 1 -- 172.21.15.102:0/10702384 <== mgr.14164 v2:172.21.15.102:6800/263535069 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f10d80621e0 con 0x7f10c4038370 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.531+0000 7f10e05f5700 1 -- 172.21.15.102:0/10702384 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f10c4038370 msgr2=0x7f10c403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:18.090 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.531+0000 7f10e05f5700 1 --2- 172.21.15.102:0/10702384 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f10c4038370 0x7f10c403a830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f10c8006fd0 tx=0x7f10c8006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:18.091 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.531+0000 7f10e05f5700 1 -- 172.21.15.102:0/10702384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 msgr2=0x7f10d819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:18.091 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.531+0000 7f10e05f5700 1 --2- 172.21.15.102:0/10702384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d819a1d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006ba0 tx=0x7f10d4011b50 comp rx=0 tx=0).stop 2024-05-13T22:39:18.091 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.531+0000 7f10e05f5700 1 -- 172.21.15.102:0/10702384 shutdown_connections 2024-05-13T22:39:18.091 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.531+0000 7f10e05f5700 1 --2- 172.21.15.102:0/10702384 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f10c4038370 0x7f10c403a830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:18.091 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.531+0000 7f10e05f5700 1 --2- 172.21.15.102:0/10702384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10d8107040 0x7f10d819a1d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:18.091 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.531+0000 7f10e05f5700 1 -- 172.21.15.102:0/10702384 >> 172.21.15.102:0/10702384 conn(0x7f10d81009f0 msgr2=0x7f10d81017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:18.091 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.531+0000 7f10e05f5700 1 -- 172.21.15.102:0/10702384 shutdown_connections 2024-05-13T22:39:18.091 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:17.531+0000 7f10e05f5700 1 -- 172.21.15.102:0/10702384 wait complete. 2024-05-13T22:39:18.091 INFO:teuthology.orchestra.run.smithi102.stdout:Fetching dashboard port number... 2024-05-13T22:39:18.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:18 smithi102 ceph-mon[28599]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:39:18.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:18 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2905970086' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-13T22:39:19.318 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 8443 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.516+0000 7fd7b1886700 1 Processor -- start 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.516+0000 7fd7b1886700 1 -- start start 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.517+0000 7fd7b1886700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac105890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.517+0000 7fd7b1886700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac105e60 con 0x7fd7ac105470 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.517+0000 7fd7aaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac105890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.517+0000 7fd7aaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac105890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54716/0 (socket says 172.21.15.102:54716) 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.517+0000 7fd7aaffd700 1 -- 172.21.15.102:0/2329623924 learned_addr learned my addr 172.21.15.102:0/2329623924 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.517+0000 7fd7aaffd700 1 -- 172.21.15.102:0/2329623924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac106670 con 0x7fd7ac105470 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.518+0000 7fd7aaffd700 1 --2- 172.21.15.102:0/2329623924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac105890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0098a0 tx=0x7fd79c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43ac4010b4ce788 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.518+0000 7fd7a9ffb700 1 -- 172.21.15.102:0/2329623924 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd79c004180 con 0x7fd7ac105470 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.518+0000 7fd7a9ffb700 1 -- 172.21.15.102:0/2329623924 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd79c0042e0 con 0x7fd7ac105470 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.518+0000 7fd7a9ffb700 1 -- 172.21.15.102:0/2329623924 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd79c011af0 con 0x7fd7ac105470 2024-05-13T22:39:19.319 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.519+0000 7fd7b1886700 1 -- 172.21.15.102:0/2329623924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 msgr2=0x7fd7ac105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.519+0000 7fd7b1886700 1 --2- 172.21.15.102:0/2329623924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac105890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0098a0 tx=0x7fd79c009bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.519+0000 7fd7b1886700 1 -- 172.21.15.102:0/2329623924 shutdown_connections 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.519+0000 7fd7b1886700 1 --2- 172.21.15.102:0/2329623924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac105890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.519+0000 7fd7b1886700 1 -- 172.21.15.102:0/2329623924 >> 172.21.15.102:0/2329623924 conn(0x7fd7ac1009f0 msgr2=0x7fd7ac102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.519+0000 7fd7b1886700 1 -- 172.21.15.102:0/2329623924 shutdown_connections 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.519+0000 7fd7b1886700 1 -- 172.21.15.102:0/2329623924 wait complete. 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.519+0000 7fd7b1886700 1 Processor -- start 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.519+0000 7fd7b1886700 1 -- start start 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7b1886700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac190660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7b1886700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac190ba0 con 0x7fd7ac105470 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7aaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac190660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:19.320 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7aaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac190660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:54718/0 (socket says 172.21.15.102:54718) 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7aaffd700 1 -- 172.21.15.102:0/2905970086 learned_addr learned my addr 172.21.15.102:0/2905970086 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7aaffd700 1 -- 172.21.15.102:0/2905970086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c009550 con 0x7fd7ac105470 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7aaffd700 1 --2- 172.21.15.102:0/2905970086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac190660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd79c000c00 tx=0x7fd79c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7b0884700 1 -- 172.21.15.102:0/2905970086 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd79c003a70 con 0x7fd7ac105470 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7b0884700 1 -- 172.21.15.102:0/2905970086 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd79c003bd0 con 0x7fd7ac105470 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7b0884700 1 -- 172.21.15.102:0/2905970086 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd79c0238c0 con 0x7fd7ac105470 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7b1886700 1 -- 172.21.15.102:0/2905970086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ac18ecb0 con 0x7fd7ac105470 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.520+0000 7fd7b1886700 1 -- 172.21.15.102:0/2905970086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac18f150 con 0x7fd7ac105470 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.521+0000 7fd7b1886700 1 -- 172.21.15.102:0/2905970086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ac0623c0 con 0x7fd7ac105470 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.521+0000 7fd7b0884700 1 -- 172.21.15.102:0/2905970086 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fd79c023af0 con 0x7fd7ac105470 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.522+0000 7fd7b0884700 1 --2- 172.21.15.102:0/2905970086 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd794038370 0x7fd79403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.522+0000 7fd7b0884700 1 -- 172.21.15.102:0/2905970086 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd79c04c380 con 0x7fd7ac105470 2024-05-13T22:39:19.321 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.522+0000 7fd7aa7fc700 1 --2- 172.21.15.102:0/2905970086 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd794038370 0x7fd79403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.523+0000 7fd7aa7fc700 1 --2- 172.21.15.102:0/2905970086 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd794038370 0x7fd79403a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0006fd0 tx=0x7fd7a0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.528+0000 7fd7b0884700 1 -- 172.21.15.102:0/2905970086 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd79c010970 con 0x7fd7ac105470 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.737+0000 7fd7b1886700 1 -- 172.21.15.102:0/2905970086 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fd7ac18fad0 con 0x7fd7ac105470 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.738+0000 7fd7b0884700 1 -- 172.21.15.102:0/2905970086 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fd79c010380 con 0x7fd7ac105470 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.740+0000 7fd7b1886700 1 -- 172.21.15.102:0/2905970086 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd794038370 msgr2=0x7fd79403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.740+0000 7fd7b1886700 1 --2- 172.21.15.102:0/2905970086 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd794038370 0x7fd79403a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0006fd0 tx=0x7fd7a0006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.740+0000 7fd7b1886700 1 -- 172.21.15.102:0/2905970086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 msgr2=0x7fd7ac190660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.740+0000 7fd7b1886700 1 --2- 172.21.15.102:0/2905970086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac190660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd79c000c00 tx=0x7fd79c011b50 comp rx=0 tx=0).stop 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.740+0000 7fd7b1886700 1 -- 172.21.15.102:0/2905970086 shutdown_connections 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.740+0000 7fd7b1886700 1 --2- 172.21.15.102:0/2905970086 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd794038370 0x7fd79403a830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.740+0000 7fd7b1886700 1 --2- 172.21.15.102:0/2905970086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7ac105470 0x7fd7ac190660 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.740+0000 7fd7b1886700 1 -- 172.21.15.102:0/2905970086 >> 172.21.15.102:0/2905970086 conn(0x7fd7ac1009f0 msgr2=0x7fd7ac0740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.740+0000 7fd7b1886700 1 -- 172.21.15.102:0/2905970086 shutdown_connections 2024-05-13T22:39:19.322 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:18.740+0000 7fd7b1886700 1 -- 172.21.15.102:0/2905970086 wait complete. 2024-05-13T22:39:19.325 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-13T22:39:19.325 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-05-13T22:39:19.332 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-13T22:39:19.332 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-05-13T22:39:19.332 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-05-13T22:39:19.332 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-13T22:39:19.333 INFO:teuthology.orchestra.run.smithi102.stdout:Ceph Dashboard is now available at: 2024-05-13T22:39:19.333 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:39:19.333 INFO:teuthology.orchestra.run.smithi102.stdout: URL: https://smithi102.front.sepia.ceph.com:8443/ 2024-05-13T22:39:19.334 INFO:teuthology.orchestra.run.smithi102.stdout: User: admin 2024-05-13T22:39:19.334 INFO:teuthology.orchestra.run.smithi102.stdout: Password: mjfhs7nzd5 2024-05-13T22:39:19.334 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:39:19.334 INFO:teuthology.orchestra.run.smithi102.stdout:Saving cluster configuration to /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config directory 2024-05-13T22:39:19.334 INFO:teuthology.orchestra.run.smithi102.stdout:Enabling autotune for osd_memory_target 2024-05-13T22:39:21.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:20 smithi102 ceph-mon[28599]: mgrmap e13: smithi102.xcijrj(active, since 6s) 2024-05-13T22:39:21.247 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.568+0000 7fcb5259b700 1 Processor -- start 2024-05-13T22:39:21.247 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.568+0000 7fcb5259b700 1 -- start start 2024-05-13T22:39:21.247 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.568+0000 7fcb5259b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c106e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:21.247 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.568+0000 7fcb5259b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c107460 con 0x7fcb4c106a70 2024-05-13T22:39:21.247 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.569+0000 7fcb4bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c106e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:21.247 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.569+0000 7fcb4bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c106e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34836/0 (socket says 172.21.15.102:34836) 2024-05-13T22:39:21.247 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.569+0000 7fcb4bfff700 1 -- 172.21.15.102:0/3724432908 learned_addr learned my addr 172.21.15.102:0/3724432908 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:21.247 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.569+0000 7fcb4bfff700 1 -- 172.21.15.102:0/3724432908 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c105680 con 0x7fcb4c106a70 2024-05-13T22:39:21.247 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.569+0000 7fcb4bfff700 1 --2- 172.21.15.102:0/3724432908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c106e90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00bb40 tx=0x7fcb3c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b029f7ae65eca62 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.569+0000 7fcb4affd700 1 -- 172.21.15.102:0/3724432908 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb3c00dc00 con 0x7fcb4c106a70 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.569+0000 7fcb4affd700 1 -- 172.21.15.102:0/3724432908 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c0044e0 con 0x7fcb4c106a70 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.569+0000 7fcb4affd700 1 -- 172.21.15.102:0/3724432908 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb3c010470 con 0x7fcb4c106a70 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.570+0000 7fcb5259b700 1 -- 172.21.15.102:0/3724432908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 msgr2=0x7fcb4c106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.570+0000 7fcb5259b700 1 --2- 172.21.15.102:0/3724432908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c106e90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00bb40 tx=0x7fcb3c00be50 comp rx=0 tx=0).stop 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.570+0000 7fcb5259b700 1 -- 172.21.15.102:0/3724432908 shutdown_connections 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.570+0000 7fcb5259b700 1 --2- 172.21.15.102:0/3724432908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c106e90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.570+0000 7fcb5259b700 1 -- 172.21.15.102:0/3724432908 >> 172.21.15.102:0/3724432908 conn(0x7fcb4c1001d0 msgr2=0x7fcb4c1025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.570+0000 7fcb5259b700 1 -- 172.21.15.102:0/3724432908 shutdown_connections 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.570+0000 7fcb5259b700 1 -- 172.21.15.102:0/3724432908 wait complete. 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.570+0000 7fcb5259b700 1 Processor -- start 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.570+0000 7fcb5259b700 1 -- start start 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.571+0000 7fcb5259b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:21.248 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.571+0000 7fcb5259b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c1985d0 con 0x7fcb4c106a70 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.571+0000 7fcb4bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.571+0000 7fcb4bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34844/0 (socket says 172.21.15.102:34844) 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.571+0000 7fcb4bfff700 1 -- 172.21.15.102:0/4142334802 learned_addr learned my addr 172.21.15.102:0/4142334802 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.571+0000 7fcb4bfff700 1 -- 172.21.15.102:0/4142334802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c00b7f0 con 0x7fcb4c106a70 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.571+0000 7fcb4bfff700 1 --2- 172.21.15.102:0/4142334802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c198090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c003ba0 tx=0x7fcb3c004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.571+0000 7fcb497fa700 1 -- 172.21.15.102:0/4142334802 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb3c01e070 con 0x7fcb4c106a70 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.572+0000 7fcb497fa700 1 -- 172.21.15.102:0/4142334802 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c00de00 con 0x7fcb4c106a70 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.572+0000 7fcb5259b700 1 -- 172.21.15.102:0/4142334802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4c1987d0 con 0x7fcb4c106a70 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.572+0000 7fcb497fa700 1 -- 172.21.15.102:0/4142334802 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb3c022970 con 0x7fcb4c106a70 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.572+0000 7fcb5259b700 1 -- 172.21.15.102:0/4142334802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4c198c70 con 0x7fcb4c106a70 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.572+0000 7fcb497fa700 1 -- 172.21.15.102:0/4142334802 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fcb3c02c440 con 0x7fcb4c106a70 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.572+0000 7fcb5259b700 1 -- 172.21.15.102:0/4142334802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4c191d20 con 0x7fcb4c106a70 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.573+0000 7fcb497fa700 1 --2- 172.21.15.102:0/4142334802 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcb34038410 0x7fcb3403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.573+0000 7fcb497fa700 1 -- 172.21.15.102:0/4142334802 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcb3c04e890 con 0x7fcb4c106a70 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.573+0000 7fcb4b7fe700 1 --2- 172.21.15.102:0/4142334802 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcb34038410 0x7fcb3403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.573+0000 7fcb4b7fe700 1 --2- 172.21.15.102:0/4142334802 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcb34038410 0x7fcb3403a8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb40006fd0 tx=0x7fcb40006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.577+0000 7fcb497fa700 1 -- 172.21.15.102:0/4142334802 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcb3c02a930 con 0x7fcb4c106a70 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.763+0000 7fcb5259b700 1 -- 172.21.15.102:0/4142334802 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fcb4c02d090 con 0x7fcb4c106a70 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.763+0000 7fcb497fa700 1 -- 172.21.15.102:0/4142334802 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fcb3c027030 con 0x7fcb4c106a70 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.765+0000 7fcb5259b700 1 -- 172.21.15.102:0/4142334802 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcb34038410 msgr2=0x7fcb3403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.765+0000 7fcb5259b700 1 --2- 172.21.15.102:0/4142334802 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcb34038410 0x7fcb3403a8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb40006fd0 tx=0x7fcb40006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.765+0000 7fcb5259b700 1 -- 172.21.15.102:0/4142334802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 msgr2=0x7fcb4c198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.765+0000 7fcb5259b700 1 --2- 172.21.15.102:0/4142334802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c198090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c003ba0 tx=0x7fcb3c004070 comp rx=0 tx=0).stop 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.766+0000 7fcb5259b700 1 -- 172.21.15.102:0/4142334802 shutdown_connections 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.766+0000 7fcb5259b700 1 --2- 172.21.15.102:0/4142334802 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcb34038410 0x7fcb3403a8d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.766+0000 7fcb5259b700 1 --2- 172.21.15.102:0/4142334802 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb4c106a70 0x7fcb4c198090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:21.250 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.766+0000 7fcb5259b700 1 -- 172.21.15.102:0/4142334802 >> 172.21.15.102:0/4142334802 conn(0x7fcb4c1001d0 msgr2=0x7fcb4c18ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:21.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.766+0000 7fcb5259b700 1 -- 172.21.15.102:0/4142334802 shutdown_connections 2024-05-13T22:39:21.251 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:20.766+0000 7fcb5259b700 1 -- 172.21.15.102:0/4142334802 wait complete. 2024-05-13T22:39:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.678+0000 7f3449259700 1 Processor -- start 2024-05-13T22:39:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.679+0000 7f3449259700 1 -- start start 2024-05-13T22:39:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.679+0000 7f3449259700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f3444106f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.679+0000 7f3449259700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34441074e0 con 0x7f3444106af0 2024-05-13T22:39:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.679+0000 7f3442d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f3444106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.679+0000 7f3442d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f3444106f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34850/0 (socket says 172.21.15.102:34850) 2024-05-13T22:39:22.462 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.679+0000 7f3442d9d700 1 -- 172.21.15.102:0/2756678098 learned_addr learned my addr 172.21.15.102:0/2756678098 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.679+0000 7f3442d9d700 1 -- 172.21.15.102:0/2756678098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3444107620 con 0x7f3444106af0 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.679+0000 7f3442d9d700 1 --2- 172.21.15.102:0/2756678098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f3444106f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f34340098a0 tx=0x7f3434009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44b06b582229daf server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.680+0000 7f3441d9b700 1 -- 172.21.15.102:0/2756678098 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3434004180 con 0x7f3444106af0 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.680+0000 7f3441d9b700 1 -- 172.21.15.102:0/2756678098 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f34340042e0 con 0x7f3444106af0 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.680+0000 7f3441d9b700 1 -- 172.21.15.102:0/2756678098 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3434011af0 con 0x7f3444106af0 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.680+0000 7f3449259700 1 -- 172.21.15.102:0/2756678098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 msgr2=0x7f3444106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.680+0000 7f3449259700 1 --2- 172.21.15.102:0/2756678098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f3444106f10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f34340098a0 tx=0x7f3434009bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.680+0000 7f3449259700 1 -- 172.21.15.102:0/2756678098 shutdown_connections 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.680+0000 7f3449259700 1 --2- 172.21.15.102:0/2756678098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f3444106f10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.680+0000 7f3449259700 1 -- 172.21.15.102:0/2756678098 >> 172.21.15.102:0/2756678098 conn(0x7f3444100190 msgr2=0x7f34441025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.680+0000 7f3449259700 1 -- 172.21.15.102:0/2756678098 shutdown_connections 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.680+0000 7f3449259700 1 -- 172.21.15.102:0/2756678098 wait complete. 2024-05-13T22:39:22.463 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.681+0000 7f3449259700 1 Processor -- start 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.681+0000 7f3449259700 1 -- start start 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.681+0000 7f3449259700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f344419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.681+0000 7f3449259700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344419c900 con 0x7f3444106af0 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.681+0000 7f3442d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f344419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.681+0000 7f3442d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f344419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34852/0 (socket says 172.21.15.102:34852) 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.681+0000 7f3442d9d700 1 -- 172.21.15.102:0/4114730578 learned_addr learned my addr 172.21.15.102:0/4114730578 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.682+0000 7f3442d9d700 1 -- 172.21.15.102:0/4114730578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434009550 con 0x7f3444106af0 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.682+0000 7f3442d9d700 1 --2- 172.21.15.102:0/4114730578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f344419c3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3434011ad0 tx=0x7f3434011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.682+0000 7f343bfff700 1 -- 172.21.15.102:0/4114730578 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3434003ab0 con 0x7f3444106af0 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.682+0000 7f343bfff700 1 -- 172.21.15.102:0/4114730578 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3434003c10 con 0x7f3444106af0 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.682+0000 7f343bfff700 1 -- 172.21.15.102:0/4114730578 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34340198e0 con 0x7f3444106af0 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.682+0000 7f3449259700 1 -- 172.21.15.102:0/4114730578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344419cb00 con 0x7f3444106af0 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.682+0000 7f3449259700 1 -- 172.21.15.102:0/4114730578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344419cfa0 con 0x7f3444106af0 2024-05-13T22:39:22.464 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.683+0000 7f343bfff700 1 -- 172.21.15.102:0/4114730578 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f3434019b50 con 0x7f3444106af0 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.683+0000 7f3449259700 1 -- 172.21.15.102:0/4114730578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3444196150 con 0x7f3444106af0 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.683+0000 7f343bfff700 1 --2- 172.21.15.102:0/4114730578 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f3430038460 0x7f343003a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.683+0000 7f343bfff700 1 -- 172.21.15.102:0/4114730578 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f343404c460 con 0x7f3444106af0 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.683+0000 7f344259c700 1 --2- 172.21.15.102:0/4114730578 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f3430038460 0x7f343003a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.684+0000 7f344259c700 1 --2- 172.21.15.102:0/4114730578 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f3430038460 0x7f343003a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f342c006fd0 tx=0x7f342c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.688+0000 7f343bfff700 1 -- 172.21.15.102:0/4114730578 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f343402b300 con 0x7f3444106af0 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.982+0000 7f3449259700 1 -- 172.21.15.102:0/4114730578 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f344402cc70 con 0x7f3444106af0 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.984+0000 7f343bfff700 1 -- 172.21.15.102:0/4114730578 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f3434010b70 con 0x7f3444106af0 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.987+0000 7f3449259700 1 -- 172.21.15.102:0/4114730578 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f3430038460 msgr2=0x7f343003a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.987+0000 7f3449259700 1 --2- 172.21.15.102:0/4114730578 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f3430038460 0x7f343003a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f342c006fd0 tx=0x7f342c006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.987+0000 7f3449259700 1 -- 172.21.15.102:0/4114730578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 msgr2=0x7f344419c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.987+0000 7f3449259700 1 --2- 172.21.15.102:0/4114730578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f344419c3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3434011ad0 tx=0x7f3434011bb0 comp rx=0 tx=0).stop 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.987+0000 7f3449259700 1 -- 172.21.15.102:0/4114730578 shutdown_connections 2024-05-13T22:39:22.465 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.987+0000 7f3449259700 1 --2- 172.21.15.102:0/4114730578 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f3430038460 0x7f343003a920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.987+0000 7f3449259700 1 --2- 172.21.15.102:0/4114730578 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3444106af0 0x7f344419c3c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.987+0000 7f3449259700 1 -- 172.21.15.102:0/4114730578 >> 172.21.15.102:0/4114730578 conn(0x7f3444100190 msgr2=0x7f3444193290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.987+0000 7f3449259700 1 -- 172.21.15.102:0/4114730578 shutdown_connections 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-05-13T22:39:21.987+0000 7f3449259700 1 -- 172.21.15.102:0/4114730578 wait complete. 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout:Or, if you are only running a single cluster on this host: 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:39:22.466 INFO:teuthology.orchestra.run.smithi102.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-13T22:39:22.467 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:39:22.467 INFO:teuthology.orchestra.run.smithi102.stdout: ceph telemetry on 2024-05-13T22:39:22.467 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:39:22.467 INFO:teuthology.orchestra.run.smithi102.stdout:For more information see: 2024-05-13T22:39:22.467 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:39:22.467 INFO:teuthology.orchestra.run.smithi102.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-13T22:39:22.467 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:39:22.467 INFO:teuthology.orchestra.run.smithi102.stdout:Bootstrap complete. 2024-05-13T22:39:22.493 INFO:tasks.cephadm:Fetching config... 2024-05-13T22:39:22.493 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:39:22.493 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-13T22:39:22.509 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-13T22:39:22.510 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:39:22.510 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-13T22:39:22.566 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-13T22:39:22.566 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:39:22.566 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/keyring of=/dev/stdout 2024-05-13T22:39:22.630 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-13T22:39:22.631 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:39:22.631 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-13T22:39:22.687 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-13T22:39:22.687 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGf9rikECSrcy4I6yD+IeGLptnhpU/3IoIlfHjXRW5m+WoWYqndYUAeDbUBmqFKjdZd/IX3WM+5d8AffQV2T4o4Sao9YOqSMxyBAAyQsD86pzwtnWrA2LRAPJIPs/pSLb/KsprC9hJeNHLpnhTQNDpTvehouztrXQSZdXwfis7DmDi2SqU8u2EeX13+nPcYB+OO1miwlUso+KAVnVYliTfnWoGi0AC2FEzU3oKF5OsCf74PqCVgMDsZjXCWSz9rV2Y2qUcVQN+Y5KStpfUpizwxQgoGbTgyM34kBcDMrbqXtnbob1KDt/zoZy6FduQ88Y9UOQAo2xx4ZW2XQZ467D674vUi+QJqpOq6BZJJC/mkPkZNN5U1ixxUNLuOI/rDfKQehII8noRWfNil3n+ochLYXBh1CfnIpGkdetgTJfe1BPJEaQlUTV8HaZVnnsNIcr/3ldN7ibE1ldb2n3Eq47Kwi0syCDFe8yyHfaVEGp4akJlKMVbsJ7E2Wu5Cz454QE= ceph-62275d5c-1179-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-13T22:39:22.762 INFO:teuthology.orchestra.run.smithi102.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGf9rikECSrcy4I6yD+IeGLptnhpU/3IoIlfHjXRW5m+WoWYqndYUAeDbUBmqFKjdZd/IX3WM+5d8AffQV2T4o4Sao9YOqSMxyBAAyQsD86pzwtnWrA2LRAPJIPs/pSLb/KsprC9hJeNHLpnhTQNDpTvehouztrXQSZdXwfis7DmDi2SqU8u2EeX13+nPcYB+OO1miwlUso+KAVnVYliTfnWoGi0AC2FEzU3oKF5OsCf74PqCVgMDsZjXCWSz9rV2Y2qUcVQN+Y5KStpfUpizwxQgoGbTgyM34kBcDMrbqXtnbob1KDt/zoZy6FduQ88Y9UOQAo2xx4ZW2XQZ467D674vUi+QJqpOq6BZJJC/mkPkZNN5U1ixxUNLuOI/rDfKQehII8noRWfNil3n+ochLYXBh1CfnIpGkdetgTJfe1BPJEaQlUTV8HaZVnnsNIcr/3ldN7ibE1ldb2n3Eq47Kwi0syCDFe8yyHfaVEGp4akJlKMVbsJ7E2Wu5Cz454QE= ceph-62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:39:22.777 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGf9rikECSrcy4I6yD+IeGLptnhpU/3IoIlfHjXRW5m+WoWYqndYUAeDbUBmqFKjdZd/IX3WM+5d8AffQV2T4o4Sao9YOqSMxyBAAyQsD86pzwtnWrA2LRAPJIPs/pSLb/KsprC9hJeNHLpnhTQNDpTvehouztrXQSZdXwfis7DmDi2SqU8u2EeX13+nPcYB+OO1miwlUso+KAVnVYliTfnWoGi0AC2FEzU3oKF5OsCf74PqCVgMDsZjXCWSz9rV2Y2qUcVQN+Y5KStpfUpizwxQgoGbTgyM34kBcDMrbqXtnbob1KDt/zoZy6FduQ88Y9UOQAo2xx4ZW2XQZ467D674vUi+QJqpOq6BZJJC/mkPkZNN5U1ixxUNLuOI/rDfKQehII8noRWfNil3n+ochLYXBh1CfnIpGkdetgTJfe1BPJEaQlUTV8HaZVnnsNIcr/3ldN7ibE1ldb2n3Eq47Kwi0syCDFe8yyHfaVEGp4akJlKMVbsJ7E2Wu5Cz454QE= ceph-62275d5c-1179-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-13T22:39:22.818 INFO:teuthology.orchestra.run.smithi130.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGf9rikECSrcy4I6yD+IeGLptnhpU/3IoIlfHjXRW5m+WoWYqndYUAeDbUBmqFKjdZd/IX3WM+5d8AffQV2T4o4Sao9YOqSMxyBAAyQsD86pzwtnWrA2LRAPJIPs/pSLb/KsprC9hJeNHLpnhTQNDpTvehouztrXQSZdXwfis7DmDi2SqU8u2EeX13+nPcYB+OO1miwlUso+KAVnVYliTfnWoGi0AC2FEzU3oKF5OsCf74PqCVgMDsZjXCWSz9rV2Y2qUcVQN+Y5KStpfUpizwxQgoGbTgyM34kBcDMrbqXtnbob1KDt/zoZy6FduQ88Y9UOQAo2xx4ZW2XQZ467D674vUi+QJqpOq6BZJJC/mkPkZNN5U1ixxUNLuOI/rDfKQehII8noRWfNil3n+ochLYXBh1CfnIpGkdetgTJfe1BPJEaQlUTV8HaZVnnsNIcr/3ldN7ibE1ldb2n3Eq47Kwi0syCDFe8yyHfaVEGp4akJlKMVbsJ7E2Wu5Cz454QE= ceph-62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:39:22.831 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-13T22:39:23.032 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:39:23.183 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:22 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/4114730578' entity='client.admin' 2024-05-13T22:39:25.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.108+0000 7efd235f0700 1 -- 172.21.15.102:0/4285542912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c0ff7a0 msgr2=0x7efd1c0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:25.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.108+0000 7efd235f0700 1 --2- 172.21.15.102:0/4285542912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c0ff7a0 0x7efd1c0ffbc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009910 tx=0x7efd0c009c20 comp rx=0 tx=0).stop 2024-05-13T22:39:25.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.108+0000 7efd235f0700 1 -- 172.21.15.102:0/4285542912 shutdown_connections 2024-05-13T22:39:25.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.108+0000 7efd235f0700 1 --2- 172.21.15.102:0/4285542912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c0ff7a0 0x7efd1c0ffbc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:25.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.108+0000 7efd235f0700 1 -- 172.21.15.102:0/4285542912 >> 172.21.15.102:0/4285542912 conn(0x7efd1c0fad20 msgr2=0x7efd1c0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:25.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.108+0000 7efd235f0700 1 -- 172.21.15.102:0/4285542912 shutdown_connections 2024-05-13T22:39:25.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.108+0000 7efd235f0700 1 -- 172.21.15.102:0/4285542912 wait complete. 2024-05-13T22:39:25.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.109+0000 7efd235f0700 1 Processor -- start 2024-05-13T22:39:25.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.109+0000 7efd235f0700 1 -- start start 2024-05-13T22:39:25.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.109+0000 7efd235f0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c0ff7a0 0x7efd1c1948c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:25.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.109+0000 7efd235f0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c194e00 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.109+0000 7efd2138c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c0ff7a0 0x7efd1c1948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:25.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.109+0000 7efd2138c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c0ff7a0 0x7efd1c1948c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34870/0 (socket says 172.21.15.102:34870) 2024-05-13T22:39:25.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.109+0000 7efd2138c700 1 -- 172.21.15.102:0/1150159956 learned_addr learned my addr 172.21.15.102:0/1150159956 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:25.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.109+0000 7efd2138c700 1 -- 172.21.15.102:0/1150159956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c0095f0 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.110+0000 7efd2138c700 1 --2- 172.21.15.102:0/1150159956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c0ff7a0 0x7efd1c1948c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005b40 tx=0x7efd0c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:25.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.110+0000 7efd127fc700 1 -- 172.21.15.102:0/1150159956 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd0c01f070 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.110+0000 7efd127fc700 1 -- 172.21.15.102:0/1150159956 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efd0c00be10 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.110+0000 7efd235f0700 1 -- 172.21.15.102:0/1150159956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c195000 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.110+0000 7efd127fc700 1 -- 172.21.15.102:0/1150159956 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd0c024620 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.110+0000 7efd235f0700 1 -- 172.21.15.102:0/1150159956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c1954a0 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.111+0000 7efd127fc700 1 -- 172.21.15.102:0/1150159956 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7efd0c00f460 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.111+0000 7efd127fc700 1 --2- 172.21.15.102:0/1150159956 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7efd080383c0 0x7efd0803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:25.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.111+0000 7efd127fc700 1 -- 172.21.15.102:0/1150159956 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efd0c04e660 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.111+0000 7efd235f0700 1 -- 172.21.15.102:0/1150159956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c0623c0 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.111+0000 7efd20b8b700 1 --2- 172.21.15.102:0/1150159956 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7efd080383c0 0x7efd0803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:25.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.112+0000 7efd20b8b700 1 --2- 172.21.15.102:0/1150159956 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7efd080383c0 0x7efd0803a880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efd18006fd0 tx=0x7efd18006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:25.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.116+0000 7efd127fc700 1 -- 172.21.15.102:0/1150159956 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efd0c029070 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.313+0000 7efd235f0700 1 -- 172.21.15.102:0/1150159956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7efd1c18e270 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.316+0000 7efd127fc700 1 -- 172.21.15.102:0/1150159956 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7efd0c02c720 con 0x7efd1c0ff7a0 2024-05-13T22:39:25.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.318+0000 7efd235f0700 1 -- 172.21.15.102:0/1150159956 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7efd080383c0 msgr2=0x7efd0803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:25.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.318+0000 7efd235f0700 1 --2- 172.21.15.102:0/1150159956 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7efd080383c0 0x7efd0803a880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efd18006fd0 tx=0x7efd18006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:25.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.318+0000 7efd235f0700 1 -- 172.21.15.102:0/1150159956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c0ff7a0 msgr2=0x7efd1c1948c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:25.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.318+0000 7efd235f0700 1 --2- 172.21.15.102:0/1150159956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c0ff7a0 0x7efd1c1948c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005b40 tx=0x7efd0c004740 comp rx=0 tx=0).stop 2024-05-13T22:39:25.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.318+0000 7efd235f0700 1 -- 172.21.15.102:0/1150159956 shutdown_connections 2024-05-13T22:39:25.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.318+0000 7efd235f0700 1 --2- 172.21.15.102:0/1150159956 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7efd080383c0 0x7efd0803a880 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:25.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.318+0000 7efd235f0700 1 --2- 172.21.15.102:0/1150159956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd1c0ff7a0 0x7efd1c1948c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:25.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.318+0000 7efd235f0700 1 -- 172.21.15.102:0/1150159956 >> 172.21.15.102:0/1150159956 conn(0x7efd1c0fad20 msgr2=0x7efd1c0fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:25.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.319+0000 7efd235f0700 1 -- 172.21.15.102:0/1150159956 shutdown_connections 2024-05-13T22:39:25.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:25.319+0000 7efd235f0700 1 -- 172.21.15.102:0/1150159956 wait complete. 2024-05-13T22:39:26.150 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-13T22:39:26.150 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-13T22:39:26.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:26 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1150159956' entity='client.admin' 2024-05-13T22:39:26.902 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:39:28.153 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:27 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:28.153 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:27 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:28.153 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:27 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:39:28.153 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:27 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:28.153 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:27 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:39:28.153 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:27 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-13T22:39:28.153 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:27 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:39:28.153 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:27 smithi102 ceph-mon[28599]: Deploying daemon ceph-exporter.smithi102 on smithi102 2024-05-13T22:39:29.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.392+0000 7fa8525b2700 1 -- 172.21.15.102:0/2981923331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c0ff7a0 msgr2=0x7fa84c0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:29.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.392+0000 7fa8525b2700 1 --2- 172.21.15.102:0/2981923331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c0ff7a0 0x7fa84c0ffbc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa840009910 tx=0x7fa840009c20 comp rx=0 tx=0).stop 2024-05-13T22:39:29.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.392+0000 7fa8525b2700 1 -- 172.21.15.102:0/2981923331 shutdown_connections 2024-05-13T22:39:29.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.392+0000 7fa8525b2700 1 --2- 172.21.15.102:0/2981923331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c0ff7a0 0x7fa84c0ffbc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:29.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.392+0000 7fa8525b2700 1 -- 172.21.15.102:0/2981923331 >> 172.21.15.102:0/2981923331 conn(0x7fa84c0fad20 msgr2=0x7fa84c0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:29.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.392+0000 7fa8525b2700 1 -- 172.21.15.102:0/2981923331 shutdown_connections 2024-05-13T22:39:29.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.392+0000 7fa8525b2700 1 -- 172.21.15.102:0/2981923331 wait complete. 2024-05-13T22:39:29.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.393+0000 7fa8525b2700 1 Processor -- start 2024-05-13T22:39:29.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.393+0000 7fa8525b2700 1 -- start start 2024-05-13T22:39:29.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.393+0000 7fa8525b2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c0ff7a0 0x7fa84c0fcce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:29.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.393+0000 7fa8525b2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c0fd220 con 0x7fa84c0ff7a0 2024-05-13T22:39:29.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.393+0000 7fa84bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c0ff7a0 0x7fa84c0fcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:29.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.394+0000 7fa84bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c0ff7a0 0x7fa84c0fcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33372/0 (socket says 172.21.15.102:33372) 2024-05-13T22:39:29.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.394+0000 7fa84bfff700 1 -- 172.21.15.102:0/4229432905 learned_addr learned my addr 172.21.15.102:0/4229432905 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:29.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.394+0000 7fa84bfff700 1 -- 172.21.15.102:0/4229432905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8400095f0 con 0x7fa84c0ff7a0 2024-05-13T22:39:29.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.394+0000 7fa84bfff700 1 --2- 172.21.15.102:0/4229432905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c0ff7a0 0x7fa84c0fcce0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa840005b40 tx=0x7fa840004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:29.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.394+0000 7fa8497fa700 1 -- 172.21.15.102:0/4229432905 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa84001d070 con 0x7fa84c0ff7a0 2024-05-13T22:39:29.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.394+0000 7fa8497fa700 1 -- 172.21.15.102:0/4229432905 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa84000be10 con 0x7fa84c0ff7a0 2024-05-13T22:39:29.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.394+0000 7fa8525b2700 1 -- 172.21.15.102:0/4229432905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c0fb330 con 0x7fa84c0ff7a0 2024-05-13T22:39:29.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.394+0000 7fa8497fa700 1 -- 172.21.15.102:0/4229432905 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa84000f460 con 0x7fa84c0ff7a0 2024-05-13T22:39:29.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.394+0000 7fa8525b2700 1 -- 172.21.15.102:0/4229432905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c0fb7d0 con 0x7fa84c0ff7a0 2024-05-13T22:39:29.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.395+0000 7fa8497fa700 1 -- 172.21.15.102:0/4229432905 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fa84000f5c0 con 0x7fa84c0ff7a0 2024-05-13T22:39:29.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.395+0000 7fa8525b2700 1 -- 172.21.15.102:0/4229432905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c0623c0 con 0x7fa84c0ff7a0 2024-05-13T22:39:29.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.395+0000 7fa8497fa700 1 --2- 172.21.15.102:0/4229432905 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fa834038410 0x7fa83403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:29.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.395+0000 7fa8497fa700 1 -- 172.21.15.102:0/4229432905 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa84004c380 con 0x7fa84c0ff7a0 2024-05-13T22:39:29.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.395+0000 7fa84b7fe700 1 --2- 172.21.15.102:0/4229432905 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fa834038410 0x7fa83403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:29.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.396+0000 7fa84b7fe700 1 --2- 172.21.15.102:0/4229432905 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fa834038410 0x7fa83403a8d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa83c006fd0 tx=0x7fa83c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:29.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.400+0000 7fa8497fa700 1 -- 172.21.15.102:0/4229432905 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa84002a360 con 0x7fa84c0ff7a0 2024-05-13T22:39:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.614+0000 7fa8525b2700 1 -- 172.21.15.102:0/4229432905 --> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fa84c0fbad0 con 0x7fa834038410 2024-05-13T22:39:29.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.618+0000 7fa8497fa700 1 -- 172.21.15.102:0/4229432905 <== mgr.14164 v2:172.21.15.102:6800/263535069 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa84c0fbad0 con 0x7fa834038410 2024-05-13T22:39:29.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.620+0000 7fa8525b2700 1 -- 172.21.15.102:0/4229432905 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fa834038410 msgr2=0x7fa83403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:29.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.620+0000 7fa8525b2700 1 --2- 172.21.15.102:0/4229432905 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fa834038410 0x7fa83403a8d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa83c006fd0 tx=0x7fa83c006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:29.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.620+0000 7fa8525b2700 1 -- 172.21.15.102:0/4229432905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c0ff7a0 msgr2=0x7fa84c0fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:29.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.620+0000 7fa8525b2700 1 --2- 172.21.15.102:0/4229432905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c0ff7a0 0x7fa84c0fcce0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa840005b40 tx=0x7fa840004740 comp rx=0 tx=0).stop 2024-05-13T22:39:29.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.620+0000 7fa8525b2700 1 -- 172.21.15.102:0/4229432905 shutdown_connections 2024-05-13T22:39:29.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.620+0000 7fa8525b2700 1 --2- 172.21.15.102:0/4229432905 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fa834038410 0x7fa83403a8d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:29.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.620+0000 7fa8525b2700 1 --2- 172.21.15.102:0/4229432905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa84c0ff7a0 0x7fa84c0fcce0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:29.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.620+0000 7fa8525b2700 1 -- 172.21.15.102:0/4229432905 >> 172.21.15.102:0/4229432905 conn(0x7fa84c0fad20 msgr2=0x7fa84c068810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:29.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.620+0000 7fa8525b2700 1 -- 172.21.15.102:0/4229432905 shutdown_connections 2024-05-13T22:39:29.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:29.620+0000 7fa8525b2700 1 -- 172.21.15.102:0/4229432905 wait complete. 2024-05-13T22:39:30.178 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi130 2024-05-13T22:39:30.179 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:39:30.179 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.conf 2024-05-13T22:39:30.198 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:39:30.198 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:39:30.256 INFO:tasks.cephadm:Adding host smithi130 to orchestrator... 2024-05-13T22:39:30.257 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph orch host add smithi130 2024-05-13T22:39:30.624 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:39:30.900 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:30 smithi102 ceph-mon[28599]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:39:30.900 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:30 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:31.830 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:31 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:31.830 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:31 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:31.830 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:31 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:31.831 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:31 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:31.831 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:31 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi102", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:39:31.831 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:31 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi102", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-13T22:39:31.831 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:31 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:39:31.831 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:31 smithi102 ceph-mon[28599]: Deploying daemon crash.smithi102 on smithi102 2024-05-13T22:39:32.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.717+0000 7f9f33310700 1 -- 172.21.15.102:0/2376884770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f2c0ff7a0 msgr2=0x7f9f2c0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.717+0000 7f9f33310700 1 --2- 172.21.15.102:0/2376884770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f2c0ff7a0 0x7f9f2c0ffbc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009910 tx=0x7f9f1c009c20 comp rx=0 tx=0).stop 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.717+0000 7f9f33310700 1 -- 172.21.15.102:0/2376884770 shutdown_connections 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.717+0000 7f9f33310700 1 --2- 172.21.15.102:0/2376884770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f2c0ff7a0 0x7f9f2c0ffbc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.717+0000 7f9f33310700 1 -- 172.21.15.102:0/2376884770 >> 172.21.15.102:0/2376884770 conn(0x7f9f2c0fad20 msgr2=0x7f9f2c0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.717+0000 7f9f33310700 1 -- 172.21.15.102:0/2376884770 shutdown_connections 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.717+0000 7f9f33310700 1 -- 172.21.15.102:0/2376884770 wait complete. 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f33310700 1 Processor -- start 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f33310700 1 -- start start 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f33310700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f2c0ff7a0 0x7f9f2c1959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f33310700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c195f00 con 0x7f9f2c0ff7a0 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f310ac700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f2c0ff7a0 0x7f9f2c1959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f310ac700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f2c0ff7a0 0x7f9f2c1959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33394/0 (socket says 172.21.15.102:33394) 2024-05-13T22:39:32.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f310ac700 1 -- 172.21.15.102:0/2029601411 learned_addr learned my addr 172.21.15.102:0/2029601411 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:32.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f310ac700 1 -- 172.21.15.102:0/2029601411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c0095f0 con 0x7f9f2c0ff7a0 2024-05-13T22:39:32.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f310ac700 1 --2- 172.21.15.102:0/2029601411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f2c0ff7a0 0x7f9f2c1959c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c005950 tx=0x7f9f1c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:32.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f2a7fc700 1 -- 172.21.15.102:0/2029601411 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f1c01d070 con 0x7f9f2c0ff7a0 2024-05-13T22:39:32.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f2a7fc700 1 -- 172.21.15.102:0/2029601411 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9f1c00be10 con 0x7f9f2c0ff7a0 2024-05-13T22:39:32.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f33310700 1 -- 172.21.15.102:0/2029601411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c196100 con 0x7f9f2c0ff7a0 2024-05-13T22:39:32.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.718+0000 7f9f2a7fc700 1 -- 172.21.15.102:0/2029601411 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f1c00f460 con 0x7f9f2c0ff7a0 2024-05-13T22:39:32.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.719+0000 7f9f33310700 1 -- 172.21.15.102:0/2029601411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c1965a0 con 0x7f9f2c0ff7a0 2024-05-13T22:39:32.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.719+0000 7f9f2a7fc700 1 -- 172.21.15.102:0/2029601411 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f9f1c00f690 con 0x7f9f2c0ff7a0 2024-05-13T22:39:32.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.719+0000 7f9f2a7fc700 1 --2- 172.21.15.102:0/2029601411 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f9f140383c0 0x7f9f1403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:32.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.719+0000 7f9f2a7fc700 1 -- 172.21.15.102:0/2029601411 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9f1c04c530 con 0x7f9f2c0ff7a0 2024-05-13T22:39:32.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.720+0000 7f9f308ab700 1 --2- 172.21.15.102:0/2029601411 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f9f140383c0 0x7f9f1403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:32.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.720+0000 7f9f33310700 1 -- 172.21.15.102:0/2029601411 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2c18f3d0 con 0x7f9f2c0ff7a0 2024-05-13T22:39:32.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.720+0000 7f9f308ab700 1 --2- 172.21.15.102:0/2029601411 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f9f140383c0 0x7f9f1403a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000ad30 tx=0x7f9f200093f0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:32.704 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.725+0000 7f9f2a7fc700 1 -- 172.21.15.102:0/2029601411 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9f1c027070 con 0x7f9f2c0ff7a0 2024-05-13T22:39:32.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:32.922+0000 7f9f33310700 1 -- 172.21.15.102:0/2029601411 --> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f2c0611d0 con 0x7f9f140383c0 2024-05-13T22:39:33.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:33 smithi102 ceph-mon[28599]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:39:35.011 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:34 smithi102 ceph-mon[28599]: Deploying cephadm binary to smithi130 2024-05-13T22:39:35.011 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:34 smithi102 ceph-mon[28599]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:35.011 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:34 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:35.011 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:34 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:35.011 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:34 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:35.012 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:34 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:35.012 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:34 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:35.012 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:34 smithi102 ceph-mon[28599]: Deploying daemon node-exporter.smithi102 on smithi102 2024-05-13T22:39:35.528 INFO:teuthology.orchestra.run.smithi102.stdout:Added host 'smithi130' with addr '172.21.15.130' 2024-05-13T22:39:35.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:35.547+0000 7f9f2a7fc700 1 -- 172.21.15.102:0/2029601411 <== mgr.14164 v2:172.21.15.102:6800/263535069 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9f2c0611d0 con 0x7f9f140383c0 2024-05-13T22:39:35.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:35.550+0000 7f9f33310700 1 -- 172.21.15.102:0/2029601411 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f9f140383c0 msgr2=0x7f9f1403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:35.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:35.550+0000 7f9f33310700 1 --2- 172.21.15.102:0/2029601411 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f9f140383c0 0x7f9f1403a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000ad30 tx=0x7f9f200093f0 comp rx=0 tx=0).stop 2024-05-13T22:39:35.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:35.550+0000 7f9f33310700 1 -- 172.21.15.102:0/2029601411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f2c0ff7a0 msgr2=0x7f9f2c1959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:35.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:35.550+0000 7f9f33310700 1 --2- 172.21.15.102:0/2029601411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f2c0ff7a0 0x7f9f2c1959c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c005950 tx=0x7f9f1c009e60 comp rx=0 tx=0).stop 2024-05-13T22:39:35.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:35.550+0000 7f9f33310700 1 -- 172.21.15.102:0/2029601411 shutdown_connections 2024-05-13T22:39:35.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:35.550+0000 7f9f33310700 1 --2- 172.21.15.102:0/2029601411 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f9f140383c0 0x7f9f1403a880 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:35.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:35.550+0000 7f9f33310700 1 --2- 172.21.15.102:0/2029601411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f2c0ff7a0 0x7f9f2c1959c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:35.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:35.550+0000 7f9f33310700 1 -- 172.21.15.102:0/2029601411 >> 172.21.15.102:0/2029601411 conn(0x7f9f2c0fad20 msgr2=0x7f9f2c069100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:35.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:35.550+0000 7f9f33310700 1 -- 172.21.15.102:0/2029601411 shutdown_connections 2024-05-13T22:39:35.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:35.550+0000 7f9f33310700 1 -- 172.21.15.102:0/2029601411 wait complete. 2024-05-13T22:39:36.380 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-13T22:39:36.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:36 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:36.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:36 smithi102 ceph-mon[28599]: Added host smithi130 2024-05-13T22:39:36.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:36 smithi102 ceph-mon[28599]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:37.099 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:39:38.983 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:38 smithi102 ceph-mon[28599]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:38.983 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:38 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:38.983 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:38 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:38.983 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:38 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:38.983 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:38 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:38.983 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:38 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:39.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.344+0000 7f0cd9e83700 1 -- 172.21.15.102:0/495105110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd41008a0 msgr2=0x7f0cd4100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:39.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.344+0000 7f0cd9e83700 1 --2- 172.21.15.102:0/495105110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd41008a0 0x7f0cd4100cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009910 tx=0x7f0cc4009c20 comp rx=0 tx=0).stop 2024-05-13T22:39:39.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.344+0000 7f0cd9e83700 1 -- 172.21.15.102:0/495105110 shutdown_connections 2024-05-13T22:39:39.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.344+0000 7f0cd9e83700 1 --2- 172.21.15.102:0/495105110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd41008a0 0x7f0cd4100cc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:39.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.344+0000 7f0cd9e83700 1 -- 172.21.15.102:0/495105110 >> 172.21.15.102:0/495105110 conn(0x7f0cd40fbe60 msgr2=0x7f0cd40fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:39.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.344+0000 7f0cd9e83700 1 -- 172.21.15.102:0/495105110 shutdown_connections 2024-05-13T22:39:39.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.344+0000 7f0cd9e83700 1 -- 172.21.15.102:0/495105110 wait complete. 2024-05-13T22:39:39.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.345+0000 7f0cd9e83700 1 Processor -- start 2024-05-13T22:39:39.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.345+0000 7f0cd9e83700 1 -- start start 2024-05-13T22:39:39.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.345+0000 7f0cd9e83700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd41008a0 0x7f0cd41959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:39.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.345+0000 7f0cd9e83700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd4195f00 con 0x7f0cd41008a0 2024-05-13T22:39:39.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.345+0000 7f0cd37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd41008a0 0x7f0cd41959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:39.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.345+0000 7f0cd37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd41008a0 0x7f0cd41959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36064/0 (socket says 172.21.15.102:36064) 2024-05-13T22:39:39.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.345+0000 7f0cd37fe700 1 -- 172.21.15.102:0/1253019900 learned_addr learned my addr 172.21.15.102:0/1253019900 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:39.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.346+0000 7f0cd37fe700 1 -- 172.21.15.102:0/1253019900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc40095f0 con 0x7f0cd41008a0 2024-05-13T22:39:39.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.346+0000 7f0cd37fe700 1 --2- 172.21.15.102:0/1253019900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd41008a0 0x7f0cd41959c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005b40 tx=0x7f0cc4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:39.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.346+0000 7f0cd17fa700 1 -- 172.21.15.102:0/1253019900 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cc401d070 con 0x7f0cd41008a0 2024-05-13T22:39:39.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.346+0000 7f0cd17fa700 1 -- 172.21.15.102:0/1253019900 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0cc400be10 con 0x7f0cd41008a0 2024-05-13T22:39:39.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.346+0000 7f0cd9e83700 1 -- 172.21.15.102:0/1253019900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd4196100 con 0x7f0cd41008a0 2024-05-13T22:39:39.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.346+0000 7f0cd17fa700 1 -- 172.21.15.102:0/1253019900 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cc400f460 con 0x7f0cd41008a0 2024-05-13T22:39:39.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.346+0000 7f0cd9e83700 1 -- 172.21.15.102:0/1253019900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd41965a0 con 0x7f0cd41008a0 2024-05-13T22:39:39.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.347+0000 7f0cd17fa700 1 -- 172.21.15.102:0/1253019900 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f0cc400f6d0 con 0x7f0cd41008a0 2024-05-13T22:39:39.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.347+0000 7f0cd9e83700 1 -- 172.21.15.102:0/1253019900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd418f4c0 con 0x7f0cd41008a0 2024-05-13T22:39:39.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.347+0000 7f0cd17fa700 1 --2- 172.21.15.102:0/1253019900 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cc00383c0 0x7f0cc003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:39.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.347+0000 7f0cd17fa700 1 -- 172.21.15.102:0/1253019900 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0cc404c550 con 0x7f0cd41008a0 2024-05-13T22:39:39.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.347+0000 7f0ccbfff700 1 --2- 172.21.15.102:0/1253019900 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cc00383c0 0x7f0cc003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:39.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.348+0000 7f0ccbfff700 1 --2- 172.21.15.102:0/1253019900 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cc00383c0 0x7f0cc003a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc006fd0 tx=0x7f0cbc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:39.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.352+0000 7f0cd17fa700 1 -- 172.21.15.102:0/1253019900 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0cc402a360 con 0x7f0cd41008a0 2024-05-13T22:39:39.522 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.543+0000 7f0cd9e83700 1 -- 172.21.15.102:0/1253019900 --> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0cd40611d0 con 0x7f0cc00383c0 2024-05-13T22:39:39.523 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.544+0000 7f0cd17fa700 1 -- 172.21.15.102:0/1253019900 <== mgr.14164 v2:172.21.15.102:6800/263535069 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f0cd40611d0 con 0x7f0cc00383c0 2024-05-13T22:39:39.523 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:39:39.523 INFO:teuthology.orchestra.run.smithi102.stdout:[{"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}, {"addr": "172.21.15.130", "hostname": "smithi130", "labels": [], "status": ""}] 2024-05-13T22:39:39.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.546+0000 7f0cd9e83700 1 -- 172.21.15.102:0/1253019900 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cc00383c0 msgr2=0x7f0cc003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:39.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.546+0000 7f0cd9e83700 1 --2- 172.21.15.102:0/1253019900 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cc00383c0 0x7f0cc003a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc006fd0 tx=0x7f0cbc006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:39.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.546+0000 7f0cd9e83700 1 -- 172.21.15.102:0/1253019900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd41008a0 msgr2=0x7f0cd41959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:39.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.546+0000 7f0cd9e83700 1 --2- 172.21.15.102:0/1253019900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd41008a0 0x7f0cd41959c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005b40 tx=0x7f0cc4004740 comp rx=0 tx=0).stop 2024-05-13T22:39:39.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.546+0000 7f0cd9e83700 1 -- 172.21.15.102:0/1253019900 shutdown_connections 2024-05-13T22:39:39.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.546+0000 7f0cd9e83700 1 --2- 172.21.15.102:0/1253019900 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cc00383c0 0x7f0cc003a880 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:39.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.546+0000 7f0cd9e83700 1 --2- 172.21.15.102:0/1253019900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd41008a0 0x7f0cd41959c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:39.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.546+0000 7f0cd9e83700 1 -- 172.21.15.102:0/1253019900 >> 172.21.15.102:0/1253019900 conn(0x7f0cd40fbe60 msgr2=0x7f0cd40fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:39.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.546+0000 7f0cd9e83700 1 -- 172.21.15.102:0/1253019900 shutdown_connections 2024-05-13T22:39:39.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:39.546+0000 7f0cd9e83700 1 -- 172.21.15.102:0/1253019900 wait complete. 2024-05-13T22:39:40.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:39 smithi102 ceph-mon[28599]: Deploying daemon alertmanager.smithi102 on smithi102 2024-05-13T22:39:40.562 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-13T22:39:40.563 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-13T22:39:40.765 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:39:41.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:41 smithi102 ceph-mon[28599]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-13T22:39:41.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:41 smithi102 ceph-mon[28599]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:43.330 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:43 smithi102 ceph-mon[28599]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:43.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.820+0000 7f2ad1b69700 1 -- 172.21.15.102:0/2874488657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2acc0ff7a0 msgr2=0x7f2acc0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:43.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.820+0000 7f2ad1b69700 1 --2- 172.21.15.102:0/2874488657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2acc0ff7a0 0x7f2acc0ffbc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2abc009910 tx=0x7f2abc009c20 comp rx=0 tx=0).stop 2024-05-13T22:39:43.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.820+0000 7f2ad1b69700 1 -- 172.21.15.102:0/2874488657 shutdown_connections 2024-05-13T22:39:43.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.820+0000 7f2ad1b69700 1 --2- 172.21.15.102:0/2874488657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2acc0ff7a0 0x7f2acc0ffbc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:43.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.820+0000 7f2ad1b69700 1 -- 172.21.15.102:0/2874488657 >> 172.21.15.102:0/2874488657 conn(0x7f2acc0fad20 msgr2=0x7f2acc0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:43.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.820+0000 7f2ad1b69700 1 -- 172.21.15.102:0/2874488657 shutdown_connections 2024-05-13T22:39:43.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.820+0000 7f2ad1b69700 1 -- 172.21.15.102:0/2874488657 wait complete. 2024-05-13T22:39:43.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.820+0000 7f2ad1b69700 1 Processor -- start 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.820+0000 7f2ad1b69700 1 -- start start 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2ad1b69700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2acc0ff7a0 0x7f2acc1948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2ad1b69700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc194de0 con 0x7f2acc0ff7a0 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2acb7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2acc0ff7a0 0x7f2acc1948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2acb7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2acc0ff7a0 0x7f2acc1948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36082/0 (socket says 172.21.15.102:36082) 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2acb7fe700 1 -- 172.21.15.102:0/251090776 learned_addr learned my addr 172.21.15.102:0/251090776 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2acb7fe700 1 -- 172.21.15.102:0/251090776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc0095f0 con 0x7f2acc0ff7a0 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2acb7fe700 1 --2- 172.21.15.102:0/251090776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2acc0ff7a0 0x7f2acc1948a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2abc005950 tx=0x7f2abc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2ac97fa700 1 -- 172.21.15.102:0/251090776 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2abc01d070 con 0x7f2acc0ff7a0 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2ac97fa700 1 -- 172.21.15.102:0/251090776 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2abc00be10 con 0x7f2acc0ff7a0 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2ac97fa700 1 -- 172.21.15.102:0/251090776 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2abc00f460 con 0x7f2acc0ff7a0 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2ad1b69700 1 -- 172.21.15.102:0/251090776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2acc194fe0 con 0x7f2acc0ff7a0 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.821+0000 7f2ad1b69700 1 -- 172.21.15.102:0/251090776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc195480 con 0x7f2acc0ff7a0 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.822+0000 7f2ac97fa700 1 -- 172.21.15.102:0/251090776 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f2abc00f690 con 0x7f2acc0ff7a0 2024-05-13T22:39:43.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.822+0000 7f2ad1b69700 1 -- 172.21.15.102:0/251090776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2acc18e3c0 con 0x7f2acc0ff7a0 2024-05-13T22:39:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.823+0000 7f2ac97fa700 1 --2- 172.21.15.102:0/251090776 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f2ab80383c0 0x7f2ab803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.823+0000 7f2ac97fa700 1 -- 172.21.15.102:0/251090776 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2abc04c420 con 0x7f2acc0ff7a0 2024-05-13T22:39:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.823+0000 7f2ac2ffd700 1 --2- 172.21.15.102:0/251090776 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f2ab80383c0 0x7f2ab803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.823+0000 7f2ac2ffd700 1 --2- 172.21.15.102:0/251090776 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f2ab80383c0 0x7f2ab803a880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4006fd0 tx=0x7f2ab4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:43.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:43.827+0000 7f2ac97fa700 1 -- 172.21.15.102:0/251090776 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2abc02bba0 con 0x7f2acc0ff7a0 2024-05-13T22:39:44.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.023+0000 7f2ad1b69700 1 -- 172.21.15.102:0/251090776 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f2acc02cc70 con 0x7f2acc0ff7a0 2024-05-13T22:39:44.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.035+0000 7f2ac97fa700 1 -- 172.21.15.102:0/251090776 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f2abc027070 con 0x7f2acc0ff7a0 2024-05-13T22:39:44.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.037+0000 7f2ad1b69700 1 -- 172.21.15.102:0/251090776 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f2ab80383c0 msgr2=0x7f2ab803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:44.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.037+0000 7f2ad1b69700 1 --2- 172.21.15.102:0/251090776 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f2ab80383c0 0x7f2ab803a880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4006fd0 tx=0x7f2ab4006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:44.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.037+0000 7f2ad1b69700 1 -- 172.21.15.102:0/251090776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2acc0ff7a0 msgr2=0x7f2acc1948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:44.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.037+0000 7f2ad1b69700 1 --2- 172.21.15.102:0/251090776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2acc0ff7a0 0x7f2acc1948a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2abc005950 tx=0x7f2abc009e60 comp rx=0 tx=0).stop 2024-05-13T22:39:44.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.037+0000 7f2ad1b69700 1 -- 172.21.15.102:0/251090776 shutdown_connections 2024-05-13T22:39:44.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.037+0000 7f2ad1b69700 1 --2- 172.21.15.102:0/251090776 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f2ab80383c0 0x7f2ab803a880 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:44.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.037+0000 7f2ad1b69700 1 --2- 172.21.15.102:0/251090776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2acc0ff7a0 0x7f2acc1948a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:44.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.037+0000 7f2ad1b69700 1 -- 172.21.15.102:0/251090776 >> 172.21.15.102:0/251090776 conn(0x7f2acc0fad20 msgr2=0x7f2acc0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:44.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.037+0000 7f2ad1b69700 1 -- 172.21.15.102:0/251090776 shutdown_connections 2024-05-13T22:39:44.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:39:44.037+0000 7f2ad1b69700 1 -- 172.21.15.102:0/251090776 wait complete. 2024-05-13T22:39:44.018 INFO:teuthology.orchestra.run.smithi102.stderr:adjusted tunables profile to default 2024-05-13T22:39:44.390 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:44 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/251090776' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-13T22:39:45.116 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:45 smithi102 ceph-mon[28599]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:45.116 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:45 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/251090776' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-13T22:39:45.116 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:45 smithi102 ceph-mon[28599]: osdmap e4: 0 total, 0 up, 0 in 2024-05-13T22:39:47.033 INFO:tasks.cephadm:Adding mon.smithi102 on smithi102 2024-05-13T22:39:47.033 INFO:tasks.cephadm:Adding mon.smithi130 on smithi130 2024-05-13T22:39:47.033 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph orch apply mon '2;smithi102:172.21.15.102=smithi102;smithi130:172.21.15.130=smithi130' 2024-05-13T22:39:47.236 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:39:47.280 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:47 smithi102 ceph-mon[28599]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:47.283 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:39:48.708 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.728+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3636682525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3281008b0 msgr2=0x7fd328100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:48.709 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.728+0000 7fd32dc2c700 1 --2- 172.21.15.130:0/3636682525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3281008b0 0x7fd328100cd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd318009910 tx=0x7fd318009c20 comp rx=0 tx=0).stop 2024-05-13T22:39:48.709 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.728+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3636682525 shutdown_connections 2024-05-13T22:39:48.709 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.728+0000 7fd32dc2c700 1 --2- 172.21.15.130:0/3636682525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3281008b0 0x7fd328100cd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:48.709 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.728+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3636682525 >> 172.21.15.130:0/3636682525 conn(0x7fd3280fbe30 msgr2=0x7fd3280fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:48.709 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.728+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3636682525 shutdown_connections 2024-05-13T22:39:48.709 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.728+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3636682525 wait complete. 2024-05-13T22:39:48.709 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.728+0000 7fd32dc2c700 1 Processor -- start 2024-05-13T22:39:48.709 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.728+0000 7fd32dc2c700 1 -- start start 2024-05-13T22:39:48.709 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.729+0000 7fd32dc2c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3281008b0 0x7fd328197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:48.709 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.729+0000 7fd32dc2c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd328198110 con 0x7fd3281008b0 2024-05-13T22:39:48.709 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.729+0000 7fd3277fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3281008b0 0x7fd328197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:48.710 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.729+0000 7fd3277fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3281008b0 0x7fd328197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:50390/0 (socket says 172.21.15.130:50390) 2024-05-13T22:39:48.710 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.729+0000 7fd3277fe700 1 -- 172.21.15.130:0/3628371510 learned_addr learned my addr 172.21.15.130:0/3628371510 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:39:48.710 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.729+0000 7fd3277fe700 1 -- 172.21.15.130:0/3628371510 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3180095f0 con 0x7fd3281008b0 2024-05-13T22:39:48.710 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.729+0000 7fd3277fe700 1 --2- 172.21.15.130:0/3628371510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3281008b0 0x7fd328197bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd318005b40 tx=0x7fd318004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:48.711 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.731+0000 7fd3257fa700 1 -- 172.21.15.130:0/3628371510 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd31801d070 con 0x7fd3281008b0 2024-05-13T22:39:48.711 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.731+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3628371510 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd328198310 con 0x7fd3281008b0 2024-05-13T22:39:48.711 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.731+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3628371510 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3281987b0 con 0x7fd3281008b0 2024-05-13T22:39:48.711 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.731+0000 7fd3257fa700 1 -- 172.21.15.130:0/3628371510 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd318005c00 con 0x7fd3281008b0 2024-05-13T22:39:48.711 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.731+0000 7fd3257fa700 1 -- 172.21.15.130:0/3628371510 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd318022c10 con 0x7fd3281008b0 2024-05-13T22:39:48.711 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.732+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3628371510 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd308005320 con 0x7fd3281008b0 2024-05-13T22:39:48.711 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.732+0000 7fd3257fa700 1 -- 172.21.15.130:0/3628371510 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fd318022600 con 0x7fd3281008b0 2024-05-13T22:39:48.712 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.732+0000 7fd3257fa700 1 --2- 172.21.15.130:0/3628371510 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd3140383c0 0x7fd31403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:48.712 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.732+0000 7fd31effd700 1 --2- 172.21.15.130:0/3628371510 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd3140383c0 0x7fd31403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:48.712 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.733+0000 7fd31effd700 1 --2- 172.21.15.130:0/3628371510 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd3140383c0 0x7fd31403a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd310006fd0 tx=0x7fd310006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:48.712 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.733+0000 7fd3257fa700 1 -- 172.21.15.130:0/3628371510 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd31804de40 con 0x7fd3281008b0 2024-05-13T22:39:48.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.738+0000 7fd3257fa700 1 -- 172.21.15.130:0/3628371510 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd31800be80 con 0x7fd3281008b0 2024-05-13T22:39:48.914 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.934+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3628371510 --> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi102:172.21.15.102=smithi102;smithi130:172.21.15.130=smithi130", "target": ["mon-mgr", ""]}) v1 -- 0x7fd308000ca0 con 0x7fd3140383c0 2024-05-13T22:39:48.919 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.939+0000 7fd3257fa700 1 -- 172.21.15.130:0/3628371510 <== mgr.14164 v2:172.21.15.102:6800/263535069 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd308000ca0 con 0x7fd3140383c0 2024-05-13T22:39:48.919 INFO:teuthology.orchestra.run.smithi130.stdout:Scheduled mon update... 2024-05-13T22:39:48.921 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.941+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3628371510 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd3140383c0 msgr2=0x7fd31403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:48.921 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.941+0000 7fd32dc2c700 1 --2- 172.21.15.130:0/3628371510 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd3140383c0 0x7fd31403a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd310006fd0 tx=0x7fd310006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:48.921 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.941+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3628371510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3281008b0 msgr2=0x7fd328197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:48.921 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.941+0000 7fd32dc2c700 1 --2- 172.21.15.130:0/3628371510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3281008b0 0x7fd328197bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd318005b40 tx=0x7fd318004740 comp rx=0 tx=0).stop 2024-05-13T22:39:48.921 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.942+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3628371510 shutdown_connections 2024-05-13T22:39:48.921 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.942+0000 7fd32dc2c700 1 --2- 172.21.15.130:0/3628371510 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd3140383c0 0x7fd31403a880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:48.921 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.942+0000 7fd32dc2c700 1 --2- 172.21.15.130:0/3628371510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3281008b0 0x7fd328197bd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:48.922 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.942+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3628371510 >> 172.21.15.130:0/3628371510 conn(0x7fd3280fbe30 msgr2=0x7fd3280fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:48.922 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.942+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3628371510 shutdown_connections 2024-05-13T22:39:48.922 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:48.942+0000 7fd32dc2c700 1 -- 172.21.15.130:0/3628371510 wait complete. 2024-05-13T22:39:49.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:49.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:49.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:49.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:49.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:49.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:49.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:49.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: Regenerating cephadm self-signed grafana TLS certificates 2024-05-13T22:39:49.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:49.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:49.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-13T22:39:49.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-13T22:39:49.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:49.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:48 smithi102 ceph-mon[28599]: Deploying daemon grafana.smithi102 on smithi102 2024-05-13T22:39:49.502 DEBUG:teuthology.orchestra.run.smithi130:mon.smithi130> sudo journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi130.service 2024-05-13T22:39:49.505 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:39:49.505 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:39:49.733 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:39:49.781 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:39:49.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:49 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:49.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:49 smithi102 ceph-mon[28599]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi102:172.21.15.102=smithi102;smithi130:172.21.15.130=smithi130", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:39:49.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:49 smithi102 ceph-mon[28599]: Saving service mon spec with placement smithi102:172.21.15.102=smithi102;smithi130:172.21.15.130=smithi130;count:2 2024-05-13T22:39:49.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:49 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:39:51.221 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.241+0000 7f97b2c7c700 1 -- 172.21.15.130:0/3549731879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97ac0ff7a0 msgr2=0x7f97ac0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:51.221 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.241+0000 7f97b2c7c700 1 --2- 172.21.15.130:0/3549731879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97ac0ff7a0 0x7f97ac0ffbc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009910 tx=0x7f97a4009c20 comp rx=0 tx=0).stop 2024-05-13T22:39:51.221 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.241+0000 7f97b2c7c700 1 -- 172.21.15.130:0/3549731879 shutdown_connections 2024-05-13T22:39:51.221 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.241+0000 7f97b2c7c700 1 --2- 172.21.15.130:0/3549731879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97ac0ff7a0 0x7f97ac0ffbc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:51.221 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.241+0000 7f97b2c7c700 1 -- 172.21.15.130:0/3549731879 >> 172.21.15.130:0/3549731879 conn(0x7f97ac0fad20 msgr2=0x7f97ac0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:51.221 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.241+0000 7f97b2c7c700 1 -- 172.21.15.130:0/3549731879 shutdown_connections 2024-05-13T22:39:51.221 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.241+0000 7f97b2c7c700 1 -- 172.21.15.130:0/3549731879 wait complete. 2024-05-13T22:39:51.223 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.242+0000 7f97b2c7c700 1 Processor -- start 2024-05-13T22:39:51.223 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.242+0000 7f97b2c7c700 1 -- start start 2024-05-13T22:39:51.223 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.242+0000 7f97b2c7c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97ac0ff7a0 0x7f97ac1948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:51.223 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.242+0000 7f97b2c7c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac194de0 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.223 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.243+0000 7f97b0a18700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97ac0ff7a0 0x7f97ac1948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:51.223 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.243+0000 7f97b0a18700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97ac0ff7a0 0x7f97ac1948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:40802/0 (socket says 172.21.15.130:40802) 2024-05-13T22:39:51.223 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.243+0000 7f97b0a18700 1 -- 172.21.15.130:0/1774073092 learned_addr learned my addr 172.21.15.130:0/1774073092 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:39:51.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.243+0000 7f97b0a18700 1 -- 172.21.15.130:0/1774073092 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a40095f0 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.243+0000 7f97b0a18700 1 --2- 172.21.15.130:0/1774073092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97ac0ff7a0 0x7f97ac1948a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f97a4005b40 tx=0x7f97a4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:51.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.243+0000 7f97a9ffb700 1 -- 172.21.15.130:0/1774073092 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97a401d070 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.243+0000 7f97b2c7c700 1 -- 172.21.15.130:0/1774073092 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac194fe0 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.243+0000 7f97a9ffb700 1 -- 172.21.15.130:0/1774073092 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f97a400be10 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.243+0000 7f97a9ffb700 1 -- 172.21.15.130:0/1774073092 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97a400f460 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.224 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.243+0000 7f97b2c7c700 1 -- 172.21.15.130:0/1774073092 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac195480 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.225 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.244+0000 7f97a9ffb700 1 -- 172.21.15.130:0/1774073092 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f97a400b960 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.225 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.244+0000 7f97b2c7c700 1 -- 172.21.15.130:0/1774073092 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac18e3c0 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.225 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.244+0000 7f97a9ffb700 1 --2- 172.21.15.130:0/1774073092 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f97980383c0 0x7f979803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:51.225 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.245+0000 7f97a9ffb700 1 -- 172.21.15.130:0/1774073092 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f97a404c4d0 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.225 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.245+0000 7f97abfff700 1 --2- 172.21.15.130:0/1774073092 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f97980383c0 0x7f979803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:51.226 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.245+0000 7f97abfff700 1 --2- 172.21.15.130:0/1774073092 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f97980383c0 0x7f979803a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f979c00ad30 tx=0x7f979c0093f0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:51.229 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.250+0000 7f97a9ffb700 1 -- 172.21.15.130:0/1774073092 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f97a402a360 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:51 smithi102 ceph-mon[28599]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:51.490 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.511+0000 7f97b2c7c700 1 -- 172.21.15.130:0/1774073092 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f97ac02cc70 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.491 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.511+0000 7f97a9ffb700 1 -- 172.21.15.130:0/1774073092 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f97a4027030 con 0x7f97ac0ff7a0 2024-05-13T22:39:51.491 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:39:51.491 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:39:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.513+0000 7f97b2c7c700 1 -- 172.21.15.130:0/1774073092 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f97980383c0 msgr2=0x7f979803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.513+0000 7f97b2c7c700 1 --2- 172.21.15.130:0/1774073092 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f97980383c0 0x7f979803a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f979c00ad30 tx=0x7f979c0093f0 comp rx=0 tx=0).stop 2024-05-13T22:39:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.513+0000 7f97b2c7c700 1 -- 172.21.15.130:0/1774073092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97ac0ff7a0 msgr2=0x7f97ac1948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.513+0000 7f97b2c7c700 1 --2- 172.21.15.130:0/1774073092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97ac0ff7a0 0x7f97ac1948a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f97a4005b40 tx=0x7f97a4004740 comp rx=0 tx=0).stop 2024-05-13T22:39:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.513+0000 7f97b2c7c700 1 -- 172.21.15.130:0/1774073092 shutdown_connections 2024-05-13T22:39:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.513+0000 7f97b2c7c700 1 --2- 172.21.15.130:0/1774073092 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f97980383c0 0x7f979803a880 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.513+0000 7f97b2c7c700 1 --2- 172.21.15.130:0/1774073092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97ac0ff7a0 0x7f97ac1948a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.513+0000 7f97b2c7c700 1 -- 172.21.15.130:0/1774073092 >> 172.21.15.130:0/1774073092 conn(0x7f97ac0fad20 msgr2=0x7f97ac0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.514+0000 7f97b2c7c700 1 -- 172.21.15.130:0/1774073092 shutdown_connections 2024-05-13T22:39:51.493 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:51.514+0000 7f97b2c7c700 1 -- 172.21.15.130:0/1774073092 wait complete. 2024-05-13T22:39:51.494 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:39:52.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:52 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/1774073092' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:39:53.097 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:39:53.097 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:39:53.314 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:39:53.362 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:39:53.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:53 smithi102 ceph-mon[28599]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:54.735 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.753+0000 7f8edb512700 1 -- 172.21.15.130:0/1436171740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41008b0 msgr2=0x7f8ed4100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:54.735 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.753+0000 7f8edb512700 1 --2- 172.21.15.130:0/1436171740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41008b0 0x7f8ed4100cd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009910 tx=0x7f8ec4009c20 comp rx=0 tx=0).stop 2024-05-13T22:39:54.735 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.753+0000 7f8edb512700 1 -- 172.21.15.130:0/1436171740 shutdown_connections 2024-05-13T22:39:54.735 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.753+0000 7f8edb512700 1 --2- 172.21.15.130:0/1436171740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41008b0 0x7f8ed4100cd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:54.735 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.753+0000 7f8edb512700 1 -- 172.21.15.130:0/1436171740 >> 172.21.15.130:0/1436171740 conn(0x7f8ed40fbe30 msgr2=0x7f8ed40fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:54.735 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.754+0000 7f8edb512700 1 -- 172.21.15.130:0/1436171740 shutdown_connections 2024-05-13T22:39:54.735 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.754+0000 7f8edb512700 1 -- 172.21.15.130:0/1436171740 wait complete. 2024-05-13T22:39:54.735 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.754+0000 7f8edb512700 1 Processor -- start 2024-05-13T22:39:54.735 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.754+0000 7f8edb512700 1 -- start start 2024-05-13T22:39:54.735 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.754+0000 7f8edb512700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41008b0 0x7f8ed41959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:54.735 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.754+0000 7f8edb512700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4195f00 con 0x7f8ed41008b0 2024-05-13T22:39:54.736 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.755+0000 7f8ed92ae700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41008b0 0x7f8ed41959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:54.736 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.755+0000 7f8ed92ae700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41008b0 0x7f8ed41959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:40820/0 (socket says 172.21.15.130:40820) 2024-05-13T22:39:54.736 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.755+0000 7f8ed92ae700 1 -- 172.21.15.130:0/2138940233 learned_addr learned my addr 172.21.15.130:0/2138940233 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:39:54.736 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.755+0000 7f8ed92ae700 1 -- 172.21.15.130:0/2138940233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec40095f0 con 0x7f8ed41008b0 2024-05-13T22:39:54.736 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.755+0000 7f8ed92ae700 1 --2- 172.21.15.130:0/2138940233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41008b0 0x7f8ed41959c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4005b40 tx=0x7f8ec4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:54.736 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.755+0000 7f8eca7fc700 1 -- 172.21.15.130:0/2138940233 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ec401d070 con 0x7f8ed41008b0 2024-05-13T22:39:54.736 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.755+0000 7f8edb512700 1 -- 172.21.15.130:0/2138940233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed4196100 con 0x7f8ed41008b0 2024-05-13T22:39:54.736 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.755+0000 7f8eca7fc700 1 -- 172.21.15.130:0/2138940233 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8ec400be10 con 0x7f8ed41008b0 2024-05-13T22:39:54.736 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.755+0000 7f8eca7fc700 1 -- 172.21.15.130:0/2138940233 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ec400f460 con 0x7f8ed41008b0 2024-05-13T22:39:54.736 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.755+0000 7f8edb512700 1 -- 172.21.15.130:0/2138940233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed41965a0 con 0x7f8ed41008b0 2024-05-13T22:39:54.737 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.756+0000 7f8edb512700 1 -- 172.21.15.130:0/2138940233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed418f4d0 con 0x7f8ed41008b0 2024-05-13T22:39:54.737 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.756+0000 7f8eca7fc700 1 -- 172.21.15.130:0/2138940233 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f8ec40227e0 con 0x7f8ed41008b0 2024-05-13T22:39:54.738 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.756+0000 7f8eca7fc700 1 --2- 172.21.15.130:0/2138940233 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f8ec00383c0 0x7f8ec003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:54.738 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.756+0000 7f8eca7fc700 1 -- 172.21.15.130:0/2138940233 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ec404c4a0 con 0x7f8ed41008b0 2024-05-13T22:39:54.738 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.757+0000 7f8ed8aad700 1 --2- 172.21.15.130:0/2138940233 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f8ec00383c0 0x7f8ec003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:54.738 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.757+0000 7f8ed8aad700 1 --2- 172.21.15.130:0/2138940233 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f8ec00383c0 0x7f8ec003a880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0006fd0 tx=0x7f8ed0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:54.744 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:54.765+0000 7f8eca7fc700 1 -- 172.21.15.130:0/2138940233 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8ec402bba0 con 0x7f8ed41008b0 2024-05-13T22:39:55.005 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.026+0000 7f8edb512700 1 -- 172.21.15.130:0/2138940233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8ed402cc70 con 0x7f8ed41008b0 2024-05-13T22:39:55.006 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.027+0000 7f8eca7fc700 1 -- 172.21.15.130:0/2138940233 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f8ec4027020 con 0x7f8ed41008b0 2024-05-13T22:39:55.006 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:39:55.006 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:39:55.008 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.029+0000 7f8edb512700 1 -- 172.21.15.130:0/2138940233 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f8ec00383c0 msgr2=0x7f8ec003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:55.008 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.029+0000 7f8edb512700 1 --2- 172.21.15.130:0/2138940233 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f8ec00383c0 0x7f8ec003a880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0006fd0 tx=0x7f8ed0006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:55.008 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.029+0000 7f8edb512700 1 -- 172.21.15.130:0/2138940233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41008b0 msgr2=0x7f8ed41959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:55.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.029+0000 7f8edb512700 1 --2- 172.21.15.130:0/2138940233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41008b0 0x7f8ed41959c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4005b40 tx=0x7f8ec4004740 comp rx=0 tx=0).stop 2024-05-13T22:39:55.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.029+0000 7f8edb512700 1 -- 172.21.15.130:0/2138940233 shutdown_connections 2024-05-13T22:39:55.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.029+0000 7f8edb512700 1 --2- 172.21.15.130:0/2138940233 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f8ec00383c0 0x7f8ec003a880 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:55.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.029+0000 7f8edb512700 1 --2- 172.21.15.130:0/2138940233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ed41008b0 0x7f8ed41959c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:55.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.029+0000 7f8edb512700 1 -- 172.21.15.130:0/2138940233 >> 172.21.15.130:0/2138940233 conn(0x7f8ed40fbe30 msgr2=0x7f8ed40fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:55.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.029+0000 7f8edb512700 1 -- 172.21.15.130:0/2138940233 shutdown_connections 2024-05-13T22:39:55.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:55.029+0000 7f8edb512700 1 -- 172.21.15.130:0/2138940233 wait complete. 2024-05-13T22:39:55.009 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:39:55.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:55 smithi102 ceph-mon[28599]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:55.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:55 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/2138940233' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:39:56.583 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:39:56.583 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:39:56.788 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:39:56.835 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:39:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:57 smithi102 ceph-mon[28599]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:58.211 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.230+0000 7f04a6bca700 1 -- 172.21.15.130:0/472879198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a01008b0 msgr2=0x7f04a0100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:58.211 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.230+0000 7f04a6bca700 1 --2- 172.21.15.130:0/472879198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a01008b0 0x7f04a0100cd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0494009910 tx=0x7f0494009c20 comp rx=0 tx=0).stop 2024-05-13T22:39:58.211 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.230+0000 7f04a6bca700 1 -- 172.21.15.130:0/472879198 shutdown_connections 2024-05-13T22:39:58.211 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.230+0000 7f04a6bca700 1 --2- 172.21.15.130:0/472879198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a01008b0 0x7f04a0100cd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:58.211 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.230+0000 7f04a6bca700 1 -- 172.21.15.130:0/472879198 >> 172.21.15.130:0/472879198 conn(0x7f04a00fbe30 msgr2=0x7f04a00fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:58.211 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.230+0000 7f04a6bca700 1 -- 172.21.15.130:0/472879198 shutdown_connections 2024-05-13T22:39:58.211 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.230+0000 7f04a6bca700 1 -- 172.21.15.130:0/472879198 wait complete. 2024-05-13T22:39:58.211 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.230+0000 7f04a6bca700 1 Processor -- start 2024-05-13T22:39:58.211 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.231+0000 7f04a6bca700 1 -- start start 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.231+0000 7f04a6bca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a01008b0 0x7f04a01959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.231+0000 7f04a6bca700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a0195f00 con 0x7f04a01008b0 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.231+0000 7f04a4966700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a01008b0 0x7f04a01959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.231+0000 7f04a4966700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a01008b0 0x7f04a01959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:40844/0 (socket says 172.21.15.130:40844) 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.231+0000 7f04a4966700 1 -- 172.21.15.130:0/2165766453 learned_addr learned my addr 172.21.15.130:0/2165766453 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.231+0000 7f04a4966700 1 -- 172.21.15.130:0/2165766453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940095f0 con 0x7f04a01008b0 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.231+0000 7f04a4966700 1 --2- 172.21.15.130:0/2165766453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a01008b0 0x7f04a01959c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0494005b40 tx=0x7f0494004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.231+0000 7f049dffb700 1 -- 172.21.15.130:0/2165766453 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f049401d070 con 0x7f04a01008b0 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.232+0000 7f049dffb700 1 -- 172.21.15.130:0/2165766453 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f049400be10 con 0x7f04a01008b0 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.232+0000 7f049dffb700 1 -- 172.21.15.130:0/2165766453 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f049400f460 con 0x7f04a01008b0 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.232+0000 7f04a6bca700 1 -- 172.21.15.130:0/2165766453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a0196100 con 0x7f04a01008b0 2024-05-13T22:39:58.212 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.232+0000 7f04a6bca700 1 -- 172.21.15.130:0/2165766453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a01965a0 con 0x7f04a01008b0 2024-05-13T22:39:58.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.233+0000 7f049dffb700 1 -- 172.21.15.130:0/2165766453 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f04940227e0 con 0x7f04a01008b0 2024-05-13T22:39:58.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.233+0000 7f04a6bca700 1 -- 172.21.15.130:0/2165766453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a018f4d0 con 0x7f04a01008b0 2024-05-13T22:39:58.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.233+0000 7f049dffb700 1 --2- 172.21.15.130:0/2165766453 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f04900383c0 0x7f049003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:39:58.213 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.233+0000 7f049dffb700 1 -- 172.21.15.130:0/2165766453 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f049404c3b0 con 0x7f04a01008b0 2024-05-13T22:39:58.214 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.233+0000 7f049ffff700 1 --2- 172.21.15.130:0/2165766453 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f04900383c0 0x7f049003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:39:58.214 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.234+0000 7f049ffff700 1 --2- 172.21.15.130:0/2165766453 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f04900383c0 0x7f049003a880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f048c006fd0 tx=0x7f048c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:39:58.217 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.238+0000 7f049dffb700 1 -- 172.21.15.130:0/2165766453 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f049402bba0 con 0x7f04a01008b0 2024-05-13T22:39:58.478 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.498+0000 7f04a6bca700 1 -- 172.21.15.130:0/2165766453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f04a002cc70 con 0x7f04a01008b0 2024-05-13T22:39:58.478 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.499+0000 7f049dffb700 1 -- 172.21.15.130:0/2165766453 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f0494027020 con 0x7f04a01008b0 2024-05-13T22:39:58.479 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:39:58.479 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:39:58.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.501+0000 7f04a6bca700 1 -- 172.21.15.130:0/2165766453 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f04900383c0 msgr2=0x7f049003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:58.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.501+0000 7f04a6bca700 1 --2- 172.21.15.130:0/2165766453 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f04900383c0 0x7f049003a880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f048c006fd0 tx=0x7f048c006e40 comp rx=0 tx=0).stop 2024-05-13T22:39:58.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.501+0000 7f04a6bca700 1 -- 172.21.15.130:0/2165766453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a01008b0 msgr2=0x7f04a01959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:39:58.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.501+0000 7f04a6bca700 1 --2- 172.21.15.130:0/2165766453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a01008b0 0x7f04a01959c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0494005b40 tx=0x7f0494004740 comp rx=0 tx=0).stop 2024-05-13T22:39:58.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.501+0000 7f04a6bca700 1 -- 172.21.15.130:0/2165766453 shutdown_connections 2024-05-13T22:39:58.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.501+0000 7f04a6bca700 1 --2- 172.21.15.130:0/2165766453 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f04900383c0 0x7f049003a880 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:58.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.501+0000 7f04a6bca700 1 --2- 172.21.15.130:0/2165766453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a01008b0 0x7f04a01959c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:39:58.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.501+0000 7f04a6bca700 1 -- 172.21.15.130:0/2165766453 >> 172.21.15.130:0/2165766453 conn(0x7f04a00fbe30 msgr2=0x7f04a00fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:39:58.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.501+0000 7f04a6bca700 1 -- 172.21.15.130:0/2165766453 shutdown_connections 2024-05-13T22:39:58.481 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:39:58.501+0000 7f04a6bca700 1 -- 172.21.15.130:0/2165766453 wait complete. 2024-05-13T22:39:58.482 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:39:59.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:59 smithi102 ceph-mon[28599]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:39:59.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:39:59 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/2165766453' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:00.019 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:00.019 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:00.224 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:00.271 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:01.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:01 smithi102 ceph-mon[28599]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:01.564 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.581+0000 7fede729a700 1 -- 172.21.15.130:0/1136778950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0100890 msgr2=0x7fede0100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:01.564 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.581+0000 7fede729a700 1 --2- 172.21.15.130:0/1136778950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0100890 0x7fede0100cb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feddc009910 tx=0x7feddc009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:01.564 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.581+0000 7fede729a700 1 -- 172.21.15.130:0/1136778950 shutdown_connections 2024-05-13T22:40:01.564 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.581+0000 7fede729a700 1 --2- 172.21.15.130:0/1136778950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0100890 0x7fede0100cb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:01.564 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.581+0000 7fede729a700 1 -- 172.21.15.130:0/1136778950 >> 172.21.15.130:0/1136778950 conn(0x7fede00fbe30 msgr2=0x7fede00fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:01.564 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.582+0000 7fede729a700 1 -- 172.21.15.130:0/1136778950 shutdown_connections 2024-05-13T22:40:01.564 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.582+0000 7fede729a700 1 -- 172.21.15.130:0/1136778950 wait complete. 2024-05-13T22:40:01.564 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.582+0000 7fede729a700 1 Processor -- start 2024-05-13T22:40:01.564 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.582+0000 7fede729a700 1 -- start start 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fede729a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0100890 0x7fede0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fede729a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede0075070 con 0x7fede0100890 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fede5036700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0100890 0x7fede0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fede5036700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0100890 0x7fede0074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:44522/0 (socket says 172.21.15.130:44522) 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fede5036700 1 -- 172.21.15.130:0/2723161585 learned_addr learned my addr 172.21.15.130:0/2723161585 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fede5036700 1 -- 172.21.15.130:0/2723161585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc0095f0 con 0x7fede0100890 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fede5036700 1 --2- 172.21.15.130:0/2723161585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0100890 0x7fede0074b30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feddc005b40 tx=0x7feddc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fedd67fc700 1 -- 172.21.15.130:0/2723161585 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feddc01d070 con 0x7fede0100890 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fede729a700 1 -- 172.21.15.130:0/2723161585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede0073180 con 0x7fede0100890 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fedd67fc700 1 -- 172.21.15.130:0/2723161585 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feddc00be10 con 0x7fede0100890 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fedd67fc700 1 -- 172.21.15.130:0/2723161585 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feddc00f460 con 0x7fede0100890 2024-05-13T22:40:01.565 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.583+0000 7fede729a700 1 -- 172.21.15.130:0/2723161585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede0073620 con 0x7fede0100890 2024-05-13T22:40:01.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.584+0000 7fede729a700 1 -- 172.21.15.130:0/2723161585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede00623c0 con 0x7fede0100890 2024-05-13T22:40:01.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.584+0000 7fedd67fc700 1 -- 172.21.15.130:0/2723161585 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7feddc0227e0 con 0x7fede0100890 2024-05-13T22:40:01.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.585+0000 7fedd67fc700 1 --2- 172.21.15.130:0/2723161585 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fedcc0383c0 0x7fedcc03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:01.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.585+0000 7fedd67fc700 1 -- 172.21.15.130:0/2723161585 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feddc04c4a0 con 0x7fede0100890 2024-05-13T22:40:01.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.585+0000 7fede4835700 1 --2- 172.21.15.130:0/2723161585 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fedcc0383c0 0x7fedcc03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:01.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.585+0000 7fede4835700 1 --2- 172.21.15.130:0/2723161585 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fedcc0383c0 0x7fedcc03a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fedd0006fd0 tx=0x7fedd0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:01.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.589+0000 7fedd67fc700 1 -- 172.21.15.130:0/2723161585 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7feddc02bba0 con 0x7fede0100890 2024-05-13T22:40:01.831 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.850+0000 7fede729a700 1 -- 172.21.15.130:0/2723161585 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fede018ee90 con 0x7fede0100890 2024-05-13T22:40:01.831 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.851+0000 7fedd67fc700 1 -- 172.21.15.130:0/2723161585 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7feddc027020 con 0x7fede0100890 2024-05-13T22:40:01.831 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:01.831 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:01.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.853+0000 7fede729a700 1 -- 172.21.15.130:0/2723161585 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fedcc0383c0 msgr2=0x7fedcc03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:01.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.853+0000 7fede729a700 1 --2- 172.21.15.130:0/2723161585 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fedcc0383c0 0x7fedcc03a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fedd0006fd0 tx=0x7fedd0006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:01.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.853+0000 7fede729a700 1 -- 172.21.15.130:0/2723161585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0100890 msgr2=0x7fede0074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:01.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.853+0000 7fede729a700 1 --2- 172.21.15.130:0/2723161585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0100890 0x7fede0074b30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feddc005b40 tx=0x7feddc004740 comp rx=0 tx=0).stop 2024-05-13T22:40:01.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.853+0000 7fede729a700 1 -- 172.21.15.130:0/2723161585 shutdown_connections 2024-05-13T22:40:01.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.853+0000 7fede729a700 1 --2- 172.21.15.130:0/2723161585 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fedcc0383c0 0x7fedcc03a880 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:01.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.853+0000 7fede729a700 1 --2- 172.21.15.130:0/2723161585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede0100890 0x7fede0074b30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:01.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.853+0000 7fede729a700 1 -- 172.21.15.130:0/2723161585 >> 172.21.15.130:0/2723161585 conn(0x7fede00fbe30 msgr2=0x7fede00fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:01.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.853+0000 7fede729a700 1 -- 172.21.15.130:0/2723161585 shutdown_connections 2024-05-13T22:40:01.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:01.853+0000 7fede729a700 1 -- 172.21.15.130:0/2723161585 wait complete. 2024-05-13T22:40:01.835 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:02.282 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:02 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/2723161585' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:03.289 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:03 smithi102 ceph-mon[28599]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:03.330 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:03.330 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:03.535 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:03.582 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:05.093 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.110+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1047842003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc1008b0 msgr2=0x7f5dcc100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:05.093 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.110+0000 7f5dd3cce700 1 --2- 172.21.15.130:0/1047842003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc1008b0 0x7f5dcc100cd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009910 tx=0x7f5dbc009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:05.093 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.110+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1047842003 shutdown_connections 2024-05-13T22:40:05.093 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.110+0000 7f5dd3cce700 1 --2- 172.21.15.130:0/1047842003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc1008b0 0x7f5dcc100cd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:05.093 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.110+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1047842003 >> 172.21.15.130:0/1047842003 conn(0x7f5dcc0fbe30 msgr2=0x7f5dcc0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:05.094 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.110+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1047842003 shutdown_connections 2024-05-13T22:40:05.094 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.110+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1047842003 wait complete. 2024-05-13T22:40:05.094 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.111+0000 7f5dd3cce700 1 Processor -- start 2024-05-13T22:40:05.094 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.111+0000 7f5dd3cce700 1 -- start start 2024-05-13T22:40:05.094 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.111+0000 7f5dd3cce700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc1008b0 0x7f5dcc074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:05.094 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.111+0000 7f5dd3cce700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc075070 con 0x7f5dcc1008b0 2024-05-13T22:40:05.095 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.111+0000 7f5dd1a6a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc1008b0 0x7f5dcc074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:05.095 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.111+0000 7f5dd1a6a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc1008b0 0x7f5dcc074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:44550/0 (socket says 172.21.15.130:44550) 2024-05-13T22:40:05.095 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.111+0000 7f5dd1a6a700 1 -- 172.21.15.130:0/1043367975 learned_addr learned my addr 172.21.15.130:0/1043367975 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:05.095 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.111+0000 7f5dd1a6a700 1 -- 172.21.15.130:0/1043367975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc0095f0 con 0x7f5dcc1008b0 2024-05-13T22:40:05.095 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.111+0000 7f5dd1a6a700 1 --2- 172.21.15.130:0/1043367975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc1008b0 0x7f5dcc074b30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc005b40 tx=0x7f5dbc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:05.098 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.115+0000 7f5dc2ffd700 1 -- 172.21.15.130:0/1043367975 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5dbc01d070 con 0x7f5dcc1008b0 2024-05-13T22:40:05.098 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.115+0000 7f5dc2ffd700 1 -- 172.21.15.130:0/1043367975 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5dbc00be10 con 0x7f5dcc1008b0 2024-05-13T22:40:05.098 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.115+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1043367975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc073180 con 0x7f5dcc1008b0 2024-05-13T22:40:05.098 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.115+0000 7f5dc2ffd700 1 -- 172.21.15.130:0/1043367975 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5dbc00f460 con 0x7f5dcc1008b0 2024-05-13T22:40:05.098 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.115+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1043367975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc073620 con 0x7f5dcc1008b0 2024-05-13T22:40:05.098 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.116+0000 7f5dc2ffd700 1 -- 172.21.15.130:0/1043367975 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f5dbc00b960 con 0x7f5dcc1008b0 2024-05-13T22:40:05.099 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.116+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1043367975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc0623c0 con 0x7f5dcc1008b0 2024-05-13T22:40:05.099 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.116+0000 7f5dc2ffd700 1 --2- 172.21.15.130:0/1043367975 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5db8038410 0x7f5db803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:05.099 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.116+0000 7f5dc2ffd700 1 -- 172.21.15.130:0/1043367975 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5dbc04c510 con 0x7f5dcc1008b0 2024-05-13T22:40:05.099 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.116+0000 7f5dd1269700 1 --2- 172.21.15.130:0/1043367975 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5db8038410 0x7f5db803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:05.099 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.116+0000 7f5dd1269700 1 --2- 172.21.15.130:0/1043367975 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5db8038410 0x7f5db803a8d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006fd0 tx=0x7f5dc8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:05.101 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.121+0000 7f5dc2ffd700 1 -- 172.21.15.130:0/1043367975 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5dbc02bb60 con 0x7f5dcc1008b0 2024-05-13T22:40:05.231 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:05 smithi102 ceph-mon[28599]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:05.361 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.380+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1043367975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5dcc18eeb0 con 0x7f5dcc1008b0 2024-05-13T22:40:05.362 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.381+0000 7f5dc2ffd700 1 -- 172.21.15.130:0/1043367975 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f5dbc027030 con 0x7f5dcc1008b0 2024-05-13T22:40:05.362 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:05.362 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:05.364 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.383+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1043367975 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5db8038410 msgr2=0x7f5db803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:05.364 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.383+0000 7f5dd3cce700 1 --2- 172.21.15.130:0/1043367975 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5db8038410 0x7f5db803a8d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006fd0 tx=0x7f5dc8006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:05.364 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.383+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1043367975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc1008b0 msgr2=0x7f5dcc074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:05.364 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.383+0000 7f5dd3cce700 1 --2- 172.21.15.130:0/1043367975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc1008b0 0x7f5dcc074b30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc005b40 tx=0x7f5dbc004740 comp rx=0 tx=0).stop 2024-05-13T22:40:05.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.383+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1043367975 shutdown_connections 2024-05-13T22:40:05.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.383+0000 7f5dd3cce700 1 --2- 172.21.15.130:0/1043367975 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f5db8038410 0x7f5db803a8d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:05.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.383+0000 7f5dd3cce700 1 --2- 172.21.15.130:0/1043367975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dcc1008b0 0x7f5dcc074b30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:05.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.383+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1043367975 >> 172.21.15.130:0/1043367975 conn(0x7f5dcc0fbe30 msgr2=0x7f5dcc0fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:05.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.383+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1043367975 shutdown_connections 2024-05-13T22:40:05.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:05.383+0000 7f5dd3cce700 1 -- 172.21.15.130:0/1043367975 wait complete. 2024-05-13T22:40:05.366 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:06.414 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:06 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:06.414 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:06 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:06.414 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:06 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:06.414 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:06 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:06.414 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:06 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:06.414 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:06 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:06.414 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:06 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:06.415 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:06 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:06.415 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:06 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/1043367975' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:06.866 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:06.866 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:07.072 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:07.120 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:07 smithi102 ceph-mon[28599]: Deploying daemon prometheus.smithi102 on smithi102 2024-05-13T22:40:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:07 smithi102 ceph-mon[28599]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:08.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.587+0000 7fd04907e700 1 -- 172.21.15.130:0/720548469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0440ff7a0 msgr2=0x7fd0440ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:08.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.587+0000 7fd04907e700 1 --2- 172.21.15.130:0/720548469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0440ff7a0 0x7fd0440ffbc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd034009910 tx=0x7fd034009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:08.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.587+0000 7fd04907e700 1 -- 172.21.15.130:0/720548469 shutdown_connections 2024-05-13T22:40:08.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.587+0000 7fd04907e700 1 --2- 172.21.15.130:0/720548469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0440ff7a0 0x7fd0440ffbc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:08.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.587+0000 7fd04907e700 1 -- 172.21.15.130:0/720548469 >> 172.21.15.130:0/720548469 conn(0x7fd0440fad20 msgr2=0x7fd0440fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:08.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.587+0000 7fd04907e700 1 -- 172.21.15.130:0/720548469 shutdown_connections 2024-05-13T22:40:08.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.587+0000 7fd04907e700 1 -- 172.21.15.130:0/720548469 wait complete. 2024-05-13T22:40:08.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.587+0000 7fd04907e700 1 Processor -- start 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.588+0000 7fd04907e700 1 -- start start 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.588+0000 7fd04907e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0440ff7a0 0x7fd044197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.588+0000 7fd04907e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd044198100 con 0x7fd0440ff7a0 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.588+0000 7fd042d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0440ff7a0 0x7fd044197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.588+0000 7fd042d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0440ff7a0 0x7fd044197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:44562/0 (socket says 172.21.15.130:44562) 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.588+0000 7fd042d9d700 1 -- 172.21.15.130:0/3215660063 learned_addr learned my addr 172.21.15.130:0/3215660063 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.588+0000 7fd042d9d700 1 -- 172.21.15.130:0/3215660063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0340095f0 con 0x7fd0440ff7a0 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.588+0000 7fd042d9d700 1 --2- 172.21.15.130:0/3215660063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0440ff7a0 0x7fd044197bc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd034005b40 tx=0x7fd034004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.589+0000 7fd02bfff700 1 -- 172.21.15.130:0/3215660063 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd03401d070 con 0x7fd0440ff7a0 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.589+0000 7fd02bfff700 1 -- 172.21.15.130:0/3215660063 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd03400be10 con 0x7fd0440ff7a0 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.589+0000 7fd04907e700 1 -- 172.21.15.130:0/3215660063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd044198300 con 0x7fd0440ff7a0 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.589+0000 7fd02bfff700 1 -- 172.21.15.130:0/3215660063 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd03400f460 con 0x7fd0440ff7a0 2024-05-13T22:40:08.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.589+0000 7fd04907e700 1 -- 172.21.15.130:0/3215660063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0441987a0 con 0x7fd0440ff7a0 2024-05-13T22:40:08.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.589+0000 7fd02bfff700 1 -- 172.21.15.130:0/3215660063 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fd03400f5c0 con 0x7fd0440ff7a0 2024-05-13T22:40:08.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.590+0000 7fd02bfff700 1 --2- 172.21.15.130:0/3215660063 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd02c038410 0x7fd02c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:08.570 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.590+0000 7fd02bfff700 1 -- 172.21.15.130:0/3215660063 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd03404c400 con 0x7fd0440ff7a0 2024-05-13T22:40:08.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.590+0000 7fd04907e700 1 -- 172.21.15.130:0/3215660063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0441917d0 con 0x7fd0440ff7a0 2024-05-13T22:40:08.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.590+0000 7fd04259c700 1 --2- 172.21.15.130:0/3215660063 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd02c038410 0x7fd02c03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:08.571 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.590+0000 7fd04259c700 1 --2- 172.21.15.130:0/3215660063 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd02c038410 0x7fd02c03a8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd038006fd0 tx=0x7fd038006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:08.574 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.595+0000 7fd02bfff700 1 -- 172.21.15.130:0/3215660063 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd034027070 con 0x7fd0440ff7a0 2024-05-13T22:40:08.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.856+0000 7fd04907e700 1 -- 172.21.15.130:0/3215660063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd04402cc70 con 0x7fd0440ff7a0 2024-05-13T22:40:08.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.857+0000 7fd02bfff700 1 -- 172.21.15.130:0/3215660063 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fd03402c440 con 0x7fd0440ff7a0 2024-05-13T22:40:08.836 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:08.837 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:08.838 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.859+0000 7fd04907e700 1 -- 172.21.15.130:0/3215660063 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd02c038410 msgr2=0x7fd02c03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:08.839 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.859+0000 7fd04907e700 1 --2- 172.21.15.130:0/3215660063 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd02c038410 0x7fd02c03a8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd038006fd0 tx=0x7fd038006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:08.839 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.859+0000 7fd04907e700 1 -- 172.21.15.130:0/3215660063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0440ff7a0 msgr2=0x7fd044197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:08.839 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.859+0000 7fd04907e700 1 --2- 172.21.15.130:0/3215660063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0440ff7a0 0x7fd044197bc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd034005b40 tx=0x7fd034004740 comp rx=0 tx=0).stop 2024-05-13T22:40:08.839 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.859+0000 7fd04907e700 1 -- 172.21.15.130:0/3215660063 shutdown_connections 2024-05-13T22:40:08.839 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.859+0000 7fd04907e700 1 --2- 172.21.15.130:0/3215660063 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fd02c038410 0x7fd02c03a8d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:08.839 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.859+0000 7fd04907e700 1 --2- 172.21.15.130:0/3215660063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0440ff7a0 0x7fd044197bc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:08.839 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.859+0000 7fd04907e700 1 -- 172.21.15.130:0/3215660063 >> 172.21.15.130:0/3215660063 conn(0x7fd0440fad20 msgr2=0x7fd044069100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:08.839 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.859+0000 7fd04907e700 1 -- 172.21.15.130:0/3215660063 shutdown_connections 2024-05-13T22:40:08.839 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:08.859+0000 7fd04907e700 1 -- 172.21.15.130:0/3215660063 wait complete. 2024-05-13T22:40:08.840 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:09.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:09 smithi102 ceph-mon[28599]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:09.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:09 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:09.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:09 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/3215660063' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:10.344 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:10.344 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:10.547 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:10.595 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:11 smithi102 ceph-mon[28599]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.983+0000 7fbb186a1700 1 -- 172.21.15.130:0/117982727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb101008b0 msgr2=0x7fbb10100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.983+0000 7fbb186a1700 1 --2- 172.21.15.130:0/117982727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb101008b0 0x7fbb10100cd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009910 tx=0x7fbb00009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.983+0000 7fbb186a1700 1 -- 172.21.15.130:0/117982727 shutdown_connections 2024-05-13T22:40:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.983+0000 7fbb186a1700 1 --2- 172.21.15.130:0/117982727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb101008b0 0x7fbb10100cd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.983+0000 7fbb186a1700 1 -- 172.21.15.130:0/117982727 >> 172.21.15.130:0/117982727 conn(0x7fbb100fbe30 msgr2=0x7fbb100fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.983+0000 7fbb186a1700 1 -- 172.21.15.130:0/117982727 shutdown_connections 2024-05-13T22:40:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.983+0000 7fbb186a1700 1 -- 172.21.15.130:0/117982727 wait complete. 2024-05-13T22:40:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.984+0000 7fbb186a1700 1 Processor -- start 2024-05-13T22:40:11.964 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.984+0000 7fbb186a1700 1 -- start start 2024-05-13T22:40:11.965 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.984+0000 7fbb186a1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb101008b0 0x7fbb10074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:11.965 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.984+0000 7fbb186a1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb10075070 con 0x7fbb101008b0 2024-05-13T22:40:11.965 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.984+0000 7fbb1643d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb101008b0 0x7fbb10074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:11.965 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.984+0000 7fbb1643d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb101008b0 0x7fbb10074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:40190/0 (socket says 172.21.15.130:40190) 2024-05-13T22:40:11.965 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.984+0000 7fbb1643d700 1 -- 172.21.15.130:0/3538700334 learned_addr learned my addr 172.21.15.130:0/3538700334 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:11.965 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.985+0000 7fbb1643d700 1 -- 172.21.15.130:0/3538700334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000095f0 con 0x7fbb101008b0 2024-05-13T22:40:11.966 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.985+0000 7fbb1643d700 1 --2- 172.21.15.130:0/3538700334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb101008b0 0x7fbb10074b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005b40 tx=0x7fbb00004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.985+0000 7fbb077fe700 1 -- 172.21.15.130:0/3538700334 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb0001d070 con 0x7fbb101008b0 2024-05-13T22:40:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.985+0000 7fbb186a1700 1 -- 172.21.15.130:0/3538700334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb10073180 con 0x7fbb101008b0 2024-05-13T22:40:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.985+0000 7fbb077fe700 1 -- 172.21.15.130:0/3538700334 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbb0000be10 con 0x7fbb101008b0 2024-05-13T22:40:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.985+0000 7fbb077fe700 1 -- 172.21.15.130:0/3538700334 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb0000f460 con 0x7fbb101008b0 2024-05-13T22:40:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.985+0000 7fbb186a1700 1 -- 172.21.15.130:0/3538700334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb10073620 con 0x7fbb101008b0 2024-05-13T22:40:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.986+0000 7fbb077fe700 1 -- 172.21.15.130:0/3538700334 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fbb000227e0 con 0x7fbb101008b0 2024-05-13T22:40:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.986+0000 7fbb186a1700 1 -- 172.21.15.130:0/3538700334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb100623c0 con 0x7fbb101008b0 2024-05-13T22:40:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.986+0000 7fbb077fe700 1 --2- 172.21.15.130:0/3538700334 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fbafc0383c0 0x7fbafc03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.986+0000 7fbb077fe700 1 -- 172.21.15.130:0/3538700334 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbb0004c470 con 0x7fbb101008b0 2024-05-13T22:40:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.986+0000 7fbb15c3c700 1 --2- 172.21.15.130:0/3538700334 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fbafc0383c0 0x7fbafc03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:11.967 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.987+0000 7fbb15c3c700 1 --2- 172.21.15.130:0/3538700334 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fbafc0383c0 0x7fbafc03a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006fd0 tx=0x7fbb0c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:11.974 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:11.995+0000 7fbb077fe700 1 -- 172.21.15.130:0/3538700334 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbb0002bba0 con 0x7fbb101008b0 2024-05-13T22:40:12.241 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.262+0000 7fbb186a1700 1 -- 172.21.15.130:0/3538700334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbb1018eeb0 con 0x7fbb101008b0 2024-05-13T22:40:12.242 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.262+0000 7fbb077fe700 1 -- 172.21.15.130:0/3538700334 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fbb00027030 con 0x7fbb101008b0 2024-05-13T22:40:12.242 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:12.242 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.264+0000 7fbb186a1700 1 -- 172.21.15.130:0/3538700334 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fbafc0383c0 msgr2=0x7fbafc03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.264+0000 7fbb186a1700 1 --2- 172.21.15.130:0/3538700334 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fbafc0383c0 0x7fbafc03a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006fd0 tx=0x7fbb0c006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.264+0000 7fbb186a1700 1 -- 172.21.15.130:0/3538700334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb101008b0 msgr2=0x7fbb10074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.264+0000 7fbb186a1700 1 --2- 172.21.15.130:0/3538700334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb101008b0 0x7fbb10074b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005b40 tx=0x7fbb00004740 comp rx=0 tx=0).stop 2024-05-13T22:40:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.264+0000 7fbb186a1700 1 -- 172.21.15.130:0/3538700334 shutdown_connections 2024-05-13T22:40:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.264+0000 7fbb186a1700 1 --2- 172.21.15.130:0/3538700334 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fbafc0383c0 0x7fbafc03a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:12.244 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.264+0000 7fbb186a1700 1 --2- 172.21.15.130:0/3538700334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb101008b0 0x7fbb10074b30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:12.245 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.264+0000 7fbb186a1700 1 -- 172.21.15.130:0/3538700334 >> 172.21.15.130:0/3538700334 conn(0x7fbb100fbe30 msgr2=0x7fbb100fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:12.245 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.264+0000 7fbb186a1700 1 -- 172.21.15.130:0/3538700334 shutdown_connections 2024-05-13T22:40:12.245 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:12.264+0000 7fbb186a1700 1 -- 172.21.15.130:0/3538700334 wait complete. 2024-05-13T22:40:12.245 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:13.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:13 smithi102 ceph-mon[28599]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:13.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:13 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/3538700334' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:13.854 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:13.854 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:14.059 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:14.107 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:15.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:15 smithi102 ceph-mon[28599]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:15.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.531+0000 7fcc12904700 1 -- 172.21.15.130:0/2416310674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc0c0ff790 msgr2=0x7fcc0c0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:15.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.531+0000 7fcc12904700 1 --2- 172.21.15.130:0/2416310674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc0c0ff790 0x7fcc0c0ffbb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009910 tx=0x7fcbfc009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:15.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.531+0000 7fcc12904700 1 -- 172.21.15.130:0/2416310674 shutdown_connections 2024-05-13T22:40:15.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.531+0000 7fcc12904700 1 --2- 172.21.15.130:0/2416310674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc0c0ff790 0x7fcc0c0ffbb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:15.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.531+0000 7fcc12904700 1 -- 172.21.15.130:0/2416310674 >> 172.21.15.130:0/2416310674 conn(0x7fcc0c0fad30 msgr2=0x7fcc0c0fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:15.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.531+0000 7fcc12904700 1 -- 172.21.15.130:0/2416310674 shutdown_connections 2024-05-13T22:40:15.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.531+0000 7fcc12904700 1 -- 172.21.15.130:0/2416310674 wait complete. 2024-05-13T22:40:15.512 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.531+0000 7fcc12904700 1 Processor -- start 2024-05-13T22:40:15.512 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.531+0000 7fcc12904700 1 -- start start 2024-05-13T22:40:15.512 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.532+0000 7fcc12904700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc0c0ff790 0x7fcc0c195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:15.512 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.532+0000 7fcc12904700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c195ed0 con 0x7fcc0c0ff790 2024-05-13T22:40:15.512 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.532+0000 7fcc0bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc0c0ff790 0x7fcc0c195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:15.512 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.532+0000 7fcc0bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc0c0ff790 0x7fcc0c195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:40210/0 (socket says 172.21.15.130:40210) 2024-05-13T22:40:15.512 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.532+0000 7fcc0bfff700 1 -- 172.21.15.130:0/84630954 learned_addr learned my addr 172.21.15.130:0/84630954 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:15.512 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.532+0000 7fcc0bfff700 1 -- 172.21.15.130:0/84630954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc0095f0 con 0x7fcc0c0ff790 2024-05-13T22:40:15.513 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.532+0000 7fcc0bfff700 1 --2- 172.21.15.130:0/84630954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc0c0ff790 0x7fcc0c195990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc005950 tx=0x7fcbfc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:15.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.532+0000 7fcc097fa700 1 -- 172.21.15.130:0/84630954 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbfc01d070 con 0x7fcc0c0ff790 2024-05-13T22:40:15.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.532+0000 7fcc097fa700 1 -- 172.21.15.130:0/84630954 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcbfc00be10 con 0x7fcc0c0ff790 2024-05-13T22:40:15.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.532+0000 7fcc12904700 1 -- 172.21.15.130:0/84630954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c1960d0 con 0x7fcc0c0ff790 2024-05-13T22:40:15.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.532+0000 7fcc097fa700 1 -- 172.21.15.130:0/84630954 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbfc00f460 con 0x7fcc0c0ff790 2024-05-13T22:40:15.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.533+0000 7fcc12904700 1 -- 172.21.15.130:0/84630954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c196570 con 0x7fcc0c0ff790 2024-05-13T22:40:15.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.533+0000 7fcc097fa700 1 -- 172.21.15.130:0/84630954 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fcbfc00b960 con 0x7fcc0c0ff790 2024-05-13T22:40:15.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.533+0000 7fcc12904700 1 -- 172.21.15.130:0/84630954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c18f3c0 con 0x7fcc0c0ff790 2024-05-13T22:40:15.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.534+0000 7fcc097fa700 1 --2- 172.21.15.130:0/84630954 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcbf40383c0 0x7fcbf403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:15.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.534+0000 7fcc097fa700 1 -- 172.21.15.130:0/84630954 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcbfc04c480 con 0x7fcc0c0ff790 2024-05-13T22:40:15.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.534+0000 7fcc0b7fe700 1 --2- 172.21.15.130:0/84630954 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcbf40383c0 0x7fcbf403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:15.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.534+0000 7fcc0b7fe700 1 --2- 172.21.15.130:0/84630954 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcbf40383c0 0x7fcbf403a880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc00006fd0 tx=0x7fcc00006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:15.518 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.539+0000 7fcc097fa700 1 -- 172.21.15.130:0/84630954 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcbfc02bba0 con 0x7fcc0c0ff790 2024-05-13T22:40:15.784 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.804+0000 7fcc12904700 1 -- 172.21.15.130:0/84630954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcc0c02cc70 con 0x7fcc0c0ff790 2024-05-13T22:40:15.784 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.805+0000 7fcc097fa700 1 -- 172.21.15.130:0/84630954 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fcbfc027030 con 0x7fcc0c0ff790 2024-05-13T22:40:15.785 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:15.785 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:15.787 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.807+0000 7fcc12904700 1 -- 172.21.15.130:0/84630954 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcbf40383c0 msgr2=0x7fcbf403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:15.787 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.807+0000 7fcc12904700 1 --2- 172.21.15.130:0/84630954 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcbf40383c0 0x7fcbf403a880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc00006fd0 tx=0x7fcc00006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:15.787 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.807+0000 7fcc12904700 1 -- 172.21.15.130:0/84630954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc0c0ff790 msgr2=0x7fcc0c195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:15.787 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.807+0000 7fcc12904700 1 --2- 172.21.15.130:0/84630954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc0c0ff790 0x7fcc0c195990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc005950 tx=0x7fcbfc009e60 comp rx=0 tx=0).stop 2024-05-13T22:40:15.787 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.807+0000 7fcc12904700 1 -- 172.21.15.130:0/84630954 shutdown_connections 2024-05-13T22:40:15.787 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.807+0000 7fcc12904700 1 --2- 172.21.15.130:0/84630954 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7fcbf40383c0 0x7fcbf403a880 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:15.788 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.807+0000 7fcc12904700 1 --2- 172.21.15.130:0/84630954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc0c0ff790 0x7fcc0c195990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:15.788 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.807+0000 7fcc12904700 1 -- 172.21.15.130:0/84630954 >> 172.21.15.130:0/84630954 conn(0x7fcc0c0fad30 msgr2=0x7fcc0c069120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:15.788 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.807+0000 7fcc12904700 1 -- 172.21.15.130:0/84630954 shutdown_connections 2024-05-13T22:40:15.788 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:15.807+0000 7fcc12904700 1 -- 172.21.15.130:0/84630954 wait complete. 2024-05-13T22:40:15.788 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:16.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:16 smithi102 ceph-mon[28599]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:16.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:16 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/84630954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:16.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:16 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:16.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:16 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:16.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:16 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:16.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:16 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-05-13T22:40:17.374 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:17.375 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:17.577 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:17.625 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:18.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:18 smithi102 ceph-mon[28599]: from='mgr.14164 172.21.15.102:0/4236017481' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-05-13T22:40:18.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:18 smithi102 ceph-mon[28599]: mgrmap e14: smithi102.xcijrj(active, since 63s) 2024-05-13T22:40:19.088 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.108+0000 7f0cd726e700 1 -- 172.21.15.130:0/1070190934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd00ff7a0 msgr2=0x7f0cd00ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.108+0000 7f0cd726e700 1 --2- 172.21.15.130:0/1070190934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd00ff7a0 0x7f0cd00ffbc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009910 tx=0x7f0cc0009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.108+0000 7f0cd726e700 1 -- 172.21.15.130:0/1070190934 shutdown_connections 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.108+0000 7f0cd726e700 1 --2- 172.21.15.130:0/1070190934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd00ff7a0 0x7f0cd00ffbc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.108+0000 7f0cd726e700 1 -- 172.21.15.130:0/1070190934 >> 172.21.15.130:0/1070190934 conn(0x7f0cd00fad20 msgr2=0x7f0cd00fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.108+0000 7f0cd726e700 1 -- 172.21.15.130:0/1070190934 shutdown_connections 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.108+0000 7f0cd726e700 1 -- 172.21.15.130:0/1070190934 wait complete. 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.109+0000 7f0cd726e700 1 Processor -- start 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.109+0000 7f0cd726e700 1 -- start start 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.109+0000 7f0cd726e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd00ff7a0 0x7f0cd0068b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.109+0000 7f0cd726e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd01986d0 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.109+0000 7f0cd500a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd00ff7a0 0x7f0cd0068b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.109+0000 7f0cd500a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd00ff7a0 0x7f0cd0068b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:40224/0 (socket says 172.21.15.130:40224) 2024-05-13T22:40:19.089 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.109+0000 7f0cd500a700 1 -- 172.21.15.130:0/1540276064 learned_addr learned my addr 172.21.15.130:0/1540276064 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:19.090 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.109+0000 7f0cd500a700 1 -- 172.21.15.130:0/1540276064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc00095f0 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.091 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.110+0000 7f0cd500a700 1 --2- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd00ff7a0 0x7f0cd0068b20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0005b40 tx=0x7f0cc0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:19.091 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.110+0000 7f0cc67fc700 1 -- 172.21.15.130:0/1540276064 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cc001d070 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.091 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.110+0000 7f0cc67fc700 1 -- 172.21.15.130:0/1540276064 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0cc000be10 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.091 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.110+0000 7f0cd726e700 1 -- 172.21.15.130:0/1540276064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd0198810 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.091 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.110+0000 7f0cc67fc700 1 -- 172.21.15.130:0/1540276064 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cc000f460 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.092 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.110+0000 7f0cd726e700 1 -- 172.21.15.130:0/1540276064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd0198cb0 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.092 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.111+0000 7f0cc67fc700 1 -- 172.21.15.130:0/1540276064 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7f0cc00227e0 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.092 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.111+0000 7f0cd726e700 1 -- 172.21.15.130:0/1540276064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd0062430 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.092 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.111+0000 7f0cc67fc700 1 --2- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cbc038410 0x7f0cbc03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:19.092 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.111+0000 7f0cc67fc700 1 -- 172.21.15.130:0/1540276064 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0cc004c410 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.092 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.111+0000 7f0cd4809700 1 -- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cbc038410 msgr2=0x7f0cbc03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/263535069 2024-05-13T22:40:19.092 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.111+0000 7f0cd4809700 1 --2- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cbc038410 0x7f0cbc03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:40:19.095 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.116+0000 7f0cc67fc700 1 -- 172.21.15.130:0/1540276064 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0cd0062430 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.291 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.312+0000 7f0cd4809700 1 -- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cbc038410 msgr2=0x7f0cbc03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/263535069 2024-05-13T22:40:19.291 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.312+0000 7f0cd4809700 1 --2- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cbc038410 0x7f0cbc03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-13T22:40:19.359 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.380+0000 7f0cd726e700 1 -- 172.21.15.130:0/1540276064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0cd018e3c0 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.360 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.380+0000 7f0cc67fc700 1 -- 172.21.15.130:0/1540276064 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f0cc0027070 con 0x7f0cd00ff7a0 2024-05-13T22:40:19.360 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:19.360 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:19.362 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.382+0000 7f0cd726e700 1 -- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cbc038410 msgr2=0x7f0cbc03a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:40:19.362 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.382+0000 7f0cd726e700 1 --2- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cbc038410 0x7f0cbc03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:19.362 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.382+0000 7f0cd726e700 1 -- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd00ff7a0 msgr2=0x7f0cd0068b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:19.362 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.382+0000 7f0cd726e700 1 --2- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd00ff7a0 0x7f0cd0068b20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0005b40 tx=0x7f0cc0004740 comp rx=0 tx=0).stop 2024-05-13T22:40:19.362 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.382+0000 7f0cd726e700 1 -- 172.21.15.130:0/1540276064 shutdown_connections 2024-05-13T22:40:19.362 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.382+0000 7f0cd726e700 1 --2- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f0cbc038410 0x7f0cbc03a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:19.362 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.382+0000 7f0cd726e700 1 --2- 172.21.15.130:0/1540276064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0cd00ff7a0 0x7f0cd0068b20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:19.363 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.382+0000 7f0cd726e700 1 -- 172.21.15.130:0/1540276064 >> 172.21.15.130:0/1540276064 conn(0x7f0cd00fad20 msgr2=0x7f0cd00fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:19.363 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.383+0000 7f0cd726e700 1 -- 172.21.15.130:0/1540276064 shutdown_connections 2024-05-13T22:40:19.363 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:19.383+0000 7f0cd726e700 1 -- 172.21.15.130:0/1540276064 wait complete. 2024-05-13T22:40:19.363 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:19.581 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:19 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/1540276064' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:20.959 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:20.960 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:21.162 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:21.209 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:22.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.769+0000 7f4f990b4700 1 -- 172.21.15.130:0/2456374039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f941008a0 msgr2=0x7f4f94100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:22.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.769+0000 7f4f990b4700 1 --2- 172.21.15.130:0/2456374039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f941008a0 0x7f4f94100cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009910 tx=0x7f4f88009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:22.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.769+0000 7f4f990b4700 1 -- 172.21.15.130:0/2456374039 shutdown_connections 2024-05-13T22:40:22.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.769+0000 7f4f990b4700 1 --2- 172.21.15.130:0/2456374039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f941008a0 0x7f4f94100cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:22.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.769+0000 7f4f990b4700 1 -- 172.21.15.130:0/2456374039 >> 172.21.15.130:0/2456374039 conn(0x7f4f940fbe60 msgr2=0x7f4f940fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:22.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.769+0000 7f4f990b4700 1 -- 172.21.15.130:0/2456374039 shutdown_connections 2024-05-13T22:40:22.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.769+0000 7f4f990b4700 1 -- 172.21.15.130:0/2456374039 wait complete. 2024-05-13T22:40:22.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.770+0000 7f4f990b4700 1 Processor -- start 2024-05-13T22:40:22.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.770+0000 7f4f990b4700 1 -- start start 2024-05-13T22:40:22.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.770+0000 7f4f990b4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f941008a0 0x7f4f94195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:22.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.770+0000 7f4f990b4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f94195e90 con 0x7f4f941008a0 2024-05-13T22:40:22.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.771+0000 7f4f92d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f941008a0 0x7f4f94195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:22.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.771+0000 7f4f92d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f941008a0 0x7f4f94195950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:53858/0 (socket says 172.21.15.130:53858) 2024-05-13T22:40:22.751 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.771+0000 7f4f92d9d700 1 -- 172.21.15.130:0/3514310140 learned_addr learned my addr 172.21.15.130:0/3514310140 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:22.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.771+0000 7f4f92d9d700 1 -- 172.21.15.130:0/3514310140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f880095f0 con 0x7f4f941008a0 2024-05-13T22:40:22.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.771+0000 7f4f92d9d700 1 --2- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f941008a0 0x7f4f94195950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4f88005b40 tx=0x7f4f88004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:22.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.771+0000 7f4f7bfff700 1 -- 172.21.15.130:0/3514310140 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f8801d070 con 0x7f4f941008a0 2024-05-13T22:40:22.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.771+0000 7f4f990b4700 1 -- 172.21.15.130:0/3514310140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f94196090 con 0x7f4f941008a0 2024-05-13T22:40:22.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.771+0000 7f4f7bfff700 1 -- 172.21.15.130:0/3514310140 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4f8800be10 con 0x7f4f941008a0 2024-05-13T22:40:22.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.771+0000 7f4f7bfff700 1 -- 172.21.15.130:0/3514310140 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f8800f460 con 0x7f4f941008a0 2024-05-13T22:40:22.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.771+0000 7f4f990b4700 1 -- 172.21.15.130:0/3514310140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f94196530 con 0x7f4f941008a0 2024-05-13T22:40:22.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.772+0000 7f4f7bfff700 1 -- 172.21.15.130:0/3514310140 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7f4f8800b960 con 0x7f4f941008a0 2024-05-13T22:40:22.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.772+0000 7f4f990b4700 1 -- 172.21.15.130:0/3514310140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9418f4c0 con 0x7f4f941008a0 2024-05-13T22:40:22.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.772+0000 7f4f7bfff700 1 --2- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f4f7c038410 0x7f4f7c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:22.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.772+0000 7f4f7bfff700 1 -- 172.21.15.130:0/3514310140 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4f8804c4d0 con 0x7f4f941008a0 2024-05-13T22:40:22.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.772+0000 7f4f9259c700 1 -- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f4f7c038410 msgr2=0x7f4f7c03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/263535069 2024-05-13T22:40:22.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.772+0000 7f4f9259c700 1 --2- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f4f7c038410 0x7f4f7c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:40:22.756 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.777+0000 7f4f7bfff700 1 -- 172.21.15.130:0/3514310140 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4f8802a360 con 0x7f4f941008a0 2024-05-13T22:40:22.952 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.973+0000 7f4f9259c700 1 -- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f4f7c038410 msgr2=0x7f4f7c03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.102:6800/263535069 2024-05-13T22:40:22.952 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:22.973+0000 7f4f9259c700 1 --2- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f4f7c038410 0x7f4f7c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-13T22:40:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.042+0000 7f4f990b4700 1 -- 172.21.15.130:0/3514310140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4f9402cc70 con 0x7f4f941008a0 2024-05-13T22:40:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.042+0000 7f4f7bfff700 1 -- 172.21.15.130:0/3514310140 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f4f88027030 con 0x7f4f941008a0 2024-05-13T22:40:23.022 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:23.022 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.044+0000 7f4f990b4700 1 -- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f4f7c038410 msgr2=0x7f4f7c03a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:40:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.044+0000 7f4f990b4700 1 --2- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f4f7c038410 0x7f4f7c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.044+0000 7f4f990b4700 1 -- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f941008a0 msgr2=0x7f4f94195950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.044+0000 7f4f990b4700 1 --2- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f941008a0 0x7f4f94195950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4f88005b40 tx=0x7f4f88004740 comp rx=0 tx=0).stop 2024-05-13T22:40:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.045+0000 7f4f990b4700 1 -- 172.21.15.130:0/3514310140 shutdown_connections 2024-05-13T22:40:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.045+0000 7f4f990b4700 1 --2- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:6800/263535069,v1:172.21.15.102:6801/263535069] conn(0x7f4f7c038410 0x7f4f7c03a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.045+0000 7f4f990b4700 1 --2- 172.21.15.130:0/3514310140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f941008a0 0x7f4f94195950 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:23.024 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.045+0000 7f4f990b4700 1 -- 172.21.15.130:0/3514310140 >> 172.21.15.130:0/3514310140 conn(0x7f4f940fbe60 msgr2=0x7f4f940fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:23.025 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.045+0000 7f4f990b4700 1 -- 172.21.15.130:0/3514310140 shutdown_connections 2024-05-13T22:40:23.025 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:23.045+0000 7f4f990b4700 1 -- 172.21.15.130:0/3514310140 wait complete. 2024-05-13T22:40:23.025 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:23.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:23 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/3514310140' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:24.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: Active manager daemon smithi102.xcijrj restarted 2024-05-13T22:40:24.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: Activating manager daemon smithi102.xcijrj 2024-05-13T22:40:24.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: osdmap e5: 0 total, 0 up, 0 in 2024-05-13T22:40:24.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: mgrmap e15: smithi102.xcijrj(active, starting, since 0.00373441s) 2024-05-13T22:40:24.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:40:24.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi102.xcijrj", "id": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:40:24.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-13T22:40:24.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-13T22:40:24.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-13T22:40:24.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: Manager daemon smithi102.xcijrj is now available 2024-05-13T22:40:24.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:40:24.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:40:24.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:24.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:40:24.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:40:24.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/trash_purge_schedule"}]: dispatch 2024-05-13T22:40:24.641 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:24.641 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:24.847 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:24.895 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:25.618 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:25 smithi102 ceph-mon[28599]: [13/May/2024:22:40:24] ENGINE Bus STARTING 2024-05-13T22:40:25.618 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:25 smithi102 ceph-mon[28599]: [13/May/2024:22:40:24] ENGINE Serving on http://172.21.15.102:8765 2024-05-13T22:40:25.618 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:25 smithi102 ceph-mon[28599]: [13/May/2024:22:40:24] ENGINE Serving on https://172.21.15.102:7150 2024-05-13T22:40:25.618 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:25 smithi102 ceph-mon[28599]: [13/May/2024:22:40:24] ENGINE Bus STARTED 2024-05-13T22:40:25.618 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:25 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:25.618 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:25 smithi102 ceph-mon[28599]: mgrmap e16: smithi102.xcijrj(active, since 1.00561s) 2024-05-13T22:40:25.618 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:25 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:26.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.464+0000 7f601b1ec700 1 -- 172.21.15.130:0/526711994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60140ff7a0 msgr2=0x7f60140ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:26.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.464+0000 7f601b1ec700 1 --2- 172.21.15.130:0/526711994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60140ff7a0 0x7f60140ffbc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6008009910 tx=0x7f6008009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:26.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.464+0000 7f601b1ec700 1 -- 172.21.15.130:0/526711994 shutdown_connections 2024-05-13T22:40:26.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.464+0000 7f601b1ec700 1 --2- 172.21.15.130:0/526711994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60140ff7a0 0x7f60140ffbc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:26.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.464+0000 7f601b1ec700 1 -- 172.21.15.130:0/526711994 >> 172.21.15.130:0/526711994 conn(0x7f60140fad20 msgr2=0x7f60140fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:26.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.464+0000 7f601b1ec700 1 -- 172.21.15.130:0/526711994 shutdown_connections 2024-05-13T22:40:26.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.464+0000 7f601b1ec700 1 -- 172.21.15.130:0/526711994 wait complete. 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.465+0000 7f601b1ec700 1 Processor -- start 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.465+0000 7f601b1ec700 1 -- start start 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.465+0000 7f601b1ec700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60140ff7a0 0x7f6014068b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.465+0000 7f601b1ec700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60141986d0 con 0x7f60140ff7a0 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.465+0000 7f6018f88700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60140ff7a0 0x7f6014068b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.465+0000 7f6018f88700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60140ff7a0 0x7f6014068b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:53872/0 (socket says 172.21.15.130:53872) 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.465+0000 7f6018f88700 1 -- 172.21.15.130:0/3427982148 learned_addr learned my addr 172.21.15.130:0/3427982148 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.465+0000 7f6018f88700 1 -- 172.21.15.130:0/3427982148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080095f0 con 0x7f60140ff7a0 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.466+0000 7f6018f88700 1 --2- 172.21.15.130:0/3427982148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60140ff7a0 0x7f6014068b20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6008005b40 tx=0x7f6008004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.466+0000 7f6011ffb700 1 -- 172.21.15.130:0/3427982148 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f600801d070 con 0x7f60140ff7a0 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.466+0000 7f6011ffb700 1 -- 172.21.15.130:0/3427982148 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f600800be10 con 0x7f60140ff7a0 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.466+0000 7f601b1ec700 1 -- 172.21.15.130:0/3427982148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6014198810 con 0x7f60140ff7a0 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.466+0000 7f6011ffb700 1 -- 172.21.15.130:0/3427982148 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f600800f460 con 0x7f60140ff7a0 2024-05-13T22:40:26.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.466+0000 7f601b1ec700 1 -- 172.21.15.130:0/3427982148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6014198cb0 con 0x7f60140ff7a0 2024-05-13T22:40:26.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.467+0000 7f6011ffb700 1 -- 172.21.15.130:0/3427982148 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 45204+0+0 (secure 0 0 0) 0x7f600800b960 con 0x7f60140ff7a0 2024-05-13T22:40:26.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.467+0000 7f601b1ec700 1 -- 172.21.15.130:0/3427982148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601418e3c0 con 0x7f60140ff7a0 2024-05-13T22:40:26.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.467+0000 7f6011ffb700 1 --2- 172.21.15.130:0/3427982148 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6004038270 0x7f600403a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:26.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.467+0000 7f6011ffb700 1 -- 172.21.15.130:0/3427982148 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f600804c3e0 con 0x7f60140ff7a0 2024-05-13T22:40:26.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.467+0000 7f6013fff700 1 --2- 172.21.15.130:0/3427982148 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6004038270 0x7f600403a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:26.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.468+0000 7f6013fff700 1 --2- 172.21.15.130:0/3427982148 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6004038270 0x7f600403a730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6000006fd0 tx=0x7f6000006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:26.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.472+0000 7f6011ffb700 1 -- 172.21.15.130:0/3427982148 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f600802bba0 con 0x7f60140ff7a0 2024-05-13T22:40:26.592 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.613+0000 7f6011ffb700 1 -- 172.21.15.130:0/3427982148 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f60080227e0 con 0x7f60140ff7a0 2024-05-13T22:40:26.719 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.740+0000 7f601b1ec700 1 -- 172.21.15.130:0/3427982148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f601402cc70 con 0x7f60140ff7a0 2024-05-13T22:40:26.720 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.741+0000 7f6011ffb700 1 -- 172.21.15.130:0/3427982148 <== mon.0 v2:172.21.15.102:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f600802ce60 con 0x7f60140ff7a0 2024-05-13T22:40:26.720 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:26.720 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:26.722 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.743+0000 7f601b1ec700 1 -- 172.21.15.130:0/3427982148 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6004038270 msgr2=0x7f600403a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:26.723 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.743+0000 7f601b1ec700 1 --2- 172.21.15.130:0/3427982148 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6004038270 0x7f600403a730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6000006fd0 tx=0x7f6000006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:26.723 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.743+0000 7f601b1ec700 1 -- 172.21.15.130:0/3427982148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60140ff7a0 msgr2=0x7f6014068b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:26.723 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.743+0000 7f601b1ec700 1 --2- 172.21.15.130:0/3427982148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60140ff7a0 0x7f6014068b20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6008005b40 tx=0x7f6008004740 comp rx=0 tx=0).stop 2024-05-13T22:40:26.723 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.743+0000 7f601b1ec700 1 -- 172.21.15.130:0/3427982148 shutdown_connections 2024-05-13T22:40:26.723 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.743+0000 7f601b1ec700 1 --2- 172.21.15.130:0/3427982148 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6004038270 0x7f600403a730 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:26.723 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.743+0000 7f601b1ec700 1 --2- 172.21.15.130:0/3427982148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60140ff7a0 0x7f6014068b20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:26.723 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.743+0000 7f601b1ec700 1 -- 172.21.15.130:0/3427982148 >> 172.21.15.130:0/3427982148 conn(0x7f60140fad20 msgr2=0x7f60140fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:26.723 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.743+0000 7f601b1ec700 1 -- 172.21.15.130:0/3427982148 shutdown_connections 2024-05-13T22:40:26.723 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:26.743+0000 7f601b1ec700 1 -- 172.21.15.130:0/3427982148 wait complete. 2024-05-13T22:40:26.724 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:26.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:26 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:27.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:27 smithi102 ceph-mon[28599]: mgrmap e17: smithi102.xcijrj(active, since 2s) 2024-05-13T22:40:27.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:27 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/3427982148' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:28.284 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:28.285 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:28.487 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:28.541 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-13T22:40:29.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:28 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:29.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:28 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:29.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:28 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:29.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:28 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:29.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:28 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:40:30.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.970+0000 7fde49b17700 1 -- 172.21.15.130:0/3509371081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde44105770 msgr2=0x7fde44105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:30.970 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.970+0000 7fde49b17700 1 --2- 172.21.15.130:0/3509371081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde44105770 0x7fde44105b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fde34009910 tx=0x7fde34009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.970+0000 7fde49b17700 1 -- 172.21.15.130:0/3509371081 shutdown_connections 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.970+0000 7fde49b17700 1 --2- 172.21.15.130:0/3509371081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde44105770 0x7fde44105b50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.970+0000 7fde49b17700 1 -- 172.21.15.130:0/3509371081 >> 172.21.15.130:0/3509371081 conn(0x7fde44075df0 msgr2=0x7fde44076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.970+0000 7fde49b17700 1 -- 172.21.15.130:0/3509371081 shutdown_connections 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.970+0000 7fde49b17700 1 -- 172.21.15.130:0/3509371081 wait complete. 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.971+0000 7fde49b17700 1 Processor -- start 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.971+0000 7fde49b17700 1 -- start start 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.971+0000 7fde49b17700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde44105770 0x7fde44195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.971+0000 7fde49b17700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde441962b0 con 0x7fde44105770 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.971+0000 7fde437fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde44105770 0x7fde44195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.971+0000 7fde437fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde44105770 0x7fde44195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:59772/0 (socket says 172.21.15.130:59772) 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.971+0000 7fde437fe700 1 -- 172.21.15.130:0/2349607149 learned_addr learned my addr 172.21.15.130:0/2349607149 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.971+0000 7fde437fe700 1 -- 172.21.15.130:0/2349607149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde340095f0 con 0x7fde44105770 2024-05-13T22:40:30.971 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.972+0000 7fde437fe700 1 --2- 172.21.15.130:0/2349607149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde44105770 0x7fde44195d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fde34005b40 tx=0x7fde34004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.972+0000 7fde40ff9700 1 -- 172.21.15.130:0/2349607149 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde3401d070 con 0x7fde44105770 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.972+0000 7fde40ff9700 1 -- 172.21.15.130:0/2349607149 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fde340044d0 con 0x7fde44105770 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.972+0000 7fde49b17700 1 -- 172.21.15.130:0/2349607149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde441964b0 con 0x7fde44105770 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.972+0000 7fde40ff9700 1 -- 172.21.15.130:0/2349607149 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde34003930 con 0x7fde44105770 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.972+0000 7fde49b17700 1 -- 172.21.15.130:0/2349607149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde44196870 con 0x7fde44105770 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.973+0000 7fde40ff9700 1 -- 172.21.15.130:0/2349607149 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fde34003bf0 con 0x7fde44105770 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.973+0000 7fde49b17700 1 -- 172.21.15.130:0/2349607149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde440623c0 con 0x7fde44105770 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.973+0000 7fde40ff9700 1 --2- 172.21.15.130:0/2349607149 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fde2c0384b0 0x7fde2c03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.973+0000 7fde40ff9700 1 -- 172.21.15.130:0/2349607149 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fde3404c3a0 con 0x7fde44105770 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.973+0000 7fde42ffd700 1 --2- 172.21.15.130:0/2349607149 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fde2c0384b0 0x7fde2c03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.973+0000 7fde42ffd700 1 --2- 172.21.15.130:0/2349607149 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fde2c0384b0 0x7fde2c03a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde38006fd0 tx=0x7fde38006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:30.972 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:30.979+0000 7fde40ff9700 1 -- 172.21.15.130:0/2349607149 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fde3402a540 con 0x7fde44105770 2024-05-13T22:40:31.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:31.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:31.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:31.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:31.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd/host:smithi130", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:40:31.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:40:31.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:40:31.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:30 smithi102 ceph-mon[28599]: Updating smithi102:/etc/ceph/ceph.conf 2024-05-13T22:40:31.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:30 smithi102 ceph-mon[28599]: Updating smithi130:/etc/ceph/ceph.conf 2024-05-13T22:40:31.223 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.243+0000 7fde49b17700 1 -- 172.21.15.130:0/2349607149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fde4419a860 con 0x7fde44105770 2024-05-13T22:40:31.223 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.244+0000 7fde40ff9700 1 -- 172.21.15.130:0/2349607149 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fde3402a360 con 0x7fde44105770 2024-05-13T22:40:31.223 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:31.223 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:31.225 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.246+0000 7fde49b17700 1 -- 172.21.15.130:0/2349607149 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fde2c0384b0 msgr2=0x7fde2c03a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:31.225 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.246+0000 7fde49b17700 1 --2- 172.21.15.130:0/2349607149 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fde2c0384b0 0x7fde2c03a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde38006fd0 tx=0x7fde38006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:31.225 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.246+0000 7fde49b17700 1 -- 172.21.15.130:0/2349607149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde44105770 msgr2=0x7fde44195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:31.225 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.246+0000 7fde49b17700 1 --2- 172.21.15.130:0/2349607149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde44105770 0x7fde44195d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fde34005b40 tx=0x7fde34004c80 comp rx=0 tx=0).stop 2024-05-13T22:40:31.226 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.246+0000 7fde49b17700 1 -- 172.21.15.130:0/2349607149 shutdown_connections 2024-05-13T22:40:31.226 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.246+0000 7fde49b17700 1 --2- 172.21.15.130:0/2349607149 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fde2c0384b0 0x7fde2c03a970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:31.226 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.246+0000 7fde49b17700 1 --2- 172.21.15.130:0/2349607149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde44105770 0x7fde44195d70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:31.226 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.246+0000 7fde49b17700 1 -- 172.21.15.130:0/2349607149 >> 172.21.15.130:0/2349607149 conn(0x7fde44075df0 msgr2=0x7fde440fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:31.226 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.246+0000 7fde49b17700 1 -- 172.21.15.130:0/2349607149 shutdown_connections 2024-05-13T22:40:31.226 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:31.246+0000 7fde49b17700 1 -- 172.21.15.130:0/2349607149 wait complete. 2024-05-13T22:40:31.226 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:32.362 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:40:32.362 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:40:32.362 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:40:32.362 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:40:32.362 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:40:32.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:40:32.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:32.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:32.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:32.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:32.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:32.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:40:32.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-13T22:40:32.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:40:32.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:32 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/2349607149' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:32.821 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:32.821 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:33.031 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:40:33.298 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:33 smithi102 ceph-mon[28599]: Deploying daemon ceph-exporter.smithi130 on smithi130 2024-05-13T22:40:35.026 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.881+0000 7f7881603700 1 -- 172.21.15.130:0/394366227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f787c105710 msgr2=0x7f787c105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:35.026 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.881+0000 7f7881603700 1 --2- 172.21.15.130:0/394366227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f787c105710 0x7f787c105af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f786c009910 tx=0x7f786c009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:35.026 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.881+0000 7f7881603700 1 -- 172.21.15.130:0/394366227 shutdown_connections 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.881+0000 7f7881603700 1 --2- 172.21.15.130:0/394366227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f787c105710 0x7f787c105af0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.881+0000 7f7881603700 1 -- 172.21.15.130:0/394366227 >> 172.21.15.130:0/394366227 conn(0x7f787c0fb220 msgr2=0x7f787c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.881+0000 7f7881603700 1 -- 172.21.15.130:0/394366227 shutdown_connections 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.881+0000 7f7881603700 1 -- 172.21.15.130:0/394366227 wait complete. 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.881+0000 7f7881603700 1 Processor -- start 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.882+0000 7f7881603700 1 -- start start 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.882+0000 7f7881603700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f787c105710 0x7f787c195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.882+0000 7f7881603700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c196250 con 0x7f787c105710 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.882+0000 7f787affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f787c105710 0x7f787c195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.882+0000 7f787affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f787c105710 0x7f787c195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:59792/0 (socket says 172.21.15.130:59792) 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.882+0000 7f787affd700 1 -- 172.21.15.130:0/2704904459 learned_addr learned my addr 172.21.15.130:0/2704904459 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.883+0000 7f787affd700 1 -- 172.21.15.130:0/2704904459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0095f0 con 0x7f787c105710 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.883+0000 7f787affd700 1 --2- 172.21.15.130:0/2704904459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f787c105710 0x7f787c195d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f786c005b40 tx=0x7f786c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:35.027 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.883+0000 7f7863fff700 1 -- 172.21.15.130:0/2704904459 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f786c01d070 con 0x7f787c105710 2024-05-13T22:40:35.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.883+0000 7f7881603700 1 -- 172.21.15.130:0/2704904459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c196450 con 0x7f787c105710 2024-05-13T22:40:35.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.883+0000 7f7863fff700 1 -- 172.21.15.130:0/2704904459 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f786c0044d0 con 0x7f787c105710 2024-05-13T22:40:35.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.883+0000 7f7863fff700 1 -- 172.21.15.130:0/2704904459 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f786c003910 con 0x7f787c105710 2024-05-13T22:40:35.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.883+0000 7f7881603700 1 -- 172.21.15.130:0/2704904459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c1968d0 con 0x7f787c105710 2024-05-13T22:40:35.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.884+0000 7f7863fff700 1 -- 172.21.15.130:0/2704904459 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f786c003b90 con 0x7f787c105710 2024-05-13T22:40:35.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.884+0000 7f7881603700 1 -- 172.21.15.130:0/2704904459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c0623c0 con 0x7f787c105710 2024-05-13T22:40:35.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.884+0000 7f7863fff700 1 --2- 172.21.15.130:0/2704904459 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78640384b0 0x7f786403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:35.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.884+0000 7f7863fff700 1 -- 172.21.15.130:0/2704904459 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f786c04c420 con 0x7f787c105710 2024-05-13T22:40:35.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.884+0000 7f787a7fc700 1 --2- 172.21.15.130:0/2704904459 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78640384b0 0x7f786403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:35.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.885+0000 7f787a7fc700 1 --2- 172.21.15.130:0/2704904459 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78640384b0 0x7f786403a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7870006fd0 tx=0x7f7870006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:35.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:34.889+0000 7f7863fff700 1 -- 172.21.15.130:0/2704904459 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f786c02a540 con 0x7f787c105710 2024-05-13T22:40:35.142 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.163+0000 7f7881603700 1 -- 172.21.15.130:0/2704904459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f787c19a840 con 0x7f787c105710 2024-05-13T22:40:35.143 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.164+0000 7f7863fff700 1 -- 172.21.15.130:0/2704904459 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f786c02a360 con 0x7f787c105710 2024-05-13T22:40:35.143 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:35.143 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:35.145 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.166+0000 7f7881603700 1 -- 172.21.15.130:0/2704904459 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78640384b0 msgr2=0x7f786403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:35.145 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.166+0000 7f7881603700 1 --2- 172.21.15.130:0/2704904459 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78640384b0 0x7f786403a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7870006fd0 tx=0x7f7870006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:35.145 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.166+0000 7f7881603700 1 -- 172.21.15.130:0/2704904459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f787c105710 msgr2=0x7f787c195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:35.145 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.166+0000 7f7881603700 1 --2- 172.21.15.130:0/2704904459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f787c105710 0x7f787c195d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f786c005b40 tx=0x7f786c004c80 comp rx=0 tx=0).stop 2024-05-13T22:40:35.146 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.166+0000 7f7881603700 1 -- 172.21.15.130:0/2704904459 shutdown_connections 2024-05-13T22:40:35.146 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.166+0000 7f7881603700 1 --2- 172.21.15.130:0/2704904459 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78640384b0 0x7f786403a970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:35.146 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.166+0000 7f7881603700 1 --2- 172.21.15.130:0/2704904459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f787c105710 0x7f787c195d10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:35.146 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.166+0000 7f7881603700 1 -- 172.21.15.130:0/2704904459 >> 172.21.15.130:0/2704904459 conn(0x7f787c0fb220 msgr2=0x7f787c107ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:35.146 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.166+0000 7f7881603700 1 -- 172.21.15.130:0/2704904459 shutdown_connections 2024-05-13T22:40:35.146 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:35.166+0000 7f7881603700 1 -- 172.21.15.130:0/2704904459 wait complete. 2024-05-13T22:40:35.146 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:36 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/2704904459' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:36 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:36 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:36 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:36 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:36 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi130", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:40:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:36 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi130", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-13T22:40:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:36 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:40:36.696 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:36.696 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:36.949 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:40:37.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:37 smithi102 ceph-mon[28599]: Deploying daemon crash.smithi130 on smithi130 2024-05-13T22:40:38.566 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.585+0000 7ff7c4151700 1 -- 172.21.15.130:0/3631805731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7bc105740 msgr2=0x7ff7bc105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.585+0000 7ff7c4151700 1 --2- 172.21.15.130:0/3631805731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7bc105740 0x7ff7bc105b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00b0a0 tx=0x7ff7ac00b3b0 comp rx=0 tx=0).stop 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.585+0000 7ff7c4151700 1 -- 172.21.15.130:0/3631805731 shutdown_connections 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.585+0000 7ff7c4151700 1 --2- 172.21.15.130:0/3631805731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7bc105740 0x7ff7bc105b20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.585+0000 7ff7c4151700 1 -- 172.21.15.130:0/3631805731 >> 172.21.15.130:0/3631805731 conn(0x7ff7bc0fb250 msgr2=0x7ff7bc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.585+0000 7ff7c4151700 1 -- 172.21.15.130:0/3631805731 shutdown_connections 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.585+0000 7ff7c4151700 1 -- 172.21.15.130:0/3631805731 wait complete. 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.586+0000 7ff7c4151700 1 Processor -- start 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.586+0000 7ff7c4151700 1 -- start start 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.586+0000 7ff7c4151700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7bc105740 0x7ff7bc195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.586+0000 7ff7c4151700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc196290 con 0x7ff7bc105740 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.586+0000 7ff7c1eed700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7bc105740 0x7ff7bc195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.586+0000 7ff7c1eed700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7bc105740 0x7ff7bc195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:59830/0 (socket says 172.21.15.130:59830) 2024-05-13T22:40:38.567 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.586+0000 7ff7c1eed700 1 -- 172.21.15.130:0/1706994837 learned_addr learned my addr 172.21.15.130:0/1706994837 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:38.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.587+0000 7ff7c1eed700 1 -- 172.21.15.130:0/1706994837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac009d00 con 0x7ff7bc105740 2024-05-13T22:40:38.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.587+0000 7ff7c1eed700 1 --2- 172.21.15.130:0/1706994837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7bc105740 0x7ff7bc195d50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00d010 tx=0x7ff7ac003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:38.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.587+0000 7ff7baffd700 1 -- 172.21.15.130:0/1706994837 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7ac00e040 con 0x7ff7bc105740 2024-05-13T22:40:38.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.587+0000 7ff7baffd700 1 -- 172.21.15.130:0/1706994837 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff7ac0044f0 con 0x7ff7bc105740 2024-05-13T22:40:38.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.587+0000 7ff7c4151700 1 -- 172.21.15.130:0/1706994837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc196490 con 0x7ff7bc105740 2024-05-13T22:40:38.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.587+0000 7ff7baffd700 1 -- 172.21.15.130:0/1706994837 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7ac01d920 con 0x7ff7bc105740 2024-05-13T22:40:38.568 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.587+0000 7ff7c4151700 1 -- 172.21.15.130:0/1706994837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc196910 con 0x7ff7bc105740 2024-05-13T22:40:38.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.588+0000 7ff7c4151700 1 -- 172.21.15.130:0/1706994837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7bc0623c0 con 0x7ff7bc105740 2024-05-13T22:40:38.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.588+0000 7ff7baffd700 1 -- 172.21.15.130:0/1706994837 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7ff7ac004040 con 0x7ff7bc105740 2024-05-13T22:40:38.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.588+0000 7ff7baffd700 1 --2- 172.21.15.130:0/1706994837 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff7a40384b0 0x7ff7a403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:38.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.588+0000 7ff7baffd700 1 -- 172.21.15.130:0/1706994837 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff7ac04c420 con 0x7ff7bc105740 2024-05-13T22:40:38.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.588+0000 7ff7c16ec700 1 --2- 172.21.15.130:0/1706994837 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff7a40384b0 0x7ff7a403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:38.569 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.589+0000 7ff7c16ec700 1 --2- 172.21.15.130:0/1706994837 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff7a40384b0 0x7ff7a403a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0006fd0 tx=0x7ff7b0006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:38.573 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.594+0000 7ff7baffd700 1 -- 172.21.15.130:0/1706994837 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff7ac011570 con 0x7ff7bc105740 2024-05-13T22:40:38.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.876+0000 7ff7c4151700 1 -- 172.21.15.130:0/1706994837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff7bc19a800 con 0x7ff7bc105740 2024-05-13T22:40:38.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.877+0000 7ff7baffd700 1 -- 172.21.15.130:0/1706994837 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7ff7ac011390 con 0x7ff7bc105740 2024-05-13T22:40:38.856 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:38.857 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:38.861 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.879+0000 7ff7c4151700 1 -- 172.21.15.130:0/1706994837 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff7a40384b0 msgr2=0x7ff7a403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:38.861 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.879+0000 7ff7c4151700 1 --2- 172.21.15.130:0/1706994837 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff7a40384b0 0x7ff7a403a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0006fd0 tx=0x7ff7b0006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:38.861 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.879+0000 7ff7c4151700 1 -- 172.21.15.130:0/1706994837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7bc105740 msgr2=0x7ff7bc195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:38.861 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.879+0000 7ff7c4151700 1 --2- 172.21.15.130:0/1706994837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7bc105740 0x7ff7bc195d50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00d010 tx=0x7ff7ac003ad0 comp rx=0 tx=0).stop 2024-05-13T22:40:38.861 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.879+0000 7ff7c4151700 1 -- 172.21.15.130:0/1706994837 shutdown_connections 2024-05-13T22:40:38.861 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.879+0000 7ff7c4151700 1 --2- 172.21.15.130:0/1706994837 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff7a40384b0 0x7ff7a403a970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:38.861 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.879+0000 7ff7c4151700 1 --2- 172.21.15.130:0/1706994837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7bc105740 0x7ff7bc195d50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:38.861 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.879+0000 7ff7c4151700 1 -- 172.21.15.130:0/1706994837 >> 172.21.15.130:0/1706994837 conn(0x7ff7bc0fb250 msgr2=0x7ff7bc069160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:38.862 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.879+0000 7ff7c4151700 1 -- 172.21.15.130:0/1706994837 shutdown_connections 2024-05-13T22:40:38.862 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:38.879+0000 7ff7c4151700 1 -- 172.21.15.130:0/1706994837 wait complete. 2024-05-13T22:40:38.862 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:39.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:39.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:40:39.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:39 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/1706994837' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:39.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:39.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:39.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:39.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:40.777 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:40.777 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:40.975 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:40:41.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:41 smithi102 ceph-mon[28599]: Deploying daemon node-exporter.smithi130 on smithi130 2024-05-13T22:40:43.116 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.135+0000 7fb3296c8700 1 -- 172.21.15.130:0/1501402247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb324101a30 msgr2=0x7fb324101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:43.116 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.135+0000 7fb3296c8700 1 --2- 172.21.15.130:0/1501402247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb324101a30 0x7fb324101e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb314009910 tx=0x7fb314009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:43.116 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.135+0000 7fb3296c8700 1 -- 172.21.15.130:0/1501402247 shutdown_connections 2024-05-13T22:40:43.116 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.135+0000 7fb3296c8700 1 --2- 172.21.15.130:0/1501402247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb324101a30 0x7fb324101e10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:43.116 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.135+0000 7fb3296c8700 1 -- 172.21.15.130:0/1501402247 >> 172.21.15.130:0/1501402247 conn(0x7fb3240fb220 msgr2=0x7fb3240fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:43.116 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.135+0000 7fb3296c8700 1 -- 172.21.15.130:0/1501402247 shutdown_connections 2024-05-13T22:40:43.116 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.135+0000 7fb3296c8700 1 -- 172.21.15.130:0/1501402247 wait complete. 2024-05-13T22:40:43.116 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.135+0000 7fb3296c8700 1 Processor -- start 2024-05-13T22:40:43.116 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.136+0000 7fb3296c8700 1 -- start start 2024-05-13T22:40:43.117 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.136+0000 7fb3296c8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb324101a30 0x7fb324195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:43.117 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.136+0000 7fb3296c8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3241962f0 con 0x7fb324101a30 2024-05-13T22:40:43.117 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.136+0000 7fb322ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb324101a30 0x7fb324195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:43.117 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.136+0000 7fb322ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb324101a30 0x7fb324195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:45482/0 (socket says 172.21.15.130:45482) 2024-05-13T22:40:43.117 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.136+0000 7fb322ffd700 1 -- 172.21.15.130:0/3119519038 learned_addr learned my addr 172.21.15.130:0/3119519038 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:43.117 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.137+0000 7fb322ffd700 1 -- 172.21.15.130:0/3119519038 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140095f0 con 0x7fb324101a30 2024-05-13T22:40:43.117 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.137+0000 7fb322ffd700 1 --2- 172.21.15.130:0/3119519038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb324101a30 0x7fb324195db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb314005b40 tx=0x7fb314004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:43.118 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.137+0000 7fb30bfff700 1 -- 172.21.15.130:0/3119519038 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb31401d070 con 0x7fb324101a30 2024-05-13T22:40:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.137+0000 7fb3296c8700 1 -- 172.21.15.130:0/3119519038 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3241964f0 con 0x7fb324101a30 2024-05-13T22:40:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.137+0000 7fb30bfff700 1 -- 172.21.15.130:0/3119519038 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb3140044d0 con 0x7fb324101a30 2024-05-13T22:40:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.137+0000 7fb30bfff700 1 -- 172.21.15.130:0/3119519038 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb314003930 con 0x7fb324101a30 2024-05-13T22:40:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.137+0000 7fb3296c8700 1 -- 172.21.15.130:0/3119519038 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3241968b0 con 0x7fb324101a30 2024-05-13T22:40:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.138+0000 7fb3296c8700 1 -- 172.21.15.130:0/3119519038 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3240623c0 con 0x7fb324101a30 2024-05-13T22:40:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.138+0000 7fb30bfff700 1 -- 172.21.15.130:0/3119519038 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fb314003c10 con 0x7fb324101a30 2024-05-13T22:40:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.138+0000 7fb30bfff700 1 --2- 172.21.15.130:0/3119519038 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb30c0384b0 0x7fb30c03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.138+0000 7fb30bfff700 1 -- 172.21.15.130:0/3119519038 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb31404c510 con 0x7fb324101a30 2024-05-13T22:40:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.138+0000 7fb3227fc700 1 --2- 172.21.15.130:0/3119519038 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb30c0384b0 0x7fb30c03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:43.119 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.139+0000 7fb3227fc700 1 --2- 172.21.15.130:0/3119519038 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb30c0384b0 0x7fb30c03a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb318006fd0 tx=0x7fb318006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:43.123 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.144+0000 7fb30bfff700 1 -- 172.21.15.130:0/3119519038 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb31402a540 con 0x7fb324101a30 2024-05-13T22:40:43.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.430+0000 7fb3296c8700 1 -- 172.21.15.130:0/3119519038 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb3241079f0 con 0x7fb324101a30 2024-05-13T22:40:43.410 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.431+0000 7fb30bfff700 1 -- 172.21.15.130:0/3119519038 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fb31402a360 con 0x7fb324101a30 2024-05-13T22:40:43.410 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:43.410 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:43.412 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.433+0000 7fb3296c8700 1 -- 172.21.15.130:0/3119519038 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb30c0384b0 msgr2=0x7fb30c03a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:43.412 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.433+0000 7fb3296c8700 1 --2- 172.21.15.130:0/3119519038 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb30c0384b0 0x7fb30c03a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb318006fd0 tx=0x7fb318006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:43.412 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.433+0000 7fb3296c8700 1 -- 172.21.15.130:0/3119519038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb324101a30 msgr2=0x7fb324195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:43.412 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.433+0000 7fb3296c8700 1 --2- 172.21.15.130:0/3119519038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb324101a30 0x7fb324195db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb314005b40 tx=0x7fb314004c80 comp rx=0 tx=0).stop 2024-05-13T22:40:43.412 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.433+0000 7fb3296c8700 1 -- 172.21.15.130:0/3119519038 shutdown_connections 2024-05-13T22:40:43.412 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.433+0000 7fb3296c8700 1 --2- 172.21.15.130:0/3119519038 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb30c0384b0 0x7fb30c03a970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:43.413 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.433+0000 7fb3296c8700 1 --2- 172.21.15.130:0/3119519038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb324101a30 0x7fb324195db0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:43.413 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.433+0000 7fb3296c8700 1 -- 172.21.15.130:0/3119519038 >> 172.21.15.130:0/3119519038 conn(0x7fb3240fb220 msgr2=0x7fb324105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:43.413 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.433+0000 7fb3296c8700 1 -- 172.21.15.130:0/3119519038 shutdown_connections 2024-05-13T22:40:43.413 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:43.433+0000 7fb3296c8700 1 -- 172.21.15.130:0/3119519038 wait complete. 2024-05-13T22:40:43.413 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:43.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:43 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/3119519038' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:44.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:44 smithi102 ceph-mon[28599]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:44.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:44 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:45.427 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:45.428 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:46.402 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:40:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:45 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:45 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:45 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:45 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:45 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi130.uoqoxl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:40:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:45 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi130.uoqoxl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-13T22:40:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:45 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:40:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:45 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:40:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:45 smithi102 ceph-mon[28599]: Deploying daemon mgr.smithi130.uoqoxl on smithi130 2024-05-13T22:40:47.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:47 smithi102 ceph-mon[28599]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:48.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.091+0000 7f88819a5700 1 -- 172.21.15.130:0/362973861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c105780 msgr2=0x7f887c105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:48.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.091+0000 7f88819a5700 1 --2- 172.21.15.130:0/362973861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c105780 0x7f887c105b60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f886c009910 tx=0x7f886c009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:48.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.091+0000 7f88819a5700 1 -- 172.21.15.130:0/362973861 shutdown_connections 2024-05-13T22:40:48.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.091+0000 7f88819a5700 1 --2- 172.21.15.130:0/362973861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c105780 0x7f887c105b60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:48.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.091+0000 7f88819a5700 1 -- 172.21.15.130:0/362973861 >> 172.21.15.130:0/362973861 conn(0x7f887c0752b0 msgr2=0x7f887c0756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:48.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.091+0000 7f88819a5700 1 -- 172.21.15.130:0/362973861 shutdown_connections 2024-05-13T22:40:48.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.091+0000 7f88819a5700 1 -- 172.21.15.130:0/362973861 wait complete. 2024-05-13T22:40:48.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.092+0000 7f88819a5700 1 Processor -- start 2024-05-13T22:40:48.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.092+0000 7f88819a5700 1 -- start start 2024-05-13T22:40:48.072 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.092+0000 7f88819a5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c105780 0x7f887c195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:48.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.092+0000 7f88819a5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1962e0 con 0x7f887c105780 2024-05-13T22:40:48.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.093+0000 7f887affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c105780 0x7f887c195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:48.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.093+0000 7f887affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c105780 0x7f887c195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:45500/0 (socket says 172.21.15.130:45500) 2024-05-13T22:40:48.073 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.093+0000 7f887affd700 1 -- 172.21.15.130:0/786861668 learned_addr learned my addr 172.21.15.130:0/786861668 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:48.074 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.093+0000 7f887affd700 1 -- 172.21.15.130:0/786861668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c0095f0 con 0x7f887c105780 2024-05-13T22:40:48.074 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.093+0000 7f887affd700 1 --2- 172.21.15.130:0/786861668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c105780 0x7f887c195da0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f886c005b40 tx=0x7f886c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:48.074 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.093+0000 7f88809a3700 1 -- 172.21.15.130:0/786861668 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f886c01d070 con 0x7f887c105780 2024-05-13T22:40:48.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.093+0000 7f88819a5700 1 -- 172.21.15.130:0/786861668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c1964e0 con 0x7f887c105780 2024-05-13T22:40:48.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.093+0000 7f88809a3700 1 -- 172.21.15.130:0/786861668 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f886c0044d0 con 0x7f887c105780 2024-05-13T22:40:48.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.093+0000 7f88809a3700 1 -- 172.21.15.130:0/786861668 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f886c003930 con 0x7f887c105780 2024-05-13T22:40:48.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.093+0000 7f88819a5700 1 -- 172.21.15.130:0/786861668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c1968a0 con 0x7f887c105780 2024-05-13T22:40:48.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.094+0000 7f88809a3700 1 -- 172.21.15.130:0/786861668 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f886c00b960 con 0x7f887c105780 2024-05-13T22:40:48.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.094+0000 7f88819a5700 1 -- 172.21.15.130:0/786861668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c0623c0 con 0x7f887c105780 2024-05-13T22:40:48.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.095+0000 7f88809a3700 1 --2- 172.21.15.130:0/786861668 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88640384b0 0x7f886403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:48.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.095+0000 7f88809a3700 1 -- 172.21.15.130:0/786861668 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f886c04c510 con 0x7f887c105780 2024-05-13T22:40:48.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.095+0000 7f887a7fc700 1 --2- 172.21.15.130:0/786861668 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88640384b0 0x7f886403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:48.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.095+0000 7f887a7fc700 1 --2- 172.21.15.130:0/786861668 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88640384b0 0x7f886403a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8870006fd0 tx=0x7f8870006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:48.079 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.100+0000 7f88809a3700 1 -- 172.21.15.130:0/786861668 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f886c02a540 con 0x7f887c105780 2024-05-13T22:40:48.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.385+0000 7f88819a5700 1 -- 172.21.15.130:0/786861668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f887c19a890 con 0x7f887c105780 2024-05-13T22:40:48.365 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.385+0000 7f88809a3700 1 -- 172.21.15.130:0/786861668 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f886c02a360 con 0x7f887c105780 2024-05-13T22:40:48.365 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:48.365 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:48.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.388+0000 7f88819a5700 1 -- 172.21.15.130:0/786861668 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88640384b0 msgr2=0x7f886403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:48.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.388+0000 7f88819a5700 1 --2- 172.21.15.130:0/786861668 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88640384b0 0x7f886403a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8870006fd0 tx=0x7f8870006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:48.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.388+0000 7f88819a5700 1 -- 172.21.15.130:0/786861668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c105780 msgr2=0x7f887c195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:48.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.388+0000 7f88819a5700 1 --2- 172.21.15.130:0/786861668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c105780 0x7f887c195da0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f886c005b40 tx=0x7f886c004c80 comp rx=0 tx=0).stop 2024-05-13T22:40:48.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.388+0000 7f88819a5700 1 -- 172.21.15.130:0/786861668 shutdown_connections 2024-05-13T22:40:48.367 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.388+0000 7f88819a5700 1 --2- 172.21.15.130:0/786861668 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88640384b0 0x7f886403a970 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:48.368 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.388+0000 7f88819a5700 1 --2- 172.21.15.130:0/786861668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c105780 0x7f887c195da0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:48.368 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.388+0000 7f88819a5700 1 -- 172.21.15.130:0/786861668 >> 172.21.15.130:0/786861668 conn(0x7f887c0752b0 msgr2=0x7f887c067fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:48.368 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.388+0000 7f88819a5700 1 -- 172.21.15.130:0/786861668 shutdown_connections 2024-05-13T22:40:48.368 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:48.388+0000 7f88819a5700 1 -- 172.21.15.130:0/786861668 wait complete. 2024-05-13T22:40:48.368 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:49.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:49 smithi102 ceph-mon[28599]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:49.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:49 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:49.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:49 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:49.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:49 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:49.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:49 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:49.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:49 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:40:49.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:49 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:40:49.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:49 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/786861668' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:49.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:49 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:50.078 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:50.079 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:50.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:50 smithi102 ceph-mon[28599]: Deploying daemon mon.smithi130 on smithi130 2024-05-13T22:40:51.238 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi130/config 2024-05-13T22:40:51.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:51 smithi102 ceph-mon[28599]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:52.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.891+0000 7fe951b48700 1 -- 172.21.15.130:0/2524984995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe94c105840 msgr2=0x7fe94c105c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:52.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.891+0000 7fe951b48700 1 --2- 172.21.15.130:0/2524984995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe94c105840 0x7fe94c105c20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009910 tx=0x7fe93c009c20 comp rx=0 tx=0).stop 2024-05-13T22:40:52.875 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.891+0000 7fe951b48700 1 -- 172.21.15.130:0/2524984995 shutdown_connections 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.891+0000 7fe951b48700 1 --2- 172.21.15.130:0/2524984995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe94c105840 0x7fe94c105c20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.891+0000 7fe951b48700 1 -- 172.21.15.130:0/2524984995 >> 172.21.15.130:0/2524984995 conn(0x7fe94c0fb350 msgr2=0x7fe94c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.891+0000 7fe951b48700 1 -- 172.21.15.130:0/2524984995 shutdown_connections 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.891+0000 7fe951b48700 1 -- 172.21.15.130:0/2524984995 wait complete. 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.892+0000 7fe951b48700 1 Processor -- start 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.892+0000 7fe951b48700 1 -- start start 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.893+0000 7fe951b48700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe94c105840 0x7fe94c196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.893+0000 7fe951b48700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c1971e0 con 0x7fe94c105840 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.893+0000 7fe94b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe94c105840 0x7fe94c196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.893+0000 7fe94b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe94c105840 0x7fe94c196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:57976/0 (socket says 172.21.15.130:57976) 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.893+0000 7fe94b7fe700 1 -- 172.21.15.130:0/800884311 learned_addr learned my addr 172.21.15.130:0/800884311 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.893+0000 7fe94b7fe700 1 -- 172.21.15.130:0/800884311 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c0095f0 con 0x7fe94c105840 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.893+0000 7fe94b7fe700 1 --2- 172.21.15.130:0/800884311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe94c105840 0x7fe94c196ca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe93c005b40 tx=0x7fe93c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.894+0000 7fe948ff9700 1 -- 172.21.15.130:0/800884311 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe93c01d070 con 0x7fe94c105840 2024-05-13T22:40:52.876 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.894+0000 7fe951b48700 1 -- 172.21.15.130:0/800884311 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c1973e0 con 0x7fe94c105840 2024-05-13T22:40:52.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.894+0000 7fe948ff9700 1 -- 172.21.15.130:0/800884311 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe93c0044d0 con 0x7fe94c105840 2024-05-13T22:40:52.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.894+0000 7fe948ff9700 1 -- 172.21.15.130:0/800884311 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe93c003910 con 0x7fe94c105840 2024-05-13T22:40:52.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.894+0000 7fe951b48700 1 -- 172.21.15.130:0/800884311 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c197860 con 0x7fe94c105840 2024-05-13T22:40:52.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.895+0000 7fe951b48700 1 -- 172.21.15.130:0/800884311 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94c103af0 con 0x7fe94c105840 2024-05-13T22:40:52.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.895+0000 7fe948ff9700 1 -- 172.21.15.130:0/800884311 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fe93c003b90 con 0x7fe94c105840 2024-05-13T22:40:52.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.895+0000 7fe948ff9700 1 --2- 172.21.15.130:0/800884311 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe9340384b0 0x7fe93403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:52.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.895+0000 7fe948ff9700 1 -- 172.21.15.130:0/800884311 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fe93c04c4c0 con 0x7fe94c105840 2024-05-13T22:40:52.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.895+0000 7fe94affd700 1 --2- 172.21.15.130:0/800884311 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe9340384b0 0x7fe93403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:52.877 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.896+0000 7fe94affd700 1 --2- 172.21.15.130:0/800884311 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe9340384b0 0x7fe93403a970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe940006fd0 tx=0x7fe940006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:52.879 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:52.900+0000 7fe948ff9700 1 -- 172.21.15.130:0/800884311 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe93c02a540 con 0x7fe94c105840 2024-05-13T22:40:53.161 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.181+0000 7fe951b48700 1 -- 172.21.15.130:0/800884311 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe94c197aa0 con 0x7fe94c105840 2024-05-13T22:40:53.161 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.181+0000 7fe948ff9700 1 -- 172.21.15.130:0/800884311 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fe93c02a360 con 0x7fe94c105840 2024-05-13T22:40:53.161 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:53.161 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:38:11.166577Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-13T22:40:53.163 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.184+0000 7fe951b48700 1 -- 172.21.15.130:0/800884311 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe9340384b0 msgr2=0x7fe93403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:53.163 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.184+0000 7fe951b48700 1 --2- 172.21.15.130:0/800884311 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe9340384b0 0x7fe93403a970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe940006fd0 tx=0x7fe940006e40 comp rx=0 tx=0).stop 2024-05-13T22:40:53.163 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.184+0000 7fe951b48700 1 -- 172.21.15.130:0/800884311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe94c105840 msgr2=0x7fe94c196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:53.163 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.184+0000 7fe951b48700 1 --2- 172.21.15.130:0/800884311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe94c105840 0x7fe94c196ca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe93c005b40 tx=0x7fe93c004c80 comp rx=0 tx=0).stop 2024-05-13T22:40:53.163 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.184+0000 7fe951b48700 1 -- 172.21.15.130:0/800884311 shutdown_connections 2024-05-13T22:40:53.163 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.184+0000 7fe951b48700 1 --2- 172.21.15.130:0/800884311 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe9340384b0 0x7fe93403a970 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:53.163 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.184+0000 7fe951b48700 1 --2- 172.21.15.130:0/800884311 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe94c105840 0x7fe94c196ca0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:53.163 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.184+0000 7fe951b48700 1 -- 172.21.15.130:0/800884311 >> 172.21.15.130:0/800884311 conn(0x7fe94c0fb350 msgr2=0x7fe94c107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:53.163 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.184+0000 7fe951b48700 1 -- 172.21.15.130:0/800884311 shutdown_connections 2024-05-13T22:40:53.164 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:53.184+0000 7fe951b48700 1 -- 172.21.15.130:0/800884311 wait complete. 2024-05-13T22:40:53.164 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-05-13T22:40:53.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:53 smithi102 ceph-mon[28599]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:55.042 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-13T22:40:55.043 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-13T22:40:55.247 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi130/config 2024-05-13T22:40:58.327 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.347+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/795470431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f28106940 msgr2=0x7f2f28106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:58.328 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.347+0000 7f2f2e0a4700 1 --2- 172.21.15.130:0/795470431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f28106940 0x7f2f28106d20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2f18005fa0 tx=0x7f2f1800fe80 comp rx=0 tx=0).stop 2024-05-13T22:40:58.328 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.347+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/795470431 shutdown_connections 2024-05-13T22:40:58.328 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.347+0000 7f2f2e0a4700 1 --2- 172.21.15.130:0/795470431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f28106940 0x7f2f28106d20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:58.328 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.347+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/795470431 >> 172.21.15.130:0/795470431 conn(0x7f2f280fc430 msgr2=0x7f2f280fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:58.328 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.347+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/795470431 shutdown_connections 2024-05-13T22:40:58.328 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.347+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/795470431 wait complete. 2024-05-13T22:40:58.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.348+0000 7f2f2e0a4700 1 Processor -- start 2024-05-13T22:40:58.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.348+0000 7f2f2e0a4700 1 -- start start 2024-05-13T22:40:58.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.348+0000 7f2f2e0a4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f28106940 0x7f2f28195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:58.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.348+0000 7f2f2e0a4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f28196430 0x7f2f2819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:58.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.348+0000 7f2f2e0a4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f28196a50 con 0x7f2f28106940 2024-05-13T22:40:58.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.348+0000 7f2f2e0a4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f28196bc0 con 0x7f2f28196430 2024-05-13T22:40:58.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.348+0000 7f2f277fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f28106940 0x7f2f28195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:58.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.348+0000 7f2f1ffff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f28196430 0x7f2f2819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:58.330 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.348+0000 7f2f277fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f28106940 0x7f2f28195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.130:58060/0 (socket says 172.21.15.130:58060) 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.348+0000 7f2f277fe700 1 -- 172.21.15.130:0/49692116 learned_addr learned my addr 172.21.15.130:0/49692116 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.349+0000 7f2f1ffff700 1 -- 172.21.15.130:0/49692116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f28196430 msgr2=0x7f2f2819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.349+0000 7f2f1ffff700 1 -- 172.21.15.130:0/49692116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f28196430 msgr2=0x7f2f2819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.349+0000 7f2f1ffff700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f28196430 0x7f2f2819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.349+0000 7f2f1ffff700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f28196430 0x7f2f2819a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.349+0000 7f2f277fe700 1 -- 172.21.15.130:0/49692116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f28196430 msgr2=0x7f2f2819a8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.349+0000 7f2f277fe700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f28196430 0x7f2f2819a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.349+0000 7f2f277fe700 1 -- 172.21.15.130:0/49692116 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1800f880 con 0x7f2f28106940 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.349+0000 7f2f277fe700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f28106940 0x7f2f28195ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2f18003c00 tx=0x7f2f18003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.350+0000 7f2f257fa700 1 -- 172.21.15.130:0/49692116 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f18021070 con 0x7f2f28106940 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.350+0000 7f2f257fa700 1 -- 172.21.15.130:0/49692116 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2f180048d0 con 0x7f2f28106940 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.350+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/49692116 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2819aea0 con 0x7f2f28106940 2024-05-13T22:40:58.331 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.350+0000 7f2f257fa700 1 -- 172.21.15.130:0/49692116 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f18012e80 con 0x7f2f28106940 2024-05-13T22:40:58.332 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.350+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/49692116 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2819b3f0 con 0x7f2f28106940 2024-05-13T22:40:58.332 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.351+0000 7f2f257fa700 1 -- 172.21.15.130:0/49692116 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f2f1802f440 con 0x7f2f28106940 2024-05-13T22:40:58.332 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.351+0000 7f2f257fa700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f14038630 0x7f2f1403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:40:58.332 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.351+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/49692116 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f28105bf0 con 0x7f2f28106940 2024-05-13T22:40:58.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.351+0000 7f2f257fa700 1 -- 172.21.15.130:0/49692116 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f2f18018070 con 0x7f2f28106940 2024-05-13T22:40:58.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.351+0000 7f2f1ffff700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f14038630 0x7f2f1403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:40:58.333 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.352+0000 7f2f1ffff700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f14038630 0x7f2f1403aaf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009850 tx=0x7f2f10008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:40:58.336 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.356+0000 7f2f257fa700 1 -- 172.21.15.130:0/49692116 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2f1802a070 con 0x7f2f28106940 2024-05-13T22:40:58.386 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.406+0000 7f2f257fa700 1 -- 172.21.15.130:0/49692116 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f2f1802fbb0 con 0x7f2f28106940 2024-05-13T22:40:58.636 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.655+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/49692116 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2f28197250 con 0x7f2f28106940 2024-05-13T22:40:58.636 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.656+0000 7f2f257fa700 1 -- 172.21.15.130:0/49692116 <== mon.0 v2:172.21.15.102:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1026 (secure 0 0 0) 0x7f2f1802a020 con 0x7f2f28106940 2024-05-13T22:40:58.636 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-05-13T22:40:58.636 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":2,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","modified":"2024-05-13T22:40:53.326926Z","created":"2024-05-13T22:38:11.166577Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi102","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi130","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-13T22:40:58.638 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.658+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/49692116 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f14038630 msgr2=0x7f2f1403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:58.638 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.658+0000 7f2f2e0a4700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f14038630 0x7f2f1403aaf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009850 tx=0x7f2f10008040 comp rx=0 tx=0).stop 2024-05-13T22:40:58.638 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.658+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/49692116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f28106940 msgr2=0x7f2f28195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:40:58.638 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.658+0000 7f2f2e0a4700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f28106940 0x7f2f28195ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2f18003c00 tx=0x7f2f18003ce0 comp rx=0 tx=0).stop 2024-05-13T22:40:58.639 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.658+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/49692116 shutdown_connections 2024-05-13T22:40:58.639 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.658+0000 7f2f2e0a4700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f14038630 0x7f2f1403aaf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:58.639 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.658+0000 7f2f2e0a4700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f28106940 0x7f2f28195ef0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:58.639 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.658+0000 7f2f2e0a4700 1 --2- 172.21.15.130:0/49692116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f28196430 0x7f2f2819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:40:58.639 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.658+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/49692116 >> 172.21.15.130:0/49692116 conn(0x7f2f280fc430 msgr2=0x7f2f280fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:40:58.639 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.658+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/49692116 shutdown_connections 2024-05-13T22:40:58.639 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:40:58.658+0000 7f2f2e0a4700 1 -- 172.21.15.130:0/49692116 wait complete. 2024-05-13T22:40:58.640 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 2 2024-05-13T22:40:58.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:40:58.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: mon.smithi102 calling monitor election 2024-05-13T22:40:58.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/2387203348' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/crt"}]: dispatch 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: mon.smithi130 calling monitor election 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: mon.smithi102 is new leader, mons smithi102,smithi130 in quorum (ranks 0,1) 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: monmap e2: 2 mons at {smithi102=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0],smithi130=[v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: fsmap 2024-05-13T22:40:58.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: osdmap e5: 0 total, 0 up, 0 in 2024-05-13T22:40:58.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: mgrmap e17: smithi102.xcijrj(active, since 34s) 2024-05-13T22:40:58.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: Standby manager daemon smithi130.uoqoxl started 2024-05-13T22:40:58.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/2387203348' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:40:58.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: overall HEALTH_OK 2024-05-13T22:40:58.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/2387203348' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/key"}]: dispatch 2024-05-13T22:40:58.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/2387203348' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:40:58.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:58.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:58.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:58 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:59.445 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-13T22:40:59.445 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-13T22:40:59.661 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:40:59.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:59 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:59.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:59 smithi102 ceph-mon[28599]: mgrmap e18: smithi102.xcijrj(active, since 34s), standbys: smithi130.uoqoxl 2024-05-13T22:40:59.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:59 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi130.uoqoxl", "id": "smithi130.uoqoxl"}]: dispatch 2024-05-13T22:40:59.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:59 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:59.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:59 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:40:59.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:59 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/49692116' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-13T22:40:59.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:59 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:40:59.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:40:59 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:41:01.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.126+0000 7fb7e88b6700 1 -- 172.21.15.102:0/941476777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7e4105820 msgr2=0x7fb7e4105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:41:01.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.126+0000 7fb7e88b6700 1 --2- 172.21.15.102:0/941476777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7e4105820 0x7fb7e4105c00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4009910 tx=0x7fb7d4009c20 comp rx=0 tx=0).stop 2024-05-13T22:41:01.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.126+0000 7fb7e88b6700 1 -- 172.21.15.102:0/941476777 shutdown_connections 2024-05-13T22:41:01.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.126+0000 7fb7e88b6700 1 --2- 172.21.15.102:0/941476777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7e4105820 0x7fb7e4105c00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:01.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.126+0000 7fb7e88b6700 1 -- 172.21.15.102:0/941476777 >> 172.21.15.102:0/941476777 conn(0x7fb7e4075e80 msgr2=0x7fb7e4076290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:41:01.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.126+0000 7fb7e88b6700 1 -- 172.21.15.102:0/941476777 shutdown_connections 2024-05-13T22:41:01.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.126+0000 7fb7e88b6700 1 -- 172.21.15.102:0/941476777 wait complete. 2024-05-13T22:41:01.107 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:01 smithi102 ceph-mon[28599]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:01.107 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:01 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:01.107 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:01 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:01.107 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:01 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:01.107 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:01 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:41:01.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.127+0000 7fb7e88b6700 1 Processor -- start 2024-05-13T22:41:01.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.127+0000 7fb7e88b6700 1 -- start start 2024-05-13T22:41:01.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.127+0000 7fb7e88b6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7e4105820 0x7fb7e4193ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:41:01.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.127+0000 7fb7e88b6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7e419cb70 0x7fb7e4194020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.127+0000 7fb7e88b6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e4194700 con 0x7fb7e4105820 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.127+0000 7fb7e88b6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e4194870 con 0x7fb7e419cb70 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7e4105820 0x7fb7e4193ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7e4105820 0x7fb7e4193ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47956/0 (socket says 172.21.15.102:47956) 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e259c700 1 -- 172.21.15.102:0/3583336592 learned_addr learned my addr 172.21.15.102:0/3583336592 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e1d9b700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7e419cb70 0x7fb7e4194020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e1d9b700 1 -- 172.21.15.102:0/3583336592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7e419cb70 msgr2=0x7fb7e4194020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e1d9b700 1 -- 172.21.15.102:0/3583336592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7e419cb70 msgr2=0x7fb7e4194020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e1d9b700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7e419cb70 0x7fb7e4194020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e1d9b700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7e419cb70 0x7fb7e4194020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e259c700 1 -- 172.21.15.102:0/3583336592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7e419cb70 msgr2=0x7fb7e4194020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e259c700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7e419cb70 0x7fb7e4194020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:01.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e259c700 1 -- 172.21.15.102:0/3583336592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d40095f0 con 0x7fb7e4105820 2024-05-13T22:41:01.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.128+0000 7fb7e259c700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7e4105820 0x7fb7e4193ac0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4005b40 tx=0x7fb7d400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:41:01.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.129+0000 7fb7d37fe700 1 -- 172.21.15.102:0/3583336592 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d401d070 con 0x7fb7e4105820 2024-05-13T22:41:01.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.129+0000 7fb7d37fe700 1 -- 172.21.15.102:0/3583336592 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb7d4005140 con 0x7fb7e4105820 2024-05-13T22:41:01.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.129+0000 7fb7d37fe700 1 -- 172.21.15.102:0/3583336592 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d4017440 con 0x7fb7e4105820 2024-05-13T22:41:01.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.129+0000 7fb7e88b6700 1 -- 172.21.15.102:0/3583336592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e41986f0 con 0x7fb7e4105820 2024-05-13T22:41:01.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.129+0000 7fb7e88b6700 1 -- 172.21.15.102:0/3583336592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e4198c40 con 0x7fb7e4105820 2024-05-13T22:41:01.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.130+0000 7fb7e88b6700 1 -- 172.21.15.102:0/3583336592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e40623c0 con 0x7fb7e4105820 2024-05-13T22:41:01.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.131+0000 7fb7d37fe700 1 -- 172.21.15.102:0/3583336592 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fb7d400f6f0 con 0x7fb7e4105820 2024-05-13T22:41:01.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.131+0000 7fb7d37fe700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb7cc06c5c0 0x7fb7cc06ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:41:01.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.131+0000 7fb7d37fe700 1 -- 172.21.15.102:0/3583336592 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb7d408b980 con 0x7fb7e4105820 2024-05-13T22:41:01.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.131+0000 7fb7e1d9b700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb7cc06c5c0 0x7fb7cc06ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:41:01.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.132+0000 7fb7e1d9b700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb7cc06c5c0 0x7fb7cc06ea80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb7e41952b0 tx=0x7fb7d8008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:41:01.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.136+0000 7fb7d37fe700 1 -- 172.21.15.102:0/3583336592 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb7d40572c0 con 0x7fb7e4105820 2024-05-13T22:41:01.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.351+0000 7fb7e88b6700 1 -- 172.21.15.102:0/3583336592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb7e4194ff0 con 0x7fb7e4105820 2024-05-13T22:41:01.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.351+0000 7fb7d37fe700 1 -- 172.21.15.102:0/3583336592 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7fb7d4026090 con 0x7fb7e4105820 2024-05-13T22:41:01.331 INFO:teuthology.orchestra.run.smithi102.stdout:# minimal ceph.conf for 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:41:01.331 INFO:teuthology.orchestra.run.smithi102.stdout:[global] 2024-05-13T22:41:01.331 INFO:teuthology.orchestra.run.smithi102.stdout: fsid = 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:41:01.331 INFO:teuthology.orchestra.run.smithi102.stdout: mon_host = [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] 2024-05-13T22:41:01.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.353+0000 7fb7e88b6700 1 -- 172.21.15.102:0/3583336592 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb7cc06c5c0 msgr2=0x7fb7cc06ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:41:01.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.353+0000 7fb7e88b6700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb7cc06c5c0 0x7fb7cc06ea80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb7e41952b0 tx=0x7fb7d8008040 comp rx=0 tx=0).stop 2024-05-13T22:41:01.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.353+0000 7fb7e88b6700 1 -- 172.21.15.102:0/3583336592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7e4105820 msgr2=0x7fb7e4193ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:41:01.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.353+0000 7fb7e88b6700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7e4105820 0x7fb7e4193ac0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4005b40 tx=0x7fb7d400bda0 comp rx=0 tx=0).stop 2024-05-13T22:41:01.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.354+0000 7fb7e88b6700 1 -- 172.21.15.102:0/3583336592 shutdown_connections 2024-05-13T22:41:01.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.354+0000 7fb7e88b6700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb7cc06c5c0 0x7fb7cc06ea80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:01.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.354+0000 7fb7e88b6700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7e4105820 0x7fb7e4193ac0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:01.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.354+0000 7fb7e88b6700 1 --2- 172.21.15.102:0/3583336592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7e419cb70 0x7fb7e4194020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:01.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.354+0000 7fb7e88b6700 1 -- 172.21.15.102:0/3583336592 >> 172.21.15.102:0/3583336592 conn(0x7fb7e4075e80 msgr2=0x7fb7e40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:41:01.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.354+0000 7fb7e88b6700 1 -- 172.21.15.102:0/3583336592 shutdown_connections 2024-05-13T22:41:01.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:01.354+0000 7fb7e88b6700 1 -- 172.21.15.102:0/3583336592 wait complete. 2024-05-13T22:41:01.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:01 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:41:01.974 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-13T22:41:01.975 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:41:01.975 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.conf 2024-05-13T22:41:02.002 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:41:02.002 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:41:02.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:02 smithi102 ceph-mon[28599]: Updating smithi102:/etc/ceph/ceph.conf 2024-05-13T22:41:02.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:02 smithi102 ceph-mon[28599]: Updating smithi130:/etc/ceph/ceph.conf 2024-05-13T22:41:02.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:02 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3583336592' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:02.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:02 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:02.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:02 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:02.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:02 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:02.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:02 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:02.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:02 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:02.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:02 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:41:02.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:02 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:41:02.065 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:02 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:02.069 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:41:02.069 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.conf 2024-05-13T22:41:02.102 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:41:02.102 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:41:02.171 INFO:tasks.cephadm:Deploying OSDs... 2024-05-13T22:41:02.171 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:41:02.171 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-05-13T22:41:02.187 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-13T22:41:02.187 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-05-13T22:41:02.243 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-13T22:41:02.243 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-13T22:41:02.243 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-13T22:41:02.243 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-13T22:41:02.244 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-05-13T22:41:02.244 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-05-13 22:40:15.448664946 +0000 2024-05-13T22:41:02.244 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-05-13 22:33:58.682183552 +0000 2024-05-13T22:41:02.244 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-05-13 22:33:58.682183552 +0000 2024-05-13T22:41:02.244 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-05-13 22:33:58.682183552 +0000 2024-05-13T22:41:02.244 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-13T22:41:02.308 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-05-13T22:41:02.308 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-05-13T22:41:02.308 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000252496 s, 2.0 MB/s 2024-05-13T22:41:02.310 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-13T22:41:02.366 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-05-13T22:41:02.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:02 smithi130 ceph-mon[36591]: Updating smithi102:/etc/ceph/ceph.conf 2024-05-13T22:41:02.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:02 smithi130 ceph-mon[36591]: Updating smithi130:/etc/ceph/ceph.conf 2024-05-13T22:41:02.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:02 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3583336592' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:02.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:02 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:02.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:02 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:02.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:02 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:02.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:02 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:02.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:02 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:02.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:02 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:41:02.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:02 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:41:02.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:02 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:02.423 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-13T22:41:02.424 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-13T22:41:02.424 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-13T22:41:02.424 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-13T22:41:02.424 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-05-13T22:41:02.424 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-05-13 22:40:15.448664946 +0000 2024-05-13T22:41:02.424 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-05-13 22:33:59.031176252 +0000 2024-05-13T22:41:02.424 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-05-13 22:33:59.031176252 +0000 2024-05-13T22:41:02.424 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-05-13 22:33:59.031176252 +0000 2024-05-13T22:41:02.424 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-13T22:41:02.489 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-05-13T22:41:02.489 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-05-13T22:41:02.489 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000204004 s, 2.5 MB/s 2024-05-13T22:41:02.490 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-13T22:41:02.547 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-05-13T22:41:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-13T22:41:02.602 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-13T22:41:02.602 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-13T22:41:02.602 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-13T22:41:02.602 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-05-13T22:41:02.602 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-05-13 22:40:15.449664926 +0000 2024-05-13T22:41:02.603 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-05-13 22:33:59.372169120 +0000 2024-05-13T22:41:02.603 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-05-13 22:33:59.372169120 +0000 2024-05-13T22:41:02.603 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-05-13 22:33:59.372169120 +0000 2024-05-13T22:41:02.603 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-13T22:41:02.667 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-05-13T22:41:02.667 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-05-13T22:41:02.667 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000194883 s, 2.6 MB/s 2024-05-13T22:41:02.668 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-13T22:41:02.725 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-05-13T22:41:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-13T22:41:02.781 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-13T22:41:02.781 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-13T22:41:02.781 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-13T22:41:02.781 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-05-13T22:41:02.781 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-05-13 22:40:15.449664926 +0000 2024-05-13T22:41:02.781 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-05-13 22:33:59.710162051 +0000 2024-05-13T22:41:02.781 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-05-13 22:33:59.710162051 +0000 2024-05-13T22:41:02.782 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-05-13 22:33:59.710162051 +0000 2024-05-13T22:41:02.782 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-13T22:41:02.844 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-05-13T22:41:02.844 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-05-13T22:41:02.844 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000173729 s, 2.9 MB/s 2024-05-13T22:41:02.845 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-13T22:41:02.901 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:41:02.901 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/scratch_devs of=/dev/stdout 2024-05-13T22:41:02.921 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-13T22:41:02.921 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_1 2024-05-13T22:41:02.980 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-13T22:41:02.980 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-13T22:41:02.981 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-13T22:41:02.981 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-13T22:41:02.981 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-05-13T22:41:02.981 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-05-13 22:40:51.714085167 +0000 2024-05-13T22:41:02.981 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-05-13 22:33:46.080651687 +0000 2024-05-13T22:41:02.981 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-05-13 22:33:46.080651687 +0000 2024-05-13T22:41:02.981 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-05-13 22:33:46.080651687 +0000 2024-05-13T22:41:02.981 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-13T22:41:03.048 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-05-13T22:41:03.049 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-05-13T22:41:03.049 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000233431 s, 2.2 MB/s 2024-05-13T22:41:03.050 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-13T22:41:03.109 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_2 2024-05-13T22:41:03.167 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-13T22:41:03.167 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-13T22:41:03.167 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-13T22:41:03.167 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-13T22:41:03.167 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-05-13T22:41:03.167 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-05-13 22:40:51.714085167 +0000 2024-05-13T22:41:03.167 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-05-13 22:33:46.431645392 +0000 2024-05-13T22:41:03.167 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-05-13 22:33:46.431645392 +0000 2024-05-13T22:41:03.167 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-05-13 22:33:46.431645392 +0000 2024-05-13T22:41:03.167 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-13T22:41:03.233 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-05-13T22:41:03.233 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-05-13T22:41:03.233 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.00023364 s, 2.2 MB/s 2024-05-13T22:41:03.234 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-13T22:41:03.295 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_3 2024-05-13T22:41:03.351 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-13T22:41:03.351 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-13T22:41:03.351 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-13T22:41:03.351 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-13T22:41:03.351 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-05-13T22:41:03.351 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-05-13 22:40:51.715085149 +0000 2024-05-13T22:41:03.352 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-05-13 22:33:46.780639133 +0000 2024-05-13T22:41:03.352 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-05-13 22:33:46.780639133 +0000 2024-05-13T22:41:03.352 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-05-13 22:33:46.780639133 +0000 2024-05-13T22:41:03.352 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-13T22:41:03.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:03 smithi130 ceph-mon[36591]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:41:03.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:03 smithi130 ceph-mon[36591]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:41:03.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:03 smithi130 ceph-mon[36591]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:03.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:03 smithi130 ceph-mon[36591]: Reconfiguring mon.smithi102 (unknown last config time)... 2024-05-13T22:41:03.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:03 smithi130 ceph-mon[36591]: Reconfiguring daemon mon.smithi102 on smithi102 2024-05-13T22:41:03.416 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-05-13T22:41:03.417 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-05-13T22:41:03.417 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.00018548 s, 2.8 MB/s 2024-05-13T22:41:03.418 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-13T22:41:03.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:03 smithi102 ceph-mon[28599]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:41:03.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:03 smithi102 ceph-mon[28599]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:41:03.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:03 smithi102 ceph-mon[28599]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:03.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:03 smithi102 ceph-mon[28599]: Reconfiguring mon.smithi102 (unknown last config time)... 2024-05-13T22:41:03.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:03 smithi102 ceph-mon[28599]: Reconfiguring daemon mon.smithi102 on smithi102 2024-05-13T22:41:03.476 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_4 2024-05-13T22:41:03.533 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-13T22:41:03.533 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-13T22:41:03.533 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-13T22:41:03.533 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-13T22:41:03.533 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-05-13T22:41:03.533 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-05-13 22:40:51.715085149 +0000 2024-05-13T22:41:03.533 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-05-13 22:33:47.169632156 +0000 2024-05-13T22:41:03.534 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-05-13 22:33:47.169632156 +0000 2024-05-13T22:41:03.534 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-05-13 22:33:47.169632156 +0000 2024-05-13T22:41:03.534 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-13T22:41:03.597 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-05-13T22:41:03.597 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-05-13T22:41:03.597 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000244342 s, 2.1 MB/s 2024-05-13T22:41:03.598 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-13T22:41:03.656 INFO:tasks.cephadm:Deploying osd.0 on smithi102 with /dev/vg_nvme/lv_4... 2024-05-13T22:41:03.656 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-13T22:41:04.090 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:41:04.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:04 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:04.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:04 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:04.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:04 smithi102 ceph-mon[28599]: Reconfiguring mgr.smithi102.xcijrj (unknown last config time)... 2024-05-13T22:41:04.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:04 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi102.xcijrj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:41:04.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:04 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:41:04.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:04 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:04.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:04 smithi102 ceph-mon[28599]: Reconfiguring daemon mgr.smithi102.xcijrj on smithi102 2024-05-13T22:41:04.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:04 smithi102 ceph-mon[28599]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:04.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:04 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:04.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:04 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:04.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:04 smithi130 ceph-mon[36591]: Reconfiguring mgr.smithi102.xcijrj (unknown last config time)... 2024-05-13T22:41:04.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:04 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi102.xcijrj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:41:04.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:04 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:41:04.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:04 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:04.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:04 smithi130 ceph-mon[36591]: Reconfiguring daemon mgr.smithi102.xcijrj on smithi102 2024-05-13T22:41:04.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:04 smithi130 ceph-mon[36591]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:05.532 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:05.532 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:05.533 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:05 smithi102 ceph-mon[28599]: Reconfiguring ceph-exporter.smithi102 (monmap changed)... 2024-05-13T22:41:05.533 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:41:05.533 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:05.533 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:05 smithi102 ceph-mon[28599]: Reconfiguring daemon ceph-exporter.smithi102 on smithi102 2024-05-13T22:41:05.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:05.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:05.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:05 smithi130 ceph-mon[36591]: Reconfiguring ceph-exporter.smithi102 (monmap changed)... 2024-05-13T22:41:05.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:41:05.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:05.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:05 smithi130 ceph-mon[36591]: Reconfiguring daemon ceph-exporter.smithi102 on smithi102 2024-05-13T22:41:06.507 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-13T22:41:06.507 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-13T22:41:06.508 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-05-13T22:41:06.508 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-05-13T22:41:06.508 INFO:teuthology.orchestra.run.smithi102.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112778 s, 930 MB/s 2024-05-13T22:41:06.508 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-05-13T22:41:06.609 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:06 smithi102 ceph-mon[28599]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:06.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:06 smithi130 ceph-mon[36591]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:07.088 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_4 2024-05-13T22:41:07.628 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:41:08.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:08 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:08.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:08 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:08.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:08 smithi102 ceph-mon[28599]: Reconfiguring crash.smithi102 (monmap changed)... 2024-05-13T22:41:08.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:08 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi102", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:41:08.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:08 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:08.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:08 smithi102 ceph-mon[28599]: Reconfiguring daemon crash.smithi102 on smithi102 2024-05-13T22:41:08.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:08 smithi102 ceph-mon[28599]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:08.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:08 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:08.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:08 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:08.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:08 smithi130 ceph-mon[36591]: Reconfiguring crash.smithi102 (monmap changed)... 2024-05-13T22:41:08.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:08 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi102", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:41:08.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:08 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:08.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:08 smithi130 ceph-mon[36591]: Reconfiguring daemon crash.smithi102 on smithi102 2024-05-13T22:41:08.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:08 smithi130 ceph-mon[36591]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:09.735 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:09 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:41:09.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:09 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:41:10.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.298+0000 7f5834f0e700 1 -- 172.21.15.102:0/1024120713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58300ffab0 msgr2=0x7f58300ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:41:10.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.298+0000 7f5834f0e700 1 --2- 172.21.15.102:0/1024120713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58300ffab0 0x7f58300ffed0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5820009910 tx=0x7f5820009c20 comp rx=0 tx=0).stop 2024-05-13T22:41:10.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.298+0000 7f5834f0e700 1 -- 172.21.15.102:0/1024120713 shutdown_connections 2024-05-13T22:41:10.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.298+0000 7f5834f0e700 1 --2- 172.21.15.102:0/1024120713 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5830100cb0 0x7f5830101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:10.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.298+0000 7f5834f0e700 1 --2- 172.21.15.102:0/1024120713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58300ffab0 0x7f58300ffed0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:10.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.298+0000 7f5834f0e700 1 -- 172.21.15.102:0/1024120713 >> 172.21.15.102:0/1024120713 conn(0x7f58300fb030 msgr2=0x7f58300fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:41:10.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.298+0000 7f5834f0e700 1 -- 172.21.15.102:0/1024120713 shutdown_connections 2024-05-13T22:41:10.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.298+0000 7f5834f0e700 1 -- 172.21.15.102:0/1024120713 wait complete. 2024-05-13T22:41:10.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f5834f0e700 1 Processor -- start 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f5834f0e700 1 -- start start 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f5834f0e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58300ffab0 0x7f58301952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f5834f0e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5830100cb0 0x7f5830195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f5834f0e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5830195e40 con 0x7f58300ffab0 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f5834f0e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5830195f80 con 0x7f5830100cb0 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f582e59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58300ffab0 0x7f58301952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f582e59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58300ffab0 0x7f58301952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48162/0 (socket says 172.21.15.102:48162) 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f582e59c700 1 -- 172.21.15.102:0/3199971766 learned_addr learned my addr 172.21.15.102:0/3199971766 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f582dd9b700 1 --2- 172.21.15.102:0/3199971766 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5830100cb0 0x7f5830195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f582e59c700 1 -- 172.21.15.102:0/3199971766 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5830100cb0 msgr2=0x7f5830195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f582e59c700 1 --2- 172.21.15.102:0/3199971766 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5830100cb0 0x7f5830195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.299+0000 7f582e59c700 1 -- 172.21.15.102:0/3199971766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58200095f0 con 0x7f58300ffab0 2024-05-13T22:41:10.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.300+0000 7f582e59c700 1 --2- 172.21.15.102:0/3199971766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58300ffab0 0x7f58301952e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f582000bda0 tx=0x7f582000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:41:10.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.300+0000 7f581f7fe700 1 -- 172.21.15.102:0/3199971766 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f582001f070 con 0x7f58300ffab0 2024-05-13T22:41:10.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.300+0000 7f581f7fe700 1 -- 172.21.15.102:0/3199971766 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5820004c50 con 0x7f58300ffab0 2024-05-13T22:41:10.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.300+0000 7f5834f0e700 1 -- 172.21.15.102:0/3199971766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5830068410 con 0x7f58300ffab0 2024-05-13T22:41:10.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.300+0000 7f581f7fe700 1 -- 172.21.15.102:0/3199971766 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58200247c0 con 0x7f58300ffab0 2024-05-13T22:41:10.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.300+0000 7f5834f0e700 1 -- 172.21.15.102:0/3199971766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5830068980 con 0x7f58300ffab0 2024-05-13T22:41:10.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.301+0000 7f5834f0e700 1 -- 172.21.15.102:0/3199971766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5830108690 con 0x7f58300ffab0 2024-05-13T22:41:10.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.302+0000 7f581f7fe700 1 -- 172.21.15.102:0/3199971766 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f58200190f0 con 0x7f58300ffab0 2024-05-13T22:41:10.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.302+0000 7f581f7fe700 1 --2- 172.21.15.102:0/3199971766 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f581806c4b0 0x7f581806e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:41:10.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.303+0000 7f581f7fe700 1 -- 172.21.15.102:0/3199971766 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f582008bc70 con 0x7f58300ffab0 2024-05-13T22:41:10.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.303+0000 7f582dd9b700 1 --2- 172.21.15.102:0/3199971766 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f581806c4b0 0x7f581806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:41:10.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.303+0000 7f582dd9b700 1 --2- 172.21.15.102:0/3199971766 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f581806c4b0 0x7f581806e970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5824006fd0 tx=0x7f5824008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:41:10.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.308+0000 7f581f7fe700 1 -- 172.21.15.102:0/3199971766 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f582005afa0 con 0x7f58300ffab0 2024-05-13T22:41:10.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:10.507+0000 7f5834f0e700 1 -- 172.21.15.102:0/3199971766 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5830105600 con 0x7f581806c4b0 2024-05-13T22:41:10.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:10 smithi102 ceph-mon[28599]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:10.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:10 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:10.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:10 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:10.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:10 smithi102 ceph-mon[28599]: Reconfiguring alertmanager.smithi102 (dependencies changed)... 2024-05-13T22:41:10.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:10 smithi102 ceph-mon[28599]: Reconfiguring daemon alertmanager.smithi102 on smithi102 2024-05-13T22:41:10.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:10 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:41:10.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:10 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:41:10.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:10 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:11.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:10 smithi130 ceph-mon[36591]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:11.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:10 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:11.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:10 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:11.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:10 smithi130 ceph-mon[36591]: Reconfiguring alertmanager.smithi102 (dependencies changed)... 2024-05-13T22:41:11.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:10 smithi130 ceph-mon[36591]: Reconfiguring daemon alertmanager.smithi102 on smithi102 2024-05-13T22:41:11.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:10 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:41:11.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:10 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:41:11.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:10 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:12.104 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:11 smithi102 ceph-mon[28599]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:41:12.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:11 smithi130 ceph-mon[36591]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:41:13.040 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:12 smithi102 ceph-mon[28599]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:13.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:12 smithi130 ceph-mon[36591]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:15.130 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:14 smithi102 ceph-mon[28599]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:15.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:14 smithi130 ceph-mon[36591]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:16.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:16 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:16.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:16 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:16.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:16 smithi130 ceph-mon[36591]: Reconfiguring grafana.smithi102 (dependencies changed)... 2024-05-13T22:41:16.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:16 smithi130 ceph-mon[36591]: Reconfiguring daemon grafana.smithi102 on smithi102 2024-05-13T22:41:16.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:16 smithi130 ceph-mon[36591]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:16.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:16 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:16.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:16 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:16.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:16 smithi102 ceph-mon[28599]: Reconfiguring grafana.smithi102 (dependencies changed)... 2024-05-13T22:41:16.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:16 smithi102 ceph-mon[28599]: Reconfiguring daemon grafana.smithi102 on smithi102 2024-05-13T22:41:16.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:16 smithi102 ceph-mon[28599]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:17.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:17 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1182861801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d8cb2e3-52d6-45f5-94a1-3e8cc6535efc"}]: dispatch 2024-05-13T22:41:17.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:17 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1182861801' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d8cb2e3-52d6-45f5-94a1-3e8cc6535efc"}]': finished 2024-05-13T22:41:17.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:17 smithi102 ceph-mon[28599]: osdmap e6: 1 total, 0 up, 1 in 2024-05-13T22:41:17.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:17 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:41:17.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:17 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1370502652' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:41:17.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:17 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1182861801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d8cb2e3-52d6-45f5-94a1-3e8cc6535efc"}]: dispatch 2024-05-13T22:41:17.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:17 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1182861801' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d8cb2e3-52d6-45f5-94a1-3e8cc6535efc"}]': finished 2024-05-13T22:41:17.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:17 smithi130 ceph-mon[36591]: osdmap e6: 1 total, 0 up, 1 in 2024-05-13T22:41:17.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:17 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:41:17.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:17 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1370502652' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:41:18.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:18 smithi130 ceph-mon[36591]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:18.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:18 smithi102 ceph-mon[28599]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:21.117 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:21 smithi102 ceph-mon[28599]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:21.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:21 smithi130 ceph-mon[36591]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:23.081 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:23 smithi102 ceph-mon[28599]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:23.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:23 smithi130 ceph-mon[36591]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:24.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:41:24.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:24 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:41:25.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:25 smithi130 ceph-mon[36591]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:25.418 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:25 smithi102 ceph-mon[28599]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:27.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:26 smithi102 ceph-mon[28599]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:27.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:26 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:27.068 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:26 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:27.069 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:26 smithi102 ceph-mon[28599]: Reconfiguring prometheus.smithi102 (dependencies changed)... 2024-05-13T22:41:27.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:26 smithi130 ceph-mon[36591]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:27.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:26 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:27.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:26 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:27.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:26 smithi130 ceph-mon[36591]: Reconfiguring prometheus.smithi102 (dependencies changed)... 2024-05-13T22:41:28.140 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:27 smithi102 ceph-mon[28599]: Reconfiguring daemon prometheus.smithi102 on smithi102 2024-05-13T22:41:28.140 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:27 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-13T22:41:28.140 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:27 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:28.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:27 smithi130 ceph-mon[36591]: Reconfiguring daemon prometheus.smithi102 on smithi102 2024-05-13T22:41:28.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:27 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-13T22:41:28.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:27 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:29.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:28 smithi130 ceph-mon[36591]: Deploying daemon osd.0 on smithi102 2024-05-13T22:41:29.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:28 smithi130 ceph-mon[36591]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:29.164 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:28 smithi102 ceph-mon[28599]: Deploying daemon osd.0 on smithi102 2024-05-13T22:41:29.164 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:28 smithi102 ceph-mon[28599]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:31.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:31 smithi130 ceph-mon[36591]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:31.408 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:31 smithi102 ceph-mon[28599]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:32.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:32 smithi130 ceph-mon[36591]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:32.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:32.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:32.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:32 smithi130 ceph-mon[36591]: Reconfiguring ceph-exporter.smithi130 (monmap changed)... 2024-05-13T22:41:32.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:41:32.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:32.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:32 smithi130 ceph-mon[36591]: Reconfiguring daemon ceph-exporter.smithi130 on smithi130 2024-05-13T22:41:33.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:32 smithi102 ceph-mon[28599]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:33.074 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:33.075 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:33.075 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:32 smithi102 ceph-mon[28599]: Reconfiguring ceph-exporter.smithi130 (monmap changed)... 2024-05-13T22:41:33.075 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:41:33.075 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:33.075 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:32 smithi102 ceph-mon[28599]: Reconfiguring daemon ceph-exporter.smithi130 on smithi130 2024-05-13T22:41:34.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:34 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:34.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:34 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:34.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:34 smithi102 ceph-mon[28599]: Reconfiguring crash.smithi130 (monmap changed)... 2024-05-13T22:41:34.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:34 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi130", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:41:34.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:34 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:34.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:34 smithi102 ceph-mon[28599]: Reconfiguring daemon crash.smithi130 on smithi130 2024-05-13T22:41:34.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:34 smithi102 ceph-mon[28599]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:34.490 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:34 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:34.490 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:34 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:34.490 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:34 smithi130 ceph-mon[36591]: Reconfiguring crash.smithi130 (monmap changed)... 2024-05-13T22:41:34.490 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:34 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi130", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:41:34.490 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:34 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:34.491 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:34 smithi130 ceph-mon[36591]: Reconfiguring daemon crash.smithi130 on smithi130 2024-05-13T22:41:34.491 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:34 smithi130 ceph-mon[36591]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:35.656 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:35 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:35.657 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:35 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:35.657 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:35 smithi102 ceph-mon[28599]: Reconfiguring mgr.smithi130.uoqoxl (monmap changed)... 2024-05-13T22:41:35.657 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:35 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi130.uoqoxl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:41:35.657 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:35 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:41:35.657 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:35 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:35.657 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:35 smithi102 ceph-mon[28599]: Reconfiguring daemon mgr.smithi130.uoqoxl on smithi130 2024-05-13T22:41:35.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:35 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:35.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:35 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:35.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:35 smithi130 ceph-mon[36591]: Reconfiguring mgr.smithi130.uoqoxl (monmap changed)... 2024-05-13T22:41:35.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:35 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi130.uoqoxl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:41:35.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:35 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:41:35.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:35 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:35.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:35 smithi130 ceph-mon[36591]: Reconfiguring daemon mgr.smithi130.uoqoxl on smithi130 2024-05-13T22:41:37.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:37 smithi130 ceph-mon[36591]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:37.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:37 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:37.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:37 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:37.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:37 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:41:37.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:37 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:41:37.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:37 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:37.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:37 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:37.153 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:37 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:37 smithi102 ceph-mon[28599]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:37 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:37 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:37 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:41:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:37 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:41:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:37 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:37 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:37 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:38.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: Reconfiguring mon.smithi130 (monmap changed)... 2024-05-13T22:41:38.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: Reconfiguring daemon mon.smithi130 on smithi130 2024-05-13T22:41:38.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:38.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:38.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-13T22:41:38.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi102.front.sepia.ceph.com:9093"}]: dispatch 2024-05-13T22:41:38.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:38.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-13T22:41:38.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi102.front.sepia.ceph.com:3000"}]: dispatch 2024-05-13T22:41:38.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:38.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:41:38.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi102.front.sepia.ceph.com:9095"}]: dispatch 2024-05-13T22:41:38.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:38.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:41:38.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: Reconfiguring mon.smithi130 (monmap changed)... 2024-05-13T22:41:38.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: Reconfiguring daemon mon.smithi130 on smithi130 2024-05-13T22:41:38.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:38.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:38.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-13T22:41:38.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi102.front.sepia.ceph.com:9093"}]: dispatch 2024-05-13T22:41:38.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:38.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-13T22:41:38.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi102.front.sepia.ceph.com:3000"}]: dispatch 2024-05-13T22:41:38.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:38.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:41:38.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi102.front.sepia.ceph.com:9095"}]: dispatch 2024-05-13T22:41:38.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:38.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:41:39.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:39 smithi130 ceph-mon[36591]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-13T22:41:39.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:39 smithi130 ceph-mon[36591]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi102.front.sepia.ceph.com:9093"}]: dispatch 2024-05-13T22:41:39.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:39 smithi130 ceph-mon[36591]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-13T22:41:39.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:39 smithi130 ceph-mon[36591]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi102.front.sepia.ceph.com:3000"}]: dispatch 2024-05-13T22:41:39.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:39 smithi130 ceph-mon[36591]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:41:39.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:39 smithi130 ceph-mon[36591]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi102.front.sepia.ceph.com:9095"}]: dispatch 2024-05-13T22:41:39.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:39 smithi130 ceph-mon[36591]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:39.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:41:39.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:39 smithi102 ceph-mon[28599]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-13T22:41:39.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:39 smithi102 ceph-mon[28599]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi102.front.sepia.ceph.com:9093"}]: dispatch 2024-05-13T22:41:39.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:39 smithi102 ceph-mon[28599]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-13T22:41:39.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:39 smithi102 ceph-mon[28599]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi102.front.sepia.ceph.com:3000"}]: dispatch 2024-05-13T22:41:39.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:39 smithi102 ceph-mon[28599]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:41:39.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:39 smithi102 ceph-mon[28599]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi102.front.sepia.ceph.com:9095"}]: dispatch 2024-05-13T22:41:39.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:39 smithi102 ceph-mon[28599]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:39.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:41:39.868 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 0 on host 'smithi102' 2024-05-13T22:41:39.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.887+0000 7f581f7fe700 1 -- 172.21.15.102:0/3199971766 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5830105600 con 0x7f581806c4b0 2024-05-13T22:41:39.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.890+0000 7f5834f0e700 1 -- 172.21.15.102:0/3199971766 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f581806c4b0 msgr2=0x7f581806e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:41:39.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.890+0000 7f5834f0e700 1 --2- 172.21.15.102:0/3199971766 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f581806c4b0 0x7f581806e970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5824006fd0 tx=0x7f5824008040 comp rx=0 tx=0).stop 2024-05-13T22:41:39.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.890+0000 7f5834f0e700 1 -- 172.21.15.102:0/3199971766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58300ffab0 msgr2=0x7f58301952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:41:39.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.890+0000 7f5834f0e700 1 --2- 172.21.15.102:0/3199971766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58300ffab0 0x7f58301952e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f582000bda0 tx=0x7f582000be80 comp rx=0 tx=0).stop 2024-05-13T22:41:39.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.890+0000 7f5834f0e700 1 -- 172.21.15.102:0/3199971766 shutdown_connections 2024-05-13T22:41:39.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.890+0000 7f5834f0e700 1 --2- 172.21.15.102:0/3199971766 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f581806c4b0 0x7f581806e970 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:39.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.890+0000 7f5834f0e700 1 --2- 172.21.15.102:0/3199971766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58300ffab0 0x7f58301952e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:39.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.890+0000 7f5834f0e700 1 --2- 172.21.15.102:0/3199971766 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5830100cb0 0x7f5830195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:39.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.890+0000 7f5834f0e700 1 -- 172.21.15.102:0/3199971766 >> 172.21.15.102:0/3199971766 conn(0x7f58300fb030 msgr2=0x7f5830103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:41:39.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.890+0000 7f5834f0e700 1 -- 172.21.15.102:0/3199971766 shutdown_connections 2024-05-13T22:41:39.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:39.890+0000 7f5834f0e700 1 -- 172.21.15.102:0/3199971766 wait complete. 2024-05-13T22:41:40.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:40 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:40.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:40 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:40.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:40 smithi102 ceph-mon[28599]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:40.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:40 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:40.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:40 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:40.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:40 smithi102 ceph-mon[28599]: from='osd.0 [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-13T22:41:40.591 DEBUG:teuthology.orchestra.run.smithi102:osd.0> sudo journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.0.service 2024-05-13T22:41:40.593 INFO:tasks.cephadm:Deploying osd.1 on smithi102 with /dev/vg_nvme/lv_3... 2024-05-13T22:41:40.593 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-13T22:41:40.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:40 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:40.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:40 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:40.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:40 smithi130 ceph-mon[36591]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:40.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:40 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:40.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:40 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:40.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:40 smithi130 ceph-mon[36591]: from='osd.0 [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-13T22:41:41.271 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:41:42.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:42 smithi130 ceph-mon[36591]: from='osd.0 [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-13T22:41:42.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:42 smithi130 ceph-mon[36591]: osdmap e7: 1 total, 0 up, 1 in 2024-05-13T22:41:42.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:41:42.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:42 smithi130 ceph-mon[36591]: from='osd.0 [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:41:42.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:42 smithi102 ceph-mon[28599]: from='osd.0 [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-13T22:41:42.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:42 smithi102 ceph-mon[28599]: osdmap e7: 1 total, 0 up, 1 in 2024-05-13T22:41:42.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:41:42.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:42 smithi102 ceph-mon[28599]: from='osd.0 [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:41:42.429 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:41:42 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[47079]: 2024-05-13T22:41:42.055+0000 7f66f565a700 -1 osd.0 0 waiting for initial osdmap 2024-05-13T22:41:42.429 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:41:42 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[47079]: 2024-05-13T22:41:42.059+0000 7f66edc48700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:41:43.269 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:43 smithi102 ceph-mon[28599]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:43.269 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:43 smithi102 ceph-mon[28599]: from='osd.0 [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-05-13T22:41:43.269 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:43 smithi102 ceph-mon[28599]: osdmap e8: 1 total, 0 up, 1 in 2024-05-13T22:41:43.269 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:43 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:41:43.269 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:43 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:41:43.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:43 smithi130 ceph-mon[36591]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:43.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:43 smithi130 ceph-mon[36591]: from='osd.0 [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-05-13T22:41:43.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:43 smithi130 ceph-mon[36591]: osdmap e8: 1 total, 0 up, 1 in 2024-05-13T22:41:43.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:43 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:41:43.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:43 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:41:43.767 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-13T22:41:43.768 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-13T22:41:43.768 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-05-13T22:41:43.768 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-05-13T22:41:43.768 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013755 s, 762 MB/s 2024-05-13T22:41:43.768 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-05-13T22:41:44.307 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:44 smithi102 ceph-mon[28599]: purged_snaps scrub starts 2024-05-13T22:41:44.307 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:44 smithi102 ceph-mon[28599]: purged_snaps scrub ok 2024-05-13T22:41:44.307 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:44 smithi102 ceph-mon[28599]: osd.0 [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722] boot 2024-05-13T22:41:44.307 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:44 smithi102 ceph-mon[28599]: osdmap e9: 1 total, 1 up, 1 in 2024-05-13T22:41:44.307 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:44 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:41:44.375 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_3 2024-05-13T22:41:44.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:44 smithi130 ceph-mon[36591]: purged_snaps scrub starts 2024-05-13T22:41:44.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:44 smithi130 ceph-mon[36591]: purged_snaps scrub ok 2024-05-13T22:41:44.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:44 smithi130 ceph-mon[36591]: osd.0 [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722] boot 2024-05-13T22:41:44.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:44 smithi130 ceph-mon[36591]: osdmap e9: 1 total, 1 up, 1 in 2024-05-13T22:41:44.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:44 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:41:44.692 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:41:45.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:45 smithi102 ceph-mon[28599]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:45.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:45 smithi102 ceph-mon[28599]: osdmap e10: 1 total, 1 up, 1 in 2024-05-13T22:41:45.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:45 smithi130 ceph-mon[36591]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-13T22:41:45.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:45 smithi130 ceph-mon[36591]: osdmap e10: 1 total, 1 up, 1 in 2024-05-13T22:41:46.587 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:46 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:46.587 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:46 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:46.587 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:46 smithi102 ceph-mon[28599]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:46.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:46 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:46.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:46 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:46.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:46 smithi130 ceph-mon[36591]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:46.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.824+0000 7f14213b1700 1 -- 172.21.15.102:0/2440390703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f141c103f70 msgr2=0x7f141c106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:41:46.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.824+0000 7f14213b1700 1 --2- 172.21.15.102:0/2440390703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f141c103f70 0x7f141c106360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1410009910 tx=0x7f1410009c20 comp rx=0 tx=0).stop 2024-05-13T22:41:46.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.824+0000 7f14213b1700 1 -- 172.21.15.102:0/2440390703 shutdown_connections 2024-05-13T22:41:46.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.824+0000 7f14213b1700 1 --2- 172.21.15.102:0/2440390703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f141c103f70 0x7f141c106360 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:46.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.824+0000 7f14213b1700 1 --2- 172.21.15.102:0/2440390703 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f141c101640 0x7f141c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.824+0000 7f14213b1700 1 -- 172.21.15.102:0/2440390703 >> 172.21.15.102:0/2440390703 conn(0x7f141c0fb050 msgr2=0x7f141c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.824+0000 7f14213b1700 1 -- 172.21.15.102:0/2440390703 shutdown_connections 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.824+0000 7f14213b1700 1 -- 172.21.15.102:0/2440390703 wait complete. 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.825+0000 7f14213b1700 1 Processor -- start 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.825+0000 7f14213b1700 1 -- start start 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.825+0000 7f14213b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f141c101640 0x7f141c196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.825+0000 7f14213b1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f141c103f70 0x7f141c1968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.825+0000 7f14213b1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c196ef0 con 0x7f141c101640 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.825+0000 7f14213b1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c197030 con 0x7f141c103f70 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.825+0000 7f141affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f141c101640 0x7f141c196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.825+0000 7f141affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f141c101640 0x7f141c196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59618/0 (socket says 172.21.15.102:59618) 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.825+0000 7f141affd700 1 -- 172.21.15.102:0/2480907710 learned_addr learned my addr 172.21.15.102:0/2480907710 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:41:46.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.825+0000 7f141a7fc700 1 --2- 172.21.15.102:0/2480907710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f141c103f70 0x7f141c1968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:41:46.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.826+0000 7f141affd700 1 -- 172.21.15.102:0/2480907710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f141c103f70 msgr2=0x7f141c1968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:41:46.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.826+0000 7f141affd700 1 --2- 172.21.15.102:0/2480907710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f141c103f70 0x7f141c1968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:41:46.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.826+0000 7f141affd700 1 -- 172.21.15.102:0/2480907710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14100095f0 con 0x7f141c101640 2024-05-13T22:41:46.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.826+0000 7f141affd700 1 --2- 172.21.15.102:0/2480907710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f141c101640 0x7f141c196390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f140c00b4b0 tx=0x7f140c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:41:46.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.826+0000 7f1403fff700 1 -- 172.21.15.102:0/2480907710 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f140c011800 con 0x7f141c101640 2024-05-13T22:41:46.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.826+0000 7f1403fff700 1 -- 172.21.15.102:0/2480907710 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f140c011e10 con 0x7f141c101640 2024-05-13T22:41:46.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.826+0000 7f1403fff700 1 -- 172.21.15.102:0/2480907710 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f140c00f2e0 con 0x7f141c101640 2024-05-13T22:41:46.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.826+0000 7f14213b1700 1 -- 172.21.15.102:0/2480907710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c19bae0 con 0x7f141c101640 2024-05-13T22:41:46.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.826+0000 7f14213b1700 1 -- 172.21.15.102:0/2480907710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c19c030 con 0x7f141c101640 2024-05-13T22:41:46.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.827+0000 7f14213b1700 1 -- 172.21.15.102:0/2480907710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141c1905a0 con 0x7f141c101640 2024-05-13T22:41:46.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.828+0000 7f1403fff700 1 -- 172.21.15.102:0/2480907710 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f140c00f510 con 0x7f141c101640 2024-05-13T22:41:46.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.828+0000 7f1403fff700 1 --2- 172.21.15.102:0/2480907710 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f140406c470 0x7f140406e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:41:46.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.828+0000 7f1403fff700 1 -- 172.21.15.102:0/2480907710 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f140c08a540 con 0x7f141c101640 2024-05-13T22:41:46.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.828+0000 7f141a7fc700 1 --2- 172.21.15.102:0/2480907710 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f140406c470 0x7f140406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:41:46.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.829+0000 7f141a7fc700 1 --2- 172.21.15.102:0/2480907710 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f140406c470 0x7f140406e930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1410006010 tx=0x7f141000b560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:41:46.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:46.833+0000 7f1403fff700 1 -- 172.21.15.102:0/2480907710 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f140c059640 con 0x7f141c101640 2024-05-13T22:41:47.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:41:47.038+0000 7f14213b1700 1 -- 172.21.15.102:0/2480907710 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f141c0611d0 con 0x7f140406c470 2024-05-13T22:41:47.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:47 smithi130 ceph-mon[36591]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:41:47.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:47 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:41:47.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:47 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:41:47.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:47 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:47.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:47 smithi102 ceph-mon[28599]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:41:47.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:47 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:41:47.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:47 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:41:47.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:47 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:48.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:48 smithi102 ceph-mon[28599]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:48 smithi130 ceph-mon[36591]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:51.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:51 smithi102 ceph-mon[28599]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:51.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:51 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:51.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:51 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:51.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:51 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:41:51.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:51 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:51.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:51 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:51.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:51 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:41:51.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:51 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:51.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:51 smithi130 ceph-mon[36591]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:51.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:51 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:51.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:51 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:51.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:51 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:41:51.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:51 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:51.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:51 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:51.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:51 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:41:51.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:51 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:52.348 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:52 smithi130 ceph-mon[36591]: Detected new or changed devices on smithi102 2024-05-13T22:41:52.348 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:52 smithi130 ceph-mon[36591]: Adjusting osd_memory_target on smithi102 to 11840M 2024-05-13T22:41:52.348 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:52 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:41:52.349 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:52 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1815868487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d3eacb1-362c-4b6a-a959-7dea19c6521e"}]: dispatch 2024-05-13T22:41:52.349 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:52 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1815868487' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d3eacb1-362c-4b6a-a959-7dea19c6521e"}]': finished 2024-05-13T22:41:52.349 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:52 smithi130 ceph-mon[36591]: osdmap e11: 2 total, 1 up, 2 in 2024-05-13T22:41:52.349 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:52 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:41:52.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:52 smithi102 ceph-mon[28599]: Detected new or changed devices on smithi102 2024-05-13T22:41:52.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:52 smithi102 ceph-mon[28599]: Adjusting osd_memory_target on smithi102 to 11840M 2024-05-13T22:41:52.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:52 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:41:52.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:52 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1815868487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d3eacb1-362c-4b6a-a959-7dea19c6521e"}]: dispatch 2024-05-13T22:41:52.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:52 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1815868487' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d3eacb1-362c-4b6a-a959-7dea19c6521e"}]': finished 2024-05-13T22:41:52.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:52 smithi102 ceph-mon[28599]: osdmap e11: 2 total, 1 up, 2 in 2024-05-13T22:41:52.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:52 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:41:53.317 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:53 smithi102 ceph-mon[28599]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:53.317 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:53 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1586381638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:41:53.317 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:53 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:53.318 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:53 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:53.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:53 smithi130 ceph-mon[36591]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:53.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:53 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1586381638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:41:53.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:53 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:53.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:53 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:54.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:54 smithi102 ceph-mon[28599]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:54.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:54 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:41:54.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:54 smithi130 ceph-mon[36591]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:54.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:54 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:41:56.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:56 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:56.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:56 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:56.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:56 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:56.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:56 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:41:56.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:56 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:56.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:56 smithi130 ceph-mon[36591]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:56.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:56 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:56.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:56 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:56.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:56 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:41:56.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:56 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:41:56.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:56 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:41:56.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:56 smithi102 ceph-mon[28599]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:58.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:58 smithi102 ceph-mon[28599]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:41:59.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:58 smithi130 ceph-mon[36591]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:00.010 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:59 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-13T22:42:00.010 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:41:59 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:00.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:59 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-13T22:42:00.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:41:59 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:01.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:01 smithi102 ceph-mon[28599]: Deploying daemon osd.1 on smithi102 2024-05-13T22:42:01.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:01 smithi102 ceph-mon[28599]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:01.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:01 smithi130 ceph-mon[36591]: Deploying daemon osd.1 on smithi102 2024-05-13T22:42:01.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:01 smithi130 ceph-mon[36591]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:03.320 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:03 smithi102 ceph-mon[28599]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:03.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:03 smithi130 ceph-mon[36591]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:05.213 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:05 smithi130 ceph-mon[36591]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:05.213 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:05.213 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:05.213 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:42:05.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:05 smithi102 ceph-mon[28599]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:05.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:05.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:05.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:42:07.375 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:07 smithi102 ceph-mon[28599]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:07.375 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:07 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:07.375 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:07 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:07.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:07 smithi130 ceph-mon[36591]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:07.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:07 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:07.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:07 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:08.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.052+0000 7f1403fff700 1 -- 172.21.15.102:0/2480907710 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f141c0611d0 con 0x7f140406c470 2024-05-13T22:42:08.032 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 1 on host 'smithi102' 2024-05-13T22:42:08.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.055+0000 7f14213b1700 1 -- 172.21.15.102:0/2480907710 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f140406c470 msgr2=0x7f140406e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:42:08.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.055+0000 7f14213b1700 1 --2- 172.21.15.102:0/2480907710 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f140406c470 0x7f140406e930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1410006010 tx=0x7f141000b560 comp rx=0 tx=0).stop 2024-05-13T22:42:08.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.055+0000 7f14213b1700 1 -- 172.21.15.102:0/2480907710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f141c101640 msgr2=0x7f141c196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:42:08.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.055+0000 7f14213b1700 1 --2- 172.21.15.102:0/2480907710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f141c101640 0x7f141c196390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f140c00b4b0 tx=0x7f140c00b7c0 comp rx=0 tx=0).stop 2024-05-13T22:42:08.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.055+0000 7f14213b1700 1 -- 172.21.15.102:0/2480907710 shutdown_connections 2024-05-13T22:42:08.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.055+0000 7f14213b1700 1 --2- 172.21.15.102:0/2480907710 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f140406c470 0x7f140406e930 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:08.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.055+0000 7f14213b1700 1 --2- 172.21.15.102:0/2480907710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f141c101640 0x7f141c196390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:08.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.055+0000 7f14213b1700 1 --2- 172.21.15.102:0/2480907710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f141c103f70 0x7f141c1968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:08.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.055+0000 7f14213b1700 1 -- 172.21.15.102:0/2480907710 >> 172.21.15.102:0/2480907710 conn(0x7f141c0fb050 msgr2=0x7f141c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:42:08.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.055+0000 7f14213b1700 1 -- 172.21.15.102:0/2480907710 shutdown_connections 2024-05-13T22:42:08.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:08.055+0000 7f14213b1700 1 -- 172.21.15.102:0/2480907710 wait complete. 2024-05-13T22:42:08.979 DEBUG:teuthology.orchestra.run.smithi102:osd.1> sudo journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.1.service 2024-05-13T22:42:08.982 INFO:tasks.cephadm:Deploying osd.2 on smithi102 with /dev/vg_nvme/lv_2... 2024-05-13T22:42:08.983 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-13T22:42:09.246 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:09 smithi102 ceph-mon[28599]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:09.246 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:09 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:09.246 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:09 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:09.246 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:09 smithi102 ceph-mon[28599]: from='osd.1 [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-13T22:42:09.246 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:09 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:42:09.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:09 smithi130 ceph-mon[36591]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:09.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:09 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:09.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:09 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:09.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:09 smithi130 ceph-mon[36591]: from='osd.1 [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-13T22:42:09.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:09 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:42:09.650 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:42:10.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:10 smithi130 ceph-mon[36591]: from='osd.1 [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-13T22:42:10.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:10 smithi130 ceph-mon[36591]: osdmap e12: 2 total, 1 up, 2 in 2024-05-13T22:42:10.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:10 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:42:10.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:10 smithi130 ceph-mon[36591]: from='osd.1 [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:42:10.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:10 smithi102 ceph-mon[28599]: from='osd.1 [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-13T22:42:10.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:10 smithi102 ceph-mon[28599]: osdmap e12: 2 total, 1 up, 2 in 2024-05-13T22:42:10.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:10 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:42:10.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:10 smithi102 ceph-mon[28599]: from='osd.1 [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:42:10.429 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:42:10 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[54781]: 2024-05-13T22:42:10.070+0000 7f8a2f015700 -1 osd.1 0 waiting for initial osdmap 2024-05-13T22:42:10.429 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:42:10 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[54781]: 2024-05-13T22:42:10.075+0000 7f8a27e04700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:42:11.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:11 smithi130 ceph-mon[36591]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:11 smithi130 ceph-mon[36591]: from='osd.1 [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-05-13T22:42:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:11 smithi130 ceph-mon[36591]: osdmap e13: 2 total, 1 up, 2 in 2024-05-13T22:42:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:42:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:42:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:11 smithi102 ceph-mon[28599]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:11 smithi102 ceph-mon[28599]: from='osd.1 [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-05-13T22:42:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:11 smithi102 ceph-mon[28599]: osdmap e13: 2 total, 1 up, 2 in 2024-05-13T22:42:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:42:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:42:12.136 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-13T22:42:12.136 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-13T22:42:12.137 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-05-13T22:42:12.137 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-05-13T22:42:12.137 INFO:teuthology.orchestra.run.smithi102.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0135393 s, 774 MB/s 2024-05-13T22:42:12.137 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-05-13T22:42:12.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:12 smithi130 ceph-mon[36591]: purged_snaps scrub starts 2024-05-13T22:42:12.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:12 smithi130 ceph-mon[36591]: purged_snaps scrub ok 2024-05-13T22:42:12.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:12 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:42:12.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:12 smithi130 ceph-mon[36591]: osd.1 [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323] boot 2024-05-13T22:42:12.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:12 smithi130 ceph-mon[36591]: osdmap e14: 2 total, 2 up, 2 in 2024-05-13T22:42:12.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:12 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:42:12.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:12 smithi102 ceph-mon[28599]: purged_snaps scrub starts 2024-05-13T22:42:12.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:12 smithi102 ceph-mon[28599]: purged_snaps scrub ok 2024-05-13T22:42:12.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:12 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:42:12.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:12 smithi102 ceph-mon[28599]: osd.1 [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323] boot 2024-05-13T22:42:12.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:12 smithi102 ceph-mon[28599]: osdmap e14: 2 total, 2 up, 2 in 2024-05-13T22:42:12.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:12 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:42:12.811 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_2 2024-05-13T22:42:13.153 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:42:13.333 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:13 smithi102 ceph-mon[28599]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:13.333 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:13 smithi102 ceph-mon[28599]: osdmap e15: 2 total, 2 up, 2 in 2024-05-13T22:42:13.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:13 smithi130 ceph-mon[36591]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-13T22:42:13.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:13 smithi130 ceph-mon[36591]: osdmap e15: 2 total, 2 up, 2 in 2024-05-13T22:42:14.512 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:14 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:14.512 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:14 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:14.512 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:14 smithi102 ceph-mon[28599]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:14.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:14 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:14.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:14 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:14.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:14 smithi130 ceph-mon[36591]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:15.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.060+0000 7f45e1d80700 1 -- 172.21.15.102:0/2368103758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc0ffab0 msgr2=0x7f45dc0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:42:15.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.060+0000 7f45e1d80700 1 --2- 172.21.15.102:0/2368103758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc0ffab0 0x7f45dc0ffed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00b0a0 tx=0x7f45cc00b3b0 comp rx=0 tx=0).stop 2024-05-13T22:42:15.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.060+0000 7f45e1d80700 1 -- 172.21.15.102:0/2368103758 shutdown_connections 2024-05-13T22:42:15.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.060+0000 7f45e1d80700 1 --2- 172.21.15.102:0/2368103758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45dc100cb0 0x7f45dc101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:15.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.060+0000 7f45e1d80700 1 --2- 172.21.15.102:0/2368103758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc0ffab0 0x7f45dc0ffed0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:15.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.060+0000 7f45e1d80700 1 -- 172.21.15.102:0/2368103758 >> 172.21.15.102:0/2368103758 conn(0x7f45dc0fb050 msgr2=0x7f45dc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:42:15.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.060+0000 7f45e1d80700 1 -- 172.21.15.102:0/2368103758 shutdown_connections 2024-05-13T22:42:15.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.060+0000 7f45e1d80700 1 -- 172.21.15.102:0/2368103758 wait complete. 2024-05-13T22:42:15.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.060+0000 7f45e1d80700 1 Processor -- start 2024-05-13T22:42:15.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.060+0000 7f45e1d80700 1 -- start start 2024-05-13T22:42:15.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45e1d80700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45dc0ffab0 0x7f45dc1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:42:15.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45e1d80700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc100cb0 0x7f45dc196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:42:15.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45e1d80700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc196eb0 con 0x7f45dc100cb0 2024-05-13T22:42:15.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45e1d80700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc196ff0 con 0x7f45dc0ffab0 2024-05-13T22:42:15.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45daffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc100cb0 0x7f45dc196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:42:15.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45daffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc100cb0 0x7f45dc196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47572/0 (socket says 172.21.15.102:47572) 2024-05-13T22:42:15.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45daffd700 1 -- 172.21.15.102:0/3218660889 learned_addr learned my addr 172.21.15.102:0/3218660889 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:42:15.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45daffd700 1 -- 172.21.15.102:0/3218660889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45dc0ffab0 msgr2=0x7f45dc1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45db7fe700 1 --2- 172.21.15.102:0/3218660889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45dc0ffab0 0x7f45dc1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45daffd700 1 --2- 172.21.15.102:0/3218660889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45dc0ffab0 0x7f45dc1963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45daffd700 1 -- 172.21.15.102:0/3218660889 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc009d00 con 0x7f45dc100cb0 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45daffd700 1 --2- 172.21.15.102:0/3218660889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc100cb0 0x7f45dc196920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f45d000c930 tx=0x7f45d000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45d8ff9700 1 -- 172.21.15.102:0/3218660889 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d0007ad0 con 0x7f45dc100cb0 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45d8ff9700 1 -- 172.21.15.102:0/3218660889 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f45d0007c30 con 0x7f45dc100cb0 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45d8ff9700 1 -- 172.21.15.102:0/3218660889 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d0018630 con 0x7f45dc100cb0 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45e1d80700 1 -- 172.21.15.102:0/3218660889 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45dc19bab0 con 0x7f45dc100cb0 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.061+0000 7f45e1d80700 1 -- 172.21.15.102:0/3218660889 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc19c050 con 0x7f45dc100cb0 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.062+0000 7f45e1d80700 1 -- 172.21.15.102:0/3218660889 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45dc108610 con 0x7f45dc100cb0 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.063+0000 7f45d8ff9700 1 -- 172.21.15.102:0/3218660889 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f45d000f450 con 0x7f45dc100cb0 2024-05-13T22:42:15.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.063+0000 7f45d8ff9700 1 --2- 172.21.15.102:0/3218660889 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f45c406c470 0x7f45c406e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:42:15.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.063+0000 7f45d8ff9700 1 -- 172.21.15.102:0/3218660889 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f45d008ac20 con 0x7f45dc100cb0 2024-05-13T22:42:15.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.063+0000 7f45db7fe700 1 --2- 172.21.15.102:0/3218660889 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f45c406c470 0x7f45c406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:42:15.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.064+0000 7f45db7fe700 1 --2- 172.21.15.102:0/3218660889 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f45c406c470 0x7f45c406e930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45cc015040 tx=0x7f45cc000f40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:42:15.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.068+0000 7f45d8ff9700 1 -- 172.21.15.102:0/3218660889 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f45d0059be0 con 0x7f45dc100cb0 2024-05-13T22:42:15.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:15.268+0000 7f45e1d80700 1 -- 172.21.15.102:0/3218660889 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f45dc0693b0 con 0x7f45c406c470 2024-05-13T22:42:16.119 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:16 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:42:16.119 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:16 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:42:16.119 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:16 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:16.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:16 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:42:16.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:16 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:42:16.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:16 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:17.337 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:17 smithi102 ceph-mon[28599]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:42:17.337 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:17 smithi102 ceph-mon[28599]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:17.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:17 smithi130 ceph-mon[36591]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:42:17.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:17 smithi130 ceph-mon[36591]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:19.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:19 smithi130 ceph-mon[36591]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:19.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:19 smithi130 ceph-mon[36591]: Detected new or changed devices on smithi102 2024-05-13T22:42:19.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:19 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:19.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:19 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:19.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:19 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:42:19.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:19 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:42:19.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:19 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:19.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:19 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:19.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:19 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:42:19.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:19 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:19.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:19 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:42:19.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:19 smithi102 ceph-mon[28599]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:19.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:19 smithi102 ceph-mon[28599]: Detected new or changed devices on smithi102 2024-05-13T22:42:19.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:19 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:19.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:19 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:19.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:19 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:42:19.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:19 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:42:19.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:19 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:19.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:19 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:19.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:19 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:42:19.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:19 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:19.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:19 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:42:20.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:20 smithi130 ceph-mon[36591]: Adjusting osd_memory_target on smithi102 to 5920M 2024-05-13T22:42:20.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:20 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3371418289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "205b05a9-9d68-48ed-b87b-c4c677882eed"}]: dispatch 2024-05-13T22:42:20.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:20 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3371418289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "205b05a9-9d68-48ed-b87b-c4c677882eed"}]': finished 2024-05-13T22:42:20.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:20 smithi130 ceph-mon[36591]: osdmap e16: 3 total, 2 up, 3 in 2024-05-13T22:42:20.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:20 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:42:20.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:20 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3520087049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:42:20.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:20 smithi102 ceph-mon[28599]: Adjusting osd_memory_target on smithi102 to 5920M 2024-05-13T22:42:20.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:20 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3371418289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "205b05a9-9d68-48ed-b87b-c4c677882eed"}]: dispatch 2024-05-13T22:42:20.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:20 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3371418289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "205b05a9-9d68-48ed-b87b-c4c677882eed"}]': finished 2024-05-13T22:42:20.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:20 smithi102 ceph-mon[28599]: osdmap e16: 3 total, 2 up, 3 in 2024-05-13T22:42:20.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:20 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:42:20.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:20 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3520087049' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:42:21.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:21 smithi102 ceph-mon[28599]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:21.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:21 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:21.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:21 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:21.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:21 smithi130 ceph-mon[36591]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:21.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:21 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:21.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:21 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:22.212 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:22 smithi102 ceph-mon[28599]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:22 smithi130 ceph-mon[36591]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:23.791 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:23 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:23.791 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:23 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:23.791 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:23 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:23.791 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:23 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:42:23.791 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:23 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:23.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:23 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:23.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:23 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:23.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:23 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:23.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:23 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:42:23.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:23 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:24.922 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:24 smithi102 ceph-mon[28599]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:24.922 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:42:24.923 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:24 smithi130 ceph-mon[36591]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:24.923 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:24 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:42:27.283 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:27 smithi102 ceph-mon[28599]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:27.283 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:27 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-13T22:42:27.283 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:27 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:27.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:27 smithi130 ceph-mon[36591]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:27.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:27 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-13T22:42:27.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:27 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:28.173 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:28 smithi102 ceph-mon[28599]: Deploying daemon osd.2 on smithi102 2024-05-13T22:42:28.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:28 smithi130 ceph-mon[36591]: Deploying daemon osd.2 on smithi102 2024-05-13T22:42:29.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:29 smithi130 ceph-mon[36591]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:29.410 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:29 smithi102 ceph-mon[28599]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:31.392 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:31 smithi102 ceph-mon[28599]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:31.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:31 smithi130 ceph-mon[36591]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:32.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:32.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:32.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:42:32.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:32 smithi130 ceph-mon[36591]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:32.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:32.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:32.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:42:32.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:32 smithi102 ceph-mon[28599]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:35.032 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:34 smithi102 ceph-mon[28599]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:35.103 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 2 on host 'smithi102' 2024-05-13T22:42:35.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.123+0000 7f45d8ff9700 1 -- 172.21.15.102:0/3218660889 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f45dc0693b0 con 0x7f45c406c470 2024-05-13T22:42:35.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.126+0000 7f45e1d80700 1 -- 172.21.15.102:0/3218660889 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f45c406c470 msgr2=0x7f45c406e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:42:35.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.126+0000 7f45e1d80700 1 --2- 172.21.15.102:0/3218660889 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f45c406c470 0x7f45c406e930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45cc015040 tx=0x7f45cc000f40 comp rx=0 tx=0).stop 2024-05-13T22:42:35.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.126+0000 7f45e1d80700 1 -- 172.21.15.102:0/3218660889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc100cb0 msgr2=0x7f45dc196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:42:35.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.126+0000 7f45e1d80700 1 --2- 172.21.15.102:0/3218660889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc100cb0 0x7f45dc196920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f45d000c930 tx=0x7f45d000ccf0 comp rx=0 tx=0).stop 2024-05-13T22:42:35.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.126+0000 7f45e1d80700 1 -- 172.21.15.102:0/3218660889 shutdown_connections 2024-05-13T22:42:35.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.126+0000 7f45e1d80700 1 --2- 172.21.15.102:0/3218660889 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f45c406c470 0x7f45c406e930 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:35.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.126+0000 7f45e1d80700 1 --2- 172.21.15.102:0/3218660889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f45dc0ffab0 0x7f45dc1963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:35.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.126+0000 7f45e1d80700 1 --2- 172.21.15.102:0/3218660889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc100cb0 0x7f45dc196920 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:35.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.126+0000 7f45e1d80700 1 -- 172.21.15.102:0/3218660889 >> 172.21.15.102:0/3218660889 conn(0x7f45dc0fb050 msgr2=0x7f45dc068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:42:35.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.126+0000 7f45e1d80700 1 -- 172.21.15.102:0/3218660889 shutdown_connections 2024-05-13T22:42:35.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:42:35.126+0000 7f45e1d80700 1 -- 172.21.15.102:0/3218660889 wait complete. 2024-05-13T22:42:35.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:34 smithi130 ceph-mon[36591]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:35.951 DEBUG:teuthology.orchestra.run.smithi102:osd.2> sudo journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.2.service 2024-05-13T22:42:35.954 INFO:tasks.cephadm:Deploying osd.3 on smithi130 with /dev/vg_nvme/lv_4... 2024-05-13T22:42:35.954 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-13T22:42:36.173 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi130/config 2024-05-13T22:42:36.190 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:36 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:36 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:36 smithi102 ceph-mon[28599]: from='osd.2 [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-13T22:42:36.295 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:36 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:36.296 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:36 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:36.296 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:36 smithi130 ceph-mon[36591]: from='osd.2 [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-13T22:42:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:37 smithi102 ceph-mon[28599]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:37 smithi102 ceph-mon[28599]: from='osd.2 [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-13T22:42:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:37 smithi102 ceph-mon[28599]: osdmap e17: 3 total, 2 up, 3 in 2024-05-13T22:42:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:37 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:42:37.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:37 smithi102 ceph-mon[28599]: from='osd.2 [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:42:37.179 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:42:37 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[62121]: 2024-05-13T22:42:37.106+0000 7fd50673f700 -1 osd.2 0 waiting for initial osdmap 2024-05-13T22:42:37.179 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:42:37 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[62121]: 2024-05-13T22:42:37.111+0000 7fd501d33700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:42:37.350 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:37 smithi130 ceph-mon[36591]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:37.350 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:37 smithi130 ceph-mon[36591]: from='osd.2 [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-13T22:42:37.350 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:37 smithi130 ceph-mon[36591]: osdmap e17: 3 total, 2 up, 3 in 2024-05-13T22:42:37.350 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:37 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:42:37.350 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:37 smithi130 ceph-mon[36591]: from='osd.2 [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:42:37.990 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-13T22:42:37.990 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-13T22:42:37.990 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-05-13T22:42:37.990 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-05-13T22:42:37.990 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111181 s, 943 MB/s 2024-05-13T22:42:37.990 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-05-13T22:42:38.289 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:38 smithi130 ceph-mon[36591]: from='osd.2 [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-05-13T22:42:38.289 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:38 smithi130 ceph-mon[36591]: osdmap e18: 3 total, 2 up, 3 in 2024-05-13T22:42:38.289 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:42:38.290 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:42:38.290 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:38 smithi130 ceph-mon[36591]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:38.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:38 smithi102 ceph-mon[28599]: from='osd.2 [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-05-13T22:42:38.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:38 smithi102 ceph-mon[28599]: osdmap e18: 3 total, 2 up, 3 in 2024-05-13T22:42:38.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:42:38.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:42:38.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:38 smithi102 ceph-mon[28599]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-13T22:42:38.580 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_4 2024-05-13T22:42:38.787 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi130/config 2024-05-13T22:42:39.202 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:39 smithi130 ceph-mon[36591]: purged_snaps scrub starts 2024-05-13T22:42:39.202 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:39 smithi130 ceph-mon[36591]: purged_snaps scrub ok 2024-05-13T22:42:39.202 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:39 smithi130 ceph-mon[36591]: osd.2 [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373] boot 2024-05-13T22:42:39.202 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:39 smithi130 ceph-mon[36591]: osdmap e19: 3 total, 3 up, 3 in 2024-05-13T22:42:39.202 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:42:39.203 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:39.203 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:39.203 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:42:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:39 smithi102 ceph-mon[28599]: purged_snaps scrub starts 2024-05-13T22:42:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:39 smithi102 ceph-mon[28599]: purged_snaps scrub ok 2024-05-13T22:42:39.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:39 smithi102 ceph-mon[28599]: osd.2 [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373] boot 2024-05-13T22:42:39.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:39 smithi102 ceph-mon[28599]: osdmap e19: 3 total, 3 up, 3 in 2024-05-13T22:42:39.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:42:39.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:39.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:39.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:42:40.259 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:40 smithi130 ceph-mon[36591]: osdmap e20: 3 total, 3 up, 3 in 2024-05-13T22:42:40.260 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:40 smithi130 ceph-mon[36591]: pgmap v76: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:42:40.260 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:40 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-13T22:42:40.394 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.414+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/2633900637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d440feea0 msgr2=0x7f3d440ff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.414+0000 7f3d4b7a8700 1 --2- 172.21.15.130:0/2633900637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d440feea0 0x7f3d440ff2c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000b0a0 tx=0x7f3d4000b3b0 comp rx=0 tx=0).stop 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.414+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/2633900637 shutdown_connections 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.414+0000 7f3d4b7a8700 1 --2- 172.21.15.130:0/2633900637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d44101f20 0x7f3d44102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.414+0000 7f3d4b7a8700 1 --2- 172.21.15.130:0/2633900637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d440feea0 0x7f3d440ff2c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.414+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/2633900637 >> 172.21.15.130:0/2633900637 conn(0x7f3d440fa7c0 msgr2=0x7f3d440fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.414+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/2633900637 shutdown_connections 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.414+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/2633900637 wait complete. 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d4b7a8700 1 Processor -- start 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d4b7a8700 1 -- start start 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d4b7a8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d440feea0 0x7f3d441963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d4b7a8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d44101f20 0x7f3d44196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:42:40.395 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d4b7a8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d44196f20 con 0x7f3d44101f20 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d4b7a8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d44197060 con 0x7f3d440feea0 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d49544700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d440feea0 0x7f3d441963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d49544700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d440feea0 0x7f3d441963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:37950/0 (socket says 172.21.15.130:37950) 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d49544700 1 -- 172.21.15.130:0/1919925617 learned_addr learned my addr 172.21.15.130:0/1919925617 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d48d43700 1 --2- 172.21.15.130:0/1919925617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d44101f20 0x7f3d44196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d49544700 1 -- 172.21.15.130:0/1919925617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d44101f20 msgr2=0x7f3d44196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d49544700 1 --2- 172.21.15.130:0/1919925617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d44101f20 0x7f3d44196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d49544700 1 -- 172.21.15.130:0/1919925617 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d40009d00 con 0x7f3d440feea0 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.415+0000 7f3d49544700 1 --2- 172.21.15.130:0/1919925617 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d440feea0 0x7f3d441963c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3d400077b0 tx=0x7f3d40009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.416+0000 7f3d3a7fc700 1 -- 172.21.15.130:0/1919925617 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d4000e040 con 0x7f3d440feea0 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.416+0000 7f3d3a7fc700 1 -- 172.21.15.130:0/1919925617 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3d40003de0 con 0x7f3d440feea0 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.416+0000 7f3d3a7fc700 1 -- 172.21.15.130:0/1919925617 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d4001d7f0 con 0x7f3d440feea0 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.416+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/1919925617 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4419bab0 con 0x7f3d440feea0 2024-05-13T22:42:40.396 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.416+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/1919925617 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4419bfd0 con 0x7f3d440feea0 2024-05-13T22:42:40.397 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.417+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/1919925617 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d44109740 con 0x7f3d440feea0 2024-05-13T22:42:40.397 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.417+0000 7f3d3a7fc700 1 -- 172.21.15.130:0/1919925617 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f3d40004780 con 0x7f3d440feea0 2024-05-13T22:42:40.398 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.417+0000 7f3d3a7fc700 1 --2- 172.21.15.130:0/1919925617 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f3d3006c460 0x7f3d3006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:42:40.398 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.418+0000 7f3d3a7fc700 1 -- 172.21.15.130:0/1919925617 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3221+0+0 (secure 0 0 0) 0x7f3d4008c8d0 con 0x7f3d440feea0 2024-05-13T22:42:40.398 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.418+0000 7f3d48d43700 1 --2- 172.21.15.130:0/1919925617 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f3d3006c460 0x7f3d3006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:42:40.398 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.418+0000 7f3d48d43700 1 --2- 172.21.15.130:0/1919925617 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f3d3006c460 0x7f3d3006e920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3d34004630 tx=0x7f3d3400b3c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:42:40.401 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.422+0000 7f3d3a7fc700 1 -- 172.21.15.130:0/1919925617 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3d4005b5a0 con 0x7f3d440feea0 2024-05-13T22:42:40.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:40 smithi102 ceph-mon[28599]: osdmap e20: 3 total, 3 up, 3 in 2024-05-13T22:42:40.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:40 smithi102 ceph-mon[28599]: pgmap v76: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:42:40.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:40 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-13T22:42:40.598 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:40.619+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/1919925617 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d44106870 con 0x7f3d3006c460 2024-05-13T22:42:41.371 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:41 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-13T22:42:41.371 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:41 smithi130 ceph-mon[36591]: osdmap e21: 3 total, 3 up, 3 in 2024-05-13T22:42:41.371 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:41 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-13T22:42:41.371 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:41 smithi130 ceph-mon[36591]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:42:41.371 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:41 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:42:41.371 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:41 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:42:41.371 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:41 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:41.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:41 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-13T22:42:41.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:41 smithi102 ceph-mon[28599]: osdmap e21: 3 total, 3 up, 3 in 2024-05-13T22:42:41.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:41 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-13T22:42:41.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:41 smithi102 ceph-mon[28599]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:42:41.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:41 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:42:41.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:41 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:42:41.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:41 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:42.156 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:42:41 smithi102 sudo[66877]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-13T22:42:42.156 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:42:41 smithi102 sudo[66877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-13T22:42:42.157 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:42:41 smithi102 sudo[66877]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-13T22:42:42.157 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:42:41 smithi102 sudo[66877]: pam_unix(sudo:session): session closed for user root 2024-05-13T22:42:42.225 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-13T22:42:42.225 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: osdmap e22: 3 total, 3 up, 3 in 2024-05-13T22:42:42.225 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: Detected new or changed devices on smithi102 2024-05-13T22:42:42.225 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:42.225 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:42.225 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:42:42.226 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:42:42.226 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:42:42.226 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: Adjusting osd_memory_target on smithi102 to 3946M 2024-05-13T22:42:42.226 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:42.226 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:42.226 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:42:42.226 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:42.226 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:42:42.226 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:42 smithi130 ceph-mon[36591]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:42:42.309 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:42.329+0000 7f3d3a7fc700 1 -- 172.21.15.130:0/1919925617 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f3d40057ee0 con 0x7f3d440feea0 2024-05-13T22:42:42.428 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:42:42 smithi102 sudo[66889]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-13T22:42:42.428 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:42:42 smithi102 sudo[66889]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-13T22:42:42.429 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:42:42 smithi102 sudo[66889]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-13T22:42:42.429 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:42:42 smithi102 sudo[66889]: pam_unix(sudo:session): session closed for user root 2024-05-13T22:42:42.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-13T22:42:42.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: osdmap e22: 3 total, 3 up, 3 in 2024-05-13T22:42:42.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: Detected new or changed devices on smithi102 2024-05-13T22:42:42.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:42.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:42.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:42:42.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:42:42.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:42:42.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: Adjusting osd_memory_target on smithi102 to 3946M 2024-05-13T22:42:42.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:42.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:42.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:42:42.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:42.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:42:42.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:42 smithi102 ceph-mon[28599]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:42:42.701 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:42:42 smithi102 sudo[66906]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-13T22:42:42.701 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:42:42 smithi102 sudo[66906]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-13T22:42:42.701 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:42:42 smithi102 sudo[66906]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-13T22:42:42.701 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:42:42 smithi102 sudo[66906]: pam_unix(sudo:session): session closed for user root 2024-05-13T22:42:42.964 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:42:42 smithi102 sudo[67011]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-13T22:42:42.964 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:42:42 smithi102 sudo[67011]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-13T22:42:42.964 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:42:42 smithi102 sudo[67011]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-13T22:42:42.964 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:42:42 smithi102 sudo[67011]: pam_unix(sudo:session): session closed for user root 2024-05-13T22:42:42.964 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:42:42 smithi102 sudo[67033]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-13T22:42:42.964 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:42:42 smithi102 sudo[67033]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-13T22:42:42.965 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:42:42 smithi102 sudo[67033]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-13T22:42:42.965 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:42:42 smithi102 sudo[67033]: pam_unix(sudo:session): session closed for user root 2024-05-13T22:42:43.428 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67037]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-13T22:42:43.428 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67037]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-13T22:42:43.428 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67037]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-13T22:42:43.429 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67037]: pam_unix(sudo:session): session closed for user root 2024-05-13T22:42:43.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:43.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 ceph-mon[28599]: osdmap e23: 3 total, 3 up, 3 in 2024-05-13T22:42:43.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 ceph-mon[28599]: mgrmap e19: smithi102.xcijrj(active, since 2m), standbys: smithi130.uoqoxl 2024-05-13T22:42:43.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/3532361965' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d827066c-3201-479e-b142-27721c2761fa"}]: dispatch 2024-05-13T22:42:43.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 ceph-mon[28599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d827066c-3201-479e-b142-27721c2761fa"}]: dispatch 2024-05-13T22:42:43.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 ceph-mon[28599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d827066c-3201-479e-b142-27721c2761fa"}]': finished 2024-05-13T22:42:43.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 ceph-mon[28599]: osdmap e24: 4 total, 3 up, 4 in 2024-05-13T22:42:43.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:42:43.436 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:43 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:43.436 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:43 smithi130 ceph-mon[36591]: osdmap e23: 3 total, 3 up, 3 in 2024-05-13T22:42:43.436 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:43 smithi130 ceph-mon[36591]: mgrmap e19: smithi102.xcijrj(active, since 2m), standbys: smithi130.uoqoxl 2024-05-13T22:42:43.436 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:43 smithi130 ceph-mon[36591]: from='client.? 172.21.15.130:0/3532361965' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d827066c-3201-479e-b142-27721c2761fa"}]: dispatch 2024-05-13T22:42:43.436 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:43 smithi130 ceph-mon[36591]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d827066c-3201-479e-b142-27721c2761fa"}]: dispatch 2024-05-13T22:42:43.436 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:43 smithi130 ceph-mon[36591]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d827066c-3201-479e-b142-27721c2761fa"}]': finished 2024-05-13T22:42:43.436 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:43 smithi130 ceph-mon[36591]: osdmap e24: 4 total, 3 up, 4 in 2024-05-13T22:42:43.436 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:43 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:42:43.726 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67085]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-13T22:42:43.726 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67085]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-13T22:42:43.726 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67085]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-13T22:42:43.726 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67085]: pam_unix(sudo:session): session closed for user root 2024-05-13T22:42:44.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67115]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-13T22:42:44.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-13T22:42:44.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67115]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-13T22:42:44.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:43 smithi102 sudo[67115]: pam_unix(sudo:session): session closed for user root 2024-05-13T22:42:44.359 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:43 smithi130 sudo[40598]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-13T22:42:44.359 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:43 smithi130 sudo[40598]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-13T22:42:44.359 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:43 smithi130 sudo[40598]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-13T22:42:44.360 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 sudo[40598]: pam_unix(sudo:session): session closed for user root 2024-05-13T22:42:44.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 sudo[40625]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-13T22:42:44.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 sudo[40625]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-13T22:42:44.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 sudo[40625]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-13T22:42:44.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 sudo[40625]: pam_unix(sudo:session): session closed for user root 2024-05-13T22:42:44.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 ceph-mon[36591]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-13T22:42:44.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 ceph-mon[36591]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:42:44.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 ceph-mon[36591]: from='client.? 172.21.15.130:0/2129035996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:42:44.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 ceph-mon[36591]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-13T22:42:44.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:42:44.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:42:44.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:42:44.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:44 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:42:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:44 smithi102 ceph-mon[28599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-13T22:42:44.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:44 smithi102 ceph-mon[28599]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:42:44.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:44 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/2129035996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:42:44.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:44 smithi102 ceph-mon[28599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-13T22:42:44.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:44 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:42:44.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:44 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:42:44.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:44 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:42:44.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:44 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:42:45.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:45 smithi130 ceph-mon[36591]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-13T22:42:45.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:45 smithi130 ceph-mon[36591]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-13T22:42:45.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:45 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:45.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:45 smithi102 ceph-mon[28599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-13T22:42:45.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:45 smithi102 ceph-mon[28599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-13T22:42:45.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:45 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:46.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:46 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:46.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:46 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:46.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:46 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:46.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:46 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:42:46.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:46 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:46.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:46 smithi130 ceph-mon[36591]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:42:46.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:46 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:46.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:46 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:46.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:46 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:46.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:46 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:42:46.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:46 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:46.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:46 smithi102 ceph-mon[28599]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:42:48.990 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:48 smithi130 ceph-mon[36591]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 244 KiB/s wr, 16 op/s 2024-05-13T22:42:49.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:48 smithi102 ceph-mon[28599]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 244 KiB/s wr, 16 op/s 2024-05-13T22:42:51.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:51 smithi130 ceph-mon[36591]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 178 KiB/s wr, 12 op/s 2024-05-13T22:42:51.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:51 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-13T22:42:51.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:51 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:51.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:51 smithi102 ceph-mon[28599]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 178 KiB/s wr, 12 op/s 2024-05-13T22:42:51.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:51 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-13T22:42:51.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:51 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:42:52.230 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:52 smithi130 ceph-mon[36591]: Deploying daemon osd.3 on smithi130 2024-05-13T22:42:52.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:52 smithi102 ceph-mon[28599]: Deploying daemon osd.3 on smithi130 2024-05-13T22:42:53.329 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:53 smithi130 ceph-mon[36591]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 140 KiB/s wr, 9 op/s 2024-05-13T22:42:53.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:53 smithi102 ceph-mon[28599]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 140 KiB/s wr, 9 op/s 2024-05-13T22:42:54.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:54 smithi130 ceph-mon[36591]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 123 KiB/s wr, 8 op/s 2024-05-13T22:42:54.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:54 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:42:54.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:54 smithi102 ceph-mon[28599]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 123 KiB/s wr, 8 op/s 2024-05-13T22:42:54.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:54 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:42:57.104 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:56 smithi130 ceph-mon[36591]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-05-13T22:42:57.104 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:56 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:57.104 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:56 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:57.104 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:56 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:42:57.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:56 smithi102 ceph-mon[28599]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-05-13T22:42:57.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:56 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:57.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:56 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:42:57.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:56 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:42:58.847 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:58 smithi130 ceph-mon[36591]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-05-13T22:42:58.912 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.933+0000 7f3d3a7fc700 1 -- 172.21.15.130:0/1919925617 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3d44106870 con 0x7f3d3006c460 2024-05-13T22:42:58.913 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 3 on host 'smithi130' 2024-05-13T22:42:58.915 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.936+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/1919925617 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f3d3006c460 msgr2=0x7f3d3006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:42:58.915 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.936+0000 7f3d4b7a8700 1 --2- 172.21.15.130:0/1919925617 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f3d3006c460 0x7f3d3006e920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3d34004630 tx=0x7f3d3400b3c0 comp rx=0 tx=0).stop 2024-05-13T22:42:58.916 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.936+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/1919925617 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d440feea0 msgr2=0x7f3d441963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:42:58.916 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.936+0000 7f3d4b7a8700 1 --2- 172.21.15.130:0/1919925617 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d440feea0 0x7f3d441963c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3d400077b0 tx=0x7f3d40009760 comp rx=0 tx=0).stop 2024-05-13T22:42:58.916 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.936+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/1919925617 shutdown_connections 2024-05-13T22:42:58.916 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.936+0000 7f3d4b7a8700 1 --2- 172.21.15.130:0/1919925617 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f3d3006c460 0x7f3d3006e920 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:58.916 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.936+0000 7f3d4b7a8700 1 --2- 172.21.15.130:0/1919925617 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d440feea0 0x7f3d441963c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:58.916 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.936+0000 7f3d4b7a8700 1 --2- 172.21.15.130:0/1919925617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d44101f20 0x7f3d44196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:42:58.916 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.936+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/1919925617 >> 172.21.15.130:0/1919925617 conn(0x7f3d440fa7c0 msgr2=0x7f3d44105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:42:58.916 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.937+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/1919925617 shutdown_connections 2024-05-13T22:42:58.916 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:42:58.937+0000 7f3d4b7a8700 1 -- 172.21.15.130:0/1919925617 wait complete. 2024-05-13T22:42:59.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:58 smithi102 ceph-mon[28599]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-05-13T22:42:59.874 DEBUG:teuthology.orchestra.run.smithi130:osd.3> sudo journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.3.service 2024-05-13T22:42:59.876 INFO:tasks.cephadm:Deploying osd.4 on smithi130 with /dev/vg_nvme/lv_3... 2024-05-13T22:42:59.877 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-13T22:43:00.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:59 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:00.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:59 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:00.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:59 smithi130 ceph-mon[36591]: from='osd.3 [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-13T22:43:00.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:42:59 smithi130 ceph-mon[36591]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-13T22:43:00.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:59 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:00.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:59 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:00.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:59 smithi102 ceph-mon[28599]: from='osd.3 [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-13T22:43:00.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:42:59 smithi102 ceph-mon[28599]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-13T22:43:00.362 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi130/config 2024-05-13T22:43:01.163 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:00 smithi130 ceph-mon[36591]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:43:01.163 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:00 smithi130 ceph-mon[36591]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-13T22:43:01.163 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:00 smithi130 ceph-mon[36591]: osdmap e25: 4 total, 3 up, 4 in 2024-05-13T22:43:01.163 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:00 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:43:01.163 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:00 smithi130 ceph-mon[36591]: from='osd.3 [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:01.163 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:00 smithi130 ceph-mon[36591]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:01.163 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:00 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:01.163 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:00 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:01.164 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:43:00 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[42623]: 2024-05-13T22:43:00.934+0000 7fd4e5254700 -1 osd.3 0 waiting for initial osdmap 2024-05-13T22:43:01.164 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:43:00 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[42623]: 2024-05-13T22:43:00.943+0000 7fd4e184a700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:43:01.165 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:00 smithi102 ceph-mon[28599]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:43:01.166 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:00 smithi102 ceph-mon[28599]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-13T22:43:01.166 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:00 smithi102 ceph-mon[28599]: osdmap e25: 4 total, 3 up, 4 in 2024-05-13T22:43:01.166 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:00 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:43:01.166 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:00 smithi102 ceph-mon[28599]: from='osd.3 [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:01.166 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:00 smithi102 ceph-mon[28599]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:01.166 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:00 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:01.166 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:00 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:02.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:01 smithi130 ceph-mon[36591]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-05-13T22:43:02.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:01 smithi130 ceph-mon[36591]: osdmap e26: 4 total, 3 up, 4 in 2024-05-13T22:43:02.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:01 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:43:02.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:01 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:43:02.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:01 smithi102 ceph-mon[28599]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-05-13T22:43:02.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:01 smithi102 ceph-mon[28599]: osdmap e26: 4 total, 3 up, 4 in 2024-05-13T22:43:02.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:01 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:43:02.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:01 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:43:02.876 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-13T22:43:02.876 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-13T22:43:02.876 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-05-13T22:43:02.876 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-05-13T22:43:02.876 INFO:teuthology.orchestra.run.smithi130.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131689 s, 796 MB/s 2024-05-13T22:43:02.877 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 2024-05-13T22:43:02.877 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-05-13T22:43:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:02 smithi130 ceph-mon[36591]: purged_snaps scrub starts 2024-05-13T22:43:03.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:02 smithi130 ceph-mon[36591]: purged_snaps scrub ok 2024-05-13T22:43:03.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:02 smithi130 ceph-mon[36591]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:43:03.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:02 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:43:03.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:02 smithi130 ceph-mon[36591]: osd.3 [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705] boot 2024-05-13T22:43:03.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:02 smithi130 ceph-mon[36591]: osdmap e27: 4 total, 4 up, 4 in 2024-05-13T22:43:03.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:02 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:43:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:02 smithi102 ceph-mon[28599]: purged_snaps scrub starts 2024-05-13T22:43:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:02 smithi102 ceph-mon[28599]: purged_snaps scrub ok 2024-05-13T22:43:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:02 smithi102 ceph-mon[28599]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-13T22:43:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:02 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:43:03.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:02 smithi102 ceph-mon[28599]: osd.3 [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705] boot 2024-05-13T22:43:03.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:02 smithi102 ceph-mon[28599]: osdmap e27: 4 total, 4 up, 4 in 2024-05-13T22:43:03.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:02 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:43:03.639 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_3 2024-05-13T22:43:04.132 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi130/config 2024-05-13T22:43:04.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:03 smithi130 ceph-mon[36591]: osdmap e28: 4 total, 4 up, 4 in 2024-05-13T22:43:04.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:03 smithi102 ceph-mon[28599]: osdmap e28: 4 total, 4 up, 4 in 2024-05-13T22:43:05.295 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:04 smithi130 ceph-mon[36591]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:05.295 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:04 smithi130 ceph-mon[36591]: osdmap e29: 4 total, 4 up, 4 in 2024-05-13T22:43:05.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:04 smithi102 ceph-mon[28599]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:05.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:04 smithi102 ceph-mon[28599]: osdmap e29: 4 total, 4 up, 4 in 2024-05-13T22:43:06.043 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: osdmap e30: 4 total, 4 up, 4 in 2024-05-13T22:43:06.043 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:06.043 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:06.043 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:06.043 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:06.043 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:06.044 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:06.044 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:06.044 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:43:06.044 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:06.044 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:06.044 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:05 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:06.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: osdmap e30: 4 total, 4 up, 4 in 2024-05-13T22:43:06.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:06.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:06.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:06.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:06.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:06.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:06.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:06.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:43:06.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:06.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:06.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:05 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:06.300 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.319+0000 7ff16ab85700 1 -- 172.21.15.130:0/1989201451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff1640ffab0 msgr2=0x7ff1640ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:06.300 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.319+0000 7ff16ab85700 1 --2- 172.21.15.130:0/1989201451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff1640ffab0 0x7ff1640ffed0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff158009910 tx=0x7ff158009c20 comp rx=0 tx=0).stop 2024-05-13T22:43:06.300 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.319+0000 7ff16ab85700 1 -- 172.21.15.130:0/1989201451 shutdown_connections 2024-05-13T22:43:06.300 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.319+0000 7ff16ab85700 1 --2- 172.21.15.130:0/1989201451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff164100cb0 0x7ff164101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:06.300 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.319+0000 7ff16ab85700 1 --2- 172.21.15.130:0/1989201451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff1640ffab0 0x7ff1640ffed0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:06.300 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.319+0000 7ff16ab85700 1 -- 172.21.15.130:0/1989201451 >> 172.21.15.130:0/1989201451 conn(0x7ff1640fb010 msgr2=0x7ff1640fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:43:06.300 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.319+0000 7ff16ab85700 1 -- 172.21.15.130:0/1989201451 shutdown_connections 2024-05-13T22:43:06.300 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.319+0000 7ff16ab85700 1 -- 172.21.15.130:0/1989201451 wait complete. 2024-05-13T22:43:06.300 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.319+0000 7ff16ab85700 1 Processor -- start 2024-05-13T22:43:06.300 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.319+0000 7ff16ab85700 1 -- start start 2024-05-13T22:43:06.301 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff16ab85700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1640ffab0 0x7ff164069990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:06.301 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff16ab85700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff164100cb0 0x7ff16419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:06.301 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff16ab85700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff164196f90 con 0x7ff1640ffab0 2024-05-13T22:43:06.301 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff16ab85700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16419da60 con 0x7ff164100cb0 2024-05-13T22:43:06.301 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff163fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff164100cb0 0x7ff16419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:06.301 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff163fff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff164100cb0 0x7ff16419d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:38508/0 (socket says 172.21.15.130:38508) 2024-05-13T22:43:06.301 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff163fff700 1 -- 172.21.15.130:0/1533389090 learned_addr learned my addr 172.21.15.130:0/1533389090 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:43:06.301 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff168921700 1 --2- 172.21.15.130:0/1533389090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1640ffab0 0x7ff164069990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:06.301 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff163fff700 1 -- 172.21.15.130:0/1533389090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1640ffab0 msgr2=0x7ff164069990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:06.301 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff163fff700 1 --2- 172.21.15.130:0/1533389090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1640ffab0 0x7ff164069990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:06.302 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff163fff700 1 -- 172.21.15.130:0/1533389090 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1580095f0 con 0x7ff164100cb0 2024-05-13T22:43:06.302 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff163fff700 1 --2- 172.21.15.130:0/1533389090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff164100cb0 0x7ff16419d520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff15000b4b0 tx=0x7ff15000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:43:06.302 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.320+0000 7ff161ffb700 1 -- 172.21.15.130:0/1533389090 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff150011800 con 0x7ff164100cb0 2024-05-13T22:43:06.302 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.321+0000 7ff161ffb700 1 -- 172.21.15.130:0/1533389090 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff150011e10 con 0x7ff164100cb0 2024-05-13T22:43:06.302 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.321+0000 7ff16ab85700 1 -- 172.21.15.130:0/1533389090 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16419dd40 con 0x7ff164100cb0 2024-05-13T22:43:06.302 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.321+0000 7ff161ffb700 1 -- 172.21.15.130:0/1533389090 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff15000f2e0 con 0x7ff164100cb0 2024-05-13T22:43:06.302 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.321+0000 7ff16ab85700 1 -- 172.21.15.130:0/1533389090 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16419e310 con 0x7ff164100cb0 2024-05-13T22:43:06.303 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.322+0000 7ff16ab85700 1 -- 172.21.15.130:0/1533389090 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff164108690 con 0x7ff164100cb0 2024-05-13T22:43:06.303 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.322+0000 7ff161ffb700 1 -- 172.21.15.130:0/1533389090 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff150010380 con 0x7ff164100cb0 2024-05-13T22:43:06.303 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.322+0000 7ff161ffb700 1 --2- 172.21.15.130:0/1533389090 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff15406c4e0 0x7ff15406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:06.303 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.323+0000 7ff161ffb700 1 -- 172.21.15.130:0/1533389090 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7ff15008adf0 con 0x7ff164100cb0 2024-05-13T22:43:06.304 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.323+0000 7ff168921700 1 --2- 172.21.15.130:0/1533389090 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff15406c4e0 0x7ff15406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:06.304 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.323+0000 7ff168921700 1 --2- 172.21.15.130:0/1533389090 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff15406c4e0 0x7ff15406e9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff158006010 tx=0x7ff15800b580 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:43:06.306 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.327+0000 7ff161ffb700 1 -- 172.21.15.130:0/1533389090 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff150059700 con 0x7ff164100cb0 2024-05-13T22:43:06.502 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:06.524+0000 7ff16ab85700 1 -- 172.21.15.130:0/1533389090 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff164105600 con 0x7ff15406c4e0 2024-05-13T22:43:07.190 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:06 smithi130 ceph-mon[36591]: Detected new or changed devices on smithi130 2024-05-13T22:43:07.190 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:06 smithi130 ceph-mon[36591]: Adjusting osd_memory_target on smithi130 to 14912M 2024-05-13T22:43:07.190 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:06 smithi130 ceph-mon[36591]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:07.190 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:06 smithi130 ceph-mon[36591]: osdmap e31: 4 total, 4 up, 4 in 2024-05-13T22:43:07.190 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:06 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:43:07.190 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:06 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:43:07.190 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:06 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:07.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:06 smithi102 ceph-mon[28599]: Detected new or changed devices on smithi130 2024-05-13T22:43:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:06 smithi102 ceph-mon[28599]: Adjusting osd_memory_target on smithi130 to 14912M 2024-05-13T22:43:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:06 smithi102 ceph-mon[28599]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:06 smithi102 ceph-mon[28599]: osdmap e31: 4 total, 4 up, 4 in 2024-05-13T22:43:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:06 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:43:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:06 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:43:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:06 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:08.264 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:07 smithi130 ceph-mon[36591]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:43:08.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:07 smithi102 ceph-mon[28599]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:43:09.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:08 smithi130 ceph-mon[36591]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-05-13T22:43:09.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:08 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:43:09.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:08 smithi102 ceph-mon[28599]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-05-13T22:43:09.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:08 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:43:10.331 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:09 smithi130 ceph-mon[36591]: from='client.? 172.21.15.130:0/1371519398' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d89edd4-8877-4cb8-a0af-019f9451bd39"}]: dispatch 2024-05-13T22:43:10.331 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:09 smithi130 ceph-mon[36591]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d89edd4-8877-4cb8-a0af-019f9451bd39"}]: dispatch 2024-05-13T22:43:10.331 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:09 smithi130 ceph-mon[36591]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d89edd4-8877-4cb8-a0af-019f9451bd39"}]': finished 2024-05-13T22:43:10.331 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:09 smithi130 ceph-mon[36591]: osdmap e32: 5 total, 4 up, 5 in 2024-05-13T22:43:10.331 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:09 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:43:10.332 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:09 smithi130 ceph-mon[36591]: from='client.? 172.21.15.130:0/2977514417' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:43:10.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:09 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/1371519398' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d89edd4-8877-4cb8-a0af-019f9451bd39"}]: dispatch 2024-05-13T22:43:10.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:09 smithi102 ceph-mon[28599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d89edd4-8877-4cb8-a0af-019f9451bd39"}]: dispatch 2024-05-13T22:43:10.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:09 smithi102 ceph-mon[28599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d89edd4-8877-4cb8-a0af-019f9451bd39"}]': finished 2024-05-13T22:43:10.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:09 smithi102 ceph-mon[28599]: osdmap e32: 5 total, 4 up, 5 in 2024-05-13T22:43:10.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:09 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:43:10.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:09 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/2977514417' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:43:11.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:11 smithi130 ceph-mon[36591]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-13T22:43:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:11 smithi102 ceph-mon[28599]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-13T22:43:13.257 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:13 smithi130 ceph-mon[36591]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-13T22:43:13.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:13 smithi102 ceph-mon[28599]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-13T22:43:15.354 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:15 smithi130 ceph-mon[36591]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-13T22:43:15.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:15 smithi102 ceph-mon[28599]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-13T22:43:17.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:17 smithi130 ceph-mon[36591]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-13T22:43:17.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:17 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-13T22:43:17.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:17 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:17.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:17 smithi102 ceph-mon[28599]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-13T22:43:17.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:17 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-13T22:43:17.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:17 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:18.341 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:18 smithi130 ceph-mon[36591]: Deploying daemon osd.4 on smithi130 2024-05-13T22:43:18.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:18 smithi102 ceph-mon[28599]: Deploying daemon osd.4 on smithi130 2024-05-13T22:43:19.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:19 smithi130 ceph-mon[36591]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:19.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:19 smithi102 ceph-mon[28599]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:21.248 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:21 smithi130 ceph-mon[36591]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:21.319 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:21 smithi102 ceph-mon[28599]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:23.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:22 smithi130 ceph-mon[36591]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:23.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:22 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:23.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:22 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:23.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:22 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:43:23.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:22 smithi102 ceph-mon[28599]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:23.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:22 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:23.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:22 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:23.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:22 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:43:24.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:23 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:43:24.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:23 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:43:25.085 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:24 smithi130 ceph-mon[36591]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:25.166 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 4 on host 'smithi130' 2024-05-13T22:43:25.167 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.187+0000 7ff161ffb700 1 -- 172.21.15.130:0/1533389090 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff164105600 con 0x7ff15406c4e0 2024-05-13T22:43:25.169 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.189+0000 7ff16ab85700 1 -- 172.21.15.130:0/1533389090 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff15406c4e0 msgr2=0x7ff15406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:25.169 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.189+0000 7ff16ab85700 1 --2- 172.21.15.130:0/1533389090 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff15406c4e0 0x7ff15406e9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff158006010 tx=0x7ff15800b580 comp rx=0 tx=0).stop 2024-05-13T22:43:25.169 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.190+0000 7ff16ab85700 1 -- 172.21.15.130:0/1533389090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff164100cb0 msgr2=0x7ff16419d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:25.169 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.190+0000 7ff16ab85700 1 --2- 172.21.15.130:0/1533389090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff164100cb0 0x7ff16419d520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff15000b4b0 tx=0x7ff15000b870 comp rx=0 tx=0).stop 2024-05-13T22:43:25.169 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.190+0000 7ff16ab85700 1 -- 172.21.15.130:0/1533389090 shutdown_connections 2024-05-13T22:43:25.169 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.190+0000 7ff16ab85700 1 --2- 172.21.15.130:0/1533389090 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7ff15406c4e0 0x7ff15406e9a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:25.169 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.190+0000 7ff16ab85700 1 --2- 172.21.15.130:0/1533389090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1640ffab0 0x7ff164069990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:25.169 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.190+0000 7ff16ab85700 1 --2- 172.21.15.130:0/1533389090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff164100cb0 0x7ff16419d520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:25.169 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.190+0000 7ff16ab85700 1 -- 172.21.15.130:0/1533389090 >> 172.21.15.130:0/1533389090 conn(0x7ff1640fb010 msgr2=0x7ff164103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:43:25.169 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.190+0000 7ff16ab85700 1 -- 172.21.15.130:0/1533389090 shutdown_connections 2024-05-13T22:43:25.169 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:25.190+0000 7ff16ab85700 1 -- 172.21.15.130:0/1533389090 wait complete. 2024-05-13T22:43:25.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:24 smithi102 ceph-mon[28599]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:25.851 DEBUG:teuthology.orchestra.run.smithi130:osd.4> sudo journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.4.service 2024-05-13T22:43:25.854 INFO:tasks.cephadm:Deploying osd.5 on smithi130 with /dev/vg_nvme/lv_2... 2024-05-13T22:43:25.854 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-13T22:43:26.358 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:26 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:26.358 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:26 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:26.358 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:26 smithi102 ceph-mon[28599]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:26.358 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:26 smithi102 ceph-mon[28599]: from='osd.4 [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-13T22:43:26.358 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:26 smithi102 ceph-mon[28599]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-13T22:43:26.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:26 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:26.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:26 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:26.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:26 smithi130 ceph-mon[36591]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:26.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:26 smithi130 ceph-mon[36591]: from='osd.4 [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-13T22:43:26.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:26 smithi130 ceph-mon[36591]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-13T22:43:26.632 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi130/config 2024-05-13T22:43:27.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:27 smithi130 ceph-mon[36591]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-13T22:43:27.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:27 smithi130 ceph-mon[36591]: osdmap e33: 5 total, 4 up, 5 in 2024-05-13T22:43:27.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:27 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:43:27.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:27 smithi130 ceph-mon[36591]: from='osd.4 [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:27.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:27 smithi130 ceph-mon[36591]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:27.402 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:43:27 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[49455]: 2024-05-13T22:43:27.149+0000 7ff23a3c2700 -1 osd.4 0 waiting for initial osdmap 2024-05-13T22:43:27.402 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:43:27 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[49455]: 2024-05-13T22:43:27.159+0000 7ff2361b7700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:43:27.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:27 smithi102 ceph-mon[28599]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-13T22:43:27.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:27 smithi102 ceph-mon[28599]: osdmap e33: 5 total, 4 up, 5 in 2024-05-13T22:43:27.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:27 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:43:27.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:27 smithi102 ceph-mon[28599]: from='osd.4 [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:27.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:27 smithi102 ceph-mon[28599]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:28.337 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:28 smithi130 ceph-mon[36591]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-05-13T22:43:28.337 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:28 smithi130 ceph-mon[36591]: osdmap e34: 5 total, 4 up, 5 in 2024-05-13T22:43:28.337 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:28 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:43:28.337 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:28 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:43:28.337 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:28 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:28.337 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:28 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:28.338 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:28 smithi130 ceph-mon[36591]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:28.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:28 smithi102 ceph-mon[28599]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-05-13T22:43:28.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:28 smithi102 ceph-mon[28599]: osdmap e34: 5 total, 4 up, 5 in 2024-05-13T22:43:28.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:28 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:43:28.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:28 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:43:28.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:28 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:28.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:28 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:28.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:28 smithi102 ceph-mon[28599]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-13T22:43:29.163 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-13T22:43:29.163 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-13T22:43:29.163 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-05-13T22:43:29.163 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-05-13T22:43:29.163 INFO:teuthology.orchestra.run.smithi130.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122457 s, 856 MB/s 2024-05-13T22:43:29.163 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-05-13T22:43:29.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:29 smithi130 ceph-mon[36591]: purged_snaps scrub starts 2024-05-13T22:43:29.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:29 smithi130 ceph-mon[36591]: purged_snaps scrub ok 2024-05-13T22:43:29.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:29 smithi130 ceph-mon[36591]: osd.4 [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717] boot 2024-05-13T22:43:29.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:29 smithi130 ceph-mon[36591]: osdmap e35: 5 total, 5 up, 5 in 2024-05-13T22:43:29.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:29 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:43:29.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:29 smithi102 ceph-mon[28599]: purged_snaps scrub starts 2024-05-13T22:43:29.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:29 smithi102 ceph-mon[28599]: purged_snaps scrub ok 2024-05-13T22:43:29.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:29 smithi102 ceph-mon[28599]: osd.4 [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717] boot 2024-05-13T22:43:29.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:29 smithi102 ceph-mon[28599]: osdmap e35: 5 total, 5 up, 5 in 2024-05-13T22:43:29.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:29 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:43:29.924 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_2 2024-05-13T22:43:30.454 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi130/config 2024-05-13T22:43:30.577 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:30 smithi130 ceph-mon[36591]: osdmap e36: 5 total, 5 up, 5 in 2024-05-13T22:43:30.577 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:30 smithi130 ceph-mon[36591]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:30.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:30 smithi102 ceph-mon[28599]: osdmap e36: 5 total, 5 up, 5 in 2024-05-13T22:43:30.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:30 smithi102 ceph-mon[28599]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:32.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.629+0000 7f6ca905f700 1 -- 172.21.15.130:0/2523096899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ca40ffab0 msgr2=0x7f6ca40ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:32.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.629+0000 7f6ca905f700 1 --2- 172.21.15.130:0/2523096899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ca40ffab0 0x7f6ca40ffed0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009910 tx=0x7f6c94009c20 comp rx=0 tx=0).stop 2024-05-13T22:43:32.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.629+0000 7f6ca905f700 1 -- 172.21.15.130:0/2523096899 shutdown_connections 2024-05-13T22:43:32.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.629+0000 7f6ca905f700 1 --2- 172.21.15.130:0/2523096899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ca4100cb0 0x7f6ca4101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:32.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.629+0000 7f6ca905f700 1 --2- 172.21.15.130:0/2523096899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ca40ffab0 0x7f6ca40ffed0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:32.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.629+0000 7f6ca905f700 1 -- 172.21.15.130:0/2523096899 >> 172.21.15.130:0/2523096899 conn(0x7f6ca40fb030 msgr2=0x7f6ca40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:43:32.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.629+0000 7f6ca905f700 1 -- 172.21.15.130:0/2523096899 shutdown_connections 2024-05-13T22:43:32.609 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.629+0000 7f6ca905f700 1 -- 172.21.15.130:0/2523096899 wait complete. 2024-05-13T22:43:32.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.630+0000 7f6ca905f700 1 Processor -- start 2024-05-13T22:43:32.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.630+0000 7f6ca905f700 1 -- start start 2024-05-13T22:43:32.611 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca905f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ca40ffab0 0x7f6ca4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca905f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ca4100cb0 0x7f6ca4196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca905f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4196ea0 con 0x7f6ca4100cb0 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca905f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4196fe0 con 0x7f6ca40ffab0 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca2d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ca40ffab0 0x7f6ca4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca2d9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ca40ffab0 0x7f6ca4196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:33126/0 (socket says 172.21.15.130:33126) 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca2d9d700 1 -- 172.21.15.130:0/4266031394 learned_addr learned my addr 172.21.15.130:0/4266031394 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca259c700 1 --2- 172.21.15.130:0/4266031394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ca4100cb0 0x7f6ca4196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca2d9d700 1 -- 172.21.15.130:0/4266031394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ca4100cb0 msgr2=0x7f6ca4196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca2d9d700 1 --2- 172.21.15.130:0/4266031394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ca4100cb0 0x7f6ca4196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca2d9d700 1 -- 172.21.15.130:0/4266031394 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c940095f0 con 0x7f6ca40ffab0 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca2d9d700 1 --2- 172.21.15.130:0/4266031394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ca40ffab0 0x7f6ca4196340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400bda0 tx=0x7f6c9400be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:43:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6c8bfff700 1 -- 172.21.15.130:0/4266031394 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c9401d070 con 0x7f6ca40ffab0 2024-05-13T22:43:32.613 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6ca905f700 1 -- 172.21.15.130:0/4266031394 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca419ba30 con 0x7f6ca40ffab0 2024-05-13T22:43:32.613 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6c8bfff700 1 -- 172.21.15.130:0/4266031394 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6c9400f460 con 0x7f6ca40ffab0 2024-05-13T22:43:32.613 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.631+0000 7f6c8bfff700 1 -- 172.21.15.130:0/4266031394 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c94021640 con 0x7f6ca40ffab0 2024-05-13T22:43:32.613 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.632+0000 7f6ca905f700 1 -- 172.21.15.130:0/4266031394 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca419bf20 con 0x7f6ca40ffab0 2024-05-13T22:43:32.613 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.633+0000 7f6ca905f700 1 -- 172.21.15.130:0/4266031394 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca4108610 con 0x7f6ca40ffab0 2024-05-13T22:43:32.613 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.633+0000 7f6c8bfff700 1 -- 172.21.15.130:0/4266031394 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f6c9400fa00 con 0x7f6ca40ffab0 2024-05-13T22:43:32.614 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.633+0000 7f6c8bfff700 1 --2- 172.21.15.130:0/4266031394 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6c8c06c4e0 0x7f6c8c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:32.614 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.634+0000 7f6c8bfff700 1 -- 172.21.15.130:0/4266031394 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f6c9408c630 con 0x7f6ca40ffab0 2024-05-13T22:43:32.614 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.634+0000 7f6ca259c700 1 --2- 172.21.15.130:0/4266031394 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6c8c06c4e0 0x7f6c8c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:32.614 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.634+0000 7f6ca259c700 1 --2- 172.21.15.130:0/4266031394 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6c8c06c4e0 0x7f6c8c06e9a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c98005950 tx=0x7f6c980058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:43:32.616 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.638+0000 7f6c8bfff700 1 -- 172.21.15.130:0/4266031394 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6c9405ad90 con 0x7f6ca40ffab0 2024-05-13T22:43:32.817 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:32.838+0000 7f6ca905f700 1 -- 172.21.15.130:0/4266031394 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ca4068b20 con 0x7f6c8c06c4e0 2024-05-13T22:43:33.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: Detected new or changed devices on smithi130 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: Adjusting osd_memory_target on smithi130 to 7456M 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:33.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:33.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:33.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: Detected new or changed devices on smithi130 2024-05-13T22:43:33.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:33.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:33.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:33.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:33.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: Adjusting osd_memory_target on smithi130 to 7456M 2024-05-13T22:43:33.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:33.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:33.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:33.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:33.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:43:33.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:33.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:33.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:33.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:33 smithi130 ceph-mon[36591]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:43:33.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:33 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:43:33.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:33 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:43:33.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:33 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:34.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:33 smithi102 ceph-mon[28599]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:43:34.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:33 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-13T22:43:34.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:33 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-13T22:43:34.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:33 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:34.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:34 smithi130 ceph-mon[36591]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:35.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:34 smithi102 ceph-mon[28599]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:36.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:36 smithi130 ceph-mon[36591]: from='client.? 172.21.15.130:0/1051359390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0942077-8a53-4e3b-b411-f1763c592e2c"}]: dispatch 2024-05-13T22:43:36.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:36 smithi130 ceph-mon[36591]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0942077-8a53-4e3b-b411-f1763c592e2c"}]: dispatch 2024-05-13T22:43:36.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:36 smithi130 ceph-mon[36591]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0942077-8a53-4e3b-b411-f1763c592e2c"}]': finished 2024-05-13T22:43:36.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:36 smithi130 ceph-mon[36591]: osdmap e37: 6 total, 5 up, 6 in 2024-05-13T22:43:36.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:36 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:43:36.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:36 smithi130 ceph-mon[36591]: from='client.? 172.21.15.130:0/3116827532' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:43:36.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:36 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/1051359390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0942077-8a53-4e3b-b411-f1763c592e2c"}]: dispatch 2024-05-13T22:43:36.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:36 smithi102 ceph-mon[28599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0942077-8a53-4e3b-b411-f1763c592e2c"}]: dispatch 2024-05-13T22:43:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:36 smithi102 ceph-mon[28599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0942077-8a53-4e3b-b411-f1763c592e2c"}]': finished 2024-05-13T22:43:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:36 smithi102 ceph-mon[28599]: osdmap e37: 6 total, 5 up, 6 in 2024-05-13T22:43:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:36 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:43:36.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:36 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/3116827532' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-13T22:43:37.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:37 smithi130 ceph-mon[36591]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:37.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:37 smithi102 ceph-mon[28599]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:38.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:38 smithi130 ceph-mon[36591]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:38.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:38 smithi102 ceph-mon[28599]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:43:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:43:41.133 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:41 smithi130 ceph-mon[36591]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:41.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:41 smithi102 ceph-mon[28599]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:43.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:43 smithi130 ceph-mon[36591]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:43.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:43 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-13T22:43:43.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:43 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:43.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:43 smithi102 ceph-mon[28599]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:43.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:43 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-13T22:43:43.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:43 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:44.233 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:44 smithi130 ceph-mon[36591]: Deploying daemon osd.5 on smithi130 2024-05-13T22:43:44.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:44 smithi102 ceph-mon[28599]: Deploying daemon osd.5 on smithi130 2024-05-13T22:43:45.137 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:45 smithi130 ceph-mon[36591]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:45.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:45 smithi102 ceph-mon[28599]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:46.154 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:46 smithi130 ceph-mon[36591]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:46 smithi102 ceph-mon[28599]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:49.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:48 smithi130 ceph-mon[36591]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:49.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:48 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:49.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:48 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:49.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:48 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:43:49.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:48 smithi102 ceph-mon[28599]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:49.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:48 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:49.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:48 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:49.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:48 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:43:51.128 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.149+0000 7f6c8bfff700 1 -- 172.21.15.130:0/4266031394 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6ca4068b20 con 0x7f6c8c06c4e0 2024-05-13T22:43:51.129 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 5 on host 'smithi130' 2024-05-13T22:43:51.130 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.152+0000 7f6ca905f700 1 -- 172.21.15.130:0/4266031394 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6c8c06c4e0 msgr2=0x7f6c8c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:51.130 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.152+0000 7f6ca905f700 1 --2- 172.21.15.130:0/4266031394 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6c8c06c4e0 0x7f6c8c06e9a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c98005950 tx=0x7f6c980058e0 comp rx=0 tx=0).stop 2024-05-13T22:43:51.131 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.152+0000 7f6ca905f700 1 -- 172.21.15.130:0/4266031394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ca40ffab0 msgr2=0x7f6ca4196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:51.131 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.152+0000 7f6ca905f700 1 --2- 172.21.15.130:0/4266031394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ca40ffab0 0x7f6ca4196340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400bda0 tx=0x7f6c9400be80 comp rx=0 tx=0).stop 2024-05-13T22:43:51.131 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.152+0000 7f6ca905f700 1 -- 172.21.15.130:0/4266031394 shutdown_connections 2024-05-13T22:43:51.131 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.152+0000 7f6ca905f700 1 --2- 172.21.15.130:0/4266031394 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6c8c06c4e0 0x7f6c8c06e9a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:51.131 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.152+0000 7f6ca905f700 1 --2- 172.21.15.130:0/4266031394 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ca40ffab0 0x7f6ca4196340 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:51.131 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.152+0000 7f6ca905f700 1 --2- 172.21.15.130:0/4266031394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ca4100cb0 0x7f6ca4196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:51.131 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.152+0000 7f6ca905f700 1 -- 172.21.15.130:0/4266031394 >> 172.21.15.130:0/4266031394 conn(0x7f6ca40fb030 msgr2=0x7f6ca4068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:43:51.131 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.152+0000 7f6ca905f700 1 -- 172.21.15.130:0/4266031394 shutdown_connections 2024-05-13T22:43:51.131 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:43:51.152+0000 7f6ca905f700 1 -- 172.21.15.130:0/4266031394 wait complete. 2024-05-13T22:43:51.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:51 smithi130 ceph-mon[36591]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:51.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:51 smithi102 ceph-mon[28599]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:51.973 DEBUG:teuthology.orchestra.run.smithi130:osd.5> sudo journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.5.service 2024-05-13T22:43:51.975 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-05-13T22:43:51.976 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-13T22:43:52.201 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:43:52.323 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:52 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:52.324 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:52 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:52.324 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:52 smithi102 ceph-mon[28599]: from='osd.5 [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-13T22:43:52.324 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:52 smithi102 ceph-mon[28599]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-13T22:43:52.324 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:52 smithi102 ceph-mon[28599]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:52.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:52 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:52.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:52 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:52.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:52 smithi130 ceph-mon[36591]: from='osd.5 [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-13T22:43:52.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:52 smithi130 ceph-mon[36591]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-13T22:43:52.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:52 smithi130 ceph-mon[36591]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:53.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:53 smithi130 ceph-mon[36591]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-13T22:43:53.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:53 smithi130 ceph-mon[36591]: osdmap e38: 6 total, 5 up, 6 in 2024-05-13T22:43:53.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:53 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:43:53.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:53 smithi130 ceph-mon[36591]: from='osd.5 [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:53.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:53 smithi130 ceph-mon[36591]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:53.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:53 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:53.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:53 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:53.403 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:43:53 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[56354]: 2024-05-13T22:43:53.156+0000 7f2a05542700 -1 osd.5 0 waiting for initial osdmap 2024-05-13T22:43:53.403 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:43:53 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[56354]: 2024-05-13T22:43:53.166+0000 7f29feb32700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:43:53.404 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:53 smithi102 ceph-mon[28599]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-13T22:43:53.404 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:53 smithi102 ceph-mon[28599]: osdmap e38: 6 total, 5 up, 6 in 2024-05-13T22:43:53.404 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:53 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:43:53.404 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:53 smithi102 ceph-mon[28599]: from='osd.5 [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:53.404 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:53 smithi102 ceph-mon[28599]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:43:53.404 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:53 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:53.404 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:53 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:53.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.913+0000 7f1d012a8700 1 -- 172.21.15.102:0/1790756648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cfc0ffab0 msgr2=0x7f1cfc0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:53.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.913+0000 7f1d012a8700 1 --2- 172.21.15.102:0/1790756648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cfc0ffab0 0x7f1cfc0ffed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009910 tx=0x7f1cec009c20 comp rx=0 tx=0).stop 2024-05-13T22:43:53.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.913+0000 7f1d012a8700 1 -- 172.21.15.102:0/1790756648 shutdown_connections 2024-05-13T22:43:53.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.913+0000 7f1d012a8700 1 --2- 172.21.15.102:0/1790756648 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cfc100cb0 0x7f1cfc101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:53.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.913+0000 7f1d012a8700 1 --2- 172.21.15.102:0/1790756648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cfc0ffab0 0x7f1cfc0ffed0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:53.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.913+0000 7f1d012a8700 1 -- 172.21.15.102:0/1790756648 >> 172.21.15.102:0/1790756648 conn(0x7f1cfc0fb030 msgr2=0x7f1cfc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:43:53.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.913+0000 7f1d012a8700 1 -- 172.21.15.102:0/1790756648 shutdown_connections 2024-05-13T22:43:53.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.913+0000 7f1d012a8700 1 -- 172.21.15.102:0/1790756648 wait complete. 2024-05-13T22:43:53.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.914+0000 7f1d012a8700 1 Processor -- start 2024-05-13T22:43:53.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.914+0000 7f1d012a8700 1 -- start start 2024-05-13T22:43:53.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.914+0000 7f1d012a8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cfc0ffab0 0x7f1cfc068f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.914+0000 7f1d012a8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cfc100cb0 0x7f1cfc19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.914+0000 7f1d012a8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc19b690 con 0x7f1cfc0ffab0 2024-05-13T22:43:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.914+0000 7f1d012a8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc19b800 con 0x7f1cfc100cb0 2024-05-13T22:43:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.914+0000 7f1cfaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cfc0ffab0 0x7f1cfc068f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.914+0000 7f1cfaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cfc0ffab0 0x7f1cfc068f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40056/0 (socket says 172.21.15.102:40056) 2024-05-13T22:43:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.914+0000 7f1cfaffd700 1 -- 172.21.15.102:0/872873063 learned_addr learned my addr 172.21.15.102:0/872873063 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:43:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.914+0000 7f1cfa7fc700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cfc100cb0 0x7f1cfc19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.915+0000 7f1cfaffd700 1 -- 172.21.15.102:0/872873063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cfc100cb0 msgr2=0x7f1cfc19b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.915+0000 7f1cfaffd700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cfc100cb0 0x7f1cfc19b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.915+0000 7f1cfaffd700 1 -- 172.21.15.102:0/872873063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec0095f0 con 0x7f1cfc0ffab0 2024-05-13T22:43:53.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.915+0000 7f1cfa7fc700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cfc100cb0 0x7f1cfc19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:43:53.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.915+0000 7f1cfaffd700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cfc0ffab0 0x7f1cfc068f90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00bda0 tx=0x7f1cec00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:43:53.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.915+0000 7f1cf3fff700 1 -- 172.21.15.102:0/872873063 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cec01d070 con 0x7f1cfc0ffab0 2024-05-13T22:43:53.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.915+0000 7f1d012a8700 1 -- 172.21.15.102:0/872873063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc19ba80 con 0x7f1cfc0ffab0 2024-05-13T22:43:53.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.915+0000 7f1cf3fff700 1 -- 172.21.15.102:0/872873063 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1cec00f460 con 0x7f1cfc0ffab0 2024-05-13T22:43:53.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.915+0000 7f1cf3fff700 1 -- 172.21.15.102:0/872873063 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cec021640 con 0x7f1cfc0ffab0 2024-05-13T22:43:53.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.915+0000 7f1d012a8700 1 -- 172.21.15.102:0/872873063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc19bf70 con 0x7f1cfc0ffab0 2024-05-13T22:43:53.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.916+0000 7f1d012a8700 1 -- 172.21.15.102:0/872873063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cfc108630 con 0x7f1cfc0ffab0 2024-05-13T22:43:53.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.916+0000 7f1cf3fff700 1 -- 172.21.15.102:0/872873063 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f1cec00fa00 con 0x7f1cfc0ffab0 2024-05-13T22:43:53.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.917+0000 7f1cf3fff700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1ce806c4e0 0x7f1ce806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:53.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.917+0000 7f1cf3fff700 1 -- 172.21.15.102:0/872873063 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4358+0+0 (secure 0 0 0) 0x7f1cec08c6d0 con 0x7f1cfc0ffab0 2024-05-13T22:43:53.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.917+0000 7f1cfa7fc700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1ce806c4e0 0x7f1ce806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:53.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.917+0000 7f1cfa7fc700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1ce806c4e0 0x7f1ce806e9a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0694f0 tx=0x7f1ce400b2c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:43:53.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:53.922+0000 7f1cf3fff700 1 -- 172.21.15.102:0/872873063 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1cec05ad90 con 0x7f1cfc0ffab0 2024-05-13T22:43:54.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.111+0000 7f1d012a8700 1 -- 172.21.15.102:0/872873063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1cfc108820 con 0x7f1cfc0ffab0 2024-05-13T22:43:54.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.112+0000 7f1cf3fff700 1 -- 172.21.15.102:0/872873063 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f1cec05a920 con 0x7f1cfc0ffab0 2024-05-13T22:43:54.091 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:43:54.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.114+0000 7f1d012a8700 1 -- 172.21.15.102:0/872873063 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1ce806c4e0 msgr2=0x7f1ce806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:54.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.114+0000 7f1d012a8700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1ce806c4e0 0x7f1ce806e9a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0694f0 tx=0x7f1ce400b2c0 comp rx=0 tx=0).stop 2024-05-13T22:43:54.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.114+0000 7f1d012a8700 1 -- 172.21.15.102:0/872873063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cfc0ffab0 msgr2=0x7f1cfc068f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:54.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.114+0000 7f1d012a8700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cfc0ffab0 0x7f1cfc068f90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00bda0 tx=0x7f1cec00be80 comp rx=0 tx=0).stop 2024-05-13T22:43:54.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.114+0000 7f1d012a8700 1 -- 172.21.15.102:0/872873063 shutdown_connections 2024-05-13T22:43:54.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.114+0000 7f1d012a8700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1ce806c4e0 0x7f1ce806e9a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:54.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.114+0000 7f1d012a8700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cfc0ffab0 0x7f1cfc068f90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:54.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.114+0000 7f1d012a8700 1 --2- 172.21.15.102:0/872873063 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cfc100cb0 0x7f1cfc19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:54.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.114+0000 7f1d012a8700 1 -- 172.21.15.102:0/872873063 >> 172.21.15.102:0/872873063 conn(0x7f1cfc0fb030 msgr2=0x7f1cfc103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:43:54.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.114+0000 7f1d012a8700 1 -- 172.21.15.102:0/872873063 shutdown_connections 2024-05-13T22:43:54.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:54.114+0000 7f1d012a8700 1 -- 172.21.15.102:0/872873063 wait complete. 2024-05-13T22:43:54.386 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:54 smithi102 ceph-mon[28599]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-05-13T22:43:54.387 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:54 smithi102 ceph-mon[28599]: osdmap e39: 6 total, 5 up, 6 in 2024-05-13T22:43:54.387 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:54 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:43:54.387 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:54 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:43:54.387 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:54 smithi102 ceph-mon[28599]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:54.387 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:54 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:43:54.387 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:54 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/872873063' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-13T22:43:54.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:54 smithi130 ceph-mon[36591]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-05-13T22:43:54.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:54 smithi130 ceph-mon[36591]: osdmap e39: 6 total, 5 up, 6 in 2024-05-13T22:43:54.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:54 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:43:54.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:54 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:43:54.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:54 smithi130 ceph-mon[36591]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-13T22:43:54.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:54 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:43:54.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:54 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/872873063' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-13T22:43:54.799 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1715640208,"num_in_osds":6,"osd_in_since":1715640215,"num_remapped_pgs":0} 2024-05-13T22:43:55.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:55 smithi130 ceph-mon[36591]: purged_snaps scrub starts 2024-05-13T22:43:55.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:55 smithi130 ceph-mon[36591]: purged_snaps scrub ok 2024-05-13T22:43:55.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:55 smithi130 ceph-mon[36591]: osd.5 [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161] boot 2024-05-13T22:43:55.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:55 smithi130 ceph-mon[36591]: osdmap e40: 6 total, 6 up, 6 in 2024-05-13T22:43:55.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:55 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:43:55.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:55 smithi102 ceph-mon[28599]: purged_snaps scrub starts 2024-05-13T22:43:55.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:55 smithi102 ceph-mon[28599]: purged_snaps scrub ok 2024-05-13T22:43:55.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:55 smithi102 ceph-mon[28599]: osd.5 [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161] boot 2024-05-13T22:43:55.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:55 smithi102 ceph-mon[28599]: osdmap e40: 6 total, 6 up, 6 in 2024-05-13T22:43:55.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:55 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:43:55.800 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-13T22:43:56.016 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:43:56.194 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:56 smithi130 ceph-mon[36591]: osdmap e41: 6 total, 6 up, 6 in 2024-05-13T22:43:56.195 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:56 smithi130 ceph-mon[36591]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:43:56.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:56 smithi102 ceph-mon[28599]: osdmap e41: 6 total, 6 up, 6 in 2024-05-13T22:43:56.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:56 smithi102 ceph-mon[28599]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:43:57.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: Detected new or changed devices on smithi130 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: Adjusting osd_memory_target on smithi130 to 4970M 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:57.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:57.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:57 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: Detected new or changed devices on smithi130 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: Adjusting osd_memory_target on smithi130 to 4970M 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:43:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:43:57.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:43:57.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:57 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:43:57.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.678+0000 7f0414060700 1 -- 172.21.15.102:0/1523344671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f040c101dc0 msgr2=0x7f040c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:57.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.678+0000 7f0414060700 1 --2- 172.21.15.102:0/1523344671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f040c101dc0 0x7f040c102220 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0400009910 tx=0x7f0400009c20 comp rx=0 tx=0).stop 2024-05-13T22:43:57.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.678+0000 7f0414060700 1 -- 172.21.15.102:0/1523344671 shutdown_connections 2024-05-13T22:43:57.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.678+0000 7f0414060700 1 --2- 172.21.15.102:0/1523344671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f040c101dc0 0x7f040c102220 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:57.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.678+0000 7f0414060700 1 --2- 172.21.15.102:0/1523344671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f040c100bc0 0x7f040c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:57.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.678+0000 7f0414060700 1 -- 172.21.15.102:0/1523344671 >> 172.21.15.102:0/1523344671 conn(0x7f040c0fc140 msgr2=0x7f040c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:43:57.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.678+0000 7f0414060700 1 -- 172.21.15.102:0/1523344671 shutdown_connections 2024-05-13T22:43:57.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.678+0000 7f0414060700 1 -- 172.21.15.102:0/1523344671 wait complete. 2024-05-13T22:43:57.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.679+0000 7f0414060700 1 Processor -- start 2024-05-13T22:43:57.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.679+0000 7f0414060700 1 -- start start 2024-05-13T22:43:57.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.679+0000 7f0414060700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f040c100bc0 0x7f040c196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:57.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.679+0000 7f0414060700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f040c101dc0 0x7f040c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.679+0000 7f0414060700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c196ed0 con 0x7f040c101dc0 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.679+0000 7f0414060700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c197010 con 0x7f040c100bc0 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f04115fb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f040c101dc0 0x7f040c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f04115fb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f040c101dc0 0x7f040c1968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40070/0 (socket says 172.21.15.102:40070) 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f04115fb700 1 -- 172.21.15.102:0/835346804 learned_addr learned my addr 172.21.15.102:0/835346804 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f0411dfc700 1 --2- 172.21.15.102:0/835346804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f040c100bc0 0x7f040c196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f04115fb700 1 -- 172.21.15.102:0/835346804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f040c100bc0 msgr2=0x7f040c196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f04115fb700 1 --2- 172.21.15.102:0/835346804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f040c100bc0 0x7f040c196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f04115fb700 1 -- 172.21.15.102:0/835346804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04000095f0 con 0x7f040c101dc0 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f04115fb700 1 --2- 172.21.15.102:0/835346804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f040c101dc0 0x7f040c1968b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0400006010 tx=0x7f040000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f03feffd700 1 -- 172.21.15.102:0/835346804 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f040001d070 con 0x7f040c101dc0 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f03feffd700 1 -- 172.21.15.102:0/835346804 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f040000f460 con 0x7f040c101dc0 2024-05-13T22:43:57.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f03feffd700 1 -- 172.21.15.102:0/835346804 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0400021640 con 0x7f040c101dc0 2024-05-13T22:43:57.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f0414060700 1 -- 172.21.15.102:0/835346804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040c19ba60 con 0x7f040c101dc0 2024-05-13T22:43:57.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.680+0000 7f0414060700 1 -- 172.21.15.102:0/835346804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040c19bf00 con 0x7f040c101dc0 2024-05-13T22:43:57.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.681+0000 7f0414060700 1 -- 172.21.15.102:0/835346804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040c109720 con 0x7f040c101dc0 2024-05-13T22:43:57.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.682+0000 7f03feffd700 1 -- 172.21.15.102:0/835346804 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f040000fab0 con 0x7f040c101dc0 2024-05-13T22:43:57.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.682+0000 7f03feffd700 1 --2- 172.21.15.102:0/835346804 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f03f806c4e0 0x7f03f806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:43:57.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.682+0000 7f03feffd700 1 -- 172.21.15.102:0/835346804 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f040008c780 con 0x7f040c101dc0 2024-05-13T22:43:57.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.682+0000 7f0411dfc700 1 --2- 172.21.15.102:0/835346804 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f03f806c4e0 0x7f03f806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:43:57.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.682+0000 7f0411dfc700 1 --2- 172.21.15.102:0/835346804 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f03f806c4e0 0x7f03f806e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f040c101c20 tx=0x7f040800b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:43:57.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.687+0000 7f03feffd700 1 -- 172.21.15.102:0/835346804 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f040005ad30 con 0x7f040c101dc0 2024-05-13T22:43:57.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.874+0000 7f0414060700 1 -- 172.21.15.102:0/835346804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f040c1098e0 con 0x7f040c101dc0 2024-05-13T22:43:57.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.874+0000 7f03feffd700 1 -- 172.21.15.102:0/835346804 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f040005a8c0 con 0x7f040c101dc0 2024-05-13T22:43:57.853 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:43:57.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.876+0000 7f0414060700 1 -- 172.21.15.102:0/835346804 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f03f806c4e0 msgr2=0x7f03f806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:57.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.876+0000 7f0414060700 1 --2- 172.21.15.102:0/835346804 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f03f806c4e0 0x7f03f806e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f040c101c20 tx=0x7f040800b410 comp rx=0 tx=0).stop 2024-05-13T22:43:57.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.876+0000 7f0414060700 1 -- 172.21.15.102:0/835346804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f040c101dc0 msgr2=0x7f040c1968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:43:57.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.876+0000 7f0414060700 1 --2- 172.21.15.102:0/835346804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f040c101dc0 0x7f040c1968b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0400006010 tx=0x7f040000bd70 comp rx=0 tx=0).stop 2024-05-13T22:43:57.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.876+0000 7f0414060700 1 -- 172.21.15.102:0/835346804 shutdown_connections 2024-05-13T22:43:57.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.876+0000 7f0414060700 1 --2- 172.21.15.102:0/835346804 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f03f806c4e0 0x7f03f806e9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:57.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.876+0000 7f0414060700 1 --2- 172.21.15.102:0/835346804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f040c100bc0 0x7f040c196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:57.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.876+0000 7f0414060700 1 --2- 172.21.15.102:0/835346804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f040c101dc0 0x7f040c1968b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:43:57.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.876+0000 7f0414060700 1 -- 172.21.15.102:0/835346804 >> 172.21.15.102:0/835346804 conn(0x7f040c0fc140 msgr2=0x7f040c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:43:57.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.876+0000 7f0414060700 1 -- 172.21.15.102:0/835346804 shutdown_connections 2024-05-13T22:43:57.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:43:57.876+0000 7f0414060700 1 -- 172.21.15.102:0/835346804 wait complete. 2024-05-13T22:43:58.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:58 smithi102 ceph-mon[28599]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:43:58.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:43:58 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/835346804' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-13T22:43:58.500 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1715640234,"num_in_osds":6,"osd_in_since":1715640215,"num_remapped_pgs":0} 2024-05-13T22:43:58.500 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-13T22:43:58.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:58 smithi130 ceph-mon[36591]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:43:58.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:43:58 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/835346804' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-13T22:43:58.707 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:00.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.210+0000 7f71b1a12700 1 -- 172.21.15.102:0/607903796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71ac0ffbd0 msgr2=0x7f71ac100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:00.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.210+0000 7f71b1a12700 1 --2- 172.21.15.102:0/607903796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71ac0ffbd0 0x7f71ac100050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7194009910 tx=0x7f7194009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:00.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.210+0000 7f71b1a12700 1 -- 172.21.15.102:0/607903796 shutdown_connections 2024-05-13T22:44:00.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.210+0000 7f71b1a12700 1 --2- 172.21.15.102:0/607903796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71ac0ffbd0 0x7f71ac100050 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:00.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.210+0000 7f71b1a12700 1 --2- 172.21.15.102:0/607903796 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71ac105a40 0x7f71ac105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.210+0000 7f71b1a12700 1 -- 172.21.15.102:0/607903796 >> 172.21.15.102:0/607903796 conn(0x7f71ac0755a0 msgr2=0x7f71ac0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.210+0000 7f71b1a12700 1 -- 172.21.15.102:0/607903796 shutdown_connections 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.210+0000 7f71b1a12700 1 -- 172.21.15.102:0/607903796 wait complete. 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.211+0000 7f71b1a12700 1 Processor -- start 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.211+0000 7f71b1a12700 1 -- start start 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.211+0000 7f71b1a12700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71ac0ffbd0 0x7f71ac19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.211+0000 7f71b1a12700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71ac105a40 0x7f71ac19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.211+0000 7f71b1a12700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac19d140 con 0x7f71ac105a40 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.211+0000 7f71b1a12700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac1965f0 con 0x7f71ac0ffbd0 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.211+0000 7f71aa7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71ac105a40 0x7f71ac19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.211+0000 7f71aa7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71ac105a40 0x7f71ac19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49890/0 (socket says 172.21.15.102:49890) 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.211+0000 7f71aa7fc700 1 -- 172.21.15.102:0/1803276064 learned_addr learned my addr 172.21.15.102:0/1803276064 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:00.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.212+0000 7f71aaffd700 1 --2- 172.21.15.102:0/1803276064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71ac0ffbd0 0x7f71ac19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:00.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.212+0000 7f71aa7fc700 1 -- 172.21.15.102:0/1803276064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71ac0ffbd0 msgr2=0x7f71ac19c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:00.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.212+0000 7f71aa7fc700 1 --2- 172.21.15.102:0/1803276064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71ac0ffbd0 0x7f71ac19c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:00.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.212+0000 7f71aa7fc700 1 -- 172.21.15.102:0/1803276064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71940095f0 con 0x7f71ac105a40 2024-05-13T22:44:00.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.212+0000 7f71aa7fc700 1 --2- 172.21.15.102:0/1803276064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71ac105a40 0x7f71ac19cab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7194006010 tx=0x7f719400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:00.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.212+0000 7f71b0a10700 1 -- 172.21.15.102:0/1803276064 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f719401d070 con 0x7f71ac105a40 2024-05-13T22:44:00.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.212+0000 7f71b0a10700 1 -- 172.21.15.102:0/1803276064 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f719400f460 con 0x7f71ac105a40 2024-05-13T22:44:00.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.212+0000 7f71b0a10700 1 -- 172.21.15.102:0/1803276064 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7194021640 con 0x7f71ac105a40 2024-05-13T22:44:00.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.212+0000 7f71b1a12700 1 -- 172.21.15.102:0/1803276064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac196870 con 0x7f71ac105a40 2024-05-13T22:44:00.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.212+0000 7f71b1a12700 1 -- 172.21.15.102:0/1803276064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac196d60 con 0x7f71ac105a40 2024-05-13T22:44:00.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.213+0000 7f71b1a12700 1 -- 172.21.15.102:0/1803276064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac109070 con 0x7f71ac105a40 2024-05-13T22:44:00.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.214+0000 7f71b0a10700 1 -- 172.21.15.102:0/1803276064 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f719400faf0 con 0x7f71ac105a40 2024-05-13T22:44:00.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.214+0000 7f71b0a10700 1 --2- 172.21.15.102:0/1803276064 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f719806c4e0 0x7f719806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:00.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.214+0000 7f71b0a10700 1 -- 172.21.15.102:0/1803276064 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f719408c780 con 0x7f71ac105a40 2024-05-13T22:44:00.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.214+0000 7f71aaffd700 1 --2- 172.21.15.102:0/1803276064 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f719806c4e0 0x7f719806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:00.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.214+0000 7f71aaffd700 1 --2- 172.21.15.102:0/1803276064 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f719806c4e0 0x7f719806e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f71a4006fd0 tx=0x7f71a4008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:00.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.218+0000 7f71b0a10700 1 -- 172.21.15.102:0/1803276064 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f719405adb0 con 0x7f71ac105a40 2024-05-13T22:44:00.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.408+0000 7f71b1a12700 1 -- 172.21.15.102:0/1803276064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71ac02cca0 con 0x7f71ac105a40 2024-05-13T22:44:00.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.409+0000 7f71b0a10700 1 -- 172.21.15.102:0/1803276064 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11368 (secure 0 0 0) 0x7f719405a940 con 0x7f71ac105a40 2024-05-13T22:44:00.388 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:44:00.388 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":41,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","created":"2024-05-13T22:38:14.140940+0000","modified":"2024-05-13T22:43:55.160041+0000","last_up_change":"2024-05-13T22:43:54.154797+0000","last_in_change":"2024-05-13T22:43:35.225543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-13T22:42:39.969524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d8cb2e3-52d6-45f5-94a1-3e8cc6535efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6803","nonce":2497809722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6805","nonce":2497809722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6809","nonce":2497809722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6807","nonce":2497809722}]},"public_addr":"172.21.15.102:6803/2497809722","cluster_addr":"172.21.15.102:6805/2497809722","heartbeat_back_addr":"172.21.15.102:6809/2497809722","heartbeat_front_addr":"172.21.15.102:6807/2497809722","state":["exists","up"]},{"osd":1,"uuid":"1d3eacb1-362c-4b6a-a959-7dea19c6521e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6811","nonce":2169124323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6813","nonce":2169124323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6817","nonce":2169124323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6815","nonce":2169124323}]},"public_addr":"172.21.15.102:6811/2169124323","cluster_addr":"172.21.15.102:6813/2169124323","heartbeat_back_addr":"172.21.15.102:6817/2169124323","heartbeat_front_addr":"172.21.15.102:6815/2169124323","state":["exists","up"]},{"osd":2,"uuid":"205b05a9-9d68-48ed-b87b-c4c677882eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6819","nonce":3553677373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6821","nonce":3553677373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6825","nonce":3553677373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6823","nonce":3553677373}]},"public_addr":"172.21.15.102:6819/3553677373","cluster_addr":"172.21.15.102:6821/3553677373","heartbeat_back_addr":"172.21.15.102:6825/3553677373","heartbeat_front_addr":"172.21.15.102:6823/3553677373","state":["exists","up"]},{"osd":3,"uuid":"d827066c-3201-479e-b142-27721c2761fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6801","nonce":4175190705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6803","nonce":4175190705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6807","nonce":4175190705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6805","nonce":4175190705}]},"public_addr":"172.21.15.130:6801/4175190705","cluster_addr":"172.21.15.130:6803/4175190705","heartbeat_back_addr":"172.21.15.130:6807/4175190705","heartbeat_front_addr":"172.21.15.130:6805/4175190705","state":["exists","up"]},{"osd":4,"uuid":"7d89edd4-8877-4cb8-a0af-019f9451bd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6809","nonce":574281717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6811","nonce":574281717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6815","nonce":574281717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6813","nonce":574281717}]},"public_addr":"172.21.15.130:6809/574281717","cluster_addr":"172.21.15.130:6811/574281717","heartbeat_back_addr":"172.21.15.130:6815/574281717","heartbeat_front_addr":"172.21.15.130:6813/574281717","state":["exists","up"]},{"osd":5,"uuid":"c0942077-8a53-4e3b-b411-f1763c592e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6817","nonce":957542161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6819","nonce":957542161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6823","nonce":957542161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6821","nonce":957542161}]},"public_addr":"172.21.15.130:6817/957542161","cluster_addr":"172.21.15.130:6819/957542161","heartbeat_back_addr":"172.21.15.130:6823/957542161","heartbeat_front_addr":"172.21.15.130:6821/957542161","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:41:41.451627+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:09.327795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:36.547203+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:00.627031+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:26.709919+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:52.576178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:6801/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/773571736":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/457222314":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/3850189972":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/17493483":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/402887427":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6801/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/3812624194":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/2096579260":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6800/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6801/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/633075379":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6800/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/2248276518":"2024-05-14T22:39:13.681560+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-13T22:44:00.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.411+0000 7f71b1a12700 1 -- 172.21.15.102:0/1803276064 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f719806c4e0 msgr2=0x7f719806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:00.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.411+0000 7f71b1a12700 1 --2- 172.21.15.102:0/1803276064 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f719806c4e0 0x7f719806e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f71a4006fd0 tx=0x7f71a4008040 comp rx=0 tx=0).stop 2024-05-13T22:44:00.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.411+0000 7f71b1a12700 1 -- 172.21.15.102:0/1803276064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71ac105a40 msgr2=0x7f71ac19cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:00.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.411+0000 7f71b1a12700 1 --2- 172.21.15.102:0/1803276064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71ac105a40 0x7f71ac19cab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7194006010 tx=0x7f719400be20 comp rx=0 tx=0).stop 2024-05-13T22:44:00.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.411+0000 7f71b1a12700 1 -- 172.21.15.102:0/1803276064 shutdown_connections 2024-05-13T22:44:00.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.411+0000 7f71b1a12700 1 --2- 172.21.15.102:0/1803276064 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f719806c4e0 0x7f719806e9a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:00.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.411+0000 7f71b1a12700 1 --2- 172.21.15.102:0/1803276064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71ac0ffbd0 0x7f71ac19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:00.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.411+0000 7f71b1a12700 1 --2- 172.21.15.102:0/1803276064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71ac105a40 0x7f71ac19cab0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:00.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.411+0000 7f71b1a12700 1 -- 172.21.15.102:0/1803276064 >> 172.21.15.102:0/1803276064 conn(0x7f71ac0755a0 msgr2=0x7f71ac068610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:00.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.412+0000 7f71b1a12700 1 -- 172.21.15.102:0/1803276064 shutdown_connections 2024-05-13T22:44:00.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:00.412+0000 7f71b1a12700 1 -- 172.21.15.102:0/1803276064 wait complete. 2024-05-13T22:44:01.140 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-13T22:42:39.969524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-13T22:44:01.140 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-13T22:44:01.346 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:01.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:01 smithi130 ceph-mon[36591]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:01.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:01 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1803276064' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T22:44:01.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:01 smithi102 ceph-mon[28599]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:01.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:01 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1803276064' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T22:44:02.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.968+0000 7fdcca807700 1 -- 172.21.15.102:0/1083256215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc4100de0 msgr2=0x7fdcc41011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:02.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.968+0000 7fdcca807700 1 --2- 172.21.15.102:0/1083256215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc4100de0 0x7fdcc41011c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009910 tx=0x7fdcb4009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:02.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.968+0000 7fdcca807700 1 -- 172.21.15.102:0/1083256215 shutdown_connections 2024-05-13T22:44:02.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.968+0000 7fdcca807700 1 --2- 172.21.15.102:0/1083256215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdcc4101790 0x7fdcc41057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:02.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.968+0000 7fdcca807700 1 --2- 172.21.15.102:0/1083256215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc4100de0 0x7fdcc41011c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:02.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.968+0000 7fdcca807700 1 -- 172.21.15.102:0/1083256215 >> 172.21.15.102:0/1083256215 conn(0x7fdcc40fc670 msgr2=0x7fdcc40fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:02.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.968+0000 7fdcca807700 1 -- 172.21.15.102:0/1083256215 shutdown_connections 2024-05-13T22:44:02.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.968+0000 7fdcca807700 1 -- 172.21.15.102:0/1083256215 wait complete. 2024-05-13T22:44:02.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.968+0000 7fdcca807700 1 Processor -- start 2024-05-13T22:44:02.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.968+0000 7fdcca807700 1 -- start start 2024-05-13T22:44:02.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.968+0000 7fdcca807700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc4100de0 0x7fdcc419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:02.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcca807700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdcc4101790 0x7fdcc419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:02.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcca807700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc419d100 con 0x7fdcc4100de0 2024-05-13T22:44:02.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcca807700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc41965b0 con 0x7fdcc4101790 2024-05-13T22:44:02.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcc3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc4100de0 0x7fdcc419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:02.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcc3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc4100de0 0x7fdcc419c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49916/0 (socket says 172.21.15.102:49916) 2024-05-13T22:44:02.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcc3fff700 1 -- 172.21.15.102:0/2658600318 learned_addr learned my addr 172.21.15.102:0/2658600318 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:02.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcbbfff700 1 --2- 172.21.15.102:0/2658600318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdcc4101790 0x7fdcc419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:02.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcc3fff700 1 -- 172.21.15.102:0/2658600318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdcc4101790 msgr2=0x7fdcc419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:02.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcc3fff700 1 --2- 172.21.15.102:0/2658600318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdcc4101790 0x7fdcc419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:02.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcc3fff700 1 -- 172.21.15.102:0/2658600318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb40095f0 con 0x7fdcc4100de0 2024-05-13T22:44:02.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcc3fff700 1 --2- 172.21.15.102:0/2658600318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc4100de0 0x7fdcc419c530 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4006010 tx=0x7fdcb400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:02.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcc1ffb700 1 -- 172.21.15.102:0/2658600318 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcb401d070 con 0x7fdcc4100de0 2024-05-13T22:44:02.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcca807700 1 -- 172.21.15.102:0/2658600318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc4196830 con 0x7fdcc4100de0 2024-05-13T22:44:02.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcc1ffb700 1 -- 172.21.15.102:0/2658600318 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdcb4004fa0 con 0x7fdcc4100de0 2024-05-13T22:44:02.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcc1ffb700 1 -- 172.21.15.102:0/2658600318 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcb400f690 con 0x7fdcc4100de0 2024-05-13T22:44:02.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.969+0000 7fdcca807700 1 -- 172.21.15.102:0/2658600318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc4196d20 con 0x7fdcc4100de0 2024-05-13T22:44:02.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.970+0000 7fdcca807700 1 -- 172.21.15.102:0/2658600318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc4109180 con 0x7fdcc4100de0 2024-05-13T22:44:02.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.971+0000 7fdcc1ffb700 1 -- 172.21.15.102:0/2658600318 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fdcb4022470 con 0x7fdcc4100de0 2024-05-13T22:44:02.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.971+0000 7fdcc1ffb700 1 --2- 172.21.15.102:0/2658600318 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fdcb006c4e0 0x7fdcb006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:02.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.971+0000 7fdcc1ffb700 1 -- 172.21.15.102:0/2658600318 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdcb408cc10 con 0x7fdcc4100de0 2024-05-13T22:44:02.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.971+0000 7fdcbbfff700 1 --2- 172.21.15.102:0/2658600318 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fdcb006c4e0 0x7fdcb006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:02.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.972+0000 7fdcbbfff700 1 --2- 172.21.15.102:0/2658600318 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fdcb006c4e0 0x7fdcb006e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdcc41059f0 tx=0x7fdcac00a300 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:02.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:02.976+0000 7fdcc1ffb700 1 -- 172.21.15.102:0/2658600318 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdcb405b1c0 con 0x7fdcc4100de0 2024-05-13T22:44:03.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.173+0000 7fdcca807700 1 -- 172.21.15.102:0/2658600318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcc4197000 con 0x7fdcc4100de0 2024-05-13T22:44:03.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.174+0000 7fdcc1ffb700 1 -- 172.21.15.102:0/2658600318 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcb405ad50 con 0x7fdcc4100de0 2024-05-13T22:44:03.154 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-05-13T22:44:03.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.176+0000 7fdcca807700 1 -- 172.21.15.102:0/2658600318 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fdcb006c4e0 msgr2=0x7fdcb006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:03.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.176+0000 7fdcca807700 1 --2- 172.21.15.102:0/2658600318 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fdcb006c4e0 0x7fdcb006e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdcc41059f0 tx=0x7fdcac00a300 comp rx=0 tx=0).stop 2024-05-13T22:44:03.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.176+0000 7fdcca807700 1 -- 172.21.15.102:0/2658600318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc4100de0 msgr2=0x7fdcc419c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:03.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.176+0000 7fdcca807700 1 --2- 172.21.15.102:0/2658600318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc4100de0 0x7fdcc419c530 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4006010 tx=0x7fdcb400bd70 comp rx=0 tx=0).stop 2024-05-13T22:44:03.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.176+0000 7fdcca807700 1 -- 172.21.15.102:0/2658600318 shutdown_connections 2024-05-13T22:44:03.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.176+0000 7fdcca807700 1 --2- 172.21.15.102:0/2658600318 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fdcb006c4e0 0x7fdcb006e9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:03.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.176+0000 7fdcca807700 1 --2- 172.21.15.102:0/2658600318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcc4100de0 0x7fdcc419c530 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:03.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.176+0000 7fdcca807700 1 --2- 172.21.15.102:0/2658600318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdcc4101790 0x7fdcc419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:03.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.176+0000 7fdcca807700 1 -- 172.21.15.102:0/2658600318 >> 172.21.15.102:0/2658600318 conn(0x7fdcc40fc670 msgr2=0x7fdcc40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:03.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.176+0000 7fdcca807700 1 -- 172.21.15.102:0/2658600318 shutdown_connections 2024-05-13T22:44:03.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:03.176+0000 7fdcca807700 1 -- 172.21.15.102:0/2658600318 wait complete. 2024-05-13T22:44:03.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:03 smithi130 ceph-mon[36591]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:03.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:03 smithi102 ceph-mon[28599]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:03.830 INFO:tasks.cephadm:Setting up client nodes... 2024-05-13T22:44:03.830 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-13T22:44:04.042 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:04.368 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:04 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2658600318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-13T22:44:04.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:04 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2658600318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-13T22:44:05.304 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:05 smithi102 ceph-mon[28599]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:05 smithi130 ceph-mon[36591]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:05.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.564+0000 7fe443670700 1 -- 172.21.15.102:0/4130227841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43c101dc0 msgr2=0x7fe43c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:05.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.564+0000 7fe443670700 1 --2- 172.21.15.102:0/4130227841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43c101dc0 0x7fe43c102220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009910 tx=0x7fe42c009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:05.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 -- 172.21.15.102:0/4130227841 shutdown_connections 2024-05-13T22:44:05.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 --2- 172.21.15.102:0/4130227841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43c101dc0 0x7fe43c102220 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:05.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 --2- 172.21.15.102:0/4130227841 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe43c100bc0 0x7fe43c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:05.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 -- 172.21.15.102:0/4130227841 >> 172.21.15.102:0/4130227841 conn(0x7fe43c0fc140 msgr2=0x7fe43c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:05.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 -- 172.21.15.102:0/4130227841 shutdown_connections 2024-05-13T22:44:05.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 -- 172.21.15.102:0/4130227841 wait complete. 2024-05-13T22:44:05.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 Processor -- start 2024-05-13T22:44:05.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 -- start start 2024-05-13T22:44:05.547 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43c100bc0 0x7fe43c196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe43c101dc0 0x7fe43c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c196ed0 con 0x7fe43c100bc0 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.565+0000 7fe443670700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c197010 con 0x7fe43c101dc0 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe44140c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43c100bc0 0x7fe43c196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe44140c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43c100bc0 0x7fe43c196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:49932/0 (socket says 172.21.15.102:49932) 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe44140c700 1 -- 172.21.15.102:0/437274144 learned_addr learned my addr 172.21.15.102:0/437274144 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe440c0b700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe43c101dc0 0x7fe43c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe44140c700 1 -- 172.21.15.102:0/437274144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe43c101dc0 msgr2=0x7fe43c1968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe44140c700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe43c101dc0 0x7fe43c1968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe44140c700 1 -- 172.21.15.102:0/437274144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c0095f0 con 0x7fe43c100bc0 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe440c0b700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe43c101dc0 0x7fe43c1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe44140c700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43c100bc0 0x7fe43c196370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe43800c8f0 tx=0x7fe43800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe4327fc700 1 -- 172.21.15.102:0/437274144 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe438007a70 con 0x7fe43c100bc0 2024-05-13T22:44:05.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe4327fc700 1 -- 172.21.15.102:0/437274144 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe438007bd0 con 0x7fe43c100bc0 2024-05-13T22:44:05.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe443670700 1 -- 172.21.15.102:0/437274144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43c19bac0 con 0x7fe43c100bc0 2024-05-13T22:44:05.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe4327fc700 1 -- 172.21.15.102:0/437274144 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe438018630 con 0x7fe43c100bc0 2024-05-13T22:44:05.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.566+0000 7fe443670700 1 -- 172.21.15.102:0/437274144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43c19c010 con 0x7fe43c100bc0 2024-05-13T22:44:05.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.567+0000 7fe443670700 1 -- 172.21.15.102:0/437274144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43c109720 con 0x7fe43c100bc0 2024-05-13T22:44:05.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.568+0000 7fe4327fc700 1 -- 172.21.15.102:0/437274144 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe43800f450 con 0x7fe43c100bc0 2024-05-13T22:44:05.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.568+0000 7fe4327fc700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe42806c4e0 0x7fe42806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:05.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.568+0000 7fe4327fc700 1 -- 172.21.15.102:0/437274144 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe43808b4f0 con 0x7fe43c100bc0 2024-05-13T22:44:05.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.568+0000 7fe440c0b700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe42806c4e0 0x7fe42806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:05.550 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.569+0000 7fe440c0b700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe42806c4e0 0x7fe42806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0059d0 tx=0x7fe42c005920 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:05.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.573+0000 7fe4327fc700 1 -- 172.21.15.102:0/437274144 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe438059b50 con 0x7fe43c100bc0 2024-05-13T22:44:05.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.839+0000 7fe443670700 1 -- 172.21.15.102:0/437274144 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fe43c19c620 con 0x7fe43c100bc0 2024-05-13T22:44:05.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.843+0000 7fe4327fc700 1 -- 172.21.15.102:0/437274144 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fe4380596e0 con 0x7fe43c100bc0 2024-05-13T22:44:05.823 INFO:teuthology.orchestra.run.smithi102.stdout:[client.0] 2024-05-13T22:44:05.823 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQC1l0JmzjwxMhAAEHRefXjXO20kcYl6u9mUNw== 2024-05-13T22:44:05.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.845+0000 7fe443670700 1 -- 172.21.15.102:0/437274144 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe42806c4e0 msgr2=0x7fe42806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.846+0000 7fe443670700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe42806c4e0 0x7fe42806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0059d0 tx=0x7fe42c005920 comp rx=0 tx=0).stop 2024-05-13T22:44:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.846+0000 7fe443670700 1 -- 172.21.15.102:0/437274144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43c100bc0 msgr2=0x7fe43c196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.846+0000 7fe443670700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43c100bc0 0x7fe43c196370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe43800c8f0 tx=0x7fe43800ccb0 comp rx=0 tx=0).stop 2024-05-13T22:44:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.846+0000 7fe443670700 1 -- 172.21.15.102:0/437274144 shutdown_connections 2024-05-13T22:44:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.846+0000 7fe443670700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fe42806c4e0 0x7fe42806e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.846+0000 7fe443670700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe43c100bc0 0x7fe43c196370 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.846+0000 7fe443670700 1 --2- 172.21.15.102:0/437274144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe43c101dc0 0x7fe43c1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.846+0000 7fe443670700 1 -- 172.21.15.102:0/437274144 >> 172.21.15.102:0/437274144 conn(0x7fe43c0fc140 msgr2=0x7fe43c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.846+0000 7fe443670700 1 -- 172.21.15.102:0/437274144 shutdown_connections 2024-05-13T22:44:05.826 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:05.846+0000 7fe443670700 1 -- 172.21.15.102:0/437274144 wait complete. 2024-05-13T22:44:06.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:06 smithi130 ceph-mon[36591]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:06.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:06 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/437274144' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-13T22:44:06.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:06 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/437274144' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-13T22:44:06.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:06 smithi102 ceph-mon[28599]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:06.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:06 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/437274144' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-13T22:44:06.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:06 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/437274144' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-13T22:44:06.518 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:44:06.518 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-13T22:44:06.518 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-13T22:44:06.556 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-13T22:44:06.774 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi130/config 2024-05-13T22:44:08.322 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.341+0000 7f26d20fd700 1 -- 172.21.15.130:0/2476261116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26cc0ffab0 msgr2=0x7f26cc0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:08.323 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.341+0000 7f26d20fd700 1 --2- 172.21.15.130:0/2476261116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26cc0ffab0 0x7f26cc0ffed0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009910 tx=0x7f26bc009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:08.323 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.341+0000 7f26d20fd700 1 -- 172.21.15.130:0/2476261116 shutdown_connections 2024-05-13T22:44:08.323 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.341+0000 7f26d20fd700 1 --2- 172.21.15.130:0/2476261116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26cc100cb0 0x7f26cc101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:08.323 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.341+0000 7f26d20fd700 1 --2- 172.21.15.130:0/2476261116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26cc0ffab0 0x7f26cc0ffed0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:08.323 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.341+0000 7f26d20fd700 1 -- 172.21.15.130:0/2476261116 >> 172.21.15.130:0/2476261116 conn(0x7f26cc0fb030 msgr2=0x7f26cc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:08.323 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.341+0000 7f26d20fd700 1 -- 172.21.15.130:0/2476261116 shutdown_connections 2024-05-13T22:44:08.323 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.341+0000 7f26d20fd700 1 -- 172.21.15.130:0/2476261116 wait complete. 2024-05-13T22:44:08.323 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.342+0000 7f26d20fd700 1 Processor -- start 2024-05-13T22:44:08.323 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.342+0000 7f26d20fd700 1 -- start start 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.342+0000 7f26d20fd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26cc0ffab0 0x7f26cc1952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.342+0000 7f26d20fd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26cc100cb0 0x7f26cc195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.342+0000 7f26d20fd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc195e40 con 0x7f26cc100cb0 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.342+0000 7f26d20fd700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc195f80 con 0x7f26cc0ffab0 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.342+0000 7f26cb7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26cc0ffab0 0x7f26cc1952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.342+0000 7f26cb7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26cc0ffab0 0x7f26cc1952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:37352/0 (socket says 172.21.15.130:37352) 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.342+0000 7f26cb7fe700 1 -- 172.21.15.130:0/225432739 learned_addr learned my addr 172.21.15.130:0/225432739 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.343+0000 7f26c3fff700 1 --2- 172.21.15.130:0/225432739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26cc100cb0 0x7f26cc195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.343+0000 7f26cb7fe700 1 -- 172.21.15.130:0/225432739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26cc100cb0 msgr2=0x7f26cc195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.343+0000 7f26cb7fe700 1 --2- 172.21.15.130:0/225432739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26cc100cb0 0x7f26cc195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.343+0000 7f26cb7fe700 1 -- 172.21.15.130:0/225432739 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc0095f0 con 0x7f26cc0ffab0 2024-05-13T22:44:08.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.343+0000 7f26cb7fe700 1 --2- 172.21.15.130:0/225432739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26cc0ffab0 0x7f26cc1952e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00bda0 tx=0x7f26bc00bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:08.325 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.343+0000 7f26c97fa700 1 -- 172.21.15.130:0/225432739 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26bc01f070 con 0x7f26cc0ffab0 2024-05-13T22:44:08.325 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.343+0000 7f26c97fa700 1 -- 172.21.15.130:0/225432739 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f26bc019610 con 0x7f26cc0ffab0 2024-05-13T22:44:08.325 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.343+0000 7f26c97fa700 1 -- 172.21.15.130:0/225432739 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26bc0246e0 con 0x7f26cc0ffab0 2024-05-13T22:44:08.325 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.343+0000 7f26d20fd700 1 -- 172.21.15.130:0/225432739 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc068410 con 0x7f26cc0ffab0 2024-05-13T22:44:08.325 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.343+0000 7f26d20fd700 1 -- 172.21.15.130:0/225432739 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc0688c0 con 0x7f26cc0ffab0 2024-05-13T22:44:08.326 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.344+0000 7f26d20fd700 1 -- 172.21.15.130:0/225432739 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26cc108690 con 0x7f26cc0ffab0 2024-05-13T22:44:08.326 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.345+0000 7f26c97fa700 1 -- 172.21.15.130:0/225432739 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f26bc019780 con 0x7f26cc0ffab0 2024-05-13T22:44:08.326 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.345+0000 7f26c97fa700 1 --2- 172.21.15.130:0/225432739 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f26b806c4e0 0x7f26b806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:08.326 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.345+0000 7f26c97fa700 1 -- 172.21.15.130:0/225432739 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f26bc08edc0 con 0x7f26cc0ffab0 2024-05-13T22:44:08.326 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.346+0000 7f26c3fff700 1 --2- 172.21.15.130:0/225432739 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f26b806c4e0 0x7f26b806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:08.326 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.346+0000 7f26c3fff700 1 --2- 172.21.15.130:0/225432739 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f26b806c4e0 0x7f26b806e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009a10 tx=0x7f26b4009190 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:08.328 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.350+0000 7f26c97fa700 1 -- 172.21.15.130:0/225432739 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f26bc05d510 con 0x7f26cc0ffab0 2024-05-13T22:44:08.590 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.612+0000 7f26d20fd700 1 -- 172.21.15.130:0/225432739 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f26cc0692c0 con 0x7f26cc0ffab0 2024-05-13T22:44:08.595 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.617+0000 7f26c97fa700 1 -- 172.21.15.130:0/225432739 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f26bc0290f0 con 0x7f26cc0ffab0 2024-05-13T22:44:08.596 INFO:teuthology.orchestra.run.smithi130.stdout:[client.1] 2024-05-13T22:44:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQC4l0JmnyqbJBAAFNvjLksYUWf+Cpbh8M9Uyg== 2024-05-13T22:44:08.598 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.619+0000 7f26d20fd700 1 -- 172.21.15.130:0/225432739 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f26b806c4e0 msgr2=0x7f26b806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:08.598 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.619+0000 7f26d20fd700 1 --2- 172.21.15.130:0/225432739 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f26b806c4e0 0x7f26b806e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009a10 tx=0x7f26b4009190 comp rx=0 tx=0).stop 2024-05-13T22:44:08.598 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.619+0000 7f26d20fd700 1 -- 172.21.15.130:0/225432739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26cc0ffab0 msgr2=0x7f26cc1952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:08.598 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.619+0000 7f26d20fd700 1 --2- 172.21.15.130:0/225432739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26cc0ffab0 0x7f26cc1952e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00bda0 tx=0x7f26bc00bdd0 comp rx=0 tx=0).stop 2024-05-13T22:44:08.598 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.619+0000 7f26d20fd700 1 -- 172.21.15.130:0/225432739 shutdown_connections 2024-05-13T22:44:08.598 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.619+0000 7f26d20fd700 1 --2- 172.21.15.130:0/225432739 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f26b806c4e0 0x7f26b806e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:08.598 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.619+0000 7f26d20fd700 1 --2- 172.21.15.130:0/225432739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26cc0ffab0 0x7f26cc1952e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:08.598 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.619+0000 7f26d20fd700 1 --2- 172.21.15.130:0/225432739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26cc100cb0 0x7f26cc195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:08.599 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.619+0000 7f26d20fd700 1 -- 172.21.15.130:0/225432739 >> 172.21.15.130:0/225432739 conn(0x7f26cc0fb030 msgr2=0x7f26cc103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:08.599 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.619+0000 7f26d20fd700 1 -- 172.21.15.130:0/225432739 shutdown_connections 2024-05-13T22:44:08.599 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:44:08.619+0000 7f26d20fd700 1 -- 172.21.15.130:0/225432739 wait complete. 2024-05-13T22:44:08.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:08 smithi130 ceph-mon[36591]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:08.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:08 smithi130 ceph-mon[36591]: from='client.? 172.21.15.130:0/225432739' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-13T22:44:08.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:08 smithi130 ceph-mon[36591]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-13T22:44:08.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:08 smithi130 ceph-mon[36591]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-13T22:44:08.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:08 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:44:09.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:08 smithi102 ceph-mon[28599]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:09.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:08 smithi102 ceph-mon[28599]: from='client.? 172.21.15.130:0/225432739' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-13T22:44:09.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:08 smithi102 ceph-mon[28599]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-13T22:44:09.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:08 smithi102 ceph-mon[28599]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-13T22:44:09.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:08 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:44:09.270 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:44:09.270 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-05-13T22:44:09.270 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-13T22:44:09.308 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-13T22:44:09.308 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-13T22:44:09.308 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-13T22:44:09.523 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:11.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.236+0000 7efe97408700 1 -- 172.21.15.102:0/4023558033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe90101790 msgr2=0x7efe901057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:11.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.236+0000 7efe97408700 1 --2- 172.21.15.102:0/4023558033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe90101790 0x7efe901057e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efe84009910 tx=0x7efe84009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:11.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.236+0000 7efe97408700 1 -- 172.21.15.102:0/4023558033 shutdown_connections 2024-05-13T22:44:11.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.236+0000 7efe97408700 1 --2- 172.21.15.102:0/4023558033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe90101790 0x7efe901057e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:11.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.236+0000 7efe97408700 1 --2- 172.21.15.102:0/4023558033 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe90100de0 0x7efe901011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:11.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.236+0000 7efe97408700 1 -- 172.21.15.102:0/4023558033 >> 172.21.15.102:0/4023558033 conn(0x7efe900fc6b0 msgr2=0x7efe900fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:11.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.236+0000 7efe97408700 1 -- 172.21.15.102:0/4023558033 shutdown_connections 2024-05-13T22:44:11.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.236+0000 7efe97408700 1 -- 172.21.15.102:0/4023558033 wait complete. 2024-05-13T22:44:11.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.237+0000 7efe97408700 1 Processor -- start 2024-05-13T22:44:11.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.237+0000 7efe97408700 1 -- start start 2024-05-13T22:44:11.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.237+0000 7efe97408700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe90100de0 0x7efe9019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:11.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.237+0000 7efe97408700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe90101790 0x7efe9019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:11.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.237+0000 7efe97408700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9019d180 con 0x7efe90100de0 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.237+0000 7efe97408700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe90196630 con 0x7efe90101790 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe951a4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe90100de0 0x7efe9019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe951a4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe90100de0 0x7efe9019c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60582/0 (socket says 172.21.15.102:60582) 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe951a4700 1 -- 172.21.15.102:0/728696860 learned_addr learned my addr 172.21.15.102:0/728696860 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe949a3700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe90101790 0x7efe9019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe951a4700 1 -- 172.21.15.102:0/728696860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe90101790 msgr2=0x7efe9019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe951a4700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe90101790 0x7efe9019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe951a4700 1 -- 172.21.15.102:0/728696860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe840095f0 con 0x7efe90100de0 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe949a3700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe90101790 0x7efe9019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe951a4700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe90100de0 0x7efe9019c5b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efe8800ba50 tx=0x7efe8800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe8e7fc700 1 -- 172.21.15.102:0/728696860 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe8800ca00 con 0x7efe90100de0 2024-05-13T22:44:11.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe8e7fc700 1 -- 172.21.15.102:0/728696860 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efe8800cb60 con 0x7efe90100de0 2024-05-13T22:44:11.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe97408700 1 -- 172.21.15.102:0/728696860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe90196910 con 0x7efe90100de0 2024-05-13T22:44:11.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe8e7fc700 1 -- 172.21.15.102:0/728696860 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe88007580 con 0x7efe90100de0 2024-05-13T22:44:11.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.238+0000 7efe97408700 1 -- 172.21.15.102:0/728696860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe90196e60 con 0x7efe90100de0 2024-05-13T22:44:11.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.239+0000 7efe97408700 1 -- 172.21.15.102:0/728696860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe90109130 con 0x7efe90100de0 2024-05-13T22:44:11.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.240+0000 7efe8e7fc700 1 -- 172.21.15.102:0/728696860 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7efe8800ccd0 con 0x7efe90100de0 2024-05-13T22:44:11.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.240+0000 7efe8e7fc700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7efe7806c4e0 0x7efe7806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:11.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.240+0000 7efe949a3700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7efe7806c4e0 0x7efe7806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:11.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.240+0000 7efe8e7fc700 1 -- 172.21.15.102:0/728696860 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efe88006e40 con 0x7efe90100de0 2024-05-13T22:44:11.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.241+0000 7efe949a3700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7efe7806c4e0 0x7efe7806e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe900fcfd0 tx=0x7efe8401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:11.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.245+0000 7efe8e7fc700 1 -- 172.21.15.102:0/728696860 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efe88059050 con 0x7efe90100de0 2024-05-13T22:44:11.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:11 smithi130 ceph-mon[36591]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:11 smithi102 ceph-mon[28599]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:11.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.488+0000 7efe97408700 1 -- 172.21.15.102:0/728696860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efe9004ea90 con 0x7efe90100de0 2024-05-13T22:44:11.470 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.491+0000 7efe8e7fc700 1 -- 172.21.15.102:0/728696860 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173046 (secure 0 0 0) 0x7efe88058e70 con 0x7efe90100de0 2024-05-13T22:44:11.473 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:44:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.494+0000 7efe97408700 1 -- 172.21.15.102:0/728696860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7efe7806c4e0 msgr2=0x7efe7806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.494+0000 7efe97408700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7efe7806c4e0 0x7efe7806e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe900fcfd0 tx=0x7efe8401a040 comp rx=0 tx=0).stop 2024-05-13T22:44:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.494+0000 7efe97408700 1 -- 172.21.15.102:0/728696860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe90100de0 msgr2=0x7efe9019c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.494+0000 7efe97408700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe90100de0 0x7efe9019c5b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efe8800ba50 tx=0x7efe8800be10 comp rx=0 tx=0).stop 2024-05-13T22:44:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.494+0000 7efe97408700 1 -- 172.21.15.102:0/728696860 shutdown_connections 2024-05-13T22:44:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.494+0000 7efe97408700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7efe7806c4e0 0x7efe7806e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.494+0000 7efe97408700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efe90100de0 0x7efe9019c5b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:11.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.494+0000 7efe97408700 1 --2- 172.21.15.102:0/728696860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efe90101790 0x7efe9019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:11.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.494+0000 7efe97408700 1 -- 172.21.15.102:0/728696860 >> 172.21.15.102:0/728696860 conn(0x7efe900fc6b0 msgr2=0x7efe900fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:11.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.494+0000 7efe97408700 1 -- 172.21.15.102:0/728696860 shutdown_connections 2024-05-13T22:44:11.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:11.494+0000 7efe97408700 1 -- 172.21.15.102:0/728696860 wait complete. 2024-05-13T22:44:12.134 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":19,"active_gid":14215,"active_name":"smithi102.xcijrj","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2571859579},{"type":"v1","addr":"172.21.15.102:6801","nonce":2571859579}]},"active_addr":"172.21.15.102:6801/2571859579","active_change":"2024-05-13T22:40:23.651534+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14240,"name":"smithi130.uoqoxl","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.102:8443/","prometheus":"http://172.21.15.102:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":1238275992}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":771792641}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":2675775894}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":233302973}]}]} 2024-05-13T22:44:12.138 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-13T22:44:12.139 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-13T22:44:12.139 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-13T22:44:12.346 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:12.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:12 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/728696860' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-13T22:44:12.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:12 smithi130 ceph-mon[36591]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:12.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:12 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/728696860' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-13T22:44:12.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:12 smithi102 ceph-mon[28599]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:13.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.868+0000 7f17b0f14700 1 -- 172.21.15.102:0/704113028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17ac0ffbd0 msgr2=0x7f17ac100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:13.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.868+0000 7f17b0f14700 1 --2- 172.21.15.102:0/704113028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17ac0ffbd0 0x7f17ac100050 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009910 tx=0x7f17a0009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:13.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.869+0000 7f17b0f14700 1 -- 172.21.15.102:0/704113028 shutdown_connections 2024-05-13T22:44:13.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.869+0000 7f17b0f14700 1 --2- 172.21.15.102:0/704113028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17ac0ffbd0 0x7f17ac100050 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:13.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.869+0000 7f17b0f14700 1 --2- 172.21.15.102:0/704113028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17ac105a40 0x7f17ac105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:13.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.869+0000 7f17b0f14700 1 -- 172.21.15.102:0/704113028 >> 172.21.15.102:0/704113028 conn(0x7f17ac0755a0 msgr2=0x7f17ac0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:13.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.869+0000 7f17b0f14700 1 -- 172.21.15.102:0/704113028 shutdown_connections 2024-05-13T22:44:13.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.869+0000 7f17b0f14700 1 -- 172.21.15.102:0/704113028 wait complete. 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17b0f14700 1 Processor -- start 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17b0f14700 1 -- start start 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17b0f14700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17ac0ffbd0 0x7f17ac19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17b0f14700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17ac105a40 0x7f17ac19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17b0f14700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac19d0d0 con 0x7f17ac105a40 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17b0f14700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac196580 con 0x7f17ac0ffbd0 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17a9d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17ac105a40 0x7f17ac19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17a9d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17ac105a40 0x7f17ac19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60608/0 (socket says 172.21.15.102:60608) 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17a9d9b700 1 -- 172.21.15.102:0/1348738128 learned_addr learned my addr 172.21.15.102:0/1348738128 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17aa59c700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17ac0ffbd0 0x7f17ac19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17a9d9b700 1 -- 172.21.15.102:0/1348738128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17ac0ffbd0 msgr2=0x7f17ac19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:13.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17a9d9b700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17ac0ffbd0 0x7f17ac19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:13.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17a9d9b700 1 -- 172.21.15.102:0/1348738128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a00095f0 con 0x7f17ac105a40 2024-05-13T22:44:13.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17aa59c700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17ac0ffbd0 0x7f17ac19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:44:13.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.870+0000 7f17a9d9b700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17ac105a40 0x7f17ac19ca40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f17a0006010 tx=0x7f17a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:13.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.871+0000 7f179b7fe700 1 -- 172.21.15.102:0/1348738128 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a001d070 con 0x7f17ac105a40 2024-05-13T22:44:13.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.871+0000 7f179b7fe700 1 -- 172.21.15.102:0/1348738128 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f17a0004f50 con 0x7f17ac105a40 2024-05-13T22:44:13.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.871+0000 7f17b0f14700 1 -- 172.21.15.102:0/1348738128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ac196800 con 0x7f17ac105a40 2024-05-13T22:44:13.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.871+0000 7f179b7fe700 1 -- 172.21.15.102:0/1348738128 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a0021640 con 0x7f17ac105a40 2024-05-13T22:44:13.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.871+0000 7f17b0f14700 1 -- 172.21.15.102:0/1348738128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ac196cf0 con 0x7f17ac105a40 2024-05-13T22:44:13.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.872+0000 7f17b0f14700 1 -- 172.21.15.102:0/1348738128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ac108fe0 con 0x7f17ac105a40 2024-05-13T22:44:13.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.872+0000 7f179b7fe700 1 -- 172.21.15.102:0/1348738128 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f17a000f460 con 0x7f17ac105a40 2024-05-13T22:44:13.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.873+0000 7f179b7fe700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f179406c4e0 0x7f179406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:13.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.873+0000 7f179b7fe700 1 -- 172.21.15.102:0/1348738128 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f17a008c740 con 0x7f17ac105a40 2024-05-13T22:44:13.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.873+0000 7f17aa59c700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f179406c4e0 0x7f179406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:13.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.873+0000 7f17aa59c700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f179406c4e0 0x7f179406e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f179c005fd0 tx=0x7f179c005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:13.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:13.878+0000 7f179b7fe700 1 -- 172.21.15.102:0/1348738128 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f17a005acf0 con 0x7f17ac105a40 2024-05-13T22:44:14.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.067+0000 7f17b0f14700 1 -- 172.21.15.102:0/1348738128 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17ac02ccf0 con 0x7f17ac105a40 2024-05-13T22:44:14.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.068+0000 7f179b7fe700 1 -- 172.21.15.102:0/1348738128 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11368 (secure 0 0 0) 0x7f17a000fe60 con 0x7f17ac105a40 2024-05-13T22:44:14.047 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:44:14.047 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":41,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","created":"2024-05-13T22:38:14.140940+0000","modified":"2024-05-13T22:43:55.160041+0000","last_up_change":"2024-05-13T22:43:54.154797+0000","last_in_change":"2024-05-13T22:43:35.225543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-13T22:42:39.969524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d8cb2e3-52d6-45f5-94a1-3e8cc6535efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6803","nonce":2497809722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6805","nonce":2497809722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6809","nonce":2497809722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6807","nonce":2497809722}]},"public_addr":"172.21.15.102:6803/2497809722","cluster_addr":"172.21.15.102:6805/2497809722","heartbeat_back_addr":"172.21.15.102:6809/2497809722","heartbeat_front_addr":"172.21.15.102:6807/2497809722","state":["exists","up"]},{"osd":1,"uuid":"1d3eacb1-362c-4b6a-a959-7dea19c6521e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6811","nonce":2169124323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6813","nonce":2169124323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6817","nonce":2169124323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6815","nonce":2169124323}]},"public_addr":"172.21.15.102:6811/2169124323","cluster_addr":"172.21.15.102:6813/2169124323","heartbeat_back_addr":"172.21.15.102:6817/2169124323","heartbeat_front_addr":"172.21.15.102:6815/2169124323","state":["exists","up"]},{"osd":2,"uuid":"205b05a9-9d68-48ed-b87b-c4c677882eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6819","nonce":3553677373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6821","nonce":3553677373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6825","nonce":3553677373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6823","nonce":3553677373}]},"public_addr":"172.21.15.102:6819/3553677373","cluster_addr":"172.21.15.102:6821/3553677373","heartbeat_back_addr":"172.21.15.102:6825/3553677373","heartbeat_front_addr":"172.21.15.102:6823/3553677373","state":["exists","up"]},{"osd":3,"uuid":"d827066c-3201-479e-b142-27721c2761fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6801","nonce":4175190705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6803","nonce":4175190705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6807","nonce":4175190705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6805","nonce":4175190705}]},"public_addr":"172.21.15.130:6801/4175190705","cluster_addr":"172.21.15.130:6803/4175190705","heartbeat_back_addr":"172.21.15.130:6807/4175190705","heartbeat_front_addr":"172.21.15.130:6805/4175190705","state":["exists","up"]},{"osd":4,"uuid":"7d89edd4-8877-4cb8-a0af-019f9451bd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6809","nonce":574281717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6811","nonce":574281717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6815","nonce":574281717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6813","nonce":574281717}]},"public_addr":"172.21.15.130:6809/574281717","cluster_addr":"172.21.15.130:6811/574281717","heartbeat_back_addr":"172.21.15.130:6815/574281717","heartbeat_front_addr":"172.21.15.130:6813/574281717","state":["exists","up"]},{"osd":5,"uuid":"c0942077-8a53-4e3b-b411-f1763c592e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6817","nonce":957542161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6819","nonce":957542161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6823","nonce":957542161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6821","nonce":957542161}]},"public_addr":"172.21.15.130:6817/957542161","cluster_addr":"172.21.15.130:6819/957542161","heartbeat_back_addr":"172.21.15.130:6823/957542161","heartbeat_front_addr":"172.21.15.130:6821/957542161","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:41:41.451627+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:09.327795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:36.547203+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:00.627031+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:26.709919+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:52.576178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:6801/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/773571736":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/457222314":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/3850189972":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/17493483":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/402887427":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6801/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/3812624194":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/2096579260":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6800/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6801/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/633075379":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6800/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/2248276518":"2024-05-14T22:39:13.681560+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-13T22:44:14.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.070+0000 7f17b0f14700 1 -- 172.21.15.102:0/1348738128 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f179406c4e0 msgr2=0x7f179406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:14.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.070+0000 7f17b0f14700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f179406c4e0 0x7f179406e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f179c005fd0 tx=0x7f179c005ee0 comp rx=0 tx=0).stop 2024-05-13T22:44:14.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.070+0000 7f17b0f14700 1 -- 172.21.15.102:0/1348738128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17ac105a40 msgr2=0x7f17ac19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:14.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.070+0000 7f17b0f14700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17ac105a40 0x7f17ac19ca40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f17a0006010 tx=0x7f17a000be20 comp rx=0 tx=0).stop 2024-05-13T22:44:14.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.070+0000 7f17b0f14700 1 -- 172.21.15.102:0/1348738128 shutdown_connections 2024-05-13T22:44:14.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.070+0000 7f17b0f14700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f179406c4e0 0x7f179406e9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:14.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.070+0000 7f17b0f14700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17ac0ffbd0 0x7f17ac19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:14.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.070+0000 7f17b0f14700 1 --2- 172.21.15.102:0/1348738128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17ac105a40 0x7f17ac19ca40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:14.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.070+0000 7f17b0f14700 1 -- 172.21.15.102:0/1348738128 >> 172.21.15.102:0/1348738128 conn(0x7f17ac0755a0 msgr2=0x7f17ac068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:14.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.070+0000 7f17b0f14700 1 -- 172.21.15.102:0/1348738128 shutdown_connections 2024-05-13T22:44:14.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:14.070+0000 7f17b0f14700 1 -- 172.21.15.102:0/1348738128 wait complete. 2024-05-13T22:44:14.806 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-13T22:44:14.806 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-13T22:44:15.010 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:15.133 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:14 smithi102 ceph-mon[28599]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:15.133 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:14 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1348738128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T22:44:15.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:14 smithi130 ceph-mon[36591]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:15.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:14 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1348738128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T22:44:16.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.516+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2496664686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b04104a50 msgr2=0x7f1b04104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:16.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.516+0000 7f1b0c3a1700 1 --2- 172.21.15.102:0/2496664686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b04104a50 0x7f1b04104e30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00b0a0 tx=0x7f1afc00b3b0 comp rx=0 tx=0).stop 2024-05-13T22:44:16.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.516+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2496664686 shutdown_connections 2024-05-13T22:44:16.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.516+0000 7f1b0c3a1700 1 --2- 172.21.15.102:0/2496664686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b04100140 0x7f1b041005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:16.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.516+0000 7f1b0c3a1700 1 --2- 172.21.15.102:0/2496664686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b04104a50 0x7f1b04104e30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:16.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.516+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2496664686 >> 172.21.15.102:0/2496664686 conn(0x7f1b04075e30 msgr2=0x7f1b040fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:16.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.516+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2496664686 shutdown_connections 2024-05-13T22:44:16.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.516+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2496664686 wait complete. 2024-05-13T22:44:16.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0c3a1700 1 Processor -- start 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0c3a1700 1 -- start start 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0c3a1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b04100140 0x7f1b041974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0c3a1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b04104a50 0x7f1b04197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0c3a1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b04198030 con 0x7f1b04100140 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0c3a1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b041981a0 con 0x7f1b04104a50 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0a13d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b04100140 0x7f1b041974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0a13d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b04100140 0x7f1b041974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60620/0 (socket says 172.21.15.102:60620) 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0a13d700 1 -- 172.21.15.102:0/2456531525 learned_addr learned my addr 172.21.15.102:0/2456531525 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0a13d700 1 -- 172.21.15.102:0/2456531525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b04104a50 msgr2=0x7f1b04197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0a13d700 1 --2- 172.21.15.102:0/2456531525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b04104a50 0x7f1b04197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0a13d700 1 -- 172.21.15.102:0/2456531525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1afc009d00 con 0x7f1b04100140 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.517+0000 7f1b0a13d700 1 --2- 172.21.15.102:0/2456531525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b04100140 0x7f1b041974d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1afc000f80 tx=0x7f1afc00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.518+0000 7f1b037fe700 1 -- 172.21.15.102:0/2456531525 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1afc00e040 con 0x7f1b04100140 2024-05-13T22:44:16.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.518+0000 7f1b037fe700 1 -- 172.21.15.102:0/2456531525 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1afc003de0 con 0x7f1b04100140 2024-05-13T22:44:16.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.518+0000 7f1b037fe700 1 -- 172.21.15.102:0/2456531525 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1afc01d9a0 con 0x7f1b04100140 2024-05-13T22:44:16.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.518+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2456531525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b041a0540 con 0x7f1b04100140 2024-05-13T22:44:16.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.518+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2456531525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b041a0a30 con 0x7f1b04100140 2024-05-13T22:44:16.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.519+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2456531525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b04108020 con 0x7f1b04100140 2024-05-13T22:44:16.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.519+0000 7f1b037fe700 1 -- 172.21.15.102:0/2456531525 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f1afc004040 con 0x7f1b04100140 2024-05-13T22:44:16.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.520+0000 7f1b037fe700 1 --2- 172.21.15.102:0/2456531525 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1aec06c4e0 0x7f1aec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:16.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.520+0000 7f1b037fe700 1 -- 172.21.15.102:0/2456531525 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1afc08cd70 con 0x7f1b04100140 2024-05-13T22:44:16.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.520+0000 7f1b0993c700 1 --2- 172.21.15.102:0/2456531525 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1aec06c4e0 0x7f1aec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:16.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.520+0000 7f1b0993c700 1 --2- 172.21.15.102:0/2456531525 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1aec06c4e0 0x7f1aec06e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1af8005fd0 tx=0x7f1af8005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:16.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.524+0000 7f1b037fe700 1 -- 172.21.15.102:0/2456531525 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1afc057910 con 0x7f1b04100140 2024-05-13T22:44:16.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.712+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2456531525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b0404ea90 con 0x7f1b04100140 2024-05-13T22:44:16.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.713+0000 7f1b037fe700 1 -- 172.21.15.102:0/2456531525 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11368 (secure 0 0 0) 0x7f1afc05af30 con 0x7f1b04100140 2024-05-13T22:44:16.691 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:44:16.691 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":41,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","created":"2024-05-13T22:38:14.140940+0000","modified":"2024-05-13T22:43:55.160041+0000","last_up_change":"2024-05-13T22:43:54.154797+0000","last_in_change":"2024-05-13T22:43:35.225543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-13T22:42:39.969524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d8cb2e3-52d6-45f5-94a1-3e8cc6535efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6803","nonce":2497809722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6805","nonce":2497809722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6809","nonce":2497809722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6807","nonce":2497809722}]},"public_addr":"172.21.15.102:6803/2497809722","cluster_addr":"172.21.15.102:6805/2497809722","heartbeat_back_addr":"172.21.15.102:6809/2497809722","heartbeat_front_addr":"172.21.15.102:6807/2497809722","state":["exists","up"]},{"osd":1,"uuid":"1d3eacb1-362c-4b6a-a959-7dea19c6521e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6811","nonce":2169124323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6813","nonce":2169124323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6817","nonce":2169124323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6815","nonce":2169124323}]},"public_addr":"172.21.15.102:6811/2169124323","cluster_addr":"172.21.15.102:6813/2169124323","heartbeat_back_addr":"172.21.15.102:6817/2169124323","heartbeat_front_addr":"172.21.15.102:6815/2169124323","state":["exists","up"]},{"osd":2,"uuid":"205b05a9-9d68-48ed-b87b-c4c677882eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6819","nonce":3553677373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6821","nonce":3553677373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6825","nonce":3553677373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6823","nonce":3553677373}]},"public_addr":"172.21.15.102:6819/3553677373","cluster_addr":"172.21.15.102:6821/3553677373","heartbeat_back_addr":"172.21.15.102:6825/3553677373","heartbeat_front_addr":"172.21.15.102:6823/3553677373","state":["exists","up"]},{"osd":3,"uuid":"d827066c-3201-479e-b142-27721c2761fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6801","nonce":4175190705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6803","nonce":4175190705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6807","nonce":4175190705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6805","nonce":4175190705}]},"public_addr":"172.21.15.130:6801/4175190705","cluster_addr":"172.21.15.130:6803/4175190705","heartbeat_back_addr":"172.21.15.130:6807/4175190705","heartbeat_front_addr":"172.21.15.130:6805/4175190705","state":["exists","up"]},{"osd":4,"uuid":"7d89edd4-8877-4cb8-a0af-019f9451bd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6809","nonce":574281717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6811","nonce":574281717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6815","nonce":574281717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6813","nonce":574281717}]},"public_addr":"172.21.15.130:6809/574281717","cluster_addr":"172.21.15.130:6811/574281717","heartbeat_back_addr":"172.21.15.130:6815/574281717","heartbeat_front_addr":"172.21.15.130:6813/574281717","state":["exists","up"]},{"osd":5,"uuid":"c0942077-8a53-4e3b-b411-f1763c592e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6817","nonce":957542161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6819","nonce":957542161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6823","nonce":957542161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6821","nonce":957542161}]},"public_addr":"172.21.15.130:6817/957542161","cluster_addr":"172.21.15.130:6819/957542161","heartbeat_back_addr":"172.21.15.130:6823/957542161","heartbeat_front_addr":"172.21.15.130:6821/957542161","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:41:41.451627+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:09.327795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:36.547203+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:00.627031+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:26.709919+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:52.576178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:6801/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/773571736":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/457222314":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/3850189972":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/17493483":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/402887427":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6801/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/3812624194":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/2096579260":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6800/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6801/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/633075379":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6800/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/2248276518":"2024-05-14T22:39:13.681560+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-13T22:44:16.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.715+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2456531525 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1aec06c4e0 msgr2=0x7f1aec06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:16.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.715+0000 7f1b0c3a1700 1 --2- 172.21.15.102:0/2456531525 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1aec06c4e0 0x7f1aec06e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1af8005fd0 tx=0x7f1af8005ee0 comp rx=0 tx=0).stop 2024-05-13T22:44:16.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.715+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2456531525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b04100140 msgr2=0x7f1b041974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:16.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.715+0000 7f1b0c3a1700 1 --2- 172.21.15.102:0/2456531525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b04100140 0x7f1b041974d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1afc000f80 tx=0x7f1afc00cdd0 comp rx=0 tx=0).stop 2024-05-13T22:44:16.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.715+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2456531525 shutdown_connections 2024-05-13T22:44:16.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.715+0000 7f1b0c3a1700 1 --2- 172.21.15.102:0/2456531525 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f1aec06c4e0 0x7f1aec06e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:16.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.715+0000 7f1b0c3a1700 1 --2- 172.21.15.102:0/2456531525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b04100140 0x7f1b041974d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:16.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.715+0000 7f1b0c3a1700 1 --2- 172.21.15.102:0/2456531525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b04104a50 0x7f1b04197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:16.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.715+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2456531525 >> 172.21.15.102:0/2456531525 conn(0x7f1b04075e30 msgr2=0x7f1b04103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:16.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.715+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2456531525 shutdown_connections 2024-05-13T22:44:16.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:16.715+0000 7f1b0c3a1700 1 -- 172.21.15.102:0/2456531525 wait complete. 2024-05-13T22:44:17.390 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:17 smithi102 ceph-mon[28599]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:17.390 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:17 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2456531525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T22:44:17.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:17 smithi130 ceph-mon[36591]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:17.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:17 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2456531525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T22:44:17.414 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-13T22:44:17.415 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-13T22:44:17.415 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-13T22:44:17.415 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-13T22:44:17.416 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-13T22:44:17.416 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-13T22:44:17.673 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:17.692 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:18.365 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:18.366 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:18.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:18 smithi130 ceph-mon[36591]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:18.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:18 smithi102 ceph-mon[28599]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:20.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.047+0000 7f08bed22700 1 -- 172.21.15.102:0/1388564170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b8105a50 msgr2=0x7f08b8105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:20.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.047+0000 7f08bed22700 1 --2- 172.21.15.102:0/1388564170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b8105a50 0x7f08b8105e30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009910 tx=0x7f08ac009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:20.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.047+0000 7f08bed22700 1 -- 172.21.15.102:0/1388564170 shutdown_connections 2024-05-13T22:44:20.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.047+0000 7f08bed22700 1 --2- 172.21.15.102:0/1388564170 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b80ffa50 0x7f08b80ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.047+0000 7f08bed22700 1 --2- 172.21.15.102:0/1388564170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b8105a50 0x7f08b8105e30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.047+0000 7f08bed22700 1 -- 172.21.15.102:0/1388564170 >> 172.21.15.102:0/1388564170 conn(0x7f08b80fb540 msgr2=0x7f08b80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.047+0000 7f08bed22700 1 -- 172.21.15.102:0/1388564170 shutdown_connections 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.047+0000 7f08bed22700 1 -- 172.21.15.102:0/1388564170 wait complete. 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.047+0000 7f08bed22700 1 Processor -- start 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.047+0000 7f08bed22700 1 -- start start 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bed22700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b80ffa50 0x7f08b81956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bed22700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b8105a50 0x7f08b8195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bed22700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b8196290 con 0x7f08b80ffa50 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bed22700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b8198fc0 con 0x7f08b8105a50 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bcabe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b80ffa50 0x7f08b81956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bcabe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b80ffa50 0x7f08b81956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47224/0 (socket says 172.21.15.102:47224) 2024-05-13T22:44:20.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bcabe700 1 -- 172.21.15.102:0/3853687175 learned_addr learned my addr 172.21.15.102:0/3853687175 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:20.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08b7fff700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b8105a50 0x7f08b8195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:20.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bcabe700 1 -- 172.21.15.102:0/3853687175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b8105a50 msgr2=0x7f08b8195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:20.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bcabe700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b8105a50 0x7f08b8195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:20.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bcabe700 1 -- 172.21.15.102:0/3853687175 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac0095f0 con 0x7f08b80ffa50 2024-05-13T22:44:20.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bcabe700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b80ffa50 0x7f08b81956c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f08ac006010 tx=0x7f08ac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:20.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08b5ffb700 1 -- 172.21.15.102:0/3853687175 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08ac01d070 con 0x7f08b80ffa50 2024-05-13T22:44:20.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08b5ffb700 1 -- 172.21.15.102:0/3853687175 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f08ac004fa0 con 0x7f08b80ffa50 2024-05-13T22:44:20.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08bed22700 1 -- 172.21.15.102:0/3853687175 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b8199240 con 0x7f08b80ffa50 2024-05-13T22:44:20.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.048+0000 7f08b5ffb700 1 -- 172.21.15.102:0/3853687175 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08ac00f690 con 0x7f08b80ffa50 2024-05-13T22:44:20.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.049+0000 7f08bed22700 1 -- 172.21.15.102:0/3853687175 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b8199730 con 0x7f08b80ffa50 2024-05-13T22:44:20.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.049+0000 7f08bed22700 1 -- 172.21.15.102:0/3853687175 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08b810cfd0 con 0x7f08b80ffa50 2024-05-13T22:44:20.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.050+0000 7f08b5ffb700 1 -- 172.21.15.102:0/3853687175 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f08ac022470 con 0x7f08b80ffa50 2024-05-13T22:44:20.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.050+0000 7f08b5ffb700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f08a806c4e0 0x7f08a806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:20.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.051+0000 7f08b5ffb700 1 -- 172.21.15.102:0/3853687175 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f08ac08d6e0 con 0x7f08b80ffa50 2024-05-13T22:44:20.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.051+0000 7f08b7fff700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f08a806c4e0 0x7f08a806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:20.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.051+0000 7f08b5ffb700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722] conn(0x7f08a80721c0 0x7f08a80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:20.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.051+0000 7f08b5ffb700 1 -- 172.21.15.102:0/3853687175 --> [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08a8074c90 con 0x7f08a80721c0 2024-05-13T22:44:20.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.051+0000 7f08b5ffb700 1 -- 172.21.15.102:0/3853687175 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08ac08db40 con 0x7f08b80ffa50 2024-05-13T22:44:20.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.051+0000 7f08bd2bf700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722] conn(0x7f08a80721c0 0x7f08a80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:20.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.051+0000 7f08b7fff700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f08a806c4e0 0x7f08a806e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f08b8196ce0 tx=0x7f08a4009190 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:20.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.051+0000 7f08bd2bf700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722] conn(0x7f08a80721c0 0x7f08a80745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:20.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.052+0000 7f08b5ffb700 1 -- 172.21.15.102:0/3853687175 <== osd.0 v2:172.21.15.102:6802/2497809722 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f08a8074c90 con 0x7f08a80721c0 2024-05-13T22:44:20.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.067+0000 7f08bed22700 1 -- 172.21.15.102:0/3853687175 --> [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08b804ea90 con 0x7f08a80721c0 2024-05-13T22:44:20.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.067+0000 7f08b5ffb700 1 -- 172.21.15.102:0/3853687175 <== osd.0 v2:172.21.15.102:6802/2497809722 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f08b804ea90 con 0x7f08a80721c0 2024-05-13T22:44:20.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.067+0000 7f08bed22700 1 -- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722] conn(0x7f08a80721c0 msgr2=0x7f08a80745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.067+0000 7f08bed22700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722] conn(0x7f08a80721c0 0x7f08a80745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 -- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f08a806c4e0 msgr2=0x7f08a806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f08a806c4e0 0x7f08a806e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f08b8196ce0 tx=0x7f08a4009190 comp rx=0 tx=0).stop 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 -- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b80ffa50 msgr2=0x7f08b81956c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b80ffa50 0x7f08b81956c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f08ac006010 tx=0x7f08ac00bd70 comp rx=0 tx=0).stop 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 -- 172.21.15.102:0/3853687175 shutdown_connections 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6802/2497809722,v1:172.21.15.102:6803/2497809722] conn(0x7f08a80721c0 0x7f08a80745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f08a806c4e0 0x7f08a806e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b80ffa50 0x7f08b81956c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 --2- 172.21.15.102:0/3853687175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08b8105a50 0x7f08b8195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 -- 172.21.15.102:0/3853687175 >> 172.21.15.102:0/3853687175 conn(0x7f08b80fb540 msgr2=0x7f08b80fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 -- 172.21.15.102:0/3853687175 shutdown_connections 2024-05-13T22:44:20.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:20.068+0000 7f08bed22700 1 -- 172.21.15.102:0/3853687175 wait complete. 2024-05-13T22:44:20.854 INFO:teuthology.orchestra.run.smithi102.stdout:38654705699 2024-05-13T22:44:20.854 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-13T22:44:21.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:21 smithi102 ceph-mon[28599]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:21.180 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:21.180 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:21.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:21 smithi130 ceph-mon[36591]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:22.356 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:22 smithi102 ceph-mon[28599]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:22 smithi130 ceph-mon[36591]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:22.603 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:24.035 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:23 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:44:24.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:23 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:44:24.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.325+0000 7f5925f90700 1 -- 172.21.15.102:0/2942114067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920101770 msgr2=0x7f59201057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:24.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.325+0000 7f5925f90700 1 --2- 172.21.15.102:0/2942114067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920101770 0x7f59201057c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5908009910 tx=0x7f5908009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:24.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.325+0000 7f5925f90700 1 -- 172.21.15.102:0/2942114067 shutdown_connections 2024-05-13T22:44:24.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.325+0000 7f5925f90700 1 --2- 172.21.15.102:0/2942114067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920101770 0x7f59201057c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:24.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.325+0000 7f5925f90700 1 --2- 172.21.15.102:0/2942114067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920100dc0 0x7f59201011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.325+0000 7f5925f90700 1 -- 172.21.15.102:0/2942114067 >> 172.21.15.102:0/2942114067 conn(0x7f59200fc650 msgr2=0x7f59200fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.325+0000 7f5925f90700 1 -- 172.21.15.102:0/2942114067 shutdown_connections 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.325+0000 7f5925f90700 1 -- 172.21.15.102:0/2942114067 wait complete. 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.326+0000 7f5925f90700 1 Processor -- start 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.326+0000 7f5925f90700 1 -- start start 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.326+0000 7f5925f90700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920100dc0 0x7f5920073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.326+0000 7f5925f90700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920101770 0x7f59200735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.326+0000 7f5925f90700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5920073b10 con 0x7f5920101770 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.326+0000 7f5925f90700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5920073c50 con 0x7f5920100dc0 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.326+0000 7f591effd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920101770 0x7f59200735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.326+0000 7f591effd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920101770 0x7f59200735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47254/0 (socket says 172.21.15.102:47254) 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.326+0000 7f591f7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920100dc0 0x7f5920073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:24.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.326+0000 7f591effd700 1 -- 172.21.15.102:0/2616528505 learned_addr learned my addr 172.21.15.102:0/2616528505 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:24.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.327+0000 7f591effd700 1 -- 172.21.15.102:0/2616528505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920100dc0 msgr2=0x7f5920073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:24.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.327+0000 7f591effd700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920100dc0 0x7f5920073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:24.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.327+0000 7f591effd700 1 -- 172.21.15.102:0/2616528505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080095f0 con 0x7f5920101770 2024-05-13T22:44:24.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.327+0000 7f591effd700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920101770 0x7f59200735d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f59080092e0 tx=0x7f590800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:24.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.327+0000 7f591cff9700 1 -- 172.21.15.102:0/2616528505 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f590801d070 con 0x7f5920101770 2024-05-13T22:44:24.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.327+0000 7f5925f90700 1 -- 172.21.15.102:0/2616528505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5920073ed0 con 0x7f5920101770 2024-05-13T22:44:24.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.327+0000 7f591cff9700 1 -- 172.21.15.102:0/2616528505 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5908005dc0 con 0x7f5920101770 2024-05-13T22:44:24.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.327+0000 7f591cff9700 1 -- 172.21.15.102:0/2616528505 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5908017400 con 0x7f5920101770 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.327+0000 7f5925f90700 1 -- 172.21.15.102:0/2616528505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59201a2c90 con 0x7f5920101770 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.328+0000 7f5925f90700 1 -- 172.21.15.102:0/2616528505 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f592010e060 con 0x7f5920101770 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.328+0000 7f591cff9700 1 -- 172.21.15.102:0/2616528505 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f590800f460 con 0x7f5920101770 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.329+0000 7f591cff9700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590c06c4e0 0x7f590c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.329+0000 7f591cff9700 1 -- 172.21.15.102:0/2616528505 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f590808ce20 con 0x7f5920101770 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.329+0000 7f591f7fe700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590c06c4e0 0x7f590c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.329+0000 7f591cff9700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161] conn(0x7f590c0721c0 0x7f590c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.329+0000 7f591cff9700 1 -- 172.21.15.102:0/2616528505 --> [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f590c074c90 con 0x7f590c0721c0 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.329+0000 7f591cff9700 1 -- 172.21.15.102:0/2616528505 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f590808d280 con 0x7f5920101770 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.329+0000 7f591ffff700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161] conn(0x7f590c0721c0 0x7f590c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.329+0000 7f591f7fe700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590c06c4e0 0x7f590c06e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5910009550 tx=0x7f59100091b0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:24.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.330+0000 7f591ffff700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161] conn(0x7f590c0721c0 0x7f590c0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:24.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.330+0000 7f591cff9700 1 -- 172.21.15.102:0/2616528505 <== osd.5 v2:172.21.15.130:6816/957542161 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f590c074c90 con 0x7f590c0721c0 2024-05-13T22:44:24.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.346+0000 7f5925f90700 1 -- 172.21.15.102:0/2616528505 --> [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f592004ea90 con 0x7f590c0721c0 2024-05-13T22:44:24.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f591cff9700 1 -- 172.21.15.102:0/2616528505 <== osd.5 v2:172.21.15.130:6816/957542161 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f592004ea90 con 0x7f590c0721c0 2024-05-13T22:44:24.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 -- 172.21.15.102:0/2616528505 >> [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161] conn(0x7f590c0721c0 msgr2=0x7f590c0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:24.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161] conn(0x7f590c0721c0 0x7f590c0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:24.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 -- 172.21.15.102:0/2616528505 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590c06c4e0 msgr2=0x7f590c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:24.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590c06c4e0 0x7f590c06e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5910009550 tx=0x7f59100091b0 comp rx=0 tx=0).stop 2024-05-13T22:44:24.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 -- 172.21.15.102:0/2616528505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920101770 msgr2=0x7f59200735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:24.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920101770 0x7f59200735d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f59080092e0 tx=0x7f590800bd70 comp rx=0 tx=0).stop 2024-05-13T22:44:24.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 -- 172.21.15.102:0/2616528505 shutdown_connections 2024-05-13T22:44:24.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590c06c4e0 0x7f590c06e9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:24.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920100dc0 0x7f5920073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:24.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.130:6816/957542161,v1:172.21.15.130:6817/957542161] conn(0x7f590c0721c0 0x7f590c0745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:24.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 --2- 172.21.15.102:0/2616528505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920101770 0x7f59200735d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:24.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 -- 172.21.15.102:0/2616528505 >> 172.21.15.102:0/2616528505 conn(0x7f59200fc650 msgr2=0x7f59200fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:24.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 -- 172.21.15.102:0/2616528505 shutdown_connections 2024-05-13T22:44:24.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:24.347+0000 7f5925f90700 1 -- 172.21.15.102:0/2616528505 wait complete. 2024-05-13T22:44:25.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:24 smithi130 ceph-mon[36591]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:25.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:24 smithi102 ceph-mon[28599]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:26.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.023+0000 7f89b0b60700 1 -- 172.21.15.102:0/2911681646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89ac0ffa40 msgr2=0x7f89ac0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:26.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.023+0000 7f89b0b60700 1 --2- 172.21.15.102:0/2911681646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89ac0ffa40 0x7f89ac0ffec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009910 tx=0x7f89a0009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.023+0000 7f89b0b60700 1 -- 172.21.15.102:0/2911681646 shutdown_connections 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.023+0000 7f89b0b60700 1 --2- 172.21.15.102:0/2911681646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89ac0ffa40 0x7f89ac0ffec0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.023+0000 7f89b0b60700 1 --2- 172.21.15.102:0/2911681646 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89ac105a40 0x7f89ac105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.023+0000 7f89b0b60700 1 -- 172.21.15.102:0/2911681646 >> 172.21.15.102:0/2911681646 conn(0x7f89ac0fb570 msgr2=0x7f89ac0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.023+0000 7f89b0b60700 1 -- 172.21.15.102:0/2911681646 shutdown_connections 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.023+0000 7f89b0b60700 1 -- 172.21.15.102:0/2911681646 wait complete. 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.024+0000 7f89b0b60700 1 Processor -- start 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.024+0000 7f89b0b60700 1 -- start start 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.024+0000 7f89b0b60700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89ac0ffa40 0x7f89ac196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.024+0000 7f89b0b60700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89ac105a40 0x7f89ac196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.024+0000 7f89b0b60700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac1972e0 con 0x7f89ac105a40 2024-05-13T22:44:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.024+0000 7f89b0b60700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac19a010 con 0x7f89ac0ffa40 2024-05-13T22:44:26.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.024+0000 7f89a9d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89ac105a40 0x7f89ac196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:26.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.024+0000 7f89a9d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89ac105a40 0x7f89ac196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47278/0 (socket says 172.21.15.102:47278) 2024-05-13T22:44:26.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.024+0000 7f89a9d9b700 1 -- 172.21.15.102:0/1549544213 learned_addr learned my addr 172.21.15.102:0/1549544213 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:26.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.024+0000 7f89aa59c700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89ac0ffa40 0x7f89ac196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:26.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.025+0000 7f89a9d9b700 1 -- 172.21.15.102:0/1549544213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89ac0ffa40 msgr2=0x7f89ac196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:26.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.025+0000 7f89a9d9b700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89ac0ffa40 0x7f89ac196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.005 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.025+0000 7f89a9d9b700 1 -- 172.21.15.102:0/1549544213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00095f0 con 0x7f89ac105a40 2024-05-13T22:44:26.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.025+0000 7f89a9d9b700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89ac105a40 0x7f89ac196c50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f89a0006010 tx=0x7f89a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:26.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.025+0000 7f899b7fe700 1 -- 172.21.15.102:0/1549544213 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89a001d070 con 0x7f89ac105a40 2024-05-13T22:44:26.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.025+0000 7f89b0b60700 1 -- 172.21.15.102:0/1549544213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ac19a290 con 0x7f89ac105a40 2024-05-13T22:44:26.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.025+0000 7f899b7fe700 1 -- 172.21.15.102:0/1549544213 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f89a000f460 con 0x7f89ac105a40 2024-05-13T22:44:26.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.025+0000 7f899b7fe700 1 -- 172.21.15.102:0/1549544213 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89a0021640 con 0x7f89ac105a40 2024-05-13T22:44:26.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.025+0000 7f89b0b60700 1 -- 172.21.15.102:0/1549544213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ac19a780 con 0x7f89ac105a40 2024-05-13T22:44:26.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.026+0000 7f89b0b60700 1 -- 172.21.15.102:0/1549544213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89ac10cec0 con 0x7f89ac105a40 2024-05-13T22:44:26.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.027+0000 7f899b7fe700 1 -- 172.21.15.102:0/1549544213 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f89a000faf0 con 0x7f89ac105a40 2024-05-13T22:44:26.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.027+0000 7f899b7fe700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f899406c4e0 0x7f899406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:26.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.027+0000 7f899b7fe700 1 -- 172.21.15.102:0/1549544213 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f89a008cfd0 con 0x7f89ac105a40 2024-05-13T22:44:26.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.027+0000 7f89aa59c700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f899406c4e0 0x7f899406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:26.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.027+0000 7f899b7fe700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373] conn(0x7f89940721c0 0x7f89940745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:26.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.027+0000 7f899b7fe700 1 -- 172.21.15.102:0/1549544213 --> [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8994074c90 con 0x7f89940721c0 2024-05-13T22:44:26.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.027+0000 7f899b7fe700 1 -- 172.21.15.102:0/1549544213 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89a008d430 con 0x7f89ac105a40 2024-05-13T22:44:26.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.027+0000 7f89aad9d700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373] conn(0x7f89940721c0 0x7f89940745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:26.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.028+0000 7f89aa59c700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f899406c4e0 0x7f899406e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f899c005fd0 tx=0x7f899c005db0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:26.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.028+0000 7f89aad9d700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373] conn(0x7f89940721c0 0x7f89940745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:26.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.028+0000 7f899b7fe700 1 -- 172.21.15.102:0/1549544213 <== osd.2 v2:172.21.15.102:6818/3553677373 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8994074c90 con 0x7f89940721c0 2024-05-13T22:44:26.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.047+0000 7f89b0b60700 1 -- 172.21.15.102:0/1549544213 --> [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f89ac04ea90 con 0x7f89940721c0 2024-05-13T22:44:26.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f899b7fe700 1 -- 172.21.15.102:0/1549544213 <== osd.2 v2:172.21.15.102:6818/3553677373 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f89ac04ea90 con 0x7f89940721c0 2024-05-13T22:44:26.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 -- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373] conn(0x7f89940721c0 msgr2=0x7f89940745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373] conn(0x7f89940721c0 0x7f89940745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 -- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f899406c4e0 msgr2=0x7f899406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f899406c4e0 0x7f899406e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f899c005fd0 tx=0x7f899c005db0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 -- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89ac105a40 msgr2=0x7f89ac196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89ac105a40 0x7f89ac196c50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f89a0006010 tx=0x7f89a000be20 comp rx=0 tx=0).stop 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 -- 172.21.15.102:0/1549544213 shutdown_connections 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6818/3553677373,v1:172.21.15.102:6819/3553677373] conn(0x7f89940721c0 0x7f89940745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f899406c4e0 0x7f899406e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f89ac0ffa40 0x7f89ac196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 --2- 172.21.15.102:0/1549544213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f89ac105a40 0x7f89ac196c50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.048+0000 7f89b0b60700 1 -- 172.21.15.102:0/1549544213 >> 172.21.15.102:0/1549544213 conn(0x7f89ac0fb570 msgr2=0x7f89ac069070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.049+0000 7f89b0b60700 1 -- 172.21.15.102:0/1549544213 shutdown_connections 2024-05-13T22:44:26.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.049+0000 7f89b0b60700 1 -- 172.21.15.102:0/1549544213 wait complete. 2024-05-13T22:44:26.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.494+0000 7f168fc74700 1 -- 172.21.15.102:0/4270070041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16880ffa40 msgr2=0x7f16880ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:26.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.494+0000 7f168fc74700 1 --2- 172.21.15.102:0/4270070041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16880ffa40 0x7f16880ffec0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1678009910 tx=0x7f1678009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:26.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.494+0000 7f168fc74700 1 -- 172.21.15.102:0/4270070041 shutdown_connections 2024-05-13T22:44:26.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.494+0000 7f168fc74700 1 --2- 172.21.15.102:0/4270070041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16880ffa40 0x7f16880ffec0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.475 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.494+0000 7f168fc74700 1 --2- 172.21.15.102:0/4270070041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1688105a40 0x7f1688105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.494+0000 7f168fc74700 1 -- 172.21.15.102:0/4270070041 >> 172.21.15.102:0/4270070041 conn(0x7f16880fb570 msgr2=0x7f16880fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.494+0000 7f168fc74700 1 -- 172.21.15.102:0/4270070041 shutdown_connections 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.494+0000 7f168fc74700 1 -- 172.21.15.102:0/4270070041 wait complete. 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.495+0000 7f168fc74700 1 Processor -- start 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.495+0000 7f168fc74700 1 -- start start 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.495+0000 7f168fc74700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16880ffa40 0x7f1688195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.495+0000 7f168fc74700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1688105a40 0x7f1688195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.495+0000 7f168fc74700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1688196240 con 0x7f16880ffa40 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.495+0000 7f168fc74700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1688199fd0 con 0x7f1688105a40 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.495+0000 7f168da10700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16880ffa40 0x7f1688195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.495+0000 7f168da10700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16880ffa40 0x7f1688195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47294/0 (socket says 172.21.15.102:47294) 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.495+0000 7f168da10700 1 -- 172.21.15.102:0/3942622111 learned_addr learned my addr 172.21.15.102:0/3942622111 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:26.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.495+0000 7f168d20f700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1688105a40 0x7f1688195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:26.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.496+0000 7f168da10700 1 -- 172.21.15.102:0/3942622111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1688105a40 msgr2=0x7f1688195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:26.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.496+0000 7f168da10700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1688105a40 0x7f1688195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.496+0000 7f168da10700 1 -- 172.21.15.102:0/3942622111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780095f0 con 0x7f16880ffa40 2024-05-13T22:44:26.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.496+0000 7f168d20f700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1688105a40 0x7f1688195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:44:26.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.496+0000 7f168da10700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16880ffa40 0x7f1688195620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f168400d8d0 tx=0x7f168400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:26.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.496+0000 7f167effd700 1 -- 172.21.15.102:0/3942622111 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1684009940 con 0x7f16880ffa40 2024-05-13T22:44:26.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.496+0000 7f167effd700 1 -- 172.21.15.102:0/3942622111 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1684010460 con 0x7f16880ffa40 2024-05-13T22:44:26.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.496+0000 7f168fc74700 1 -- 172.21.15.102:0/3942622111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168819a2b0 con 0x7f16880ffa40 2024-05-13T22:44:26.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.496+0000 7f167effd700 1 -- 172.21.15.102:0/3942622111 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f168400f5f0 con 0x7f16880ffa40 2024-05-13T22:44:26.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.496+0000 7f168fc74700 1 -- 172.21.15.102:0/3942622111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168819a800 con 0x7f16880ffa40 2024-05-13T22:44:26.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.497+0000 7f168fc74700 1 -- 172.21.15.102:0/3942622111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f168810cf40 con 0x7f16880ffa40 2024-05-13T22:44:26.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.498+0000 7f167effd700 1 -- 172.21.15.102:0/3942622111 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f16840105d0 con 0x7f16880ffa40 2024-05-13T22:44:26.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.498+0000 7f167effd700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f167406c4e0 0x7f167406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:26.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.498+0000 7f167effd700 1 -- 172.21.15.102:0/3942622111 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f168408ba00 con 0x7f16880ffa40 2024-05-13T22:44:26.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.498+0000 7f168d20f700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f167406c4e0 0x7f167406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:26.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.498+0000 7f167effd700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323] conn(0x7f16740721c0 0x7f16740745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:26.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.498+0000 7f167effd700 1 -- 172.21.15.102:0/3942622111 --> [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1674074c90 con 0x7f16740721c0 2024-05-13T22:44:26.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.498+0000 7f167effd700 1 -- 172.21.15.102:0/3942622111 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f168408be60 con 0x7f16880ffa40 2024-05-13T22:44:26.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.498+0000 7f168e211700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323] conn(0x7f16740721c0 0x7f16740745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:26.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.499+0000 7f168d20f700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f167406c4e0 0x7f167406e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1678005b40 tx=0x7f167801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:26.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.499+0000 7f168e211700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323] conn(0x7f16740721c0 0x7f16740745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:26.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.499+0000 7f167effd700 1 -- 172.21.15.102:0/3942622111 <== osd.1 v2:172.21.15.102:6810/2169124323 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1674074c90 con 0x7f16740721c0 2024-05-13T22:44:26.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 -- 172.21.15.102:0/3942622111 --> [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f168804ea90 con 0x7f16740721c0 2024-05-13T22:44:26.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f167effd700 1 -- 172.21.15.102:0/3942622111 <== osd.1 v2:172.21.15.102:6810/2169124323 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f168804ea90 con 0x7f16740721c0 2024-05-13T22:44:26.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 -- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323] conn(0x7f16740721c0 msgr2=0x7f16740745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:26.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323] conn(0x7f16740721c0 0x7f16740745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 -- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f167406c4e0 msgr2=0x7f167406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:26.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f167406c4e0 0x7f167406e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1678005b40 tx=0x7f167801a040 comp rx=0 tx=0).stop 2024-05-13T22:44:26.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 -- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16880ffa40 msgr2=0x7f1688195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:26.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16880ffa40 0x7f1688195620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f168400d8d0 tx=0x7f168400dc90 comp rx=0 tx=0).stop 2024-05-13T22:44:26.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 -- 172.21.15.102:0/3942622111 shutdown_connections 2024-05-13T22:44:26.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6810/2169124323,v1:172.21.15.102:6811/2169124323] conn(0x7f16740721c0 0x7f16740745e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f167406c4e0 0x7f167406e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16880ffa40 0x7f1688195620 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 --2- 172.21.15.102:0/3942622111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1688105a40 0x7f1688195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:26.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.515+0000 7f168fc74700 1 -- 172.21.15.102:0/3942622111 >> 172.21.15.102:0/3942622111 conn(0x7f16880fb570 msgr2=0x7f16880fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:26.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.516+0000 7f168fc74700 1 -- 172.21.15.102:0/3942622111 shutdown_connections 2024-05-13T22:44:26.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:26.516+0000 7f168fc74700 1 -- 172.21.15.102:0/3942622111 wait complete. 2024-05-13T22:44:27.334 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:27 smithi102 ceph-mon[28599]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:27.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:27 smithi130 ceph-mon[36591]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:28.289 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:28 smithi102 ceph-mon[28599]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:28.316 INFO:teuthology.orchestra.run.smithi102.stdout:81604378648 2024-05-13T22:44:28.316 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-13T22:44:28.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:28 smithi130 ceph-mon[36591]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.062+0000 7f71efa6c700 1 -- 172.21.15.102:0/287493281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e80ffbd0 msgr2=0x7f71e8100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.062+0000 7f71efa6c700 1 --2- 172.21.15.102:0/287493281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e80ffbd0 0x7f71e8100050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009910 tx=0x7f71e4009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.062+0000 7f71efa6c700 1 -- 172.21.15.102:0/287493281 shutdown_connections 2024-05-13T22:44:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.062+0000 7f71efa6c700 1 --2- 172.21.15.102:0/287493281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e80ffbd0 0x7f71e8100050 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:29.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.062+0000 7f71efa6c700 1 --2- 172.21.15.102:0/287493281 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e8105a40 0x7f71e8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.062+0000 7f71efa6c700 1 -- 172.21.15.102:0/287493281 >> 172.21.15.102:0/287493281 conn(0x7f71e80755a0 msgr2=0x7f71e80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.062+0000 7f71efa6c700 1 -- 172.21.15.102:0/287493281 shutdown_connections 2024-05-13T22:44:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.062+0000 7f71efa6c700 1 -- 172.21.15.102:0/287493281 wait complete. 2024-05-13T22:44:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71efa6c700 1 Processor -- start 2024-05-13T22:44:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71efa6c700 1 -- start start 2024-05-13T22:44:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71efa6c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e80ffbd0 0x7f71e819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71efa6c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e8105a40 0x7f71e819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71efa6c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e819d140 con 0x7f71e80ffbd0 2024-05-13T22:44:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71efa6c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81965f0 con 0x7f71e8105a40 2024-05-13T22:44:29.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71ed808700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e80ffbd0 0x7f71e819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:29.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71ed808700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e80ffbd0 0x7f71e819c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34908/0 (socket says 172.21.15.102:34908) 2024-05-13T22:44:29.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71ed808700 1 -- 172.21.15.102:0/850495948 learned_addr learned my addr 172.21.15.102:0/850495948 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:29.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71ed007700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e8105a40 0x7f71e819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:29.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71ed808700 1 -- 172.21.15.102:0/850495948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e8105a40 msgr2=0x7f71e819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:29.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71ed808700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e8105a40 0x7f71e819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:29.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71ed808700 1 -- 172.21.15.102:0/850495948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e40095f0 con 0x7f71e80ffbd0 2024-05-13T22:44:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71ed808700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e80ffbd0 0x7f71e819c570 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f71d800b4b0 tx=0x7f71d800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71deffd700 1 -- 172.21.15.102:0/850495948 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71d8011800 con 0x7f71e80ffbd0 2024-05-13T22:44:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71efa6c700 1 -- 172.21.15.102:0/850495948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e81968d0 con 0x7f71e80ffbd0 2024-05-13T22:44:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71deffd700 1 -- 172.21.15.102:0/850495948 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f71d8011e10 con 0x7f71e80ffbd0 2024-05-13T22:44:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.063+0000 7f71deffd700 1 -- 172.21.15.102:0/850495948 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71d800f2e0 con 0x7f71e80ffbd0 2024-05-13T22:44:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.064+0000 7f71efa6c700 1 -- 172.21.15.102:0/850495948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e8196e20 con 0x7f71e80ffbd0 2024-05-13T22:44:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.064+0000 7f71efa6c700 1 -- 172.21.15.102:0/850495948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f71e810cfc0 con 0x7f71e80ffbd0 2024-05-13T22:44:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.065+0000 7f71deffd700 1 -- 172.21.15.102:0/850495948 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f71d8010380 con 0x7f71e80ffbd0 2024-05-13T22:44:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.065+0000 7f71deffd700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f71d406c4e0 0x7f71d406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:29.156 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.065+0000 7f71deffd700 1 -- 172.21.15.102:0/850495948 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f71d808b130 con 0x7f71e80ffbd0 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.065+0000 7f71ed007700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f71d406c4e0 0x7f71d406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.066+0000 7f71deffd700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705] conn(0x7f71d40721c0 0x7f71d40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.066+0000 7f71deffd700 1 -- 172.21.15.102:0/850495948 --> [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71d4074c90 con 0x7f71d40721c0 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.066+0000 7f71deffd700 1 -- 172.21.15.102:0/850495948 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71d808d800 con 0x7f71e80ffbd0 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.066+0000 7f71ee009700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705] conn(0x7f71d40721c0 0x7f71d40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.066+0000 7f71ed007700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f71d406c4e0 0x7f71d406e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f71e8104d20 tx=0x7f71e401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.066+0000 7f71ee009700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705] conn(0x7f71d40721c0 0x7f71d40745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.067+0000 7f71deffd700 1 -- 172.21.15.102:0/850495948 <== osd.3 v2:172.21.15.130:6800/4175190705 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f71d4074c90 con 0x7f71d40721c0 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.082+0000 7f71efa6c700 1 -- 172.21.15.102:0/850495948 --> [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f71e804ea90 con 0x7f71d40721c0 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71deffd700 1 -- 172.21.15.102:0/850495948 <== osd.3 v2:172.21.15.130:6800/4175190705 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f71e804ea90 con 0x7f71d40721c0 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 -- 172.21.15.102:0/850495948 >> [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705] conn(0x7f71d40721c0 msgr2=0x7f71d40745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705] conn(0x7f71d40721c0 0x7f71d40745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:29.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 -- 172.21.15.102:0/850495948 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f71d406c4e0 msgr2=0x7f71d406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f71d406c4e0 0x7f71d406e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f71e8104d20 tx=0x7f71e401a040 comp rx=0 tx=0).stop 2024-05-13T22:44:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 -- 172.21.15.102:0/850495948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e80ffbd0 msgr2=0x7f71e819c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e80ffbd0 0x7f71e819c570 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f71d800b4b0 tx=0x7f71d800b870 comp rx=0 tx=0).stop 2024-05-13T22:44:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 -- 172.21.15.102:0/850495948 shutdown_connections 2024-05-13T22:44:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.130:6800/4175190705,v1:172.21.15.130:6801/4175190705] conn(0x7f71d40721c0 0x7f71d40745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f71d406c4e0 0x7f71d406e9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e80ffbd0 0x7f71e819c570 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 --2- 172.21.15.102:0/850495948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e8105a40 0x7f71e819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.083+0000 7f71efa6c700 1 -- 172.21.15.102:0/850495948 >> 172.21.15.102:0/850495948 conn(0x7f71e80755a0 msgr2=0x7f71e8068610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.084+0000 7f71efa6c700 1 -- 172.21.15.102:0/850495948 shutdown_connections 2024-05-13T22:44:29.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:29.084+0000 7f71efa6c700 1 -- 172.21.15.102:0/850495948 wait complete. 2024-05-13T22:44:29.786 INFO:teuthology.orchestra.run.smithi102.stdout:171798691848 2024-05-13T22:44:29.787 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-13T22:44:30.181 INFO:teuthology.orchestra.run.smithi102.stdout:60129542174 2024-05-13T22:44:30.181 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-13T22:44:30.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.909+0000 7f2fb233e700 1 -- 172.21.15.102:0/822197708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fac0ffbd0 msgr2=0x7f2fac100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:30.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.909+0000 7f2fb233e700 1 --2- 172.21.15.102:0/822197708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fac0ffbd0 0x7f2fac100050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009910 tx=0x7f2fa0009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:30.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.909+0000 7f2fb233e700 1 -- 172.21.15.102:0/822197708 shutdown_connections 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.909+0000 7f2fb233e700 1 --2- 172.21.15.102:0/822197708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fac0ffbd0 0x7f2fac100050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.909+0000 7f2fb233e700 1 --2- 172.21.15.102:0/822197708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fac105a40 0x7f2fac105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.909+0000 7f2fb233e700 1 -- 172.21.15.102:0/822197708 >> 172.21.15.102:0/822197708 conn(0x7f2fac0755a0 msgr2=0x7f2fac0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.909+0000 7f2fb233e700 1 -- 172.21.15.102:0/822197708 shutdown_connections 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.909+0000 7f2fb233e700 1 -- 172.21.15.102:0/822197708 wait complete. 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.909+0000 7f2fb233e700 1 Processor -- start 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.909+0000 7f2fb233e700 1 -- start start 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fb233e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fac0ffbd0 0x7f2fac19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fb233e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fac105a40 0x7f2fac19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fb233e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac19d0d0 con 0x7f2fac105a40 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fb233e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac196580 con 0x7f2fac0ffbd0 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fac105a40 0x7f2fac19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fac105a40 0x7f2fac19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34950/0 (socket says 172.21.15.102:34950) 2024-05-13T22:44:30.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fab7fe700 1 -- 172.21.15.102:0/3280766430 learned_addr learned my addr 172.21.15.102:0/3280766430 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:30.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fabfff700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fac0ffbd0 0x7f2fac19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:30.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fab7fe700 1 -- 172.21.15.102:0/3280766430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fac0ffbd0 msgr2=0x7f2fac19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:30.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fab7fe700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fac0ffbd0 0x7f2fac19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:30.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fab7fe700 1 -- 172.21.15.102:0/3280766430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa00095f0 con 0x7f2fac105a40 2024-05-13T22:44:30.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.910+0000 7f2fab7fe700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fac105a40 0x7f2fac19ca40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0006010 tx=0x7f2fa000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:30.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.911+0000 7f2fa97fa700 1 -- 172.21.15.102:0/3280766430 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fa001d070 con 0x7f2fac105a40 2024-05-13T22:44:30.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.911+0000 7f2fa97fa700 1 -- 172.21.15.102:0/3280766430 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2fa000f460 con 0x7f2fac105a40 2024-05-13T22:44:30.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.911+0000 7f2fa97fa700 1 -- 172.21.15.102:0/3280766430 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fa0021640 con 0x7f2fac105a40 2024-05-13T22:44:30.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.911+0000 7f2fb233e700 1 -- 172.21.15.102:0/3280766430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac196800 con 0x7f2fac105a40 2024-05-13T22:44:30.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.911+0000 7f2fb233e700 1 -- 172.21.15.102:0/3280766430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac196cf0 con 0x7f2fac105a40 2024-05-13T22:44:30.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.912+0000 7f2fb233e700 1 -- 172.21.15.102:0/3280766430 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2fac10cf40 con 0x7f2fac105a40 2024-05-13T22:44:30.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.912+0000 7f2fa97fa700 1 -- 172.21.15.102:0/3280766430 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2fa000faf0 con 0x7f2fac105a40 2024-05-13T22:44:30.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.913+0000 7f2fa97fa700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f9406c4e0 0x7f2f9406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:30.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.913+0000 7f2fabfff700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f9406c4e0 0x7f2f9406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:30.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.913+0000 7f2fa97fa700 1 -- 172.21.15.102:0/3280766430 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f2fa008d190 con 0x7f2fac105a40 2024-05-13T22:44:30.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.913+0000 7f2fa97fa700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717] conn(0x7f2f940721c0 0x7f2f940745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:30.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.913+0000 7f2fa97fa700 1 -- 172.21.15.102:0/3280766430 --> [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f94074c90 con 0x7f2f940721c0 2024-05-13T22:44:30.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.913+0000 7f2fa97fa700 1 -- 172.21.15.102:0/3280766430 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa008d5f0 con 0x7f2fac105a40 2024-05-13T22:44:30.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.913+0000 7f2fabfff700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f9406c4e0 0x7f2f9406e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c005fd0 tx=0x7f2f9c00a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:30.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.913+0000 7f2fb08db700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717] conn(0x7f2f940721c0 0x7f2f940745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:30.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.914+0000 7f2fb08db700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717] conn(0x7f2f940721c0 0x7f2f940745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:30.893 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.914+0000 7f2fa97fa700 1 -- 172.21.15.102:0/3280766430 <== osd.4 v2:172.21.15.130:6808/574281717 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2f94074c90 con 0x7f2f940721c0 2024-05-13T22:44:30.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.930+0000 7f2fb233e700 1 -- 172.21.15.102:0/3280766430 --> [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2fac04ea90 con 0x7f2f940721c0 2024-05-13T22:44:30.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.930+0000 7f2fa97fa700 1 -- 172.21.15.102:0/3280766430 <== osd.4 v2:172.21.15.130:6808/574281717 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2fac04ea90 con 0x7f2f940721c0 2024-05-13T22:44:30.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 -- 172.21.15.102:0/3280766430 >> [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717] conn(0x7f2f940721c0 msgr2=0x7f2f940745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717] conn(0x7f2f940721c0 0x7f2f940745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 -- 172.21.15.102:0/3280766430 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f9406c4e0 msgr2=0x7f2f9406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f9406c4e0 0x7f2f9406e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c005fd0 tx=0x7f2f9c00a560 comp rx=0 tx=0).stop 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 -- 172.21.15.102:0/3280766430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fac105a40 msgr2=0x7f2fac19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fac105a40 0x7f2fac19ca40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0006010 tx=0x7f2fa000be20 comp rx=0 tx=0).stop 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 -- 172.21.15.102:0/3280766430 shutdown_connections 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f2f9406c4e0 0x7f2f9406e9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fac0ffbd0 0x7f2fac19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.130:6808/574281717,v1:172.21.15.130:6809/574281717] conn(0x7f2f940721c0 0x7f2f940745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 --2- 172.21.15.102:0/3280766430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fac105a40 0x7f2fac19ca40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 -- 172.21.15.102:0/3280766430 >> 172.21.15.102:0/3280766430 conn(0x7f2fac0755a0 msgr2=0x7f2fac068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 -- 172.21.15.102:0/3280766430 shutdown_connections 2024-05-13T22:44:30.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.931+0000 7f2fb233e700 1 -- 172.21.15.102:0/3280766430 wait complete. 2024-05-13T22:44:31.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:31 smithi130 ceph-mon[36591]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:31.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:31 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3653368326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-13T22:44:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.814+0000 7f9a6d522700 1 -- 172.21.15.102:0/949592654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a68104a50 msgr2=0x7f9a68104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:31.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.814+0000 7f9a6d522700 1 --2- 172.21.15.102:0/949592654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a68104a50 0x7f9a68104e30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009910 tx=0x7f9a50009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.814+0000 7f9a6d522700 1 -- 172.21.15.102:0/949592654 shutdown_connections 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.814+0000 7f9a6d522700 1 --2- 172.21.15.102:0/949592654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a68100180 0x7f9a68100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.814+0000 7f9a6d522700 1 --2- 172.21.15.102:0/949592654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a68104a50 0x7f9a68104e30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.814+0000 7f9a6d522700 1 -- 172.21.15.102:0/949592654 >> 172.21.15.102:0/949592654 conn(0x7f9a68075e30 msgr2=0x7f9a680fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.814+0000 7f9a6d522700 1 -- 172.21.15.102:0/949592654 shutdown_connections 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.814+0000 7f9a6d522700 1 -- 172.21.15.102:0/949592654 wait complete. 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a6d522700 1 Processor -- start 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a6d522700 1 -- start start 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a6d522700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a68100180 0x7f9a68197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a6d522700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a68104a50 0x7f9a68197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a6d522700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a68198070 con 0x7f9a68104a50 2024-05-13T22:44:31.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a6d522700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a681981b0 con 0x7f9a68100180 2024-05-13T22:44:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a667fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a68104a50 0x7f9a68197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a667fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a68104a50 0x7f9a68197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34934/0 (socket says 172.21.15.102:34934) 2024-05-13T22:44:31.303 INFO:teuthology.orchestra.run.smithi102.stdout:38654705701 2024-05-13T22:44:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a667fc700 1 -- 172.21.15.102:0/3653368326 learned_addr learned my addr 172.21.15.102:0/3653368326 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a66ffd700 1 --2- 172.21.15.102:0/3653368326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a68100180 0x7f9a68197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:31.303 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a667fc700 1 -- 172.21.15.102:0/3653368326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a68100180 msgr2=0x7f9a68197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.815+0000 7f9a667fc700 1 --2- 172.21.15.102:0/3653368326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a68100180 0x7f9a68197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.816+0000 7f9a667fc700 1 -- 172.21.15.102:0/3653368326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500095f0 con 0x7f9a68104a50 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.816+0000 7f9a667fc700 1 --2- 172.21.15.102:0/3653368326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a68104a50 0x7f9a68197a50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800d8d0 tx=0x7f9a5800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.816+0000 7f9a5ffff700 1 -- 172.21.15.102:0/3653368326 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a580098a0 con 0x7f9a68104a50 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.816+0000 7f9a5ffff700 1 -- 172.21.15.102:0/3653368326 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9a58010460 con 0x7f9a68104a50 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.816+0000 7f9a6d522700 1 -- 172.21.15.102:0/3653368326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a681a0580 con 0x7f9a68104a50 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.816+0000 7f9a5ffff700 1 -- 172.21.15.102:0/3653368326 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a5800f5f0 con 0x7f9a68104a50 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.816+0000 7f9a6d522700 1 -- 172.21.15.102:0/3653368326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a681a0a40 con 0x7f9a68104a50 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.817+0000 7f9a6d522700 1 -- 172.21.15.102:0/3653368326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a68108020 con 0x7f9a68104a50 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.817+0000 7f9a5ffff700 1 -- 172.21.15.102:0/3653368326 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9a5800f820 con 0x7f9a68104a50 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.818+0000 7f9a5ffff700 1 --2- 172.21.15.102:0/3653368326 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9a5406c4e0 0x7f9a5406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.818+0000 7f9a66ffd700 1 --2- 172.21.15.102:0/3653368326 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9a5406c4e0 0x7f9a5406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.818+0000 7f9a5ffff700 1 -- 172.21.15.102:0/3653368326 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9a5808b3c0 con 0x7f9a68104a50 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.819+0000 7f9a66ffd700 1 --2- 172.21.15.102:0/3653368326 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9a5406c4e0 0x7f9a5406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006010 tx=0x7f9a5001a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:31.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:30.823+0000 7f9a5ffff700 1 -- 172.21.15.102:0/3653368326 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9a58055ee0 con 0x7f9a68104a50 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.021+0000 7f9a6d522700 1 -- 172.21.15.102:0/3653368326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9a6804ea90 con 0x7f9a68104a50 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.021+0000 7f9a5ffff700 1 -- 172.21.15.102:0/3653368326 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9a58059500 con 0x7f9a68104a50 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.023+0000 7f9a6d522700 1 -- 172.21.15.102:0/3653368326 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9a5406c4e0 msgr2=0x7f9a5406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.023+0000 7f9a6d522700 1 --2- 172.21.15.102:0/3653368326 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9a5406c4e0 0x7f9a5406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006010 tx=0x7f9a5001a040 comp rx=0 tx=0).stop 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.023+0000 7f9a6d522700 1 -- 172.21.15.102:0/3653368326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a68104a50 msgr2=0x7f9a68197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.023+0000 7f9a6d522700 1 --2- 172.21.15.102:0/3653368326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a68104a50 0x7f9a68197a50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800d8d0 tx=0x7f9a5800dbe0 comp rx=0 tx=0).stop 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.024+0000 7f9a6d522700 1 -- 172.21.15.102:0/3653368326 shutdown_connections 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.024+0000 7f9a6d522700 1 --2- 172.21.15.102:0/3653368326 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9a5406c4e0 0x7f9a5406e9a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.024+0000 7f9a6d522700 1 --2- 172.21.15.102:0/3653368326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9a68100180 0x7f9a68197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.024+0000 7f9a6d522700 1 --2- 172.21.15.102:0/3653368326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a68104a50 0x7f9a68197a50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.024+0000 7f9a6d522700 1 -- 172.21.15.102:0/3653368326 >> 172.21.15.102:0/3653368326 conn(0x7f9a68075e30 msgr2=0x7f9a68103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.024+0000 7f9a6d522700 1 -- 172.21.15.102:0/3653368326 shutdown_connections 2024-05-13T22:44:31.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:31.024+0000 7f9a6d522700 1 -- 172.21.15.102:0/3653368326 wait complete. 2024-05-13T22:44:31.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:31 smithi102 ceph-mon[28599]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:31.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:31 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3653368326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-13T22:44:31.984 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-05-13T22:44:31.985 DEBUG:teuthology.parallel:result is None 2024-05-13T22:44:32.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:32 smithi102 ceph-mon[28599]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:32.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:32 smithi130 ceph-mon[36591]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:32.453 INFO:teuthology.orchestra.run.smithi102.stdout:150323855375 2024-05-13T22:44:32.454 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-13T22:44:32.762 INFO:teuthology.orchestra.run.smithi102.stdout:115964117013 2024-05-13T22:44:32.763 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-13T22:44:32.776 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:34.042 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:35.039 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:34 smithi102 ceph-mon[28599]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:35.039 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:34 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2477608168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-13T22:44:35.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:34 smithi130 ceph-mon[36591]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:35.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:34 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2477608168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-13T22:44:35.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.502+0000 7fbaeb80a700 1 -- 172.21.15.102:0/1721585196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbae4105a40 msgr2=0x7fbae4105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:35.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.502+0000 7fbaeb80a700 1 --2- 172.21.15.102:0/1721585196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbae4105a40 0x7fbae4105e20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009910 tx=0x7fbad4009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:35.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.502+0000 7fbaeb80a700 1 -- 172.21.15.102:0/1721585196 shutdown_connections 2024-05-13T22:44:35.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.502+0000 7fbaeb80a700 1 --2- 172.21.15.102:0/1721585196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbae40ffa40 0x7fbae40ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:35.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.502+0000 7fbaeb80a700 1 --2- 172.21.15.102:0/1721585196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbae4105a40 0x7fbae4105e20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:35.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.502+0000 7fbaeb80a700 1 -- 172.21.15.102:0/1721585196 >> 172.21.15.102:0/1721585196 conn(0x7fbae40fb550 msgr2=0x7fbae40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:35.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.502+0000 7fbaeb80a700 1 -- 172.21.15.102:0/1721585196 shutdown_connections 2024-05-13T22:44:35.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.502+0000 7fbaeb80a700 1 -- 172.21.15.102:0/1721585196 wait complete. 2024-05-13T22:44:35.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbaeb80a700 1 Processor -- start 2024-05-13T22:44:35.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbaeb80a700 1 -- start start 2024-05-13T22:44:35.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbaeb80a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbae40ffa40 0x7fbae419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:35.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbaeb80a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbae4105a40 0x7fbae419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:35.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbaeb80a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae419d1d0 con 0x7fbae4105a40 2024-05-13T22:44:35.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbaeb80a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4196680 con 0x7fbae40ffa40 2024-05-13T22:44:35.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbae8da5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbae4105a40 0x7fbae419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:35.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbae8da5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbae4105a40 0x7fbae419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34962/0 (socket says 172.21.15.102:34962) 2024-05-13T22:44:35.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbae8da5700 1 -- 172.21.15.102:0/2477608168 learned_addr learned my addr 172.21.15.102:0/2477608168 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:35.216 INFO:teuthology.orchestra.run.smithi102.stdout:60129542176 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbae8da5700 1 -- 172.21.15.102:0/2477608168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbae40ffa40 msgr2=0x7fbae419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbae95a6700 1 --2- 172.21.15.102:0/2477608168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbae40ffa40 0x7fbae419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbae8da5700 1 --2- 172.21.15.102:0/2477608168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbae40ffa40 0x7fbae419c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.503+0000 7fbae8da5700 1 -- 172.21.15.102:0/2477608168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad40095f0 con 0x7fbae4105a40 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.504+0000 7fbae8da5700 1 --2- 172.21.15.102:0/2477608168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbae4105a40 0x7fbae419cb40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbae000b4b0 tx=0x7fbae000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.504+0000 7fbada7fc700 1 -- 172.21.15.102:0/2477608168 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbae0011800 con 0x7fbae4105a40 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.504+0000 7fbaeb80a700 1 -- 172.21.15.102:0/2477608168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae4196960 con 0x7fbae4105a40 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.504+0000 7fbada7fc700 1 -- 172.21.15.102:0/2477608168 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbae0011e10 con 0x7fbae4105a40 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.504+0000 7fbada7fc700 1 -- 172.21.15.102:0/2477608168 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbae000f2e0 con 0x7fbae4105a40 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.504+0000 7fbaeb80a700 1 -- 172.21.15.102:0/2477608168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae4196eb0 con 0x7fbae4105a40 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.505+0000 7fbaeb80a700 1 -- 172.21.15.102:0/2477608168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae4109050 con 0x7fbae4105a40 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.505+0000 7fbada7fc700 1 -- 172.21.15.102:0/2477608168 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbae0010380 con 0x7fbae4105a40 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.506+0000 7fbada7fc700 1 --2- 172.21.15.102:0/2477608168 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fbad006c4e0 0x7fbad006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:35.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.506+0000 7fbada7fc700 1 -- 172.21.15.102:0/2477608168 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbae008b130 con 0x7fbae4105a40 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.506+0000 7fbae95a6700 1 --2- 172.21.15.102:0/2477608168 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fbad006c4e0 0x7fbad006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.506+0000 7fbae95a6700 1 --2- 172.21.15.102:0/2477608168 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fbad006c4e0 0x7fbad006e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b5c0 tx=0x7fbad4005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.510+0000 7fbada7fc700 1 -- 172.21.15.102:0/2477608168 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbae00596e0 con 0x7fbae4105a40 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.708+0000 7fbaeb80a700 1 -- 172.21.15.102:0/2477608168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbae4197130 con 0x7fbae4105a40 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.709+0000 7fbada7fc700 1 -- 172.21.15.102:0/2477608168 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbae0059270 con 0x7fbae4105a40 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.711+0000 7fbaeb80a700 1 -- 172.21.15.102:0/2477608168 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fbad006c4e0 msgr2=0x7fbad006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.711+0000 7fbaeb80a700 1 --2- 172.21.15.102:0/2477608168 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fbad006c4e0 0x7fbad006e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b5c0 tx=0x7fbad4005960 comp rx=0 tx=0).stop 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.711+0000 7fbaeb80a700 1 -- 172.21.15.102:0/2477608168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbae4105a40 msgr2=0x7fbae419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.711+0000 7fbaeb80a700 1 --2- 172.21.15.102:0/2477608168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbae4105a40 0x7fbae419cb40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbae000b4b0 tx=0x7fbae000b870 comp rx=0 tx=0).stop 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.711+0000 7fbaeb80a700 1 -- 172.21.15.102:0/2477608168 shutdown_connections 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.711+0000 7fbaeb80a700 1 --2- 172.21.15.102:0/2477608168 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fbad006c4e0 0x7fbad006e9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:35.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.711+0000 7fbaeb80a700 1 --2- 172.21.15.102:0/2477608168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbae40ffa40 0x7fbae419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:35.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.711+0000 7fbaeb80a700 1 --2- 172.21.15.102:0/2477608168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbae4105a40 0x7fbae419cb40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:35.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.711+0000 7fbaeb80a700 1 -- 172.21.15.102:0/2477608168 >> 172.21.15.102:0/2477608168 conn(0x7fbae40fb550 msgr2=0x7fbae4069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:35.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.711+0000 7fbaeb80a700 1 -- 172.21.15.102:0/2477608168 shutdown_connections 2024-05-13T22:44:35.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:34.711+0000 7fbaeb80a700 1 -- 172.21.15.102:0/2477608168 wait complete. 2024-05-13T22:44:35.817 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542176 for osd.1 2024-05-13T22:44:35.818 DEBUG:teuthology.parallel:result is None 2024-05-13T22:44:35.829 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:36.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:36 smithi130 ceph-mon[36591]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.278+0000 7f15b6fbc700 1 -- 172.21.15.102:0/4226623117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b0100dd0 msgr2=0x7f15b01011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.278+0000 7f15b6fbc700 1 --2- 172.21.15.102:0/4226623117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b0100dd0 0x7f15b01011b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009910 tx=0x7f15a4009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.278+0000 7f15b6fbc700 1 -- 172.21.15.102:0/4226623117 shutdown_connections 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.278+0000 7f15b6fbc700 1 --2- 172.21.15.102:0/4226623117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15b0101780 0x7f15b01057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.278+0000 7f15b6fbc700 1 --2- 172.21.15.102:0/4226623117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b0100dd0 0x7f15b01011b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.278+0000 7f15b6fbc700 1 -- 172.21.15.102:0/4226623117 >> 172.21.15.102:0/4226623117 conn(0x7f15b00fc640 msgr2=0x7f15b00fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.278+0000 7f15b6fbc700 1 -- 172.21.15.102:0/4226623117 shutdown_connections 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.279+0000 7f15b6fbc700 1 -- 172.21.15.102:0/4226623117 wait complete. 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.279+0000 7f15b6fbc700 1 Processor -- start 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.279+0000 7f15b6fbc700 1 -- start start 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.279+0000 7f15b6fbc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b0100dd0 0x7f15b0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.279+0000 7f15b6fbc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15b0101780 0x7f15b00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.279+0000 7f15b6fbc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0073b10 con 0x7f15b0100dd0 2024-05-13T22:44:36.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.279+0000 7f15b6fbc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0073c50 con 0x7f15b0101780 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.279+0000 7f15b4d58700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b0100dd0 0x7f15b0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.279+0000 7f15b4d58700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b0100dd0 0x7f15b0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34980/0 (socket says 172.21.15.102:34980) 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.279+0000 7f15b4d58700 1 -- 172.21.15.102:0/2304073967 learned_addr learned my addr 172.21.15.102:0/2304073967 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.280+0000 7f15affff700 1 --2- 172.21.15.102:0/2304073967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15b0101780 0x7f15b00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.280+0000 7f15b4d58700 1 -- 172.21.15.102:0/2304073967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15b0101780 msgr2=0x7f15b00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.280+0000 7f15b4d58700 1 --2- 172.21.15.102:0/2304073967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15b0101780 0x7f15b00735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.280+0000 7f15b4d58700 1 -- 172.21.15.102:0/2304073967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a40095f0 con 0x7f15b0100dd0 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.280+0000 7f15b4d58700 1 --2- 172.21.15.102:0/2304073967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b0100dd0 0x7f15b0073090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15a4006010 tx=0x7f15a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.280+0000 7f15adffb700 1 -- 172.21.15.102:0/2304073967 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15a401d070 con 0x7f15b0100dd0 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.280+0000 7f15adffb700 1 -- 172.21.15.102:0/2304073967 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f15a400f460 con 0x7f15b0100dd0 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.280+0000 7f15b6fbc700 1 -- 172.21.15.102:0/2304073967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b0073ed0 con 0x7f15b0100dd0 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.280+0000 7f15adffb700 1 -- 172.21.15.102:0/2304073967 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15a4021640 con 0x7f15b0100dd0 2024-05-13T22:44:36.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.280+0000 7f15b6fbc700 1 -- 172.21.15.102:0/2304073967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b01a2ce0 con 0x7f15b0100dd0 2024-05-13T22:44:36.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.281+0000 7f15b6fbc700 1 -- 172.21.15.102:0/2304073967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b0109170 con 0x7f15b0100dd0 2024-05-13T22:44:36.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.282+0000 7f15adffb700 1 -- 172.21.15.102:0/2304073967 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f15a400faf0 con 0x7f15b0100dd0 2024-05-13T22:44:36.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.282+0000 7f15adffb700 1 --2- 172.21.15.102:0/2304073967 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f15a006c4e0 0x7f15a006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:36.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.282+0000 7f15adffb700 1 -- 172.21.15.102:0/2304073967 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f15a408c830 con 0x7f15b0100dd0 2024-05-13T22:44:36.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.282+0000 7f15affff700 1 --2- 172.21.15.102:0/2304073967 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f15a006c4e0 0x7f15a006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:36.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.283+0000 7f15affff700 1 --2- 172.21.15.102:0/2304073967 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f15a006c4e0 0x7f15a006e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f159c006fd0 tx=0x7f159c008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:36.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.287+0000 7f15adffb700 1 -- 172.21.15.102:0/2304073967 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f15a4057350 con 0x7f15b0100dd0 2024-05-13T22:44:36.276 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:36.277 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:36.381 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:36 smithi102 ceph-mon[28599]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:36.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.485+0000 7f15b6fbc700 1 -- 172.21.15.102:0/2304073967 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f15b004ea90 con 0x7f15b0100dd0 2024-05-13T22:44:36.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.485+0000 7f15adffb700 1 -- 172.21.15.102:0/2304073967 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15a405a970 con 0x7f15b0100dd0 2024-05-13T22:44:36.464 INFO:teuthology.orchestra.run.smithi102.stdout:81604378650 2024-05-13T22:44:36.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.487+0000 7f15b6fbc700 1 -- 172.21.15.102:0/2304073967 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f15a006c4e0 msgr2=0x7f15a006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:36.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.487+0000 7f15b6fbc700 1 --2- 172.21.15.102:0/2304073967 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f15a006c4e0 0x7f15a006e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f159c006fd0 tx=0x7f159c008040 comp rx=0 tx=0).stop 2024-05-13T22:44:36.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.487+0000 7f15b6fbc700 1 -- 172.21.15.102:0/2304073967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b0100dd0 msgr2=0x7f15b0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:36.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.487+0000 7f15b6fbc700 1 --2- 172.21.15.102:0/2304073967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b0100dd0 0x7f15b0073090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15a4006010 tx=0x7f15a400be20 comp rx=0 tx=0).stop 2024-05-13T22:44:36.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.488+0000 7f15b6fbc700 1 -- 172.21.15.102:0/2304073967 shutdown_connections 2024-05-13T22:44:36.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.488+0000 7f15b6fbc700 1 --2- 172.21.15.102:0/2304073967 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f15a006c4e0 0x7f15a006e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:36.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.488+0000 7f15b6fbc700 1 --2- 172.21.15.102:0/2304073967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15b0100dd0 0x7f15b0073090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:36.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.488+0000 7f15b6fbc700 1 --2- 172.21.15.102:0/2304073967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f15b0101780 0x7f15b00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:36.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.488+0000 7f15b6fbc700 1 -- 172.21.15.102:0/2304073967 >> 172.21.15.102:0/2304073967 conn(0x7f15b00fc640 msgr2=0x7f15b00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:36.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.488+0000 7f15b6fbc700 1 -- 172.21.15.102:0/2304073967 shutdown_connections 2024-05-13T22:44:36.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:36.488+0000 7f15b6fbc700 1 -- 172.21.15.102:0/2304073967 wait complete. 2024-05-13T22:44:37.057 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378650 for osd.2 2024-05-13T22:44:37.057 DEBUG:teuthology.parallel:result is None 2024-05-13T22:44:37.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:37 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2304073967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-13T22:44:37.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:37 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2304073967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-13T22:44:38.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:38 smithi102 ceph-mon[28599]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:38.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:38 smithi130 ceph-mon[36591]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:39.134 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:44:39.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.183+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3775120385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa494100dc0 msgr2=0x7fa4941011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:39.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.183+0000 7fa49c0b6700 1 --2- 172.21.15.102:0/3775120385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa494100dc0 0x7fa4941011a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa488009910 tx=0x7fa488009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:39.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.183+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3775120385 shutdown_connections 2024-05-13T22:44:39.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.183+0000 7fa49c0b6700 1 --2- 172.21.15.102:0/3775120385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa494101770 0x7fa4941057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:39.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.183+0000 7fa49c0b6700 1 --2- 172.21.15.102:0/3775120385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa494100dc0 0x7fa4941011a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:39.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.183+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3775120385 >> 172.21.15.102:0/3775120385 conn(0x7fa4940fc650 msgr2=0x7fa4940fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:39.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.183+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3775120385 shutdown_connections 2024-05-13T22:44:39.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.183+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3775120385 wait complete. 2024-05-13T22:44:39.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa49c0b6700 1 Processor -- start 2024-05-13T22:44:39.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa49c0b6700 1 -- start start 2024-05-13T22:44:39.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa49c0b6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa494100dc0 0x7fa494073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:39.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa49c0b6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa494101770 0x7fa4940735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:39.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa49c0b6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa494073b10 con 0x7fa494101770 2024-05-13T22:44:39.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa49c0b6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa494073c50 con 0x7fa494100dc0 2024-05-13T22:44:39.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa499651700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa494101770 0x7fa4940735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:39.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa499651700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa494101770 0x7fa4940735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53724/0 (socket says 172.21.15.102:53724) 2024-05-13T22:44:39.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa499651700 1 -- 172.21.15.102:0/3888108478 learned_addr learned my addr 172.21.15.102:0/3888108478 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:39.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa499e52700 1 --2- 172.21.15.102:0/3888108478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa494100dc0 0x7fa494073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:39.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa499651700 1 -- 172.21.15.102:0/3888108478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa494100dc0 msgr2=0x7fa494073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:39.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa499651700 1 --2- 172.21.15.102:0/3888108478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa494100dc0 0x7fa494073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:39.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.184+0000 7fa499651700 1 -- 172.21.15.102:0/3888108478 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4880095f0 con 0x7fa494101770 2024-05-13T22:44:39.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.185+0000 7fa499651700 1 --2- 172.21.15.102:0/3888108478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa494101770 0x7fa4940735d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa49000d8d0 tx=0x7fa49000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:39.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.185+0000 7fa486ffd700 1 -- 172.21.15.102:0/3888108478 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4900041f0 con 0x7fa494101770 2024-05-13T22:44:39.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.185+0000 7fa486ffd700 1 -- 172.21.15.102:0/3888108478 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa490007620 con 0x7fa494101770 2024-05-13T22:44:39.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.185+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3888108478 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4941a2940 con 0x7fa494101770 2024-05-13T22:44:39.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.185+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3888108478 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4941a2d50 con 0x7fa494101770 2024-05-13T22:44:39.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.185+0000 7fa486ffd700 1 -- 172.21.15.102:0/3888108478 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4900056b0 con 0x7fa494101770 2024-05-13T22:44:39.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.186+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3888108478 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa494109160 con 0x7fa494101770 2024-05-13T22:44:39.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.186+0000 7fa486ffd700 1 -- 172.21.15.102:0/3888108478 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa490004750 con 0x7fa494101770 2024-05-13T22:44:39.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.187+0000 7fa486ffd700 1 --2- 172.21.15.102:0/3888108478 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa48006c4e0 0x7fa48006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:39.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.187+0000 7fa486ffd700 1 -- 172.21.15.102:0/3888108478 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa49008b3d0 con 0x7fa494101770 2024-05-13T22:44:39.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.187+0000 7fa499e52700 1 --2- 172.21.15.102:0/3888108478 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa48006c4e0 0x7fa48006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:39.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.187+0000 7fa499e52700 1 --2- 172.21.15.102:0/3888108478 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa48006c4e0 0x7fa48006e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa488006010 tx=0x7fa48801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:39.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.191+0000 7fa486ffd700 1 -- 172.21.15.102:0/3888108478 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa490055ef0 con 0x7fa494101770 2024-05-13T22:44:39.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.381+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3888108478 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa49404ea90 con 0x7fa494101770 2024-05-13T22:44:39.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.381+0000 7fa486ffd700 1 -- 172.21.15.102:0/3888108478 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa490059510 con 0x7fa494101770 2024-05-13T22:44:39.360 INFO:teuthology.orchestra.run.smithi102.stdout:150323855376 2024-05-13T22:44:39.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.383+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3888108478 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa48006c4e0 msgr2=0x7fa48006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:39.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.383+0000 7fa49c0b6700 1 --2- 172.21.15.102:0/3888108478 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa48006c4e0 0x7fa48006e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa488006010 tx=0x7fa48801a040 comp rx=0 tx=0).stop 2024-05-13T22:44:39.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.383+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3888108478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa494101770 msgr2=0x7fa4940735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:39.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.383+0000 7fa49c0b6700 1 --2- 172.21.15.102:0/3888108478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa494101770 0x7fa4940735d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa49000d8d0 tx=0x7fa49000dbe0 comp rx=0 tx=0).stop 2024-05-13T22:44:39.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.383+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3888108478 shutdown_connections 2024-05-13T22:44:39.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.383+0000 7fa49c0b6700 1 --2- 172.21.15.102:0/3888108478 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa48006c4e0 0x7fa48006e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:39.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.383+0000 7fa49c0b6700 1 --2- 172.21.15.102:0/3888108478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa494100dc0 0x7fa494073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:39.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.383+0000 7fa49c0b6700 1 --2- 172.21.15.102:0/3888108478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa494101770 0x7fa4940735d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:39.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.383+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3888108478 >> 172.21.15.102:0/3888108478 conn(0x7fa4940fc650 msgr2=0x7fa4940fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:39.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.384+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3888108478 shutdown_connections 2024-05-13T22:44:39.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:39.384+0000 7fa49c0b6700 1 -- 172.21.15.102:0/3888108478 wait complete. 2024-05-13T22:44:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:44:40.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:40 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3888108478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-13T22:44:40.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:40 smithi130 ceph-mon[36591]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:40.403 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:40 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3888108478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-13T22:44:40.403 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:40 smithi102 ceph-mon[28599]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:40.971 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855375 got 150323855376 for osd.4 2024-05-13T22:44:40.971 DEBUG:teuthology.parallel:result is None 2024-05-13T22:44:41.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.182+0000 7fc122397700 1 -- 172.21.15.102:0/1546231933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11c101c70 msgr2=0x7fc11c102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:41.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.182+0000 7fc122397700 1 --2- 172.21.15.102:0/1546231933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11c101c70 0x7fc11c102050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009910 tx=0x7fc10c009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:41.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.182+0000 7fc122397700 1 -- 172.21.15.102:0/1546231933 shutdown_connections 2024-05-13T22:44:41.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.182+0000 7fc122397700 1 --2- 172.21.15.102:0/1546231933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc11c102620 0x7fc11c10ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:41.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.182+0000 7fc122397700 1 --2- 172.21.15.102:0/1546231933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11c101c70 0x7fc11c102050 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:41.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.182+0000 7fc122397700 1 -- 172.21.15.102:0/1546231933 >> 172.21.15.102:0/1546231933 conn(0x7fc11c0fb570 msgr2=0x7fc11c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:41.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.182+0000 7fc122397700 1 -- 172.21.15.102:0/1546231933 shutdown_connections 2024-05-13T22:44:41.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.182+0000 7fc122397700 1 -- 172.21.15.102:0/1546231933 wait complete. 2024-05-13T22:44:41.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.183+0000 7fc122397700 1 Processor -- start 2024-05-13T22:44:41.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.183+0000 7fc122397700 1 -- start start 2024-05-13T22:44:41.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.183+0000 7fc122397700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11c101c70 0x7fc11c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:41.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.183+0000 7fc122397700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc11c102620 0x7fc11c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:41.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.183+0000 7fc122397700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19d120 con 0x7fc11c101c70 2024-05-13T22:44:41.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.183+0000 7fc122397700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c196620 con 0x7fc11c102620 2024-05-13T22:44:41.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.183+0000 7fc11bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11c101c70 0x7fc11c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:41.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.183+0000 7fc11bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11c101c70 0x7fc11c19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53740/0 (socket says 172.21.15.102:53740) 2024-05-13T22:44:41.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.183+0000 7fc11bfff700 1 -- 172.21.15.102:0/2886732391 learned_addr learned my addr 172.21.15.102:0/2886732391 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:41.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.184+0000 7fc11b7fe700 1 --2- 172.21.15.102:0/2886732391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc11c102620 0x7fc11c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:41.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.184+0000 7fc11bfff700 1 -- 172.21.15.102:0/2886732391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc11c102620 msgr2=0x7fc11c19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:41.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.184+0000 7fc11bfff700 1 --2- 172.21.15.102:0/2886732391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc11c102620 0x7fc11c19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:41.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.184+0000 7fc11bfff700 1 -- 172.21.15.102:0/2886732391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c0095f0 con 0x7fc11c101c70 2024-05-13T22:44:41.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.184+0000 7fc11bfff700 1 --2- 172.21.15.102:0/2886732391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11c101c70 0x7fc11c19c550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc10c006010 tx=0x7fc10c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:41.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.184+0000 7fc1197fa700 1 -- 172.21.15.102:0/2886732391 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc10c01d070 con 0x7fc11c101c70 2024-05-13T22:44:41.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.184+0000 7fc1197fa700 1 -- 172.21.15.102:0/2886732391 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc10c00f460 con 0x7fc11c101c70 2024-05-13T22:44:41.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.184+0000 7fc122397700 1 -- 172.21.15.102:0/2886732391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c1968a0 con 0x7fc11c101c70 2024-05-13T22:44:41.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.184+0000 7fc1197fa700 1 -- 172.21.15.102:0/2886732391 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc10c021640 con 0x7fc11c101c70 2024-05-13T22:44:41.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.184+0000 7fc122397700 1 -- 172.21.15.102:0/2886732391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c196d90 con 0x7fc11c101c70 2024-05-13T22:44:41.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.185+0000 7fc122397700 1 -- 172.21.15.102:0/2886732391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c108210 con 0x7fc11c101c70 2024-05-13T22:44:41.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.186+0000 7fc1197fa700 1 -- 172.21.15.102:0/2886732391 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc10c00faf0 con 0x7fc11c101c70 2024-05-13T22:44:41.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.186+0000 7fc1197fa700 1 --2- 172.21.15.102:0/2886732391 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc10406c4e0 0x7fc10406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:41.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.186+0000 7fc1197fa700 1 -- 172.21.15.102:0/2886732391 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc10c08c870 con 0x7fc11c101c70 2024-05-13T22:44:41.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.186+0000 7fc11b7fe700 1 --2- 172.21.15.102:0/2886732391 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc10406c4e0 0x7fc10406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:41.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.187+0000 7fc11b7fe700 1 --2- 172.21.15.102:0/2886732391 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc10406c4e0 0x7fc10406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc110006fd0 tx=0x7fc110008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:41.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.191+0000 7fc1197fa700 1 -- 172.21.15.102:0/2886732391 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc10c05ae20 con 0x7fc11c101c70 2024-05-13T22:44:41.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.382+0000 7fc122397700 1 -- 172.21.15.102:0/2886732391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc11c197070 con 0x7fc11c101c70 2024-05-13T22:44:41.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.383+0000 7fc1197fa700 1 -- 172.21.15.102:0/2886732391 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc10c05a9b0 con 0x7fc11c101c70 2024-05-13T22:44:41.361 INFO:teuthology.orchestra.run.smithi102.stdout:115964117015 2024-05-13T22:44:41.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.385+0000 7fc122397700 1 -- 172.21.15.102:0/2886732391 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc10406c4e0 msgr2=0x7fc10406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:41.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.385+0000 7fc122397700 1 --2- 172.21.15.102:0/2886732391 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc10406c4e0 0x7fc10406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc110006fd0 tx=0x7fc110008040 comp rx=0 tx=0).stop 2024-05-13T22:44:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.385+0000 7fc122397700 1 -- 172.21.15.102:0/2886732391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11c101c70 msgr2=0x7fc11c19c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.385+0000 7fc122397700 1 --2- 172.21.15.102:0/2886732391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11c101c70 0x7fc11c19c550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc10c006010 tx=0x7fc10c00be20 comp rx=0 tx=0).stop 2024-05-13T22:44:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.385+0000 7fc122397700 1 -- 172.21.15.102:0/2886732391 shutdown_connections 2024-05-13T22:44:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.385+0000 7fc122397700 1 --2- 172.21.15.102:0/2886732391 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc10406c4e0 0x7fc10406e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.385+0000 7fc122397700 1 --2- 172.21.15.102:0/2886732391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc11c101c70 0x7fc11c19c550 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.385+0000 7fc122397700 1 --2- 172.21.15.102:0/2886732391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc11c102620 0x7fc11c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.385+0000 7fc122397700 1 -- 172.21.15.102:0/2886732391 >> 172.21.15.102:0/2886732391 conn(0x7fc11c0fb570 msgr2=0x7fc11c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.385+0000 7fc122397700 1 -- 172.21.15.102:0/2886732391 shutdown_connections 2024-05-13T22:44:41.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:41.385+0000 7fc122397700 1 -- 172.21.15.102:0/2886732391 wait complete. 2024-05-13T22:44:41.670 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:41 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2886732391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-13T22:44:41.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:41 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2886732391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-13T22:44:42.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.386+0000 7f59205af700 1 -- 172.21.15.102:0/655227602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5918100dc0 msgr2=0x7f59181011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:42.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.386+0000 7f59205af700 1 --2- 172.21.15.102:0/655227602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5918100dc0 0x7f59181011a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5914009910 tx=0x7f5914009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:42.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 -- 172.21.15.102:0/655227602 shutdown_connections 2024-05-13T22:44:42.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 --2- 172.21.15.102:0/655227602 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5918101770 0x7f59181057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:42.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 --2- 172.21.15.102:0/655227602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5918100dc0 0x7f59181011a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:42.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 -- 172.21.15.102:0/655227602 >> 172.21.15.102:0/655227602 conn(0x7f59180fc650 msgr2=0x7f59180fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:42.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 -- 172.21.15.102:0/655227602 shutdown_connections 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 -- 172.21.15.102:0/655227602 wait complete. 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 Processor -- start 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 -- start start 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5918100dc0 0x7f591819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5918101770 0x7f591819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591819d110 con 0x7f5918100dc0 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f59205af700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59181965c0 con 0x7f5918101770 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.387+0000 7f591e34b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5918100dc0 0x7f591819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f591e34b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5918100dc0 0x7f591819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53756/0 (socket says 172.21.15.102:53756) 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f591e34b700 1 -- 172.21.15.102:0/1212648968 learned_addr learned my addr 172.21.15.102:0/1212648968 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f591e34b700 1 -- 172.21.15.102:0/1212648968 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5918101770 msgr2=0x7f591819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f591db4a700 1 --2- 172.21.15.102:0/1212648968 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5918101770 0x7f591819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:42.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f591e34b700 1 --2- 172.21.15.102:0/1212648968 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5918101770 0x7f591819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:42.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f591e34b700 1 -- 172.21.15.102:0/1212648968 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59140095f0 con 0x7f5918100dc0 2024-05-13T22:44:42.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f591e34b700 1 --2- 172.21.15.102:0/1212648968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5918100dc0 0x7f591819c540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5914006010 tx=0x7f591400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:42.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f590f7fe700 1 -- 172.21.15.102:0/1212648968 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f591401d070 con 0x7f5918100dc0 2024-05-13T22:44:42.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f590f7fe700 1 -- 172.21.15.102:0/1212648968 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f591400f460 con 0x7f5918100dc0 2024-05-13T22:44:42.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f59205af700 1 -- 172.21.15.102:0/1212648968 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5918196840 con 0x7f5918100dc0 2024-05-13T22:44:42.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f590f7fe700 1 -- 172.21.15.102:0/1212648968 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5914021640 con 0x7f5918100dc0 2024-05-13T22:44:42.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.388+0000 7f59205af700 1 -- 172.21.15.102:0/1212648968 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5918196d30 con 0x7f5918100dc0 2024-05-13T22:44:42.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.389+0000 7f59205af700 1 -- 172.21.15.102:0/1212648968 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5918109160 con 0x7f5918100dc0 2024-05-13T22:44:42.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.390+0000 7f590f7fe700 1 -- 172.21.15.102:0/1212648968 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f591400faf0 con 0x7f5918100dc0 2024-05-13T22:44:42.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.390+0000 7f590f7fe700 1 --2- 172.21.15.102:0/1212648968 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590406c4e0 0x7f590406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:42.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.390+0000 7f590f7fe700 1 -- 172.21.15.102:0/1212648968 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f591408c7d0 con 0x7f5918100dc0 2024-05-13T22:44:42.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.390+0000 7f591db4a700 1 --2- 172.21.15.102:0/1212648968 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590406c4e0 0x7f590406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:42.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.390+0000 7f591db4a700 1 --2- 172.21.15.102:0/1212648968 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590406c4e0 0x7f590406e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f59181059b0 tx=0x7f5908005cf0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:42.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.394+0000 7f590f7fe700 1 -- 172.21.15.102:0/1212648968 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f591405ad80 con 0x7f5918100dc0 2024-05-13T22:44:42.513 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:42 smithi102 ceph-mon[28599]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:42.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.589+0000 7f59205af700 1 -- 172.21.15.102:0/1212648968 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f591804ea90 con 0x7f5918100dc0 2024-05-13T22:44:42.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.590+0000 7f590f7fe700 1 -- 172.21.15.102:0/1212648968 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f591405a910 con 0x7f5918100dc0 2024-05-13T22:44:42.569 INFO:teuthology.orchestra.run.smithi102.stdout:171798691852 2024-05-13T22:44:42.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.592+0000 7f59205af700 1 -- 172.21.15.102:0/1212648968 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590406c4e0 msgr2=0x7f590406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:42.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.592+0000 7f59205af700 1 --2- 172.21.15.102:0/1212648968 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590406c4e0 0x7f590406e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f59181059b0 tx=0x7f5908005cf0 comp rx=0 tx=0).stop 2024-05-13T22:44:42.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.592+0000 7f59205af700 1 -- 172.21.15.102:0/1212648968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5918100dc0 msgr2=0x7f591819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:42.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.592+0000 7f59205af700 1 --2- 172.21.15.102:0/1212648968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5918100dc0 0x7f591819c540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5914006010 tx=0x7f591400be20 comp rx=0 tx=0).stop 2024-05-13T22:44:42.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.592+0000 7f59205af700 1 -- 172.21.15.102:0/1212648968 shutdown_connections 2024-05-13T22:44:42.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.592+0000 7f59205af700 1 --2- 172.21.15.102:0/1212648968 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f590406c4e0 0x7f590406e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:42.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.592+0000 7f59205af700 1 --2- 172.21.15.102:0/1212648968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5918100dc0 0x7f591819c540 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:42.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.592+0000 7f59205af700 1 --2- 172.21.15.102:0/1212648968 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5918101770 0x7f591819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:42.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.592+0000 7f59205af700 1 -- 172.21.15.102:0/1212648968 >> 172.21.15.102:0/1212648968 conn(0x7f59180fc650 msgr2=0x7f59180fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:42.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.592+0000 7f59205af700 1 -- 172.21.15.102:0/1212648968 shutdown_connections 2024-05-13T22:44:42.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:42.592+0000 7f59205af700 1 -- 172.21.15.102:0/1212648968 wait complete. 2024-05-13T22:44:42.820 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117013 got 115964117015 for osd.3 2024-05-13T22:44:42.820 DEBUG:teuthology.parallel:result is None 2024-05-13T22:44:42.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:42 smithi130 ceph-mon[36591]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:43.374 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691848 got 171798691852 for osd.5 2024-05-13T22:44:43.374 DEBUG:teuthology.parallel:result is None 2024-05-13T22:44:43.374 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-13T22:44:43.374 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-13T22:44:43.580 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:43.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:43 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1212648968' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-13T22:44:43.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:43 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1212648968' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-13T22:44:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:44 smithi102 ceph-mon[28599]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:44.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:44 smithi130 ceph-mon[36591]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:45.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.203+0000 7f0eaf65c700 1 -- 172.21.15.102:0/2441721844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ea8101750 msgr2=0x7f0ea81057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:45.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.203+0000 7f0eaf65c700 1 --2- 172.21.15.102:0/2441721844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ea8101750 0x7f0ea81057a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009910 tx=0x7f0ea4009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.203+0000 7f0eaf65c700 1 -- 172.21.15.102:0/2441721844 shutdown_connections 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.203+0000 7f0eaf65c700 1 --2- 172.21.15.102:0/2441721844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ea8101750 0x7f0ea81057a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.203+0000 7f0eaf65c700 1 --2- 172.21.15.102:0/2441721844 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ea8100da0 0x7f0ea8101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.203+0000 7f0eaf65c700 1 -- 172.21.15.102:0/2441721844 >> 172.21.15.102:0/2441721844 conn(0x7f0ea80fc630 msgr2=0x7f0ea80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.203+0000 7f0eaf65c700 1 -- 172.21.15.102:0/2441721844 shutdown_connections 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.203+0000 7f0eaf65c700 1 -- 172.21.15.102:0/2441721844 wait complete. 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.204+0000 7f0eaf65c700 1 Processor -- start 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.204+0000 7f0eaf65c700 1 -- start start 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.204+0000 7f0eaf65c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ea8100da0 0x7f0ea8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.204+0000 7f0eaf65c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ea8101750 0x7f0ea80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.204+0000 7f0eaf65c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea8073b10 con 0x7f0ea8101750 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.204+0000 7f0eaf65c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea8073c50 con 0x7f0ea8100da0 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.204+0000 7f0eacbf7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ea8101750 0x7f0ea80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:45.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.204+0000 7f0eacbf7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ea8101750 0x7f0ea80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53784/0 (socket says 172.21.15.102:53784) 2024-05-13T22:44:45.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.204+0000 7f0eacbf7700 1 -- 172.21.15.102:0/344078848 learned_addr learned my addr 172.21.15.102:0/344078848 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:45.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.205+0000 7f0eacbf7700 1 -- 172.21.15.102:0/344078848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ea8100da0 msgr2=0x7f0ea8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:45.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.205+0000 7f0eacbf7700 1 --2- 172.21.15.102:0/344078848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ea8100da0 0x7f0ea8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:45.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.205+0000 7f0eacbf7700 1 -- 172.21.15.102:0/344078848 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea40095f0 con 0x7f0ea8101750 2024-05-13T22:44:45.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.205+0000 7f0eacbf7700 1 --2- 172.21.15.102:0/344078848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ea8101750 0x7f0ea80735d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40092e0 tx=0x7f0ea400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:45.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.205+0000 7f0e9e7fc700 1 -- 172.21.15.102:0/344078848 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ea401d070 con 0x7f0ea8101750 2024-05-13T22:44:45.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.205+0000 7f0e9e7fc700 1 -- 172.21.15.102:0/344078848 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ea4022470 con 0x7f0ea8101750 2024-05-13T22:44:45.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.205+0000 7f0eaf65c700 1 -- 172.21.15.102:0/344078848 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea8073ed0 con 0x7f0ea8101750 2024-05-13T22:44:45.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.205+0000 7f0e9e7fc700 1 -- 172.21.15.102:0/344078848 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ea400f690 con 0x7f0ea8101750 2024-05-13T22:44:45.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.205+0000 7f0eaf65c700 1 -- 172.21.15.102:0/344078848 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea81a2be0 con 0x7f0ea8101750 2024-05-13T22:44:45.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.206+0000 7f0eaf65c700 1 -- 172.21.15.102:0/344078848 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea81091a0 con 0x7f0ea8101750 2024-05-13T22:44:45.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.206+0000 7f0e9e7fc700 1 -- 172.21.15.102:0/344078848 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0ea40225e0 con 0x7f0ea8101750 2024-05-13T22:44:45.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.207+0000 7f0e9e7fc700 1 --2- 172.21.15.102:0/344078848 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f0e9406c4e0 0x7f0e9406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:45.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.207+0000 7f0e9e7fc700 1 -- 172.21.15.102:0/344078848 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0ea408c990 con 0x7f0ea8101750 2024-05-13T22:44:45.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.207+0000 7f0ead3f8700 1 --2- 172.21.15.102:0/344078848 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f0e9406c4e0 0x7f0e9406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:45.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.207+0000 7f0ead3f8700 1 --2- 172.21.15.102:0/344078848 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f0e9406c4e0 0x7f0e9406e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0e98005fd0 tx=0x7f0e98005d00 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:45.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.211+0000 7f0e9e7fc700 1 -- 172.21.15.102:0/344078848 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0ea40575e0 con 0x7f0ea8101750 2024-05-13T22:44:45.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.400+0000 7f0eaf65c700 1 -- 172.21.15.102:0/344078848 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ea80611d0 con 0x7f0e9406c4e0 2024-05-13T22:44:45.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.402+0000 7f0e9e7fc700 1 -- 172.21.15.102:0/344078848 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27770 (secure 0 0 0) 0x7f0ea80611d0 con 0x7f0e9406c4e0 2024-05-13T22:44:45.381 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:44:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.404+0000 7f0eaf65c700 1 -- 172.21.15.102:0/344078848 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f0e9406c4e0 msgr2=0x7f0e9406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.404+0000 7f0eaf65c700 1 --2- 172.21.15.102:0/344078848 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f0e9406c4e0 0x7f0e9406e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0e98005fd0 tx=0x7f0e98005d00 comp rx=0 tx=0).stop 2024-05-13T22:44:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.404+0000 7f0eaf65c700 1 -- 172.21.15.102:0/344078848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ea8101750 msgr2=0x7f0ea80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.404+0000 7f0eaf65c700 1 --2- 172.21.15.102:0/344078848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ea8101750 0x7f0ea80735d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40092e0 tx=0x7f0ea400be20 comp rx=0 tx=0).stop 2024-05-13T22:44:45.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.405+0000 7f0eaf65c700 1 -- 172.21.15.102:0/344078848 shutdown_connections 2024-05-13T22:44:45.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.405+0000 7f0eaf65c700 1 --2- 172.21.15.102:0/344078848 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f0e9406c4e0 0x7f0e9406e9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:45.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.405+0000 7f0eaf65c700 1 --2- 172.21.15.102:0/344078848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ea8100da0 0x7f0ea8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:45.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.405+0000 7f0eaf65c700 1 --2- 172.21.15.102:0/344078848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ea8101750 0x7f0ea80735d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:45.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.405+0000 7f0eaf65c700 1 -- 172.21.15.102:0/344078848 >> 172.21.15.102:0/344078848 conn(0x7f0ea80fc630 msgr2=0x7f0ea80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:45.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.405+0000 7f0eaf65c700 1 -- 172.21.15.102:0/344078848 shutdown_connections 2024-05-13T22:44:45.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:45.405+0000 7f0eaf65c700 1 -- 172.21.15.102:0/344078848 wait complete. 2024-05-13T22:44:45.384 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-05-13T22:44:45.998 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-05-13T22:44:43.699108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002366"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-13T22:43:54.158442+0000","last_change":"2024-05-13T22:43:28.149339+0000","last_active":"2024-05-13T22:43:54.158442+0000","last_peered":"2024-05-13T22:43:54.158442+0000","last_clean":"2024-05-13T22:43:54.158442+0000","last_became_active":"2024-05-13T22:43:28.148937+0000","last_became_peered":"2024-05-13T22:43:28.148937+0000","last_unstale":"2024-05-13T22:43:54.158442+0000","last_undegraded":"2024-05-13T22:43:54.158442+0000","last_fullsized":"2024-05-13T22:43:54.158442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-13T22:42:40.185397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-13T22:42:40.185397+0000","last_clean_scrub_stamp":"2024-05-13T22:42:40.185397+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T01:53:45.013620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 13 22:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.58799999999999997}]},{"osd":1,"last update":"Mon May 13 22:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.54500000000000004}]},{"osd":2,"last update":"Mon May 13 22:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon May 13 22:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.36599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 13 22:44:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.55700000000000005}]},{"osd":1,"last update":"Mon May 13 22:44:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.501},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.439}]},{"osd":2,"last update":"Mon May 13 22:44:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.54200000000000004}]},{"osd":4,"last update":"Mon May 13 22:44:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.371},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.45100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 13 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35099999999999998,"15min":0.35699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.498,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.38100000000000001}]},{"osd":1,"last update":"Mon May 13 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.48599999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.36799999999999999,"15min":0.373},"min":{"1min":0.24099999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51600000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.41599999999999998}]},{"osd":3,"last update":"Mon May 13 22:44:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.72399999999999998}]},{"osd":4,"last update":"Mon May 13 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon May 13 22:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.316},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.221}]},{"osd":2,"last update":"Mon May 13 22:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.254}]},{"osd":3,"last update":"Mon May 13 22:44:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.55500000000000005}]},{"osd":4,"last update":"Mon May 13 22:44:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 13 22:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.29099999999999998},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.246},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.32200000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.27300000000000002}]},{"osd":2,"last update":"Mon May 13 22:44:41 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34699999999999998,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.187,"15min":0.187},"max":{"1min":0.56100000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.378,"15min":0.38300000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.47999999999999998,"5min":0.753,"15min":0.753},"last":0.308}]},{"osd":3,"last update":"Mon May 13 22:44:03 2024","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":19.488,"5min":19.488,"15min":19.488},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55400000000000005}]},{"osd":4,"last update":"Mon May 13 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":19.451000000000001,"5min":19.451000000000001,"15min":19.451000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-13T22:44:46.000 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-13T22:44:46.205 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:46.327 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:46 smithi102 ceph-mon[28599]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-13T22:44:46.328 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:46 smithi102 ceph-mon[28599]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:46 smithi130 ceph-mon[36591]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-13T22:44:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:46 smithi130 ceph-mon[36591]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:47.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.819+0000 7f599af1e700 1 -- 172.21.15.102:0/3689237576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5994101750 msgr2=0x7f59941057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:47.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.819+0000 7f599af1e700 1 --2- 172.21.15.102:0/3689237576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5994101750 0x7f59941057a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f598400b0a0 tx=0x7f598400b3b0 comp rx=0 tx=0).stop 2024-05-13T22:44:47.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.819+0000 7f599af1e700 1 -- 172.21.15.102:0/3689237576 shutdown_connections 2024-05-13T22:44:47.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.819+0000 7f599af1e700 1 --2- 172.21.15.102:0/3689237576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5994101750 0x7f59941057a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:47.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.819+0000 7f599af1e700 1 --2- 172.21.15.102:0/3689237576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5994100da0 0x7f5994101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:47.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.819+0000 7f599af1e700 1 -- 172.21.15.102:0/3689237576 >> 172.21.15.102:0/3689237576 conn(0x7f59940fc630 msgr2=0x7f59940fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:47.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.819+0000 7f599af1e700 1 -- 172.21.15.102:0/3689237576 shutdown_connections 2024-05-13T22:44:47.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.819+0000 7f599af1e700 1 -- 172.21.15.102:0/3689237576 wait complete. 2024-05-13T22:44:47.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f599af1e700 1 Processor -- start 2024-05-13T22:44:47.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f599af1e700 1 -- start start 2024-05-13T22:44:47.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f599af1e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5994100da0 0x7f599419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:47.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f599af1e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5994101750 0x7f599419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:47.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f599af1e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599419d0d0 con 0x7f5994100da0 2024-05-13T22:44:47.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f599af1e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5994196580 con 0x7f5994101750 2024-05-13T22:44:47.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f5998cba700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5994100da0 0x7f599419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:47.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f5998cba700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5994100da0 0x7f599419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53792/0 (socket says 172.21.15.102:53792) 2024-05-13T22:44:47.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f5998cba700 1 -- 172.21.15.102:0/1601694764 learned_addr learned my addr 172.21.15.102:0/1601694764 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:47.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f5998cba700 1 -- 172.21.15.102:0/1601694764 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5994101750 msgr2=0x7f599419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:47.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f5998cba700 1 --2- 172.21.15.102:0/1601694764 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5994101750 0x7f599419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:47.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.820+0000 7f5998cba700 1 -- 172.21.15.102:0/1601694764 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984009d00 con 0x7f5994100da0 2024-05-13T22:44:47.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.821+0000 7f5998cba700 1 --2- 172.21.15.102:0/1601694764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5994100da0 0x7f599419c500 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d8d0 tx=0x7f598c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:47.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.821+0000 7f5991ffb700 1 -- 172.21.15.102:0/1601694764 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f598c009940 con 0x7f5994100da0 2024-05-13T22:44:47.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.821+0000 7f5991ffb700 1 -- 172.21.15.102:0/1601694764 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f598c010460 con 0x7f5994100da0 2024-05-13T22:44:47.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.821+0000 7f5991ffb700 1 -- 172.21.15.102:0/1601694764 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f598c00f5f0 con 0x7f5994100da0 2024-05-13T22:44:47.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.821+0000 7f599af1e700 1 -- 172.21.15.102:0/1601694764 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5994196860 con 0x7f5994100da0 2024-05-13T22:44:47.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.821+0000 7f599af1e700 1 -- 172.21.15.102:0/1601694764 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5994196d60 con 0x7f5994100da0 2024-05-13T22:44:47.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.822+0000 7f599af1e700 1 -- 172.21.15.102:0/1601694764 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59941091a0 con 0x7f5994100da0 2024-05-13T22:44:47.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.822+0000 7f5991ffb700 1 -- 172.21.15.102:0/1601694764 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f598c0105d0 con 0x7f5994100da0 2024-05-13T22:44:47.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.823+0000 7f5991ffb700 1 --2- 172.21.15.102:0/1601694764 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f598006c4e0 0x7f598006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:47.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.823+0000 7f5993fff700 1 --2- 172.21.15.102:0/1601694764 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f598006c4e0 0x7f598006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:47.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.823+0000 7f5991ffb700 1 -- 172.21.15.102:0/1601694764 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f598c08b4f0 con 0x7f5994100da0 2024-05-13T22:44:47.803 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.823+0000 7f5993fff700 1 --2- 172.21.15.102:0/1601694764 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f598006c4e0 0x7f598006e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f598400b830 tx=0x7f5984006000 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:47.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:47.827+0000 7f5991ffb700 1 -- 172.21.15.102:0/1601694764 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f598c059aa0 con 0x7f5994100da0 2024-05-13T22:44:47.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.015+0000 7f599af1e700 1 -- 172.21.15.102:0/1601694764 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59941970b0 con 0x7f598006c4e0 2024-05-13T22:44:47.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.016+0000 7f5991ffb700 1 -- 172.21.15.102:0/1601694764 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27826 (secure 0 0 0) 0x7f59941970b0 con 0x7f598006c4e0 2024-05-13T22:44:47.995 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:44:47.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.018+0000 7f599af1e700 1 -- 172.21.15.102:0/1601694764 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f598006c4e0 msgr2=0x7f598006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:47.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.018+0000 7f599af1e700 1 --2- 172.21.15.102:0/1601694764 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f598006c4e0 0x7f598006e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f598400b830 tx=0x7f5984006000 comp rx=0 tx=0).stop 2024-05-13T22:44:47.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.018+0000 7f599af1e700 1 -- 172.21.15.102:0/1601694764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5994100da0 msgr2=0x7f599419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:47.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.018+0000 7f599af1e700 1 --2- 172.21.15.102:0/1601694764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5994100da0 0x7f599419c500 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d8d0 tx=0x7f598c00dc90 comp rx=0 tx=0).stop 2024-05-13T22:44:47.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.018+0000 7f599af1e700 1 -- 172.21.15.102:0/1601694764 shutdown_connections 2024-05-13T22:44:47.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.018+0000 7f599af1e700 1 --2- 172.21.15.102:0/1601694764 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f598006c4e0 0x7f598006e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:47.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.018+0000 7f599af1e700 1 --2- 172.21.15.102:0/1601694764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5994100da0 0x7f599419c500 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:47.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.018+0000 7f599af1e700 1 --2- 172.21.15.102:0/1601694764 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5994101750 0x7f599419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:47.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.018+0000 7f599af1e700 1 -- 172.21.15.102:0/1601694764 >> 172.21.15.102:0/1601694764 conn(0x7f59940fc630 msgr2=0x7f59940fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:47.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.018+0000 7f599af1e700 1 -- 172.21.15.102:0/1601694764 shutdown_connections 2024-05-13T22:44:47.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:48.019+0000 7f599af1e700 1 -- 172.21.15.102:0/1601694764 wait complete. 2024-05-13T22:44:47.998 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-05-13T22:44:48.663 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-05-13T22:44:47.699711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002119"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-13T22:43:54.158442+0000","last_change":"2024-05-13T22:43:28.149339+0000","last_active":"2024-05-13T22:43:54.158442+0000","last_peered":"2024-05-13T22:43:54.158442+0000","last_clean":"2024-05-13T22:43:54.158442+0000","last_became_active":"2024-05-13T22:43:28.148937+0000","last_became_peered":"2024-05-13T22:43:28.148937+0000","last_unstale":"2024-05-13T22:43:54.158442+0000","last_undegraded":"2024-05-13T22:43:54.158442+0000","last_fullsized":"2024-05-13T22:43:54.158442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-13T22:42:40.185397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-13T22:42:40.185397+0000","last_clean_scrub_stamp":"2024-05-13T22:42:40.185397+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T01:53:45.013620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 13 22:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.58799999999999997}]},{"osd":1,"last update":"Mon May 13 22:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.54500000000000004}]},{"osd":2,"last update":"Mon May 13 22:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon May 13 22:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.36599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 13 22:44:04 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.55700000000000005}]},{"osd":1,"last update":"Mon May 13 22:44:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.501},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.439}]},{"osd":2,"last update":"Mon May 13 22:44:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.54200000000000004}]},{"osd":4,"last update":"Mon May 13 22:44:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.371},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.45100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 13 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35099999999999998,"15min":0.35699999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45300000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.33700000000000002,"15min":0.33900000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.498,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.38100000000000001}]},{"osd":1,"last update":"Mon May 13 22:44:39 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.48599999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.36799999999999999,"15min":0.373},"min":{"1min":0.24099999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.51600000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.41599999999999998}]},{"osd":3,"last update":"Mon May 13 22:44:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.72399999999999998}]},{"osd":4,"last update":"Mon May 13 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon May 13 22:44:16 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.316},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.47599999999999998}]},{"osd":2,"last update":"Mon May 13 22:43:43 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.50700000000000001}]},{"osd":3,"last update":"Mon May 13 22:44:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.89200000000000002}]},{"osd":4,"last update":"Mon May 13 22:44:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 13 22:44:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.29799999999999999,"15min":0.29099999999999998},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.32800000000000001,"15min":0.32200000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.253}]},{"osd":2,"last update":"Mon May 13 22:44:41 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34699999999999998,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.187,"15min":0.187},"max":{"1min":0.56100000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.378,"15min":0.38300000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.47999999999999998,"5min":0.753,"15min":0.753},"last":0.31}]},{"osd":3,"last update":"Mon May 13 22:44:03 2024","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":19.488,"5min":19.488,"15min":19.488},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.69599999999999995}]},{"osd":4,"last update":"Mon May 13 22:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":19.451000000000001,"5min":19.451000000000001,"15min":19.451000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-13T22:44:48.664 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-13T22:44:48.664 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-13T22:44:48.664 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-13T22:44:48.664 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-13T22:44:48.869 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:48.933 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:48 smithi102 ceph-mon[28599]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:48.934 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:48 smithi102 ceph-mon[28599]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-13T22:44:49.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:48 smithi130 ceph-mon[36591]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:49.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:48 smithi130 ceph-mon[36591]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-13T22:44:50.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.348+0000 7f8bb3206700 1 -- 172.21.15.102:0/709310567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bac100da0 msgr2=0x7f8bac101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:50.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.348+0000 7f8bb3206700 1 --2- 172.21.15.102:0/709310567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bac100da0 0x7f8bac101180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009910 tx=0x7f8ba4009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:50.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.348+0000 7f8bb3206700 1 -- 172.21.15.102:0/709310567 shutdown_connections 2024-05-13T22:44:50.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.348+0000 7f8bb3206700 1 --2- 172.21.15.102:0/709310567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bac101750 0x7f8bac1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:50.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.348+0000 7f8bb3206700 1 --2- 172.21.15.102:0/709310567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bac100da0 0x7f8bac101180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:50.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.348+0000 7f8bb3206700 1 -- 172.21.15.102:0/709310567 >> 172.21.15.102:0/709310567 conn(0x7f8bac0fc630 msgr2=0x7f8bac0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:50.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.348+0000 7f8bb3206700 1 -- 172.21.15.102:0/709310567 shutdown_connections 2024-05-13T22:44:50.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.348+0000 7f8bb3206700 1 -- 172.21.15.102:0/709310567 wait complete. 2024-05-13T22:44:50.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8bb3206700 1 Processor -- start 2024-05-13T22:44:50.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8bb3206700 1 -- start start 2024-05-13T22:44:50.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8bb3206700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bac100da0 0x7f8bac199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:50.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8bb3206700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bac101750 0x7f8bac199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:50.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8bb3206700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac19f090 con 0x7f8bac101750 2024-05-13T22:44:50.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8bb3206700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac19f200 con 0x7f8bac100da0 2024-05-13T22:44:50.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8babfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bac101750 0x7f8bac199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:50.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8babfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bac101750 0x7f8bac199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38822/0 (socket says 172.21.15.102:38822) 2024-05-13T22:44:50.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8babfff700 1 -- 172.21.15.102:0/4253617018 learned_addr learned my addr 172.21.15.102:0/4253617018 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:50.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8bb0fa2700 1 --2- 172.21.15.102:0/4253617018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bac100da0 0x7f8bac199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:50.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8babfff700 1 -- 172.21.15.102:0/4253617018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bac100da0 msgr2=0x7f8bac199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:50.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.349+0000 7f8babfff700 1 --2- 172.21.15.102:0/4253617018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bac100da0 0x7f8bac199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:50.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.350+0000 7f8babfff700 1 -- 172.21.15.102:0/4253617018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba40095f0 con 0x7f8bac101750 2024-05-13T22:44:50.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.350+0000 7f8babfff700 1 --2- 172.21.15.102:0/4253617018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bac101750 0x7f8bac199860 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c007e90 tx=0x7f8b9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:50.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.350+0000 7f8ba9ffb700 1 -- 172.21.15.102:0/4253617018 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b9c00dda0 con 0x7f8bac101750 2024-05-13T22:44:50.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.350+0000 7f8ba9ffb700 1 -- 172.21.15.102:0/4253617018 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8b9c00f040 con 0x7f8bac101750 2024-05-13T22:44:50.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.350+0000 7f8bb3206700 1 -- 172.21.15.102:0/4253617018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac19f4e0 con 0x7f8bac101750 2024-05-13T22:44:50.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.350+0000 7f8ba9ffb700 1 -- 172.21.15.102:0/4253617018 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b9c012610 con 0x7f8bac101750 2024-05-13T22:44:50.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.350+0000 7f8bb3206700 1 -- 172.21.15.102:0/4253617018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac19fa30 con 0x7f8bac101750 2024-05-13T22:44:50.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.351+0000 7f8bb3206700 1 -- 172.21.15.102:0/4253617018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bac04ea90 con 0x7f8bac101750 2024-05-13T22:44:50.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.352+0000 7f8ba9ffb700 1 -- 172.21.15.102:0/4253617018 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8b9c012880 con 0x7f8bac101750 2024-05-13T22:44:50.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.352+0000 7f8ba9ffb700 1 --2- 172.21.15.102:0/4253617018 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b9806c4e0 0x7f8b9806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:50.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.352+0000 7f8bb0fa2700 1 --2- 172.21.15.102:0/4253617018 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b9806c4e0 0x7f8b9806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:50.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.352+0000 7f8ba9ffb700 1 -- 172.21.15.102:0/4253617018 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8b9c08ad50 con 0x7f8bac101750 2024-05-13T22:44:50.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.353+0000 7f8bb0fa2700 1 --2- 172.21.15.102:0/4253617018 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b9806c4e0 0x7f8b9806e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400b5c0 tx=0x7f8ba4005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.357+0000 7f8ba9ffb700 1 -- 172.21.15.102:0/4253617018 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8b9c0558f0 con 0x7f8bac101750 2024-05-13T22:44:50.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.591+0000 7f8bb3206700 1 -- 172.21.15.102:0/4253617018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8bac19fd40 con 0x7f8bac101750 2024-05-13T22:44:50.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.591+0000 7f8ba9ffb700 1 -- 172.21.15.102:0/4253617018 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8b9c058f10 con 0x7f8bac101750 2024-05-13T22:44:50.570 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:44:50.570 INFO:teuthology.orchestra.run.smithi102.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-13T22:44:50.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.593+0000 7f8bb3206700 1 -- 172.21.15.102:0/4253617018 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b9806c4e0 msgr2=0x7f8b9806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:50.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.593+0000 7f8bb3206700 1 --2- 172.21.15.102:0/4253617018 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b9806c4e0 0x7f8b9806e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400b5c0 tx=0x7f8ba4005960 comp rx=0 tx=0).stop 2024-05-13T22:44:50.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.593+0000 7f8bb3206700 1 -- 172.21.15.102:0/4253617018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bac101750 msgr2=0x7f8bac199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:50.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.593+0000 7f8bb3206700 1 --2- 172.21.15.102:0/4253617018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bac101750 0x7f8bac199860 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c007e90 tx=0x7f8b9c00d3b0 comp rx=0 tx=0).stop 2024-05-13T22:44:50.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.593+0000 7f8bb3206700 1 -- 172.21.15.102:0/4253617018 shutdown_connections 2024-05-13T22:44:50.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.593+0000 7f8bb3206700 1 --2- 172.21.15.102:0/4253617018 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b9806c4e0 0x7f8b9806e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:50.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.593+0000 7f8bb3206700 1 --2- 172.21.15.102:0/4253617018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bac100da0 0x7f8bac199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:50.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.593+0000 7f8bb3206700 1 --2- 172.21.15.102:0/4253617018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8bac101750 0x7f8bac199860 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:50.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.593+0000 7f8bb3206700 1 -- 172.21.15.102:0/4253617018 >> 172.21.15.102:0/4253617018 conn(0x7f8bac0fc630 msgr2=0x7f8bac0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:50.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.594+0000 7f8bb3206700 1 -- 172.21.15.102:0/4253617018 shutdown_connections 2024-05-13T22:44:50.573 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:50.594+0000 7f8bb3206700 1 -- 172.21.15.102:0/4253617018 wait complete. 2024-05-13T22:44:50.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:50 smithi102 ceph-mon[28599]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:50.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:50 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/4253617018' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-13T22:44:50.991 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:50 smithi130 ceph-mon[36591]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:50.991 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:50 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/4253617018' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-13T22:44:51.120 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-13T22:44:51.120 INFO:tasks.cephadm:Setup complete, yielding 2024-05-13T22:44:51.121 INFO:teuthology.run_tasks:Running task print... 2024-05-13T22:44:51.136 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-05-13T22:44:51.136 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-13T22:44:51.168 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:44:51.168 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-05-13T22:44:51.373 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:52.841 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:52 smithi102 ceph-mon[28599]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:52.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.867+0000 7f5176aaa700 1 -- 172.21.15.102:0/2473886016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5170101dc0 msgr2=0x7f5170102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:52.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.867+0000 7f5176aaa700 1 --2- 172.21.15.102:0/2473886016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5170101dc0 0x7f5170102220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f515c009910 tx=0x7f515c009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:52.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.867+0000 7f5176aaa700 1 -- 172.21.15.102:0/2473886016 shutdown_connections 2024-05-13T22:44:52.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.867+0000 7f5176aaa700 1 --2- 172.21.15.102:0/2473886016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5170101dc0 0x7f5170102220 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:52.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f5176aaa700 1 --2- 172.21.15.102:0/2473886016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5170100bc0 0x7f5170100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:52.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f5176aaa700 1 -- 172.21.15.102:0/2473886016 >> 172.21.15.102:0/2473886016 conn(0x7f51700fc140 msgr2=0x7f51700fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:52.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f5176aaa700 1 -- 172.21.15.102:0/2473886016 shutdown_connections 2024-05-13T22:44:52.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f5176aaa700 1 -- 172.21.15.102:0/2473886016 wait complete. 2024-05-13T22:44:52.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f5176aaa700 1 Processor -- start 2024-05-13T22:44:52.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f5176aaa700 1 -- start start 2024-05-13T22:44:52.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f5176aaa700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5170100bc0 0x7f5170196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f5176aaa700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5170101dc0 0x7f5170196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f5176aaa700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5170196f90 con 0x7f5170101dc0 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f5176aaa700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51701970d0 con 0x7f5170100bc0 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f516ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5170101dc0 0x7f5170196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f516ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5170101dc0 0x7f5170196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38836/0 (socket says 172.21.15.102:38836) 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.868+0000 7f516ffff700 1 -- 172.21.15.102:0/4205151240 learned_addr learned my addr 172.21.15.102:0/4205151240 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.869+0000 7f5174846700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5170100bc0 0x7f5170196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.869+0000 7f516ffff700 1 -- 172.21.15.102:0/4205151240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5170100bc0 msgr2=0x7f5170196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.869+0000 7f516ffff700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5170100bc0 0x7f5170196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.869+0000 7f516ffff700 1 -- 172.21.15.102:0/4205151240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c0095f0 con 0x7f5170101dc0 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.869+0000 7f5174846700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5170100bc0 0x7f5170196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.869+0000 7f516ffff700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5170101dc0 0x7f5170196970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f515c006010 tx=0x7f515c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:52.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.869+0000 7f516dffb700 1 -- 172.21.15.102:0/4205151240 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f515c01d070 con 0x7f5170101dc0 2024-05-13T22:44:52.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.869+0000 7f516dffb700 1 -- 172.21.15.102:0/4205151240 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f515c00f460 con 0x7f5170101dc0 2024-05-13T22:44:52.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.869+0000 7f5176aaa700 1 -- 172.21.15.102:0/4205151240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517019bb20 con 0x7f5170101dc0 2024-05-13T22:44:52.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.869+0000 7f516dffb700 1 -- 172.21.15.102:0/4205151240 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f515c021640 con 0x7f5170101dc0 2024-05-13T22:44:52.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.869+0000 7f5176aaa700 1 -- 172.21.15.102:0/4205151240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517019c090 con 0x7f5170101dc0 2024-05-13T22:44:52.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.870+0000 7f5176aaa700 1 -- 172.21.15.102:0/4205151240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51701097c0 con 0x7f5170101dc0 2024-05-13T22:44:52.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.871+0000 7f516dffb700 1 -- 172.21.15.102:0/4205151240 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f515c00fab0 con 0x7f5170101dc0 2024-05-13T22:44:52.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.871+0000 7f516dffb700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f516006c4e0 0x7f516006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:52.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.871+0000 7f516dffb700 1 -- 172.21.15.102:0/4205151240 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f515c08c900 con 0x7f5170101dc0 2024-05-13T22:44:52.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.871+0000 7f5174846700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f516006c4e0 0x7f516006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:52.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.871+0000 7f5174846700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f516006c4e0 0x7f516006e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5170101c20 tx=0x7f5164008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:52.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:52.875+0000 7f516dffb700 1 -- 172.21.15.102:0/4205151240 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f515c05af30 con 0x7f5170101dc0 2024-05-13T22:44:53.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.062+0000 7f5176aaa700 1 -- 172.21.15.102:0/4205151240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f517019ca80 con 0x7f5170101dc0 2024-05-13T22:44:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.067+0000 7f516dffb700 1 -- 172.21.15.102:0/4205151240 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f515c05aac0 con 0x7f5170101dc0 2024-05-13T22:44:53.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.069+0000 7f5176aaa700 1 -- 172.21.15.102:0/4205151240 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f516006c4e0 msgr2=0x7f516006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:53.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.069+0000 7f5176aaa700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f516006c4e0 0x7f516006e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5170101c20 tx=0x7f5164008040 comp rx=0 tx=0).stop 2024-05-13T22:44:53.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.069+0000 7f5176aaa700 1 -- 172.21.15.102:0/4205151240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5170101dc0 msgr2=0x7f5170196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:53.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.069+0000 7f5176aaa700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5170101dc0 0x7f5170196970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f515c006010 tx=0x7f515c00be20 comp rx=0 tx=0).stop 2024-05-13T22:44:53.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.069+0000 7f5176aaa700 1 -- 172.21.15.102:0/4205151240 shutdown_connections 2024-05-13T22:44:53.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.069+0000 7f5176aaa700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f516006c4e0 0x7f516006e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:53.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.069+0000 7f5176aaa700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5170100bc0 0x7f5170196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:53.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.069+0000 7f5176aaa700 1 --2- 172.21.15.102:0/4205151240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5170101dc0 0x7f5170196970 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:53.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.069+0000 7f5176aaa700 1 -- 172.21.15.102:0/4205151240 >> 172.21.15.102:0/4205151240 conn(0x7f51700fc140 msgr2=0x7f5170104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:53.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.069+0000 7f5176aaa700 1 -- 172.21.15.102:0/4205151240 shutdown_connections 2024-05-13T22:44:53.050 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:53.069+0000 7f5176aaa700 1 -- 172.21.15.102:0/4205151240 wait complete. 2024-05-13T22:44:53.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:52 smithi130 ceph-mon[36591]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:53.734 INFO:teuthology.run_tasks:Running task print... 2024-05-13T22:44:53.743 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-05-13T22:44:53.743 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-13T22:44:53.769 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:44:53.769 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-13T22:44:53.974 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:54.318 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:54 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/4205151240' entity='client.admin' 2024-05-13T22:44:54.318 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:54 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:44:54.318 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:54 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:44:54.318 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:54 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:44:54.318 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:54 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:44:54.318 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:54 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:44:54.323 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:54 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/4205151240' entity='client.admin' 2024-05-13T22:44:54.323 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:54 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:44:54.323 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:54 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:44:54.323 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:54 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:44:54.323 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:54 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:44:54.324 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:54 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:44:55.231 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:55 smithi102 ceph-mon[28599]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:55.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:55 smithi130 ceph-mon[36591]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:55.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.443+0000 7f4dad1be700 1 -- 172.21.15.102:0/1290712821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da80ffab0 msgr2=0x7f4da80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:55.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.443+0000 7f4dad1be700 1 --2- 172.21.15.102:0/1290712821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da80ffab0 0x7f4da80ffed0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009910 tx=0x7f4d98009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:55.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.443+0000 7f4dad1be700 1 -- 172.21.15.102:0/1290712821 shutdown_connections 2024-05-13T22:44:55.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.443+0000 7f4dad1be700 1 --2- 172.21.15.102:0/1290712821 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4da8100cb0 0x7f4da8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:55.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.443+0000 7f4dad1be700 1 --2- 172.21.15.102:0/1290712821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da80ffab0 0x7f4da80ffed0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:55.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.443+0000 7f4dad1be700 1 -- 172.21.15.102:0/1290712821 >> 172.21.15.102:0/1290712821 conn(0x7f4da80fb030 msgr2=0x7f4da80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:55.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.443+0000 7f4dad1be700 1 -- 172.21.15.102:0/1290712821 shutdown_connections 2024-05-13T22:44:55.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.443+0000 7f4dad1be700 1 -- 172.21.15.102:0/1290712821 wait complete. 2024-05-13T22:44:55.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.444+0000 7f4dad1be700 1 Processor -- start 2024-05-13T22:44:55.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.444+0000 7f4dad1be700 1 -- start start 2024-05-13T22:44:55.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.444+0000 7f4dad1be700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4da80ffab0 0x7f4da8196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:55.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.444+0000 7f4dad1be700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da8100cb0 0x7f4da8196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:55.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.444+0000 7f4dad1be700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da8196e80 con 0x7f4da8100cb0 2024-05-13T22:44:55.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.444+0000 7f4dad1be700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da8196fc0 con 0x7f4da80ffab0 2024-05-13T22:44:55.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4da659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da8100cb0 0x7f4da8196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4da659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da8100cb0 0x7f4da8196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38854/0 (socket says 172.21.15.102:38854) 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4da659c700 1 -- 172.21.15.102:0/1870679887 learned_addr learned my addr 172.21.15.102:0/1870679887 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4da6d9d700 1 --2- 172.21.15.102:0/1870679887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4da80ffab0 0x7f4da8196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4da659c700 1 -- 172.21.15.102:0/1870679887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4da80ffab0 msgr2=0x7f4da8196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4da659c700 1 --2- 172.21.15.102:0/1870679887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4da80ffab0 0x7f4da8196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4da659c700 1 -- 172.21.15.102:0/1870679887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d980095f0 con 0x7f4da8100cb0 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4da659c700 1 --2- 172.21.15.102:0/1870679887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da8100cb0 0x7f4da8196860 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00d900 tx=0x7f4d9c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4d8ffff700 1 -- 172.21.15.102:0/1870679887 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d9c0041f0 con 0x7f4da8100cb0 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4d8ffff700 1 -- 172.21.15.102:0/1870679887 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4d9c004ce0 con 0x7f4da8100cb0 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4dad1be700 1 -- 172.21.15.102:0/1870679887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da819ba70 con 0x7f4da8100cb0 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4d8ffff700 1 -- 172.21.15.102:0/1870679887 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d9c00b750 con 0x7f4da8100cb0 2024-05-13T22:44:55.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.445+0000 7f4dad1be700 1 -- 172.21.15.102:0/1870679887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da819bfc0 con 0x7f4da8100cb0 2024-05-13T22:44:55.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.446+0000 7f4dad1be700 1 -- 172.21.15.102:0/1870679887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da8108610 con 0x7f4da8100cb0 2024-05-13T22:44:55.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.447+0000 7f4d8ffff700 1 -- 172.21.15.102:0/1870679887 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4d9c004e50 con 0x7f4da8100cb0 2024-05-13T22:44:55.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.447+0000 7f4d8ffff700 1 --2- 172.21.15.102:0/1870679887 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4d9006c4e0 0x7f4d9006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:55.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.447+0000 7f4d8ffff700 1 -- 172.21.15.102:0/1870679887 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4d9c08b470 con 0x7f4da8100cb0 2024-05-13T22:44:55.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.447+0000 7f4da6d9d700 1 --2- 172.21.15.102:0/1870679887 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4d9006c4e0 0x7f4d9006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:55.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.448+0000 7f4da6d9d700 1 --2- 172.21.15.102:0/1870679887 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4d9006c4e0 0x7f4d9006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d98006010 tx=0x7f4d98005ec0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:55.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.452+0000 7f4d8ffff700 1 -- 172.21.15.102:0/1870679887 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4d9c059a20 con 0x7f4da8100cb0 2024-05-13T22:44:55.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.657+0000 7f4dad1be700 1 -- 172.21.15.102:0/1870679887 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4da8068ae0 con 0x7f4d9006c4e0 2024-05-13T22:44:55.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.658+0000 7f4d8ffff700 1 -- 172.21.15.102:0/1870679887 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f4da8068ae0 con 0x7f4d9006c4e0 2024-05-13T22:44:55.638 INFO:teuthology.orchestra.run.smithi102.stdout:Backend: cephadm 2024-05-13T22:44:55.638 INFO:teuthology.orchestra.run.smithi102.stdout:Available: Yes 2024-05-13T22:44:55.638 INFO:teuthology.orchestra.run.smithi102.stdout:Paused: No 2024-05-13T22:44:55.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.661+0000 7f4dad1be700 1 -- 172.21.15.102:0/1870679887 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4d9006c4e0 msgr2=0x7f4d9006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:55.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.661+0000 7f4dad1be700 1 --2- 172.21.15.102:0/1870679887 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4d9006c4e0 0x7f4d9006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d98006010 tx=0x7f4d98005ec0 comp rx=0 tx=0).stop 2024-05-13T22:44:55.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.661+0000 7f4dad1be700 1 -- 172.21.15.102:0/1870679887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da8100cb0 msgr2=0x7f4da8196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:55.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.661+0000 7f4dad1be700 1 --2- 172.21.15.102:0/1870679887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da8100cb0 0x7f4da8196860 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00d900 tx=0x7f4d9c00dc10 comp rx=0 tx=0).stop 2024-05-13T22:44:55.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.661+0000 7f4dad1be700 1 -- 172.21.15.102:0/1870679887 shutdown_connections 2024-05-13T22:44:55.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.661+0000 7f4dad1be700 1 --2- 172.21.15.102:0/1870679887 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4d9006c4e0 0x7f4d9006e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:55.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.661+0000 7f4dad1be700 1 --2- 172.21.15.102:0/1870679887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4da80ffab0 0x7f4da8196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:55.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.661+0000 7f4dad1be700 1 --2- 172.21.15.102:0/1870679887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4da8100cb0 0x7f4da8196860 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:55.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.661+0000 7f4dad1be700 1 -- 172.21.15.102:0/1870679887 >> 172.21.15.102:0/1870679887 conn(0x7f4da80fb030 msgr2=0x7f4da80683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:55.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.661+0000 7f4dad1be700 1 -- 172.21.15.102:0/1870679887 shutdown_connections 2024-05-13T22:44:55.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:55.661+0000 7f4dad1be700 1 -- 172.21.15.102:0/1870679887 wait complete. 2024-05-13T22:44:56.290 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-13T22:44:56.302 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:56 smithi102 ceph-mon[28599]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:44:56.302 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:56 smithi102 ceph-mon[28599]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:56.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:56 smithi130 ceph-mon[36591]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:44:56.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:56 smithi130 ceph-mon[36591]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:56.511 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:44:58.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.211+0000 7fa5955b9700 1 -- 172.21.15.102:0/3879851049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa590100cb0 msgr2=0x7fa590101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:58.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.211+0000 7fa5955b9700 1 --2- 172.21.15.102:0/3879851049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa590100cb0 0x7fa590101110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa578009910 tx=0x7fa578009c20 comp rx=0 tx=0).stop 2024-05-13T22:44:58.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.211+0000 7fa5955b9700 1 -- 172.21.15.102:0/3879851049 shutdown_connections 2024-05-13T22:44:58.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.211+0000 7fa5955b9700 1 --2- 172.21.15.102:0/3879851049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa590100cb0 0x7fa590101110 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:58.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.211+0000 7fa5955b9700 1 --2- 172.21.15.102:0/3879851049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5900ffab0 0x7fa5900ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:58.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.211+0000 7fa5955b9700 1 -- 172.21.15.102:0/3879851049 >> 172.21.15.102:0/3879851049 conn(0x7fa5900fb010 msgr2=0x7fa5900fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:58.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.211+0000 7fa5955b9700 1 -- 172.21.15.102:0/3879851049 shutdown_connections 2024-05-13T22:44:58.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.211+0000 7fa5955b9700 1 -- 172.21.15.102:0/3879851049 wait complete. 2024-05-13T22:44:58.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.212+0000 7fa5955b9700 1 Processor -- start 2024-05-13T22:44:58.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.212+0000 7fa5955b9700 1 -- start start 2024-05-13T22:44:58.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.212+0000 7fa5955b9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5900ffab0 0x7fa590196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:58.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.212+0000 7fa5955b9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa590100cb0 0x7fa5901968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:58.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.212+0000 7fa5955b9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa590196ed0 con 0x7fa590100cb0 2024-05-13T22:44:58.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.212+0000 7fa5955b9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa590197010 con 0x7fa5900ffab0 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa58e7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa590100cb0 0x7fa5901968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa58e7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa590100cb0 0x7fa5901968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38884/0 (socket says 172.21.15.102:38884) 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa58e7fc700 1 -- 172.21.15.102:0/2895098054 learned_addr learned my addr 172.21.15.102:0/2895098054 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa58effd700 1 --2- 172.21.15.102:0/2895098054 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5900ffab0 0x7fa590196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa58e7fc700 1 -- 172.21.15.102:0/2895098054 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5900ffab0 msgr2=0x7fa590196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa58e7fc700 1 --2- 172.21.15.102:0/2895098054 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5900ffab0 0x7fa590196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa58e7fc700 1 -- 172.21.15.102:0/2895098054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5780095f0 con 0x7fa590100cb0 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa58e7fc700 1 --2- 172.21.15.102:0/2895098054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa590100cb0 0x7fa5901968b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa578006010 tx=0x7fa57800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa587fff700 1 -- 172.21.15.102:0/2895098054 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57801d070 con 0x7fa590100cb0 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa5955b9700 1 -- 172.21.15.102:0/2895098054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59019ba60 con 0x7fa590100cb0 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa587fff700 1 -- 172.21.15.102:0/2895098054 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa57800f460 con 0x7fa590100cb0 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa587fff700 1 -- 172.21.15.102:0/2895098054 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa578021640 con 0x7fa590100cb0 2024-05-13T22:44:58.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.213+0000 7fa5955b9700 1 -- 172.21.15.102:0/2895098054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59019bf50 con 0x7fa590100cb0 2024-05-13T22:44:58.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.214+0000 7fa5955b9700 1 -- 172.21.15.102:0/2895098054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa590108610 con 0x7fa590100cb0 2024-05-13T22:44:58.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.215+0000 7fa587fff700 1 -- 172.21.15.102:0/2895098054 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa57800fab0 con 0x7fa590100cb0 2024-05-13T22:44:58.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.215+0000 7fa587fff700 1 --2- 172.21.15.102:0/2895098054 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa57c06c4e0 0x7fa57c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:44:58.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.215+0000 7fa58effd700 1 --2- 172.21.15.102:0/2895098054 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa57c06c4e0 0x7fa57c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:44:58.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.215+0000 7fa587fff700 1 -- 172.21.15.102:0/2895098054 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa57808c780 con 0x7fa590100cb0 2024-05-13T22:44:58.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.216+0000 7fa58effd700 1 --2- 172.21.15.102:0/2895098054 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa57c06c4e0 0x7fa57c06e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa588009550 tx=0x7fa5880091b0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:44:58.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.219+0000 7fa587fff700 1 -- 172.21.15.102:0/2895098054 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa57805ad30 con 0x7fa590100cb0 2024-05-13T22:44:58.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.415+0000 7fa5955b9700 1 -- 172.21.15.102:0/2895098054 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa590069500 con 0x7fa57c06c4e0 2024-05-13T22:44:58.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.424+0000 7fa587fff700 1 -- 172.21.15.102:0/2895098054 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2820 (secure 0 0 0) 0x7fa590069500 con 0x7fa57c06c4e0 2024-05-13T22:44:58.403 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:44:58.403 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (3m) 2m ago 5m 26.9M - 0.25.0 c8568f914cd2 c096e9e9a228 2024-05-13T22:44:58.403 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (5m) 2m ago 5m 7075k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (4m) 65s ago 4m 6929k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (5m) 2m ago 5m 6656k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (4m) 65s ago 4m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (3m) 2m ago 4m 77.6M - 9.4.7 954c08fa6188 bca1790c17aa 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:9283,8765,8443 running (6m) 2m ago 6m 481M - 18.2.1 5be31c24972a 77cd1ca053e7 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (4m) 65s ago 4m 439M - 18.2.1 5be31c24972a c8a9d782f394 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (6m) 2m ago 6m 43.7M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (4m) 65s ago 4m 42.8M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (5m) 2m ago 5m 15.9M - 1.5.0 0da6a335fe13 af297cf52099 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (4m) 65s ago 4m 15.9M - 1.5.0 0da6a335fe13 cc0f64ee7ecd 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (3m) 2m ago 3m 41.0M 3946M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (2m) 2m ago 2m 40.4M 3946M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (2m) 2m ago 2m 38.8M 3946M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:44:58.404 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (2m) 65s ago 2m 42.2M 4970M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:44:58.405 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (97s) 65s ago 96s 42.0M 4970M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:44:58.405 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (71s) 65s ago 70s 13.6M 4970M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:44:58.405 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (3m) 2m ago 4m 40.2M - 2.43.0 a07b618ecd1d a89752444eb6 2024-05-13T22:44:58.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.426+0000 7fa5955b9700 1 -- 172.21.15.102:0/2895098054 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa57c06c4e0 msgr2=0x7fa57c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:58.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.426+0000 7fa5955b9700 1 --2- 172.21.15.102:0/2895098054 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa57c06c4e0 0x7fa57c06e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa588009550 tx=0x7fa5880091b0 comp rx=0 tx=0).stop 2024-05-13T22:44:58.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.426+0000 7fa5955b9700 1 -- 172.21.15.102:0/2895098054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa590100cb0 msgr2=0x7fa5901968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:44:58.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.426+0000 7fa5955b9700 1 --2- 172.21.15.102:0/2895098054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa590100cb0 0x7fa5901968b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa578006010 tx=0x7fa57800bd70 comp rx=0 tx=0).stop 2024-05-13T22:44:58.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.426+0000 7fa5955b9700 1 -- 172.21.15.102:0/2895098054 shutdown_connections 2024-05-13T22:44:58.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.426+0000 7fa5955b9700 1 --2- 172.21.15.102:0/2895098054 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fa57c06c4e0 0x7fa57c06e9a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:58.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.426+0000 7fa5955b9700 1 --2- 172.21.15.102:0/2895098054 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5900ffab0 0x7fa590196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:58.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.426+0000 7fa5955b9700 1 --2- 172.21.15.102:0/2895098054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa590100cb0 0x7fa5901968b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:44:58.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.426+0000 7fa5955b9700 1 -- 172.21.15.102:0/2895098054 >> 172.21.15.102:0/2895098054 conn(0x7fa5900fb010 msgr2=0x7fa590068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:44:58.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.426+0000 7fa5955b9700 1 -- 172.21.15.102:0/2895098054 shutdown_connections 2024-05-13T22:44:58.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:44:58.426+0000 7fa5955b9700 1 -- 172.21.15.102:0/2895098054 wait complete. 2024-05-13T22:44:59.007 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:58 smithi102 ceph-mon[28599]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:59.030 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-13T22:44:59.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:58 smithi130 ceph-mon[36591]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:44:59.277 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:00.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:44:59 smithi130 ceph-mon[36591]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:00.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:44:59 smithi102 ceph-mon[28599]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.783+0000 7faaa2cd2700 1 -- 172.21.15.102:0/3212695398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa9c100cc0 msgr2=0x7faa9c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.783+0000 7faaa2cd2700 1 --2- 172.21.15.102:0/3212695398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa9c100cc0 0x7faa9c101120 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa88009910 tx=0x7faa88009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.783+0000 7faaa2cd2700 1 -- 172.21.15.102:0/3212695398 shutdown_connections 2024-05-13T22:45:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.783+0000 7faaa2cd2700 1 --2- 172.21.15.102:0/3212695398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa9c100cc0 0x7faa9c101120 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:00.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.783+0000 7faaa2cd2700 1 --2- 172.21.15.102:0/3212695398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa9c0ffac0 0x7faa9c0ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:00.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.783+0000 7faaa2cd2700 1 -- 172.21.15.102:0/3212695398 >> 172.21.15.102:0/3212695398 conn(0x7faa9c0fb060 msgr2=0x7faa9c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:00.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.784+0000 7faaa2cd2700 1 -- 172.21.15.102:0/3212695398 shutdown_connections 2024-05-13T22:45:00.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.784+0000 7faaa2cd2700 1 -- 172.21.15.102:0/3212695398 wait complete. 2024-05-13T22:45:00.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.784+0000 7faaa2cd2700 1 Processor -- start 2024-05-13T22:45:00.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.784+0000 7faaa2cd2700 1 -- start start 2024-05-13T22:45:00.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.784+0000 7faaa2cd2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa9c0ffac0 0x7faa9c069910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.784+0000 7faaa2cd2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa9c100cc0 0x7faa9c19b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.784+0000 7faaa2cd2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c19b740 con 0x7faa9c0ffac0 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.784+0000 7faaa2cd2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c19b880 con 0x7faa9c100cc0 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.784+0000 7faaa0a6e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa9c0ffac0 0x7faa9c069910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.784+0000 7faaa0a6e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa9c0ffac0 0x7faa9c069910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59608/0 (socket says 172.21.15.102:59608) 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.784+0000 7faaa0a6e700 1 -- 172.21.15.102:0/114835716 learned_addr learned my addr 172.21.15.102:0/114835716 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.785+0000 7faa9bfff700 1 --2- 172.21.15.102:0/114835716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa9c100cc0 0x7faa9c19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.785+0000 7faaa0a6e700 1 -- 172.21.15.102:0/114835716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa9c100cc0 msgr2=0x7faa9c19b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.785+0000 7faaa0a6e700 1 --2- 172.21.15.102:0/114835716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa9c100cc0 0x7faa9c19b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.785+0000 7faaa0a6e700 1 -- 172.21.15.102:0/114835716 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa880095f0 con 0x7faa9c0ffac0 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.785+0000 7faaa0a6e700 1 --2- 172.21.15.102:0/114835716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa9c0ffac0 0x7faa9c069910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faa9000b480 tx=0x7faa9000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.785+0000 7faa99ffb700 1 -- 172.21.15.102:0/114835716 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa90011800 con 0x7faa9c0ffac0 2024-05-13T22:45:00.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.785+0000 7faa99ffb700 1 -- 172.21.15.102:0/114835716 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faa90011e10 con 0x7faa9c0ffac0 2024-05-13T22:45:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.785+0000 7faaa2cd2700 1 -- 172.21.15.102:0/114835716 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9c19bb60 con 0x7faa9c0ffac0 2024-05-13T22:45:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.785+0000 7faa99ffb700 1 -- 172.21.15.102:0/114835716 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa9000f2e0 con 0x7faa9c0ffac0 2024-05-13T22:45:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.785+0000 7faaa2cd2700 1 -- 172.21.15.102:0/114835716 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9c19c130 con 0x7faa9c0ffac0 2024-05-13T22:45:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.786+0000 7faaa2cd2700 1 -- 172.21.15.102:0/114835716 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9c1086c0 con 0x7faa9c0ffac0 2024-05-13T22:45:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.787+0000 7faa99ffb700 1 -- 172.21.15.102:0/114835716 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7faa90010380 con 0x7faa9c0ffac0 2024-05-13T22:45:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.787+0000 7faa99ffb700 1 --2- 172.21.15.102:0/114835716 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa8c06c4e0 0x7faa8c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.787+0000 7faa9bfff700 1 --2- 172.21.15.102:0/114835716 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa8c06c4e0 0x7faa8c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.787+0000 7faa99ffb700 1 -- 172.21.15.102:0/114835716 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7faa9008b040 con 0x7faa9c0ffac0 2024-05-13T22:45:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.788+0000 7faa9bfff700 1 --2- 172.21.15.102:0/114835716 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa8c06c4e0 0x7faa8c06e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa9c069e90 tx=0x7faa8800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:00.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.792+0000 7faa99ffb700 1 -- 172.21.15.102:0/114835716 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faa900595f0 con 0x7faa9c0ffac0 2024-05-13T22:45:00.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:00 smithi102 ceph-mon[28599]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:00.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.987+0000 7faaa2cd2700 1 -- 172.21.15.102:0/114835716 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7faa9c105610 con 0x7faa8c06c4e0 2024-05-13T22:45:00.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.992+0000 7faa99ffb700 1 -- 172.21.15.102:0/114835716 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1310 (secure 0 0 0) 0x7faa9c105610 con 0x7faa8c06c4e0 2024-05-13T22:45:00.970 INFO:teuthology.orchestra.run.smithi102.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-13T22:45:00.970 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-05-13T22:45:00.971 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter 2/2 2m ago 6m * 2024-05-13T22:45:00.971 INFO:teuthology.orchestra.run.smithi102.stdout:crash 2/2 2m ago 6m * 2024-05-13T22:45:00.971 INFO:teuthology.orchestra.run.smithi102.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-05-13T22:45:00.971 INFO:teuthology.orchestra.run.smithi102.stdout:mgr 2/2 2m ago 6m count:2 2024-05-13T22:45:00.971 INFO:teuthology.orchestra.run.smithi102.stdout:mon 2/2 2m ago 5m smithi102:172.21.15.102=smithi102;smithi130:172.21.15.130=smithi130;count:2 2024-05-13T22:45:00.971 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-05-13T22:45:00.971 INFO:teuthology.orchestra.run.smithi102.stdout:osd 6 2m ago - 2024-05-13T22:45:00.971 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-05-13T22:45:00.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.994+0000 7faaa2cd2700 1 -- 172.21.15.102:0/114835716 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa8c06c4e0 msgr2=0x7faa8c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:00.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.994+0000 7faaa2cd2700 1 --2- 172.21.15.102:0/114835716 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa8c06c4e0 0x7faa8c06e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa9c069e90 tx=0x7faa8800b540 comp rx=0 tx=0).stop 2024-05-13T22:45:00.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.994+0000 7faaa2cd2700 1 -- 172.21.15.102:0/114835716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa9c0ffac0 msgr2=0x7faa9c069910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:00.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.994+0000 7faaa2cd2700 1 --2- 172.21.15.102:0/114835716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa9c0ffac0 0x7faa9c069910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faa9000b480 tx=0x7faa9000b790 comp rx=0 tx=0).stop 2024-05-13T22:45:00.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.994+0000 7faaa2cd2700 1 -- 172.21.15.102:0/114835716 shutdown_connections 2024-05-13T22:45:00.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.994+0000 7faaa2cd2700 1 --2- 172.21.15.102:0/114835716 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa8c06c4e0 0x7faa8c06e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:00.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.994+0000 7faaa2cd2700 1 --2- 172.21.15.102:0/114835716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa9c0ffac0 0x7faa9c069910 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:00.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.994+0000 7faaa2cd2700 1 --2- 172.21.15.102:0/114835716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa9c100cc0 0x7faa9c19b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:00.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.994+0000 7faaa2cd2700 1 -- 172.21.15.102:0/114835716 >> 172.21.15.102:0/114835716 conn(0x7faa9c0fb060 msgr2=0x7faa9c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:00.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.994+0000 7faaa2cd2700 1 -- 172.21.15.102:0/114835716 shutdown_connections 2024-05-13T22:45:00.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:00.994+0000 7faaa2cd2700 1 -- 172.21.15.102:0/114835716 wait complete. 2024-05-13T22:45:01.066 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:00 smithi130 ceph-mon[36591]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:01.559 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-13T22:45:01.766 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:02.066 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:01 smithi102 ceph-mon[28599]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:02.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:01 smithi130 ceph-mon[36591]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:03.055 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:02 smithi102 ceph-mon[28599]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:02 smithi130 ceph-mon[36591]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:03.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.281+0000 7fd15da47700 1 -- 172.21.15.102:0/1778380310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd158100bd0 msgr2=0x7fd158100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:03.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.281+0000 7fd15da47700 1 --2- 172.21.15.102:0/1778380310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd158100bd0 0x7fd158100ff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd140009910 tx=0x7fd140009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:03.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.281+0000 7fd15da47700 1 -- 172.21.15.102:0/1778380310 shutdown_connections 2024-05-13T22:45:03.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.281+0000 7fd15da47700 1 --2- 172.21.15.102:0/1778380310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd158101dd0 0x7fd158102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:03.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.281+0000 7fd15da47700 1 --2- 172.21.15.102:0/1778380310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd158100bd0 0x7fd158100ff0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:03.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.281+0000 7fd15da47700 1 -- 172.21.15.102:0/1778380310 >> 172.21.15.102:0/1778380310 conn(0x7fd1580fc190 msgr2=0x7fd1580fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:03.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.281+0000 7fd15da47700 1 -- 172.21.15.102:0/1778380310 shutdown_connections 2024-05-13T22:45:03.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.281+0000 7fd15da47700 1 -- 172.21.15.102:0/1778380310 wait complete. 2024-05-13T22:45:03.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.282+0000 7fd15da47700 1 Processor -- start 2024-05-13T22:45:03.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.282+0000 7fd15da47700 1 -- start start 2024-05-13T22:45:03.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.282+0000 7fd15da47700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd158100bd0 0x7fd1581963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:03.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.282+0000 7fd15da47700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd158101dd0 0x7fd158196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:03.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.282+0000 7fd15da47700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd158196f40 con 0x7fd158100bd0 2024-05-13T22:45:03.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd15da47700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd158197080 con 0x7fd158101dd0 2024-05-13T22:45:03.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd156ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd158100bd0 0x7fd1581963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd156ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd158100bd0 0x7fd1581963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59626/0 (socket says 172.21.15.102:59626) 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd156ffd700 1 -- 172.21.15.102:0/692359835 learned_addr learned my addr 172.21.15.102:0/692359835 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd1567fc700 1 --2- 172.21.15.102:0/692359835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd158101dd0 0x7fd158196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd156ffd700 1 -- 172.21.15.102:0/692359835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd158101dd0 msgr2=0x7fd158196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd156ffd700 1 --2- 172.21.15.102:0/692359835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd158101dd0 0x7fd158196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd156ffd700 1 -- 172.21.15.102:0/692359835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1400095f0 con 0x7fd158100bd0 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd156ffd700 1 --2- 172.21.15.102:0/692359835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd158100bd0 0x7fd1581963e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd14000bda0 tx=0x7fd14000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd15ca45700 1 -- 172.21.15.102:0/692359835 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd14001d070 con 0x7fd158100bd0 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd15ca45700 1 -- 172.21.15.102:0/692359835 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd140004c50 con 0x7fd158100bd0 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd15da47700 1 -- 172.21.15.102:0/692359835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15819bad0 con 0x7fd158100bd0 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd15ca45700 1 -- 172.21.15.102:0/692359835 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd14000f690 con 0x7fd158100bd0 2024-05-13T22:45:03.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.283+0000 7fd15da47700 1 -- 172.21.15.102:0/692359835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15819c040 con 0x7fd158100bd0 2024-05-13T22:45:03.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.284+0000 7fd15da47700 1 -- 172.21.15.102:0/692359835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1581097b0 con 0x7fd158100bd0 2024-05-13T22:45:03.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.285+0000 7fd15ca45700 1 -- 172.21.15.102:0/692359835 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd1400229b0 con 0x7fd158100bd0 2024-05-13T22:45:03.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.285+0000 7fd15ca45700 1 --2- 172.21.15.102:0/692359835 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd14406c4e0 0x7fd14406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:03.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.285+0000 7fd15ca45700 1 -- 172.21.15.102:0/692359835 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd14008cc40 con 0x7fd158100bd0 2024-05-13T22:45:03.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.285+0000 7fd1567fc700 1 --2- 172.21.15.102:0/692359835 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd14406c4e0 0x7fd14406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:03.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.286+0000 7fd1567fc700 1 --2- 172.21.15.102:0/692359835 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd14406c4e0 0x7fd14406e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd148006fd0 tx=0x7fd148008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:03.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.290+0000 7fd15ca45700 1 -- 172.21.15.102:0/692359835 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd14005b2a0 con 0x7fd158100bd0 2024-05-13T22:45:03.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.484+0000 7fd15da47700 1 -- 172.21.15.102:0/692359835 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd158106720 con 0x7fd14406c4e0 2024-05-13T22:45:03.464 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.486+0000 7fd15ca45700 1 -- 172.21.15.102:0/692359835 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7fd158106720 con 0x7fd14406c4e0 2024-05-13T22:45:03.464 INFO:teuthology.orchestra.run.smithi102.stdout:HOST ADDR LABELS STATUS 2024-05-13T22:45:03.464 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102 172.21.15.102 2024-05-13T22:45:03.464 INFO:teuthology.orchestra.run.smithi102.stdout:smithi130 172.21.15.130 2024-05-13T22:45:03.464 INFO:teuthology.orchestra.run.smithi102.stdout:2 hosts in cluster 2024-05-13T22:45:03.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.488+0000 7fd15da47700 1 -- 172.21.15.102:0/692359835 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd14406c4e0 msgr2=0x7fd14406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:03.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.488+0000 7fd15da47700 1 --2- 172.21.15.102:0/692359835 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd14406c4e0 0x7fd14406e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd148006fd0 tx=0x7fd148008040 comp rx=0 tx=0).stop 2024-05-13T22:45:03.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.488+0000 7fd15da47700 1 -- 172.21.15.102:0/692359835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd158100bd0 msgr2=0x7fd1581963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:03.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.488+0000 7fd15da47700 1 --2- 172.21.15.102:0/692359835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd158100bd0 0x7fd1581963e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd14000bda0 tx=0x7fd14000be80 comp rx=0 tx=0).stop 2024-05-13T22:45:03.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.488+0000 7fd15da47700 1 -- 172.21.15.102:0/692359835 shutdown_connections 2024-05-13T22:45:03.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.488+0000 7fd15da47700 1 --2- 172.21.15.102:0/692359835 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd14406c4e0 0x7fd14406e9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:03.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.488+0000 7fd15da47700 1 --2- 172.21.15.102:0/692359835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd158100bd0 0x7fd1581963e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:03.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.488+0000 7fd15da47700 1 --2- 172.21.15.102:0/692359835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd158101dd0 0x7fd158196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:03.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.488+0000 7fd15da47700 1 -- 172.21.15.102:0/692359835 >> 172.21.15.102:0/692359835 conn(0x7fd1580fc190 msgr2=0x7fd158105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:03.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.488+0000 7fd15da47700 1 -- 172.21.15.102:0/692359835 shutdown_connections 2024-05-13T22:45:03.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:03.488+0000 7fd15da47700 1 -- 172.21.15.102:0/692359835 wait complete. 2024-05-13T22:45:04.125 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-13T22:45:04.333 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:05.127 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:04 smithi102 ceph-mon[28599]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:05.127 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:04 smithi102 ceph-mon[28599]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:05.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:04 smithi130 ceph-mon[36591]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:05.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:04 smithi130 ceph-mon[36591]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:05.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.922+0000 7fad23bdd700 1 -- 172.21.15.102:0/3554247502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ff4d0 msgr2=0x7fad1c0ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:05.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.922+0000 7fad23bdd700 1 --2- 172.21.15.102:0/3554247502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ff4d0 0x7fad1c0ff8f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fad18009910 tx=0x7fad18009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:05.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.923+0000 7fad23bdd700 1 -- 172.21.15.102:0/3554247502 shutdown_connections 2024-05-13T22:45:05.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.923+0000 7fad23bdd700 1 --2- 172.21.15.102:0/3554247502 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ffe30 0x7fad1c1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:05.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.923+0000 7fad23bdd700 1 --2- 172.21.15.102:0/3554247502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ff4d0 0x7fad1c0ff8f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:05.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.923+0000 7fad23bdd700 1 -- 172.21.15.102:0/3554247502 >> 172.21.15.102:0/3554247502 conn(0x7fad1c0fb050 msgr2=0x7fad1c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:05.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.923+0000 7fad23bdd700 1 -- 172.21.15.102:0/3554247502 shutdown_connections 2024-05-13T22:45:05.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.923+0000 7fad23bdd700 1 -- 172.21.15.102:0/3554247502 wait complete. 2024-05-13T22:45:05.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.923+0000 7fad23bdd700 1 Processor -- start 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.923+0000 7fad23bdd700 1 -- start start 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad23bdd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ff4d0 0x7fad1c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad23bdd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffe30 0x7fad1c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad23bdd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c199110 con 0x7fad1c0ffe30 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad23bdd700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c199250 con 0x7fad1c0ff4d0 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad21178700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffe30 0x7fad1c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad21178700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffe30 0x7fad1c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59634/0 (socket says 172.21.15.102:59634) 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad21178700 1 -- 172.21.15.102:0/1857056730 learned_addr learned my addr 172.21.15.102:0/1857056730 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad21979700 1 --2- 172.21.15.102:0/1857056730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ff4d0 0x7fad1c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad21178700 1 -- 172.21.15.102:0/1857056730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ff4d0 msgr2=0x7fad1c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad21178700 1 --2- 172.21.15.102:0/1857056730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ff4d0 0x7fad1c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad21178700 1 -- 172.21.15.102:0/1857056730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad180095f0 con 0x7fad1c0ffe30 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad21178700 1 --2- 172.21.15.102:0/1857056730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffe30 0x7fad1c198af0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00d8d0 tx=0x7fad0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:05.904 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad12ffd700 1 -- 172.21.15.102:0/1857056730 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad0c009940 con 0x7fad1c0ffe30 2024-05-13T22:45:05.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad23bdd700 1 -- 172.21.15.102:0/1857056730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c19dd00 con 0x7fad1c0ffe30 2024-05-13T22:45:05.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad12ffd700 1 -- 172.21.15.102:0/1857056730 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fad0c010460 con 0x7fad1c0ffe30 2024-05-13T22:45:05.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad12ffd700 1 -- 172.21.15.102:0/1857056730 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad0c00f5f0 con 0x7fad1c0ffe30 2024-05-13T22:45:05.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.924+0000 7fad23bdd700 1 -- 172.21.15.102:0/1857056730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c19e250 con 0x7fad1c0ffe30 2024-05-13T22:45:05.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.925+0000 7fad23bdd700 1 -- 172.21.15.102:0/1857056730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c10b930 con 0x7fad1c0ffe30 2024-05-13T22:45:05.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.926+0000 7fad12ffd700 1 -- 172.21.15.102:0/1857056730 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fad0c0105d0 con 0x7fad1c0ffe30 2024-05-13T22:45:05.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.926+0000 7fad12ffd700 1 --2- 172.21.15.102:0/1857056730 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fad0806c4e0 0x7fad0806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:05.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.926+0000 7fad12ffd700 1 -- 172.21.15.102:0/1857056730 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fad0c08b4f0 con 0x7fad1c0ffe30 2024-05-13T22:45:05.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.926+0000 7fad21979700 1 --2- 172.21.15.102:0/1857056730 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fad0806c4e0 0x7fad0806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:05.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.927+0000 7fad21979700 1 --2- 172.21.15.102:0/1857056730 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fad0806c4e0 0x7fad0806e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fad1800b5c0 tx=0x7fad18005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:05.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:05.931+0000 7fad12ffd700 1 -- 172.21.15.102:0/1857056730 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fad0c059aa0 con 0x7fad1c0ffe30 2024-05-13T22:45:06.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.139+0000 7fad23bdd700 1 -- 172.21.15.102:0/1857056730 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fad1c109220 con 0x7fad0806c4e0 2024-05-13T22:45:06.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.141+0000 7fad12ffd700 1 -- 172.21.15.102:0/1857056730 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fad1c109220 con 0x7fad0806c4e0 2024-05-13T22:45:06.119 INFO:teuthology.orchestra.run.smithi102.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-13T22:45:06.119 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-13T22:45:06.119 INFO:teuthology.orchestra.run.smithi102.stdout:smithi130 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500Q1400BGN 372G No 69s ago Has a FileSystem, LVM detected 2024-05-13T22:45:06.121 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.143+0000 7fad23bdd700 1 -- 172.21.15.102:0/1857056730 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fad0806c4e0 msgr2=0x7fad0806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:06.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.143+0000 7fad23bdd700 1 --2- 172.21.15.102:0/1857056730 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fad0806c4e0 0x7fad0806e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fad1800b5c0 tx=0x7fad18005960 comp rx=0 tx=0).stop 2024-05-13T22:45:06.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.143+0000 7fad23bdd700 1 -- 172.21.15.102:0/1857056730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffe30 msgr2=0x7fad1c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:06.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.143+0000 7fad23bdd700 1 --2- 172.21.15.102:0/1857056730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffe30 0x7fad1c198af0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00d8d0 tx=0x7fad0c00dc90 comp rx=0 tx=0).stop 2024-05-13T22:45:06.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.143+0000 7fad23bdd700 1 -- 172.21.15.102:0/1857056730 shutdown_connections 2024-05-13T22:45:06.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.143+0000 7fad23bdd700 1 --2- 172.21.15.102:0/1857056730 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fad0806c4e0 0x7fad0806e9a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:06.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.143+0000 7fad23bdd700 1 --2- 172.21.15.102:0/1857056730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ff4d0 0x7fad1c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:06.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.143+0000 7fad23bdd700 1 --2- 172.21.15.102:0/1857056730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffe30 0x7fad1c198af0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:06.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.143+0000 7fad23bdd700 1 -- 172.21.15.102:0/1857056730 >> 172.21.15.102:0/1857056730 conn(0x7fad1c0fb050 msgr2=0x7fad1c107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:06.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.143+0000 7fad23bdd700 1 -- 172.21.15.102:0/1857056730 shutdown_connections 2024-05-13T22:45:06.122 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:06.143+0000 7fad23bdd700 1 -- 172.21.15.102:0/1857056730 wait complete. 2024-05-13T22:45:06.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:06 smithi130 ceph-mon[36591]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:06.418 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:06 smithi102 ceph-mon[28599]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:06.734 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-13T22:45:06.743 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:45:06.744 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-05-13T22:45:06.954 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:07.346 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:07 smithi102 ceph-mon[28599]: from='client.14472 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:07.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:07 smithi130 ceph-mon[36591]: from='client.14472 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:08.155 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:08 smithi102 ceph-mon[28599]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:08.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.420+0000 7eff437cf700 1 -- 172.21.15.102:0/1802979524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c101dc0 msgr2=0x7eff3c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:08.399 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.420+0000 7eff437cf700 1 --2- 172.21.15.102:0/1802979524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c101dc0 0x7eff3c102220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009910 tx=0x7eff2c009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.420+0000 7eff437cf700 1 -- 172.21.15.102:0/1802979524 shutdown_connections 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.420+0000 7eff437cf700 1 --2- 172.21.15.102:0/1802979524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c101dc0 0x7eff3c102220 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.420+0000 7eff437cf700 1 --2- 172.21.15.102:0/1802979524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff3c100bc0 0x7eff3c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.420+0000 7eff437cf700 1 -- 172.21.15.102:0/1802979524 >> 172.21.15.102:0/1802979524 conn(0x7eff3c0fc140 msgr2=0x7eff3c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.420+0000 7eff437cf700 1 -- 172.21.15.102:0/1802979524 shutdown_connections 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.420+0000 7eff437cf700 1 -- 172.21.15.102:0/1802979524 wait complete. 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff437cf700 1 Processor -- start 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff437cf700 1 -- start start 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff437cf700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff3c100bc0 0x7eff3c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff437cf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c101dc0 0x7eff3c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff437cf700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0736c0 con 0x7eff3c101dc0 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff437cf700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c073800 con 0x7eff3c100bc0 2024-05-13T22:45:08.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff40d6a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c101dc0 0x7eff3c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:08.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff40d6a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c101dc0 0x7eff3c073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:59658/0 (socket says 172.21.15.102:59658) 2024-05-13T22:45:08.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff40d6a700 1 -- 172.21.15.102:0/3251045738 learned_addr learned my addr 172.21.15.102:0/3251045738 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:08.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff4156b700 1 --2- 172.21.15.102:0/3251045738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff3c100bc0 0x7eff3c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:08.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff40d6a700 1 -- 172.21.15.102:0/3251045738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff3c100bc0 msgr2=0x7eff3c074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:08.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff40d6a700 1 --2- 172.21.15.102:0/3251045738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff3c100bc0 0x7eff3c074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:08.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff40d6a700 1 -- 172.21.15.102:0/3251045738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0095f0 con 0x7eff3c101dc0 2024-05-13T22:45:08.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff40d6a700 1 --2- 172.21.15.102:0/3251045738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c101dc0 0x7eff3c073180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006010 tx=0x7eff2c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:08.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:08 smithi130 ceph-mon[36591]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:08.402 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff327fc700 1 -- 172.21.15.102:0/3251045738 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff2c01d070 con 0x7eff3c101dc0 2024-05-13T22:45:08.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff437cf700 1 -- 172.21.15.102:0/3251045738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c073a80 con 0x7eff3c101dc0 2024-05-13T22:45:08.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff327fc700 1 -- 172.21.15.102:0/3251045738 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7eff2c00f460 con 0x7eff3c101dc0 2024-05-13T22:45:08.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff327fc700 1 -- 172.21.15.102:0/3251045738 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff2c021640 con 0x7eff3c101dc0 2024-05-13T22:45:08.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.421+0000 7eff437cf700 1 -- 172.21.15.102:0/3251045738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c073f70 con 0x7eff3c101dc0 2024-05-13T22:45:08.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.422+0000 7eff437cf700 1 -- 172.21.15.102:0/3251045738 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c1097c0 con 0x7eff3c101dc0 2024-05-13T22:45:08.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.423+0000 7eff327fc700 1 -- 172.21.15.102:0/3251045738 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7eff2c00fab0 con 0x7eff3c101dc0 2024-05-13T22:45:08.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.423+0000 7eff327fc700 1 --2- 172.21.15.102:0/3251045738 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7eff2806c4e0 0x7eff2806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:08.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.423+0000 7eff327fc700 1 -- 172.21.15.102:0/3251045738 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7eff2c08c8f0 con 0x7eff3c101dc0 2024-05-13T22:45:08.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.423+0000 7eff4156b700 1 --2- 172.21.15.102:0/3251045738 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7eff2806c4e0 0x7eff2806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:08.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.424+0000 7eff4156b700 1 --2- 172.21.15.102:0/3251045738 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7eff2806c4e0 0x7eff2806e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff38006fd0 tx=0x7eff38008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:08.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.428+0000 7eff327fc700 1 -- 172.21.15.102:0/3251045738 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7eff2c05b040 con 0x7eff3c101dc0 2024-05-13T22:45:08.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:08.643+0000 7eff437cf700 1 -- 172.21.15.102:0/3251045738 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7eff3c106710 con 0x7eff2806c4e0 2024-05-13T22:45:09.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:09 smithi130 ceph-mon[36591]: from='client.14476 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:09.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:09 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-13T22:45:09.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:09 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:45:09.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:09 smithi102 ceph-mon[28599]: from='client.14476 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:09.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:09 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-13T22:45:09.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:09 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:45:10.139 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.160+0000 7eff327fc700 1 -- 172.21.15.102:0/3251045738 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7eff3c106710 con 0x7eff2806c4e0 2024-05-13T22:45:10.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.163+0000 7eff437cf700 1 -- 172.21.15.102:0/3251045738 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7eff2806c4e0 msgr2=0x7eff2806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:10.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.163+0000 7eff437cf700 1 --2- 172.21.15.102:0/3251045738 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7eff2806c4e0 0x7eff2806e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff38006fd0 tx=0x7eff38008040 comp rx=0 tx=0).stop 2024-05-13T22:45:10.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.163+0000 7eff437cf700 1 -- 172.21.15.102:0/3251045738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c101dc0 msgr2=0x7eff3c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:10.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.163+0000 7eff437cf700 1 --2- 172.21.15.102:0/3251045738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c101dc0 0x7eff3c073180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006010 tx=0x7eff2c00be20 comp rx=0 tx=0).stop 2024-05-13T22:45:10.142 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.164+0000 7eff437cf700 1 -- 172.21.15.102:0/3251045738 shutdown_connections 2024-05-13T22:45:10.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.164+0000 7eff437cf700 1 --2- 172.21.15.102:0/3251045738 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7eff2806c4e0 0x7eff2806e9a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:10.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.164+0000 7eff437cf700 1 --2- 172.21.15.102:0/3251045738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff3c100bc0 0x7eff3c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:10.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.164+0000 7eff437cf700 1 --2- 172.21.15.102:0/3251045738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff3c101dc0 0x7eff3c073180 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:10.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.164+0000 7eff437cf700 1 -- 172.21.15.102:0/3251045738 >> 172.21.15.102:0/3251045738 conn(0x7eff3c0fc140 msgr2=0x7eff3c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:10.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.164+0000 7eff437cf700 1 -- 172.21.15.102:0/3251045738 shutdown_connections 2024-05-13T22:45:10.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:10.164+0000 7eff437cf700 1 -- 172.21.15.102:0/3251045738 wait complete. 2024-05-13T22:45:10.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:10 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-13T22:45:10.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:10 smithi130 ceph-mon[36591]: osdmap e42: 6 total, 6 up, 6 in 2024-05-13T22:45:10.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:10 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-13T22:45:10.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:10 smithi130 ceph-mon[36591]: pgmap v173: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:10.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:10 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-13T22:45:10.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:10 smithi102 ceph-mon[28599]: osdmap e42: 6 total, 6 up, 6 in 2024-05-13T22:45:10.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:10 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-13T22:45:10.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:10 smithi102 ceph-mon[28599]: pgmap v173: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:10.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:10 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102[28595]: 2024-05-13T22:45:10.143+0000 7efcba4ce700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-13T22:45:10.852 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-13T22:45:11.070 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:11.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-13T22:45:11.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-13T22:45:11.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: osdmap e43: 6 total, 6 up, 6 in 2024-05-13T22:45:11.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-13T22:45:11.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: osdmap e44: 6 total, 6 up, 6 in 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: fsmap cephfs:0 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: Saving service mds.cephfs spec with placement count:4 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.cgkcwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.cgkcwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-13T22:45:11.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:11 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-13T22:45:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-13T22:45:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: osdmap e43: 6 total, 6 up, 6 in 2024-05-13T22:45:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-13T22:45:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-13T22:45:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-13T22:45:11.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: osdmap e44: 6 total, 6 up, 6 in 2024-05-13T22:45:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-13T22:45:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: fsmap cephfs:0 2024-05-13T22:45:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: Saving service mds.cephfs spec with placement count:4 2024-05-13T22:45:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:45:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:45:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.cgkcwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:45:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.cgkcwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-13T22:45:11.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:11 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:12.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:12 smithi102 ceph-mon[28599]: Deploying daemon mds.cephfs.smithi102.cgkcwz on smithi102 2024-05-13T22:45:12.401 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:12 smithi102 ceph-mon[28599]: osdmap e45: 6 total, 6 up, 6 in 2024-05-13T22:45:12.401 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:12 smithi102 ceph-mon[28599]: pgmap v177: 65 pgs: 8 creating+activating, 6 active+clean, 51 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:12.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:12 smithi130 ceph-mon[36591]: Deploying daemon mds.cephfs.smithi102.cgkcwz on smithi102 2024-05-13T22:45:12.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:12 smithi130 ceph-mon[36591]: osdmap e45: 6 total, 6 up, 6 in 2024-05-13T22:45:12.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:12 smithi130 ceph-mon[36591]: pgmap v177: 65 pgs: 8 creating+activating, 6 active+clean, 51 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:12.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.666+0000 7f78c5463700 1 -- 172.21.15.102:0/3975049204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c0073790 msgr2=0x7f78c0073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:12.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.666+0000 7f78c5463700 1 --2- 172.21.15.102:0/3975049204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c0073790 0x7f78c0073c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009910 tx=0x7f78b4009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:12.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.666+0000 7f78c5463700 1 -- 172.21.15.102:0/3975049204 shutdown_connections 2024-05-13T22:45:12.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.666+0000 7f78c5463700 1 --2- 172.21.15.102:0/3975049204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c0073790 0x7f78c0073c10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:12.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.666+0000 7f78c5463700 1 --2- 172.21.15.102:0/3975049204 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c0074d60 0x7f78c00731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:12.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.666+0000 7f78c5463700 1 -- 172.21.15.102:0/3975049204 >> 172.21.15.102:0/3975049204 conn(0x7f78c00fc140 msgr2=0x7f78c00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:12.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.667+0000 7f78c5463700 1 -- 172.21.15.102:0/3975049204 shutdown_connections 2024-05-13T22:45:12.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.667+0000 7f78c5463700 1 -- 172.21.15.102:0/3975049204 wait complete. 2024-05-13T22:45:12.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.667+0000 7f78c5463700 1 Processor -- start 2024-05-13T22:45:12.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.667+0000 7f78c5463700 1 -- start start 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.667+0000 7f78c5463700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c0073790 0x7f78c01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.667+0000 7f78c5463700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c0074d60 0x7f78c0198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.667+0000 7f78c5463700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c0199140 con 0x7f78c0074d60 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.667+0000 7f78c5463700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c0199280 con 0x7f78c0073790 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78be7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c0074d60 0x7f78c0198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78be7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c0074d60 0x7f78c0198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42366/0 (socket says 172.21.15.102:42366) 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78be7fc700 1 -- 172.21.15.102:0/846582228 learned_addr learned my addr 172.21.15.102:0/846582228 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78beffd700 1 --2- 172.21.15.102:0/846582228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c0073790 0x7f78c01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78be7fc700 1 -- 172.21.15.102:0/846582228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c0073790 msgr2=0x7f78c01985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78be7fc700 1 --2- 172.21.15.102:0/846582228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c0073790 0x7f78c01985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78be7fc700 1 -- 172.21.15.102:0/846582228 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40095f0 con 0x7f78c0074d60 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78be7fc700 1 --2- 172.21.15.102:0/846582228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c0074d60 0x7f78c0198b20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006010 tx=0x7f78b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78a7fff700 1 -- 172.21.15.102:0/846582228 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78b401d070 con 0x7f78c0074d60 2024-05-13T22:45:12.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78a7fff700 1 -- 172.21.15.102:0/846582228 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f78b400f460 con 0x7f78c0074d60 2024-05-13T22:45:12.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78a7fff700 1 -- 172.21.15.102:0/846582228 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78b4021640 con 0x7f78c0074d60 2024-05-13T22:45:12.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78c5463700 1 -- 172.21.15.102:0/846582228 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c019dcd0 con 0x7f78c0074d60 2024-05-13T22:45:12.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.668+0000 7f78c5463700 1 -- 172.21.15.102:0/846582228 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c019e1f0 con 0x7f78c0074d60 2024-05-13T22:45:12.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.669+0000 7f78c5463700 1 -- 172.21.15.102:0/846582228 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c010b9b0 con 0x7f78c0074d60 2024-05-13T22:45:12.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.670+0000 7f78a7fff700 1 -- 172.21.15.102:0/846582228 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f78b400fab0 con 0x7f78c0074d60 2024-05-13T22:45:12.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.670+0000 7f78a7fff700 1 --2- 172.21.15.102:0/846582228 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78a806c4e0 0x7f78a806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:12.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.670+0000 7f78beffd700 1 --2- 172.21.15.102:0/846582228 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78a806c4e0 0x7f78a806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:12.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.670+0000 7f78a7fff700 1 -- 172.21.15.102:0/846582228 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f78b408cbe0 con 0x7f78c0074d60 2024-05-13T22:45:12.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.671+0000 7f78beffd700 1 --2- 172.21.15.102:0/846582228 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78a806c4e0 0x7f78a806e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009a10 tx=0x7f78b0009190 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:12.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.675+0000 7f78a7fff700 1 -- 172.21.15.102:0/846582228 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f78b405ae50 con 0x7f78c0074d60 2024-05-13T22:45:12.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.912+0000 7f78c5463700 1 -- 172.21.15.102:0/846582228 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f78c019e800 con 0x7f78c0074d60 2024-05-13T22:45:12.891 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.912+0000 7f78a7fff700 1 -- 172.21.15.102:0/846582228 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1093 (secure 0 0 0) 0x7f78b405a9e0 con 0x7f78c0074d60 2024-05-13T22:45:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:e2 2024-05-13T22:45:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:45:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:45:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:45:12.891 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:45:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:45:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:45:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 2 2024-05-13T22:45:12.891 INFO:teuthology.orchestra.run.smithi102.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-13T22:45:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:45:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:45:10.143970+0000 2024-05-13T22:45:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:45:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:45:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:45:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:45:12.892 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:45:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:45:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:45:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 0 2024-05-13T22:45:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:45:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:45:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:in 2024-05-13T22:45:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:up {} 2024-05-13T22:45:12.893 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:45:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:45:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:45:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:45:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:45:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:45:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:45:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:45:12.894 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 0 2024-05-13T22:45:12.895 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:45:12.895 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:45:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.914+0000 7f78c5463700 1 -- 172.21.15.102:0/846582228 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78a806c4e0 msgr2=0x7f78a806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.914+0000 7f78c5463700 1 --2- 172.21.15.102:0/846582228 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78a806c4e0 0x7f78a806e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009a10 tx=0x7f78b0009190 comp rx=0 tx=0).stop 2024-05-13T22:45:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.914+0000 7f78c5463700 1 -- 172.21.15.102:0/846582228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c0074d60 msgr2=0x7f78c0198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.914+0000 7f78c5463700 1 --2- 172.21.15.102:0/846582228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c0074d60 0x7f78c0198b20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006010 tx=0x7f78b400bd70 comp rx=0 tx=0).stop 2024-05-13T22:45:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.914+0000 7f78c5463700 1 -- 172.21.15.102:0/846582228 shutdown_connections 2024-05-13T22:45:12.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.914+0000 7f78c5463700 1 --2- 172.21.15.102:0/846582228 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f78a806c4e0 0x7f78a806e9a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.914+0000 7f78c5463700 1 --2- 172.21.15.102:0/846582228 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c0073790 0x7f78c01985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.914+0000 7f78c5463700 1 --2- 172.21.15.102:0/846582228 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c0074d60 0x7f78c0198b20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.914+0000 7f78c5463700 1 -- 172.21.15.102:0/846582228 >> 172.21.15.102:0/846582228 conn(0x7f78c00fc140 msgr2=0x7f78c0106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.914+0000 7f78c5463700 1 -- 172.21.15.102:0/846582228 shutdown_connections 2024-05-13T22:45:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:12.914+0000 7f78c5463700 1 -- 172.21.15.102:0/846582228 wait complete. 2024-05-13T22:45:12.897 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 2 2024-05-13T22:45:13.232 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:13 smithi102 ceph-mon[28599]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-13T22:45:13.233 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:13 smithi102 ceph-mon[28599]: osdmap e46: 6 total, 6 up, 6 in 2024-05-13T22:45:13.233 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:13 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/846582228' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:45:13.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:13 smithi130 ceph-mon[36591]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-13T22:45:13.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:13 smithi130 ceph-mon[36591]: osdmap e46: 6 total, 6 up, 6 in 2024-05-13T22:45:13.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:13 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/846582228' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:45:14.358 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-13T22:45:14.367 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:45:14.367 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-05-13T22:45:14.391 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:14 smithi102 ceph-mon[28599]: pgmap v179: 65 pgs: 8 creating+activating, 48 active+clean, 9 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:14.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:14 smithi130 ceph-mon[36591]: pgmap v179: 65 pgs: 8 creating+activating, 48 active+clean, 9 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:14.897 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:16.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:16.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:16.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:16.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.hzjanp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:45:16.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.hzjanp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-13T22:45:16.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:16.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: Deploying daemon mds.cephfs.smithi130.hzjanp on smithi130 2024-05-13T22:45:16.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: daemon mds.cephfs.smithi102.cgkcwz assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-13T22:45:16.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-13T22:45:16.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-13T22:45:16.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: Cluster is now healthy 2024-05-13T22:45:16.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.102:6826/3231884180,v1:172.21.15.102:6827/3231884180] up:boot 2024-05-13T22:45:16.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:creating} 2024-05-13T22:45:16.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:45:16.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:15 smithi130 ceph-mon[36591]: daemon mds.cephfs.smithi102.cgkcwz is now active in filesystem cephfs as rank 0 2024-05-13T22:45:16.158 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:16.158 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:16.158 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:16.158 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.hzjanp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:45:16.158 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.hzjanp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-13T22:45:16.159 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:16.159 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: Deploying daemon mds.cephfs.smithi130.hzjanp on smithi130 2024-05-13T22:45:16.159 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: daemon mds.cephfs.smithi102.cgkcwz assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-13T22:45:16.159 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-13T22:45:16.159 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-13T22:45:16.159 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: Cluster is now healthy 2024-05-13T22:45:16.159 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.102:6826/3231884180,v1:172.21.15.102:6827/3231884180] up:boot 2024-05-13T22:45:16.159 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:creating} 2024-05-13T22:45:16.159 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:45:16.159 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:15 smithi102 ceph-mon[28599]: daemon mds.cephfs.smithi102.cgkcwz is now active in filesystem cephfs as rank 0 2024-05-13T22:45:16.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.281+0000 7f17293a0700 1 -- 172.21.15.102:0/1556745481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1724100cb0 msgr2=0x7f1724101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.281+0000 7f17293a0700 1 --2- 172.21.15.102:0/1556745481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1724100cb0 0x7f1724101110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1714009910 tx=0x7f1714009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.281+0000 7f17293a0700 1 -- 172.21.15.102:0/1556745481 shutdown_connections 2024-05-13T22:45:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.281+0000 7f17293a0700 1 --2- 172.21.15.102:0/1556745481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1724100cb0 0x7f1724101110 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.281+0000 7f17293a0700 1 --2- 172.21.15.102:0/1556745481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17240ffab0 0x7f17240ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.281+0000 7f17293a0700 1 -- 172.21.15.102:0/1556745481 >> 172.21.15.102:0/1556745481 conn(0x7f17240fb030 msgr2=0x7f17240fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.281+0000 7f17293a0700 1 -- 172.21.15.102:0/1556745481 shutdown_connections 2024-05-13T22:45:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.281+0000 7f17293a0700 1 -- 172.21.15.102:0/1556745481 wait complete. 2024-05-13T22:45:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.282+0000 7f17293a0700 1 Processor -- start 2024-05-13T22:45:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.282+0000 7f17293a0700 1 -- start start 2024-05-13T22:45:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.282+0000 7f17293a0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17240ffab0 0x7f17241963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:16.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.282+0000 7f17293a0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1724100cb0 0x7f1724196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.282+0000 7f17293a0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724196f40 con 0x7f1724100cb0 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.282+0000 7f17293a0700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724197080 con 0x7f17240ffab0 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.282+0000 7f17227fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1724100cb0 0x7f1724196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.282+0000 7f17227fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1724100cb0 0x7f1724196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42390/0 (socket says 172.21.15.102:42390) 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.282+0000 7f17227fc700 1 -- 172.21.15.102:0/3033507795 learned_addr learned my addr 172.21.15.102:0/3033507795 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.282+0000 7f1722ffd700 1 --2- 172.21.15.102:0/3033507795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17240ffab0 0x7f17241963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.283+0000 7f17227fc700 1 -- 172.21.15.102:0/3033507795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17240ffab0 msgr2=0x7f17241963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.283+0000 7f17227fc700 1 --2- 172.21.15.102:0/3033507795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17240ffab0 0x7f17241963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.283+0000 7f17227fc700 1 -- 172.21.15.102:0/3033507795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17140095f0 con 0x7f1724100cb0 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.283+0000 7f17227fc700 1 --2- 172.21.15.102:0/3033507795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1724100cb0 0x7f1724196920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1714006010 tx=0x7f171400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.283+0000 7f171bfff700 1 -- 172.21.15.102:0/3033507795 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f171401d070 con 0x7f1724100cb0 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.283+0000 7f171bfff700 1 -- 172.21.15.102:0/3033507795 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f171400f460 con 0x7f1724100cb0 2024-05-13T22:45:16.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.283+0000 7f17293a0700 1 -- 172.21.15.102:0/3033507795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172419bad0 con 0x7f1724100cb0 2024-05-13T22:45:16.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.283+0000 7f171bfff700 1 -- 172.21.15.102:0/3033507795 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1714021640 con 0x7f1724100cb0 2024-05-13T22:45:16.264 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.283+0000 7f17293a0700 1 -- 172.21.15.102:0/3033507795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172419c040 con 0x7f1724100cb0 2024-05-13T22:45:16.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.284+0000 7f17293a0700 1 -- 172.21.15.102:0/3033507795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17241086b0 con 0x7f1724100cb0 2024-05-13T22:45:16.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.284+0000 7f171bfff700 1 -- 172.21.15.102:0/3033507795 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f171400fab0 con 0x7f1724100cb0 2024-05-13T22:45:16.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.285+0000 7f171bfff700 1 --2- 172.21.15.102:0/3033507795 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f171006c4e0 0x7f171006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:16.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.285+0000 7f171bfff700 1 -- 172.21.15.102:0/3033507795 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f171408cbf0 con 0x7f1724100cb0 2024-05-13T22:45:16.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.285+0000 7f1722ffd700 1 --2- 172.21.15.102:0/3033507795 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f171006c4e0 0x7f171006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:16.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.285+0000 7f1722ffd700 1 --2- 172.21.15.102:0/3033507795 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f171006c4e0 0x7f171006e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1724100b10 tx=0x7f170c009190 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:16.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.290+0000 7f171bfff700 1 -- 172.21.15.102:0/3033507795 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f171405ae60 con 0x7f1724100cb0 2024-05-13T22:45:16.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:16.522+0000 7f17293a0700 1 -- 172.21.15.102:0/3033507795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f172419c740 con 0x7f1724100cb0 2024-05-13T22:45:17.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.124+0000 7f171bfff700 1 -- 172.21.15.102:0/3033507795 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v5) v1 ==== 105+0+0 (secure 0 0 0) 0x7f171405a9f0 con 0x7f1724100cb0 2024-05-13T22:45:17.105 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.127+0000 7f17293a0700 1 -- 172.21.15.102:0/3033507795 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f171006c4e0 msgr2=0x7f171006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:17.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.127+0000 7f17293a0700 1 --2- 172.21.15.102:0/3033507795 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f171006c4e0 0x7f171006e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1724100b10 tx=0x7f170c009190 comp rx=0 tx=0).stop 2024-05-13T22:45:17.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.127+0000 7f17293a0700 1 -- 172.21.15.102:0/3033507795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1724100cb0 msgr2=0x7f1724196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:17.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.127+0000 7f17293a0700 1 --2- 172.21.15.102:0/3033507795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1724100cb0 0x7f1724196920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1714006010 tx=0x7f171400bd70 comp rx=0 tx=0).stop 2024-05-13T22:45:17.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.127+0000 7f17293a0700 1 -- 172.21.15.102:0/3033507795 shutdown_connections 2024-05-13T22:45:17.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.127+0000 7f17293a0700 1 --2- 172.21.15.102:0/3033507795 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f171006c4e0 0x7f171006e9a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:17.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.127+0000 7f17293a0700 1 --2- 172.21.15.102:0/3033507795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17240ffab0 0x7f17241963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:17.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.127+0000 7f17293a0700 1 --2- 172.21.15.102:0/3033507795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1724100cb0 0x7f1724196920 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:17.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.127+0000 7f17293a0700 1 -- 172.21.15.102:0/3033507795 >> 172.21.15.102:0/3033507795 conn(0x7f17240fb030 msgr2=0x7f1724068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:17.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.127+0000 7f17293a0700 1 -- 172.21.15.102:0/3033507795 shutdown_connections 2024-05-13T22:45:17.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:17.127+0000 7f17293a0700 1 -- 172.21.15.102:0/3033507795 wait complete. 2024-05-13T22:45:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:16 smithi130 ceph-mon[36591]: pgmap v180: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:16 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.102:6826/3231884180,v1:172.21.15.102:6827/3231884180] up:active 2024-05-13T22:45:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:16 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 2024-05-13T22:45:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:16 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3033507795' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-05-13T22:45:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:16 smithi102 ceph-mon[28599]: pgmap v180: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:45:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:16 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.102:6826/3231884180,v1:172.21.15.102:6827/3231884180] up:active 2024-05-13T22:45:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:16 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 2024-05-13T22:45:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:16 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3033507795' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-05-13T22:45:17.748 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-13T22:45:17.757 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:45:17.757 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-05-13T22:45:18.329 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:18 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3033507795' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-05-13T22:45:18.329 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:18 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 2024-05-13T22:45:18.329 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:18 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:18.329 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:18 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:18.329 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:18 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:18.330 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:18 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.sdylgv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:45:18.330 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:18 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.sdylgv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-13T22:45:18.330 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:18 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:18.330 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:18 smithi102 ceph-mon[28599]: Deploying daemon mds.cephfs.smithi102.sdylgv on smithi102 2024-05-13T22:45:18.330 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:18 smithi102 ceph-mon[28599]: pgmap v181: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 270 B/s wr, 0 op/s 2024-05-13T22:45:18.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:18 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3033507795' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-05-13T22:45:18.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:18 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 2024-05-13T22:45:18.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:18 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:18.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:18 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:18.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:18 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:18.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:18 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.sdylgv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:45:18.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:18 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.sdylgv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-13T22:45:18.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:18 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:18.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:18 smithi130 ceph-mon[36591]: Deploying daemon mds.cephfs.smithi102.sdylgv on smithi102 2024-05-13T22:45:18.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:18 smithi130 ceph-mon[36591]: pgmap v181: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 270 B/s wr, 0 op/s 2024-05-13T22:45:18.527 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:19.328 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:19 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.130:6824/1291036486,v1:172.21.15.130:6825/1291036486] up:boot 2024-05-13T22:45:19.329 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:19 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby 2024-05-13T22:45:19.329 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:19 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:45:19.329 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:19 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby 2024-05-13T22:45:19.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:19 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.130:6824/1291036486,v1:172.21.15.130:6825/1291036486] up:boot 2024-05-13T22:45:19.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:19 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby 2024-05-13T22:45:19.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:19 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:45:19.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:19 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby 2024-05-13T22:45:20.275 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:20 smithi102 ceph-mon[28599]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-13T22:45:20.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:20 smithi130 ceph-mon[36591]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-13T22:45:20.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.489+0000 7f88f5037700 1 -- 172.21.15.102:0/2001465559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f00ffab0 msgr2=0x7f88f00ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:20.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.489+0000 7f88f5037700 1 --2- 172.21.15.102:0/2001465559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f00ffab0 0x7f88f00ffed0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009910 tx=0x7f88e0009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:20.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.489+0000 7f88f5037700 1 -- 172.21.15.102:0/2001465559 shutdown_connections 2024-05-13T22:45:20.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.489+0000 7f88f5037700 1 --2- 172.21.15.102:0/2001465559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88f0100cb0 0x7f88f0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:20.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.489+0000 7f88f5037700 1 --2- 172.21.15.102:0/2001465559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f00ffab0 0x7f88f00ffed0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:20.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.489+0000 7f88f5037700 1 -- 172.21.15.102:0/2001465559 >> 172.21.15.102:0/2001465559 conn(0x7f88f00fb030 msgr2=0x7f88f00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:20.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.489+0000 7f88f5037700 1 -- 172.21.15.102:0/2001465559 shutdown_connections 2024-05-13T22:45:20.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.489+0000 7f88f5037700 1 -- 172.21.15.102:0/2001465559 wait complete. 2024-05-13T22:45:20.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.490+0000 7f88f5037700 1 Processor -- start 2024-05-13T22:45:20.469 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.490+0000 7f88f5037700 1 -- start start 2024-05-13T22:45:20.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88f5037700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88f00ffab0 0x7f88f0196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:20.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88f5037700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f0100cb0 0x7f88f0196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:20.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88f5037700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f0196e80 con 0x7f88f0100cb0 2024-05-13T22:45:20.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88f5037700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f0196fc0 con 0x7f88f00ffab0 2024-05-13T22:45:20.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88ee59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f0100cb0 0x7f88f0196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:20.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88ee59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f0100cb0 0x7f88f0196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52034/0 (socket says 172.21.15.102:52034) 2024-05-13T22:45:20.471 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88ee59c700 1 -- 172.21.15.102:0/2226565499 learned_addr learned my addr 172.21.15.102:0/2226565499 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88eed9d700 1 --2- 172.21.15.102:0/2226565499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88f00ffab0 0x7f88f0196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88ee59c700 1 -- 172.21.15.102:0/2226565499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88f00ffab0 msgr2=0x7f88f0196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88ee59c700 1 --2- 172.21.15.102:0/2226565499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88f00ffab0 0x7f88f0196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88ee59c700 1 -- 172.21.15.102:0/2226565499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e00095f0 con 0x7f88f0100cb0 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88ee59c700 1 --2- 172.21.15.102:0/2226565499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f0100cb0 0x7f88f0196860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f88e400d8d0 tx=0x7f88e400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88d7fff700 1 -- 172.21.15.102:0/2226565499 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88e40098a0 con 0x7f88f0100cb0 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88d7fff700 1 -- 172.21.15.102:0/2226565499 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f88e4010460 con 0x7f88f0100cb0 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88f5037700 1 -- 172.21.15.102:0/2226565499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f019ba70 con 0x7f88f0100cb0 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88d7fff700 1 -- 172.21.15.102:0/2226565499 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88e400f5f0 con 0x7f88f0100cb0 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.491+0000 7f88f5037700 1 -- 172.21.15.102:0/2226565499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f019bfc0 con 0x7f88f0100cb0 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.492+0000 7f88f5037700 1 -- 172.21.15.102:0/2226565499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f0108610 con 0x7f88f0100cb0 2024-05-13T22:45:20.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.493+0000 7f88d7fff700 1 -- 172.21.15.102:0/2226565499 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f88e4009a00 con 0x7f88f0100cb0 2024-05-13T22:45:20.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.493+0000 7f88d7fff700 1 --2- 172.21.15.102:0/2226565499 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88d806c4e0 0x7f88d806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:20.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.493+0000 7f88eed9d700 1 --2- 172.21.15.102:0/2226565499 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88d806c4e0 0x7f88d806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:20.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.493+0000 7f88d7fff700 1 -- 172.21.15.102:0/2226565499 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f88e408b710 con 0x7f88f0100cb0 2024-05-13T22:45:20.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.494+0000 7f88eed9d700 1 --2- 172.21.15.102:0/2226565499 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88d806c4e0 0x7f88d806e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f88e0006010 tx=0x7f88e000b560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:20.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.498+0000 7f88d7fff700 1 -- 172.21.15.102:0/2226565499 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f88e4059980 con 0x7f88f0100cb0 2024-05-13T22:45:20.723 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:20.745+0000 7f88f5037700 1 -- 172.21.15.102:0/2226565499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f88f0108800 con 0x7f88f0100cb0 2024-05-13T22:45:21.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.146+0000 7f88d7fff700 1 -- 172.21.15.102:0/2226565499 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v8) v1 ==== 121+0+0 (secure 0 0 0) 0x7f88e4059510 con 0x7f88f0100cb0 2024-05-13T22:45:21.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.148+0000 7f88f5037700 1 -- 172.21.15.102:0/2226565499 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88d806c4e0 msgr2=0x7f88d806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:21.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.148+0000 7f88f5037700 1 --2- 172.21.15.102:0/2226565499 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88d806c4e0 0x7f88d806e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f88e0006010 tx=0x7f88e000b560 comp rx=0 tx=0).stop 2024-05-13T22:45:21.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.148+0000 7f88f5037700 1 -- 172.21.15.102:0/2226565499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f0100cb0 msgr2=0x7f88f0196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:21.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.148+0000 7f88f5037700 1 --2- 172.21.15.102:0/2226565499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f0100cb0 0x7f88f0196860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f88e400d8d0 tx=0x7f88e400dbe0 comp rx=0 tx=0).stop 2024-05-13T22:45:21.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.148+0000 7f88f5037700 1 -- 172.21.15.102:0/2226565499 shutdown_connections 2024-05-13T22:45:21.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.148+0000 7f88f5037700 1 --2- 172.21.15.102:0/2226565499 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f88d806c4e0 0x7f88d806e9a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:21.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.148+0000 7f88f5037700 1 --2- 172.21.15.102:0/2226565499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f88f00ffab0 0x7f88f0196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:21.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.148+0000 7f88f5037700 1 --2- 172.21.15.102:0/2226565499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88f0100cb0 0x7f88f0196860 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:21.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.148+0000 7f88f5037700 1 -- 172.21.15.102:0/2226565499 >> 172.21.15.102:0/2226565499 conn(0x7f88f00fb030 msgr2=0x7f88f00683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:21.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.148+0000 7f88f5037700 1 -- 172.21.15.102:0/2226565499 shutdown_connections 2024-05-13T22:45:21.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:21.148+0000 7f88f5037700 1 -- 172.21.15.102:0/2226565499 wait complete. 2024-05-13T22:45:21.138 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:21 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2226565499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-13T22:45:21.138 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:21 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:21.138 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:21 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:21.138 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:21 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:21.138 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:21 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.ovlxvg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:45:21.139 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:21 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.ovlxvg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-13T22:45:21.139 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:21 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:21.139 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:21 smithi102 ceph-mon[28599]: Deploying daemon mds.cephfs.smithi130.ovlxvg on smithi130 2024-05-13T22:45:21.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:21 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2226565499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-13T22:45:21.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:21 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:21.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:21 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:21.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:21 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:21.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:21 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.ovlxvg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:45:21.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:21 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.ovlxvg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-13T22:45:21.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:21 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:21.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:21 smithi130 ceph-mon[36591]: Deploying daemon mds.cephfs.smithi130.ovlxvg on smithi130 2024-05-13T22:45:21.791 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-13T22:45:21.800 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:45:21.801 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-05-13T22:45:22.009 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:22 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2226565499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-13T22:45:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:22 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] up:boot 2024-05-13T22:45:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:22 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 2 up:standby 2024-05-13T22:45:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:22 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:45:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:22 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 1 up:standby 2024-05-13T22:45:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:22 smithi130 ceph-mon[36591]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-13T22:45:22.423 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:22 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2226565499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-13T22:45:22.423 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:22 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] up:boot 2024-05-13T22:45:22.423 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:22 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 2 up:standby 2024-05-13T22:45:22.423 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:22 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:45:22.423 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:22 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 1 up:standby 2024-05-13T22:45:22.424 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:22 smithi102 ceph-mon[28599]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-13T22:45:23.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.597+0000 7f8b884f9700 1 -- 172.21.15.102:0/3915135088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b80100bc0 msgr2=0x7f8b80100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.597+0000 7f8b884f9700 1 --2- 172.21.15.102:0/3915135088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b80100bc0 0x7f8b80100fe0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009910 tx=0x7f8b7c009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.597+0000 7f8b884f9700 1 -- 172.21.15.102:0/3915135088 shutdown_connections 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.597+0000 7f8b884f9700 1 --2- 172.21.15.102:0/3915135088 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b80101dc0 0x7f8b80102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.597+0000 7f8b884f9700 1 --2- 172.21.15.102:0/3915135088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b80100bc0 0x7f8b80100fe0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.597+0000 7f8b884f9700 1 -- 172.21.15.102:0/3915135088 >> 172.21.15.102:0/3915135088 conn(0x7f8b800fc140 msgr2=0x7f8b800fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.597+0000 7f8b884f9700 1 -- 172.21.15.102:0/3915135088 shutdown_connections 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.597+0000 7f8b884f9700 1 -- 172.21.15.102:0/3915135088 wait complete. 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.598+0000 7f8b884f9700 1 Processor -- start 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.598+0000 7f8b884f9700 1 -- start start 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.598+0000 7f8b884f9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b80100bc0 0x7f8b80196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.598+0000 7f8b884f9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b80101dc0 0x7f8b801968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:23.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.598+0000 7f8b884f9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80196ed0 con 0x7f8b80100bc0 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.598+0000 7f8b884f9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80197010 con 0x7f8b80101dc0 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.598+0000 7f8b86295700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b80100bc0 0x7f8b80196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.598+0000 7f8b86295700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b80100bc0 0x7f8b80196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52074/0 (socket says 172.21.15.102:52074) 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b86295700 1 -- 172.21.15.102:0/2874415034 learned_addr learned my addr 172.21.15.102:0/2874415034 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b85a94700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b80101dc0 0x7f8b801968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b86295700 1 -- 172.21.15.102:0/2874415034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b80101dc0 msgr2=0x7f8b801968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b86295700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b80101dc0 0x7f8b801968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b86295700 1 -- 172.21.15.102:0/2874415034 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0095f0 con 0x7f8b80100bc0 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b85a94700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b80101dc0 0x7f8b801968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b86295700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b80100bc0 0x7f8b80196370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00bda0 tx=0x7f8b7c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b777fe700 1 -- 172.21.15.102:0/2874415034 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b7c01d070 con 0x7f8b80100bc0 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b884f9700 1 -- 172.21.15.102:0/2874415034 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8019ba60 con 0x7f8b80100bc0 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b777fe700 1 -- 172.21.15.102:0/2874415034 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8b7c00f460 con 0x7f8b80100bc0 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b777fe700 1 -- 172.21.15.102:0/2874415034 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b7c021640 con 0x7f8b80100bc0 2024-05-13T22:45:23.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.599+0000 7f8b884f9700 1 -- 172.21.15.102:0/2874415034 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8019bf50 con 0x7f8b80100bc0 2024-05-13T22:45:23.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.600+0000 7f8b884f9700 1 -- 172.21.15.102:0/2874415034 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b80109720 con 0x7f8b80100bc0 2024-05-13T22:45:23.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.601+0000 7f8b777fe700 1 -- 172.21.15.102:0/2874415034 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8b7c00fa00 con 0x7f8b80100bc0 2024-05-13T22:45:23.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.601+0000 7f8b777fe700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b6c06c4e0 0x7f8b6c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:23.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.601+0000 7f8b777fe700 1 -- 172.21.15.102:0/2874415034 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f8b7c08cb20 con 0x7f8b80100bc0 2024-05-13T22:45:23.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.601+0000 7f8b85a94700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b6c06c4e0 0x7f8b6c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:23.581 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.602+0000 7f8b85a94700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b6c06c4e0 0x7f8b6c06e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8b70005fd0 tx=0x7f8b70005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:23.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.606+0000 7f8b777fe700 1 -- 172.21.15.102:0/2874415034 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8b7c05ad90 con 0x7f8b80100bc0 2024-05-13T22:45:23.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:23.852+0000 7f8b884f9700 1 -- 172.21.15.102:0/2874415034 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f8b801098e0 con 0x7f8b80100bc0 2024-05-13T22:45:24.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.766+0000 7f8b777fe700 1 -- 172.21.15.102:0/2874415034 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v11) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8b7c05a920 con 0x7f8b80100bc0 2024-05-13T22:45:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.769+0000 7f8b884f9700 1 -- 172.21.15.102:0/2874415034 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b6c06c4e0 msgr2=0x7f8b6c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.769+0000 7f8b884f9700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b6c06c4e0 0x7f8b6c06e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8b70005fd0 tx=0x7f8b70005ee0 comp rx=0 tx=0).stop 2024-05-13T22:45:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.769+0000 7f8b884f9700 1 -- 172.21.15.102:0/2874415034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b80100bc0 msgr2=0x7f8b80196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:24.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.769+0000 7f8b884f9700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b80100bc0 0x7f8b80196370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00bda0 tx=0x7f8b7c00be80 comp rx=0 tx=0).stop 2024-05-13T22:45:24.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.769+0000 7f8b884f9700 1 -- 172.21.15.102:0/2874415034 shutdown_connections 2024-05-13T22:45:24.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.769+0000 7f8b884f9700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8b6c06c4e0 0x7f8b6c06e9a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:24.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.769+0000 7f8b884f9700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b80100bc0 0x7f8b80196370 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:24.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.769+0000 7f8b884f9700 1 --2- 172.21.15.102:0/2874415034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b80101dc0 0x7f8b801968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:24.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.769+0000 7f8b884f9700 1 -- 172.21.15.102:0/2874415034 >> 172.21.15.102:0/2874415034 conn(0x7f8b800fc140 msgr2=0x7f8b80104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:24.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.769+0000 7f8b884f9700 1 -- 172.21.15.102:0/2874415034 shutdown_connections 2024-05-13T22:45:24.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:24.769+0000 7f8b884f9700 1 -- 172.21.15.102:0/2874415034 wait complete. 2024-05-13T22:45:24.748 INFO:teuthology.orchestra.run.smithi102.stderr:inline data disabled 2024-05-13T22:45:24.758 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] up:boot 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 709 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2874415034' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-05-13T22:45:24.759 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:24 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:45:24.803 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] up:boot 2024-05-13T22:45:24.803 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:45:24.803 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:45:24.803 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.803 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:45:24.803 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 709 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-13T22:45:24.803 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.803 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.803 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.803 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.803 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:24.804 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2874415034' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-05-13T22:45:24.804 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:24 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:45:25.480 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-13T22:45:25.491 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:45:25.491 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-13T22:45:26.050 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:25 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2874415034' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-05-13T22:45:26.050 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:25 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:45:26.050 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:25 smithi102 ceph-mon[28599]: Dropping low affinity standby-replay daemon mds.cephfs.smithi130.hzjanp in favor of higher affinity standby. 2024-05-13T22:45:26.050 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:25 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] up:standby 2024-05-13T22:45:26.050 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:25 smithi102 ceph-mon[28599]: Dropping low affinity active daemon mds.cephfs.smithi102.cgkcwz in favor of higher affinity standby. 2024-05-13T22:45:26.050 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:25 smithi102 ceph-mon[28599]: Replacing daemon mds.cephfs.smithi102.cgkcwz as rank 0 with standby daemon mds.cephfs.smithi102.sdylgv 2024-05-13T22:45:26.050 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:25 smithi102 ceph-mon[28599]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-13T22:45:26.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:25 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 1 up:standby 2024-05-13T22:45:26.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:25 smithi102 ceph-mon[28599]: osdmap e47: 6 total, 6 up, 6 in 2024-05-13T22:45:26.051 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:25 smithi102 ceph-mon[28599]: fsmap cephfs:1/1 {0=cephfs.smithi102.sdylgv=up:replay} 1 up:standby 2024-05-13T22:45:26.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:25 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2874415034' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-05-13T22:45:26.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:25 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:45:26.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:25 smithi130 ceph-mon[36591]: Dropping low affinity standby-replay daemon mds.cephfs.smithi130.hzjanp in favor of higher affinity standby. 2024-05-13T22:45:26.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:25 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] up:standby 2024-05-13T22:45:26.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:25 smithi130 ceph-mon[36591]: Dropping low affinity active daemon mds.cephfs.smithi102.cgkcwz in favor of higher affinity standby. 2024-05-13T22:45:26.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:25 smithi130 ceph-mon[36591]: Replacing daemon mds.cephfs.smithi102.cgkcwz as rank 0 with standby daemon mds.cephfs.smithi102.sdylgv 2024-05-13T22:45:26.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:25 smithi130 ceph-mon[36591]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-13T22:45:26.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:25 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 1 up:standby 2024-05-13T22:45:26.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:25 smithi130 ceph-mon[36591]: osdmap e47: 6 total, 6 up, 6 in 2024-05-13T22:45:26.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:25 smithi130 ceph-mon[36591]: fsmap cephfs:1/1 {0=cephfs.smithi102.sdylgv=up:replay} 1 up:standby 2024-05-13T22:45:26.256 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:26 smithi102 ceph-mon[28599]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-13T22:45:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:26 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:26 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:26 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] up:boot 2024-05-13T22:45:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:26 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] up:boot 2024-05-13T22:45:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:26 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] up:reconnect 2024-05-13T22:45:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:26 smithi102 ceph-mon[28599]: fsmap cephfs:1/1 {0=cephfs.smithi102.sdylgv=up:reconnect} 3 up:standby 2024-05-13T22:45:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:26 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:45:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:26 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:45:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:26 smithi130 ceph-mon[36591]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-13T22:45:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:26 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:26 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:26 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] up:boot 2024-05-13T22:45:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:26 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] up:boot 2024-05-13T22:45:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:26 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] up:reconnect 2024-05-13T22:45:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:26 smithi130 ceph-mon[36591]: fsmap cephfs:1/1 {0=cephfs.smithi102.sdylgv=up:reconnect} 3 up:standby 2024-05-13T22:45:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:26 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:45:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:26 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:45:28.339 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:28 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] up:rejoin 2024-05-13T22:45:28.339 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:28 smithi102 ceph-mon[28599]: fsmap cephfs:1/1 {0=cephfs.smithi102.sdylgv=up:rejoin} 3 up:standby 2024-05-13T22:45:28.339 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:28 smithi102 ceph-mon[28599]: daemon mds.cephfs.smithi102.sdylgv is now active in filesystem cephfs as rank 0 2024-05-13T22:45:28.339 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:28 smithi102 ceph-mon[28599]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-13T22:45:28.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.409+0000 7fb333463700 1 -- 172.21.15.102:0/892806040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb32c1016c0 msgr2=0x7fb32c103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:28.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.409+0000 7fb333463700 1 --2- 172.21.15.102:0/892806040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb32c1016c0 0x7fb32c103ab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb328009910 tx=0x7fb328009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:28.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.410+0000 7fb333463700 1 -- 172.21.15.102:0/892806040 shutdown_connections 2024-05-13T22:45:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.410+0000 7fb333463700 1 --2- 172.21.15.102:0/892806040 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb32c103ff0 0x7fb32c1063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.410+0000 7fb333463700 1 --2- 172.21.15.102:0/892806040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb32c1016c0 0x7fb32c103ab0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.410+0000 7fb333463700 1 -- 172.21.15.102:0/892806040 >> 172.21.15.102:0/892806040 conn(0x7fb32c0fb030 msgr2=0x7fb32c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.410+0000 7fb333463700 1 -- 172.21.15.102:0/892806040 shutdown_connections 2024-05-13T22:45:28.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.410+0000 7fb333463700 1 -- 172.21.15.102:0/892806040 wait complete. 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.410+0000 7fb333463700 1 Processor -- start 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb333463700 1 -- start start 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb333463700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb32c1016c0 0x7fb32c196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb333463700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb32c103ff0 0x7fb32c1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb333463700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c196ec0 con 0x7fb32c103ff0 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb333463700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c197000 con 0x7fb32c1016c0 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb3309fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb32c103ff0 0x7fb32c1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb3309fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb32c103ff0 0x7fb32c1968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52114/0 (socket says 172.21.15.102:52114) 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb3309fe700 1 -- 172.21.15.102:0/2004455256 learned_addr learned my addr 172.21.15.102:0/2004455256 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb3311ff700 1 --2- 172.21.15.102:0/2004455256 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb32c1016c0 0x7fb32c196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb3309fe700 1 -- 172.21.15.102:0/2004455256 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb32c1016c0 msgr2=0x7fb32c196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:28.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb3309fe700 1 --2- 172.21.15.102:0/2004455256 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb32c1016c0 0x7fb32c196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:28.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb3309fe700 1 -- 172.21.15.102:0/2004455256 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3280095f0 con 0x7fb32c103ff0 2024-05-13T22:45:28.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.411+0000 7fb3309fe700 1 --2- 172.21.15.102:0/2004455256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb32c103ff0 0x7fb32c1968a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00b4b0 tx=0x7fb31c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:28.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.412+0000 7fb3227fc700 1 -- 172.21.15.102:0/2004455256 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31c011800 con 0x7fb32c103ff0 2024-05-13T22:45:28.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.412+0000 7fb3227fc700 1 -- 172.21.15.102:0/2004455256 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb31c011e10 con 0x7fb32c103ff0 2024-05-13T22:45:28.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.412+0000 7fb333463700 1 -- 172.21.15.102:0/2004455256 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32c19bab0 con 0x7fb32c103ff0 2024-05-13T22:45:28.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.412+0000 7fb3227fc700 1 -- 172.21.15.102:0/2004455256 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb31c00f2e0 con 0x7fb32c103ff0 2024-05-13T22:45:28.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.412+0000 7fb333463700 1 -- 172.21.15.102:0/2004455256 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c19c080 con 0x7fb32c103ff0 2024-05-13T22:45:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.413+0000 7fb333463700 1 -- 172.21.15.102:0/2004455256 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c1905a0 con 0x7fb32c103ff0 2024-05-13T22:45:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.413+0000 7fb3227fc700 1 -- 172.21.15.102:0/2004455256 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb31c00f510 con 0x7fb32c103ff0 2024-05-13T22:45:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.414+0000 7fb3227fc700 1 --2- 172.21.15.102:0/2004455256 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb31806c4e0 0x7fb31806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.414+0000 7fb3311ff700 1 --2- 172.21.15.102:0/2004455256 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb31806c4e0 0x7fb31806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.414+0000 7fb3227fc700 1 -- 172.21.15.102:0/2004455256 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb31c08b4f0 con 0x7fb32c103ff0 2024-05-13T22:45:28.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.414+0000 7fb3311ff700 1 --2- 172.21.15.102:0/2004455256 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb31806c4e0 0x7fb31806e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb328006010 tx=0x7fb32800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:28.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.418+0000 7fb3227fc700 1 -- 172.21.15.102:0/2004455256 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb31c059770 con 0x7fb32c103ff0 2024-05-13T22:45:28.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:28 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] up:rejoin 2024-05-13T22:45:28.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:28 smithi130 ceph-mon[36591]: fsmap cephfs:1/1 {0=cephfs.smithi102.sdylgv=up:rejoin} 3 up:standby 2024-05-13T22:45:28.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:28 smithi130 ceph-mon[36591]: daemon mds.cephfs.smithi102.sdylgv is now active in filesystem cephfs as rank 0 2024-05-13T22:45:28.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:28 smithi130 ceph-mon[36591]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-05-13T22:45:28.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.686+0000 7fb333463700 1 -- 172.21.15.102:0/2004455256 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb32c02cc70 con 0x7fb32c103ff0 2024-05-13T22:45:28.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.687+0000 7fb3227fc700 1 -- 172.21.15.102:0/2004455256 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1856 (secure 0 0 0) 0x7fb31c059300 con 0x7fb32c103ff0 2024-05-13T22:45:28.666 INFO:teuthology.orchestra.run.smithi102.stdout:e17 2024-05-13T22:45:28.666 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:45:28.666 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:45:28.666 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:45:28.666 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:45:28.666 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:45:28.666 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:45:28.666 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 17 2024-05-13T22:45:28.666 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:45:28.666 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 47 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=14498} 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:45:28.667 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:45:28.668 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:45:28.669 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:45:28.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.690+0000 7fb333463700 1 -- 172.21.15.102:0/2004455256 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb31806c4e0 msgr2=0x7fb31806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:28.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.690+0000 7fb333463700 1 --2- 172.21.15.102:0/2004455256 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb31806c4e0 0x7fb31806e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb328006010 tx=0x7fb32800b540 comp rx=0 tx=0).stop 2024-05-13T22:45:28.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.690+0000 7fb333463700 1 -- 172.21.15.102:0/2004455256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb32c103ff0 msgr2=0x7fb32c1968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:28.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.690+0000 7fb333463700 1 --2- 172.21.15.102:0/2004455256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb32c103ff0 0x7fb32c1968a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00b4b0 tx=0x7fb31c00b870 comp rx=0 tx=0).stop 2024-05-13T22:45:28.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.690+0000 7fb333463700 1 -- 172.21.15.102:0/2004455256 shutdown_connections 2024-05-13T22:45:28.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.690+0000 7fb333463700 1 --2- 172.21.15.102:0/2004455256 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb31806c4e0 0x7fb31806e9a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:28.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.690+0000 7fb333463700 1 --2- 172.21.15.102:0/2004455256 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb32c1016c0 0x7fb32c196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:28.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.690+0000 7fb333463700 1 --2- 172.21.15.102:0/2004455256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb32c103ff0 0x7fb32c1968a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:28.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.690+0000 7fb333463700 1 -- 172.21.15.102:0/2004455256 >> 172.21.15.102:0/2004455256 conn(0x7fb32c0fb030 msgr2=0x7fb32c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:28.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.691+0000 7fb333463700 1 -- 172.21.15.102:0/2004455256 shutdown_connections 2024-05-13T22:45:28.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:28.691+0000 7fb333463700 1 -- 172.21.15.102:0/2004455256 wait complete. 2024-05-13T22:45:28.670 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:45:29.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:29 smithi102 ceph-mon[28599]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-13T22:45:29.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:29 smithi102 ceph-mon[28599]: Cluster is now healthy 2024-05-13T22:45:29.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:29 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] up:active 2024-05-13T22:45:29.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:29 smithi102 ceph-mon[28599]: mds.? [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] up:standby 2024-05-13T22:45:29.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:29 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 3 up:standby 2024-05-13T22:45:29.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:29 smithi102 ceph-mon[28599]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:45:29.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:29 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2004455256' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:45:29.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:29 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:29.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:29 smithi130 ceph-mon[36591]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-13T22:45:29.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:29 smithi130 ceph-mon[36591]: Cluster is now healthy 2024-05-13T22:45:29.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:29 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] up:active 2024-05-13T22:45:29.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:29 smithi130 ceph-mon[36591]: mds.? [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] up:standby 2024-05-13T22:45:29.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:29 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 3 up:standby 2024-05-13T22:45:29.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:29 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:45:29.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:29 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2004455256' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:45:29.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:29 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:29.578 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-05-13T22:45:30.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:30 smithi102 ceph-mon[28599]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-13T22:45:30.298 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:30 smithi130 ceph-mon[36591]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-13T22:45:31.581 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:31 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:31.582 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:31 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:31.582 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:31 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:31.582 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:31 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:45:31.582 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:31 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:31.582 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:31 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:45:31.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:31 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:31.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:31 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:31.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:31 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:31.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:31 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:45:31.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:31 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:31.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:31 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:45:31.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.948+0000 7f2409fce700 1 -- 172.21.15.102:0/1069222277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2404100bc0 msgr2=0x7f2404100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:31.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.948+0000 7f2409fce700 1 --2- 172.21.15.102:0/1069222277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2404100bc0 0x7f2404100fe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009910 tx=0x7f23f4009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:31.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.948+0000 7f2409fce700 1 -- 172.21.15.102:0/1069222277 shutdown_connections 2024-05-13T22:45:31.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.948+0000 7f2409fce700 1 --2- 172.21.15.102:0/1069222277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2404101dc0 0x7f2404102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:31.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.948+0000 7f2409fce700 1 --2- 172.21.15.102:0/1069222277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2404100bc0 0x7f2404100fe0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:31.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.948+0000 7f2409fce700 1 -- 172.21.15.102:0/1069222277 >> 172.21.15.102:0/1069222277 conn(0x7f24040fc140 msgr2=0x7f24040fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:31.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.948+0000 7f2409fce700 1 -- 172.21.15.102:0/1069222277 shutdown_connections 2024-05-13T22:45:31.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.948+0000 7f2409fce700 1 -- 172.21.15.102:0/1069222277 wait complete. 2024-05-13T22:45:31.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.948+0000 7f2409fce700 1 Processor -- start 2024-05-13T22:45:31.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f2409fce700 1 -- start start 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f2409fce700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2404100bc0 0x7f2404074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f2409fce700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2404101dc0 0x7f2404073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f2409fce700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24040736c0 con 0x7f2404101dc0 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f2409fce700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2404073800 con 0x7f2404100bc0 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f24037fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2404100bc0 0x7f2404074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f2402ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2404101dc0 0x7f2404073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f24037fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2404100bc0 0x7f2404074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.102:34192/0 (socket says 172.21.15.102:34192) 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f24037fe700 1 -- 172.21.15.102:0/3290412150 learned_addr learned my addr 172.21.15.102:0/3290412150 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f2402ffd700 1 -- 172.21.15.102:0/3290412150 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2404100bc0 msgr2=0x7f2404074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f2402ffd700 1 --2- 172.21.15.102:0/3290412150 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2404100bc0 0x7f2404074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f2402ffd700 1 -- 172.21.15.102:0/3290412150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f40095f0 con 0x7f2404101dc0 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f2402ffd700 1 --2- 172.21.15.102:0/3290412150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2404101dc0 0x7f2404073180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f23f800c8f0 tx=0x7f23f800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:31.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.949+0000 7f2400ff9700 1 -- 172.21.15.102:0/3290412150 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23f800e850 con 0x7f2404101dc0 2024-05-13T22:45:31.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.950+0000 7f2409fce700 1 -- 172.21.15.102:0/3290412150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2404073ae0 con 0x7f2404101dc0 2024-05-13T22:45:31.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.950+0000 7f2400ff9700 1 -- 172.21.15.102:0/3290412150 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f23f800ee60 con 0x7f2404101dc0 2024-05-13T22:45:31.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.950+0000 7f2400ff9700 1 -- 172.21.15.102:0/3290412150 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23f8018630 con 0x7f2404101dc0 2024-05-13T22:45:31.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.950+0000 7f2409fce700 1 -- 172.21.15.102:0/3290412150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240419e3d0 con 0x7f2404101dc0 2024-05-13T22:45:31.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.951+0000 7f2409fce700 1 -- 172.21.15.102:0/3290412150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2404109720 con 0x7f2404101dc0 2024-05-13T22:45:31.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.952+0000 7f2400ff9700 1 -- 172.21.15.102:0/3290412150 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f23f800f8a0 con 0x7f2404101dc0 2024-05-13T22:45:31.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.952+0000 7f2400ff9700 1 --2- 172.21.15.102:0/3290412150 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f23ec06c4e0 0x7f23ec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:31.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.952+0000 7f2400ff9700 1 -- 172.21.15.102:0/3290412150 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f23f808b6a0 con 0x7f2404101dc0 2024-05-13T22:45:31.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.952+0000 7f24037fe700 1 --2- 172.21.15.102:0/3290412150 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f23ec06c4e0 0x7f23ec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:31.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.953+0000 7f24037fe700 1 --2- 172.21.15.102:0/3290412150 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f23ec06c4e0 0x7f23ec06e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23f4006010 tx=0x7f23f400b580 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:31.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:31.957+0000 7f2400ff9700 1 -- 172.21.15.102:0/3290412150 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f23f80598b0 con 0x7f2404101dc0 2024-05-13T22:45:32.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.202+0000 7f2409fce700 1 -- 172.21.15.102:0/3290412150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24041098e0 con 0x7f2404101dc0 2024-05-13T22:45:32.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.203+0000 7f2400ff9700 1 -- 172.21.15.102:0/3290412150 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4753 (secure 0 0 0) 0x7f24041098e0 con 0x7f2404101dc0 2024-05-13T22:45:32.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.204+0000 7f2409fce700 1 -- 172.21.15.102:0/3290412150 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f23ec06c4e0 msgr2=0x7f23ec06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:32.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.204+0000 7f2409fce700 1 --2- 172.21.15.102:0/3290412150 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f23ec06c4e0 0x7f23ec06e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23f4006010 tx=0x7f23f400b580 comp rx=0 tx=0).stop 2024-05-13T22:45:32.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.204+0000 7f2409fce700 1 -- 172.21.15.102:0/3290412150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2404101dc0 msgr2=0x7f2404073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:32.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.204+0000 7f2409fce700 1 --2- 172.21.15.102:0/3290412150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2404101dc0 0x7f2404073180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f23f800c8f0 tx=0x7f23f800cc00 comp rx=0 tx=0).stop 2024-05-13T22:45:32.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.205+0000 7f2409fce700 1 -- 172.21.15.102:0/3290412150 shutdown_connections 2024-05-13T22:45:32.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.205+0000 7f2409fce700 1 --2- 172.21.15.102:0/3290412150 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f23ec06c4e0 0x7f23ec06e9a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:32.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.205+0000 7f2409fce700 1 --2- 172.21.15.102:0/3290412150 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2404100bc0 0x7f2404074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:32.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.205+0000 7f2409fce700 1 --2- 172.21.15.102:0/3290412150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2404101dc0 0x7f2404073180 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:32.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.205+0000 7f2409fce700 1 -- 172.21.15.102:0/3290412150 >> 172.21.15.102:0/3290412150 conn(0x7f24040fc140 msgr2=0x7f2404104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:32.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.205+0000 7f2409fce700 1 -- 172.21.15.102:0/3290412150 shutdown_connections 2024-05-13T22:45:32.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:32.205+0000 7f2409fce700 1 -- 172.21.15.102:0/3290412150 wait complete. 2024-05-13T22:45:32.186 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:45:32.194 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:45:32.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:32 smithi130 ceph-mon[36591]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-13T22:45:32.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:32 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3290412150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-13T22:45:32.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:32 smithi102 ceph-mon[28599]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-13T22:45:32.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:32 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3290412150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-13T22:45:33.229 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-05-13T22:45:33.887 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:34.141 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:33 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:34.141 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:33 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:34.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:33 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:34.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:33 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:34.948 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:34 smithi102 ceph-mon[28599]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-13T22:45:35.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:34 smithi130 ceph-mon[36591]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-13T22:45:35.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.728+0000 7fb110f60700 1 -- 172.21.15.102:0/2878793083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb10c101df0 msgr2=0x7fb10c102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:35.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.728+0000 7fb110f60700 1 --2- 172.21.15.102:0/2878793083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb10c101df0 0x7fb10c102250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb100009910 tx=0x7fb100009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:35.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.728+0000 7fb110f60700 1 -- 172.21.15.102:0/2878793083 shutdown_connections 2024-05-13T22:45:35.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.728+0000 7fb110f60700 1 --2- 172.21.15.102:0/2878793083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb10c101df0 0x7fb10c102250 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:35.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.728+0000 7fb110f60700 1 --2- 172.21.15.102:0/2878793083 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb10c100bf0 0x7fb10c101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:35.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.728+0000 7fb110f60700 1 -- 172.21.15.102:0/2878793083 >> 172.21.15.102:0/2878793083 conn(0x7fb10c0fc190 msgr2=0x7fb10c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:35.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.728+0000 7fb110f60700 1 -- 172.21.15.102:0/2878793083 shutdown_connections 2024-05-13T22:45:35.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.728+0000 7fb110f60700 1 -- 172.21.15.102:0/2878793083 wait complete. 2024-05-13T22:45:35.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.729+0000 7fb110f60700 1 Processor -- start 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.729+0000 7fb110f60700 1 -- start start 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.729+0000 7fb110f60700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb10c100bf0 0x7fb10c196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.729+0000 7fb110f60700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb10c101df0 0x7fb10c196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.729+0000 7fb110f60700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c196e70 con 0x7fb10c101df0 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.729+0000 7fb110f60700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c196fb0 con 0x7fb10c100bf0 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.729+0000 7fb109d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb10c101df0 0x7fb10c196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb109d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb10c101df0 0x7fb10c196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53474/0 (socket says 172.21.15.102:53474) 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb109d9b700 1 -- 172.21.15.102:0/4178539686 learned_addr learned my addr 172.21.15.102:0/4178539686 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb10a59c700 1 --2- 172.21.15.102:0/4178539686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb10c100bf0 0x7fb10c196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb109d9b700 1 -- 172.21.15.102:0/4178539686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb10c100bf0 msgr2=0x7fb10c196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb109d9b700 1 --2- 172.21.15.102:0/4178539686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb10c100bf0 0x7fb10c196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb109d9b700 1 -- 172.21.15.102:0/4178539686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000095f0 con 0x7fb10c101df0 2024-05-13T22:45:35.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb109d9b700 1 --2- 172.21.15.102:0/4178539686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb10c101df0 0x7fb10c196850 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb100006010 tx=0x7fb10000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:35.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb0fb7fe700 1 -- 172.21.15.102:0/4178539686 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb10001d070 con 0x7fb10c101df0 2024-05-13T22:45:35.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb110f60700 1 -- 172.21.15.102:0/4178539686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c19ba00 con 0x7fb10c101df0 2024-05-13T22:45:35.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb0fb7fe700 1 -- 172.21.15.102:0/4178539686 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb10000f460 con 0x7fb10c101df0 2024-05-13T22:45:35.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb0fb7fe700 1 -- 172.21.15.102:0/4178539686 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb100021640 con 0x7fb10c101df0 2024-05-13T22:45:35.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.730+0000 7fb110f60700 1 -- 172.21.15.102:0/4178539686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c19bef0 con 0x7fb10c101df0 2024-05-13T22:45:35.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.731+0000 7fb110f60700 1 -- 172.21.15.102:0/4178539686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c109750 con 0x7fb10c101df0 2024-05-13T22:45:35.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.732+0000 7fb0fb7fe700 1 -- 172.21.15.102:0/4178539686 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb10000fab0 con 0x7fb10c101df0 2024-05-13T22:45:35.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.732+0000 7fb0fb7fe700 1 --2- 172.21.15.102:0/4178539686 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb0f406c4e0 0x7fb0f406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:35.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.732+0000 7fb10a59c700 1 --2- 172.21.15.102:0/4178539686 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb0f406c4e0 0x7fb0f406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:35.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.732+0000 7fb0fb7fe700 1 -- 172.21.15.102:0/4178539686 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb10008cb20 con 0x7fb10c101df0 2024-05-13T22:45:35.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.732+0000 7fb10a59c700 1 --2- 172.21.15.102:0/4178539686 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb0f406c4e0 0x7fb0f406e9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb10c101c50 tx=0x7fb0fc00b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:35.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:35.737+0000 7fb0fb7fe700 1 -- 172.21.15.102:0/4178539686 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb10005ad30 con 0x7fb10c101df0 2024-05-13T22:45:35.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.009+0000 7fb110f60700 1 -- 172.21.15.102:0/4178539686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fb10c109910 con 0x7fb10c101df0 2024-05-13T22:45:35.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.009+0000 7fb0fb7fe700 1 -- 172.21.15.102:0/4178539686 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+83 (secure 0 0 0) 0x7fb10005a8c0 con 0x7fb10c101df0 2024-05-13T22:45:35.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.011+0000 7fb110f60700 1 -- 172.21.15.102:0/4178539686 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb0f406c4e0 msgr2=0x7fb0f406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:35.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.011+0000 7fb110f60700 1 --2- 172.21.15.102:0/4178539686 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb0f406c4e0 0x7fb0f406e9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb10c101c50 tx=0x7fb0fc00b410 comp rx=0 tx=0).stop 2024-05-13T22:45:35.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.011+0000 7fb110f60700 1 -- 172.21.15.102:0/4178539686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb10c101df0 msgr2=0x7fb10c196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:35.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.011+0000 7fb110f60700 1 --2- 172.21.15.102:0/4178539686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb10c101df0 0x7fb10c196850 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb100006010 tx=0x7fb10000bd70 comp rx=0 tx=0).stop 2024-05-13T22:45:35.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.012+0000 7fb110f60700 1 -- 172.21.15.102:0/4178539686 shutdown_connections 2024-05-13T22:45:35.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.012+0000 7fb110f60700 1 --2- 172.21.15.102:0/4178539686 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb0f406c4e0 0x7fb0f406e9a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:35.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.012+0000 7fb110f60700 1 --2- 172.21.15.102:0/4178539686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb10c100bf0 0x7fb10c196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:35.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.012+0000 7fb110f60700 1 --2- 172.21.15.102:0/4178539686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb10c101df0 0x7fb10c196850 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:35.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.012+0000 7fb110f60700 1 -- 172.21.15.102:0/4178539686 >> 172.21.15.102:0/4178539686 conn(0x7fb10c0fc190 msgr2=0x7fb10c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:35.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.012+0000 7fb110f60700 1 -- 172.21.15.102:0/4178539686 shutdown_connections 2024-05-13T22:45:35.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.012+0000 7fb110f60700 1 -- 172.21.15.102:0/4178539686 wait complete. 2024-05-13T22:45:36.001 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:45:36.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:36 smithi130 ceph-mon[36591]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.3 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-13T22:45:36.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:36 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/4178539686' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-13T22:45:36.285 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:36 smithi102 ceph-mon[28599]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.3 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-13T22:45:36.285 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:36 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/4178539686' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-13T22:45:36.597 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-05-13T22:45:36.610 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-13T22:45:36.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.750+0000 7f4a5334e640 1 -- 172.21.15.102:0/1323777554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a4c0fe900 msgr2=0x7f4a4c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:36.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.750+0000 7f4a5334e640 1 --2- 172.21.15.102:0/1323777554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a4c0fe900 0x7f4a4c100d30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009880 tx=0x7f4a4802bce0 comp rx=0 tx=0).stop 2024-05-13T22:45:36.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.750+0000 7f4a5334e640 1 -- 172.21.15.102:0/1323777554 shutdown_connections 2024-05-13T22:45:36.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.750+0000 7f4a5334e640 1 --2- 172.21.15.102:0/1323777554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a4c101270 0x7f4a4c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:36.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.750+0000 7f4a5334e640 1 --2- 172.21.15.102:0/1323777554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a4c0fe900 0x7f4a4c100d30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:36.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.750+0000 7f4a5334e640 1 -- 172.21.15.102:0/1323777554 >> 172.21.15.102:0/1323777554 conn(0x7f4a4c0fa500 msgr2=0x7f4a4c0fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:36.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.750+0000 7f4a5334e640 1 -- 172.21.15.102:0/1323777554 shutdown_connections 2024-05-13T22:45:36.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.750+0000 7f4a5334e640 1 -- 172.21.15.102:0/1323777554 wait complete. 2024-05-13T22:45:36.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a5334e640 1 Processor -- start 2024-05-13T22:45:36.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a5334e640 1 -- start start 2024-05-13T22:45:36.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a5334e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a4c0fe900 0x7f4a4c19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:36.731 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a5334e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a4c101270 0x7f4a4c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a5334e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c19c990 con 0x7f4a4c101270 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a5334e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c19cb00 con 0x7f4a4c0fe900 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a508e9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a4c101270 0x7f4a4c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a508e9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a4c101270 0x7f4a4c19c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53502/0 (socket says 172.21.15.102:53502) 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a508e9640 1 -- 172.21.15.102:0/3009290530 learned_addr learned my addr 172.21.15.102:0/3009290530 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a510ea640 1 --2- 172.21.15.102:0/3009290530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a4c0fe900 0x7f4a4c19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a508e9640 1 -- 172.21.15.102:0/3009290530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a4c0fe900 msgr2=0x7f4a4c19be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a508e9640 1 --2- 172.21.15.102:0/3009290530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a4c0fe900 0x7f4a4c19be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.751+0000 7f4a508e9640 1 -- 172.21.15.102:0/3009290530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a48009560 con 0x7f4a4c101270 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.752+0000 7f4a508e9640 1 --2- 172.21.15.102:0/3009290530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a4c101270 0x7f4a4c19c3c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00d6f0 tx=0x7f4a3c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.752+0000 7f4a427fc640 1 -- 172.21.15.102:0/3009290530 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a3c009a30 con 0x7f4a4c101270 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.752+0000 7f4a427fc640 1 -- 172.21.15.102:0/3009290530 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4a3c0054d0 con 0x7f4a4c101270 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.752+0000 7f4a5334e640 1 -- 172.21.15.102:0/3009290530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c1a1590 con 0x7f4a4c101270 2024-05-13T22:45:36.732 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.752+0000 7f4a427fc640 1 -- 172.21.15.102:0/3009290530 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a3c010460 con 0x7f4a4c101270 2024-05-13T22:45:36.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.752+0000 7f4a5334e640 1 -- 172.21.15.102:0/3009290530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c1a1af0 con 0x7f4a4c101270 2024-05-13T22:45:36.733 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.753+0000 7f4a5334e640 1 -- 172.21.15.102:0/3009290530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c195ff0 con 0x7f4a4c101270 2024-05-13T22:45:36.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.753+0000 7f4a427fc640 1 -- 172.21.15.102:0/3009290530 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4a3c009b90 con 0x7f4a4c101270 2024-05-13T22:45:36.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.754+0000 7f4a427fc640 1 --2- 172.21.15.102:0/3009290530 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4a3806c510 0x7f4a3806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:36.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.754+0000 7f4a510ea640 1 --2- 172.21.15.102:0/3009290530 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4a3806c510 0x7f4a3806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:36.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.754+0000 7f4a427fc640 1 -- 172.21.15.102:0/3009290530 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4a3c08b420 con 0x7f4a4c101270 2024-05-13T22:45:36.734 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.754+0000 7f4a510ea640 1 --2- 172.21.15.102:0/3009290530 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4a3806c510 0x7f4a3806e9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a4802c7c0 tx=0x7f4a48005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.758+0000 7f4a427fc640 1 -- 172.21.15.102:0/3009290530 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4a3c055ba0 con 0x7f4a4c101270 2024-05-13T22:45:36.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.935+0000 7f4a5334e640 1 -- 172.21.15.102:0/3009290530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a4c10f350 con 0x7f4a4c101270 2024-05-13T22:45:36.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.936+0000 7f4a427fc640 1 -- 172.21.15.102:0/3009290530 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15173 (secure 0 0 0) 0x7f4a3c0591c0 con 0x7f4a4c101270 2024-05-13T22:45:36.914 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:45:36.915 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":47,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","created":"2024-05-13T22:38:14.140940+0000","modified":"2024-05-13T22:45:25.114247+0000","last_up_change":"2024-05-13T22:43:54.154797+0000","last_in_change":"2024-05-13T22:43:35.225543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-13T22:42:39.969524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-13T22:45:08.878601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-13T22:45:09.369354+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d8cb2e3-52d6-45f5-94a1-3e8cc6535efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6803","nonce":2497809722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6805","nonce":2497809722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6809","nonce":2497809722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2497809722},{"type":"v1","addr":"172.21.15.102:6807","nonce":2497809722}]},"public_addr":"172.21.15.102:6803/2497809722","cluster_addr":"172.21.15.102:6805/2497809722","heartbeat_back_addr":"172.21.15.102:6809/2497809722","heartbeat_front_addr":"172.21.15.102:6807/2497809722","state":["exists","up"]},{"osd":1,"uuid":"1d3eacb1-362c-4b6a-a959-7dea19c6521e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6811","nonce":2169124323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6813","nonce":2169124323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6817","nonce":2169124323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":2169124323},{"type":"v1","addr":"172.21.15.102:6815","nonce":2169124323}]},"public_addr":"172.21.15.102:6811/2169124323","cluster_addr":"172.21.15.102:6813/2169124323","heartbeat_back_addr":"172.21.15.102:6817/2169124323","heartbeat_front_addr":"172.21.15.102:6815/2169124323","state":["exists","up"]},{"osd":2,"uuid":"205b05a9-9d68-48ed-b87b-c4c677882eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6819","nonce":3553677373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6821","nonce":3553677373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6825","nonce":3553677373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3553677373},{"type":"v1","addr":"172.21.15.102:6823","nonce":3553677373}]},"public_addr":"172.21.15.102:6819/3553677373","cluster_addr":"172.21.15.102:6821/3553677373","heartbeat_back_addr":"172.21.15.102:6825/3553677373","heartbeat_front_addr":"172.21.15.102:6823/3553677373","state":["exists","up"]},{"osd":3,"uuid":"d827066c-3201-479e-b142-27721c2761fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6801","nonce":4175190705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6803","nonce":4175190705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6807","nonce":4175190705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4175190705},{"type":"v1","addr":"172.21.15.130:6805","nonce":4175190705}]},"public_addr":"172.21.15.130:6801/4175190705","cluster_addr":"172.21.15.130:6803/4175190705","heartbeat_back_addr":"172.21.15.130:6807/4175190705","heartbeat_front_addr":"172.21.15.130:6805/4175190705","state":["exists","up"]},{"osd":4,"uuid":"7d89edd4-8877-4cb8-a0af-019f9451bd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6809","nonce":574281717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6811","nonce":574281717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6815","nonce":574281717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":574281717},{"type":"v1","addr":"172.21.15.130:6813","nonce":574281717}]},"public_addr":"172.21.15.130:6809/574281717","cluster_addr":"172.21.15.130:6811/574281717","heartbeat_back_addr":"172.21.15.130:6815/574281717","heartbeat_front_addr":"172.21.15.130:6813/574281717","state":["exists","up"]},{"osd":5,"uuid":"c0942077-8a53-4e3b-b411-f1763c592e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6817","nonce":957542161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6819","nonce":957542161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6823","nonce":957542161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":957542161},{"type":"v1","addr":"172.21.15.130:6821","nonce":957542161}]},"public_addr":"172.21.15.130:6817/957542161","cluster_addr":"172.21.15.130:6819/957542161","heartbeat_back_addr":"172.21.15.130:6823/957542161","heartbeat_front_addr":"172.21.15.130:6821/957542161","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:41:41.451627+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:09.327795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:36.547203+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:00.627031+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:26.709919+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:52.576178+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:6827/3231884180":"2024-05-14T22:45:25.114218+0000","172.21.15.102:6800/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/633075379":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6801/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:6800/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/2096579260":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6826/3231884180":"2024-05-14T22:45:25.114218+0000","172.21.15.102:0/3812624194":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6801/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/17493483":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/3850189972":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/457222314":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/2248276518":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/773571736":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/402887427":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6801/263535069":"2024-05-14T22:40:23.651424+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-13T22:45:36.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.938+0000 7f4a5334e640 1 -- 172.21.15.102:0/3009290530 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4a3806c510 msgr2=0x7f4a3806e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:36.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.938+0000 7f4a5334e640 1 --2- 172.21.15.102:0/3009290530 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4a3806c510 0x7f4a3806e9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a4802c7c0 tx=0x7f4a48005fd0 comp rx=0 tx=0).stop 2024-05-13T22:45:36.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.938+0000 7f4a5334e640 1 -- 172.21.15.102:0/3009290530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a4c101270 msgr2=0x7f4a4c19c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:36.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.938+0000 7f4a5334e640 1 --2- 172.21.15.102:0/3009290530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a4c101270 0x7f4a4c19c3c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00d6f0 tx=0x7f4a3c00dbb0 comp rx=0 tx=0).stop 2024-05-13T22:45:36.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.938+0000 7f4a5334e640 1 -- 172.21.15.102:0/3009290530 shutdown_connections 2024-05-13T22:45:36.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.938+0000 7f4a5334e640 1 --2- 172.21.15.102:0/3009290530 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4a3806c510 0x7f4a3806e9d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:36.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.938+0000 7f4a5334e640 1 --2- 172.21.15.102:0/3009290530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a4c101270 0x7f4a4c19c3c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:36.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.938+0000 7f4a5334e640 1 --2- 172.21.15.102:0/3009290530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a4c0fe900 0x7f4a4c19be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:36.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.939+0000 7f4a5334e640 1 -- 172.21.15.102:0/3009290530 >> 172.21.15.102:0/3009290530 conn(0x7f4a4c0fa500 msgr2=0x7f4a4c0fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:36.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.939+0000 7f4a5334e640 1 -- 172.21.15.102:0/3009290530 shutdown_connections 2024-05-13T22:45:36.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:36.939+0000 7f4a5334e640 1 -- 172.21.15.102:0/3009290530 wait complete. 2024-05-13T22:45:36.928 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-13T22:42:39.969524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-13T22:45:08.878601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-13T22:45:09.369354+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-13T22:45:36.929 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-13T22:45:37.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.062+0000 7fadcff51640 1 -- 172.21.15.102:0/2519001392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81012c0 msgr2=0x7fadc8101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.062+0000 7fadcff51640 1 --2- 172.21.15.102:0/2519001392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81012c0 0x7fadc8101740 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009880 tx=0x7fadb802bce0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.062+0000 7fadcff51640 1 -- 172.21.15.102:0/2519001392 shutdown_connections 2024-05-13T22:45:37.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.062+0000 7fadcff51640 1 --2- 172.21.15.102:0/2519001392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81012c0 0x7fadc8101740 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.062+0000 7fadcff51640 1 --2- 172.21.15.102:0/2519001392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc81000d0 0x7fadc81004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.062+0000 7fadcff51640 1 -- 172.21.15.102:0/2519001392 >> 172.21.15.102:0/2519001392 conn(0x7fadc80fb620 msgr2=0x7fadc80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:37.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.062+0000 7fadcff51640 1 -- 172.21.15.102:0/2519001392 shutdown_connections 2024-05-13T22:45:37.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.062+0000 7fadcff51640 1 -- 172.21.15.102:0/2519001392 wait complete. 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.062+0000 7fadcff51640 1 Processor -- start 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.062+0000 7fadcff51640 1 -- start start 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcff51640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81000d0 0x7fadc8195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcff51640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc81012c0 0x7fadc8195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcff51640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc81963a0 con 0x7fadc81000d0 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcff51640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8196510 con 0x7fadc81012c0 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcdced640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81000d0 0x7fadc8195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcdced640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81000d0 0x7fadc8195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53530/0 (socket says 172.21.15.102:53530) 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcdced640 1 -- 172.21.15.102:0/215260877 learned_addr learned my addr 172.21.15.102:0/215260877 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcd4ec640 1 --2- 172.21.15.102:0/215260877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc81012c0 0x7fadc8195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcdced640 1 -- 172.21.15.102:0/215260877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc81012c0 msgr2=0x7fadc8195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcdced640 1 --2- 172.21.15.102:0/215260877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc81012c0 0x7fadc8195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcdced640 1 -- 172.21.15.102:0/215260877 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb8009560 con 0x7fadc81000d0 2024-05-13T22:45:37.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcdced640 1 --2- 172.21.15.102:0/215260877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81000d0 0x7fadc8195920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fadc400c8f0 tx=0x7fadc400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:37.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadbeffd640 1 -- 172.21.15.102:0/215260877 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc40041f0 con 0x7fadc81000d0 2024-05-13T22:45:37.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadbeffd640 1 -- 172.21.15.102:0/215260877 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fadc4007e00 con 0x7fadc81000d0 2024-05-13T22:45:37.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadbeffd640 1 -- 172.21.15.102:0/215260877 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc400f450 con 0x7fadc81000d0 2024-05-13T22:45:37.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcff51640 1 -- 172.21.15.102:0/215260877 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc819afe0 con 0x7fadc81000d0 2024-05-13T22:45:37.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.063+0000 7fadcff51640 1 -- 172.21.15.102:0/215260877 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc819b4b0 con 0x7fadc81000d0 2024-05-13T22:45:37.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.064+0000 7fadcff51640 1 -- 172.21.15.102:0/215260877 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc8108cb0 con 0x7fadc81000d0 2024-05-13T22:45:37.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.065+0000 7fadbeffd640 1 -- 172.21.15.102:0/215260877 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fadc4004350 con 0x7fadc81000d0 2024-05-13T22:45:37.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.065+0000 7fadbeffd640 1 --2- 172.21.15.102:0/215260877 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fadb406c450 0x7fadb406e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.065+0000 7fadbeffd640 1 -- 172.21.15.102:0/215260877 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fadc4013070 con 0x7fadc81000d0 2024-05-13T22:45:37.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.065+0000 7fadcd4ec640 1 --2- 172.21.15.102:0/215260877 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fadb406c450 0x7fadb406e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.066+0000 7fadcd4ec640 1 --2- 172.21.15.102:0/215260877 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fadb406c450 0x7fadb406e910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fadc8196e30 tx=0x7fadb8000bc0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:37.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.070+0000 7fadbeffd640 1 -- 172.21.15.102:0/215260877 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fadc4056070 con 0x7fadc81000d0 2024-05-13T22:45:37.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:37 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3009290530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T22:45:37.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.241+0000 7fadcff51640 1 -- 172.21.15.102:0/215260877 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fadc819b9e0 con 0x7fadc81000d0 2024-05-13T22:45:37.220 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.242+0000 7fadbeffd640 1 -- 172.21.15.102:0/215260877 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadc4059690 con 0x7fadc81000d0 2024-05-13T22:45:37.220 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-05-13T22:45:37.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.244+0000 7fadcff51640 1 -- 172.21.15.102:0/215260877 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fadb406c450 msgr2=0x7fadb406e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.222 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.244+0000 7fadcff51640 1 --2- 172.21.15.102:0/215260877 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fadb406c450 0x7fadb406e910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fadc8196e30 tx=0x7fadb8000bc0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.244+0000 7fadcff51640 1 -- 172.21.15.102:0/215260877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81000d0 msgr2=0x7fadc8195920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.244+0000 7fadcff51640 1 --2- 172.21.15.102:0/215260877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81000d0 0x7fadc8195920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fadc400c8f0 tx=0x7fadc400cdb0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.244+0000 7fadcff51640 1 -- 172.21.15.102:0/215260877 shutdown_connections 2024-05-13T22:45:37.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.244+0000 7fadcff51640 1 --2- 172.21.15.102:0/215260877 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fadb406c450 0x7fadb406e910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.244+0000 7fadcff51640 1 --2- 172.21.15.102:0/215260877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc81012c0 0x7fadc8195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.244+0000 7fadcff51640 1 --2- 172.21.15.102:0/215260877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc81000d0 0x7fadc8195920 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.244+0000 7fadcff51640 1 -- 172.21.15.102:0/215260877 >> 172.21.15.102:0/215260877 conn(0x7fadc80fb620 msgr2=0x7fadc81044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:37.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.244+0000 7fadcff51640 1 -- 172.21.15.102:0/215260877 shutdown_connections 2024-05-13T22:45:37.223 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.244+0000 7fadcff51640 1 -- 172.21.15.102:0/215260877 wait complete. 2024-05-13T22:45:37.233 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-13T22:45:37.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.357+0000 7f8f6c608640 1 -- 172.21.15.102:0/802019127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f641001a0 msgr2=0x7f8f64100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.357+0000 7f8f6c608640 1 --2- 172.21.15.102:0/802019127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f641001a0 0x7f8f64100620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009880 tx=0x7f8f5802bce0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.357+0000 7f8f6c608640 1 -- 172.21.15.102:0/802019127 shutdown_connections 2024-05-13T22:45:37.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.357+0000 7f8f6c608640 1 --2- 172.21.15.102:0/802019127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f641001a0 0x7f8f64100620 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.357+0000 7f8f6c608640 1 --2- 172.21.15.102:0/802019127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f640fefb0 0x7f8f640ff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.357+0000 7f8f6c608640 1 -- 172.21.15.102:0/802019127 >> 172.21.15.102:0/802019127 conn(0x7f8f640fa520 msgr2=0x7f8f640fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:37.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f6c608640 1 -- 172.21.15.102:0/802019127 shutdown_connections 2024-05-13T22:45:37.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f6c608640 1 -- 172.21.15.102:0/802019127 wait complete. 2024-05-13T22:45:37.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f6c608640 1 Processor -- start 2024-05-13T22:45:37.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f6c608640 1 -- start start 2024-05-13T22:45:37.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f6c608640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f640fefb0 0x7f8f64068f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f6c608640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f641001a0 0x7f8f6419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f6c608640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6419ad20 con 0x7f8f640fefb0 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f6c608640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6419ae60 con 0x7f8f641001a0 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f6a3a4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f640fefb0 0x7f8f64068f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f6a3a4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f640fefb0 0x7f8f64068f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53554/0 (socket says 172.21.15.102:53554) 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f6a3a4640 1 -- 172.21.15.102:0/611903493 learned_addr learned my addr 172.21.15.102:0/611903493 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.358+0000 7f8f69ba3640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f641001a0 0x7f8f6419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.359+0000 7f8f6a3a4640 1 -- 172.21.15.102:0/611903493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f641001a0 msgr2=0x7f8f6419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.359+0000 7f8f6a3a4640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f641001a0 0x7f8f6419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.359+0000 7f8f6a3a4640 1 -- 172.21.15.102:0/611903493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58009560 con 0x7f8f640fefb0 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.359+0000 7f8f69ba3640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f641001a0 0x7f8f6419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.359+0000 7f8f6a3a4640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f640fefb0 0x7f8f64068f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00d8d0 tx=0x7f8f5c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.359+0000 7f8f637fe640 1 -- 172.21.15.102:0/611903493 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f5c009a30 con 0x7f8f640fefb0 2024-05-13T22:45:37.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.359+0000 7f8f637fe640 1 -- 172.21.15.102:0/611903493 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8f5c009b90 con 0x7f8f640fefb0 2024-05-13T22:45:37.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.359+0000 7f8f6c608640 1 -- 172.21.15.102:0/611903493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6419b140 con 0x7f8f640fefb0 2024-05-13T22:45:37.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.359+0000 7f8f637fe640 1 -- 172.21.15.102:0/611903493 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f5c00f660 con 0x7f8f640fefb0 2024-05-13T22:45:37.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.359+0000 7f8f6c608640 1 -- 172.21.15.102:0/611903493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6419b6a0 con 0x7f8f640fefb0 2024-05-13T22:45:37.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.360+0000 7f8f6c608640 1 -- 172.21.15.102:0/611903493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f64107d00 con 0x7f8f640fefb0 2024-05-13T22:45:37.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.361+0000 7f8f637fe640 1 -- 172.21.15.102:0/611903493 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8f5c009d00 con 0x7f8f640fefb0 2024-05-13T22:45:37.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.361+0000 7f8f637fe640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8f4c06c450 0x7f8f4c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.361+0000 7f8f637fe640 1 -- 172.21.15.102:0/611903493 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8f5c08b6a0 con 0x7f8f640fefb0 2024-05-13T22:45:37.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.361+0000 7f8f69ba3640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8f4c06c450 0x7f8f4c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.361+0000 7f8f69ba3640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8f4c06c450 0x7f8f4c06e910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f64068ca0 tx=0x7f8f5802c7c0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:37.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.365+0000 7f8f637fe640 1 -- 172.21.15.102:0/611903493 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8f5c055e20 con 0x7f8f640fefb0 2024-05-13T22:45:37.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:37 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3009290530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T22:45:37.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.551+0000 7f8f6c608640 1 -- 172.21.15.102:0/611903493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f8f6419b8c0 con 0x7f8f640fefb0 2024-05-13T22:45:37.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.551+0000 7f8f637fe640 1 -- 172.21.15.102:0/611903493 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8f5c059440 con 0x7f8f640fefb0 2024-05-13T22:45:37.530 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-05-13T22:45:37.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.553+0000 7f8f6c608640 1 -- 172.21.15.102:0/611903493 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8f4c06c450 msgr2=0x7f8f4c06e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.553+0000 7f8f6c608640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8f4c06c450 0x7f8f4c06e910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f64068ca0 tx=0x7f8f5802c7c0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.554+0000 7f8f6c608640 1 -- 172.21.15.102:0/611903493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f640fefb0 msgr2=0x7f8f64068f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.554+0000 7f8f6c608640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f640fefb0 0x7f8f64068f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00d8d0 tx=0x7f8f5c00dd90 comp rx=0 tx=0).stop 2024-05-13T22:45:37.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.554+0000 7f8f6c608640 1 -- 172.21.15.102:0/611903493 shutdown_connections 2024-05-13T22:45:37.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.554+0000 7f8f6c608640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f8f4c06c450 0x7f8f4c06e910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.554+0000 7f8f6c608640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f641001a0 0x7f8f6419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.554+0000 7f8f6c608640 1 --2- 172.21.15.102:0/611903493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f640fefb0 0x7f8f64068f30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.554+0000 7f8f6c608640 1 -- 172.21.15.102:0/611903493 >> 172.21.15.102:0/611903493 conn(0x7f8f640fa520 msgr2=0x7f8f641033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:37.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.554+0000 7f8f6c608640 1 -- 172.21.15.102:0/611903493 shutdown_connections 2024-05-13T22:45:37.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.554+0000 7f8f6c608640 1 -- 172.21.15.102:0/611903493 wait complete. 2024-05-13T22:45:37.544 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-13T22:45:37.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.670+0000 7f3978fc5640 1 -- 172.21.15.102:0/3611028675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3974100180 msgr2=0x7f3974100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.670+0000 7f3978fc5640 1 --2- 172.21.15.102:0/3611028675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3974100180 0x7f3974100600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3968009880 tx=0x7f396802bce0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.670+0000 7f3978fc5640 1 -- 172.21.15.102:0/3611028675 shutdown_connections 2024-05-13T22:45:37.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.670+0000 7f3978fc5640 1 --2- 172.21.15.102:0/3611028675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3974100180 0x7f3974100600 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.670+0000 7f3978fc5640 1 --2- 172.21.15.102:0/3611028675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f39740fef90 0x7f39740ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.671+0000 7f3978fc5640 1 -- 172.21.15.102:0/3611028675 >> 172.21.15.102:0/3611028675 conn(0x7f39740fa4e0 msgr2=0x7f39740fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:37.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.671+0000 7f3978fc5640 1 -- 172.21.15.102:0/3611028675 shutdown_connections 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.671+0000 7f3978fc5640 1 -- 172.21.15.102:0/3611028675 wait complete. 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.671+0000 7f3978fc5640 1 Processor -- start 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.671+0000 7f3978fc5640 1 -- start start 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.671+0000 7f3978fc5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f39740fef90 0x7f3974195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.671+0000 7f3978fc5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3974100180 0x7f3974195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.671+0000 7f3978fc5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3974196450 con 0x7f3974100180 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.671+0000 7f3978fc5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39741965c0 con 0x7f39740fef90 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f3971d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3974100180 0x7f3974195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f3971d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3974100180 0x7f3974195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53576/0 (socket says 172.21.15.102:53576) 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f3971d9b640 1 -- 172.21.15.102:0/251728860 learned_addr learned my addr 172.21.15.102:0/251728860 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f397259c640 1 --2- 172.21.15.102:0/251728860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f39740fef90 0x7f3974195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f3971d9b640 1 -- 172.21.15.102:0/251728860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f39740fef90 msgr2=0x7f3974195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f3971d9b640 1 --2- 172.21.15.102:0/251728860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f39740fef90 0x7f3974195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f3971d9b640 1 -- 172.21.15.102:0/251728860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3968009560 con 0x7f3974100180 2024-05-13T22:45:37.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f3971d9b640 1 --2- 172.21.15.102:0/251728860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3974100180 0x7f3974195e80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3968009230 tx=0x7f3968004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:37.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f39637fe640 1 -- 172.21.15.102:0/251728860 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3968004520 con 0x7f3974100180 2024-05-13T22:45:37.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f39637fe640 1 -- 172.21.15.102:0/251728860 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f39680054b0 con 0x7f3974100180 2024-05-13T22:45:37.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f3978fc5640 1 -- 172.21.15.102:0/251728860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397419aff0 con 0x7f3974100180 2024-05-13T22:45:37.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f39637fe640 1 -- 172.21.15.102:0/251728860 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3968034670 con 0x7f3974100180 2024-05-13T22:45:37.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.672+0000 7f3978fc5640 1 -- 172.21.15.102:0/251728860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397419b4c0 con 0x7f3974100180 2024-05-13T22:45:37.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.673+0000 7f3978fc5640 1 -- 172.21.15.102:0/251728860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3974107d20 con 0x7f3974100180 2024-05-13T22:45:37.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.674+0000 7f39637fe640 1 -- 172.21.15.102:0/251728860 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f39680038b0 con 0x7f3974100180 2024-05-13T22:45:37.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.674+0000 7f39637fe640 1 --2- 172.21.15.102:0/251728860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f395c06c450 0x7f395c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.674+0000 7f397259c640 1 --2- 172.21.15.102:0/251728860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f395c06c450 0x7f395c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.674+0000 7f39637fe640 1 -- 172.21.15.102:0/251728860 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f39680adda0 con 0x7f3974100180 2024-05-13T22:45:37.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.674+0000 7f397259c640 1 --2- 172.21.15.102:0/251728860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f395c06c450 0x7f395c06e910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3964005fd0 tx=0x7f396400b390 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:37.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.678+0000 7f39637fe640 1 -- 172.21.15.102:0/251728860 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f39680785d0 con 0x7f3974100180 2024-05-13T22:45:37.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.855+0000 7f3978fc5640 1 -- 172.21.15.102:0/251728860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f397419b6b0 con 0x7f3974100180 2024-05-13T22:45:37.833 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.855+0000 7f39637fe640 1 -- 172.21.15.102:0/251728860 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f396807bbf0 con 0x7f3974100180 2024-05-13T22:45:37.834 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-05-13T22:45:37.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.857+0000 7f3978fc5640 1 -- 172.21.15.102:0/251728860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f395c06c450 msgr2=0x7f395c06e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.857+0000 7f3978fc5640 1 --2- 172.21.15.102:0/251728860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f395c06c450 0x7f395c06e910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3964005fd0 tx=0x7f396400b390 comp rx=0 tx=0).stop 2024-05-13T22:45:37.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.857+0000 7f3978fc5640 1 -- 172.21.15.102:0/251728860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3974100180 msgr2=0x7f3974195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.857+0000 7f3978fc5640 1 --2- 172.21.15.102:0/251728860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3974100180 0x7f3974195e80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3968009230 tx=0x7f3968004e30 comp rx=0 tx=0).stop 2024-05-13T22:45:37.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.857+0000 7f3978fc5640 1 -- 172.21.15.102:0/251728860 shutdown_connections 2024-05-13T22:45:37.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.857+0000 7f3978fc5640 1 --2- 172.21.15.102:0/251728860 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f395c06c450 0x7f395c06e910 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.857+0000 7f3978fc5640 1 --2- 172.21.15.102:0/251728860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3974100180 0x7f3974195e80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.857+0000 7f3978fc5640 1 --2- 172.21.15.102:0/251728860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f39740fef90 0x7f3974195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.857+0000 7f3978fc5640 1 -- 172.21.15.102:0/251728860 >> 172.21.15.102:0/251728860 conn(0x7f39740fa4e0 msgr2=0x7f3974067350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:37.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.857+0000 7f3978fc5640 1 -- 172.21.15.102:0/251728860 shutdown_connections 2024-05-13T22:45:37.837 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.857+0000 7f3978fc5640 1 -- 172.21.15.102:0/251728860 wait complete. 2024-05-13T22:45:37.847 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-13T22:45:37.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.969+0000 7fc0464e5640 1 -- 172.21.15.102:0/2972506059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0400ff1c0 msgr2=0x7fc0400ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.969+0000 7fc0464e5640 1 --2- 172.21.15.102:0/2972506059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0400ff1c0 0x7fc0400ff640 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc034009880 tx=0x7fc03402bce0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.969+0000 7fc0464e5640 1 -- 172.21.15.102:0/2972506059 shutdown_connections 2024-05-13T22:45:37.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.969+0000 7fc0464e5640 1 --2- 172.21.15.102:0/2972506059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0400ff1c0 0x7fc0400ff640 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.969+0000 7fc0464e5640 1 --2- 172.21.15.102:0/2972506059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0401005c0 0x7fc0401009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.948 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.969+0000 7fc0464e5640 1 -- 172.21.15.102:0/2972506059 >> 172.21.15.102:0/2972506059 conn(0x7fc0400f9cb0 msgr2=0x7fc0400fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:37.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.969+0000 7fc0464e5640 1 -- 172.21.15.102:0/2972506059 shutdown_connections 2024-05-13T22:45:37.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.969+0000 7fc0464e5640 1 -- 172.21.15.102:0/2972506059 wait complete. 2024-05-13T22:45:37.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.970+0000 7fc0464e5640 1 Processor -- start 2024-05-13T22:45:37.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.970+0000 7fc0464e5640 1 -- start start 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.970+0000 7fc0464e5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0400ff1c0 0x7fc0400fec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.970+0000 7fc0464e5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0401005c0 0x7fc04019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.970+0000 7fc0464e5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04019ac10 con 0x7fc0401005c0 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.970+0000 7fc0464e5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04019ad50 con 0x7fc0400ff1c0 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.970+0000 7fc03f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0401005c0 0x7fc04019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.970+0000 7fc03f7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0401005c0 0x7fc04019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53592/0 (socket says 172.21.15.102:53592) 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.970+0000 7fc03f7fe640 1 -- 172.21.15.102:0/366290882 learned_addr learned my addr 172.21.15.102:0/366290882 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.970+0000 7fc03ffff640 1 --2- 172.21.15.102:0/366290882 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0400ff1c0 0x7fc0400fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.971+0000 7fc03f7fe640 1 -- 172.21.15.102:0/366290882 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0400ff1c0 msgr2=0x7fc0400fec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.971+0000 7fc03f7fe640 1 --2- 172.21.15.102:0/366290882 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0400ff1c0 0x7fc0400fec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.971+0000 7fc03f7fe640 1 -- 172.21.15.102:0/366290882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc034009560 con 0x7fc0401005c0 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.971+0000 7fc03f7fe640 1 --2- 172.21.15.102:0/366290882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0401005c0 0x7fc04019a640 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc034009230 tx=0x7fc034004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:37.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.971+0000 7fc03d7fa640 1 -- 172.21.15.102:0/366290882 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc034004520 con 0x7fc0401005c0 2024-05-13T22:45:37.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.971+0000 7fc03d7fa640 1 -- 172.21.15.102:0/366290882 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc0340054b0 con 0x7fc0401005c0 2024-05-13T22:45:37.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.971+0000 7fc0464e5640 1 -- 172.21.15.102:0/366290882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04019afa0 con 0x7fc0401005c0 2024-05-13T22:45:37.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.971+0000 7fc03d7fa640 1 -- 172.21.15.102:0/366290882 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc034034440 con 0x7fc0401005c0 2024-05-13T22:45:37.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.971+0000 7fc0464e5640 1 -- 172.21.15.102:0/366290882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04019b470 con 0x7fc0401005c0 2024-05-13T22:45:37.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.972+0000 7fc0464e5640 1 -- 172.21.15.102:0/366290882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc040107be0 con 0x7fc0401005c0 2024-05-13T22:45:37.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.972+0000 7fc03d7fa640 1 -- 172.21.15.102:0/366290882 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc0340037f0 con 0x7fc0401005c0 2024-05-13T22:45:37.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.973+0000 7fc03d7fa640 1 --2- 172.21.15.102:0/366290882 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc02806c510 0x7fc02806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:37.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.973+0000 7fc03ffff640 1 --2- 172.21.15.102:0/366290882 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc02806c510 0x7fc02806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:37.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.973+0000 7fc03d7fa640 1 -- 172.21.15.102:0/366290882 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc0340add20 con 0x7fc0401005c0 2024-05-13T22:45:37.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.973+0000 7fc03ffff640 1 --2- 172.21.15.102:0/366290882 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc02806c510 0x7fc02806e9d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc030005fd0 tx=0x7fc03000b390 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:37.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:37.977+0000 7fc03d7fa640 1 -- 172.21.15.102:0/366290882 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc034078550 con 0x7fc0401005c0 2024-05-13T22:45:38.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.207+0000 7fc0464e5640 1 -- 172.21.15.102:0/366290882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc04019b8b0 con 0x7fc0401005c0 2024-05-13T22:45:38.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.208+0000 7fc03d7fa640 1 -- 172.21.15.102:0/366290882 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4753 (secure 0 0 0) 0x7fc03407bb70 con 0x7fc0401005c0 2024-05-13T22:45:38.186 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:45:38.186 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24281,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/244897459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":244897459},{"type":"v1","addr":"172.21.15.130:6825","nonce":244897459}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:45:28.142444+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14498},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14498":{"gid":14498,"name":"cephfs.smithi102.sdylgv","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6829/2977397703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2977397703},{"type":"v1","addr":"172.21.15.102:6829","nonce":2977397703}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14514":{"gid":14514,"name":"cephfs.smithi102.cgkcwz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6827/1826944061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1826944061},{"type":"v1","addr":"172.21.15.102:6827","nonce":1826944061}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-13T22:45:38.187 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:45:38.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.210+0000 7fc0464e5640 1 -- 172.21.15.102:0/366290882 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc02806c510 msgr2=0x7fc02806e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:38.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.210+0000 7fc0464e5640 1 --2- 172.21.15.102:0/366290882 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc02806c510 0x7fc02806e9d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc030005fd0 tx=0x7fc03000b390 comp rx=0 tx=0).stop 2024-05-13T22:45:38.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.210+0000 7fc0464e5640 1 -- 172.21.15.102:0/366290882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0401005c0 msgr2=0x7fc04019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:38.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.210+0000 7fc0464e5640 1 --2- 172.21.15.102:0/366290882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0401005c0 0x7fc04019a640 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc034009230 tx=0x7fc034004f00 comp rx=0 tx=0).stop 2024-05-13T22:45:38.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.210+0000 7fc0464e5640 1 -- 172.21.15.102:0/366290882 shutdown_connections 2024-05-13T22:45:38.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.210+0000 7fc0464e5640 1 --2- 172.21.15.102:0/366290882 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc02806c510 0x7fc02806e9d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:38.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.210+0000 7fc0464e5640 1 --2- 172.21.15.102:0/366290882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0401005c0 0x7fc04019a640 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:38.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.210+0000 7fc0464e5640 1 --2- 172.21.15.102:0/366290882 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc0400ff1c0 0x7fc0400fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:38.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.210+0000 7fc0464e5640 1 -- 172.21.15.102:0/366290882 >> 172.21.15.102:0/366290882 conn(0x7fc0400f9cb0 msgr2=0x7fc040103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:38.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.210+0000 7fc0464e5640 1 -- 172.21.15.102:0/366290882 shutdown_connections 2024-05-13T22:45:38.189 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:38.210+0000 7fc0464e5640 1 -- 172.21.15.102:0/366290882 wait complete. 2024-05-13T22:45:38.200 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 1, 'flags': 50} 2024-05-13T22:45:38.201 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-13T22:45:38.229 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-13T22:45:38.229 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-05-13T22:45:38.229 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-13T22:45:38.229 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-13T22:45:38.229 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-13T22:45:38.229 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-13T22:45:38.230 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-13T22:45:38.230 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-13T22:45:38.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:38.230 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-05-13T22:45:38.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:38.247 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link delete ceph-brx 2024-05-13T22:45:38.317 INFO:teuthology.orchestra.run.smithi102.stderr:Cannot find device "ceph-brx" 2024-05-13T22:45:38.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-13T22:45:38.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:38.319 DEBUG:teuthology.orchestra.run.smithi130:> ip netns list 2024-05-13T22:45:38.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:38.340 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link delete ceph-brx 2024-05-13T22:45:38.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:38 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/215260877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-13T22:45:38.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:38 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/611903493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-13T22:45:38.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:38 smithi130 ceph-mon[36591]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-13T22:45:38.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:38 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/251728860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-13T22:45:38.415 INFO:teuthology.orchestra.run.smithi130.stderr:Cannot find device "ceph-brx" 2024-05-13T22:45:38.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-13T22:45:38.417 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-13T22:45:38.417 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-13T22:45:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:38 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/215260877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-13T22:45:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:38 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/611903493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-13T22:45:38.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:38 smithi102 ceph-mon[28599]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-13T22:45:38.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:38 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/251728860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-13T22:45:38.622 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:39 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/366290882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-13T22:45:39.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:39.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:39.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:39.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:45:39.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:39.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:39 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:45:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:39 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/366290882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-13T22:45:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:45:39.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:45:39.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:45:39.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:39 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:45:40.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.109+0000 7f9ec13f4700 1 -- 172.21.15.102:0/691303928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ebc100da0 msgr2=0x7f9ebc101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:40.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.109+0000 7f9ec13f4700 1 --2- 172.21.15.102:0/691303928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ebc100da0 0x7f9ebc101180 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400b0a0 tx=0x7f9eb400b3b0 comp rx=0 tx=0).stop 2024-05-13T22:45:40.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.109+0000 7f9ec13f4700 1 -- 172.21.15.102:0/691303928 shutdown_connections 2024-05-13T22:45:40.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.109+0000 7f9ec13f4700 1 --2- 172.21.15.102:0/691303928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ebc101750 0x7f9ebc1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:40.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.109+0000 7f9ec13f4700 1 --2- 172.21.15.102:0/691303928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ebc100da0 0x7f9ebc101180 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:40.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.109+0000 7f9ec13f4700 1 -- 172.21.15.102:0/691303928 >> 172.21.15.102:0/691303928 conn(0x7f9ebc0fc630 msgr2=0x7f9ebc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:40.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.109+0000 7f9ec13f4700 1 -- 172.21.15.102:0/691303928 shutdown_connections 2024-05-13T22:45:40.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.109+0000 7f9ec13f4700 1 -- 172.21.15.102:0/691303928 wait complete. 2024-05-13T22:45:40.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.109+0000 7f9ec13f4700 1 Processor -- start 2024-05-13T22:45:40.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ec13f4700 1 -- start start 2024-05-13T22:45:40.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ec13f4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ebc100da0 0x7f9ebc19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:40.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ec13f4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ebc101750 0x7f9ebc19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:40.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ec13f4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc19d170 con 0x7f9ebc100da0 2024-05-13T22:45:40.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ec13f4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc196620 con 0x7f9ebc101750 2024-05-13T22:45:40.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ebaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ebc100da0 0x7f9ebc19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:40.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ebaffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ebc100da0 0x7f9ebc19c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53790/0 (socket says 172.21.15.102:53790) 2024-05-13T22:45:40.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ebaffd700 1 -- 172.21.15.102:0/3211619200 learned_addr learned my addr 172.21.15.102:0/3211619200 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:40.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ebaffd700 1 -- 172.21.15.102:0/3211619200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ebc101750 msgr2=0x7f9ebc19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:40.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9eba7fc700 1 --2- 172.21.15.102:0/3211619200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ebc101750 0x7f9ebc19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:40.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ebaffd700 1 --2- 172.21.15.102:0/3211619200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ebc101750 0x7f9ebc19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:40.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ebaffd700 1 -- 172.21.15.102:0/3211619200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4009d00 con 0x7f9ebc100da0 2024-05-13T22:45:40.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ebaffd700 1 --2- 172.21.15.102:0/3211619200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ebc100da0 0x7f9ebc19c5a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4000f80 tx=0x7f9eb400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:40.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ea3fff700 1 -- 172.21.15.102:0/3211619200 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb400e040 con 0x7f9ebc100da0 2024-05-13T22:45:40.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ea3fff700 1 -- 172.21.15.102:0/3211619200 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9eb4003de0 con 0x7f9ebc100da0 2024-05-13T22:45:40.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.110+0000 7f9ec13f4700 1 -- 172.21.15.102:0/3211619200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ebc1968a0 con 0x7f9ebc100da0 2024-05-13T22:45:40.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.111+0000 7f9ea3fff700 1 -- 172.21.15.102:0/3211619200 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb401d9a0 con 0x7f9ebc100da0 2024-05-13T22:45:40.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.111+0000 7f9ec13f4700 1 -- 172.21.15.102:0/3211619200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc196d90 con 0x7f9ebc100da0 2024-05-13T22:45:40.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.111+0000 7f9ec13f4700 1 -- 172.21.15.102:0/3211619200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ebc109140 con 0x7f9ebc100da0 2024-05-13T22:45:40.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.112+0000 7f9ea3fff700 1 -- 172.21.15.102:0/3211619200 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9eb4004040 con 0x7f9ebc100da0 2024-05-13T22:45:40.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.112+0000 7f9ea3fff700 1 --2- 172.21.15.102:0/3211619200 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9ea406c4e0 0x7f9ea406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:40.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.113+0000 7f9ea3fff700 1 -- 172.21.15.102:0/3211619200 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9eb408d1f0 con 0x7f9ebc100da0 2024-05-13T22:45:40.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.113+0000 7f9eba7fc700 1 --2- 172.21.15.102:0/3211619200 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9ea406c4e0 0x7f9ea406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:40.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.113+0000 7f9eba7fc700 1 --2- 172.21.15.102:0/3211619200 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9ea406c4e0 0x7f9ea406e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0005950 tx=0x7f9eb00058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:40.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.117+0000 7f9ea3fff700 1 -- 172.21.15.102:0/3211619200 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9eb405b400 con 0x7f9ebc100da0 2024-05-13T22:45:40.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:40 smithi102 ceph-mon[28599]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.3 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-13T22:45:40.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.348+0000 7f9ec13f4700 1 -- 172.21.15.102:0/3211619200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ebc02cca0 con 0x7f9ebc100da0 2024-05-13T22:45:40.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.348+0000 7f9ea3fff700 1 -- 172.21.15.102:0/3211619200 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f9eb405af90 con 0x7f9ebc100da0 2024-05-13T22:45:40.328 INFO:teuthology.orchestra.run.smithi102.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-13T22:45:40.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.350+0000 7f9ec13f4700 1 -- 172.21.15.102:0/3211619200 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9ea406c4e0 msgr2=0x7f9ea406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:40.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.350+0000 7f9ec13f4700 1 --2- 172.21.15.102:0/3211619200 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9ea406c4e0 0x7f9ea406e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0005950 tx=0x7f9eb00058e0 comp rx=0 tx=0).stop 2024-05-13T22:45:40.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.350+0000 7f9ec13f4700 1 -- 172.21.15.102:0/3211619200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ebc100da0 msgr2=0x7f9ebc19c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:40.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.350+0000 7f9ec13f4700 1 --2- 172.21.15.102:0/3211619200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ebc100da0 0x7f9ebc19c5a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4000f80 tx=0x7f9eb400cdd0 comp rx=0 tx=0).stop 2024-05-13T22:45:40.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.351+0000 7f9ec13f4700 1 -- 172.21.15.102:0/3211619200 shutdown_connections 2024-05-13T22:45:40.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.351+0000 7f9ec13f4700 1 --2- 172.21.15.102:0/3211619200 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f9ea406c4e0 0x7f9ea406e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:40.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.351+0000 7f9ec13f4700 1 --2- 172.21.15.102:0/3211619200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ebc100da0 0x7f9ebc19c5a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:40.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.351+0000 7f9ec13f4700 1 --2- 172.21.15.102:0/3211619200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ebc101750 0x7f9ebc19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:40.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.351+0000 7f9ec13f4700 1 -- 172.21.15.102:0/3211619200 >> 172.21.15.102:0/3211619200 conn(0x7f9ebc0fc630 msgr2=0x7f9ebc0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:40.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.351+0000 7f9ec13f4700 1 -- 172.21.15.102:0/3211619200 shutdown_connections 2024-05-13T22:45:40.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:40.351+0000 7f9ec13f4700 1 -- 172.21.15.102:0/3211619200 wait complete. 2024-05-13T22:45:40.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:40 smithi130 ceph-mon[36591]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.3 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-13T22:45:41.038 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-13T22:45:41.038 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-13T22:45:41.038 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi102.front.sepia.ceph.com 2024-05-13T22:45:41.039 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-13T22:45:41.039 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-13T22:45:41.039 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-13T22:45:41.039 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-13T22:45:41.039 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-13T22:45:41.039 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-13T22:45:41.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:41.039 DEBUG:teuthology.orchestra.run.smithi102:> ip addr 2024-05-13T22:45:41.096 INFO:teuthology.orchestra.run.smithi102.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: inet 127.0.0.1/8 scope host lo 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 ::1/128 scope host 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dc brd ff:ff:ff:ff:ff:ff 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f0 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:d8:d9:dd brd ff:ff:ff:ff:ff:ff 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: altname enp7s0f1 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-13T22:45:41.097 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:68 brd ff:ff:ff:ff:ff:ff 2024-05-13T22:45:41.098 INFO:teuthology.orchestra.run.smithi102.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-13T22:45:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: link/ether 0c:c4:7a:8f:d7:69 brd ff:ff:ff:ff:ff:ff 2024-05-13T22:45:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: inet 172.21.15.102/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-13T22:45:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft 42226sec preferred_lft 42226sec 2024-05-13T22:45:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: inet6 fe80::ec4:7aff:fe8f:d769/64 scope link 2024-05-13T22:45:41.098 INFO:teuthology.orchestra.run.smithi102.stdout: valid_lft forever preferred_lft forever 2024-05-13T22:45:41.098 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-13T22:45:41.099 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-13T22:45:41.099 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-05-13T22:45:41.099 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link add name ceph-brx type bridge 2024-05-13T22:45:41.099 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip addr flush dev ceph-brx 2024-05-13T22:45:41.099 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set ceph-brx up 2024-05-13T22:45:41.099 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-13T22:45:41.099 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-05-13T22:45:41.186 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-13T22:45:41.270 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:41 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3211619200' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-13T22:45:41.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:41.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:41.291 DEBUG:teuthology.orchestra.run.smithi102:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-13T22:45:41.317 INFO:teuthology.orchestra.run.smithi102.stdout:1 2024-05-13T22:45:41.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:41.318 DEBUG:teuthology.orchestra.run.smithi102:> route 2024-05-13T22:45:41.390 INFO:teuthology.orchestra.run.smithi102.stdout:Kernel IP routing table 2024-05-13T22:45:41.390 INFO:teuthology.orchestra.run.smithi102.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-13T22:45:41.390 INFO:teuthology.orchestra.run.smithi102.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-13T22:45:41.391 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-13T22:45:41.391 INFO:teuthology.orchestra.run.smithi102.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-13T22:45:41.391 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-13T22:45:41.391 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-05-13T22:45:41.391 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-13T22:45:41.391 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-13T22:45:41.391 DEBUG:teuthology.orchestra.run.smithi102:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-13T22:45:41.391 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-05-13T22:45:41.394 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:41 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3211619200' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-13T22:45:41.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-13T22:45:41.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:41.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:41.680 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list 2024-05-13T22:45:41.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:41.696 DEBUG:teuthology.orchestra.run.smithi102:> ip netns list-id 2024-05-13T22:45:41.752 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-13T22:45:41.752 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-05-13T22:45:41.752 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-13T22:45:41.752 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-13T22:45:41.752 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-05-13T22:45:41.840 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-13T22:45:41.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:41.870 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-13T22:45:41.871 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-13T22:45:41.871 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-05-13T22:45:41.871 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-13T22:45:41.871 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-13T22:45:41.871 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-13T22:45:41.871 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-13T22:45:41.871 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-13T22:45:41.871 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-05-13T22:45:41.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-13T22:45:41.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:41.997 DEBUG:teuthology.orchestra.run.smithi102:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-13T22:45:41.997 DEBUG:teuthology.orchestra.run.smithi102:> set -e 2024-05-13T22:45:41.997 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set brx.0 up 2024-05-13T22:45:41.997 DEBUG:teuthology.orchestra.run.smithi102:> sudo ip link set dev brx.0 master ceph-brx 2024-05-13T22:45:41.997 DEBUG:teuthology.orchestra.run.smithi102:> ') 2024-05-13T22:45:42.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-13T22:45:42.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:42.074 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-13T22:45:42.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-13T22:45:42.074 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-13T22:45:42.089 INFO:teuthology.orchestra.run.smithi102.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-13T22:45:42.089 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-13T22:45:42.090 DEBUG:teuthology.orchestra.run.smithi102:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-13T22:45:42.141 DEBUG:teuthology.orchestra.run.smithi102:> sudo modprobe fuse 2024-05-13T22:45:42.207 DEBUG:teuthology.orchestra.run.smithi102:> cat /proc/self/mounts | awk '{print $2}' 2024-05-13T22:45:42.263 INFO:teuthology.orchestra.run.smithi102.stdout:/proc 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/sys 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/dev 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/security 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/shm 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/pts 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/run 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/cgroup 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/pstore 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/bpf 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/ 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/selinux 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-05-13T22:45:42.264 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/mqueue 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/hugepages 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/debug 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/tracing 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/fuse/connections 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/config 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-sysctl.service 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/nfs/rpc_pipefs 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/1000 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay 2024-05-13T22:45:42.265 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/3e95941b3bbfd175cd4b21d4127b7f2c61f799532c046732189d12947924fdbb/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/f0a7a2e438351349c9ecfa729ebf9bd6e89a50d48e6735d1ed1abceeedebdfcd/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/0 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/f7eea855652e1ce1b862bc01aadea55e90d526d358c6eefe8db42d4d5bb7ab96/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/d3d27ce7186b26323926de8c232f57672b1ce4bc37886ea3ec4b8115002932ca/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/ba9336e81e3c1476259638468c7e61c222435a1243f448946083d89b79ce0f35/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/73e8db395280b18e6311446e8a8407daad01d2c4d1dca09ddd04e30d2e6d6d2e/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/f0ac450c3f801300c36aa7f3e692383b566b385a029ad327e214bdabc157680f/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/a02895e86766682b70b519f93fc389eb29b30a6bc039a8cd2d3b63afb7d36abb/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/d5b5485f41398d1d51c657bcb2a6dad8f1f4fc362237f68e4623cfc6c50c4166/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/2860b879c0f98b670fa67bd4a5fe6a28c9c973f92e343ccc998c2cfd3c91a258/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/ae5c11b5db6717251fcc73bb9da56ef86aad21704afb74a46faf9808dc0d5923/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/78f27a1dc1b742c3e6ee91037d00e97529b54a32197bf72b4bbad1d4f4856c61/merged 2024-05-13T22:45:42.266 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/8ba958ce4f5d5e28f22cbb044bc8a0c730b6e80e98809f09e1f5557919509757/merged 2024-05-13T22:45:42.267 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns 2024-05-13T22:45:42.267 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-13T22:45:42.267 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-13T22:45:42.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:42.267 DEBUG:teuthology.orchestra.run.smithi102:> ls /sys/fs/fuse/connections 2024-05-13T22:45:42.319 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-13T22:45:42.319 DEBUG:teuthology.orchestra.run.smithi102:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-13T22:45:42.362 DEBUG:teuthology.orchestra.run.smithi102:> sudo modprobe fuse 2024-05-13T22:45:42.381 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:42 smithi102 ceph-mon[28599]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-13T22:45:42.387 DEBUG:teuthology.orchestra.run.smithi102:> cat /proc/self/mounts | awk '{print $2}' 2024-05-13T22:45:42.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:42 smithi130 ceph-mon[36591]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/proc 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/sys 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/dev 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/security 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/shm 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/pts 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/run 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/cgroup 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/pstore 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/bpf 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/ 2024-05-13T22:45:42.443 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/selinux 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/mqueue 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/dev/hugepages 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/debug 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/tracing 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/fs/fuse/connections 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/sys/kernel/config 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-sysctl.service 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/nfs/rpc_pipefs 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/1000 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/ceph 2024-05-13T22:45:42.444 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/3e95941b3bbfd175cd4b21d4127b7f2c61f799532c046732189d12947924fdbb/merged 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/f0a7a2e438351349c9ecfa729ebf9bd6e89a50d48e6735d1ed1abceeedebdfcd/merged 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/run/user/0 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/proc/sys/fs/binfmt_misc 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/f7eea855652e1ce1b862bc01aadea55e90d526d358c6eefe8db42d4d5bb7ab96/merged 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/d3d27ce7186b26323926de8c232f57672b1ce4bc37886ea3ec4b8115002932ca/merged 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/ba9336e81e3c1476259638468c7e61c222435a1243f448946083d89b79ce0f35/merged 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/73e8db395280b18e6311446e8a8407daad01d2c4d1dca09ddd04e30d2e6d6d2e/merged 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/f0ac450c3f801300c36aa7f3e692383b566b385a029ad327e214bdabc157680f/merged 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/a02895e86766682b70b519f93fc389eb29b30a6bc039a8cd2d3b63afb7d36abb/merged 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/d5b5485f41398d1d51c657bcb2a6dad8f1f4fc362237f68e4623cfc6c50c4166/merged 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/2860b879c0f98b670fa67bd4a5fe6a28c9c973f92e343ccc998c2cfd3c91a258/merged 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/ae5c11b5db6717251fcc73bb9da56ef86aad21704afb74a46faf9808dc0d5923/merged 2024-05-13T22:45:42.445 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/78f27a1dc1b742c3e6ee91037d00e97529b54a32197bf72b4bbad1d4f4856c61/merged 2024-05-13T22:45:42.446 INFO:teuthology.orchestra.run.smithi102.stdout:/var/lib/containers/storage/overlay/8ba958ce4f5d5e28f22cbb044bc8a0c730b6e80e98809f09e1f5557919509757/merged 2024-05-13T22:45:42.446 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns 2024-05-13T22:45:42.446 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-13T22:45:42.446 INFO:teuthology.orchestra.run.smithi102.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-13T22:45:42.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:42.447 DEBUG:teuthology.orchestra.run.smithi102:> ls /sys/fs/fuse/connections 2024-05-13T22:45:42.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi102.stderr:2024-05-13T22:45:42.475+0000 7fab6f03b480 -1 init, newargv = 0x5578a9fa03f0 newargc=15 2024-05-13T22:45:42.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi102.stderr:ceph-fuse[76488]: starting ceph client 2024-05-13T22:45:42.461 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi102.stderr:ceph-fuse[76488]: starting fuse 2024-05-13T22:45:42.471 INFO:teuthology.orchestra.run.smithi102.stdout:152 2024-05-13T22:45:42.471 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [152] 2024-05-13T22:45:42.471 DEBUG:teuthology.orchestra.run.smithi102:> sudo stdin-killer -- python3 -c ' 2024-05-13T22:45:42.471 DEBUG:teuthology.orchestra.run.smithi102:> import glob 2024-05-13T22:45:42.471 DEBUG:teuthology.orchestra.run.smithi102:> import re 2024-05-13T22:45:42.471 DEBUG:teuthology.orchestra.run.smithi102:> import os 2024-05-13T22:45:42.471 DEBUG:teuthology.orchestra.run.smithi102:> import subprocess 2024-05-13T22:45:42.471 DEBUG:teuthology.orchestra.run.smithi102:> 2024-05-13T22:45:42.471 DEBUG:teuthology.orchestra.run.smithi102:> def _find_admin_socket(client_name): 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> files = glob.glob(asok_path) 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> # Given a non-glob path, it better be there 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> if "*" not in asok_path: 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> assert(len(files) == 1) 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> return files[0] 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> for f in files: 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> contents = proc_f.read() 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> if mountpoint in contents: 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> return f 2024-05-13T22:45:42.472 DEBUG:teuthology.orchestra.run.smithi102:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-13T22:45:42.473 DEBUG:teuthology.orchestra.run.smithi102:> 2024-05-13T22:45:42.473 DEBUG:teuthology.orchestra.run.smithi102:> print(_find_admin_socket("client.0")) 2024-05-13T22:45:42.473 DEBUG:teuthology.orchestra.run.smithi102:> ' 2024-05-13T22:45:42.590 INFO:teuthology.orchestra.run.smithi102.stdout:/var/run/ceph/ceph-client.0.76488.asok 2024-05-13T22:45:42.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:42.601 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76488.asok 2024-05-13T22:45:42.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:42.602 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76488.asok status 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout: "metadata": { 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout: "entity_id": "0", 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout: "hostname": "smithi102", 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout: "pid": "76488", 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout: "root": "/" 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout: "dentry_count": 0, 2024-05-13T22:45:42.737 INFO:teuthology.orchestra.run.smithi102.stdout: "dentry_pinned_count": 0, 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: "id": 14554, 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: "inst": { 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: "name": { 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "client", 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: "num": 14554 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": { 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "192.168.0.1:0", 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 2305339701 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": { 2024-05-13T22:45:42.738 INFO:teuthology.orchestra.run.smithi102.stdout: "type": "v1", 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout: "addr": "192.168.0.1:0", 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout: "nonce": 2305339701 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout: "inst_str": "client.14554 192.168.0.1:0/2305339701", 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout: "addr_str": "192.168.0.1:0/2305339701", 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout: "inode_count": 1, 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_epoch": 17, 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_epoch": 47, 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_epoch_barrier": 0, 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout: "blocklisted": false, 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout: "fs_name": "cephfs" 2024-05-13T22:45:42.739 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:45:42.745 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-13T22:45:42.952 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:44.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.384+0000 7f32b9471700 1 -- 172.21.15.102:0/1193143759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b4105a30 msgr2=0x7f32b4105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:44.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.384+0000 7f32b9471700 1 --2- 172.21.15.102:0/1193143759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b4105a30 0x7f32b4105e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009910 tx=0x7f32a4009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:44.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.384+0000 7f32b9471700 1 -- 172.21.15.102:0/1193143759 shutdown_connections 2024-05-13T22:45:44.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.384+0000 7f32b9471700 1 --2- 172.21.15.102:0/1193143759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32b40ffa30 0x7f32b40ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:44.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.384+0000 7f32b9471700 1 --2- 172.21.15.102:0/1193143759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b4105a30 0x7f32b4105e10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:44.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.384+0000 7f32b9471700 1 -- 172.21.15.102:0/1193143759 >> 172.21.15.102:0/1193143759 conn(0x7f32b40fb520 msgr2=0x7f32b40fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:44.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.384+0000 7f32b9471700 1 -- 172.21.15.102:0/1193143759 shutdown_connections 2024-05-13T22:45:44.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.385+0000 7f32b9471700 1 -- 172.21.15.102:0/1193143759 wait complete. 2024-05-13T22:45:44.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.385+0000 7f32b9471700 1 Processor -- start 2024-05-13T22:45:44.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.385+0000 7f32b9471700 1 -- start start 2024-05-13T22:45:44.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.385+0000 7f32b9471700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32b40ffa30 0x7f32b419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:44.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.385+0000 7f32b9471700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b4105a30 0x7f32b419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.385+0000 7f32b9471700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b419d120 con 0x7f32b4105a30 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.385+0000 7f32b9471700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b41965d0 con 0x7f32b40ffa30 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f32b27fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b4105a30 0x7f32b419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f32b27fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b4105a30 0x7f32b419ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53814/0 (socket says 172.21.15.102:53814) 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f32b2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32b40ffa30 0x7f32b419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f32b27fc700 1 -- 172.21.15.102:0/2383939305 learned_addr learned my addr 172.21.15.102:0/2383939305 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f32b27fc700 1 -- 172.21.15.102:0/2383939305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32b40ffa30 msgr2=0x7f32b419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f32b27fc700 1 --2- 172.21.15.102:0/2383939305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32b40ffa30 0x7f32b419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f32b27fc700 1 -- 172.21.15.102:0/2383939305 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a40095f0 con 0x7f32b4105a30 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f32b27fc700 1 --2- 172.21.15.102:0/2383939305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b4105a30 0x7f32b419ca90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b4b0 tx=0x7f32a800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f329bfff700 1 -- 172.21.15.102:0/2383939305 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a8011800 con 0x7f32b4105a30 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f329bfff700 1 -- 172.21.15.102:0/2383939305 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32a8011e10 con 0x7f32b4105a30 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f32b9471700 1 -- 172.21.15.102:0/2383939305 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b41968b0 con 0x7f32b4105a30 2024-05-13T22:45:44.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f329bfff700 1 -- 172.21.15.102:0/2383939305 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a800f2e0 con 0x7f32b4105a30 2024-05-13T22:45:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.386+0000 7f32b9471700 1 -- 172.21.15.102:0/2383939305 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b4196e00 con 0x7f32b4105a30 2024-05-13T22:45:44.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.387+0000 7f32b9471700 1 -- 172.21.15.102:0/2383939305 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b4109040 con 0x7f32b4105a30 2024-05-13T22:45:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.388+0000 7f329bfff700 1 -- 172.21.15.102:0/2383939305 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f32a8010380 con 0x7f32b4105a30 2024-05-13T22:45:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.388+0000 7f329bfff700 1 --2- 172.21.15.102:0/2383939305 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f329c06c4e0 0x7f329c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.388+0000 7f329bfff700 1 -- 172.21.15.102:0/2383939305 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f32a808b430 con 0x7f32b4105a30 2024-05-13T22:45:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.388+0000 7f32b2ffd700 1 --2- 172.21.15.102:0/2383939305 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f329c06c4e0 0x7f329c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:44.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.389+0000 7f32b2ffd700 1 --2- 172.21.15.102:0/2383939305 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f329c06c4e0 0x7f329c06e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f32a4006010 tx=0x7f32a401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:44.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.393+0000 7f329bfff700 1 -- 172.21.15.102:0/2383939305 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f32a8059640 con 0x7f32b4105a30 2024-05-13T22:45:44.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.626+0000 7f32b9471700 1 -- 172.21.15.102:0/2383939305 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f32b404ea90 con 0x7f32b4105a30 2024-05-13T22:45:44.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.627+0000 7f329bfff700 1 -- 172.21.15.102:0/2383939305 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f32a80591d0 con 0x7f32b4105a30 2024-05-13T22:45:44.606 INFO:teuthology.orchestra.run.smithi102.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-13T22:45:44.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.629+0000 7f32b9471700 1 -- 172.21.15.102:0/2383939305 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f329c06c4e0 msgr2=0x7f329c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:44.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.629+0000 7f32b9471700 1 --2- 172.21.15.102:0/2383939305 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f329c06c4e0 0x7f329c06e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f32a4006010 tx=0x7f32a401a040 comp rx=0 tx=0).stop 2024-05-13T22:45:44.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.629+0000 7f32b9471700 1 -- 172.21.15.102:0/2383939305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b4105a30 msgr2=0x7f32b419ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:44.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.629+0000 7f32b9471700 1 --2- 172.21.15.102:0/2383939305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b4105a30 0x7f32b419ca90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b4b0 tx=0x7f32a800b7c0 comp rx=0 tx=0).stop 2024-05-13T22:45:44.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.629+0000 7f32b9471700 1 -- 172.21.15.102:0/2383939305 shutdown_connections 2024-05-13T22:45:44.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.629+0000 7f32b9471700 1 --2- 172.21.15.102:0/2383939305 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f329c06c4e0 0x7f329c06e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:44.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.629+0000 7f32b9471700 1 --2- 172.21.15.102:0/2383939305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32b40ffa30 0x7f32b419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:44.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.629+0000 7f32b9471700 1 --2- 172.21.15.102:0/2383939305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b4105a30 0x7f32b419ca90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:44.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.629+0000 7f32b9471700 1 -- 172.21.15.102:0/2383939305 >> 172.21.15.102:0/2383939305 conn(0x7f32b40fb520 msgr2=0x7f32b4068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:44.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.629+0000 7f32b9471700 1 -- 172.21.15.102:0/2383939305 shutdown_connections 2024-05-13T22:45:44.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:44.629+0000 7f32b9471700 1 -- 172.21.15.102:0/2383939305 wait complete. 2024-05-13T22:45:44.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:44 smithi102 ceph-mon[28599]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-13T22:45:44.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:44 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2383939305' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-13T22:45:45.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:44 smithi130 ceph-mon[36591]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-13T22:45:45.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:44 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2383939305' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-13T22:45:45.197 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-13T22:45:45.197 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-13T22:45:45.197 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi130.front.sepia.ceph.com 2024-05-13T22:45:45.197 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-13T22:45:45.197 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-13T22:45:45.197 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-13T22:45:45.197 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-13T22:45:45.197 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-13T22:45:45.197 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-13T22:45:45.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:45.197 DEBUG:teuthology.orchestra.run.smithi130:> ip addr 2024-05-13T22:45:45.217 INFO:teuthology.orchestra.run.smithi130.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-13T22:45:45.217 INFO:teuthology.orchestra.run.smithi130.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-13T22:45:45.217 INFO:teuthology.orchestra.run.smithi130.stdout: inet 127.0.0.1/8 scope host lo 2024-05-13T22:45:45.217 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft forever preferred_lft forever 2024-05-13T22:45:45.217 INFO:teuthology.orchestra.run.smithi130.stdout: inet6 ::1/128 scope host 2024-05-13T22:45:45.217 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft forever preferred_lft forever 2024-05-13T22:45:45.217 INFO:teuthology.orchestra.run.smithi130.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-13T22:45:45.217 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:d9:37:96 brd ff:ff:ff:ff:ff:ff 2024-05-13T22:45:45.217 INFO:teuthology.orchestra.run.smithi130.stdout: altname enp7s0f0 2024-05-13T22:45:45.217 INFO:teuthology.orchestra.run.smithi130.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-13T22:45:45.218 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:d9:37:97 brd ff:ff:ff:ff:ff:ff 2024-05-13T22:45:45.218 INFO:teuthology.orchestra.run.smithi130.stdout: altname enp7s0f1 2024-05-13T22:45:45.218 INFO:teuthology.orchestra.run.smithi130.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-13T22:45:45.218 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:88:7f:36 brd ff:ff:ff:ff:ff:ff 2024-05-13T22:45:45.218 INFO:teuthology.orchestra.run.smithi130.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-13T22:45:45.218 INFO:teuthology.orchestra.run.smithi130.stdout: link/ether 0c:c4:7a:88:7f:37 brd ff:ff:ff:ff:ff:ff 2024-05-13T22:45:45.218 INFO:teuthology.orchestra.run.smithi130.stdout: inet 172.21.15.130/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-13T22:45:45.218 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft 42255sec preferred_lft 42255sec 2024-05-13T22:45:45.218 INFO:teuthology.orchestra.run.smithi130.stdout: inet6 fe80::ec4:7aff:fe88:7f37/64 scope link 2024-05-13T22:45:45.218 INFO:teuthology.orchestra.run.smithi130.stdout: valid_lft forever preferred_lft forever 2024-05-13T22:45:45.218 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-13T22:45:45.219 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-13T22:45:45.219 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2024-05-13T22:45:45.219 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link add name ceph-brx type bridge 2024-05-13T22:45:45.219 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip addr flush dev ceph-brx 2024-05-13T22:45:45.219 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link set ceph-brx up 2024-05-13T22:45:45.219 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-13T22:45:45.219 DEBUG:teuthology.orchestra.run.smithi130:> ') 2024-05-13T22:45:45.313 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:45:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-13T22:45:45.483 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:45:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:45.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:45.490 DEBUG:teuthology.orchestra.run.smithi130:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-13T22:45:45.515 INFO:teuthology.orchestra.run.smithi130.stdout:1 2024-05-13T22:45:45.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:45.517 DEBUG:teuthology.orchestra.run.smithi130:> route 2024-05-13T22:45:45.583 INFO:teuthology.orchestra.run.smithi130.stdout:Kernel IP routing table 2024-05-13T22:45:45.583 INFO:teuthology.orchestra.run.smithi130.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-13T22:45:45.583 INFO:teuthology.orchestra.run.smithi130.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-13T22:45:45.583 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-13T22:45:45.583 INFO:teuthology.orchestra.run.smithi130.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-13T22:45:45.584 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-13T22:45:45.584 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2024-05-13T22:45:45.584 DEBUG:teuthology.orchestra.run.smithi130:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-13T22:45:45.584 DEBUG:teuthology.orchestra.run.smithi130:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-13T22:45:45.584 DEBUG:teuthology.orchestra.run.smithi130:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-13T22:45:45.584 DEBUG:teuthology.orchestra.run.smithi130:> ') 2024-05-13T22:45:45.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:45:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-13T22:45:45.866 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:45:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:45.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:45.873 DEBUG:teuthology.orchestra.run.smithi130:> ip netns list 2024-05-13T22:45:45.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:45.888 DEBUG:teuthology.orchestra.run.smithi130:> ip netns list-id 2024-05-13T22:45:45.943 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-13T22:45:45.944 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2024-05-13T22:45:45.944 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-13T22:45:45.944 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-13T22:45:45.944 DEBUG:teuthology.orchestra.run.smithi130:> ') 2024-05-13T22:45:46.033 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:45:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-13T22:45:46.055 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:45:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:46.062 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-13T22:45:46.062 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-13T22:45:46.062 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2024-05-13T22:45:46.063 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-13T22:45:46.063 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-13T22:45:46.063 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-13T22:45:46.063 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-13T22:45:46.063 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-13T22:45:46.063 DEBUG:teuthology.orchestra.run.smithi130:> ') 2024-05-13T22:45:46.112 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:45:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-13T22:45:46.187 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:45:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:46.193 DEBUG:teuthology.orchestra.run.smithi130:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-13T22:45:46.193 DEBUG:teuthology.orchestra.run.smithi130:> set -e 2024-05-13T22:45:46.194 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link set brx.0 up 2024-05-13T22:45:46.194 DEBUG:teuthology.orchestra.run.smithi130:> sudo ip link set dev brx.0 master ceph-brx 2024-05-13T22:45:46.194 DEBUG:teuthology.orchestra.run.smithi130:> ') 2024-05-13T22:45:46.241 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:45:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-13T22:45:46.263 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:45:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:46.270 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-13T22:45:46.270 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-13T22:45:46.270 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-13T22:45:46.295 INFO:teuthology.orchestra.run.smithi130.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-13T22:45:46.295 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-13T22:45:46.295 DEBUG:teuthology.orchestra.run.smithi130:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-13T22:45:46.350 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2024-05-13T22:45:46.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:46 smithi130 ceph-mon[36591]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-05-13T22:45:46.415 DEBUG:teuthology.orchestra.run.smithi130:> cat /proc/self/mounts | awk '{print $2}' 2024-05-13T22:45:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:46 smithi102 ceph-mon[28599]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-05-13T22:45:46.472 INFO:teuthology.orchestra.run.smithi130.stdout:/proc 2024-05-13T22:45:46.472 INFO:teuthology.orchestra.run.smithi130.stdout:/sys 2024-05-13T22:45:46.472 INFO:teuthology.orchestra.run.smithi130.stdout:/dev 2024-05-13T22:45:46.472 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/kernel/security 2024-05-13T22:45:46.472 INFO:teuthology.orchestra.run.smithi130.stdout:/dev/shm 2024-05-13T22:45:46.472 INFO:teuthology.orchestra.run.smithi130.stdout:/dev/pts 2024-05-13T22:45:46.472 INFO:teuthology.orchestra.run.smithi130.stdout:/run 2024-05-13T22:45:46.472 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/fs/cgroup 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/fs/pstore 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/fs/bpf 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/ 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/fs/selinux 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/proc/sys/fs/binfmt_misc 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/dev/hugepages 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/dev/mqueue 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/kernel/debug 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/kernel/tracing 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/fs/fuse/connections 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/kernel/config 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/run/credentials/systemd-sysctl.service 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-13T22:45:46.473 INFO:teuthology.orchestra.run.smithi130.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-13T22:45:46.474 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/nfs/rpc_pipefs 2024-05-13T22:45:46.474 INFO:teuthology.orchestra.run.smithi130.stdout:/run/user/1000 2024-05-13T22:45:46.474 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/ceph 2024-05-13T22:45:46.474 INFO:teuthology.orchestra.run.smithi130.stdout:/run/user/0 2024-05-13T22:45:46.474 INFO:teuthology.orchestra.run.smithi130.stdout:/proc/sys/fs/binfmt_misc 2024-05-13T22:45:46.474 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay 2024-05-13T22:45:46.474 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/254d504bd574e17d0bed8a5e4887e09bbc8834e5e4add6ff2685e53f272ca738/merged 2024-05-13T22:45:46.474 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/f7278946f0923cb745aa868636a445bf7d5c4d80e57ef1502bfc0a9384ff4dfb/merged 2024-05-13T22:45:46.474 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/528eb4ea457eebc357c38dc39894457ab4cd40ae92e51dbeb15bba701e9d4d54/merged 2024-05-13T22:45:46.474 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/fc51f26e27a1da9a3be099bd606c9d434ea2385441cdb39dc6b60ecadf2f5f00/merged 2024-05-13T22:45:46.475 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/0de00bcaad74d78a611fa930ddaefc2d8be803a1141c309706cb3e3b84843984/merged 2024-05-13T22:45:46.475 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/a8a1cc493b310d3aa34ec3bd8abd11b5c6f7d8efe8fef6d3f281f038a69eb65e/merged 2024-05-13T22:45:46.475 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/62bdcde585da1fff7626cd063273866b105789773177465e36d90591da6e7b18/merged 2024-05-13T22:45:46.475 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/553981fd21a5c369b98dca0ba73676378216dad09535bc72f435296130b5ccef/merged 2024-05-13T22:45:46.475 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/b4e62b52c9f9bcc8aeecb884467ac8b2a915300115a5a01194e04fef2b39e17f/merged 2024-05-13T22:45:46.475 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/29e0b209af0cbd49b128b40e807d285daac17ddec8ceaa3e3593b1d19d74e5f7/merged 2024-05-13T22:45:46.475 INFO:teuthology.orchestra.run.smithi130.stdout:/run/netns 2024-05-13T22:45:46.475 INFO:teuthology.orchestra.run.smithi130.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-13T22:45:46.475 INFO:teuthology.orchestra.run.smithi130.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-13T22:45:46.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:46.476 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2024-05-13T22:45:46.528 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-13T22:45:46.528 DEBUG:teuthology.orchestra.run.smithi130:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-13T22:45:46.571 DEBUG:teuthology.orchestra.run.smithi130:> sudo modprobe fuse 2024-05-13T22:45:46.595 DEBUG:teuthology.orchestra.run.smithi130:> cat /proc/self/mounts | awk '{print $2}' 2024-05-13T22:45:46.651 INFO:teuthology.orchestra.run.smithi130.stdout:/proc 2024-05-13T22:45:46.651 INFO:teuthology.orchestra.run.smithi130.stdout:/sys 2024-05-13T22:45:46.651 INFO:teuthology.orchestra.run.smithi130.stdout:/dev 2024-05-13T22:45:46.651 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/kernel/security 2024-05-13T22:45:46.651 INFO:teuthology.orchestra.run.smithi130.stdout:/dev/shm 2024-05-13T22:45:46.651 INFO:teuthology.orchestra.run.smithi130.stdout:/dev/pts 2024-05-13T22:45:46.651 INFO:teuthology.orchestra.run.smithi130.stdout:/run 2024-05-13T22:45:46.651 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/fs/cgroup 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/fs/pstore 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/fs/bpf 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/ 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/fs/selinux 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/proc/sys/fs/binfmt_misc 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/dev/hugepages 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/dev/mqueue 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/kernel/debug 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/kernel/tracing 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/fs/fuse/connections 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/sys/kernel/config 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/run/credentials/systemd-sysctl.service 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-13T22:45:46.652 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/nfs/rpc_pipefs 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/run/user/1000 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/ceph 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/run/user/0 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/proc/sys/fs/binfmt_misc 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/254d504bd574e17d0bed8a5e4887e09bbc8834e5e4add6ff2685e53f272ca738/merged 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/f7278946f0923cb745aa868636a445bf7d5c4d80e57ef1502bfc0a9384ff4dfb/merged 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/528eb4ea457eebc357c38dc39894457ab4cd40ae92e51dbeb15bba701e9d4d54/merged 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/fc51f26e27a1da9a3be099bd606c9d434ea2385441cdb39dc6b60ecadf2f5f00/merged 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/0de00bcaad74d78a611fa930ddaefc2d8be803a1141c309706cb3e3b84843984/merged 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/a8a1cc493b310d3aa34ec3bd8abd11b5c6f7d8efe8fef6d3f281f038a69eb65e/merged 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/62bdcde585da1fff7626cd063273866b105789773177465e36d90591da6e7b18/merged 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/553981fd21a5c369b98dca0ba73676378216dad09535bc72f435296130b5ccef/merged 2024-05-13T22:45:46.653 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/b4e62b52c9f9bcc8aeecb884467ac8b2a915300115a5a01194e04fef2b39e17f/merged 2024-05-13T22:45:46.654 INFO:teuthology.orchestra.run.smithi130.stdout:/var/lib/containers/storage/overlay/29e0b209af0cbd49b128b40e807d285daac17ddec8ceaa3e3593b1d19d74e5f7/merged 2024-05-13T22:45:46.654 INFO:teuthology.orchestra.run.smithi130.stdout:/run/netns 2024-05-13T22:45:46.654 INFO:teuthology.orchestra.run.smithi130.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-13T22:45:46.654 INFO:teuthology.orchestra.run.smithi130.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-13T22:45:46.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:46.654 DEBUG:teuthology.orchestra.run.smithi130:> ls /sys/fs/fuse/connections 2024-05-13T22:45:46.692 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi130.stderr:2024-05-13T22:45:46.713+0000 7ff49a83a480 -1 init, newargv = 0x5598b3764510 newargc=15 2024-05-13T22:45:46.692 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi130.stderr:ceph-fuse[62614]: starting ceph client 2024-05-13T22:45:46.696 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi130.stderr:ceph-fuse[62614]: starting fuse 2024-05-13T22:45:46.707 INFO:teuthology.orchestra.run.smithi130.stdout:128 2024-05-13T22:45:46.707 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [128] 2024-05-13T22:45:46.707 DEBUG:teuthology.orchestra.run.smithi130:> sudo stdin-killer -- python3 -c ' 2024-05-13T22:45:46.707 DEBUG:teuthology.orchestra.run.smithi130:> import glob 2024-05-13T22:45:46.707 DEBUG:teuthology.orchestra.run.smithi130:> import re 2024-05-13T22:45:46.707 DEBUG:teuthology.orchestra.run.smithi130:> import os 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> import subprocess 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> def _find_admin_socket(client_name): 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> files = glob.glob(asok_path) 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> # Given a non-glob path, it better be there 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> if "*" not in asok_path: 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> assert(len(files) == 1) 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> return files[0] 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> for f in files: 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-13T22:45:46.708 DEBUG:teuthology.orchestra.run.smithi130:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-13T22:45:46.709 DEBUG:teuthology.orchestra.run.smithi130:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-13T22:45:46.709 DEBUG:teuthology.orchestra.run.smithi130:> contents = proc_f.read() 2024-05-13T22:45:46.709 DEBUG:teuthology.orchestra.run.smithi130:> if mountpoint in contents: 2024-05-13T22:45:46.709 DEBUG:teuthology.orchestra.run.smithi130:> return f 2024-05-13T22:45:46.709 DEBUG:teuthology.orchestra.run.smithi130:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-13T22:45:46.709 DEBUG:teuthology.orchestra.run.smithi130:> 2024-05-13T22:45:46.709 DEBUG:teuthology.orchestra.run.smithi130:> print(_find_admin_socket("client.1")) 2024-05-13T22:45:46.709 DEBUG:teuthology.orchestra.run.smithi130:> ' 2024-05-13T22:45:46.825 INFO:teuthology.orchestra.run.smithi130.stdout:/var/run/ceph/ceph-client.1.62614.asok 2024-05-13T22:45:46.828 INFO:teuthology.orchestra.run.smithi130.stderr:2024-05-13T22:45:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-13T22:45:46.835 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.62614.asok 2024-05-13T22:45:46.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:46.835 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.62614.asok status 2024-05-13T22:45:46.969 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-05-13T22:45:46.969 INFO:teuthology.orchestra.run.smithi130.stdout: "metadata": { 2024-05-13T22:45:46.969 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-05-13T22:45:46.969 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-05-13T22:45:46.969 INFO:teuthology.orchestra.run.smithi130.stdout: "entity_id": "1", 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: "hostname": "smithi130", 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: "pid": "62614", 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: "root": "/" 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_count": 0, 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: "dentry_pinned_count": 0, 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: "id": 24321, 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: "inst": { 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: "name": { 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "client", 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: "num": 24321 2024-05-13T22:45:46.970 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "192.168.0.1:0", 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 2826286483 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": { 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: "type": "v1", 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: "addr": "192.168.0.1:0", 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: "nonce": 2826286483 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: "inst_str": "client.24321 192.168.0.1:0/2826286483", 2024-05-13T22:45:46.971 INFO:teuthology.orchestra.run.smithi130.stdout: "addr_str": "192.168.0.1:0/2826286483", 2024-05-13T22:45:46.972 INFO:teuthology.orchestra.run.smithi130.stdout: "inode_count": 1, 2024-05-13T22:45:46.972 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_epoch": 17, 2024-05-13T22:45:46.972 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch": 47, 2024-05-13T22:45:46.972 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_epoch_barrier": 0, 2024-05-13T22:45:46.972 INFO:teuthology.orchestra.run.smithi130.stdout: "blocklisted": false, 2024-05-13T22:45:46.972 INFO:teuthology.orchestra.run.smithi130.stdout: "fs_name": "cephfs" 2024-05-13T22:45:46.972 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-05-13T22:45:46.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:46.979 DEBUG:teuthology.orchestra.run.smithi102:> stat --file-system '--printf=%T 2024-05-13T22:45:46.979 DEBUG:teuthology.orchestra.run.smithi102:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-13T22:45:46.999 INFO:teuthology.orchestra.run.smithi102.stdout:fuseblk 2024-05-13T22:45:47.000 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-13T22:45:47.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:47.000 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-13T22:45:47.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:47.074 DEBUG:teuthology.orchestra.run.smithi130:> stat --file-system '--printf=%T 2024-05-13T22:45:47.074 DEBUG:teuthology.orchestra.run.smithi130:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-13T22:45:47.090 INFO:teuthology.orchestra.run.smithi130.stdout:fuseblk 2024-05-13T22:45:47.090 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-13T22:45:47.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T22:45:47.090 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-13T22:45:47.159 INFO:teuthology.run_tasks:Running task print... 2024-05-13T22:45:47.169 INFO:teuthology.task.print:**** done client 2024-05-13T22:45:47.169 INFO:teuthology.run_tasks:Running task parallel... 2024-05-13T22:45:47.179 INFO:teuthology.task.parallel:starting parallel... 2024-05-13T22:45:47.179 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-13T22:45:47.180 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-13T22:45:47.180 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:45:47.180 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-13T22:45:47.181 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-13T22:45:47.181 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-05-13T22:45:47.185 INFO:tasks.workunit:Pulling workunits from ref c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:45:47.185 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-13T22:45:47.185 INFO:tasks.workunit:timeout=3h 2024-05-13T22:45:47.185 INFO:tasks.workunit:cleanup=True 2024-05-13T22:45:47.186 DEBUG:teuthology.orchestra.run.smithi102:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-13T22:45:47.205 INFO:teuthology.orchestra.run.smithi102.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-13T22:45:47.205 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-13T22:45:47.205 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 98h/152d Inode: 1 Links: 2 2024-05-13T22:45:47.205 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-13T22:45:47.205 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-13T22:45:47.205 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-13T22:45:47.205 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-05-13 22:45:15.111195265 +0000 2024-05-13T22:45:47.205 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-05-13 22:45:47.093835108 +0000 2024-05-13T22:45:47.205 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: - 2024-05-13T22:45:47.206 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-13T22:45:47.206 DEBUG:teuthology.orchestra.run.smithi102:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-13T22:45:47.275 DEBUG:teuthology.orchestra.run.smithi130:> stat -- /home/ubuntu/cephtest/mnt.1 2024-05-13T22:45:47.292 INFO:teuthology.orchestra.run.smithi130.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-05-13T22:45:47.292 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-13T22:45:47.292 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 80h/128d Inode: 1 Links: 3 2024-05-13T22:45:47.292 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-13T22:45:47.292 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-13T22:45:47.292 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-13T22:45:47.293 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-05-13 22:45:47.292485038 +0000 2024-05-13T22:45:47.293 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-05-13 22:45:47.292485038 +0000 2024-05-13T22:45:47.293 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: - 2024-05-13T22:45:47.293 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-05-13T22:45:47.293 DEBUG:teuthology.orchestra.run.smithi130:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-05-13T22:45:47.362 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:45:47.362 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:45:47.378 INFO:tasks.workunit.client.0.smithi102.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-13T22:45:47.402 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:47.418 INFO:tasks.workunit.client.1.smithi130.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-05-13T22:45:48.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.974+0000 7f29035cf700 1 -- 172.21.15.102:0/1812299841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc103f70 msgr2=0x7f28fc106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:48.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.974+0000 7f29035cf700 1 --2- 172.21.15.102:0/1812299841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc103f70 0x7f28fc106360 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009910 tx=0x7f28ec009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:48.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.974+0000 7f29035cf700 1 -- 172.21.15.102:0/1812299841 shutdown_connections 2024-05-13T22:45:48.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.974+0000 7f29035cf700 1 --2- 172.21.15.102:0/1812299841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc103f70 0x7f28fc106360 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.974+0000 7f29035cf700 1 --2- 172.21.15.102:0/1812299841 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28fc101640 0x7f28fc103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.974+0000 7f29035cf700 1 -- 172.21.15.102:0/1812299841 >> 172.21.15.102:0/1812299841 conn(0x7f28fc0fb050 msgr2=0x7f28fc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.974+0000 7f29035cf700 1 -- 172.21.15.102:0/1812299841 shutdown_connections 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.974+0000 7f29035cf700 1 -- 172.21.15.102:0/1812299841 wait complete. 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f29035cf700 1 Processor -- start 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f29035cf700 1 -- start start 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f29035cf700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28fc101640 0x7f28fc196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f29035cf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc103f70 0x7f28fc196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f29035cf700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc196e80 con 0x7f28fc103f70 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f29035cf700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc196fc0 con 0x7f28fc101640 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f2900b6a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc103f70 0x7f28fc196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f2900b6a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc103f70 0x7f28fc196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42720/0 (socket says 172.21.15.102:42720) 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f2900b6a700 1 -- 172.21.15.102:0/4211546028 learned_addr learned my addr 172.21.15.102:0/4211546028 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:48.955 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f290136b700 1 --2- 172.21.15.102:0/4211546028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28fc101640 0x7f28fc196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:48.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f2900b6a700 1 -- 172.21.15.102:0/4211546028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28fc101640 msgr2=0x7f28fc196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:48.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f2900b6a700 1 --2- 172.21.15.102:0/4211546028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28fc101640 0x7f28fc196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:48.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.975+0000 7f2900b6a700 1 -- 172.21.15.102:0/4211546028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec0095f0 con 0x7f28fc103f70 2024-05-13T22:45:48.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.976+0000 7f2900b6a700 1 --2- 172.21.15.102:0/4211546028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc103f70 0x7f28fc196860 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f28ec00b5c0 tx=0x7f28ec00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:48.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.976+0000 7f28f27fc700 1 -- 172.21.15.102:0/4211546028 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ec01d070 con 0x7f28fc103f70 2024-05-13T22:45:48.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.976+0000 7f29035cf700 1 -- 172.21.15.102:0/4211546028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc19ba10 con 0x7f28fc103f70 2024-05-13T22:45:48.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.976+0000 7f28f27fc700 1 -- 172.21.15.102:0/4211546028 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f28ec022470 con 0x7f28fc103f70 2024-05-13T22:45:48.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.976+0000 7f28f27fc700 1 -- 172.21.15.102:0/4211546028 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ec00f690 con 0x7f28fc103f70 2024-05-13T22:45:48.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.976+0000 7f29035cf700 1 -- 172.21.15.102:0/4211546028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc19bf00 con 0x7f28fc103f70 2024-05-13T22:45:48.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.977+0000 7f29035cf700 1 -- 172.21.15.102:0/4211546028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc190520 con 0x7f28fc103f70 2024-05-13T22:45:48.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.977+0000 7f28f27fc700 1 -- 172.21.15.102:0/4211546028 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f28ec0225e0 con 0x7f28fc103f70 2024-05-13T22:45:48.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.978+0000 7f28f27fc700 1 --2- 172.21.15.102:0/4211546028 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28e806c4e0 0x7f28e806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:48.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.978+0000 7f28f27fc700 1 -- 172.21.15.102:0/4211546028 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f28ec08d020 con 0x7f28fc103f70 2024-05-13T22:45:48.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.978+0000 7f290136b700 1 --2- 172.21.15.102:0/4211546028 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28e806c4e0 0x7f28e806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:48.957 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.978+0000 7f290136b700 1 --2- 172.21.15.102:0/4211546028 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28e806c4e0 0x7f28e806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28f8005950 tx=0x7f28f800a300 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:48.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:48.982+0000 7f28f27fc700 1 -- 172.21.15.102:0/4211546028 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f28ec05b230 con 0x7f28fc103f70 2024-05-13T22:45:49.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:48 smithi130 ceph-mon[36591]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-05-13T22:45:49.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.175+0000 7f29035cf700 1 -- 172.21.15.102:0/4211546028 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f28fc02cc70 con 0x7f28fc103f70 2024-05-13T22:45:49.154 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:48 smithi102 ceph-mon[28599]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-05-13T22:45:49.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.176+0000 7f28f27fc700 1 -- 172.21.15.102:0/4211546028 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f28ec05adc0 con 0x7f28fc103f70 2024-05-13T22:45:49.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.179+0000 7f29035cf700 1 -- 172.21.15.102:0/4211546028 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28e806c4e0 msgr2=0x7f28e806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:49.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.179+0000 7f29035cf700 1 --2- 172.21.15.102:0/4211546028 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28e806c4e0 0x7f28e806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28f8005950 tx=0x7f28f800a300 comp rx=0 tx=0).stop 2024-05-13T22:45:49.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.179+0000 7f29035cf700 1 -- 172.21.15.102:0/4211546028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc103f70 msgr2=0x7f28fc196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:49.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.179+0000 7f29035cf700 1 --2- 172.21.15.102:0/4211546028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc103f70 0x7f28fc196860 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f28ec00b5c0 tx=0x7f28ec00bd70 comp rx=0 tx=0).stop 2024-05-13T22:45:49.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.179+0000 7f29035cf700 1 -- 172.21.15.102:0/4211546028 shutdown_connections 2024-05-13T22:45:49.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.179+0000 7f29035cf700 1 --2- 172.21.15.102:0/4211546028 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28e806c4e0 0x7f28e806e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:49.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.179+0000 7f29035cf700 1 --2- 172.21.15.102:0/4211546028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28fc101640 0x7f28fc196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:49.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.179+0000 7f29035cf700 1 --2- 172.21.15.102:0/4211546028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28fc103f70 0x7f28fc196860 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:49.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.179+0000 7f29035cf700 1 -- 172.21.15.102:0/4211546028 >> 172.21.15.102:0/4211546028 conn(0x7f28fc0fb050 msgr2=0x7f28fc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:49.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.179+0000 7f29035cf700 1 -- 172.21.15.102:0/4211546028 shutdown_connections 2024-05-13T22:45:49.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:49.179+0000 7f29035cf700 1 -- 172.21.15.102:0/4211546028 wait complete. 2024-05-13T22:45:49.754 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-13T22:45:50.010 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:51.143 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:50 smithi102 ceph-mon[28599]: pgmap v198: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 3 op/s 2024-05-13T22:45:51.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:50 smithi130 ceph-mon[36591]: pgmap v198: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 3 op/s 2024-05-13T22:45:51.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.546+0000 7f11e0810700 1 -- 172.21.15.102:0/4086503886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11dc100cb0 msgr2=0x7f11dc101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:51.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.546+0000 7f11e0810700 1 --2- 172.21.15.102:0/4086503886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11dc100cb0 0x7f11dc101110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009910 tx=0x7f11c4009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:51.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.546+0000 7f11e0810700 1 -- 172.21.15.102:0/4086503886 shutdown_connections 2024-05-13T22:45:51.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.546+0000 7f11e0810700 1 --2- 172.21.15.102:0/4086503886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11dc100cb0 0x7f11dc101110 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:51.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.546+0000 7f11e0810700 1 --2- 172.21.15.102:0/4086503886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11dc0ffab0 0x7f11dc0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:51.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.546+0000 7f11e0810700 1 -- 172.21.15.102:0/4086503886 >> 172.21.15.102:0/4086503886 conn(0x7f11dc0fb050 msgr2=0x7f11dc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:51.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.546+0000 7f11e0810700 1 -- 172.21.15.102:0/4086503886 shutdown_connections 2024-05-13T22:45:51.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.546+0000 7f11e0810700 1 -- 172.21.15.102:0/4086503886 wait complete. 2024-05-13T22:45:51.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11e0810700 1 Processor -- start 2024-05-13T22:45:51.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11e0810700 1 -- start start 2024-05-13T22:45:51.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11e0810700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11dc0ffab0 0x7f11dc195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:51.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11e0810700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11dc100cb0 0x7f11dc1957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:51.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11e0810700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc195dc0 con 0x7f11dc0ffab0 2024-05-13T22:45:51.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11e0810700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc195f00 con 0x7f11dc100cb0 2024-05-13T22:45:51.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11da59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11dc0ffab0 0x7f11dc195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11da59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11dc0ffab0 0x7f11dc195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42734/0 (socket says 172.21.15.102:42734) 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11da59c700 1 -- 172.21.15.102:0/3027659991 learned_addr learned my addr 172.21.15.102:0/3027659991 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11d1d9b700 1 --2- 172.21.15.102:0/3027659991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11dc100cb0 0x7f11dc1957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11da59c700 1 -- 172.21.15.102:0/3027659991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11dc100cb0 msgr2=0x7f11dc1957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11da59c700 1 --2- 172.21.15.102:0/3027659991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11dc100cb0 0x7f11dc1957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.547+0000 7f11da59c700 1 -- 172.21.15.102:0/3027659991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40095f0 con 0x7f11dc0ffab0 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.548+0000 7f11da59c700 1 --2- 172.21.15.102:0/3027659991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11dc0ffab0 0x7f11dc195260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00d900 tx=0x7f11cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.548+0000 7f11d3fff700 1 -- 172.21.15.102:0/3027659991 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11cc0041f0 con 0x7f11dc0ffab0 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.548+0000 7f11d3fff700 1 -- 172.21.15.102:0/3027659991 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f11cc004350 con 0x7f11dc0ffab0 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.548+0000 7f11e0810700 1 -- 172.21.15.102:0/3027659991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11dc19a9b0 con 0x7f11dc0ffab0 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.548+0000 7f11d3fff700 1 -- 172.21.15.102:0/3027659991 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11cc003d20 con 0x7f11dc0ffab0 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.548+0000 7f11e0810700 1 -- 172.21.15.102:0/3027659991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11dc19af00 con 0x7f11dc0ffab0 2024-05-13T22:45:51.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.549+0000 7f11e0810700 1 -- 172.21.15.102:0/3027659991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11dc108610 con 0x7f11dc0ffab0 2024-05-13T22:45:51.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.549+0000 7f11d3fff700 1 -- 172.21.15.102:0/3027659991 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f11cc009730 con 0x7f11dc0ffab0 2024-05-13T22:45:51.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.550+0000 7f11d3fff700 1 --2- 172.21.15.102:0/3027659991 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f11c806c4e0 0x7f11c806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:51.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.550+0000 7f11d1d9b700 1 --2- 172.21.15.102:0/3027659991 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f11c806c4e0 0x7f11c806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:51.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.550+0000 7f11d3fff700 1 -- 172.21.15.102:0/3027659991 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f11cc08b680 con 0x7f11dc0ffab0 2024-05-13T22:45:51.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.550+0000 7f11d1d9b700 1 --2- 172.21.15.102:0/3027659991 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f11c806c4e0 0x7f11c806e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f11c4006010 tx=0x7f11c400b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:51.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.554+0000 7f11d3fff700 1 -- 172.21.15.102:0/3027659991 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f11cc059910 con 0x7f11dc0ffab0 2024-05-13T22:45:51.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.747+0000 7f11e0810700 1 -- 172.21.15.102:0/3027659991 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f11dc19b5f0 con 0x7f11dc0ffab0 2024-05-13T22:45:51.725 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.747+0000 7f11d3fff700 1 -- 172.21.15.102:0/3027659991 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f11cc003e80 con 0x7f11dc0ffab0 2024-05-13T22:45:51.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.749+0000 7f11e0810700 1 -- 172.21.15.102:0/3027659991 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f11c806c4e0 msgr2=0x7f11c806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:51.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.750+0000 7f11e0810700 1 --2- 172.21.15.102:0/3027659991 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f11c806c4e0 0x7f11c806e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f11c4006010 tx=0x7f11c400b540 comp rx=0 tx=0).stop 2024-05-13T22:45:51.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.750+0000 7f11e0810700 1 -- 172.21.15.102:0/3027659991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11dc0ffab0 msgr2=0x7f11dc195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:51.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.750+0000 7f11e0810700 1 --2- 172.21.15.102:0/3027659991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11dc0ffab0 0x7f11dc195260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00d900 tx=0x7f11cc00dc10 comp rx=0 tx=0).stop 2024-05-13T22:45:51.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.750+0000 7f11e0810700 1 -- 172.21.15.102:0/3027659991 shutdown_connections 2024-05-13T22:45:51.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.750+0000 7f11e0810700 1 --2- 172.21.15.102:0/3027659991 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f11c806c4e0 0x7f11c806e9a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:51.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.750+0000 7f11e0810700 1 --2- 172.21.15.102:0/3027659991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11dc0ffab0 0x7f11dc195260 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:51.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.750+0000 7f11e0810700 1 --2- 172.21.15.102:0/3027659991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f11dc100cb0 0x7f11dc1957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:51.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.750+0000 7f11e0810700 1 -- 172.21.15.102:0/3027659991 >> 172.21.15.102:0/3027659991 conn(0x7f11dc0fb050 msgr2=0x7f11dc103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:51.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.750+0000 7f11e0810700 1 -- 172.21.15.102:0/3027659991 shutdown_connections 2024-05-13T22:45:51.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:51.750+0000 7f11e0810700 1 -- 172.21.15.102:0/3027659991 wait complete. 2024-05-13T22:45:52.284 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-13T22:45:52.489 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:52.834 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:52 smithi102 ceph-mon[28599]: pgmap v199: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 682 B/s wr, 2 op/s 2024-05-13T22:45:53.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:52 smithi130 ceph-mon[36591]: pgmap v199: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 682 B/s wr, 2 op/s 2024-05-13T22:45:53.824 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:53 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:45:54.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.063+0000 7faa64241700 1 -- 172.21.15.102:0/2423596017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa5c100bc0 msgr2=0x7faa5c100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:54.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.063+0000 7faa64241700 1 --2- 172.21.15.102:0/2423596017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa5c100bc0 0x7faa5c100fe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faa58009910 tx=0x7faa58009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:54.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.063+0000 7faa64241700 1 -- 172.21.15.102:0/2423596017 shutdown_connections 2024-05-13T22:45:54.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.063+0000 7faa64241700 1 --2- 172.21.15.102:0/2423596017 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa5c101dc0 0x7faa5c102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:54.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.063+0000 7faa64241700 1 --2- 172.21.15.102:0/2423596017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa5c100bc0 0x7faa5c100fe0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:54.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.063+0000 7faa64241700 1 -- 172.21.15.102:0/2423596017 >> 172.21.15.102:0/2423596017 conn(0x7faa5c0fc140 msgr2=0x7faa5c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:54.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.063+0000 7faa64241700 1 -- 172.21.15.102:0/2423596017 shutdown_connections 2024-05-13T22:45:54.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.063+0000 7faa64241700 1 -- 172.21.15.102:0/2423596017 wait complete. 2024-05-13T22:45:54.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa64241700 1 Processor -- start 2024-05-13T22:45:54.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa64241700 1 -- start start 2024-05-13T22:45:54.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa64241700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa5c100bc0 0x7faa5c1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:54.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa64241700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa5c101dc0 0x7faa5c196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:54.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa64241700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c196f50 con 0x7faa5c101dc0 2024-05-13T22:45:54.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa64241700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c197090 con 0x7faa5c100bc0 2024-05-13T22:45:54.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa617dc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa5c101dc0 0x7faa5c196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:54.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa617dc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa5c101dc0 0x7faa5c196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42756/0 (socket says 172.21.15.102:42756) 2024-05-13T22:45:54.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa617dc700 1 -- 172.21.15.102:0/2209907267 learned_addr learned my addr 172.21.15.102:0/2209907267 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:54.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa61fdd700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa5c100bc0 0x7faa5c1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:54.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa617dc700 1 -- 172.21.15.102:0/2209907267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa5c100bc0 msgr2=0x7faa5c1963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:54.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa617dc700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa5c100bc0 0x7faa5c1963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:54.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa617dc700 1 -- 172.21.15.102:0/2209907267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa580095f0 con 0x7faa5c101dc0 2024-05-13T22:45:54.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa61fdd700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa5c100bc0 0x7faa5c1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:45:54.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.064+0000 7faa617dc700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa5c101dc0 0x7faa5c196930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00b4b0 tx=0x7faa4c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:54.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.065+0000 7faa52ffd700 1 -- 172.21.15.102:0/2209907267 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa4c011800 con 0x7faa5c101dc0 2024-05-13T22:45:54.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.065+0000 7faa52ffd700 1 -- 172.21.15.102:0/2209907267 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faa4c011e10 con 0x7faa5c101dc0 2024-05-13T22:45:54.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.065+0000 7faa64241700 1 -- 172.21.15.102:0/2209907267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c19bb40 con 0x7faa5c101dc0 2024-05-13T22:45:54.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.065+0000 7faa52ffd700 1 -- 172.21.15.102:0/2209907267 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa4c00f2e0 con 0x7faa5c101dc0 2024-05-13T22:45:54.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.065+0000 7faa64241700 1 -- 172.21.15.102:0/2209907267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c19c110 con 0x7faa5c101dc0 2024-05-13T22:45:54.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.066+0000 7faa64241700 1 -- 172.21.15.102:0/2209907267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5c1097a0 con 0x7faa5c101dc0 2024-05-13T22:45:54.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.066+0000 7faa52ffd700 1 -- 172.21.15.102:0/2209907267 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7faa4c010380 con 0x7faa5c101dc0 2024-05-13T22:45:54.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.066+0000 7faa52ffd700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa4806c4e0 0x7faa4806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:54.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.067+0000 7faa52ffd700 1 -- 172.21.15.102:0/2209907267 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7faa4c08b4d0 con 0x7faa5c101dc0 2024-05-13T22:45:54.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.067+0000 7faa61fdd700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa4806c4e0 0x7faa4806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:54.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.067+0000 7faa61fdd700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa4806c4e0 0x7faa4806e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faa5800b5c0 tx=0x7faa58005e20 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:54.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.071+0000 7faa52ffd700 1 -- 172.21.15.102:0/2209907267 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faa4c0596e0 con 0x7faa5c101dc0 2024-05-13T22:45:54.052 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:53 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:45:54.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.265+0000 7faa64241700 1 -- 172.21.15.102:0/2209907267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7faa5c19c950 con 0x7faa5c101dc0 2024-05-13T22:45:54.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.266+0000 7faa52ffd700 1 -- 172.21.15.102:0/2209907267 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7faa4c059270 con 0x7faa5c101dc0 2024-05-13T22:45:54.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.268+0000 7faa64241700 1 -- 172.21.15.102:0/2209907267 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa4806c4e0 msgr2=0x7faa4806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:54.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.268+0000 7faa64241700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa4806c4e0 0x7faa4806e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faa5800b5c0 tx=0x7faa58005e20 comp rx=0 tx=0).stop 2024-05-13T22:45:54.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.268+0000 7faa64241700 1 -- 172.21.15.102:0/2209907267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa5c101dc0 msgr2=0x7faa5c196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:54.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.268+0000 7faa64241700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa5c101dc0 0x7faa5c196930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00b4b0 tx=0x7faa4c00b870 comp rx=0 tx=0).stop 2024-05-13T22:45:54.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.268+0000 7faa64241700 1 -- 172.21.15.102:0/2209907267 shutdown_connections 2024-05-13T22:45:54.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.268+0000 7faa64241700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7faa4806c4e0 0x7faa4806e9a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:54.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.268+0000 7faa64241700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faa5c100bc0 0x7faa5c1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:54.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.268+0000 7faa64241700 1 --2- 172.21.15.102:0/2209907267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa5c101dc0 0x7faa5c196930 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:54.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.268+0000 7faa64241700 1 -- 172.21.15.102:0/2209907267 >> 172.21.15.102:0/2209907267 conn(0x7faa5c0fc140 msgr2=0x7faa5c104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:54.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.268+0000 7faa64241700 1 -- 172.21.15.102:0/2209907267 shutdown_connections 2024-05-13T22:45:54.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:54.268+0000 7faa64241700 1 -- 172.21.15.102:0/2209907267 wait complete. 2024-05-13T22:45:54.759 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-05-13T22:45:54.965 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:45:55.086 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:54 smithi102 ceph-mon[28599]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-13T22:45:55.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:54 smithi130 ceph-mon[36591]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-13T22:45:56.306 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:56 smithi102 ceph-mon[28599]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-13T22:45:56.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:56 smithi130 ceph-mon[36591]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-13T22:45:56.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.435+0000 7f9141a7c700 1 -- 172.21.15.102:0/1135696386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f913c100cd0 msgr2=0x7f913c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:56.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.435+0000 7f9141a7c700 1 --2- 172.21.15.102:0/1135696386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f913c100cd0 0x7f913c101130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9130009910 tx=0x7f9130009c20 comp rx=0 tx=0).stop 2024-05-13T22:45:56.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.435+0000 7f9141a7c700 1 -- 172.21.15.102:0/1135696386 shutdown_connections 2024-05-13T22:45:56.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.435+0000 7f9141a7c700 1 --2- 172.21.15.102:0/1135696386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f913c100cd0 0x7f913c101130 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:56.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.435+0000 7f9141a7c700 1 --2- 172.21.15.102:0/1135696386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f913c0ffad0 0x7f913c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:56.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.435+0000 7f9141a7c700 1 -- 172.21.15.102:0/1135696386 >> 172.21.15.102:0/1135696386 conn(0x7f913c0fb050 msgr2=0x7f913c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:45:56.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.435+0000 7f9141a7c700 1 -- 172.21.15.102:0/1135696386 shutdown_connections 2024-05-13T22:45:56.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.435+0000 7f9141a7c700 1 -- 172.21.15.102:0/1135696386 wait complete. 2024-05-13T22:45:56.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.435+0000 7f9141a7c700 1 Processor -- start 2024-05-13T22:45:56.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f9141a7c700 1 -- start start 2024-05-13T22:45:56.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f9141a7c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f913c0ffad0 0x7f913c069910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:56.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f9141a7c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f913c100cd0 0x7f913c19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:56.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f9141a7c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c19b730 con 0x7f913c100cd0 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f9141a7c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c19b8a0 con 0x7f913c0ffad0 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f913affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f913c100cd0 0x7f913c19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f913affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f913c100cd0 0x7f913c19b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42768/0 (socket says 172.21.15.102:42768) 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f913affd700 1 -- 172.21.15.102:0/1844295314 learned_addr learned my addr 172.21.15.102:0/1844295314 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f913affd700 1 -- 172.21.15.102:0/1844295314 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f913c0ffad0 msgr2=0x7f913c069910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f913b7fe700 1 --2- 172.21.15.102:0/1844295314 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f913c0ffad0 0x7f913c069910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f913affd700 1 --2- 172.21.15.102:0/1844295314 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f913c0ffad0 0x7f913c069910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f913affd700 1 -- 172.21.15.102:0/1844295314 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91300095f0 con 0x7f913c100cd0 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f913affd700 1 --2- 172.21.15.102:0/1844295314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f913c100cd0 0x7f913c19b160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9130006010 tx=0x7f913000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.436+0000 7f9138ff9700 1 -- 172.21.15.102:0/1844295314 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f913001d070 con 0x7f913c100cd0 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.437+0000 7f9141a7c700 1 -- 172.21.15.102:0/1844295314 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c19bb20 con 0x7f913c100cd0 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.437+0000 7f9138ff9700 1 -- 172.21.15.102:0/1844295314 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f913000f460 con 0x7f913c100cd0 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.437+0000 7f9138ff9700 1 -- 172.21.15.102:0/1844295314 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9130021640 con 0x7f913c100cd0 2024-05-13T22:45:56.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.437+0000 7f9141a7c700 1 -- 172.21.15.102:0/1844295314 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c19c090 con 0x7f913c100cd0 2024-05-13T22:45:56.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.437+0000 7f9141a7c700 1 -- 172.21.15.102:0/1844295314 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c1086d0 con 0x7f913c100cd0 2024-05-13T22:45:56.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.438+0000 7f9138ff9700 1 -- 172.21.15.102:0/1844295314 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f913000fab0 con 0x7f913c100cd0 2024-05-13T22:45:56.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.438+0000 7f9138ff9700 1 --2- 172.21.15.102:0/1844295314 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f912406c4e0 0x7f912406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:45:56.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.439+0000 7f9138ff9700 1 -- 172.21.15.102:0/1844295314 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f913008cd20 con 0x7f913c100cd0 2024-05-13T22:45:56.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.439+0000 7f913b7fe700 1 --2- 172.21.15.102:0/1844295314 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f912406c4e0 0x7f912406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:45:56.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.439+0000 7f913b7fe700 1 --2- 172.21.15.102:0/1844295314 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f912406c4e0 0x7f912406e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f913c100b30 tx=0x7f912c0072f0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:45:56.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.443+0000 7f9138ff9700 1 -- 172.21.15.102:0/1844295314 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f913005af30 con 0x7f913c100cd0 2024-05-13T22:45:56.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:45:56.648+0000 7f9141a7c700 1 -- 172.21.15.102:0/1844295314 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f913c1056d0 con 0x7f912406c4e0 2024-05-13T22:45:57.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:57 smithi130 ceph-mon[36591]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:57.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:57 smithi102 ceph-mon[28599]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:45:58.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:45:58 smithi130 ceph-mon[36591]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-13T22:45:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:45:58 smithi102 ceph-mon[28599]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-13T22:46:01.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:00 smithi130 ceph-mon[36591]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-05-13T22:46:01.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:00 smithi102 ceph-mon[28599]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-05-13T22:46:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:02 smithi102 ceph-mon[28599]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-05-13T22:46:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:02 smithi130 ceph-mon[36591]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-05-13T22:46:05.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:04 smithi130 ceph-mon[36591]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-13T22:46:05.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:04 smithi102 ceph-mon[28599]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-13T22:46:06.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:06 smithi130 ceph-mon[36591]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-05-13T22:46:06.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:06 smithi102 ceph-mon[28599]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-05-13T22:46:09.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:08 smithi130 ceph-mon[36591]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-05-13T22:46:09.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:08 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:46:09.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:08 smithi102 ceph-mon[28599]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-05-13T22:46:09.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:08 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:46:11.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:10 smithi130 ceph-mon[36591]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 341 B/s wr, 2 op/s 2024-05-13T22:46:11.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:10 smithi102 ceph-mon[28599]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 341 B/s wr, 2 op/s 2024-05-13T22:46:13.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:12 smithi130 ceph-mon[36591]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:13.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:12 smithi102 ceph-mon[28599]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:15.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:14 smithi130 ceph-mon[36591]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:15.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:14 smithi102 ceph-mon[28599]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:16.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:16 smithi130 ceph-mon[36591]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:16.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:16 smithi102 ceph-mon[28599]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:19.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:18 smithi130 ceph-mon[36591]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:19.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:18 smithi102 ceph-mon[28599]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:21.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:20 smithi130 ceph-mon[36591]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:21.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:20 smithi102 ceph-mon[28599]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:23.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:22 smithi130 ceph-mon[36591]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:23.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:22 smithi102 ceph-mon[28599]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:24.051 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:23 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:46:24.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:23 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:46:25.066 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:24 smithi130 ceph-mon[36591]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:25.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:24 smithi102 ceph-mon[28599]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:26.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:26 smithi130 ceph-mon[36591]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:26.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:26 smithi102 ceph-mon[28599]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:29.021 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:28 smithi102 ceph-mon[28599]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:29.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.108+0000 7f9138ff9700 1 -- 172.21.15.102:0/1844295314 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f913c1056d0 con 0x7f912406c4e0 2024-05-13T22:46:29.088 INFO:teuthology.orchestra.run.smithi102.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:46:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.110+0000 7f9141a7c700 1 -- 172.21.15.102:0/1844295314 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f912406c4e0 msgr2=0x7f912406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.110+0000 7f9141a7c700 1 --2- 172.21.15.102:0/1844295314 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f912406c4e0 0x7f912406e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f913c100b30 tx=0x7f912c0072f0 comp rx=0 tx=0).stop 2024-05-13T22:46:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.110+0000 7f9141a7c700 1 -- 172.21.15.102:0/1844295314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f913c100cd0 msgr2=0x7f913c19b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.110+0000 7f9141a7c700 1 --2- 172.21.15.102:0/1844295314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f913c100cd0 0x7f913c19b160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9130006010 tx=0x7f913000be20 comp rx=0 tx=0).stop 2024-05-13T22:46:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.110+0000 7f9141a7c700 1 -- 172.21.15.102:0/1844295314 shutdown_connections 2024-05-13T22:46:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.110+0000 7f9141a7c700 1 --2- 172.21.15.102:0/1844295314 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f912406c4e0 0x7f912406e9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.110+0000 7f9141a7c700 1 --2- 172.21.15.102:0/1844295314 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f913c0ffad0 0x7f913c069910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:29.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.110+0000 7f9141a7c700 1 --2- 172.21.15.102:0/1844295314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f913c100cd0 0x7f913c19b160 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:29.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.111+0000 7f9141a7c700 1 -- 172.21.15.102:0/1844295314 >> 172.21.15.102:0/1844295314 conn(0x7f913c0fb050 msgr2=0x7f913c103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:46:29.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.111+0000 7f9141a7c700 1 -- 172.21.15.102:0/1844295314 shutdown_connections 2024-05-13T22:46:29.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:29.111+0000 7f9141a7c700 1 -- 172.21.15.102:0/1844295314 wait complete. 2024-05-13T22:46:29.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:28 smithi130 ceph-mon[36591]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:29.960 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-05-13T22:46:30.265 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:30 smithi102 ceph-mon[28599]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:46:30.265 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:46:30.265 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:46:30.265 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:46:30.265 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:46:30.266 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:30 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:46:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:30 smithi130 ceph-mon[36591]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:46:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:30 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:46:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:30 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:46:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:30 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:46:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:30 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:46:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:30 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:46:30.596 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:46:31.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:31 smithi102 ceph-mon[28599]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:31.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:31 smithi102 ceph-mon[28599]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:46:31.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:31 smithi130 ceph-mon[36591]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:31.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:31 smithi130 ceph-mon[36591]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:46:33.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.023+0000 7f64fb813700 1 -- 172.21.15.102:0/3521259146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f4100cb0 msgr2=0x7f64f4101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.023+0000 7f64fb813700 1 --2- 172.21.15.102:0/3521259146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f4100cb0 0x7f64f4101110 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009910 tx=0x7f64e4009c20 comp rx=0 tx=0).stop 2024-05-13T22:46:33.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.023+0000 7f64fb813700 1 -- 172.21.15.102:0/3521259146 shutdown_connections 2024-05-13T22:46:33.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.023+0000 7f64fb813700 1 --2- 172.21.15.102:0/3521259146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f4100cb0 0x7f64f4101110 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.023+0000 7f64fb813700 1 --2- 172.21.15.102:0/3521259146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64f40ffab0 0x7f64f40ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.023+0000 7f64fb813700 1 -- 172.21.15.102:0/3521259146 >> 172.21.15.102:0/3521259146 conn(0x7f64f40fb030 msgr2=0x7f64f40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.023+0000 7f64fb813700 1 -- 172.21.15.102:0/3521259146 shutdown_connections 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.023+0000 7f64fb813700 1 -- 172.21.15.102:0/3521259146 wait complete. 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64fb813700 1 Processor -- start 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64fb813700 1 -- start start 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64fb813700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64f40ffab0 0x7f64f4068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64fb813700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f4100cb0 0x7f64f419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64fb813700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f419b690 con 0x7f64f4100cb0 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64fb813700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f419b7d0 con 0x7f64f40ffab0 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64f8dae700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f4100cb0 0x7f64f419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64f95af700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64f40ffab0 0x7f64f4068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64f8dae700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f4100cb0 0x7f64f419b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42700/0 (socket says 172.21.15.102:42700) 2024-05-13T22:46:33.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64f8dae700 1 -- 172.21.15.102:0/550400698 learned_addr learned my addr 172.21.15.102:0/550400698 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64f8dae700 1 -- 172.21.15.102:0/550400698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64f40ffab0 msgr2=0x7f64f4068f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64f8dae700 1 --2- 172.21.15.102:0/550400698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64f40ffab0 0x7f64f4068f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.024+0000 7f64f8dae700 1 -- 172.21.15.102:0/550400698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e40095f0 con 0x7f64f4100cb0 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.025+0000 7f64f8dae700 1 --2- 172.21.15.102:0/550400698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f4100cb0 0x7f64f419b0c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f64e4006010 tx=0x7f64e400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.025+0000 7f64ea7fc700 1 -- 172.21.15.102:0/550400698 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e401d070 con 0x7f64f4100cb0 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.025+0000 7f64ea7fc700 1 -- 172.21.15.102:0/550400698 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f64e400f460 con 0x7f64f4100cb0 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.025+0000 7f64ea7fc700 1 -- 172.21.15.102:0/550400698 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e4021640 con 0x7f64f4100cb0 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.025+0000 7f64fb813700 1 -- 172.21.15.102:0/550400698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f419ba50 con 0x7f64f4100cb0 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.026+0000 7f64fb813700 1 -- 172.21.15.102:0/550400698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f419be80 con 0x7f64f4100cb0 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.027+0000 7f64fb813700 1 -- 172.21.15.102:0/550400698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f4108610 con 0x7f64f4100cb0 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.027+0000 7f64ea7fc700 1 -- 172.21.15.102:0/550400698 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f64e4003a60 con 0x7f64f4100cb0 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.028+0000 7f64ea7fc700 1 --2- 172.21.15.102:0/550400698 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f64e006c530 0x7f64e006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.028+0000 7f64ea7fc700 1 -- 172.21.15.102:0/550400698 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f64e408c960 con 0x7f64f4100cb0 2024-05-13T22:46:33.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.028+0000 7f64f95af700 1 --2- 172.21.15.102:0/550400698 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f64e006c530 0x7f64e006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:33.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.028+0000 7f64f95af700 1 --2- 172.21.15.102:0/550400698 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f64e006c530 0x7f64e006e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f64f4100b10 tx=0x7f64f000b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:46:33.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.032+0000 7f64ea7fc700 1 -- 172.21.15.102:0/550400698 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f64e405ac90 con 0x7f64f4100cb0 2024-05-13T22:46:33.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:32 smithi102 ceph-mon[28599]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:33.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:46:33.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:32 smithi102 ceph-mon[28599]: Upgrade: Target is version 19.0.0-2565-gc3e55145 (unknown) 2024-05-13T22:46:33.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:32 smithi102 ceph-mon[28599]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, digests ['quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64'] 2024-05-13T22:46:33.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:46:33.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:46:33.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:32 smithi102 ceph-mon[28599]: Upgrade: Need to upgrade myself (mgr.smithi102.xcijrj) 2024-05-13T22:46:33.249 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.270+0000 7f64fb813700 1 -- 172.21.15.102:0/550400698 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64f4105600 con 0x7f64e006c530 2024-05-13T22:46:33.250 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.272+0000 7f64ea7fc700 1 -- 172.21.15.102:0/550400698 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f64f4105600 con 0x7f64e006c530 2024-05-13T22:46:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.274+0000 7f64fb813700 1 -- 172.21.15.102:0/550400698 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f64e006c530 msgr2=0x7f64e006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.274+0000 7f64fb813700 1 --2- 172.21.15.102:0/550400698 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f64e006c530 0x7f64e006e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f64f4100b10 tx=0x7f64f000b410 comp rx=0 tx=0).stop 2024-05-13T22:46:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.274+0000 7f64fb813700 1 -- 172.21.15.102:0/550400698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f4100cb0 msgr2=0x7f64f419b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.274+0000 7f64fb813700 1 --2- 172.21.15.102:0/550400698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f4100cb0 0x7f64f419b0c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f64e4006010 tx=0x7f64e400bd70 comp rx=0 tx=0).stop 2024-05-13T22:46:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.275+0000 7f64fb813700 1 -- 172.21.15.102:0/550400698 shutdown_connections 2024-05-13T22:46:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.275+0000 7f64fb813700 1 --2- 172.21.15.102:0/550400698 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f64e006c530 0x7f64e006e9f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.275+0000 7f64fb813700 1 --2- 172.21.15.102:0/550400698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64f40ffab0 0x7f64f4068f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.275+0000 7f64fb813700 1 --2- 172.21.15.102:0/550400698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64f4100cb0 0x7f64f419b0c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.275+0000 7f64fb813700 1 -- 172.21.15.102:0/550400698 >> 172.21.15.102:0/550400698 conn(0x7f64f40fb030 msgr2=0x7f64f4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:46:33.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.275+0000 7f64fb813700 1 -- 172.21.15.102:0/550400698 shutdown_connections 2024-05-13T22:46:33.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.275+0000 7f64fb813700 1 -- 172.21.15.102:0/550400698 wait complete. 2024-05-13T22:46:33.266 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:46:33.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.390+0000 7f24804cf700 1 -- 172.21.15.102:0/4164249365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2478101640 msgr2=0x7f2478103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.390+0000 7f24804cf700 1 --2- 172.21.15.102:0/4164249365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2478101640 0x7f2478103a30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2474009910 tx=0x7f2474009c20 comp rx=0 tx=0).stop 2024-05-13T22:46:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.390+0000 7f24804cf700 1 -- 172.21.15.102:0/4164249365 shutdown_connections 2024-05-13T22:46:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.390+0000 7f24804cf700 1 --2- 172.21.15.102:0/4164249365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2478103f70 0x7f2478106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.390+0000 7f24804cf700 1 --2- 172.21.15.102:0/4164249365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2478101640 0x7f2478103a30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.390+0000 7f24804cf700 1 -- 172.21.15.102:0/4164249365 >> 172.21.15.102:0/4164249365 conn(0x7f24780fb050 msgr2=0x7f24780fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:46:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.390+0000 7f24804cf700 1 -- 172.21.15.102:0/4164249365 shutdown_connections 2024-05-13T22:46:33.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.390+0000 7f24804cf700 1 -- 172.21.15.102:0/4164249365 wait complete. 2024-05-13T22:46:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.391+0000 7f24804cf700 1 Processor -- start 2024-05-13T22:46:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.391+0000 7f24804cf700 1 -- start start 2024-05-13T22:46:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.391+0000 7f24804cf700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2478101640 0x7f2478196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.391+0000 7f24804cf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2478103f70 0x7f2478196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f24804cf700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2478196e70 con 0x7f2478103f70 2024-05-13T22:46:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f24804cf700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2478196fb0 con 0x7f2478101640 2024-05-13T22:46:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f247da6a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2478103f70 0x7f2478196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f247da6a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2478103f70 0x7f2478196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42710/0 (socket says 172.21.15.102:42710) 2024-05-13T22:46:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f247da6a700 1 -- 172.21.15.102:0/2258298893 learned_addr learned my addr 172.21.15.102:0/2258298893 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:46:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f247e26b700 1 --2- 172.21.15.102:0/2258298893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2478101640 0x7f2478196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:33.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f247da6a700 1 -- 172.21.15.102:0/2258298893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2478101640 msgr2=0x7f2478196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f247da6a700 1 --2- 172.21.15.102:0/2258298893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2478101640 0x7f2478196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f247da6a700 1 -- 172.21.15.102:0/2258298893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24740095f0 con 0x7f2478103f70 2024-05-13T22:46:33.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f247da6a700 1 --2- 172.21.15.102:0/2258298893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2478103f70 0x7f2478196850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f246800e920 tx=0x7f246800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:46:33.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f246f7fe700 1 -- 172.21.15.102:0/2258298893 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f246800c9d0 con 0x7f2478103f70 2024-05-13T22:46:33.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f246f7fe700 1 -- 172.21.15.102:0/2258298893 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2468010430 con 0x7f2478103f70 2024-05-13T22:46:33.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f24804cf700 1 -- 172.21.15.102:0/2258298893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247819ba60 con 0x7f2478103f70 2024-05-13T22:46:33.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.392+0000 7f246f7fe700 1 -- 172.21.15.102:0/2258298893 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2468018650 con 0x7f2478103f70 2024-05-13T22:46:33.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.393+0000 7f24804cf700 1 -- 172.21.15.102:0/2258298893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247819bfb0 con 0x7f2478103f70 2024-05-13T22:46:33.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.394+0000 7f24804cf700 1 -- 172.21.15.102:0/2258298893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2478190520 con 0x7f2478103f70 2024-05-13T22:46:33.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.394+0000 7f246f7fe700 1 -- 172.21.15.102:0/2258298893 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f24680105a0 con 0x7f2478103f70 2024-05-13T22:46:33.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.395+0000 7f246f7fe700 1 --2- 172.21.15.102:0/2258298893 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f246406c4e0 0x7f246406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:33.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.395+0000 7f246f7fe700 1 -- 172.21.15.102:0/2258298893 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2468014070 con 0x7f2478103f70 2024-05-13T22:46:33.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.395+0000 7f247e26b700 1 --2- 172.21.15.102:0/2258298893 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f246406c4e0 0x7f246406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:33.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.396+0000 7f247e26b700 1 --2- 172.21.15.102:0/2258298893 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f246406c4e0 0x7f246406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2474009fd0 tx=0x7f2474005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:46:33.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.400+0000 7f246f7fe700 1 -- 172.21.15.102:0/2258298893 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f24680520e0 con 0x7f2478103f70 2024-05-13T22:46:33.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:32 smithi130 ceph-mon[36591]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:33.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:46:33.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:32 smithi130 ceph-mon[36591]: Upgrade: Target is version 19.0.0-2565-gc3e55145 (unknown) 2024-05-13T22:46:33.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:32 smithi130 ceph-mon[36591]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, digests ['quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64'] 2024-05-13T22:46:33.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:46:33.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:46:33.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:32 smithi130 ceph-mon[36591]: Upgrade: Need to upgrade myself (mgr.smithi102.xcijrj) 2024-05-13T22:46:33.615 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.636+0000 7f24804cf700 1 -- 172.21.15.102:0/2258298893 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24780611d0 con 0x7f246406c4e0 2024-05-13T22:46:33.616 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.638+0000 7f246f7fe700 1 -- 172.21.15.102:0/2258298893 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f24780611d0 con 0x7f246406c4e0 2024-05-13T22:46:33.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.640+0000 7f24804cf700 1 -- 172.21.15.102:0/2258298893 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f246406c4e0 msgr2=0x7f246406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.640+0000 7f24804cf700 1 --2- 172.21.15.102:0/2258298893 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f246406c4e0 0x7f246406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2474009fd0 tx=0x7f2474005940 comp rx=0 tx=0).stop 2024-05-13T22:46:33.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.640+0000 7f24804cf700 1 -- 172.21.15.102:0/2258298893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2478103f70 msgr2=0x7f2478196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.640+0000 7f24804cf700 1 --2- 172.21.15.102:0/2258298893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2478103f70 0x7f2478196850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f246800e920 tx=0x7f246800ec30 comp rx=0 tx=0).stop 2024-05-13T22:46:33.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.640+0000 7f24804cf700 1 -- 172.21.15.102:0/2258298893 shutdown_connections 2024-05-13T22:46:33.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.640+0000 7f24804cf700 1 --2- 172.21.15.102:0/2258298893 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f246406c4e0 0x7f246406e9a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.640+0000 7f24804cf700 1 --2- 172.21.15.102:0/2258298893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2478101640 0x7f2478196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.640+0000 7f24804cf700 1 --2- 172.21.15.102:0/2258298893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2478103f70 0x7f2478196850 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.640+0000 7f24804cf700 1 -- 172.21.15.102:0/2258298893 >> 172.21.15.102:0/2258298893 conn(0x7f24780fb050 msgr2=0x7f24780fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:46:33.620 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.640+0000 7f24804cf700 1 -- 172.21.15.102:0/2258298893 shutdown_connections 2024-05-13T22:46:33.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.640+0000 7f24804cf700 1 -- 172.21.15.102:0/2258298893 wait complete. 2024-05-13T22:46:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.766+0000 7f290cdfb700 1 -- 172.21.15.102:0/1654680946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2908100cb0 msgr2=0x7f2908101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.766+0000 7f290cdfb700 1 --2- 172.21.15.102:0/1654680946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2908100cb0 0x7f2908101110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009910 tx=0x7f28fc009c20 comp rx=0 tx=0).stop 2024-05-13T22:46:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.766+0000 7f290cdfb700 1 -- 172.21.15.102:0/1654680946 shutdown_connections 2024-05-13T22:46:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.766+0000 7f290cdfb700 1 --2- 172.21.15.102:0/1654680946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2908100cb0 0x7f2908101110 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.766+0000 7f290cdfb700 1 --2- 172.21.15.102:0/1654680946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29080ffab0 0x7f29080ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.766+0000 7f290cdfb700 1 -- 172.21.15.102:0/1654680946 >> 172.21.15.102:0/1654680946 conn(0x7f29080fb030 msgr2=0x7f29080fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:46:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.766+0000 7f290cdfb700 1 -- 172.21.15.102:0/1654680946 shutdown_connections 2024-05-13T22:46:33.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.767+0000 7f290cdfb700 1 -- 172.21.15.102:0/1654680946 wait complete. 2024-05-13T22:46:33.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.767+0000 7f290cdfb700 1 Processor -- start 2024-05-13T22:46:33.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.768+0000 7f290cdfb700 1 -- start start 2024-05-13T22:46:33.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.768+0000 7f290cdfb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29080ffab0 0x7f2908195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:33.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.768+0000 7f290cdfb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2908100cb0 0x7f29081957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.768+0000 7f290cdfb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2908195dc0 con 0x7f29080ffab0 2024-05-13T22:46:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.768+0000 7f2905d9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2908100cb0 0x7f29081957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.768+0000 7f2905d9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2908100cb0 0x7f29081957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.102:60930/0 (socket says 172.21.15.102:60930) 2024-05-13T22:46:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.768+0000 7f2905d9b700 1 -- 172.21.15.102:0/3022510644 learned_addr learned my addr 172.21.15.102:0/3022510644 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:46:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.768+0000 7f290659c700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29080ffab0 0x7f2908195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:33.748 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.768+0000 7f290cdfb700 1 -- 172.21.15.102:0/3022510644 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2908195f00 con 0x7f2908100cb0 2024-05-13T22:46:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.769+0000 7f2905d9b700 1 -- 172.21.15.102:0/3022510644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29080ffab0 msgr2=0x7f2908195260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.769+0000 7f2905d9b700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29080ffab0 0x7f2908195260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.769+0000 7f2905d9b700 1 -- 172.21.15.102:0/3022510644 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc0095f0 con 0x7f2908100cb0 2024-05-13T22:46:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.769+0000 7f290659c700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29080ffab0 0x7f2908195260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:46:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.769+0000 7f2905d9b700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2908100cb0 0x7f29081957a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f28fc006010 tx=0x7f28fc00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:46:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.769+0000 7f28f77fe700 1 -- 172.21.15.102:0/3022510644 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28fc01d070 con 0x7f2908100cb0 2024-05-13T22:46:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.769+0000 7f290cdfb700 1 -- 172.21.15.102:0/3022510644 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2908068410 con 0x7f2908100cb0 2024-05-13T22:46:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.769+0000 7f28f77fe700 1 -- 172.21.15.102:0/3022510644 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f28fc00f460 con 0x7f2908100cb0 2024-05-13T22:46:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.769+0000 7f28f77fe700 1 -- 172.21.15.102:0/3022510644 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28fc021640 con 0x7f2908100cb0 2024-05-13T22:46:33.749 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.769+0000 7f290cdfb700 1 -- 172.21.15.102:0/3022510644 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2908068900 con 0x7f2908100cb0 2024-05-13T22:46:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.770+0000 7f290cdfb700 1 -- 172.21.15.102:0/3022510644 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2908108610 con 0x7f2908100cb0 2024-05-13T22:46:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.771+0000 7f28f77fe700 1 -- 172.21.15.102:0/3022510644 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f28fc003a60 con 0x7f2908100cb0 2024-05-13T22:46:33.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.772+0000 7f28f77fe700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28f00708f0 0x7f28f0072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.772+0000 7f28f77fe700 1 -- 172.21.15.102:0/3022510644 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f28fc08cbc0 con 0x7f2908100cb0 2024-05-13T22:46:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.772+0000 7f290659c700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28f00708f0 0x7f28f0072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:33.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.772+0000 7f290659c700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28f00708f0 0x7f28f0072db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28f800d440 tx=0x7f28f800d620 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:46:33.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:33.778+0000 7f28f77fe700 1 -- 172.21.15.102:0/3022510644 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f28fc05ad30 con 0x7f2908100cb0 2024-05-13T22:46:33.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.000+0000 7f290cdfb700 1 -- 172.21.15.102:0/3022510644 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2908105600 con 0x7f28f00708f0 2024-05-13T22:46:33.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.012+0000 7f28f77fe700 1 -- 172.21.15.102:0/3022510644 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f2908105600 con 0x7f28f00708f0 2024-05-13T22:46:33.991 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:46:33.991 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (5m) 55s ago 6m 25.9M - 0.25.0 c8568f914cd2 c096e9e9a228 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (7m) 55s ago 7m 7457k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (5m) 61s ago 5m 7289k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (7m) 55s ago 7m 6656k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (5m) 61s ago 5m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (5m) 55s ago 6m 87.4M - 9.4.7 954c08fa6188 bca1790c17aa 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (79s) 55s ago 79s 11.9M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (73s) 55s ago 73s 15.8M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (76s) 61s ago 76s 11.2M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (70s) 61s ago 70s 14.3M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:9283,8765,8443 running (8m) 55s ago 8m 488M - 18.2.1 5be31c24972a 77cd1ca053e7 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (5m) 61s ago 5m 440M - 18.2.1 5be31c24972a c8a9d782f394 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (8m) 55s ago 8m 44.5M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (5m) 61s ago 5m 37.5M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:46:33.992 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (6m) 55s ago 6m 16.3M - 1.5.0 0da6a335fe13 af297cf52099 2024-05-13T22:46:33.993 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (5m) 61s ago 5m 15.8M - 1.5.0 0da6a335fe13 cc0f64ee7ecd 2024-05-13T22:46:33.993 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (4m) 55s ago 4m 48.0M 3946M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:46:33.993 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (4m) 55s ago 4m 47.7M 3946M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:46:33.993 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (4m) 55s ago 4m 45.7M 3946M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:46:33.993 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (3m) 61s ago 3m 47.0M 4970M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:46:33.993 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (3m) 61s ago 3m 46.6M 4970M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:46:33.993 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (2m) 61s ago 2m 45.8M 4970M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:46:33.993 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (5m) 55s ago 6m 54.0M - 2.43.0 a07b618ecd1d a89752444eb6 2024-05-13T22:46:33.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.014+0000 7f290cdfb700 1 -- 172.21.15.102:0/3022510644 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28f00708f0 msgr2=0x7f28f0072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.014+0000 7f290cdfb700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28f00708f0 0x7f28f0072db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28f800d440 tx=0x7f28f800d620 comp rx=0 tx=0).stop 2024-05-13T22:46:33.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.014+0000 7f290cdfb700 1 -- 172.21.15.102:0/3022510644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2908100cb0 msgr2=0x7f29081957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:33.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.014+0000 7f290cdfb700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2908100cb0 0x7f29081957a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f28fc006010 tx=0x7f28fc00bd70 comp rx=0 tx=0).stop 2024-05-13T22:46:33.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.014+0000 7f290cdfb700 1 -- 172.21.15.102:0/3022510644 shutdown_connections 2024-05-13T22:46:33.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.014+0000 7f290cdfb700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f28f00708f0 0x7f28f0072db0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.014+0000 7f290cdfb700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29080ffab0 0x7f2908195260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.014+0000 7f290cdfb700 1 --2- 172.21.15.102:0/3022510644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2908100cb0 0x7f29081957a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:33.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.014+0000 7f290cdfb700 1 -- 172.21.15.102:0/3022510644 >> 172.21.15.102:0/3022510644 conn(0x7f29080fb030 msgr2=0x7f2908103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:46:33.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.015+0000 7f290cdfb700 1 -- 172.21.15.102:0/3022510644 shutdown_connections 2024-05-13T22:46:33.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.015+0000 7f290cdfb700 1 -- 172.21.15.102:0/3022510644 wait complete. 2024-05-13T22:46:34.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.129+0000 7f4c21124700 1 -- 172.21.15.102:0/942633747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c1c0ffab0 msgr2=0x7f4c1c0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:34.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.129+0000 7f4c21124700 1 --2- 172.21.15.102:0/942633747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c1c0ffab0 0x7f4c1c0ffed0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009910 tx=0x7f4c0c009c20 comp rx=0 tx=0).stop 2024-05-13T22:46:34.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.129+0000 7f4c21124700 1 -- 172.21.15.102:0/942633747 shutdown_connections 2024-05-13T22:46:34.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.129+0000 7f4c21124700 1 --2- 172.21.15.102:0/942633747 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c1c100cb0 0x7f4c1c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.129+0000 7f4c21124700 1 --2- 172.21.15.102:0/942633747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c1c0ffab0 0x7f4c1c0ffed0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.129+0000 7f4c21124700 1 -- 172.21.15.102:0/942633747 >> 172.21.15.102:0/942633747 conn(0x7f4c1c0fb050 msgr2=0x7f4c1c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:46:34.109 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:33 smithi102 ceph-mon[28599]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 on smithi130 2024-05-13T22:46:34.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.129+0000 7f4c21124700 1 -- 172.21.15.102:0/942633747 shutdown_connections 2024-05-13T22:46:34.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.129+0000 7f4c21124700 1 -- 172.21.15.102:0/942633747 wait complete. 2024-05-13T22:46:34.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.130+0000 7f4c21124700 1 Processor -- start 2024-05-13T22:46:34.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.130+0000 7f4c21124700 1 -- start start 2024-05-13T22:46:34.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.130+0000 7f4c21124700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c1c0ffab0 0x7f4c1c196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:34.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.130+0000 7f4c21124700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c1c100cb0 0x7f4c1c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:34.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.130+0000 7f4c21124700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c196ed0 con 0x7f4c1c100cb0 2024-05-13T22:46:34.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.130+0000 7f4c21124700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c197010 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.131+0000 7f4c1a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c1c100cb0 0x7f4c1c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:34.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.131+0000 7f4c1ad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c1c0ffab0 0x7f4c1c196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:34.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.131+0000 7f4c1ad9d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c1c0ffab0 0x7f4c1c196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.102:60940/0 (socket says 172.21.15.102:60940) 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.131+0000 7f4c1a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c1c100cb0 0x7f4c1c1968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42740/0 (socket says 172.21.15.102:42740) 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.131+0000 7f4c1ad9d700 1 -- 172.21.15.102:0/1197851811 learned_addr learned my addr 172.21.15.102:0/1197851811 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.131+0000 7f4c1ad9d700 1 -- 172.21.15.102:0/1197851811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c1c100cb0 msgr2=0x7f4c1c1968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.131+0000 7f4c1ad9d700 1 --2- 172.21.15.102:0/1197851811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c1c100cb0 0x7f4c1c1968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.131+0000 7f4c1ad9d700 1 -- 172.21.15.102:0/1197851811 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c0095f0 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.131+0000 7f4c1a59c700 1 --2- 172.21.15.102:0/1197851811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c1c100cb0 0x7f4c1c1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.131+0000 7f4c1ad9d700 1 --2- 172.21.15.102:0/1197851811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c1c0ffab0 0x7f4c1c196370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c005b40 tx=0x7f4c0c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.131+0000 7f4c03fff700 1 -- 172.21.15.102:0/1197851811 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c0c01d070 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.132+0000 7f4c21124700 1 -- 172.21.15.102:0/1197851811 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c101ed0 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.132+0000 7f4c03fff700 1 -- 172.21.15.102:0/1197851811 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c0c005dc0 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.132+0000 7f4c03fff700 1 -- 172.21.15.102:0/1197851811 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c0c022470 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.132+0000 7f4c21124700 1 -- 172.21.15.102:0/1197851811 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c1023c0 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.133+0000 7f4c21124700 1 -- 172.21.15.102:0/1197851811 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c108610 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.134+0000 7f4c03fff700 1 -- 172.21.15.102:0/1197851811 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4c0c003680 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.134+0000 7f4c03fff700 1 --2- 172.21.15.102:0/1197851811 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4c0406c4e0 0x7f4c0406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.135+0000 7f4c03fff700 1 -- 172.21.15.102:0/1197851811 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4c0c08cd70 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.135+0000 7f4c1a59c700 1 --2- 172.21.15.102:0/1197851811 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4c0406c4e0 0x7f4c0406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:34.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.135+0000 7f4c1a59c700 1 --2- 172.21.15.102:0/1197851811 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4c0406c4e0 0x7f4c0406e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4c10011490 tx=0x7f4c10009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:46:34.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.141+0000 7f4c03fff700 1 -- 172.21.15.102:0/1197851811 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4c0c05b000 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:33 smithi130 ceph-mon[36591]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 on smithi130 2024-05-13T22:46:34.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.482+0000 7f4c21124700 1 -- 172.21.15.102:0/1197851811 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4c1c102f90 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.483+0000 7f4c03fff700 1 -- 172.21.15.102:0/1197851811 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f4c0c05ab90 con 0x7f4c1c0ffab0 2024-05-13T22:46:34.462 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:46:34.462 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:46:34.462 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:46:34.462 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:46:34.463 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:46:34.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.486+0000 7f4c21124700 1 -- 172.21.15.102:0/1197851811 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4c0406c4e0 msgr2=0x7f4c0406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:34.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.486+0000 7f4c21124700 1 --2- 172.21.15.102:0/1197851811 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4c0406c4e0 0x7f4c0406e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4c10011490 tx=0x7f4c10009260 comp rx=0 tx=0).stop 2024-05-13T22:46:34.465 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.486+0000 7f4c21124700 1 -- 172.21.15.102:0/1197851811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c1c0ffab0 msgr2=0x7f4c1c196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:34.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.486+0000 7f4c21124700 1 --2- 172.21.15.102:0/1197851811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c1c0ffab0 0x7f4c1c196370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c005b40 tx=0x7f4c0c00fa00 comp rx=0 tx=0).stop 2024-05-13T22:46:34.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.486+0000 7f4c21124700 1 -- 172.21.15.102:0/1197851811 shutdown_connections 2024-05-13T22:46:34.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.486+0000 7f4c21124700 1 --2- 172.21.15.102:0/1197851811 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f4c0406c4e0 0x7f4c0406e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.486+0000 7f4c21124700 1 --2- 172.21.15.102:0/1197851811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c1c0ffab0 0x7f4c1c196370 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.486+0000 7f4c21124700 1 --2- 172.21.15.102:0/1197851811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c1c100cb0 0x7f4c1c1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.486+0000 7f4c21124700 1 -- 172.21.15.102:0/1197851811 >> 172.21.15.102:0/1197851811 conn(0x7f4c1c0fb050 msgr2=0x7f4c1c068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:46:34.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.486+0000 7f4c21124700 1 -- 172.21.15.102:0/1197851811 shutdown_connections 2024-05-13T22:46:34.466 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.486+0000 7f4c21124700 1 -- 172.21.15.102:0/1197851811 wait complete. 2024-05-13T22:46:34.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.647+0000 7f6f9a428700 1 -- 172.21.15.102:0/2772701484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f94100cb0 msgr2=0x7f6f94101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:34.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.647+0000 7f6f9a428700 1 --2- 172.21.15.102:0/2772701484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f94100cb0 0x7f6f94101110 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009910 tx=0x7f6f88009c20 comp rx=0 tx=0).stop 2024-05-13T22:46:34.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.647+0000 7f6f9a428700 1 -- 172.21.15.102:0/2772701484 shutdown_connections 2024-05-13T22:46:34.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.647+0000 7f6f9a428700 1 --2- 172.21.15.102:0/2772701484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f94100cb0 0x7f6f94101110 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.647+0000 7f6f9a428700 1 --2- 172.21.15.102:0/2772701484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f940ffab0 0x7f6f940ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.647+0000 7f6f9a428700 1 -- 172.21.15.102:0/2772701484 >> 172.21.15.102:0/2772701484 conn(0x7f6f940fb030 msgr2=0x7f6f940fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:46:34.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.647+0000 7f6f9a428700 1 -- 172.21.15.102:0/2772701484 shutdown_connections 2024-05-13T22:46:34.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.647+0000 7f6f9a428700 1 -- 172.21.15.102:0/2772701484 wait complete. 2024-05-13T22:46:34.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.648+0000 7f6f9a428700 1 Processor -- start 2024-05-13T22:46:34.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.648+0000 7f6f9a428700 1 -- start start 2024-05-13T22:46:34.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.649+0000 7f6f9a428700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f940ffab0 0x7f6f94195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:34.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.649+0000 7f6f9a428700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f94100cb0 0x7f6f941957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:34.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.649+0000 7f6f9a428700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94195dc0 con 0x7f6f940ffab0 2024-05-13T22:46:34.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.649+0000 7f6f9a428700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94195f00 con 0x7f6f94100cb0 2024-05-13T22:46:34.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.649+0000 7f6f93fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f940ffab0 0x7f6f94195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:34.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.649+0000 7f6f93fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f940ffab0 0x7f6f94195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42752/0 (socket says 172.21.15.102:42752) 2024-05-13T22:46:34.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.649+0000 7f6f93fff700 1 -- 172.21.15.102:0/2016191464 learned_addr learned my addr 172.21.15.102:0/2016191464 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:46:34.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.649+0000 7f6f937fe700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f94100cb0 0x7f6f941957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:34.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.650+0000 7f6f93fff700 1 -- 172.21.15.102:0/2016191464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f94100cb0 msgr2=0x7f6f941957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:34.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.650+0000 7f6f93fff700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f94100cb0 0x7f6f941957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.650+0000 7f6f93fff700 1 -- 172.21.15.102:0/2016191464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f880095f0 con 0x7f6f940ffab0 2024-05-13T22:46:34.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.650+0000 7f6f937fe700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f94100cb0 0x7f6f941957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:46:34.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.650+0000 7f6f93fff700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f940ffab0 0x7f6f94195260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400ea60 tx=0x7f6f8400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:46:34.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.650+0000 7f6f917fa700 1 -- 172.21.15.102:0/2016191464 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f84013070 con 0x7f6f940ffab0 2024-05-13T22:46:34.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.650+0000 7f6f917fa700 1 -- 172.21.15.102:0/2016191464 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6f8400b940 con 0x7f6f940ffab0 2024-05-13T22:46:34.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.650+0000 7f6f9a428700 1 -- 172.21.15.102:0/2016191464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f94068470 con 0x7f6f940ffab0 2024-05-13T22:46:34.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.650+0000 7f6f917fa700 1 -- 172.21.15.102:0/2016191464 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f84003d60 con 0x7f6f940ffab0 2024-05-13T22:46:34.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.651+0000 7f6f9a428700 1 -- 172.21.15.102:0/2016191464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f940689c0 con 0x7f6f940ffab0 2024-05-13T22:46:34.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.651+0000 7f6f9a428700 1 -- 172.21.15.102:0/2016191464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f94108630 con 0x7f6f940ffab0 2024-05-13T22:46:34.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.654+0000 7f6f917fa700 1 -- 172.21.15.102:0/2016191464 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f6f84004200 con 0x7f6f940ffab0 2024-05-13T22:46:34.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.654+0000 7f6f917fa700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6f7c06c4e0 0x7f6f7c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:46:34.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.655+0000 7f6f937fe700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6f7c06c4e0 0x7f6f7c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:46:34.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.655+0000 7f6f917fa700 1 -- 172.21.15.102:0/2016191464 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6f8408d930 con 0x7f6f940ffab0 2024-05-13T22:46:34.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.655+0000 7f6f937fe700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6f7c06c4e0 0x7f6f7c06e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009fd0 tx=0x7f6f88005fd0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:46:34.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.660+0000 7f6f917fa700 1 -- 172.21.15.102:0/2016191464 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6f8405bba0 con 0x7f6f940ffab0 2024-05-13T22:46:34.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.885+0000 7f6f9a428700 1 -- 172.21.15.102:0/2016191464 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f94105600 con 0x7f6f7c06c4e0 2024-05-13T22:46:34.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.887+0000 7f6f917fa700 1 -- 172.21.15.102:0/2016191464 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f6f94105600 con 0x7f6f7c06c4e0 2024-05-13T22:46:34.866 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:46:34.866 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:46:34.866 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:46:34.866 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-13T22:46:34.866 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [], 2024-05-13T22:46:34.867 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "0/2 daemons upgraded", 2024-05-13T22:46:34.867 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 image on host smithi130", 2024-05-13T22:46:34.867 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:46:34.867 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:46:34.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.890+0000 7f6f9a428700 1 -- 172.21.15.102:0/2016191464 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6f7c06c4e0 msgr2=0x7f6f7c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:34.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.890+0000 7f6f9a428700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6f7c06c4e0 0x7f6f7c06e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009fd0 tx=0x7f6f88005fd0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.890+0000 7f6f9a428700 1 -- 172.21.15.102:0/2016191464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f940ffab0 msgr2=0x7f6f94195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:46:34.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.890+0000 7f6f9a428700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f940ffab0 0x7f6f94195260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400ea60 tx=0x7f6f8400ed70 comp rx=0 tx=0).stop 2024-05-13T22:46:34.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.890+0000 7f6f9a428700 1 -- 172.21.15.102:0/2016191464 shutdown_connections 2024-05-13T22:46:34.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.890+0000 7f6f9a428700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f6f7c06c4e0 0x7f6f7c06e9a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.890+0000 7f6f9a428700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f940ffab0 0x7f6f94195260 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.890+0000 7f6f9a428700 1 --2- 172.21.15.102:0/2016191464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f94100cb0 0x7f6f941957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:46:34.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.890+0000 7f6f9a428700 1 -- 172.21.15.102:0/2016191464 >> 172.21.15.102:0/2016191464 conn(0x7f6f940fb030 msgr2=0x7f6f94103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:46:34.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.890+0000 7f6f9a428700 1 -- 172.21.15.102:0/2016191464 shutdown_connections 2024-05-13T22:46:34.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:46:34.891+0000 7f6f9a428700 1 -- 172.21.15.102:0/2016191464 wait complete. 2024-05-13T22:46:35.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:34 smithi102 ceph-mon[28599]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:46:35.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:34 smithi102 ceph-mon[28599]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:46:35.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:34 smithi102 ceph-mon[28599]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:35.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:34 smithi102 ceph-mon[28599]: from='client.24339 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:46:35.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:34 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1197851811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:46:35.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:34 smithi130 ceph-mon[36591]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:46:35.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:34 smithi130 ceph-mon[36591]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:46:35.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:34 smithi130 ceph-mon[36591]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:35.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:34 smithi130 ceph-mon[36591]: from='client.24339 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:46:35.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:34 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1197851811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:46:36.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:35 smithi130 ceph-mon[36591]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:46:36.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:35 smithi102 ceph-mon[28599]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:46:37.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:36 smithi130 ceph-mon[36591]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:37.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:36 smithi102 ceph-mon[28599]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:38 smithi130 ceph-mon[36591]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:38 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:46:39.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:38 smithi102 ceph-mon[28599]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:39.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:38 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:46:41.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:40 smithi130 ceph-mon[36591]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:41.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:40 smithi102 ceph-mon[28599]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:43.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:42 smithi130 ceph-mon[36591]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:43.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:42 smithi102 ceph-mon[28599]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:45.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:44 smithi130 ceph-mon[36591]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:45.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:44 smithi102 ceph-mon[28599]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:46.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:46 smithi130 ceph-mon[36591]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:46 smithi102 ceph-mon[28599]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:49.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:48 smithi130 ceph-mon[36591]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:49.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:48 smithi102 ceph-mon[28599]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:51.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:50 smithi130 ceph-mon[36591]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:51.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:50 smithi102 ceph-mon[28599]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:53.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:52 smithi130 ceph-mon[36591]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:53.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:52 smithi102 ceph-mon[28599]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:54.051 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:53 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:46:54.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:53 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:46:55.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:54 smithi130 ceph-mon[36591]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:55.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:54 smithi102 ceph-mon[28599]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:46:56.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:56 smithi130 ceph-mon[36591]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:56.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:56 smithi102 ceph-mon[28599]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:59.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:46:58 smithi130 ceph-mon[36591]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:46:59.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:46:58 smithi102 ceph-mon[28599]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:47:01.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:00 smithi130 ceph-mon[36591]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:47:01.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:00 smithi102 ceph-mon[28599]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:47:01.508 INFO:tasks.workunit.client.0.smithi102.stderr:Note: switching to 'c3e55145f4f4b3a500bb66ee08c938a215bc231d'. 2024-05-13T22:47:01.508 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr:state without impacting any branches by switching back to a branch. 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr: git switch -c 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr:Or undo this operation with: 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr: git switch - 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-13T22:47:01.509 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-05-13T22:47:01.510 INFO:tasks.workunit.client.0.smithi102.stderr:HEAD is now at c3e55145f4f Merge pull request #57426 from zdover23/wip-doc-2024-05-13-backport-57420-to-squid 2024-05-13T22:47:01.513 DEBUG:teuthology.orchestra.run.smithi102:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-13T22:47:01.539 INFO:tasks.workunit.client.0.smithi102.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-13T22:47:01.541 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-13T22:47:01.541 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-13T22:47:02.253 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-13T22:47:02.306 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-13T22:47:02.346 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-13T22:47:02.347 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-13T22:47:02.347 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-13T22:47:02.386 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-13T22:47:02.389 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T22:47:02.389 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-13T22:47:02.446 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-05-13T22:47:02.448 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-13T22:47:02.448 DEBUG:teuthology.orchestra.run.smithi102:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c3e55145f4f4b3a500bb66ee08c938a215bc231d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-05-13T22:47:02.511 INFO:tasks.workunit.client.0.smithi102.stderr:+ mkdir -p fsstress 2024-05-13T22:47:02.513 INFO:tasks.workunit.client.0.smithi102.stderr:+ pushd fsstress 2024-05-13T22:47:02.514 INFO:tasks.workunit.client.0.smithi102.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-13T22:47:02.515 INFO:tasks.workunit.client.0.smithi102.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-13T22:47:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:02 smithi130 ceph-mon[36591]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:47:03.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:02 smithi102 ceph-mon[28599]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:47:03.449 INFO:tasks.workunit.client.0.smithi102.stderr:+ tar xzf ltp-full.tgz 2024-05-13T22:47:05.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.024+0000 7fb1d6c78700 1 -- 172.21.15.102:0/4056865845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d0100bf0 msgr2=0x7fb1d0101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:05.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.024+0000 7fb1d6c78700 1 --2- 172.21.15.102:0/4056865845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d0100bf0 0x7fb1d0101010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009910 tx=0x7fb1c4009c20 comp rx=0 tx=0).stop 2024-05-13T22:47:05.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.024+0000 7fb1d6c78700 1 -- 172.21.15.102:0/4056865845 shutdown_connections 2024-05-13T22:47:05.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.024+0000 7fb1d6c78700 1 --2- 172.21.15.102:0/4056865845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb1d0101df0 0x7fb1d0102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.024+0000 7fb1d6c78700 1 --2- 172.21.15.102:0/4056865845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d0100bf0 0x7fb1d0101010 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.024+0000 7fb1d6c78700 1 -- 172.21.15.102:0/4056865845 >> 172.21.15.102:0/4056865845 conn(0x7fb1d00fc190 msgr2=0x7fb1d00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:05.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.024+0000 7fb1d6c78700 1 -- 172.21.15.102:0/4056865845 shutdown_connections 2024-05-13T22:47:05.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.024+0000 7fb1d6c78700 1 -- 172.21.15.102:0/4056865845 wait complete. 2024-05-13T22:47:05.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.025+0000 7fb1d6c78700 1 Processor -- start 2024-05-13T22:47:05.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.025+0000 7fb1d6c78700 1 -- start start 2024-05-13T22:47:05.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.025+0000 7fb1d6c78700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d0100bf0 0x7fb1d0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:05.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.025+0000 7fb1d6c78700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb1d0101df0 0x7fb1d0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:05.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.025+0000 7fb1d6c78700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00736c0 con 0x7fb1d0100bf0 2024-05-13T22:47:05.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.025+0000 7fb1d6c78700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d0073800 con 0x7fb1d0101df0 2024-05-13T22:47:05.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.025+0000 7fb1d4a14700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d0100bf0 0x7fb1d0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:05.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.025+0000 7fb1d4a14700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d0100bf0 0x7fb1d0074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44540/0 (socket says 172.21.15.102:44540) 2024-05-13T22:47:05.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.025+0000 7fb1d4a14700 1 -- 172.21.15.102:0/1913041664 learned_addr learned my addr 172.21.15.102:0/1913041664 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:47:05.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.025+0000 7fb1cffff700 1 --2- 172.21.15.102:0/1913041664 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb1d0101df0 0x7fb1d0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:05.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.026+0000 7fb1d4a14700 1 -- 172.21.15.102:0/1913041664 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb1d0101df0 msgr2=0x7fb1d0073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:05.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.026+0000 7fb1d4a14700 1 --2- 172.21.15.102:0/1913041664 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb1d0101df0 0x7fb1d0073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.026+0000 7fb1d4a14700 1 -- 172.21.15.102:0/1913041664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c40095f0 con 0x7fb1d0100bf0 2024-05-13T22:47:05.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.026+0000 7fb1d4a14700 1 --2- 172.21.15.102:0/1913041664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d0100bf0 0x7fb1d0074b30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400bda0 tx=0x7fb1c400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:05.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.026+0000 7fb1cdffb700 1 -- 172.21.15.102:0/1913041664 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1c401d070 con 0x7fb1d0100bf0 2024-05-13T22:47:05.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.026+0000 7fb1cdffb700 1 -- 172.21.15.102:0/1913041664 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb1c400f460 con 0x7fb1d0100bf0 2024-05-13T22:47:05.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.026+0000 7fb1d6c78700 1 -- 172.21.15.102:0/1913041664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d0073a80 con 0x7fb1d0100bf0 2024-05-13T22:47:05.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.026+0000 7fb1cdffb700 1 -- 172.21.15.102:0/1913041664 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1c4021620 con 0x7fb1d0100bf0 2024-05-13T22:47:05.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.026+0000 7fb1d6c78700 1 -- 172.21.15.102:0/1913041664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d0073f70 con 0x7fb1d0100bf0 2024-05-13T22:47:05.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.027+0000 7fb1d6c78700 1 -- 172.21.15.102:0/1913041664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d0109750 con 0x7fb1d0100bf0 2024-05-13T22:47:05.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.028+0000 7fb1cdffb700 1 -- 172.21.15.102:0/1913041664 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb1c400f5d0 con 0x7fb1d0100bf0 2024-05-13T22:47:05.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.029+0000 7fb1cdffb700 1 --2- 172.21.15.102:0/1913041664 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb1c006c4e0 0x7fb1c006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:05.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.029+0000 7fb1cdffb700 1 -- 172.21.15.102:0/1913041664 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb1c408cb20 con 0x7fb1d0100bf0 2024-05-13T22:47:05.012 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.029+0000 7fb1cffff700 1 --2- 172.21.15.102:0/1913041664 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb1c006c4e0 0x7fb1c006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:05.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.029+0000 7fb1cffff700 1 --2- 172.21.15.102:0/1913041664 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb1c006c4e0 0x7fb1c006e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc007c60 tx=0x7fb1bc0073d0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:05.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.035+0000 7fb1cdffb700 1 -- 172.21.15.102:0/1913041664 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb1c405ad30 con 0x7fb1d0100bf0 2024-05-13T22:47:05.087 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:04 smithi130 ceph-mon[36591]: pgmap v235: 65 pgs: 65 active+clean; 597 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-13T22:47:05.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:04 smithi102 ceph-mon[28599]: pgmap v235: 65 pgs: 65 active+clean; 597 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-13T22:47:05.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.331+0000 7fb1d6c78700 1 -- 172.21.15.102:0/1913041664 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1d0106740 con 0x7fb1c006c4e0 2024-05-13T22:47:05.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.333+0000 7fb1cdffb700 1 -- 172.21.15.102:0/1913041664 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fb1d0106740 con 0x7fb1c006c4e0 2024-05-13T22:47:05.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.335+0000 7fb1d6c78700 1 -- 172.21.15.102:0/1913041664 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb1c006c4e0 msgr2=0x7fb1c006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:05.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.335+0000 7fb1d6c78700 1 --2- 172.21.15.102:0/1913041664 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb1c006c4e0 0x7fb1c006e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc007c60 tx=0x7fb1bc0073d0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.335+0000 7fb1d6c78700 1 -- 172.21.15.102:0/1913041664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d0100bf0 msgr2=0x7fb1d0074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:05.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.335+0000 7fb1d6c78700 1 --2- 172.21.15.102:0/1913041664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d0100bf0 0x7fb1d0074b30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400bda0 tx=0x7fb1c400bdd0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.335+0000 7fb1d6c78700 1 -- 172.21.15.102:0/1913041664 shutdown_connections 2024-05-13T22:47:05.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.335+0000 7fb1d6c78700 1 --2- 172.21.15.102:0/1913041664 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fb1c006c4e0 0x7fb1c006e9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.335+0000 7fb1d6c78700 1 --2- 172.21.15.102:0/1913041664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1d0100bf0 0x7fb1d0074b30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.335+0000 7fb1d6c78700 1 --2- 172.21.15.102:0/1913041664 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb1d0101df0 0x7fb1d0073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.335+0000 7fb1d6c78700 1 -- 172.21.15.102:0/1913041664 >> 172.21.15.102:0/1913041664 conn(0x7fb1d00fc190 msgr2=0x7fb1d0105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:05.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.335+0000 7fb1d6c78700 1 -- 172.21.15.102:0/1913041664 shutdown_connections 2024-05-13T22:47:05.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.335+0000 7fb1d6c78700 1 -- 172.21.15.102:0/1913041664 wait complete. 2024-05-13T22:47:05.326 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:47:05.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.466+0000 7fd3c8bef700 1 -- 172.21.15.102:0/3200055479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40ffd60 msgr2=0x7fd3c41001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:05.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.466+0000 7fd3c8bef700 1 --2- 172.21.15.102:0/3200055479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40ffd60 0x7fd3c41001c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009910 tx=0x7fd3b4009c20 comp rx=0 tx=0).stop 2024-05-13T22:47:05.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.466+0000 7fd3c8bef700 1 -- 172.21.15.102:0/3200055479 shutdown_connections 2024-05-13T22:47:05.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.466+0000 7fd3c8bef700 1 --2- 172.21.15.102:0/3200055479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40ffd60 0x7fd3c41001c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.466+0000 7fd3c8bef700 1 --2- 172.21.15.102:0/3200055479 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3c4101020 0x7fd3c4101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.466+0000 7fd3c8bef700 1 -- 172.21.15.102:0/3200055479 >> 172.21.15.102:0/3200055479 conn(0x7fd3c40fa790 msgr2=0x7fd3c40fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:05.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.466+0000 7fd3c8bef700 1 -- 172.21.15.102:0/3200055479 shutdown_connections 2024-05-13T22:47:05.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.466+0000 7fd3c8bef700 1 -- 172.21.15.102:0/3200055479 wait complete. 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.467+0000 7fd3c8bef700 1 Processor -- start 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.467+0000 7fd3c8bef700 1 -- start start 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.467+0000 7fd3c8bef700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40ffd60 0x7fd3c40ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.467+0000 7fd3c8bef700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3c4101020 0x7fd3c419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.467+0000 7fd3c8bef700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c4196f00 con 0x7fd3c40ffd60 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.467+0000 7fd3c8bef700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c419d9d0 con 0x7fd3c4101020 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.467+0000 7fd3c259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40ffd60 0x7fd3c40ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.467+0000 7fd3c259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40ffd60 0x7fd3c40ff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44570/0 (socket says 172.21.15.102:44570) 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.467+0000 7fd3c259c700 1 -- 172.21.15.102:0/2018887705 learned_addr learned my addr 172.21.15.102:0/2018887705 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.468+0000 7fd3c1d9b700 1 --2- 172.21.15.102:0/2018887705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3c4101020 0x7fd3c419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.468+0000 7fd3c259c700 1 -- 172.21.15.102:0/2018887705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3c4101020 msgr2=0x7fd3c419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.468+0000 7fd3c259c700 1 --2- 172.21.15.102:0/2018887705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3c4101020 0x7fd3c419d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.468+0000 7fd3c259c700 1 -- 172.21.15.102:0/2018887705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b40095f0 con 0x7fd3c40ffd60 2024-05-13T22:47:05.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.468+0000 7fd3c259c700 1 --2- 172.21.15.102:0/2018887705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40ffd60 0x7fd3c40ff7b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800e920 tx=0x7fd3b800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:05.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.468+0000 7fd3b37fe700 1 -- 172.21.15.102:0/2018887705 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3b8009790 con 0x7fd3c40ffd60 2024-05-13T22:47:05.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.468+0000 7fd3c8bef700 1 -- 172.21.15.102:0/2018887705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c419dc80 con 0x7fd3c40ffd60 2024-05-13T22:47:05.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.468+0000 7fd3c8bef700 1 -- 172.21.15.102:0/2018887705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c419e1a0 con 0x7fd3c40ffd60 2024-05-13T22:47:05.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.468+0000 7fd3b37fe700 1 -- 172.21.15.102:0/2018887705 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd3b800ce80 con 0x7fd3c40ffd60 2024-05-13T22:47:05.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.468+0000 7fd3b37fe700 1 -- 172.21.15.102:0/2018887705 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3b8003d60 con 0x7fd3c40ffd60 2024-05-13T22:47:05.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.470+0000 7fd3c8bef700 1 -- 172.21.15.102:0/2018887705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a4005320 con 0x7fd3c40ffd60 2024-05-13T22:47:05.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.470+0000 7fd3b37fe700 1 -- 172.21.15.102:0/2018887705 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd3b800c9d0 con 0x7fd3c40ffd60 2024-05-13T22:47:05.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.471+0000 7fd3b37fe700 1 --2- 172.21.15.102:0/2018887705 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd3ac06c4e0 0x7fd3ac06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:05.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.471+0000 7fd3b37fe700 1 -- 172.21.15.102:0/2018887705 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd3b8014070 con 0x7fd3c40ffd60 2024-05-13T22:47:05.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.471+0000 7fd3c1d9b700 1 --2- 172.21.15.102:0/2018887705 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd3ac06c4e0 0x7fd3ac06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:05.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.472+0000 7fd3c1d9b700 1 --2- 172.21.15.102:0/2018887705 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd3ac06c4e0 0x7fd3ac06e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009fd0 tx=0x7fd3b4005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:05.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.477+0000 7fd3b37fe700 1 -- 172.21.15.102:0/2018887705 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd3b8059cd0 con 0x7fd3c40ffd60 2024-05-13T22:47:05.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.768+0000 7fd3c8bef700 1 -- 172.21.15.102:0/2018887705 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3a4000bf0 con 0x7fd3ac06c4e0 2024-05-13T22:47:05.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.769+0000 7fd3b37fe700 1 -- 172.21.15.102:0/2018887705 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fd3a4000bf0 con 0x7fd3ac06c4e0 2024-05-13T22:47:05.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.772+0000 7fd3c8bef700 1 -- 172.21.15.102:0/2018887705 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd3ac06c4e0 msgr2=0x7fd3ac06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:05.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.772+0000 7fd3c8bef700 1 --2- 172.21.15.102:0/2018887705 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd3ac06c4e0 0x7fd3ac06e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009fd0 tx=0x7fd3b4005940 comp rx=0 tx=0).stop 2024-05-13T22:47:05.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.772+0000 7fd3c8bef700 1 -- 172.21.15.102:0/2018887705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40ffd60 msgr2=0x7fd3c40ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:05.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.772+0000 7fd3c8bef700 1 --2- 172.21.15.102:0/2018887705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40ffd60 0x7fd3c40ff7b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800e920 tx=0x7fd3b800ec30 comp rx=0 tx=0).stop 2024-05-13T22:47:05.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.772+0000 7fd3c8bef700 1 -- 172.21.15.102:0/2018887705 shutdown_connections 2024-05-13T22:47:05.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.772+0000 7fd3c8bef700 1 --2- 172.21.15.102:0/2018887705 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fd3ac06c4e0 0x7fd3ac06e9a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.772+0000 7fd3c8bef700 1 --2- 172.21.15.102:0/2018887705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3c40ffd60 0x7fd3c40ff7b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.772+0000 7fd3c8bef700 1 --2- 172.21.15.102:0/2018887705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3c4101020 0x7fd3c419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.772+0000 7fd3c8bef700 1 -- 172.21.15.102:0/2018887705 >> 172.21.15.102:0/2018887705 conn(0x7fd3c40fa790 msgr2=0x7fd3c4103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:05.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.772+0000 7fd3c8bef700 1 -- 172.21.15.102:0/2018887705 shutdown_connections 2024-05-13T22:47:05.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.772+0000 7fd3c8bef700 1 -- 172.21.15.102:0/2018887705 wait complete. 2024-05-13T22:47:05.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.900+0000 7fc247378700 1 -- 172.21.15.102:0/1881706549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2400ff490 msgr2=0x7fc2400ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:05.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.900+0000 7fc247378700 1 --2- 172.21.15.102:0/1881706549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2400ff490 0x7fc2400ff8b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009910 tx=0x7fc23c009c20 comp rx=0 tx=0).stop 2024-05-13T22:47:05.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.900+0000 7fc247378700 1 -- 172.21.15.102:0/1881706549 shutdown_connections 2024-05-13T22:47:05.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.900+0000 7fc247378700 1 --2- 172.21.15.102:0/1881706549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2400ffdf0 0x7fc240100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.900+0000 7fc247378700 1 --2- 172.21.15.102:0/1881706549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2400ff490 0x7fc2400ff8b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.900+0000 7fc247378700 1 -- 172.21.15.102:0/1881706549 >> 172.21.15.102:0/1881706549 conn(0x7fc2400fb030 msgr2=0x7fc2400fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:05.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.900+0000 7fc247378700 1 -- 172.21.15.102:0/1881706549 shutdown_connections 2024-05-13T22:47:05.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.900+0000 7fc247378700 1 -- 172.21.15.102:0/1881706549 wait complete. 2024-05-13T22:47:05.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.901+0000 7fc247378700 1 Processor -- start 2024-05-13T22:47:05.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.901+0000 7fc247378700 1 -- start start 2024-05-13T22:47:05.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.901+0000 7fc247378700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2400ff490 0x7fc240198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:05.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.901+0000 7fc247378700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2400ffdf0 0x7fc240198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:05.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.901+0000 7fc247378700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2401990b0 con 0x7fc2400ffdf0 2024-05-13T22:47:05.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.901+0000 7fc247378700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2401991f0 con 0x7fc2400ff490 2024-05-13T22:47:05.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.901+0000 7fc244913700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2400ffdf0 0x7fc240198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.901+0000 7fc244913700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2400ffdf0 0x7fc240198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44588/0 (socket says 172.21.15.102:44588) 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.901+0000 7fc245114700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2400ff490 0x7fc240198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.901+0000 7fc244913700 1 -- 172.21.15.102:0/3554460329 learned_addr learned my addr 172.21.15.102:0/3554460329 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.902+0000 7fc244913700 1 -- 172.21.15.102:0/3554460329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2400ff490 msgr2=0x7fc240198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.902+0000 7fc244913700 1 --2- 172.21.15.102:0/3554460329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2400ff490 0x7fc240198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.902+0000 7fc244913700 1 -- 172.21.15.102:0/3554460329 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c0095f0 con 0x7fc2400ffdf0 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.902+0000 7fc244913700 1 --2- 172.21.15.102:0/3554460329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2400ffdf0 0x7fc240198a90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc23000c8f0 tx=0x7fc23000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.902+0000 7fc2367fc700 1 -- 172.21.15.102:0/3554460329 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc230004a00 con 0x7fc2400ffdf0 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.902+0000 7fc247378700 1 -- 172.21.15.102:0/3554460329 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24019dca0 con 0x7fc2400ffdf0 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.902+0000 7fc2367fc700 1 -- 172.21.15.102:0/3554460329 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc2300054d0 con 0x7fc2400ffdf0 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.902+0000 7fc2367fc700 1 -- 172.21.15.102:0/3554460329 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc230007ce0 con 0x7fc2400ffdf0 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.902+0000 7fc247378700 1 -- 172.21.15.102:0/3554460329 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24019e1f0 con 0x7fc2400ffdf0 2024-05-13T22:47:05.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.903+0000 7fc247378700 1 -- 172.21.15.102:0/3554460329 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24010b930 con 0x7fc2400ffdf0 2024-05-13T22:47:05.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.904+0000 7fc2367fc700 1 -- 172.21.15.102:0/3554460329 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc230005020 con 0x7fc2400ffdf0 2024-05-13T22:47:05.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.904+0000 7fc2367fc700 1 --2- 172.21.15.102:0/3554460329 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc22c06c4e0 0x7fc22c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:05.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.905+0000 7fc2367fc700 1 -- 172.21.15.102:0/3554460329 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc230013070 con 0x7fc2400ffdf0 2024-05-13T22:47:05.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.905+0000 7fc245114700 1 --2- 172.21.15.102:0/3554460329 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc22c06c4e0 0x7fc22c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:05.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.905+0000 7fc245114700 1 --2- 172.21.15.102:0/3554460329 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc22c06c4e0 0x7fc22c06e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b5c0 tx=0x7fc23c0052a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:05.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:05.909+0000 7fc2367fc700 1 -- 172.21.15.102:0/3554460329 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc230059890 con 0x7fc2400ffdf0 2024-05-13T22:47:06.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.174+0000 7fc247378700 1 -- 172.21.15.102:0/3554460329 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2401091e0 con 0x7fc22c06c4e0 2024-05-13T22:47:06.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.185+0000 7fc2367fc700 1 -- 172.21.15.102:0/3554460329 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7fc2401091e0 con 0x7fc22c06c4e0 2024-05-13T22:47:06.163 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (5m) 87s ago 7m 25.9M - 0.25.0 c8568f914cd2 c096e9e9a228 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (7m) 87s ago 7m 7457k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (6m) 93s ago 6m 7289k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (7m) 87s ago 7m 6656k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (6m) 93s ago 6m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (5m) 87s ago 7m 87.4M - 9.4.7 954c08fa6188 bca1790c17aa 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (111s) 87s ago 111s 11.9M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (105s) 87s ago 105s 15.8M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (109s) 93s ago 108s 11.2M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (103s) 93s ago 102s 14.3M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:9283,8765,8443 running (8m) 87s ago 8m 488M - 18.2.1 5be31c24972a 77cd1ca053e7 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (6m) 93s ago 6m 440M - 18.2.1 5be31c24972a c8a9d782f394 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (8m) 87s ago 8m 44.5M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:47:06.164 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (6m) 93s ago 6m 37.5M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:47:06.165 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (7m) 87s ago 7m 16.3M - 1.5.0 0da6a335fe13 af297cf52099 2024-05-13T22:47:06.165 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (6m) 93s ago 6m 15.8M - 1.5.0 0da6a335fe13 cc0f64ee7ecd 2024-05-13T22:47:06.165 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (5m) 87s ago 5m 48.0M 3946M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:47:06.165 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (5m) 87s ago 5m 47.7M 3946M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:47:06.165 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (4m) 87s ago 4m 45.7M 3946M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:47:06.165 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (4m) 93s ago 4m 47.0M 4970M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:47:06.165 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (3m) 93s ago 3m 46.6M 4970M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:47:06.165 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (3m) 93s ago 3m 45.8M 4970M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:47:06.165 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (5m) 87s ago 6m 54.0M - 2.43.0 a07b618ecd1d a89752444eb6 2024-05-13T22:47:06.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.188+0000 7fc247378700 1 -- 172.21.15.102:0/3554460329 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc22c06c4e0 msgr2=0x7fc22c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:06.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.188+0000 7fc247378700 1 --2- 172.21.15.102:0/3554460329 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc22c06c4e0 0x7fc22c06e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b5c0 tx=0x7fc23c0052a0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.188+0000 7fc247378700 1 -- 172.21.15.102:0/3554460329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2400ffdf0 msgr2=0x7fc240198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:06.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.188+0000 7fc247378700 1 --2- 172.21.15.102:0/3554460329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2400ffdf0 0x7fc240198a90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc23000c8f0 tx=0x7fc23000cc00 comp rx=0 tx=0).stop 2024-05-13T22:47:06.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.188+0000 7fc247378700 1 -- 172.21.15.102:0/3554460329 shutdown_connections 2024-05-13T22:47:06.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.188+0000 7fc247378700 1 --2- 172.21.15.102:0/3554460329 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7fc22c06c4e0 0x7fc22c06e9a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.188+0000 7fc247378700 1 --2- 172.21.15.102:0/3554460329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2400ff490 0x7fc240198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.188+0000 7fc247378700 1 --2- 172.21.15.102:0/3554460329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2400ffdf0 0x7fc240198a90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.188+0000 7fc247378700 1 -- 172.21.15.102:0/3554460329 >> 172.21.15.102:0/3554460329 conn(0x7fc2400fb030 msgr2=0x7fc240107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:06.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.188+0000 7fc247378700 1 -- 172.21.15.102:0/3554460329 shutdown_connections 2024-05-13T22:47:06.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.188+0000 7fc247378700 1 -- 172.21.15.102:0/3554460329 wait complete. 2024-05-13T22:47:06.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.313+0000 7f33eb857700 1 -- 172.21.15.102:0/2508722419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33e4101dc0 msgr2=0x7f33e4102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:06.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.313+0000 7f33eb857700 1 --2- 172.21.15.102:0/2508722419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33e4101dc0 0x7f33e4102220 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009910 tx=0x7f33d4009c20 comp rx=0 tx=0).stop 2024-05-13T22:47:06.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.314+0000 7f33eb857700 1 -- 172.21.15.102:0/2508722419 shutdown_connections 2024-05-13T22:47:06.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.314+0000 7f33eb857700 1 --2- 172.21.15.102:0/2508722419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33e4101dc0 0x7f33e4102220 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.314+0000 7f33eb857700 1 --2- 172.21.15.102:0/2508722419 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33e4100bc0 0x7f33e4100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.314+0000 7f33eb857700 1 -- 172.21.15.102:0/2508722419 >> 172.21.15.102:0/2508722419 conn(0x7f33e40fc140 msgr2=0x7f33e40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:06.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.314+0000 7f33eb857700 1 -- 172.21.15.102:0/2508722419 shutdown_connections 2024-05-13T22:47:06.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.314+0000 7f33eb857700 1 -- 172.21.15.102:0/2508722419 wait complete. 2024-05-13T22:47:06.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.314+0000 7f33eb857700 1 Processor -- start 2024-05-13T22:47:06.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.314+0000 7f33eb857700 1 -- start start 2024-05-13T22:47:06.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33eb857700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33e4100bc0 0x7f33e4074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:06.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33eb857700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33e4101dc0 0x7f33e4073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:06.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33eb857700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40736c0 con 0x7f33e4100bc0 2024-05-13T22:47:06.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33eb857700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4073800 con 0x7f33e4101dc0 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33e95f3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33e4100bc0 0x7f33e4074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33e95f3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33e4100bc0 0x7f33e4074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44600/0 (socket says 172.21.15.102:44600) 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33e95f3700 1 -- 172.21.15.102:0/2995180494 learned_addr learned my addr 172.21.15.102:0/2995180494 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33e8df2700 1 --2- 172.21.15.102:0/2995180494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33e4101dc0 0x7f33e4073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33e95f3700 1 -- 172.21.15.102:0/2995180494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33e4101dc0 msgr2=0x7f33e4073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33e95f3700 1 --2- 172.21.15.102:0/2995180494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33e4101dc0 0x7f33e4073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33e95f3700 1 -- 172.21.15.102:0/2995180494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d40095f0 con 0x7f33e4100bc0 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.315+0000 7f33e95f3700 1 --2- 172.21.15.102:0/2995180494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33e4100bc0 0x7f33e4074b30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f33e000d900 tx=0x7f33e000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.316+0000 7f33da7fc700 1 -- 172.21.15.102:0/2995180494 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33e000b920 con 0x7f33e4100bc0 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.316+0000 7f33eb857700 1 -- 172.21.15.102:0/2995180494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e4073ae0 con 0x7f33e4100bc0 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.316+0000 7f33da7fc700 1 -- 172.21.15.102:0/2995180494 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f33e0010460 con 0x7f33e4100bc0 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.316+0000 7f33da7fc700 1 -- 172.21.15.102:0/2995180494 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33e000f590 con 0x7f33e4100bc0 2024-05-13T22:47:06.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.316+0000 7f33eb857700 1 -- 172.21.15.102:0/2995180494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e419e3d0 con 0x7f33e4100bc0 2024-05-13T22:47:06.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.317+0000 7f33eb857700 1 -- 172.21.15.102:0/2995180494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e4109720 con 0x7f33e4100bc0 2024-05-13T22:47:06.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.317+0000 7f33da7fc700 1 -- 172.21.15.102:0/2995180494 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f33e00105d0 con 0x7f33e4100bc0 2024-05-13T22:47:06.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.318+0000 7f33da7fc700 1 --2- 172.21.15.102:0/2995180494 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f33d006c4e0 0x7f33d006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:06.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.318+0000 7f33da7fc700 1 -- 172.21.15.102:0/2995180494 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f33e008b500 con 0x7f33e4100bc0 2024-05-13T22:47:06.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.318+0000 7f33e8df2700 1 --2- 172.21.15.102:0/2995180494 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f33d006c4e0 0x7f33d006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:06.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.319+0000 7f33e8df2700 1 --2- 172.21.15.102:0/2995180494 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f33d006c4e0 0x7f33d006e9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f33d400b5c0 tx=0x7f33d4005ee0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:06.302 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.324+0000 7f33da7fc700 1 -- 172.21.15.102:0/2995180494 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f33e0059710 con 0x7f33e4100bc0 2024-05-13T22:47:06.303 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:06 smithi102 ceph-mon[28599]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:06.303 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:06 smithi102 ceph-mon[28599]: pgmap v236: 65 pgs: 65 active+clean; 9.4 MiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 762 KiB/s wr, 19 op/s 2024-05-13T22:47:06.303 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:06 smithi102 ceph-mon[28599]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:06.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:06 smithi130 ceph-mon[36591]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:06.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:06 smithi130 ceph-mon[36591]: pgmap v236: 65 pgs: 65 active+clean; 9.4 MiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 762 KiB/s wr, 19 op/s 2024-05-13T22:47:06.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:06 smithi130 ceph-mon[36591]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:06.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.687+0000 7f33eb857700 1 -- 172.21.15.102:0/2995180494 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f33e419e980 con 0x7f33e4100bc0 2024-05-13T22:47:06.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.688+0000 7f33da7fc700 1 -- 172.21.15.102:0/2995180494 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f33e00592a0 con 0x7f33e4100bc0 2024-05-13T22:47:06.666 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:47:06.666 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:47:06.666 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:47:06.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:47:06.666 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:47:06.666 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:47:06.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:47:06.667 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:47:06.667 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:47:06.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:47:06.667 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:47:06.667 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:47:06.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:47:06.667 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:47:06.667 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-05-13T22:47:06.667 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:47:06.667 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:47:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.690+0000 7f33eb857700 1 -- 172.21.15.102:0/2995180494 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f33d006c4e0 msgr2=0x7f33d006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.690+0000 7f33eb857700 1 --2- 172.21.15.102:0/2995180494 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f33d006c4e0 0x7f33d006e9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f33d400b5c0 tx=0x7f33d4005ee0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.690+0000 7f33eb857700 1 -- 172.21.15.102:0/2995180494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33e4100bc0 msgr2=0x7f33e4074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.690+0000 7f33eb857700 1 --2- 172.21.15.102:0/2995180494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33e4100bc0 0x7f33e4074b30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f33e000d900 tx=0x7f33e000dc10 comp rx=0 tx=0).stop 2024-05-13T22:47:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.691+0000 7f33eb857700 1 -- 172.21.15.102:0/2995180494 shutdown_connections 2024-05-13T22:47:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.691+0000 7f33eb857700 1 --2- 172.21.15.102:0/2995180494 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f33d006c4e0 0x7f33d006e9a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.691+0000 7f33eb857700 1 --2- 172.21.15.102:0/2995180494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33e4100bc0 0x7f33e4074b30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.691+0000 7f33eb857700 1 --2- 172.21.15.102:0/2995180494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33e4101dc0 0x7f33e4073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.691+0000 7f33eb857700 1 -- 172.21.15.102:0/2995180494 >> 172.21.15.102:0/2995180494 conn(0x7f33e40fc140 msgr2=0x7f33e4104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:06.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.691+0000 7f33eb857700 1 -- 172.21.15.102:0/2995180494 shutdown_connections 2024-05-13T22:47:06.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.691+0000 7f33eb857700 1 -- 172.21.15.102:0/2995180494 wait complete. 2024-05-13T22:47:06.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.826+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2016152723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54a4101dc0 msgr2=0x7f54a4102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.826+0000 7f54ac5c2700 1 --2- 172.21.15.102:0/2016152723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54a4101dc0 0x7f54a4102220 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5494009910 tx=0x7f5494009c20 comp rx=0 tx=0).stop 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.826+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2016152723 shutdown_connections 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.826+0000 7f54ac5c2700 1 --2- 172.21.15.102:0/2016152723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54a4101dc0 0x7f54a4102220 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.826+0000 7f54ac5c2700 1 --2- 172.21.15.102:0/2016152723 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a4100bc0 0x7f54a4100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.826+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2016152723 >> 172.21.15.102:0/2016152723 conn(0x7f54a40fc140 msgr2=0x7f54a40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.826+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2016152723 shutdown_connections 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.826+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2016152723 wait complete. 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54ac5c2700 1 Processor -- start 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54ac5c2700 1 -- start start 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54ac5c2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a4100bc0 0x7f54a4196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54ac5c2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54a4101dc0 0x7f54a41968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:06.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54ac5c2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4196ed0 con 0x7f54a4101dc0 2024-05-13T22:47:06.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54ac5c2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4197010 con 0x7f54a4100bc0 2024-05-13T22:47:06.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54a9b5d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54a4101dc0 0x7f54a41968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:06.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54a9b5d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54a4101dc0 0x7f54a41968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44624/0 (socket says 172.21.15.102:44624) 2024-05-13T22:47:06.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54a9b5d700 1 -- 172.21.15.102:0/2793350866 learned_addr learned my addr 172.21.15.102:0/2793350866 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:47:06.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54aa35e700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a4100bc0 0x7f54a4196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:06.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54a9b5d700 1 -- 172.21.15.102:0/2793350866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a4100bc0 msgr2=0x7f54a4196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:06.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54a9b5d700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a4100bc0 0x7f54a4196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:06.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54a9b5d700 1 -- 172.21.15.102:0/2793350866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54940095f0 con 0x7f54a4101dc0 2024-05-13T22:47:06.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.827+0000 7f54aa35e700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a4100bc0 0x7f54a4196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:47:06.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.828+0000 7f54a9b5d700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54a4101dc0 0x7f54a41968b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5494000c00 tx=0x7f549400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:06.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.828+0000 7f549b7fe700 1 -- 172.21.15.102:0/2793350866 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f549401d070 con 0x7f54a4101dc0 2024-05-13T22:47:06.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.828+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2793350866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a419ba60 con 0x7f54a4101dc0 2024-05-13T22:47:06.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.828+0000 7f549b7fe700 1 -- 172.21.15.102:0/2793350866 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5494005dc0 con 0x7f54a4101dc0 2024-05-13T22:47:06.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.828+0000 7f549b7fe700 1 -- 172.21.15.102:0/2793350866 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54940226a0 con 0x7f54a4101dc0 2024-05-13T22:47:06.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.828+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2793350866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a419bf50 con 0x7f54a4101dc0 2024-05-13T22:47:06.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.829+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2793350866 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a4109720 con 0x7f54a4101dc0 2024-05-13T22:47:06.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.830+0000 7f549b7fe700 1 -- 172.21.15.102:0/2793350866 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f5494003680 con 0x7f54a4101dc0 2024-05-13T22:47:06.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.830+0000 7f549b7fe700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f549006c4e0 0x7f549006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:06.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.830+0000 7f549b7fe700 1 -- 172.21.15.102:0/2793350866 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f549408ce80 con 0x7f54a4101dc0 2024-05-13T22:47:06.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.831+0000 7f54aa35e700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f549006c4e0 0x7f549006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:06.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.831+0000 7f54aa35e700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f549006c4e0 0x7f549006e9a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f54a4101c20 tx=0x7f54a0009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:06.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:06.836+0000 7f549b7fe700 1 -- 172.21.15.102:0/2793350866 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f549405b090 con 0x7f54a4101dc0 2024-05-13T22:47:07.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.111+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2793350866 --> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f54a4106710 con 0x7f549006c4e0 2024-05-13T22:47:07.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.113+0000 7f549b7fe700 1 -- 172.21.15.102:0/2793350866 <== mgr.14215 v2:172.21.15.102:6800/2571859579 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f54a4106710 con 0x7f549006c4e0 2024-05-13T22:47:07.091 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:47:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:47:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:47:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-13T22:47:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [], 2024-05-13T22:47:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "0/2 daemons upgraded", 2024-05-13T22:47:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading mgr daemons", 2024-05-13T22:47:07.091 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:47:07.091 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:47:07.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.115+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2793350866 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f549006c4e0 msgr2=0x7f549006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:07.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.115+0000 7f54ac5c2700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f549006c4e0 0x7f549006e9a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f54a4101c20 tx=0x7f54a0009260 comp rx=0 tx=0).stop 2024-05-13T22:47:07.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.115+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2793350866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54a4101dc0 msgr2=0x7f54a41968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:07.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.115+0000 7f54ac5c2700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54a4101dc0 0x7f54a41968b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5494000c00 tx=0x7f549400f740 comp rx=0 tx=0).stop 2024-05-13T22:47:07.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.115+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2793350866 shutdown_connections 2024-05-13T22:47:07.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.115+0000 7f54ac5c2700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.102:6800/2571859579,v1:172.21.15.102:6801/2571859579] conn(0x7f549006c4e0 0x7f549006e9a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:07.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.115+0000 7f54ac5c2700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f54a4100bc0 0x7f54a4196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:07.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.115+0000 7f54ac5c2700 1 --2- 172.21.15.102:0/2793350866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54a4101dc0 0x7f54a41968b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:07.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.115+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2793350866 >> 172.21.15.102:0/2793350866 conn(0x7f54a40fc140 msgr2=0x7f54a4104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:07.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.116+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2793350866 shutdown_connections 2024-05-13T22:47:07.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:07.116+0000 7f54ac5c2700 1 -- 172.21.15.102:0/2793350866 wait complete. 2024-05-13T22:47:07.343 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:07 smithi130 ceph-mon[36591]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:07.344 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:07 smithi130 ceph-mon[36591]: Upgrade: Updating mgr.smithi130.uoqoxl 2024-05-13T22:47:07.344 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:07 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:07.344 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:07 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi130.uoqoxl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:47:07.344 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:07 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:47:07.344 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:07 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:47:07.345 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:07 smithi130 ceph-mon[36591]: Deploying daemon mgr.smithi130.uoqoxl on smithi130 2024-05-13T22:47:07.345 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:07 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2995180494' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:47:07.345 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:07 smithi130 ceph-mon[36591]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:07.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:07 smithi102 ceph-mon[28599]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:07.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:07 smithi102 ceph-mon[28599]: Upgrade: Updating mgr.smithi130.uoqoxl 2024-05-13T22:47:07.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:07 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:07.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:07 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi130.uoqoxl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:47:07.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:07 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:47:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:07 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:47:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:07 smithi102 ceph-mon[28599]: Deploying daemon mgr.smithi130.uoqoxl on smithi130 2024-05-13T22:47:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:07 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2995180494' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:47:07.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:07 smithi102 ceph-mon[28599]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:07.585 INFO:tasks.workunit.client.1.smithi130.stderr:Updating files: 91% (11417/12532) Updating files: 92% (11530/12532) Updating files: 93% (11655/12532) Updating files: 94% (11781/12532) Updating files: 95% (11906/12532) Updating files: 96% (12031/12532) Updating files: 97% (12157/12532) Updating files: 98% (12282/12532) Updating files: 99% (12407/12532) Updating files: 100% (12532/12532) Updating files: 100% (12532/12532), done. 2024-05-13T22:47:08.440 INFO:tasks.workunit.client.1.smithi130.stderr:Note: switching to 'c3e55145f4f4b3a500bb66ee08c938a215bc231d'. 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr: 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr:state without impacting any branches by switching back to a branch. 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr: 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr: 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr: git switch -c 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr: 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr:Or undo this operation with: 2024-05-13T22:47:08.441 INFO:tasks.workunit.client.1.smithi130.stderr: 2024-05-13T22:47:08.442 INFO:tasks.workunit.client.1.smithi130.stderr: git switch - 2024-05-13T22:47:08.442 INFO:tasks.workunit.client.1.smithi130.stderr: 2024-05-13T22:47:08.442 INFO:tasks.workunit.client.1.smithi130.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-13T22:47:08.442 INFO:tasks.workunit.client.1.smithi130.stderr: 2024-05-13T22:47:08.442 INFO:tasks.workunit.client.1.smithi130.stderr:HEAD is now at c3e55145f4f Merge pull request #57426 from zdover23/wip-doc-2024-05-13-backport-57420-to-squid 2024-05-13T22:47:08.445 DEBUG:teuthology.orchestra.run.smithi130:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-05-13T22:47:08.540 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:08 smithi102 ceph-mon[28599]: pgmap v237: 65 pgs: 65 active+clean; 10 MiB data, 187 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 836 KiB/s wr, 34 op/s 2024-05-13T22:47:08.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:08 smithi130 ceph-mon[36591]: pgmap v237: 65 pgs: 65 active+clean; 10 MiB data, 187 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 836 KiB/s wr, 34 op/s 2024-05-13T22:47:08.792 INFO:tasks.workunit.client.1.smithi130.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-13T22:47:08.795 INFO:tasks.workunit.client.1.smithi130.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-13T22:47:08.795 INFO:tasks.workunit.client.1.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-13T22:47:09.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:09 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:47:09.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:09 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:47:10.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:10 smithi130 ceph-mon[36591]: pgmap v238: 65 pgs: 65 active+clean; 18 MiB data, 242 MiB used, 536 GiB / 536 GiB avail; 252 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-05-13T22:47:10.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:10 smithi102 ceph-mon[28599]: pgmap v238: 65 pgs: 65 active+clean; 18 MiB data, 242 MiB used, 536 GiB / 536 GiB avail; 252 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-05-13T22:47:13.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:12 smithi130 ceph-mon[36591]: pgmap v239: 65 pgs: 65 active+clean; 21 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 252 KiB/s rd, 1.8 MiB/s wr, 150 op/s 2024-05-13T22:47:13.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:12 smithi102 ceph-mon[28599]: pgmap v239: 65 pgs: 65 active+clean; 21 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 252 KiB/s rd, 1.8 MiB/s wr, 150 op/s 2024-05-13T22:47:15.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:14 smithi130 ceph-mon[36591]: pgmap v240: 65 pgs: 65 active+clean; 30 MiB data, 293 MiB used, 536 GiB / 536 GiB avail; 652 KiB/s rd, 2.6 MiB/s wr, 234 op/s 2024-05-13T22:47:15.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:14 smithi102 ceph-mon[28599]: pgmap v240: 65 pgs: 65 active+clean; 30 MiB data, 293 MiB used, 536 GiB / 536 GiB avail; 652 KiB/s rd, 2.6 MiB/s wr, 234 op/s 2024-05-13T22:47:15.796 INFO:tasks.workunit.client.1.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-13T22:47:16.324 INFO:tasks.workunit.client.1.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-13T22:47:16.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:16 smithi130 ceph-mon[36591]: pgmap v241: 65 pgs: 65 active+clean; 35 MiB data, 315 MiB used, 536 GiB / 536 GiB avail; 651 KiB/s rd, 3.0 MiB/s wr, 287 op/s 2024-05-13T22:47:16.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:16 smithi102 ceph-mon[28599]: pgmap v241: 65 pgs: 65 active+clean; 35 MiB data, 315 MiB used, 536 GiB / 536 GiB avail; 651 KiB/s rd, 3.0 MiB/s wr, 287 op/s 2024-05-13T22:47:16.624 INFO:tasks.workunit.client.1.smithi130.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-13T22:47:16.626 INFO:tasks.workunit.client.1.smithi130.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-13T22:47:16.626 INFO:tasks.workunit.client.1.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-13T22:47:16.952 INFO:tasks.workunit.client.1.smithi130.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-13T22:47:16.956 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-05-13T22:47:16.956 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-05-13T22:47:16.972 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-05-13T22:47:16.973 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-13T22:47:16.973 DEBUG:teuthology.orchestra.run.smithi130:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c3e55145f4f4b3a500bb66ee08c938a215bc231d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-05-13T22:47:17.037 INFO:tasks.workunit.client.1.smithi130.stderr:+ mkdir -p fsstress 2024-05-13T22:47:17.040 INFO:tasks.workunit.client.1.smithi130.stderr:+ pushd fsstress 2024-05-13T22:47:17.041 INFO:tasks.workunit.client.1.smithi130.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-13T22:47:17.041 INFO:tasks.workunit.client.1.smithi130.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-13T22:47:18.355 INFO:tasks.workunit.client.1.smithi130.stderr:+ tar xzf ltp-full.tgz 2024-05-13T22:47:18.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:18 smithi102 ceph-mon[28599]: pgmap v242: 65 pgs: 65 active+clean; 36 MiB data, 324 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 2.4 MiB/s wr, 292 op/s 2024-05-13T22:47:19.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:18 smithi130 ceph-mon[36591]: pgmap v242: 65 pgs: 65 active+clean; 36 MiB data, 324 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 2.4 MiB/s wr, 292 op/s 2024-05-13T22:47:20.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:20 smithi130 ceph-mon[36591]: pgmap v243: 65 pgs: 65 active+clean; 50 MiB data, 385 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 357 op/s 2024-05-13T22:47:21.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:20 smithi102 ceph-mon[28599]: pgmap v243: 65 pgs: 65 active+clean; 50 MiB data, 385 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 357 op/s 2024-05-13T22:47:23.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:22 smithi130 ceph-mon[36591]: pgmap v244: 65 pgs: 65 active+clean; 55 MiB data, 401 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-05-13T22:47:23.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:22 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:23.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:22 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:23.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:22 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:47:23.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:22 smithi102 ceph-mon[28599]: pgmap v244: 65 pgs: 65 active+clean; 55 MiB data, 401 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-05-13T22:47:23.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:22 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:23.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:22 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:23.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:22 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:47:23.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:23 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:47:24.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:23 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:47:24.841 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:24 smithi130 ceph-mon[36591]: pgmap v245: 65 pgs: 65 active+clean; 73 MiB data, 488 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 398 op/s 2024-05-13T22:47:25.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:24 smithi102 ceph-mon[28599]: pgmap v245: 65 pgs: 65 active+clean; 73 MiB data, 488 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 398 op/s 2024-05-13T22:47:27.289 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:26 smithi130 ceph-mon[36591]: pgmap v246: 65 pgs: 65 active+clean; 75 MiB data, 517 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 400 op/s 2024-05-13T22:47:27.290 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:26 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:27.290 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:26 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:27.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:26 smithi102 ceph-mon[28599]: pgmap v246: 65 pgs: 65 active+clean; 75 MiB data, 517 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 400 op/s 2024-05-13T22:47:27.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:26 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:27.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:26 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:28.057 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:28 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:28.057 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:28 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:28.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:28 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:28.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:28 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:29.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:29 smithi102 ceph-mon[28599]: pgmap v247: 65 pgs: 65 active+clean; 76 MiB data, 512 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 374 op/s 2024-05-13T22:47:29.290 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:29 smithi130 ceph-mon[36591]: pgmap v247: 65 pgs: 65 active+clean; 76 MiB data, 512 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 374 op/s 2024-05-13T22:47:30.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:30 smithi130 ceph-mon[36591]: Standby manager daemon smithi130.uoqoxl restarted 2024-05-13T22:47:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:30 smithi130 ceph-mon[36591]: Standby manager daemon smithi130.uoqoxl started 2024-05-13T22:47:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:30 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/crt"}]: dispatch 2024-05-13T22:47:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:30 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:47:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:30 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/key"}]: dispatch 2024-05-13T22:47:30.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:30 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:47:30.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:30 smithi102 ceph-mon[28599]: Standby manager daemon smithi130.uoqoxl restarted 2024-05-13T22:47:30.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:30 smithi102 ceph-mon[28599]: Standby manager daemon smithi130.uoqoxl started 2024-05-13T22:47:30.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:30 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/crt"}]: dispatch 2024-05-13T22:47:30.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:30 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:47:30.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:30 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/key"}]: dispatch 2024-05-13T22:47:30.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:30 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:47:31.047 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:31 smithi130 ceph-mon[36591]: pgmap v248: 65 pgs: 65 active+clean; 91 MiB data, 507 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 457 op/s 2024-05-13T22:47:31.047 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:31 smithi130 ceph-mon[36591]: mgrmap e20: smithi102.xcijrj(active, since 7m), standbys: smithi130.uoqoxl 2024-05-13T22:47:31.047 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:31 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:31.047 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:31 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:31.047 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:31 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:47:31.047 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:31 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:47:31.047 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:31 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:31.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:31 smithi102 ceph-mon[28599]: pgmap v248: 65 pgs: 65 active+clean; 91 MiB data, 507 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 457 op/s 2024-05-13T22:47:31.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:31 smithi102 ceph-mon[28599]: mgrmap e20: smithi102.xcijrj(active, since 7m), standbys: smithi130.uoqoxl 2024-05-13T22:47:31.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:31 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:31.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:31 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:31.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:31 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:47:31.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:31 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:47:31.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:31 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' 2024-05-13T22:47:32.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:47:32.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr fail", "who": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:47:32.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:47:32.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:47:32.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:47:32.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:47:32.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:47:32.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:47:32.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mgr metadata", "who": "smithi130.uoqoxl", "id": "smithi130.uoqoxl"}]: dispatch 2024-05-13T22:47:32.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:47:32.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: osdmap e48: 6 total, 6 up, 6 in 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "mgr fail", "who": "smithi102.xcijrj"}]': finished 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: mgrmap e21: smithi130.uoqoxl(active, starting, since 0.00887273s) 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: Manager daemon smithi130.uoqoxl is now available 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:47:32.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:47:32.404 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi130.uoqoxl/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:47:32.404 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi130.uoqoxl/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:47:32.404 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi130.uoqoxl/trash_purge_schedule"}]: dispatch 2024-05-13T22:47:32.404 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi130.uoqoxl/trash_purge_schedule"}]: dispatch 2024-05-13T22:47:32.404 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: [13/May/2024:22:47:31] ENGINE Bus STARTING 2024-05-13T22:47:32.404 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: Deploying cephadm binary to smithi130 2024-05-13T22:47:32.404 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: [13/May/2024:22:47:31] ENGINE Serving on https://172.21.15.130:7150 2024-05-13T22:47:32.404 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:32 smithi130 ceph-mon[36591]: Deploying cephadm binary to smithi102 2024-05-13T22:47:32.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:47:32.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr fail", "who": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:47:32.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:47:32.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:47:32.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:47:32.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:47:32.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mgr metadata", "who": "smithi130.uoqoxl", "id": "smithi130.uoqoxl"}]: dispatch 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: osdmap e48: 6 total, 6 up, 6 in 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.14215 172.21.15.102:0/3521801769' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "mgr fail", "who": "smithi102.xcijrj"}]': finished 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: mgrmap e21: smithi130.uoqoxl(active, starting, since 0.00887273s) 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-13T22:47:32.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-13T22:47:32.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-13T22:47:32.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: Manager daemon smithi130.uoqoxl is now available 2024-05-13T22:47:32.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:47:32.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:47:32.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi130.uoqoxl/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:47:32.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi130.uoqoxl/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:47:32.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi130.uoqoxl/trash_purge_schedule"}]: dispatch 2024-05-13T22:47:32.431 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi130.uoqoxl/trash_purge_schedule"}]: dispatch 2024-05-13T22:47:32.431 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: [13/May/2024:22:47:31] ENGINE Bus STARTING 2024-05-13T22:47:32.431 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: Deploying cephadm binary to smithi130 2024-05-13T22:47:32.431 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: [13/May/2024:22:47:31] ENGINE Serving on https://172.21.15.130:7150 2024-05-13T22:47:32.431 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:32 smithi102 ceph-mon[28599]: Deploying cephadm binary to smithi102 2024-05-13T22:47:33.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:33 smithi130 ceph-mon[36591]: [13/May/2024:22:47:31] ENGINE Serving on http://172.21.15.130:8765 2024-05-13T22:47:33.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:33 smithi130 ceph-mon[36591]: [13/May/2024:22:47:31] ENGINE Bus STARTED 2024-05-13T22:47:33.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:33 smithi130 ceph-mon[36591]: mgrmap e22: smithi130.uoqoxl(active, since 1.00926s) 2024-05-13T22:47:33.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:33 smithi130 ceph-mon[36591]: pgmap v3: 65 pgs: 65 active+clean; 109 MiB data, 588 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:47:33.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:33 smithi102 ceph-mon[28599]: [13/May/2024:22:47:31] ENGINE Serving on http://172.21.15.130:8765 2024-05-13T22:47:33.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:33 smithi102 ceph-mon[28599]: [13/May/2024:22:47:31] ENGINE Bus STARTED 2024-05-13T22:47:33.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:33 smithi102 ceph-mon[28599]: mgrmap e22: smithi130.uoqoxl(active, since 1.00926s) 2024-05-13T22:47:33.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:33 smithi102 ceph-mon[28599]: pgmap v3: 65 pgs: 65 active+clean; 109 MiB data, 588 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:47:34.174 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:34 smithi102 ceph-mon[28599]: pgmap v4: 65 pgs: 65 active+clean; 109 MiB data, 588 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:47:34.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:34 smithi130 ceph-mon[36591]: pgmap v4: 65 pgs: 65 active+clean; 109 MiB data, 588 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:47:35.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:35 smithi102 ceph-mon[28599]: mgrmap e23: smithi130.uoqoxl(active, since 3s) 2024-05-13T22:47:35.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:35 smithi130 ceph-mon[36591]: mgrmap e23: smithi130.uoqoxl(active, since 3s) 2024-05-13T22:47:36.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:36 smithi130 ceph-mon[36591]: pgmap v5: 65 pgs: 65 active+clean; 109 MiB data, 588 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:47:36.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:36 smithi102 ceph-mon[28599]: pgmap v5: 65 pgs: 65 active+clean; 109 MiB data, 588 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:47:37.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.286+0000 7f346052f700 1 -- 172.21.15.102:0/3674565576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34580feda0 msgr2=0x7f34580ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:37.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.286+0000 7f346052f700 1 --2- 172.21.15.102:0/3674565576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34580feda0 0x7f34580ff1c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f345000b0a0 tx=0x7f345000b3b0 comp rx=0 tx=0).stop 2024-05-13T22:47:37.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.286+0000 7f346052f700 1 -- 172.21.15.102:0/3674565576 shutdown_connections 2024-05-13T22:47:37.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.286+0000 7f346052f700 1 --2- 172.21.15.102:0/3674565576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3458101d40 0x7f34581021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:37.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.286+0000 7f346052f700 1 --2- 172.21.15.102:0/3674565576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34580feda0 0x7f34580ff1c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:37.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.286+0000 7f346052f700 1 -- 172.21.15.102:0/3674565576 >> 172.21.15.102:0/3674565576 conn(0x7f34580fa7c0 msgr2=0x7f34580fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:37.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.286+0000 7f346052f700 1 -- 172.21.15.102:0/3674565576 shutdown_connections 2024-05-13T22:47:37.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.286+0000 7f346052f700 1 -- 172.21.15.102:0/3674565576 wait complete. 2024-05-13T22:47:37.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.287+0000 7f346052f700 1 Processor -- start 2024-05-13T22:47:37.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.287+0000 7f346052f700 1 -- start start 2024-05-13T22:47:37.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.287+0000 7f346052f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34580feda0 0x7f3458198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:37.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.287+0000 7f346052f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3458101d40 0x7f3458198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:37.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.287+0000 7f346052f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3458199190 con 0x7f3458101d40 2024-05-13T22:47:37.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.287+0000 7f346052f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34581992d0 con 0x7f34580feda0 2024-05-13T22:47:37.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.288+0000 7f345daca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3458101d40 0x7f3458198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:37.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.288+0000 7f345daca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3458101d40 0x7f3458198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51886/0 (socket says 172.21.15.102:51886) 2024-05-13T22:47:37.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.288+0000 7f345daca700 1 -- 172.21.15.102:0/1621214296 learned_addr learned my addr 172.21.15.102:0/1621214296 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:47:37.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.288+0000 7f345e2cb700 1 --2- 172.21.15.102:0/1621214296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34580feda0 0x7f3458198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:37.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.289+0000 7f345daca700 1 -- 172.21.15.102:0/1621214296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34580feda0 msgr2=0x7f3458198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:37.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.289+0000 7f345daca700 1 --2- 172.21.15.102:0/1621214296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34580feda0 0x7f3458198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:37.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.289+0000 7f345daca700 1 -- 172.21.15.102:0/1621214296 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3450009d00 con 0x7f3458101d40 2024-05-13T22:47:37.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.289+0000 7f345daca700 1 --2- 172.21.15.102:0/1621214296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3458101d40 0x7f3458198b70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f345400e920 tx=0x7f345400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:37.271 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.290+0000 7f344f7fe700 1 -- 172.21.15.102:0/1621214296 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f345400c9d0 con 0x7f3458101d40 2024-05-13T22:47:37.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.290+0000 7f344f7fe700 1 -- 172.21.15.102:0/1621214296 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3454010430 con 0x7f3458101d40 2024-05-13T22:47:37.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.290+0000 7f344f7fe700 1 -- 172.21.15.102:0/1621214296 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3454018650 con 0x7f3458101d40 2024-05-13T22:47:37.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.290+0000 7f346052f700 1 -- 172.21.15.102:0/1621214296 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345806a950 con 0x7f3458101d40 2024-05-13T22:47:37.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.290+0000 7f346052f700 1 -- 172.21.15.102:0/1621214296 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345806aea0 con 0x7f3458101d40 2024-05-13T22:47:37.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.291+0000 7f346052f700 1 -- 172.21.15.102:0/1621214296 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3458192740 con 0x7f3458101d40 2024-05-13T22:47:37.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.292+0000 7f344f7fe700 1 -- 172.21.15.102:0/1621214296 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 23) v1 ==== 46428+0+0 (secure 0 0 0) 0x7f34540105a0 con 0x7f3458101d40 2024-05-13T22:47:37.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.292+0000 7f344f7fe700 1 --2- 172.21.15.102:0/1621214296 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f34440393b0 0x7f344403b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:37.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.292+0000 7f344f7fe700 1 -- 172.21.15.102:0/1621214296 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f3454014070 con 0x7f3458101d40 2024-05-13T22:47:37.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.293+0000 7f345e2cb700 1 --2- 172.21.15.102:0/1621214296 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f34440393b0 0x7f344403b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:37.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.295+0000 7f345e2cb700 1 --2- 172.21.15.102:0/1621214296 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f34440393b0 0x7f344403b870 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3450015040 tx=0x7f3450000f40 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:37.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.302+0000 7f344f7fe700 1 -- 172.21.15.102:0/1621214296 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f3454018c00 con 0x7f3458101d40 2024-05-13T22:47:37.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.681+0000 7f346052f700 1 -- 172.21.15.102:0/1621214296 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34580611d0 con 0x7f34440393b0 2024-05-13T22:47:37.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.683+0000 7f344f7fe700 1 -- 172.21.15.102:0/1621214296 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f34580611d0 con 0x7f34440393b0 2024-05-13T22:47:37.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.687+0000 7f346052f700 1 -- 172.21.15.102:0/1621214296 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f34440393b0 msgr2=0x7f344403b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:37.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.687+0000 7f346052f700 1 --2- 172.21.15.102:0/1621214296 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f34440393b0 0x7f344403b870 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3450015040 tx=0x7f3450000f40 comp rx=0 tx=0).stop 2024-05-13T22:47:37.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.687+0000 7f346052f700 1 -- 172.21.15.102:0/1621214296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3458101d40 msgr2=0x7f3458198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:37.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.687+0000 7f346052f700 1 --2- 172.21.15.102:0/1621214296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3458101d40 0x7f3458198b70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f345400e920 tx=0x7f345400ec30 comp rx=0 tx=0).stop 2024-05-13T22:47:37.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.687+0000 7f346052f700 1 -- 172.21.15.102:0/1621214296 shutdown_connections 2024-05-13T22:47:37.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.687+0000 7f346052f700 1 --2- 172.21.15.102:0/1621214296 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f34440393b0 0x7f344403b870 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:37.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.687+0000 7f346052f700 1 --2- 172.21.15.102:0/1621214296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34580feda0 0x7f3458198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:37.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.687+0000 7f346052f700 1 --2- 172.21.15.102:0/1621214296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3458101d40 0x7f3458198b70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:37.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.687+0000 7f346052f700 1 -- 172.21.15.102:0/1621214296 >> 172.21.15.102:0/1621214296 conn(0x7f34580fa7c0 msgr2=0x7f34580fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:37.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.687+0000 7f346052f700 1 -- 172.21.15.102:0/1621214296 shutdown_connections 2024-05-13T22:47:37.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.687+0000 7f346052f700 1 -- 172.21.15.102:0/1621214296 wait complete. 2024-05-13T22:47:37.685 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:47:37.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.873+0000 7ff7a8689700 1 -- 172.21.15.102:0/1399552263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a0074dc0 msgr2=0x7ff7a0073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:37.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.873+0000 7ff7a8689700 1 --2- 172.21.15.102:0/1399552263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a0074dc0 0x7ff7a0073220 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009910 tx=0x7ff79c009c20 comp rx=0 tx=0).stop 2024-05-13T22:47:37.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.874+0000 7ff7a8689700 1 -- 172.21.15.102:0/1399552263 shutdown_connections 2024-05-13T22:47:37.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.874+0000 7ff7a8689700 1 --2- 172.21.15.102:0/1399552263 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7a00737f0 0x7ff7a0073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:37.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.874+0000 7ff7a8689700 1 --2- 172.21.15.102:0/1399552263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a0074dc0 0x7ff7a0073220 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:37.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.874+0000 7ff7a8689700 1 -- 172.21.15.102:0/1399552263 >> 172.21.15.102:0/1399552263 conn(0x7ff7a00fc1b0 msgr2=0x7ff7a00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:37.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.874+0000 7ff7a8689700 1 -- 172.21.15.102:0/1399552263 shutdown_connections 2024-05-13T22:47:37.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.874+0000 7ff7a8689700 1 -- 172.21.15.102:0/1399552263 wait complete. 2024-05-13T22:47:37.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.875+0000 7ff7a8689700 1 Processor -- start 2024-05-13T22:47:37.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.875+0000 7ff7a8689700 1 -- start start 2024-05-13T22:47:37.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.875+0000 7ff7a8689700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a00737f0 0x7ff7a019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:37.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.875+0000 7ff7a8689700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7a0074dc0 0x7ff7a019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:37.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.875+0000 7ff7a8689700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019d500 con 0x7ff7a00737f0 2024-05-13T22:47:37.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.875+0000 7ff7a8689700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019d640 con 0x7ff7a0074dc0 2024-05-13T22:47:37.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.876+0000 7ff7a6425700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a00737f0 0x7ff7a019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:37.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.876+0000 7ff7a6425700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a00737f0 0x7ff7a019c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51916/0 (socket says 172.21.15.102:51916) 2024-05-13T22:47:37.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.876+0000 7ff7a6425700 1 -- 172.21.15.102:0/2794400936 learned_addr learned my addr 172.21.15.102:0/2794400936 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:47:37.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.876+0000 7ff7a5c24700 1 --2- 172.21.15.102:0/2794400936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7a0074dc0 0x7ff7a019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:37.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.876+0000 7ff7a6425700 1 -- 172.21.15.102:0/2794400936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7a0074dc0 msgr2=0x7ff7a019cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:37.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.876+0000 7ff7a6425700 1 --2- 172.21.15.102:0/2794400936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7a0074dc0 0x7ff7a019cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:37.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.876+0000 7ff7a6425700 1 -- 172.21.15.102:0/2794400936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c0095f0 con 0x7ff7a00737f0 2024-05-13T22:47:37.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.876+0000 7ff7a6425700 1 --2- 172.21.15.102:0/2794400936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a00737f0 0x7ff7a019c9a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00f6c0 tx=0x7ff79c00f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:37.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.877+0000 7ff7977fe700 1 -- 172.21.15.102:0/2794400936 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff79c01d070 con 0x7ff7a00737f0 2024-05-13T22:47:37.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.877+0000 7ff7977fe700 1 -- 172.21.15.102:0/2794400936 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff79c005dc0 con 0x7ff7a00737f0 2024-05-13T22:47:37.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.877+0000 7ff7977fe700 1 -- 172.21.15.102:0/2794400936 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff79c017630 con 0x7ff7a00737f0 2024-05-13T22:47:37.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.877+0000 7ff7a8689700 1 -- 172.21.15.102:0/2794400936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a01a2090 con 0x7ff7a00737f0 2024-05-13T22:47:37.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.877+0000 7ff7a8689700 1 -- 172.21.15.102:0/2794400936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a01a2580 con 0x7ff7a00737f0 2024-05-13T22:47:37.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.878+0000 7ff7977fe700 1 -- 172.21.15.102:0/2794400936 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 23) v1 ==== 46428+0+0 (secure 0 0 0) 0x7ff79c00fad0 con 0x7ff7a00737f0 2024-05-13T22:47:37.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.878+0000 7ff7a8689700 1 -- 172.21.15.102:0/2794400936 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a010fd80 con 0x7ff7a00737f0 2024-05-13T22:47:37.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.878+0000 7ff7977fe700 1 --2- 172.21.15.102:0/2794400936 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff78c039360 0x7ff78c03b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:37.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.878+0000 7ff7977fe700 1 -- 172.21.15.102:0/2794400936 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff79c04ea30 con 0x7ff7a00737f0 2024-05-13T22:47:37.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.879+0000 7ff7a5c24700 1 --2- 172.21.15.102:0/2794400936 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff78c039360 0x7ff78c03b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:37.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.879+0000 7ff7a5c24700 1 --2- 172.21.15.102:0/2794400936 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff78c039360 0x7ff78c03b820 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0074af0 tx=0x7ff790009260 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:37.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:37.887+0000 7ff7977fe700 1 -- 172.21.15.102:0/2794400936 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7ff79c0163b0 con 0x7ff7a00737f0 2024-05-13T22:47:38.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.259+0000 7ff7a8689700 1 -- 172.21.15.102:0/2794400936 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7a0103cc0 con 0x7ff78c039360 2024-05-13T22:47:38.237 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:38 smithi102 ceph-mon[28599]: pgmap v6: 65 pgs: 65 active+clean; 109 MiB data, 588 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:47:38.238 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:38 smithi102 ceph-mon[28599]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:38.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.261+0000 7ff7977fe700 1 -- 172.21.15.102:0/2794400936 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7ff7a0103cc0 con 0x7ff78c039360 2024-05-13T22:47:38.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.263+0000 7ff7a8689700 1 -- 172.21.15.102:0/2794400936 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff78c039360 msgr2=0x7ff78c03b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:38.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.263+0000 7ff7a8689700 1 --2- 172.21.15.102:0/2794400936 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff78c039360 0x7ff78c03b820 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0074af0 tx=0x7ff790009260 comp rx=0 tx=0).stop 2024-05-13T22:47:38.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.263+0000 7ff7a8689700 1 -- 172.21.15.102:0/2794400936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a00737f0 msgr2=0x7ff7a019c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:38.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.263+0000 7ff7a8689700 1 --2- 172.21.15.102:0/2794400936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a00737f0 0x7ff7a019c9a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00f6c0 tx=0x7ff79c00f6f0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.263+0000 7ff7a8689700 1 -- 172.21.15.102:0/2794400936 shutdown_connections 2024-05-13T22:47:38.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.263+0000 7ff7a8689700 1 --2- 172.21.15.102:0/2794400936 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff78c039360 0x7ff78c03b820 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.263+0000 7ff7a8689700 1 --2- 172.21.15.102:0/2794400936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7a00737f0 0x7ff7a019c9a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.263+0000 7ff7a8689700 1 --2- 172.21.15.102:0/2794400936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7a0074dc0 0x7ff7a019cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.263+0000 7ff7a8689700 1 -- 172.21.15.102:0/2794400936 >> 172.21.15.102:0/2794400936 conn(0x7ff7a00fc1b0 msgr2=0x7ff7a01025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:38.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.263+0000 7ff7a8689700 1 -- 172.21.15.102:0/2794400936 shutdown_connections 2024-05-13T22:47:38.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.263+0000 7ff7a8689700 1 -- 172.21.15.102:0/2794400936 wait complete. 2024-05-13T22:47:38.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.371+0000 7fdf9307a700 1 -- 172.21.15.102:0/577789242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf8c100cb0 msgr2=0x7fdf8c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:38.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.371+0000 7fdf9307a700 1 --2- 172.21.15.102:0/577789242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf8c100cb0 0x7fdf8c101110 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009910 tx=0x7fdf78009c20 comp rx=0 tx=0).stop 2024-05-13T22:47:38.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.371+0000 7fdf9307a700 1 -- 172.21.15.102:0/577789242 shutdown_connections 2024-05-13T22:47:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.371+0000 7fdf9307a700 1 --2- 172.21.15.102:0/577789242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf8c100cb0 0x7fdf8c101110 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.371+0000 7fdf9307a700 1 --2- 172.21.15.102:0/577789242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf8c0ffab0 0x7fdf8c0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.371+0000 7fdf9307a700 1 -- 172.21.15.102:0/577789242 >> 172.21.15.102:0/577789242 conn(0x7fdf8c0fb050 msgr2=0x7fdf8c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.371+0000 7fdf9307a700 1 -- 172.21.15.102:0/577789242 shutdown_connections 2024-05-13T22:47:38.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.371+0000 7fdf9307a700 1 -- 172.21.15.102:0/577789242 wait complete. 2024-05-13T22:47:38.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf9307a700 1 Processor -- start 2024-05-13T22:47:38.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf9307a700 1 -- start start 2024-05-13T22:47:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf9307a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf8c0ffab0 0x7fdf8c196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf9307a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf8c100cb0 0x7fdf8c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf9307a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c196ed0 con 0x7fdf8c100cb0 2024-05-13T22:47:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf9307a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c197010 con 0x7fdf8c0ffab0 2024-05-13T22:47:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf8bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf8c100cb0 0x7fdf8c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf90e16700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf8c0ffab0 0x7fdf8c196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf8bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf8c100cb0 0x7fdf8c1968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51942/0 (socket says 172.21.15.102:51942) 2024-05-13T22:47:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf8bfff700 1 -- 172.21.15.102:0/3975042453 learned_addr learned my addr 172.21.15.102:0/3975042453 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:47:38.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf90e16700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf8c0ffab0 0x7fdf8c196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.102:47468/0 (socket says 172.21.15.102:47468) 2024-05-13T22:47:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf8bfff700 1 -- 172.21.15.102:0/3975042453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf8c0ffab0 msgr2=0x7fdf8c196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf8bfff700 1 --2- 172.21.15.102:0/3975042453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf8c0ffab0 0x7fdf8c196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf8bfff700 1 -- 172.21.15.102:0/3975042453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf780095f0 con 0x7fdf8c100cb0 2024-05-13T22:47:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.372+0000 7fdf90e16700 1 --2- 172.21.15.102:0/3975042453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf8c0ffab0 0x7fdf8c196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:47:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.373+0000 7fdf8bfff700 1 --2- 172.21.15.102:0/3975042453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf8c100cb0 0x7fdf8c1968b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf78000c00 tx=0x7fdf7800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.373+0000 7fdf89ffb700 1 -- 172.21.15.102:0/3975042453 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf7801d070 con 0x7fdf8c100cb0 2024-05-13T22:47:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.373+0000 7fdf9307a700 1 -- 172.21.15.102:0/3975042453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8c19ba60 con 0x7fdf8c100cb0 2024-05-13T22:47:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.373+0000 7fdf89ffb700 1 -- 172.21.15.102:0/3975042453 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdf78005dc0 con 0x7fdf8c100cb0 2024-05-13T22:47:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.373+0000 7fdf89ffb700 1 -- 172.21.15.102:0/3975042453 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf780226a0 con 0x7fdf8c100cb0 2024-05-13T22:47:38.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.373+0000 7fdf9307a700 1 -- 172.21.15.102:0/3975042453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8c1020a0 con 0x7fdf8c100cb0 2024-05-13T22:47:38.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.374+0000 7fdf9307a700 1 -- 172.21.15.102:0/3975042453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8c108610 con 0x7fdf8c100cb0 2024-05-13T22:47:38.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.374+0000 7fdf89ffb700 1 -- 172.21.15.102:0/3975042453 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 23) v1 ==== 46428+0+0 (secure 0 0 0) 0x7fdf7800faf0 con 0x7fdf8c100cb0 2024-05-13T22:47:38.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.375+0000 7fdf89ffb700 1 --2- 172.21.15.102:0/3975042453 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fdf7c03d770 0x7fdf7c03fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:38.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.375+0000 7fdf89ffb700 1 -- 172.21.15.102:0/3975042453 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fdf7804ea20 con 0x7fdf8c100cb0 2024-05-13T22:47:38.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.375+0000 7fdf90e16700 1 --2- 172.21.15.102:0/3975042453 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fdf7c03d770 0x7fdf7c03fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:38.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.375+0000 7fdf90e16700 1 --2- 172.21.15.102:0/3975042453 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fdf7c03d770 0x7fdf7c03fc30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c100b10 tx=0x7fdf80009260 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:38.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.381+0000 7fdf89ffb700 1 -- 172.21.15.102:0/3975042453 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fdf78022800 con 0x7fdf8c100cb0 2024-05-13T22:47:38.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:38 smithi130 ceph-mon[36591]: pgmap v6: 65 pgs: 65 active+clean; 109 MiB data, 588 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:47:38.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:38 smithi130 ceph-mon[36591]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:38.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.677+0000 7fdf9307a700 1 -- 172.21.15.102:0/3975042453 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf8c0693b0 con 0x7fdf7c03d770 2024-05-13T22:47:38.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.690+0000 7fdf89ffb700 1 -- 172.21.15.102:0/3975042453 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fdf8c0693b0 con 0x7fdf7c03d770 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (6m) 2m ago 7m 25.9M - 0.25.0 c8568f914cd2 c096e9e9a228 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (8m) 2m ago 8m 7457k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (7m) 12s ago 7m 7436k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (8m) 2m ago 8m 6656k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (7m) 12s ago 6m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (6m) 2m ago 7m 87.4M - 9.4.7 954c08fa6188 bca1790c17aa 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (2m) 2m ago 2m 11.9M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (2m) 2m ago 2m 15.8M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (2m) 12s ago 2m 12.4M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (2m) 12s ago 2m 15.4M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:47:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:9283,8765,8443 running (9m) 2m ago 9m 488M - 18.2.1 5be31c24972a 77cd1ca053e7 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (20s) 12s ago 6m 110M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9a28ef996664 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (9m) 2m ago 9m 44.5M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (6m) 12s ago 6m 38.1M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (8m) 2m ago 8m 16.3M - 1.5.0 0da6a335fe13 af297cf52099 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (6m) 12s ago 6m 15.7M - 1.5.0 0da6a335fe13 cc0f64ee7ecd 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (6m) 2m ago 6m 48.0M 3946M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (5m) 2m ago 5m 47.7M 3946M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (5m) 2m ago 5m 45.7M 3946M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (4m) 12s ago 4m 84.3M 4970M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (4m) 12s ago 4m 84.6M 4970M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (3m) 12s ago 3m 78.6M 4970M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:47:38.670 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (6m) 2m ago 7m 54.0M - 2.43.0 a07b618ecd1d a89752444eb6 2024-05-13T22:47:38.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.694+0000 7fdf9307a700 1 -- 172.21.15.102:0/3975042453 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fdf7c03d770 msgr2=0x7fdf7c03fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:38.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.694+0000 7fdf9307a700 1 --2- 172.21.15.102:0/3975042453 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fdf7c03d770 0x7fdf7c03fc30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c100b10 tx=0x7fdf80009260 comp rx=0 tx=0).stop 2024-05-13T22:47:38.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.694+0000 7fdf9307a700 1 -- 172.21.15.102:0/3975042453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf8c100cb0 msgr2=0x7fdf8c1968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:38.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.694+0000 7fdf9307a700 1 --2- 172.21.15.102:0/3975042453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf8c100cb0 0x7fdf8c1968b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf78000c00 tx=0x7fdf7800f690 comp rx=0 tx=0).stop 2024-05-13T22:47:38.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.694+0000 7fdf9307a700 1 -- 172.21.15.102:0/3975042453 shutdown_connections 2024-05-13T22:47:38.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.694+0000 7fdf9307a700 1 --2- 172.21.15.102:0/3975042453 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fdf7c03d770 0x7fdf7c03fc30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.694+0000 7fdf9307a700 1 --2- 172.21.15.102:0/3975042453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf8c0ffab0 0x7fdf8c196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.694+0000 7fdf9307a700 1 --2- 172.21.15.102:0/3975042453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf8c100cb0 0x7fdf8c1968b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.694+0000 7fdf9307a700 1 -- 172.21.15.102:0/3975042453 >> 172.21.15.102:0/3975042453 conn(0x7fdf8c0fb050 msgr2=0x7fdf8c068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:38.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.694+0000 7fdf9307a700 1 -- 172.21.15.102:0/3975042453 shutdown_connections 2024-05-13T22:47:38.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.694+0000 7fdf9307a700 1 -- 172.21.15.102:0/3975042453 wait complete. 2024-05-13T22:47:38.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.837+0000 7ff05f6e2700 1 -- 172.21.15.102:0/1123470930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff058100bc0 msgr2=0x7ff058100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:38.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.837+0000 7ff05f6e2700 1 --2- 172.21.15.102:0/1123470930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff058100bc0 0x7ff058100fe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff054009910 tx=0x7ff054009c20 comp rx=0 tx=0).stop 2024-05-13T22:47:38.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.837+0000 7ff05f6e2700 1 -- 172.21.15.102:0/1123470930 shutdown_connections 2024-05-13T22:47:38.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.837+0000 7ff05f6e2700 1 --2- 172.21.15.102:0/1123470930 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff058101dc0 0x7ff058102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.837+0000 7ff05f6e2700 1 --2- 172.21.15.102:0/1123470930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff058100bc0 0x7ff058100fe0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.837+0000 7ff05f6e2700 1 -- 172.21.15.102:0/1123470930 >> 172.21.15.102:0/1123470930 conn(0x7ff0580fc140 msgr2=0x7ff0580fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:38.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.838+0000 7ff05f6e2700 1 -- 172.21.15.102:0/1123470930 shutdown_connections 2024-05-13T22:47:38.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.838+0000 7ff05f6e2700 1 -- 172.21.15.102:0/1123470930 wait complete. 2024-05-13T22:47:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.838+0000 7ff05f6e2700 1 Processor -- start 2024-05-13T22:47:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.838+0000 7ff05f6e2700 1 -- start start 2024-05-13T22:47:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.838+0000 7ff05f6e2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff058100bc0 0x7ff058074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.838+0000 7ff05f6e2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff058101dc0 0x7ff058073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.838+0000 7ff05f6e2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0580736c0 con 0x7ff058101dc0 2024-05-13T22:47:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.838+0000 7ff05f6e2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff058073800 con 0x7ff058100bc0 2024-05-13T22:47:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.838+0000 7ff05cc7d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff058101dc0 0x7ff058073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:38.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.838+0000 7ff05d47e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff058100bc0 0x7ff058074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.838+0000 7ff05cc7d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff058101dc0 0x7ff058073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37492/0 (socket says 172.21.15.102:37492) 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.839+0000 7ff05cc7d700 1 -- 172.21.15.102:0/3806603016 learned_addr learned my addr 172.21.15.102:0/3806603016 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.839+0000 7ff05d47e700 1 -- 172.21.15.102:0/3806603016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff058101dc0 msgr2=0x7ff058073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.839+0000 7ff05d47e700 1 --2- 172.21.15.102:0/3806603016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff058101dc0 0x7ff058073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.839+0000 7ff05d47e700 1 -- 172.21.15.102:0/3806603016 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540095f0 con 0x7ff058100bc0 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.839+0000 7ff05d47e700 1 --2- 172.21.15.102:0/3806603016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff058100bc0 0x7ff058074b30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff054009910 tx=0x7ff05400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.839+0000 7ff04e7fc700 1 -- 172.21.15.102:0/3806603016 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05401d070 con 0x7ff058100bc0 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.839+0000 7ff05f6e2700 1 -- 172.21.15.102:0/3806603016 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff058073a80 con 0x7ff058100bc0 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.839+0000 7ff04e7fc700 1 -- 172.21.15.102:0/3806603016 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff05400f460 con 0x7ff058100bc0 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.839+0000 7ff04e7fc700 1 -- 172.21.15.102:0/3806603016 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff054021640 con 0x7ff058100bc0 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.839+0000 7ff05f6e2700 1 -- 172.21.15.102:0/3806603016 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff058073f70 con 0x7ff058100bc0 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.840+0000 7ff04e7fc700 1 -- 172.21.15.102:0/3806603016 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 23) v1 ==== 46428+0+0 (secure 0 0 0) 0x7ff05402b440 con 0x7ff058100bc0 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.840+0000 7ff04e7fc700 1 --2- 172.21.15.102:0/3806603016 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff044039360 0x7ff04403b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:38.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.840+0000 7ff04e7fc700 1 -- 172.21.15.102:0/3806603016 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff05404f7d0 con 0x7ff058100bc0 2024-05-13T22:47:38.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.840+0000 7ff05f6e2700 1 -- 172.21.15.102:0/3806603016 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff058109720 con 0x7ff058100bc0 2024-05-13T22:47:38.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.841+0000 7ff05cc7d700 1 --2- 172.21.15.102:0/3806603016 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff044039360 0x7ff04403b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:38.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.841+0000 7ff05cc7d700 1 --2- 172.21.15.102:0/3806603016 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff044039360 0x7ff04403b820 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff048005950 tx=0x7ff04800b500 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:38.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:38.846+0000 7ff04e7fc700 1 -- 172.21.15.102:0/3806603016 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7ff054026070 con 0x7ff058100bc0 2024-05-13T22:47:39.162 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:39 smithi102 ceph-mon[28599]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:39.162 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:39 smithi102 ceph-mon[28599]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:39.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.259+0000 7ff05f6e2700 1 -- 172.21.15.102:0/3806603016 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff05819e870 con 0x7ff058100bc0 2024-05-13T22:47:39.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.260+0000 7ff04e7fc700 1 -- 172.21.15.102:0/3806603016 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7ff054017400 con 0x7ff058100bc0 2024-05-13T22:47:39.239 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:47:39.239 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:47:39.239 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:47:39.239 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:47:39.239 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:47:39.240 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 1 2024-05-13T22:47:39.240 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:47:39.240 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:47:39.240 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:47:39.240 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:47:39.240 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:47:39.241 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:47:39.241 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:47:39.241 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:47:39.241 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-05-13T22:47:39.241 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 1 2024-05-13T22:47:39.241 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:47:39.242 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:47:39.243 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.262+0000 7ff05f6e2700 1 -- 172.21.15.102:0/3806603016 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff044039360 msgr2=0x7ff04403b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.262+0000 7ff05f6e2700 1 --2- 172.21.15.102:0/3806603016 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff044039360 0x7ff04403b820 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff048005950 tx=0x7ff04800b500 comp rx=0 tx=0).stop 2024-05-13T22:47:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.262+0000 7ff05f6e2700 1 -- 172.21.15.102:0/3806603016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff058100bc0 msgr2=0x7ff058074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.262+0000 7ff05f6e2700 1 --2- 172.21.15.102:0/3806603016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff058100bc0 0x7ff058074b30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff054009910 tx=0x7ff05400bd70 comp rx=0 tx=0).stop 2024-05-13T22:47:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.262+0000 7ff05f6e2700 1 -- 172.21.15.102:0/3806603016 shutdown_connections 2024-05-13T22:47:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.262+0000 7ff05f6e2700 1 --2- 172.21.15.102:0/3806603016 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7ff044039360 0x7ff04403b820 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:39.244 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.262+0000 7ff05f6e2700 1 --2- 172.21.15.102:0/3806603016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff058100bc0 0x7ff058074b30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:39.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.262+0000 7ff05f6e2700 1 --2- 172.21.15.102:0/3806603016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff058101dc0 0x7ff058073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:39.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.262+0000 7ff05f6e2700 1 -- 172.21.15.102:0/3806603016 >> 172.21.15.102:0/3806603016 conn(0x7ff0580fc140 msgr2=0x7ff058104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:39.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.262+0000 7ff05f6e2700 1 -- 172.21.15.102:0/3806603016 shutdown_connections 2024-05-13T22:47:39.245 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.262+0000 7ff05f6e2700 1 -- 172.21.15.102:0/3806603016 wait complete. 2024-05-13T22:47:39.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.409+0000 7f1b673fc700 1 -- 172.21.15.102:0/2306565300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b600ff240 msgr2=0x7f1b600ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:39.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.409+0000 7f1b673fc700 1 --2- 172.21.15.102:0/2306565300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b600ff240 0x7f1b600ff660 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009910 tx=0x7f1b5c009c20 comp rx=0 tx=0).stop 2024-05-13T22:47:39.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.409+0000 7f1b673fc700 1 -- 172.21.15.102:0/2306565300 shutdown_connections 2024-05-13T22:47:39.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.409+0000 7f1b673fc700 1 --2- 172.21.15.102:0/2306565300 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b60100440 0x7f1b601008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:39.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.409+0000 7f1b673fc700 1 --2- 172.21.15.102:0/2306565300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b600ff240 0x7f1b600ff660 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.410+0000 7f1b673fc700 1 -- 172.21.15.102:0/2306565300 >> 172.21.15.102:0/2306565300 conn(0x7f1b600fa7c0 msgr2=0x7f1b600fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.410+0000 7f1b673fc700 1 -- 172.21.15.102:0/2306565300 shutdown_connections 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.410+0000 7f1b673fc700 1 -- 172.21.15.102:0/2306565300 wait complete. 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.410+0000 7f1b673fc700 1 Processor -- start 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.410+0000 7f1b673fc700 1 -- start start 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.410+0000 7f1b673fc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b600ff240 0x7f1b601963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.410+0000 7f1b673fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b60100440 0x7f1b60196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.410+0000 7f1b673fc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60196f40 con 0x7f1b60100440 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.410+0000 7f1b673fc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60197080 con 0x7f1b600ff240 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b64997700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b60100440 0x7f1b60196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b64997700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b60100440 0x7f1b60196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37502/0 (socket says 172.21.15.102:37502) 2024-05-13T22:47:39.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b64997700 1 -- 172.21.15.102:0/2869690224 learned_addr learned my addr 172.21.15.102:0/2869690224 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:47:39.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b65198700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b600ff240 0x7f1b601963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:39.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b64997700 1 -- 172.21.15.102:0/2869690224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b600ff240 msgr2=0x7f1b601963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:39.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b64997700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b600ff240 0x7f1b601963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:39.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b64997700 1 -- 172.21.15.102:0/2869690224 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c0095f0 con 0x7f1b60100440 2024-05-13T22:47:39.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b65198700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b600ff240 0x7f1b601963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:47:39.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b64997700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b60100440 0x7f1b60196920 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000e940 tx=0x7f1b5000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:39.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b567fc700 1 -- 172.21.15.102:0/2869690224 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b5000cba0 con 0x7f1b60100440 2024-05-13T22:47:39.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b567fc700 1 -- 172.21.15.102:0/2869690224 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1b5000ee90 con 0x7f1b60100440 2024-05-13T22:47:39.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b673fc700 1 -- 172.21.15.102:0/2869690224 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6019bb30 con 0x7f1b60100440 2024-05-13T22:47:39.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b567fc700 1 -- 172.21.15.102:0/2869690224 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b50005650 con 0x7f1b60100440 2024-05-13T22:47:39.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.411+0000 7f1b673fc700 1 -- 172.21.15.102:0/2869690224 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6019c080 con 0x7f1b60100440 2024-05-13T22:47:39.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.412+0000 7f1b673fc700 1 -- 172.21.15.102:0/2869690224 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b60108640 con 0x7f1b60100440 2024-05-13T22:47:39.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.412+0000 7f1b567fc700 1 -- 172.21.15.102:0/2869690224 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 23) v1 ==== 46428+0+0 (secure 0 0 0) 0x7f1b50021440 con 0x7f1b60100440 2024-05-13T22:47:39.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.413+0000 7f1b567fc700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f1b4c039360 0x7f1b4c03b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:47:39.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.413+0000 7f1b567fc700 1 -- 172.21.15.102:0/2869690224 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f1b50014070 con 0x7f1b60100440 2024-05-13T22:47:39.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.413+0000 7f1b65198700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f1b4c039360 0x7f1b4c03b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:47:39.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.413+0000 7f1b65198700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f1b4c039360 0x7f1b4c03b820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00b6f0 tx=0x7f1b5c005940 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:47:39.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.417+0000 7f1b567fc700 1 -- 172.21.15.102:0/2869690224 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f1b50016330 con 0x7f1b60100440 2024-05-13T22:47:39.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:39 smithi130 ceph-mon[36591]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:39.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:39 smithi130 ceph-mon[36591]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:39.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.759+0000 7f1b673fc700 1 -- 172.21.15.102:0/2869690224 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b60105f90 con 0x7f1b4c039360 2024-05-13T22:47:39.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.761+0000 7f1b567fc700 1 -- 172.21.15.102:0/2869690224 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f1b60105f90 con 0x7f1b4c039360 2024-05-13T22:47:39.739 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:47:39.739 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:47:39.739 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:47:39.739 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-13T22:47:39.739 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [], 2024-05-13T22:47:39.739 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "1/2 daemons upgraded", 2024-05-13T22:47:39.739 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "", 2024-05-13T22:47:39.739 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:47:39.739 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:47:39.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.764+0000 7f1b673fc700 1 -- 172.21.15.102:0/2869690224 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f1b4c039360 msgr2=0x7f1b4c03b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:39.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.764+0000 7f1b673fc700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f1b4c039360 0x7f1b4c03b820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00b6f0 tx=0x7f1b5c005940 comp rx=0 tx=0).stop 2024-05-13T22:47:39.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.764+0000 7f1b673fc700 1 -- 172.21.15.102:0/2869690224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b60100440 msgr2=0x7f1b60196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:47:39.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.764+0000 7f1b673fc700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b60100440 0x7f1b60196920 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000e940 tx=0x7f1b5000ec50 comp rx=0 tx=0).stop 2024-05-13T22:47:39.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.764+0000 7f1b673fc700 1 -- 172.21.15.102:0/2869690224 shutdown_connections 2024-05-13T22:47:39.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.764+0000 7f1b673fc700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f1b4c039360 0x7f1b4c03b820 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:39.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.764+0000 7f1b673fc700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b600ff240 0x7f1b601963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:39.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.764+0000 7f1b673fc700 1 --2- 172.21.15.102:0/2869690224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b60100440 0x7f1b60196920 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:47:39.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.764+0000 7f1b673fc700 1 -- 172.21.15.102:0/2869690224 >> 172.21.15.102:0/2869690224 conn(0x7f1b600fa7c0 msgr2=0x7f1b60105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:47:39.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.764+0000 7f1b673fc700 1 -- 172.21.15.102:0/2869690224 shutdown_connections 2024-05-13T22:47:39.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:47:39.764+0000 7f1b673fc700 1 -- 172.21.15.102:0/2869690224 wait complete. 2024-05-13T22:47:40.401 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:40 smithi102 ceph-mon[28599]: pgmap v7: 65 pgs: 65 active+clean; 130 MiB data, 682 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 305 op/s 2024-05-13T22:47:40.402 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:40 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3806603016' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:47:40.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:40 smithi130 ceph-mon[36591]: pgmap v7: 65 pgs: 65 active+clean; 130 MiB data, 682 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 305 op/s 2024-05-13T22:47:40.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:40 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3806603016' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:47:41.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:41 smithi130 ceph-mon[36591]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:41.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:41 smithi130 ceph-mon[36591]: Standby manager daemon smithi102.xcijrj started 2024-05-13T22:47:41.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:41 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.102:0/52782493' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi102.xcijrj/crt"}]: dispatch 2024-05-13T22:47:41.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:41 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.102:0/52782493' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:47:41.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:41 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.102:0/52782493' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi102.xcijrj/key"}]: dispatch 2024-05-13T22:47:41.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:41 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.102:0/52782493' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:47:41.546 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:41 smithi102 ceph-mon[28599]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:47:41.546 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:41 smithi102 ceph-mon[28599]: Standby manager daemon smithi102.xcijrj started 2024-05-13T22:47:41.546 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:41 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.102:0/52782493' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi102.xcijrj/crt"}]: dispatch 2024-05-13T22:47:41.546 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:41 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.102:0/52782493' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:47:41.546 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:41 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.102:0/52782493' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi102.xcijrj/key"}]: dispatch 2024-05-13T22:47:41.546 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:41 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.102:0/52782493' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:47:42.503 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:42 smithi102 ceph-mon[28599]: pgmap v8: 65 pgs: 65 active+clean; 130 MiB data, 682 MiB used, 536 GiB / 536 GiB avail; 892 KiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-05-13T22:47:42.503 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:42 smithi102 ceph-mon[28599]: mgrmap e24: smithi130.uoqoxl(active, since 10s), standbys: smithi102.xcijrj 2024-05-13T22:47:42.503 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:42 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mgr metadata", "who": "smithi102.xcijrj", "id": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:47:42.503 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:42 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:42.504 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:42 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:42 smithi130 ceph-mon[36591]: pgmap v8: 65 pgs: 65 active+clean; 130 MiB data, 682 MiB used, 536 GiB / 536 GiB avail; 892 KiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-05-13T22:47:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:42 smithi130 ceph-mon[36591]: mgrmap e24: smithi130.uoqoxl(active, since 10s), standbys: smithi102.xcijrj 2024-05-13T22:47:42.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:42 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mgr metadata", "who": "smithi102.xcijrj", "id": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:47:42.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:42 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:42.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:42 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:44.297 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:44 smithi130 ceph-mon[36591]: pgmap v9: 65 pgs: 65 active+clean; 143 MiB data, 740 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 304 op/s 2024-05-13T22:47:44.297 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:44 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:44.297 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:44 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:44.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:44 smithi102 ceph-mon[28599]: pgmap v9: 65 pgs: 65 active+clean; 143 MiB data, 740 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 304 op/s 2024-05-13T22:47:44.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:44 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:44.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:44 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:46 smithi102 ceph-mon[28599]: pgmap v10: 65 pgs: 65 active+clean; 143 MiB data, 740 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-05-13T22:47:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:46 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:46 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:46 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:47:46.506 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:46 smithi130 ceph-mon[36591]: pgmap v10: 65 pgs: 65 active+clean; 143 MiB data, 740 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-05-13T22:47:46.507 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:46 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:46.507 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:46 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:46.507 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:46 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:47:47.021 INFO:tasks.workunit.client.0.smithi102.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-13T22:47:47.029 INFO:tasks.workunit.client.0.smithi102.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-13T22:47:47.029 INFO:tasks.workunit.client.0.smithi102.stderr:+ make 2024-05-13T22:47:47.493 INFO:tasks.workunit.client.0.smithi102.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-13T22:47:48.580 INFO:tasks.workunit.client.0.smithi102.stderr:++ readlink -f fsstress 2024-05-13T22:47:48.581 INFO:tasks.workunit.client.0.smithi102.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-13T22:47:48.581 INFO:tasks.workunit.client.0.smithi102.stderr:+ popd 2024-05-13T22:47:48.582 INFO:tasks.workunit.client.0.smithi102.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-13T22:47:48.583 INFO:tasks.workunit.client.0.smithi102.stderr:+ popd 2024-05-13T22:47:48.583 INFO:tasks.workunit.client.0.smithi102.stdout:~/cephtest/mnt.0/client.0/tmp 2024-05-13T22:47:48.583 INFO:tasks.workunit.client.0.smithi102.stderr:++ mktemp -d -p . 2024-05-13T22:47:48.586 INFO:tasks.workunit.client.0.smithi102.stderr:+ T=./tmp.ZPN04XmeAi 2024-05-13T22:47:48.586 INFO:tasks.workunit.client.0.smithi102.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.ZPN04XmeAi -l 1 -n 1000 -p 10 -v 2024-05-13T22:47:48.590 INFO:tasks.workunit.client.0.smithi102.stdout:seed = 1716241803 2024-05-13T22:47:48.596 INFO:tasks.workunit.client.0.smithi102.stdout:0/0: chown . 3090699 1 2024-05-13T22:47:48.596 INFO:tasks.workunit.client.0.smithi102.stdout:0/1: dread - no filename 2024-05-13T22:47:48.597 INFO:tasks.workunit.client.0.smithi102.stdout:1/0: chown . 181 1 2024-05-13T22:47:48.599 INFO:tasks.workunit.client.0.smithi102.stdout:2/0: chown . 0 1 2024-05-13T22:47:48.599 INFO:tasks.workunit.client.0.smithi102.stdout:2/1: write - no filename 2024-05-13T22:47:48.600 INFO:tasks.workunit.client.0.smithi102.stdout:0/2: creat f0 x:0 0 0 2024-05-13T22:47:48.600 INFO:tasks.workunit.client.0.smithi102.stdout:0/3: dread - f0 zero size 2024-05-13T22:47:48.601 INFO:tasks.workunit.client.0.smithi102.stdout:1/1: creat f0 x:0 0 0 2024-05-13T22:47:48.601 INFO:tasks.workunit.client.0.smithi102.stdout:0/4: write f0 [1033393,25616] 0 2024-05-13T22:47:48.603 INFO:tasks.workunit.client.0.smithi102.stdout:3/0: truncate - no filename 2024-05-13T22:47:48.603 INFO:tasks.workunit.client.0.smithi102.stdout:3/1: stat - no entries 2024-05-13T22:47:48.603 INFO:tasks.workunit.client.0.smithi102.stdout:3/2: rmdir - no directory 2024-05-13T22:47:48.603 INFO:tasks.workunit.client.0.smithi102.stdout:3/3: write - no filename 2024-05-13T22:47:48.606 INFO:tasks.workunit.client.0.smithi102.stdout:2/2: mknod c0 0 2024-05-13T22:47:48.606 INFO:tasks.workunit.client.0.smithi102.stdout:4/0: chown . 44517718 1 2024-05-13T22:47:48.607 INFO:tasks.workunit.client.0.smithi102.stdout:4/1: dwrite - no filename 2024-05-13T22:47:48.607 INFO:tasks.workunit.client.0.smithi102.stdout:4/2: dwrite - no filename 2024-05-13T22:47:48.607 INFO:tasks.workunit.client.0.smithi102.stdout:4/3: chown . 227 1 2024-05-13T22:47:48.607 INFO:tasks.workunit.client.0.smithi102.stdout:4/4: dread - no filename 2024-05-13T22:47:48.608 INFO:tasks.workunit.client.0.smithi102.stdout:0/5: dread f0 [0,4194304] 0 2024-05-13T22:47:48.609 INFO:tasks.workunit.client.0.smithi102.stdout:5/0: stat - no entries 2024-05-13T22:47:48.609 INFO:tasks.workunit.client.0.smithi102.stdout:5/1: dwrite - no filename 2024-05-13T22:47:48.609 INFO:tasks.workunit.client.0.smithi102.stdout:5/2: fdatasync - no filename 2024-05-13T22:47:48.609 INFO:tasks.workunit.client.0.smithi102.stdout:5/3: dread - no filename 2024-05-13T22:47:48.609 INFO:tasks.workunit.client.0.smithi102.stdout:5/4: unlink - no file 2024-05-13T22:47:48.609 INFO:tasks.workunit.client.0.smithi102.stdout:5/5: unlink - no file 2024-05-13T22:47:48.610 INFO:tasks.workunit.client.0.smithi102.stdout:2/3: symlink l1 0 2024-05-13T22:47:48.610 INFO:tasks.workunit.client.0.smithi102.stdout:2/4: write - no filename 2024-05-13T22:47:48.613 INFO:tasks.workunit.client.0.smithi102.stdout:3/4: getdents . 0 2024-05-13T22:47:48.616 INFO:tasks.workunit.client.0.smithi102.stdout:2/5: mknod c2 0 2024-05-13T22:47:48.617 INFO:tasks.workunit.client.0.smithi102.stdout:4/5: creat f0 x:0 0 0 2024-05-13T22:47:48.618 INFO:tasks.workunit.client.0.smithi102.stdout:4/6: write f0 [713770,87938] 0 2024-05-13T22:47:48.619 INFO:tasks.workunit.client.0.smithi102.stdout:4/7: truncate f0 1834222 0 2024-05-13T22:47:48.622 INFO:tasks.workunit.client.0.smithi102.stdout:5/6: creat f0 x:0 0 0 2024-05-13T22:47:48.623 INFO:tasks.workunit.client.0.smithi102.stdout:5/7: dread - f0 zero size 2024-05-13T22:47:48.624 INFO:tasks.workunit.client.0.smithi102.stdout:5/8: stat f0 0 2024-05-13T22:47:48.624 INFO:tasks.workunit.client.0.smithi102.stdout:5/9: readlink - no filename 2024-05-13T22:47:48.624 INFO:tasks.workunit.client.0.smithi102.stdout:3/5: mknod c0 0 2024-05-13T22:47:48.624 INFO:tasks.workunit.client.0.smithi102.stdout:3/6: rmdir - no directory 2024-05-13T22:47:48.624 INFO:tasks.workunit.client.0.smithi102.stdout:3/7: dread - no filename 2024-05-13T22:47:48.624 INFO:tasks.workunit.client.0.smithi102.stdout:3/8: readlink - no filename 2024-05-13T22:47:48.626 INFO:tasks.workunit.client.0.smithi102.stdout:6/0: creat f0 x:0 0 0 2024-05-13T22:47:48.626 INFO:tasks.workunit.client.0.smithi102.stdout:7/0: chown . 1911 1 2024-05-13T22:47:48.626 INFO:tasks.workunit.client.0.smithi102.stdout:3/9: chown c0 348692 1 2024-05-13T22:47:48.628 INFO:tasks.workunit.client.0.smithi102.stdout:5/10: creat f1 x:0 0 0 2024-05-13T22:47:48.635 INFO:tasks.workunit.client.0.smithi102.stdout:4/8: dwrite f0 [0,4194304] 0 2024-05-13T22:47:48.637 INFO:tasks.workunit.client.0.smithi102.stdout:3/10: creat f1 x:0 0 0 2024-05-13T22:47:48.638 INFO:tasks.workunit.client.0.smithi102.stdout:3/11: fdatasync f1 0 2024-05-13T22:47:48.640 INFO:tasks.workunit.client.0.smithi102.stdout:6/1: dwrite f0 [0,4194304] 0 2024-05-13T22:47:48.644 INFO:tasks.workunit.client.0.smithi102.stdout:4/9: dread f0 [0,4194304] 0 2024-05-13T22:47:48.646 INFO:tasks.workunit.client.0.smithi102.stdout:7/1: creat f0 x:0 0 0 2024-05-13T22:47:48.647 INFO:tasks.workunit.client.0.smithi102.stdout:9/0: link - no file 2024-05-13T22:47:48.647 INFO:tasks.workunit.client.0.smithi102.stdout:9/1: truncate - no filename 2024-05-13T22:47:48.647 INFO:tasks.workunit.client.0.smithi102.stdout:9/2: rename - no filename 2024-05-13T22:47:48.647 INFO:tasks.workunit.client.0.smithi102.stdout:9/3: dread - no filename 2024-05-13T22:47:48.649 INFO:tasks.workunit.client.0.smithi102.stdout:4/10: dread f0 [0,4194304] 0 2024-05-13T22:47:48.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: pgmap v11: 65 pgs: 65 active+clean; 143 MiB data, 740 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-05-13T22:47:48.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: Adjusting osd_memory_target on smithi102 to 1216M 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: Adjusting osd_memory_target on smithi130 to 2240M 2024-05-13T22:47:48.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:47:48.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:47:48.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: Updating smithi102:/etc/ceph/ceph.conf 2024-05-13T22:47:48.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:48 smithi130 ceph-mon[36591]: Updating smithi130:/etc/ceph/ceph.conf 2024-05-13T22:47:48.653 INFO:tasks.workunit.client.0.smithi102.stdout:5/11: link f1 f2 0 2024-05-13T22:47:48.654 INFO:tasks.workunit.client.0.smithi102.stdout:5/12: readlink - no filename 2024-05-13T22:47:48.655 INFO:tasks.workunit.client.0.smithi102.stdout:3/12: creat f2 x:0 0 0 2024-05-13T22:47:48.656 INFO:tasks.workunit.client.0.smithi102.stdout:6/2: creat f1 x:0 0 0 2024-05-13T22:47:48.657 INFO:tasks.workunit.client.0.smithi102.stdout:7/2: creat f1 x:0 0 0 2024-05-13T22:47:48.660 INFO:tasks.workunit.client.0.smithi102.stdout:8/0: mkdir d0 0 2024-05-13T22:47:48.660 INFO:tasks.workunit.client.0.smithi102.stdout:8/1: dread - no filename 2024-05-13T22:47:48.660 INFO:tasks.workunit.client.0.smithi102.stdout:8/2: write - no filename 2024-05-13T22:47:48.661 INFO:tasks.workunit.client.0.smithi102.stdout:8/3: write - no filename 2024-05-13T22:47:48.661 INFO:tasks.workunit.client.0.smithi102.stdout:8/4: fsync - no filename 2024-05-13T22:47:48.664 INFO:tasks.workunit.client.0.smithi102.stdout:5/13: symlink l3 0 2024-05-13T22:47:48.666 INFO:tasks.workunit.client.0.smithi102.stdout:5/14: dread - f2 zero size 2024-05-13T22:47:48.667 INFO:tasks.workunit.client.0.smithi102.stdout:3/13: symlink l3 0 2024-05-13T22:47:48.668 INFO:tasks.workunit.client.0.smithi102.stdout:5/15: dread - f1 zero size 2024-05-13T22:47:48.670 INFO:tasks.workunit.client.0.smithi102.stdout:9/4: mkdir d0 0 2024-05-13T22:47:48.671 INFO:tasks.workunit.client.0.smithi102.stdout:6/3: dwrite f1 [0,4194304] 0 2024-05-13T22:47:48.675 INFO:tasks.workunit.client.0.smithi102.stdout:6/4: chown f0 7816617 1 2024-05-13T22:47:48.676 INFO:tasks.workunit.client.0.smithi102.stdout:7/3: dwrite f0 [0,4194304] 0 2024-05-13T22:47:48.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: pgmap v11: 65 pgs: 65 active+clean; 143 MiB data, 740 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-05-13T22:47:48.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: Adjusting osd_memory_target on smithi102 to 1216M 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-13T22:47:48.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: Adjusting osd_memory_target on smithi130 to 2240M 2024-05-13T22:47:48.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:48.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:47:48.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:47:48.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: Updating smithi102:/etc/ceph/ceph.conf 2024-05-13T22:47:48.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:48 smithi102 ceph-mon[28599]: Updating smithi130:/etc/ceph/ceph.conf 2024-05-13T22:47:48.682 INFO:tasks.workunit.client.0.smithi102.stdout:5/16: mknod c4 0 2024-05-13T22:47:48.684 INFO:tasks.workunit.client.0.smithi102.stdout:5/17: dread - f0 zero size 2024-05-13T22:47:48.685 INFO:tasks.workunit.client.0.smithi102.stdout:3/14: dwrite f2 [0,4194304] 0 2024-05-13T22:47:48.690 INFO:tasks.workunit.client.0.smithi102.stdout:7/4: symlink l2 0 2024-05-13T22:47:48.695 INFO:tasks.workunit.client.0.smithi102.stdout:3/15: symlink l4 0 2024-05-13T22:47:48.696 INFO:tasks.workunit.client.0.smithi102.stdout:5/18: creat f5 x:0 0 0 2024-05-13T22:47:48.696 INFO:tasks.workunit.client.0.smithi102.stdout:3/16: write f1 [198409,118350] 0 2024-05-13T22:47:48.699 INFO:tasks.workunit.client.0.smithi102.stdout:7/5: creat f3 x:0 0 0 2024-05-13T22:47:48.703 INFO:tasks.workunit.client.0.smithi102.stdout:5/19: creat f6 x:0 0 0 2024-05-13T22:47:48.704 INFO:tasks.workunit.client.0.smithi102.stdout:9/5: rmdir d0 0 2024-05-13T22:47:48.706 INFO:tasks.workunit.client.0.smithi102.stdout:7/6: rename f1 to f4 0 2024-05-13T22:47:48.707 INFO:tasks.workunit.client.0.smithi102.stdout:7/7: truncate f3 758704 0 2024-05-13T22:47:48.709 INFO:tasks.workunit.client.0.smithi102.stdout:5/20: write f1 [884073,72975] 0 2024-05-13T22:47:48.711 INFO:tasks.workunit.client.0.smithi102.stdout:9/6: mkdir d1 0 2024-05-13T22:47:48.711 INFO:tasks.workunit.client.0.smithi102.stdout:9/7: readlink - no filename 2024-05-13T22:47:48.712 INFO:tasks.workunit.client.0.smithi102.stdout:7/8: creat f5 x:0 0 0 2024-05-13T22:47:48.715 INFO:tasks.workunit.client.0.smithi102.stdout:7/9: mkdir d6 0 2024-05-13T22:47:48.718 INFO:tasks.workunit.client.0.smithi102.stdout:5/21: link f5 f7 0 2024-05-13T22:47:48.720 INFO:tasks.workunit.client.0.smithi102.stdout:9/8: symlink d1/l2 0 2024-05-13T22:47:48.720 INFO:tasks.workunit.client.0.smithi102.stdout:9/9: dread - no filename 2024-05-13T22:47:48.720 INFO:tasks.workunit.client.0.smithi102.stdout:9/10: truncate - no filename 2024-05-13T22:47:48.720 INFO:tasks.workunit.client.0.smithi102.stdout:9/11: dread - no filename 2024-05-13T22:47:48.721 INFO:tasks.workunit.client.0.smithi102.stdout:9/12: rename d1 to d1/d3 22 2024-05-13T22:47:48.722 INFO:tasks.workunit.client.0.smithi102.stdout:5/22: symlink l8 0 2024-05-13T22:47:48.723 INFO:tasks.workunit.client.0.smithi102.stdout:7/10: dwrite f0 [0,4194304] 0 2024-05-13T22:47:48.724 INFO:tasks.workunit.client.0.smithi102.stdout:7/11: write f5 [2624,112694] 0 2024-05-13T22:47:48.725 INFO:tasks.workunit.client.0.smithi102.stdout:9/13: symlink d1/l4 0 2024-05-13T22:47:48.725 INFO:tasks.workunit.client.0.smithi102.stdout:9/14: dread - no filename 2024-05-13T22:47:48.729 INFO:tasks.workunit.client.0.smithi102.stdout:5/23: dread f1 [0,4194304] 0 2024-05-13T22:47:48.729 INFO:tasks.workunit.client.0.smithi102.stdout:7/12: dread f0 [0,4194304] 0 2024-05-13T22:47:48.730 INFO:tasks.workunit.client.0.smithi102.stdout:9/15: symlink d1/l5 0 2024-05-13T22:47:48.733 INFO:tasks.workunit.client.0.smithi102.stdout:5/24: rename f7 to f9 0 2024-05-13T22:47:48.734 INFO:tasks.workunit.client.0.smithi102.stdout:5/25: truncate f9 388423 0 2024-05-13T22:47:48.734 INFO:tasks.workunit.client.0.smithi102.stdout:5/26: truncate f6 559191 0 2024-05-13T22:47:48.735 INFO:tasks.workunit.client.0.smithi102.stdout:5/27: write f0 [792018,37456] 0 2024-05-13T22:47:48.735 INFO:tasks.workunit.client.0.smithi102.stdout:9/16: mkdir d1/d6 0 2024-05-13T22:47:48.738 INFO:tasks.workunit.client.0.smithi102.stdout:7/13: creat d6/f7 x:0 0 0 2024-05-13T22:47:48.741 INFO:tasks.workunit.client.0.smithi102.stdout:7/14: creat d6/f8 x:0 0 0 2024-05-13T22:47:48.743 INFO:tasks.workunit.client.0.smithi102.stdout:5/28: dwrite f6 [0,4194304] 0 2024-05-13T22:47:48.748 INFO:tasks.workunit.client.0.smithi102.stdout:5/29: symlink la 0 2024-05-13T22:47:48.750 INFO:tasks.workunit.client.0.smithi102.stdout:5/30: symlink lb 0 2024-05-13T22:47:48.752 INFO:tasks.workunit.client.0.smithi102.stdout:5/31: write f5 [590964,45030] 0 2024-05-13T22:47:48.754 INFO:tasks.workunit.client.0.smithi102.stdout:5/32: symlink lc 0 2024-05-13T22:47:48.757 INFO:tasks.workunit.client.0.smithi102.stdout:5/33: creat fd x:0 0 0 2024-05-13T22:47:48.758 INFO:tasks.workunit.client.0.smithi102.stdout:5/34: chown lb 98574353 1 2024-05-13T22:47:48.761 INFO:tasks.workunit.client.0.smithi102.stdout:5/35: dread f5 [0,4194304] 0 2024-05-13T22:47:48.763 INFO:tasks.workunit.client.0.smithi102.stdout:5/36: mkdir de 0 2024-05-13T22:47:48.764 INFO:tasks.workunit.client.0.smithi102.stdout:5/37: read f0 [199657,98532] 0 2024-05-13T22:47:48.768 INFO:tasks.workunit.client.0.smithi102.stdout:5/38: write f9 [449487,78499] 0 2024-05-13T22:47:48.772 INFO:tasks.workunit.client.0.smithi102.stdout:5/39: write f2 [1864339,44346] 0 2024-05-13T22:47:48.772 INFO:tasks.workunit.client.0.smithi102.stdout:5/40: write fd [264013,128066] 0 2024-05-13T22:47:48.775 INFO:tasks.workunit.client.0.smithi102.stdout:5/41: rename la to de/lf 0 2024-05-13T22:47:48.782 INFO:tasks.workunit.client.0.smithi102.stdout:5/42: dwrite f9 [0,4194304] 0 2024-05-13T22:47:48.784 INFO:tasks.workunit.client.0.smithi102.stdout:5/43: mkdir de/d10 0 2024-05-13T22:47:48.796 INFO:tasks.workunit.client.0.smithi102.stdout:5/44: dwrite f0 [0,4194304] 0 2024-05-13T22:47:48.801 INFO:tasks.workunit.client.0.smithi102.stdout:5/45: dread f6 [0,4194304] 0 2024-05-13T22:47:48.809 INFO:tasks.workunit.client.0.smithi102.stdout:5/46: dwrite fd [0,4194304] 0 2024-05-13T22:47:48.818 INFO:tasks.workunit.client.0.smithi102.stdout:5/47: dwrite f6 [0,4194304] 0 2024-05-13T22:47:48.822 INFO:tasks.workunit.client.0.smithi102.stdout:5/48: creat de/f11 x:0 0 0 2024-05-13T22:47:48.822 INFO:tasks.workunit.client.0.smithi102.stdout:5/49: chown f5 4031582 1 2024-05-13T22:47:48.823 INFO:tasks.workunit.client.0.smithi102.stdout:5/50: chown f5 3581104 1 2024-05-13T22:47:48.825 INFO:tasks.workunit.client.0.smithi102.stdout:5/51: mknod de/d10/c12 0 2024-05-13T22:47:48.827 INFO:tasks.workunit.client.0.smithi102.stdout:5/52: creat de/f13 x:0 0 0 2024-05-13T22:47:49.205 INFO:tasks.workunit.client.0.smithi102.stdout:0/6: fsync f0 0 2024-05-13T22:47:49.206 INFO:tasks.workunit.client.0.smithi102.stdout:0/7: write f0 [1325079,68030] 0 2024-05-13T22:47:49.209 INFO:tasks.workunit.client.0.smithi102.stdout:1/2: getdents . 0 2024-05-13T22:47:49.209 INFO:tasks.workunit.client.0.smithi102.stdout:1/3: dread - f0 zero size 2024-05-13T22:47:49.209 INFO:tasks.workunit.client.0.smithi102.stdout:0/8: dread f0 [0,4194304] 0 2024-05-13T22:47:49.210 INFO:tasks.workunit.client.0.smithi102.stdout:1/4: read - f0 zero size 2024-05-13T22:47:49.210 INFO:tasks.workunit.client.0.smithi102.stdout:1/5: stat f0 0 2024-05-13T22:47:49.210 INFO:tasks.workunit.client.0.smithi102.stdout:1/6: chown f0 25154 1 2024-05-13T22:47:49.217 INFO:tasks.workunit.client.0.smithi102.stdout:1/7: dwrite f0 [0,4194304] 0 2024-05-13T22:47:49.218 INFO:tasks.workunit.client.0.smithi102.stdout:1/8: truncate f0 4475546 0 2024-05-13T22:47:49.223 INFO:tasks.workunit.client.0.smithi102.stdout:2/6: getdents . 0 2024-05-13T22:47:49.224 INFO:tasks.workunit.client.0.smithi102.stdout:1/9: symlink l1 0 2024-05-13T22:47:49.224 INFO:tasks.workunit.client.0.smithi102.stdout:1/10: chown f0 529272379 1 2024-05-13T22:47:49.226 INFO:tasks.workunit.client.0.smithi102.stdout:2/7: rename c0 to c3 0 2024-05-13T22:47:49.231 INFO:tasks.workunit.client.0.smithi102.stdout:2/8: symlink l4 0 2024-05-13T22:47:49.231 INFO:tasks.workunit.client.0.smithi102.stdout:2/9: read - no filename 2024-05-13T22:47:49.234 INFO:tasks.workunit.client.0.smithi102.stdout:1/11: dwrite f0 [0,4194304] 0 2024-05-13T22:47:49.234 INFO:tasks.workunit.client.0.smithi102.stdout:1/12: readlink l1 0 2024-05-13T22:47:49.235 INFO:tasks.workunit.client.0.smithi102.stdout:2/10: symlink l5 0 2024-05-13T22:47:49.236 INFO:tasks.workunit.client.0.smithi102.stdout:2/11: fdatasync - no filename 2024-05-13T22:47:49.236 INFO:tasks.workunit.client.0.smithi102.stdout:2/12: truncate - no filename 2024-05-13T22:47:49.236 INFO:tasks.workunit.client.0.smithi102.stdout:2/13: dread - no filename 2024-05-13T22:47:49.236 INFO:tasks.workunit.client.0.smithi102.stdout:2/14: dread - no filename 2024-05-13T22:47:49.236 INFO:tasks.workunit.client.0.smithi102.stdout:2/15: dread - no filename 2024-05-13T22:47:49.251 INFO:tasks.workunit.client.0.smithi102.stdout:1/13: fsync f0 0 2024-05-13T22:47:49.252 INFO:tasks.workunit.client.0.smithi102.stdout:1/14: chown f0 240976467 1 2024-05-13T22:47:49.266 INFO:tasks.workunit.client.0.smithi102.stdout:1/15: mkdir d2 0 2024-05-13T22:47:49.266 INFO:tasks.workunit.client.0.smithi102.stdout:4/11: truncate f0 1074916 0 2024-05-13T22:47:49.266 INFO:tasks.workunit.client.0.smithi102.stdout:1/16: fdatasync f0 0 2024-05-13T22:47:49.276 INFO:tasks.workunit.client.0.smithi102.stdout:6/5: truncate f0 1786570 0 2024-05-13T22:47:49.285 INFO:tasks.workunit.client.0.smithi102.stdout:7/15: fsync f5 0 2024-05-13T22:47:49.285 INFO:tasks.workunit.client.0.smithi102.stdout:7/16: chown f5 1533577980 1 2024-05-13T22:47:49.346 INFO:tasks.workunit.client.0.smithi102.stdout:5/53: getdents de 0 2024-05-13T22:47:49.347 INFO:tasks.workunit.client.0.smithi102.stdout:5/54: dread - de/f13 zero size 2024-05-13T22:47:49.629 INFO:tasks.workunit.client.0.smithi102.stdout:4/12: symlink l1 0 2024-05-13T22:47:49.633 INFO:tasks.workunit.client.0.smithi102.stdout:5/55: rename f5 to de/f14 0 2024-05-13T22:47:49.634 INFO:tasks.workunit.client.0.smithi102.stdout:5/56: chown de/f13 0 1 2024-05-13T22:47:49.636 INFO:tasks.workunit.client.0.smithi102.stdout:1/17: symlink d2/l3 0 2024-05-13T22:47:49.643 INFO:tasks.workunit.client.0.smithi102.stdout:1/18: dwrite f0 [0,4194304] 0 2024-05-13T22:47:49.643 INFO:tasks.workunit.client.0.smithi102.stdout:5/57: creat de/f15 x:0 0 0 2024-05-13T22:47:49.644 INFO:tasks.workunit.client.0.smithi102.stdout:1/19: write f0 [3847522,126212] 0 2024-05-13T22:47:49.647 INFO:tasks.workunit.client.0.smithi102.stdout:5/58: dread f6 [0,4194304] 0 2024-05-13T22:47:49.648 INFO:tasks.workunit.client.0.smithi102.stdout:5/59: dread - de/f15 zero size 2024-05-13T22:47:49.648 INFO:tasks.workunit.client.0.smithi102.stdout:1/20: mkdir d2/d4 0 2024-05-13T22:47:49.652 INFO:tasks.workunit.client.0.smithi102.stdout:1/21: mkdir d2/d4/d5 0 2024-05-13T22:47:49.653 INFO:tasks.workunit.client.0.smithi102.stdout:1/22: write f0 [566429,128135] 0 2024-05-13T22:47:49.654 INFO:tasks.workunit.client.0.smithi102.stdout:1/23: write f0 [3134116,109807] 0 2024-05-13T22:47:49.655 INFO:tasks.workunit.client.0.smithi102.stdout:1/24: truncate f0 4759131 0 2024-05-13T22:47:49.660 INFO:tasks.workunit.client.0.smithi102.stdout:1/25: creat d2/d4/d5/f6 x:0 0 0 2024-05-13T22:47:49.661 INFO:tasks.workunit.client.0.smithi102.stdout:1/26: write f0 [816335,86918] 0 2024-05-13T22:47:49.664 INFO:tasks.workunit.client.0.smithi102.stdout:1/27: symlink d2/l7 0 2024-05-13T22:47:49.669 INFO:tasks.workunit.client.0.smithi102.stdout:0/9: truncate f0 15869 0 2024-05-13T22:47:49.669 INFO:tasks.workunit.client.0.smithi102.stdout:0/10: readlink - no filename 2024-05-13T22:47:49.671 INFO:tasks.workunit.client.0.smithi102.stdout:0/11: mkdir d1 0 2024-05-13T22:47:49.686 INFO:tasks.workunit.client.0.smithi102.stdout:4/13: fsync f0 0 2024-05-13T22:47:49.691 INFO:tasks.workunit.client.0.smithi102.stdout:4/14: rename l1 to l2 0 2024-05-13T22:47:49.693 INFO:tasks.workunit.client.0.smithi102.stdout:4/15: dread f0 [0,4194304] 0 2024-05-13T22:47:49.837 INFO:tasks.workunit.client.0.smithi102.stdout:5/60: fsync de/f15 0 2024-05-13T22:47:49.839 INFO:tasks.workunit.client.0.smithi102.stdout:5/61: symlink de/d10/l16 0 2024-05-13T22:47:49.840 INFO:tasks.workunit.client.0.smithi102.stdout:5/62: mknod de/c17 0 2024-05-13T22:47:49.843 INFO:tasks.workunit.client.0.smithi102.stdout:5/63: creat de/d10/f18 x:0 0 0 2024-05-13T22:47:49.851 INFO:tasks.workunit.client.0.smithi102.stdout:0/12: rename f0 to d1/f2 0 2024-05-13T22:47:49.858 INFO:tasks.workunit.client.0.smithi102.stdout:4/16: chown l2 182 1 2024-05-13T22:47:49.860 INFO:tasks.workunit.client.0.smithi102.stdout:4/17: mkdir d3 0 2024-05-13T22:47:49.864 INFO:tasks.workunit.client.0.smithi102.stdout:4/18: unlink f0 0 2024-05-13T22:47:49.866 INFO:tasks.workunit.client.0.smithi102.stdout:6/6: dwrite f0 [0,4194304] 0 2024-05-13T22:47:49.871 INFO:tasks.workunit.client.0.smithi102.stdout:4/19: mkdir d3/d4 0 2024-05-13T22:47:49.874 INFO:tasks.workunit.client.0.smithi102.stdout:6/7: link f1 f2 0 2024-05-13T22:47:49.877 INFO:tasks.workunit.client.0.smithi102.stdout:6/8: creat f3 x:0 0 0 2024-05-13T22:47:49.878 INFO:tasks.workunit.client.0.smithi102.stdout:6/9: truncate f0 4201209 0 2024-05-13T22:47:49.882 INFO:tasks.workunit.client.0.smithi102.stdout:4/20: link l2 d3/d4/l5 0 2024-05-13T22:47:49.887 INFO:tasks.workunit.client.0.smithi102.stdout:4/21: mkdir d3/d4/d6 0 2024-05-13T22:47:49.890 INFO:tasks.workunit.client.0.smithi102.stdout:4/22: mknod d3/d4/d6/c7 0 2024-05-13T22:47:49.890 INFO:tasks.workunit.client.0.smithi102.stdout:4/23: dread - no filename 2024-05-13T22:47:49.890 INFO:tasks.workunit.client.0.smithi102.stdout:4/24: chown d3 3 1 2024-05-13T22:47:49.892 INFO:tasks.workunit.client.0.smithi102.stdout:4/25: creat d3/f8 x:0 0 0 2024-05-13T22:47:49.894 INFO:tasks.workunit.client.0.smithi102.stdout:4/26: rmdir d3/d4 39 2024-05-13T22:47:49.894 INFO:tasks.workunit.client.0.smithi102.stdout:4/27: fdatasync d3/f8 0 2024-05-13T22:47:49.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:49 smithi130 ceph-mon[36591]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:47:49.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:49 smithi130 ceph-mon[36591]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:47:49.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:49 smithi130 ceph-mon[36591]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:47:49.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:49 smithi130 ceph-mon[36591]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:47:49.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:49 smithi130 ceph-mon[36591]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:47:49.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:49 smithi130 ceph-mon[36591]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:47:49.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:49 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:49.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:49 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:49.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:49 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:49.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:49 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:49.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:49 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:49.914 INFO:tasks.workunit.client.0.smithi102.stdout:5/64: dwrite f6 [0,4194304] 0 2024-05-13T22:47:49.918 INFO:tasks.workunit.client.0.smithi102.stdout:5/65: dread f6 [0,4194304] 0 2024-05-13T22:47:49.921 INFO:tasks.workunit.client.0.smithi102.stdout:5/66: write f9 [2662731,92973] 0 2024-05-13T22:47:49.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:49 smithi102 ceph-mon[28599]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:47:49.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:49 smithi102 ceph-mon[28599]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:47:49.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:49 smithi102 ceph-mon[28599]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:47:49.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:49 smithi102 ceph-mon[28599]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:47:49.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:49 smithi102 ceph-mon[28599]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:47:49.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:49 smithi102 ceph-mon[28599]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:47:49.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:49 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:49.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:49 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:49.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:49 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:49.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:49 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:49.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:49 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:47:49.928 INFO:tasks.workunit.client.0.smithi102.stdout:4/28: creat d3/d4/d6/f9 x:0 0 0 2024-05-13T22:47:49.929 INFO:tasks.workunit.client.0.smithi102.stdout:4/29: dread - d3/d4/d6/f9 zero size 2024-05-13T22:47:49.929 INFO:tasks.workunit.client.0.smithi102.stdout:6/10: getdents . 0 2024-05-13T22:47:49.930 INFO:tasks.workunit.client.0.smithi102.stdout:6/11: write f0 [836613,22979] 0 2024-05-13T22:47:49.937 INFO:tasks.workunit.client.0.smithi102.stdout:4/30: creat d3/fa x:0 0 0 2024-05-13T22:47:49.941 INFO:tasks.workunit.client.0.smithi102.stdout:4/31: unlink d3/f8 0 2024-05-13T22:47:49.944 INFO:tasks.workunit.client.0.smithi102.stdout:4/32: mkdir d3/db 0 2024-05-13T22:47:49.944 INFO:tasks.workunit.client.0.smithi102.stdout:4/33: dread - d3/fa zero size 2024-05-13T22:47:49.946 INFO:tasks.workunit.client.0.smithi102.stdout:0/13: read d1/f2 [38,36850] 0 2024-05-13T22:47:49.947 INFO:tasks.workunit.client.0.smithi102.stdout:4/34: rename d3/d4/d6/c7 to d3/d4/cc 0 2024-05-13T22:47:49.947 INFO:tasks.workunit.client.0.smithi102.stdout:0/14: rename d1 to d1/d3 22 2024-05-13T22:47:49.950 INFO:tasks.workunit.client.0.smithi102.stdout:4/35: creat d3/d4/fd x:0 0 0 2024-05-13T22:47:49.953 INFO:tasks.workunit.client.0.smithi102.stdout:0/15: creat d1/f4 x:0 0 0 2024-05-13T22:47:49.956 INFO:tasks.workunit.client.0.smithi102.stdout:0/16: creat d1/f5 x:0 0 0 2024-05-13T22:47:49.956 INFO:tasks.workunit.client.0.smithi102.stdout:6/12: write f2 [2523136,94320] 0 2024-05-13T22:47:49.957 INFO:tasks.workunit.client.0.smithi102.stdout:6/13: write f2 [1878040,129625] 0 2024-05-13T22:47:49.962 INFO:tasks.workunit.client.0.smithi102.stdout:6/14: rename f2 to f4 0 2024-05-13T22:47:49.966 INFO:tasks.workunit.client.0.smithi102.stdout:6/15: write f4 [3068957,117115] 0 2024-05-13T22:47:49.967 INFO:tasks.workunit.client.0.smithi102.stdout:6/16: write f0 [4657381,73847] 0 2024-05-13T22:47:49.967 INFO:tasks.workunit.client.0.smithi102.stdout:4/36: rmdir d3 39 2024-05-13T22:47:49.972 INFO:tasks.workunit.client.0.smithi102.stdout:0/17: rmdir d1 39 2024-05-13T22:47:49.973 INFO:tasks.workunit.client.0.smithi102.stdout:6/17: dwrite f1 [0,4194304] 0 2024-05-13T22:47:49.978 INFO:tasks.workunit.client.0.smithi102.stdout:4/37: unlink d3/d4/cc 0 2024-05-13T22:47:49.979 INFO:tasks.workunit.client.0.smithi102.stdout:4/38: dread - d3/d4/d6/f9 zero size 2024-05-13T22:47:49.981 INFO:tasks.workunit.client.0.smithi102.stdout:4/39: dread - d3/d4/d6/f9 zero size 2024-05-13T22:47:49.981 INFO:tasks.workunit.client.0.smithi102.stdout:4/40: dread - d3/d4/fd zero size 2024-05-13T22:47:49.990 INFO:tasks.workunit.client.0.smithi102.stdout:6/18: fdatasync f4 0 2024-05-13T22:47:49.991 INFO:tasks.workunit.client.0.smithi102.stdout:6/19: write f0 [578640,42665] 0 2024-05-13T22:47:49.993 INFO:tasks.workunit.client.0.smithi102.stdout:0/18: dwrite d1/f2 [0,4194304] 0 2024-05-13T22:47:49.995 INFO:tasks.workunit.client.0.smithi102.stdout:6/20: creat f5 x:0 0 0 2024-05-13T22:47:49.996 INFO:tasks.workunit.client.0.smithi102.stdout:6/21: fdatasync f0 0 2024-05-13T22:47:49.997 INFO:tasks.workunit.client.0.smithi102.stdout:6/22: fdatasync f3 0 2024-05-13T22:47:50.000 INFO:tasks.workunit.client.0.smithi102.stdout:0/19: dwrite d1/f2 [0,4194304] 0 2024-05-13T22:47:50.001 INFO:tasks.workunit.client.0.smithi102.stdout:6/23: creat f6 x:0 0 0 2024-05-13T22:47:50.005 INFO:tasks.workunit.client.0.smithi102.stdout:0/20: link d1/f2 d1/f6 0 2024-05-13T22:47:50.006 INFO:tasks.workunit.client.0.smithi102.stdout:0/21: chown d1/f2 3178 1 2024-05-13T22:47:50.007 INFO:tasks.workunit.client.0.smithi102.stdout:0/22: mkdir d1/d7 0 2024-05-13T22:47:50.008 INFO:tasks.workunit.client.0.smithi102.stdout:0/23: chown d1/f4 195172658 1 2024-05-13T22:47:50.015 INFO:tasks.workunit.client.0.smithi102.stdout:0/24: dwrite d1/f4 [0,4194304] 0 2024-05-13T22:47:50.018 INFO:tasks.workunit.client.0.smithi102.stdout:0/25: symlink d1/l8 0 2024-05-13T22:47:50.018 INFO:tasks.workunit.client.0.smithi102.stdout:0/26: chown d1 0 1 2024-05-13T22:47:50.018 INFO:tasks.workunit.client.0.smithi102.stdout:0/27: chown d1/l8 20 1 2024-05-13T22:47:50.022 INFO:tasks.workunit.client.0.smithi102.stdout:6/24: getdents . 0 2024-05-13T22:47:50.024 INFO:tasks.workunit.client.0.smithi102.stdout:0/28: creat d1/f9 x:0 0 0 2024-05-13T22:47:50.031 INFO:tasks.workunit.client.0.smithi102.stdout:6/25: dwrite f4 [4194304,4194304] 0 2024-05-13T22:47:50.033 INFO:tasks.workunit.client.0.smithi102.stdout:0/29: dwrite d1/f5 [0,4194304] 0 2024-05-13T22:47:50.034 INFO:tasks.workunit.client.0.smithi102.stdout:0/30: stat d1/f6 0 2024-05-13T22:47:50.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:50 smithi130 ceph-mon[36591]: Reconfiguring prometheus.smithi102 (dependencies changed)... 2024-05-13T22:47:50.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:50 smithi130 ceph-mon[36591]: pgmap v12: 65 pgs: 65 active+clean; 166 MiB data, 814 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 506 op/s 2024-05-13T22:47:50.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:50 smithi130 ceph-mon[36591]: Reconfiguring daemon prometheus.smithi102 on smithi102 2024-05-13T22:47:50.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:50 smithi102 ceph-mon[28599]: Reconfiguring prometheus.smithi102 (dependencies changed)... 2024-05-13T22:47:50.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:50 smithi102 ceph-mon[28599]: pgmap v12: 65 pgs: 65 active+clean; 166 MiB data, 814 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 506 op/s 2024-05-13T22:47:50.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:50 smithi102 ceph-mon[28599]: Reconfiguring daemon prometheus.smithi102 on smithi102 2024-05-13T22:47:52.491 INFO:tasks.workunit.client.0.smithi102.stdout:7/17: sync 2024-05-13T22:47:52.492 INFO:tasks.workunit.client.0.smithi102.stdout:9/17: sync 2024-05-13T22:47:52.492 INFO:tasks.workunit.client.0.smithi102.stdout:4/41: sync 2024-05-13T22:47:52.492 INFO:tasks.workunit.client.0.smithi102.stdout:9/18: fsync - no filename 2024-05-13T22:47:52.492 INFO:tasks.workunit.client.0.smithi102.stdout:0/31: sync 2024-05-13T22:47:52.492 INFO:tasks.workunit.client.0.smithi102.stdout:8/5: sync 2024-05-13T22:47:52.492 INFO:tasks.workunit.client.0.smithi102.stdout:8/6: link - no file 2024-05-13T22:47:52.492 INFO:tasks.workunit.client.0.smithi102.stdout:8/7: dwrite - no filename 2024-05-13T22:47:52.493 INFO:tasks.workunit.client.0.smithi102.stdout:8/8: truncate - no filename 2024-05-13T22:47:52.493 INFO:tasks.workunit.client.0.smithi102.stdout:8/9: truncate - no filename 2024-05-13T22:47:52.493 INFO:tasks.workunit.client.0.smithi102.stdout:2/16: sync 2024-05-13T22:47:52.493 INFO:tasks.workunit.client.0.smithi102.stdout:2/17: dread - no filename 2024-05-13T22:47:52.493 INFO:tasks.workunit.client.0.smithi102.stdout:5/67: sync 2024-05-13T22:47:52.493 INFO:tasks.workunit.client.0.smithi102.stdout:3/17: sync 2024-05-13T22:47:52.493 INFO:tasks.workunit.client.0.smithi102.stdout:1/28: sync 2024-05-13T22:47:52.494 INFO:tasks.workunit.client.0.smithi102.stdout:6/26: sync 2024-05-13T22:47:52.494 INFO:tasks.workunit.client.0.smithi102.stdout:3/18: write f1 [1229427,50406] 0 2024-05-13T22:47:52.494 INFO:tasks.workunit.client.0.smithi102.stdout:1/29: write f0 [4854370,31659] 0 2024-05-13T22:47:52.496 INFO:tasks.workunit.client.0.smithi102.stdout:7/18: dread f5 [0,4194304] 0 2024-05-13T22:47:52.498 INFO:tasks.workunit.client.0.smithi102.stdout:7/19: write f3 [945744,68248] 0 2024-05-13T22:47:52.501 INFO:tasks.workunit.client.0.smithi102.stdout:5/68: dwrite f0 [0,4194304] 0 2024-05-13T22:47:52.501 INFO:tasks.workunit.client.0.smithi102.stdout:5/69: fsync de/f11 0 2024-05-13T22:47:52.576 INFO:tasks.workunit.client.0.smithi102.stdout:9/19: creat d1/f7 x:0 0 0 2024-05-13T22:47:52.576 INFO:tasks.workunit.client.0.smithi102.stdout:9/20: write d1/f7 [112347,106350] 0 2024-05-13T22:47:52.582 INFO:tasks.workunit.client.0.smithi102.stdout:2/18: rename l4 to l6 0 2024-05-13T22:47:52.582 INFO:tasks.workunit.client.0.smithi102.stdout:2/19: dread - no filename 2024-05-13T22:47:52.584 INFO:tasks.workunit.client.0.smithi102.stdout:9/21: dwrite d1/f7 [0,4194304] 0 2024-05-13T22:47:52.585 INFO:tasks.workunit.client.0.smithi102.stdout:6/27: creat f7 x:0 0 0 2024-05-13T22:47:52.585 INFO:tasks.workunit.client.0.smithi102.stdout:6/28: truncate f7 698394 0 2024-05-13T22:47:52.586 INFO:tasks.workunit.client.0.smithi102.stdout:4/42: mkdir d3/d4/d6/de 0 2024-05-13T22:47:52.589 INFO:tasks.workunit.client.0.smithi102.stdout:3/19: creat f5 x:0 0 0 2024-05-13T22:47:52.592 INFO:tasks.workunit.client.0.smithi102.stdout:0/32: rename d1/f2 to d1/d7/fa 0 2024-05-13T22:47:52.594 INFO:tasks.workunit.client.0.smithi102.stdout:6/29: dwrite f4 [4194304,4194304] 0 2024-05-13T22:47:52.594 INFO:tasks.workunit.client.0.smithi102.stdout:0/33: chown d1/d7/fa 3660 1 2024-05-13T22:47:52.594 INFO:tasks.workunit.client.0.smithi102.stdout:1/30: symlink d2/d4/d5/l8 0 2024-05-13T22:47:52.594 INFO:tasks.workunit.client.0.smithi102.stdout:0/34: fdatasync d1/f4 0 2024-05-13T22:47:52.595 INFO:tasks.workunit.client.0.smithi102.stdout:1/31: write f0 [5216547,47345] 0 2024-05-13T22:47:52.596 INFO:tasks.workunit.client.0.smithi102.stdout:1/32: rename d2 to d2/d4/d5/d9 22 2024-05-13T22:47:52.598 INFO:tasks.workunit.client.0.smithi102.stdout:7/20: mkdir d6/d9 0 2024-05-13T22:47:52.604 INFO:tasks.workunit.client.0.smithi102.stdout:0/35: dwrite d1/f4 [0,4194304] 0 2024-05-13T22:47:52.606 INFO:tasks.workunit.client.0.smithi102.stdout:1/33: dwrite f0 [0,4194304] 0 2024-05-13T22:47:52.608 INFO:tasks.workunit.client.0.smithi102.stdout:9/22: unlink d1/l5 0 2024-05-13T22:47:52.612 INFO:tasks.workunit.client.0.smithi102.stdout:4/43: creat d3/d4/ff x:0 0 0 2024-05-13T22:47:52.614 INFO:tasks.workunit.client.0.smithi102.stdout:0/36: dwrite d1/f5 [0,4194304] 0 2024-05-13T22:47:52.617 INFO:tasks.workunit.client.0.smithi102.stdout:3/20: rename c0 to c6 0 2024-05-13T22:47:52.625 INFO:tasks.workunit.client.0.smithi102.stdout:8/10: rmdir d0 0 2024-05-13T22:47:52.626 INFO:tasks.workunit.client.0.smithi102.stdout:8/11: chown . 3 1 2024-05-13T22:47:52.630 INFO:tasks.workunit.client.0.smithi102.stdout:1/34: unlink f0 0 2024-05-13T22:47:52.632 INFO:tasks.workunit.client.0.smithi102.stdout:9/23: symlink d1/l8 0 2024-05-13T22:47:52.636 INFO:tasks.workunit.client.0.smithi102.stdout:0/37: creat d1/d7/fb x:0 0 0 2024-05-13T22:47:52.637 INFO:tasks.workunit.client.0.smithi102.stdout:0/38: dread - d1/f9 zero size 2024-05-13T22:47:52.638 INFO:tasks.workunit.client.0.smithi102.stdout:3/21: unlink f1 0 2024-05-13T22:47:52.641 INFO:tasks.workunit.client.0.smithi102.stdout:7/21: symlink d6/d9/la 0 2024-05-13T22:47:52.641 INFO:tasks.workunit.client.0.smithi102.stdout:0/39: dread d1/d7/fa [0,4194304] 0 2024-05-13T22:47:52.642 INFO:tasks.workunit.client.0.smithi102.stdout:8/12: creat f1 x:0 0 0 2024-05-13T22:47:52.646 INFO:tasks.workunit.client.0.smithi102.stdout:0/40: dread d1/f6 [0,4194304] 0 2024-05-13T22:47:52.647 INFO:tasks.workunit.client.0.smithi102.stdout:0/41: truncate d1/f9 242731 0 2024-05-13T22:47:52.647 INFO:tasks.workunit.client.0.smithi102.stdout:1/35: creat d2/d4/d5/fa x:0 0 0 2024-05-13T22:47:52.648 INFO:tasks.workunit.client.0.smithi102.stdout:9/24: mknod d1/c9 0 2024-05-13T22:47:52.649 INFO:tasks.workunit.client.0.smithi102.stdout:0/42: dread d1/f9 [0,4194304] 0 2024-05-13T22:47:52.652 INFO:tasks.workunit.client.0.smithi102.stdout:3/22: creat f7 x:0 0 0 2024-05-13T22:47:52.655 INFO:tasks.workunit.client.0.smithi102.stdout:7/22: rename f0 to d6/d9/fb 0 2024-05-13T22:47:52.657 INFO:tasks.workunit.client.0.smithi102.stdout:8/13: mkdir d2 0 2024-05-13T22:47:52.659 INFO:tasks.workunit.client.0.smithi102.stdout:8/14: truncate f1 444844 0 2024-05-13T22:47:52.659 INFO:tasks.workunit.client.0.smithi102.stdout:3/23: dwrite f5 [0,4194304] 0 2024-05-13T22:47:52.659 INFO:tasks.workunit.client.0.smithi102.stdout:8/15: stat f1 0 2024-05-13T22:47:52.660 INFO:tasks.workunit.client.0.smithi102.stdout:3/24: write f7 [480606,64706] 0 2024-05-13T22:47:52.660 INFO:tasks.workunit.client.0.smithi102.stdout:8/16: chown f1 5 1 2024-05-13T22:47:52.661 INFO:tasks.workunit.client.0.smithi102.stdout:8/17: read f1 [443053,2367] 0 2024-05-13T22:47:52.662 INFO:tasks.workunit.client.0.smithi102.stdout:1/36: creat d2/d4/fb x:0 0 0 2024-05-13T22:47:52.662 INFO:tasks.workunit.client.0.smithi102.stdout:1/37: dread - d2/d4/d5/fa zero size 2024-05-13T22:47:52.666 INFO:tasks.workunit.client.0.smithi102.stdout:0/43: mknod d1/cc 0 2024-05-13T22:47:52.668 INFO:tasks.workunit.client.0.smithi102.stdout:4/44: creat d3/f10 x:0 0 0 2024-05-13T22:47:52.671 INFO:tasks.workunit.client.0.smithi102.stdout:1/38: dwrite d2/d4/fb [0,4194304] 0 2024-05-13T22:47:52.674 INFO:tasks.workunit.client.0.smithi102.stdout:7/23: creat d6/fc x:0 0 0 2024-05-13T22:47:52.685 INFO:tasks.workunit.client.0.smithi102.stdout:1/39: mknod d2/d4/cc 0 2024-05-13T22:47:52.686 INFO:tasks.workunit.client.0.smithi102.stdout:7/24: write f4 [845549,20247] 0 2024-05-13T22:47:52.686 INFO:tasks.workunit.client.0.smithi102.stdout:7/25: chown f5 19 1 2024-05-13T22:47:52.687 INFO:tasks.workunit.client.0.smithi102.stdout:7/26: write f5 [720163,130025] 0 2024-05-13T22:47:52.692 INFO:tasks.workunit.client.0.smithi102.stdout:1/40: creat d2/d4/fd x:0 0 0 2024-05-13T22:47:52.694 INFO:tasks.workunit.client.0.smithi102.stdout:1/41: truncate d2/d4/d5/fa 607657 0 2024-05-13T22:47:52.695 INFO:tasks.workunit.client.0.smithi102.stdout:7/27: dwrite f5 [0,4194304] 0 2024-05-13T22:47:52.698 INFO:tasks.workunit.client.0.smithi102.stdout:7/28: read f3 [894405,107897] 0 2024-05-13T22:47:52.698 INFO:tasks.workunit.client.0.smithi102.stdout:4/45: rename l2 to d3/l11 0 2024-05-13T22:47:52.699 INFO:tasks.workunit.client.0.smithi102.stdout:7/29: read - d6/f8 zero size 2024-05-13T22:47:52.700 INFO:tasks.workunit.client.0.smithi102.stdout:4/46: fdatasync d3/d4/fd 0 2024-05-13T22:47:52.700 INFO:tasks.workunit.client.0.smithi102.stdout:7/30: chown d6/d9 196753 1 2024-05-13T22:47:52.700 INFO:tasks.workunit.client.0.smithi102.stdout:7/31: write f3 [97280,44701] 0 2024-05-13T22:47:52.707 INFO:tasks.workunit.client.0.smithi102.stdout:1/42: link d2/d4/fb d2/fe 0 2024-05-13T22:47:52.709 INFO:tasks.workunit.client.0.smithi102.stdout:1/43: dread d2/d4/d5/fa [0,4194304] 0 2024-05-13T22:47:52.710 INFO:tasks.workunit.client.0.smithi102.stdout:1/44: chown d2/d4/d5/fa 20931 1 2024-05-13T22:47:52.711 INFO:tasks.workunit.client.0.smithi102.stdout:4/47: mknod d3/c12 0 2024-05-13T22:47:52.716 INFO:tasks.workunit.client.0.smithi102.stdout:1/45: unlink d2/d4/fb 0 2024-05-13T22:47:52.719 INFO:tasks.workunit.client.0.smithi102.stdout:4/48: dwrite d3/d4/d6/f9 [0,4194304] 0 2024-05-13T22:47:52.722 INFO:tasks.workunit.client.0.smithi102.stdout:1/46: unlink d2/d4/d5/f6 0 2024-05-13T22:47:52.729 INFO:tasks.workunit.client.0.smithi102.stdout:1/47: creat d2/ff x:0 0 0 2024-05-13T22:47:52.731 INFO:tasks.workunit.client.0.smithi102.stdout:1/48: mkdir d2/d10 0 2024-05-13T22:47:52.734 INFO:tasks.workunit.client.0.smithi102.stdout:1/49: dread d2/d4/d5/fa [0,4194304] 0 2024-05-13T22:47:52.737 INFO:tasks.workunit.client.0.smithi102.stdout:1/50: symlink d2/l11 0 2024-05-13T22:47:52.737 INFO:tasks.workunit.client.0.smithi102.stdout:1/51: read - d2/ff zero size 2024-05-13T22:47:52.740 INFO:tasks.workunit.client.0.smithi102.stdout:1/52: creat d2/d10/f12 x:0 0 0 2024-05-13T22:47:52.867 INFO:tasks.workunit.client.0.smithi102.stdout:9/25: fsync d1/f7 0 2024-05-13T22:47:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:2/20: getdents . 0 2024-05-13T22:47:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:2/21: write - no filename 2024-05-13T22:47:52.870 INFO:tasks.workunit.client.0.smithi102.stdout:2/22: dwrite - no filename 2024-05-13T22:47:52.871 INFO:tasks.workunit.client.0.smithi102.stdout:9/26: creat d1/fa x:0 0 0 2024-05-13T22:47:52.873 INFO:tasks.workunit.client.0.smithi102.stdout:5/70: dwrite f0 [4194304,4194304] 0 2024-05-13T22:47:52.873 INFO:tasks.workunit.client.0.smithi102.stdout:2/23: mkdir d7 0 2024-05-13T22:47:52.874 INFO:tasks.workunit.client.0.smithi102.stdout:6/30: getdents . 0 2024-05-13T22:47:52.876 INFO:tasks.workunit.client.0.smithi102.stdout:6/31: read f7 [424635,81496] 0 2024-05-13T22:47:52.879 INFO:tasks.workunit.client.0.smithi102.stdout:5/71: dread fd [0,4194304] 0 2024-05-13T22:47:52.880 INFO:tasks.workunit.client.0.smithi102.stdout:5/72: write f0 [1322457,15805] 0 2024-05-13T22:47:52.883 INFO:tasks.workunit.client.0.smithi102.stdout:2/24: rename l6 to d7/l8 0 2024-05-13T22:47:52.883 INFO:tasks.workunit.client.0.smithi102.stdout:2/25: write - no filename 2024-05-13T22:47:52.885 INFO:tasks.workunit.client.0.smithi102.stdout:6/32: dwrite f4 [0,4194304] 0 2024-05-13T22:47:52.885 INFO:tasks.workunit.client.0.smithi102.stdout:6/33: readlink - no filename 2024-05-13T22:47:52.888 INFO:tasks.workunit.client.0.smithi102.stdout:9/27: mkdir d1/d6/db 0 2024-05-13T22:47:52.889 INFO:tasks.workunit.client.0.smithi102.stdout:9/28: fdatasync d1/f7 0 2024-05-13T22:47:52.889 INFO:tasks.workunit.client.0.smithi102.stdout:6/34: dread f4 [4194304,4194304] 0 2024-05-13T22:47:52.890 INFO:tasks.workunit.client.0.smithi102.stdout:6/35: rmdir - no directory 2024-05-13T22:47:52.891 INFO:tasks.workunit.client.0.smithi102.stdout:9/29: write d1/fa [807918,128746] 0 2024-05-13T22:47:52.896 INFO:tasks.workunit.client.0.smithi102.stdout:2/26: mkdir d7/d9 0 2024-05-13T22:47:52.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:52 smithi130 ceph-mon[36591]: pgmap v13: 65 pgs: 65 active+clean; 166 MiB data, 814 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-05-13T22:47:52.901 INFO:tasks.workunit.client.0.smithi102.stdout:6/36: creat f8 x:0 0 0 2024-05-13T22:47:52.902 INFO:tasks.workunit.client.0.smithi102.stdout:6/37: rmdir - no directory 2024-05-13T22:47:52.902 INFO:tasks.workunit.client.0.smithi102.stdout:6/38: chown f3 1667003 1 2024-05-13T22:47:52.903 INFO:tasks.workunit.client.0.smithi102.stdout:9/30: creat d1/fc x:0 0 0 2024-05-13T22:47:52.907 INFO:tasks.workunit.client.0.smithi102.stdout:3/25: rename c6 to c8 0 2024-05-13T22:47:52.909 INFO:tasks.workunit.client.0.smithi102.stdout:9/31: symlink d1/d6/ld 0 2024-05-13T22:47:52.910 INFO:tasks.workunit.client.0.smithi102.stdout:5/73: link lc de/l19 0 2024-05-13T22:47:52.913 INFO:tasks.workunit.client.0.smithi102.stdout:3/26: creat f9 x:0 0 0 2024-05-13T22:47:52.913 INFO:tasks.workunit.client.0.smithi102.stdout:8/18: fsync f1 0 2024-05-13T22:47:52.915 INFO:tasks.workunit.client.0.smithi102.stdout:9/32: unlink d1/fc 0 2024-05-13T22:47:52.916 INFO:tasks.workunit.client.0.smithi102.stdout:5/74: symlink de/d10/l1a 0 2024-05-13T22:47:52.917 INFO:tasks.workunit.client.0.smithi102.stdout:5/75: write f9 [894861,73535] 0 2024-05-13T22:47:52.919 INFO:tasks.workunit.client.0.smithi102.stdout:2/27: link c2 d7/d9/ca 0 2024-05-13T22:47:52.919 INFO:tasks.workunit.client.0.smithi102.stdout:2/28: stat d7/d9 0 2024-05-13T22:47:52.921 INFO:tasks.workunit.client.0.smithi102.stdout:3/27: creat fa x:0 0 0 2024-05-13T22:47:52.925 INFO:tasks.workunit.client.0.smithi102.stdout:5/76: symlink de/d10/l1b 0 2024-05-13T22:47:52.927 INFO:tasks.workunit.client.0.smithi102.stdout:8/19: symlink d2/l3 0 2024-05-13T22:47:52.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:52 smithi102 ceph-mon[28599]: pgmap v13: 65 pgs: 65 active+clean; 166 MiB data, 814 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-05-13T22:47:52.929 INFO:tasks.workunit.client.0.smithi102.stdout:3/28: symlink lb 0 2024-05-13T22:47:52.929 INFO:tasks.workunit.client.0.smithi102.stdout:3/29: rmdir - no directory 2024-05-13T22:47:52.933 INFO:tasks.workunit.client.0.smithi102.stdout:2/29: creat d7/d9/fb x:0 0 0 2024-05-13T22:47:52.934 INFO:tasks.workunit.client.0.smithi102.stdout:2/30: truncate d7/d9/fb 406892 0 2024-05-13T22:47:52.934 INFO:tasks.workunit.client.0.smithi102.stdout:5/77: creat de/d10/f1c x:0 0 0 2024-05-13T22:47:52.936 INFO:tasks.workunit.client.0.smithi102.stdout:2/31: dread d7/d9/fb [0,4194304] 0 2024-05-13T22:47:52.937 INFO:tasks.workunit.client.0.smithi102.stdout:3/30: rename lb to lc 0 2024-05-13T22:47:52.938 INFO:tasks.workunit.client.0.smithi102.stdout:2/32: read d7/d9/fb [11886,19245] 0 2024-05-13T22:47:52.938 INFO:tasks.workunit.client.0.smithi102.stdout:3/31: write f9 [577833,51620] 0 2024-05-13T22:47:52.942 INFO:tasks.workunit.client.0.smithi102.stdout:0/44: truncate d1/f6 2139362 0 2024-05-13T22:47:52.946 INFO:tasks.workunit.client.0.smithi102.stdout:5/78: dwrite fd [0,4194304] 0 2024-05-13T22:47:52.947 INFO:tasks.workunit.client.0.smithi102.stdout:3/32: rename f9 to fd 0 2024-05-13T22:47:52.956 INFO:tasks.workunit.client.0.smithi102.stdout:3/33: dwrite fa [0,4194304] 0 2024-05-13T22:47:52.956 INFO:tasks.workunit.client.0.smithi102.stdout:3/34: rmdir - no directory 2024-05-13T22:47:52.960 INFO:tasks.workunit.client.0.smithi102.stdout:5/79: creat de/f1d x:0 0 0 2024-05-13T22:47:52.961 INFO:tasks.workunit.client.0.smithi102.stdout:5/80: write de/f11 [323827,9525] 0 2024-05-13T22:47:52.968 INFO:tasks.workunit.client.0.smithi102.stdout:3/35: mkdir de 0 2024-05-13T22:47:52.969 INFO:tasks.workunit.client.0.smithi102.stdout:5/81: creat de/f1e x:0 0 0 2024-05-13T22:47:52.974 INFO:tasks.workunit.client.0.smithi102.stdout:5/82: creat de/d10/f1f x:0 0 0 2024-05-13T22:47:52.975 INFO:tasks.workunit.client.0.smithi102.stdout:3/36: dwrite f7 [0,4194304] 0 2024-05-13T22:47:52.976 INFO:tasks.workunit.client.0.smithi102.stdout:1/53: rmdir d2 39 2024-05-13T22:47:52.980 INFO:tasks.workunit.client.0.smithi102.stdout:5/83: rename de/d10/c12 to de/c20 0 2024-05-13T22:47:52.987 INFO:tasks.workunit.client.0.smithi102.stdout:3/37: creat de/ff x:0 0 0 2024-05-13T22:47:52.989 INFO:tasks.workunit.client.0.smithi102.stdout:5/84: creat de/f21 x:0 0 0 2024-05-13T22:47:52.991 INFO:tasks.workunit.client.0.smithi102.stdout:5/85: write de/f13 [15220,28259] 0 2024-05-13T22:47:52.991 INFO:tasks.workunit.client.0.smithi102.stdout:1/54: write d2/ff [416980,48720] 0 2024-05-13T22:47:52.991 INFO:tasks.workunit.client.0.smithi102.stdout:5/86: chown de/f11 252764155 1 2024-05-13T22:47:52.992 INFO:tasks.workunit.client.0.smithi102.stdout:3/38: mkdir de/d10 0 2024-05-13T22:47:53.019 INFO:tasks.workunit.client.0.smithi102.stdout:2/33: getdents d7 0 2024-05-13T22:47:53.033 INFO:tasks.workunit.client.0.smithi102.stdout:8/20: getdents d2 0 2024-05-13T22:47:53.034 INFO:tasks.workunit.client.0.smithi102.stdout:8/21: dread f1 [0,4194304] 0 2024-05-13T22:47:53.036 INFO:tasks.workunit.client.0.smithi102.stdout:8/22: rename f1 to d2/f4 0 2024-05-13T22:47:53.037 INFO:tasks.workunit.client.0.smithi102.stdout:9/33: truncate d1/fa 455375 0 2024-05-13T22:47:53.039 INFO:tasks.workunit.client.0.smithi102.stdout:2/34: rmdir d7/d9 39 2024-05-13T22:47:53.039 INFO:tasks.workunit.client.0.smithi102.stdout:2/35: chown l1 35468 1 2024-05-13T22:47:53.041 INFO:tasks.workunit.client.0.smithi102.stdout:9/34: creat d1/d6/fe x:0 0 0 2024-05-13T22:47:53.042 INFO:tasks.workunit.client.0.smithi102.stdout:9/35: stat d1/f7 0 2024-05-13T22:47:53.043 INFO:tasks.workunit.client.0.smithi102.stdout:3/39: readlink lc 0 2024-05-13T22:47:53.044 INFO:tasks.workunit.client.0.smithi102.stdout:9/36: chown d1/d6/ld 47 1 2024-05-13T22:47:53.045 INFO:tasks.workunit.client.0.smithi102.stdout:3/40: chown l3 17093808 1 2024-05-13T22:47:53.045 INFO:tasks.workunit.client.0.smithi102.stdout:8/23: dwrite d2/f4 [0,4194304] 0 2024-05-13T22:47:53.046 INFO:tasks.workunit.client.0.smithi102.stdout:8/24: read d2/f4 [1830427,42996] 0 2024-05-13T22:47:53.047 INFO:tasks.workunit.client.0.smithi102.stdout:3/41: truncate de/ff 350792 0 2024-05-13T22:47:53.047 INFO:tasks.workunit.client.0.smithi102.stdout:3/42: chown fa 258 1 2024-05-13T22:47:53.051 INFO:tasks.workunit.client.0.smithi102.stdout:0/45: rename d1/d7/fa to d1/d7/fd 0 2024-05-13T22:47:53.052 INFO:tasks.workunit.client.0.smithi102.stdout:2/36: symlink d7/d9/lc 0 2024-05-13T22:47:53.054 INFO:tasks.workunit.client.0.smithi102.stdout:9/37: mknod d1/d6/cf 0 2024-05-13T22:47:53.064 INFO:tasks.workunit.client.0.smithi102.stdout:9/38: dwrite d1/d6/fe [0,4194304] 0 2024-05-13T22:47:53.065 INFO:tasks.workunit.client.0.smithi102.stdout:0/46: dread d1/f9 [0,4194304] 0 2024-05-13T22:47:53.066 INFO:tasks.workunit.client.0.smithi102.stdout:9/39: rename d1 to d1/d10 22 2024-05-13T22:47:53.067 INFO:tasks.workunit.client.0.smithi102.stdout:2/37: dread d7/d9/fb [0,4194304] 0 2024-05-13T22:47:53.068 INFO:tasks.workunit.client.0.smithi102.stdout:0/47: write d1/f5 [1813852,83319] 0 2024-05-13T22:47:53.071 INFO:tasks.workunit.client.0.smithi102.stdout:3/43: dwrite fd [0,4194304] 0 2024-05-13T22:47:53.074 INFO:tasks.workunit.client.0.smithi102.stdout:9/40: dread d1/d6/fe [0,4194304] 0 2024-05-13T22:47:53.075 INFO:tasks.workunit.client.0.smithi102.stdout:9/41: readlink d1/l8 0 2024-05-13T22:47:53.079 INFO:tasks.workunit.client.0.smithi102.stdout:9/42: dread d1/d6/fe [0,4194304] 0 2024-05-13T22:47:53.087 INFO:tasks.workunit.client.0.smithi102.stdout:5/87: truncate f0 717683 0 2024-05-13T22:47:53.089 INFO:tasks.workunit.client.0.smithi102.stdout:1/55: chown d2/d4/d5 1741819922 1 2024-05-13T22:47:53.094 INFO:tasks.workunit.client.0.smithi102.stdout:9/43: creat d1/d6/f11 x:0 0 0 2024-05-13T22:47:53.098 INFO:tasks.workunit.client.0.smithi102.stdout:3/44: creat de/d10/f11 x:0 0 0 2024-05-13T22:47:53.098 INFO:tasks.workunit.client.0.smithi102.stdout:3/45: stat f2 0 2024-05-13T22:47:53.100 INFO:tasks.workunit.client.0.smithi102.stdout:5/88: symlink de/d10/l22 0 2024-05-13T22:47:53.106 INFO:tasks.workunit.client.0.smithi102.stdout:9/44: creat d1/d6/f12 x:0 0 0 2024-05-13T22:47:53.109 INFO:tasks.workunit.client.0.smithi102.stdout:5/89: dwrite de/f21 [0,4194304] 0 2024-05-13T22:47:53.114 INFO:tasks.workunit.client.0.smithi102.stdout:8/25: dwrite d2/f4 [4194304,4194304] 0 2024-05-13T22:47:53.116 INFO:tasks.workunit.client.0.smithi102.stdout:9/45: dread d1/d6/fe [0,4194304] 0 2024-05-13T22:47:53.120 INFO:tasks.workunit.client.0.smithi102.stdout:8/26: dread d2/f4 [0,4194304] 0 2024-05-13T22:47:53.121 INFO:tasks.workunit.client.0.smithi102.stdout:0/48: getdents d1/d7 0 2024-05-13T22:47:53.125 INFO:tasks.workunit.client.0.smithi102.stdout:8/27: dread d2/f4 [4194304,4194304] 0 2024-05-13T22:47:53.125 INFO:tasks.workunit.client.0.smithi102.stdout:5/90: symlink de/l23 0 2024-05-13T22:47:53.126 INFO:tasks.workunit.client.0.smithi102.stdout:8/28: chown d2 9520320 1 2024-05-13T22:47:53.126 INFO:tasks.workunit.client.0.smithi102.stdout:5/91: write de/f21 [1597053,80978] 0 2024-05-13T22:47:53.128 INFO:tasks.workunit.client.0.smithi102.stdout:3/46: link de/ff de/d10/f12 0 2024-05-13T22:47:53.140 INFO:tasks.workunit.client.0.smithi102.stdout:2/38: dwrite d7/d9/fb [0,4194304] 0 2024-05-13T22:47:53.141 INFO:tasks.workunit.client.0.smithi102.stdout:8/29: mknod d2/c5 0 2024-05-13T22:47:53.142 INFO:tasks.workunit.client.0.smithi102.stdout:8/30: stat d2/l3 0 2024-05-13T22:47:53.143 INFO:tasks.workunit.client.0.smithi102.stdout:5/92: creat de/d10/f24 x:0 0 0 2024-05-13T22:47:53.148 INFO:tasks.workunit.client.0.smithi102.stdout:1/56: creat d2/f13 x:0 0 0 2024-05-13T22:47:53.153 INFO:tasks.workunit.client.0.smithi102.stdout:9/46: truncate d1/f7 2513778 0 2024-05-13T22:47:53.160 INFO:tasks.workunit.client.0.smithi102.stdout:8/31: rename d2/l3 to d2/l6 0 2024-05-13T22:47:53.162 INFO:tasks.workunit.client.0.smithi102.stdout:5/93: mkdir de/d25 0 2024-05-13T22:47:53.168 INFO:tasks.workunit.client.0.smithi102.stdout:3/47: rmdir de 39 2024-05-13T22:47:53.175 INFO:tasks.workunit.client.0.smithi102.stdout:5/94: rename de/f15 to de/d25/f26 0 2024-05-13T22:47:53.177 INFO:tasks.workunit.client.0.smithi102.stdout:8/32: dwrite d2/f4 [0,4194304] 0 2024-05-13T22:47:53.187 INFO:tasks.workunit.client.0.smithi102.stdout:5/95: rename de/lf to de/l27 0 2024-05-13T22:47:53.192 INFO:tasks.workunit.client.0.smithi102.stdout:8/33: mkdir d2/d7 0 2024-05-13T22:47:53.192 INFO:tasks.workunit.client.0.smithi102.stdout:2/39: write d7/d9/fb [4235293,67805] 0 2024-05-13T22:47:53.194 INFO:tasks.workunit.client.0.smithi102.stdout:2/40: write d7/d9/fb [2501297,13097] 0 2024-05-13T22:47:53.194 INFO:tasks.workunit.client.0.smithi102.stdout:3/48: symlink de/l13 0 2024-05-13T22:47:53.196 INFO:tasks.workunit.client.0.smithi102.stdout:1/57: fdatasync d2/d4/d5/fa 0 2024-05-13T22:47:53.198 INFO:tasks.workunit.client.0.smithi102.stdout:5/96: mkdir de/d10/d28 0 2024-05-13T22:47:53.202 INFO:tasks.workunit.client.0.smithi102.stdout:3/49: creat de/d10/f14 x:0 0 0 2024-05-13T22:47:53.207 INFO:tasks.workunit.client.0.smithi102.stdout:3/50: mknod de/d10/c15 0 2024-05-13T22:47:53.210 INFO:tasks.workunit.client.0.smithi102.stdout:5/97: symlink de/d10/d28/l29 0 2024-05-13T22:47:53.210 INFO:tasks.workunit.client.0.smithi102.stdout:0/49: dwrite d1/d7/fd [0,4194304] 0 2024-05-13T22:47:53.213 INFO:tasks.workunit.client.0.smithi102.stdout:0/50: write d1/f6 [2751768,66980] 0 2024-05-13T22:47:53.214 INFO:tasks.workunit.client.0.smithi102.stdout:0/51: chown d1/cc 10610 1 2024-05-13T22:47:53.217 INFO:tasks.workunit.client.0.smithi102.stdout:3/51: dwrite de/d10/f11 [0,4194304] 0 2024-05-13T22:47:53.217 INFO:tasks.workunit.client.0.smithi102.stdout:5/98: mkdir de/d25/d2a 0 2024-05-13T22:47:53.221 INFO:tasks.workunit.client.0.smithi102.stdout:0/52: dread d1/f5 [0,4194304] 0 2024-05-13T22:47:53.223 INFO:tasks.workunit.client.0.smithi102.stdout:3/52: dread fd [0,4194304] 0 2024-05-13T22:47:53.224 INFO:tasks.workunit.client.0.smithi102.stdout:9/47: truncate d1/f7 2735697 0 2024-05-13T22:47:53.230 INFO:tasks.workunit.client.0.smithi102.stdout:0/53: dwrite d1/f6 [0,4194304] 0 2024-05-13T22:47:53.231 INFO:tasks.workunit.client.0.smithi102.stdout:0/54: write d1/d7/fd [1681464,2580] 0 2024-05-13T22:47:53.232 INFO:tasks.workunit.client.0.smithi102.stdout:0/55: write d1/d7/fd [3889126,43746] 0 2024-05-13T22:47:53.236 INFO:tasks.workunit.client.0.smithi102.stdout:3/53: mknod de/c16 0 2024-05-13T22:47:53.238 INFO:tasks.workunit.client.0.smithi102.stdout:9/48: creat d1/d6/f13 x:0 0 0 2024-05-13T22:47:53.248 INFO:tasks.workunit.client.0.smithi102.stdout:0/56: mkdir d1/de 0 2024-05-13T22:47:53.250 INFO:tasks.workunit.client.0.smithi102.stdout:3/54: symlink de/l17 0 2024-05-13T22:47:53.251 INFO:tasks.workunit.client.0.smithi102.stdout:0/57: dread d1/f4 [0,4194304] 0 2024-05-13T22:47:53.252 INFO:tasks.workunit.client.0.smithi102.stdout:9/49: mkdir d1/d14 0 2024-05-13T22:47:53.255 INFO:tasks.workunit.client.0.smithi102.stdout:0/58: dread d1/f4 [0,4194304] 0 2024-05-13T22:47:53.255 INFO:tasks.workunit.client.0.smithi102.stdout:0/59: chown d1 4 1 2024-05-13T22:47:53.258 INFO:tasks.workunit.client.0.smithi102.stdout:3/55: mknod de/d10/c18 0 2024-05-13T22:47:53.258 INFO:tasks.workunit.client.0.smithi102.stdout:3/56: dread - de/d10/f14 zero size 2024-05-13T22:47:53.261 INFO:tasks.workunit.client.0.smithi102.stdout:5/99: getdents de/d25 0 2024-05-13T22:47:53.262 INFO:tasks.workunit.client.0.smithi102.stdout:5/100: chown de/f21 3333795 1 2024-05-13T22:47:53.262 INFO:tasks.workunit.client.0.smithi102.stdout:3/57: dread de/d10/f11 [0,4194304] 0 2024-05-13T22:47:53.263 INFO:tasks.workunit.client.0.smithi102.stdout:5/101: chown f2 105573286 1 2024-05-13T22:47:53.263 INFO:tasks.workunit.client.0.smithi102.stdout:3/58: chown fa 1 1 2024-05-13T22:47:53.263 INFO:tasks.workunit.client.0.smithi102.stdout:0/60: symlink d1/lf 0 2024-05-13T22:47:53.275 INFO:tasks.workunit.client.0.smithi102.stdout:0/61: write d1/f4 [4099901,64245] 0 2024-05-13T22:47:53.278 INFO:tasks.workunit.client.0.smithi102.stdout:9/50: rmdir d1/d14 0 2024-05-13T22:47:53.279 INFO:tasks.workunit.client.0.smithi102.stdout:9/51: dread - d1/d6/f12 zero size 2024-05-13T22:47:53.279 INFO:tasks.workunit.client.0.smithi102.stdout:9/52: dread - d1/d6/f13 zero size 2024-05-13T22:47:53.280 INFO:tasks.workunit.client.0.smithi102.stdout:5/102: creat de/d25/d2a/f2b x:0 0 0 2024-05-13T22:47:53.283 INFO:tasks.workunit.client.0.smithi102.stdout:0/62: symlink d1/l10 0 2024-05-13T22:47:53.283 INFO:tasks.workunit.client.0.smithi102.stdout:0/63: chown d1/d7 6090953 1 2024-05-13T22:47:53.284 INFO:tasks.workunit.client.0.smithi102.stdout:0/64: write d1/f4 [989861,1152] 0 2024-05-13T22:47:53.284 INFO:tasks.workunit.client.0.smithi102.stdout:0/65: read d1/f6 [2895755,67953] 0 2024-05-13T22:47:53.286 INFO:tasks.workunit.client.0.smithi102.stdout:9/53: mknod d1/c15 0 2024-05-13T22:47:53.288 INFO:tasks.workunit.client.0.smithi102.stdout:5/103: mkdir de/d25/d2c 0 2024-05-13T22:47:53.384 INFO:tasks.workunit.client.0.smithi102.stdout:5/104: write de/d10/f1f [695176,56009] 0 2024-05-13T22:47:53.386 INFO:tasks.workunit.client.0.smithi102.stdout:0/66: dwrite d1/f6 [0,4194304] 0 2024-05-13T22:47:53.386 INFO:tasks.workunit.client.0.smithi102.stdout:3/59: write de/d10/f12 [1119919,5205] 0 2024-05-13T22:47:53.387 INFO:tasks.workunit.client.0.smithi102.stdout:9/54: write d1/d6/fe [4020561,105050] 0 2024-05-13T22:47:53.389 INFO:tasks.workunit.client.0.smithi102.stdout:3/60: creat de/f19 x:0 0 0 2024-05-13T22:47:53.390 INFO:tasks.workunit.client.0.smithi102.stdout:0/67: write d1/d7/fd [4690513,44390] 0 2024-05-13T22:47:53.391 INFO:tasks.workunit.client.0.smithi102.stdout:9/55: creat d1/f16 x:0 0 0 2024-05-13T22:47:53.392 INFO:tasks.workunit.client.0.smithi102.stdout:0/68: write d1/d7/fd [5646542,70167] 0 2024-05-13T22:47:53.394 INFO:tasks.workunit.client.0.smithi102.stdout:0/69: write d1/d7/fd [4902118,125453] 0 2024-05-13T22:47:53.395 INFO:tasks.workunit.client.0.smithi102.stdout:5/105: dwrite de/d10/f24 [0,4194304] 0 2024-05-13T22:47:53.399 INFO:tasks.workunit.client.0.smithi102.stdout:0/70: symlink d1/d7/l11 0 2024-05-13T22:47:53.409 INFO:tasks.workunit.client.0.smithi102.stdout:5/106: link l8 de/d25/d2c/l2d 0 2024-05-13T22:47:53.409 INFO:tasks.workunit.client.0.smithi102.stdout:5/107: chown f2 981854524 1 2024-05-13T22:47:53.412 INFO:tasks.workunit.client.0.smithi102.stdout:5/108: creat de/d10/f2e x:0 0 0 2024-05-13T22:47:53.419 INFO:tasks.workunit.client.0.smithi102.stdout:5/109: dwrite de/d10/f1c [0,4194304] 0 2024-05-13T22:47:53.426 INFO:tasks.workunit.client.0.smithi102.stdout:5/110: fsync de/d10/f24 0 2024-05-13T22:47:53.429 INFO:tasks.workunit.client.0.smithi102.stdout:5/111: symlink de/d25/d2c/l2f 0 2024-05-13T22:47:53.430 INFO:tasks.workunit.client.0.smithi102.stdout:5/112: write de/d10/f1f [1754212,37546] 0 2024-05-13T22:47:53.548 INFO:tasks.workunit.client.0.smithi102.stdout:7/32: sync 2024-05-13T22:47:53.548 INFO:tasks.workunit.client.0.smithi102.stdout:7/33: stat d6/fc 0 2024-05-13T22:47:53.549 INFO:tasks.workunit.client.0.smithi102.stdout:3/61: write fd [365234,129609] 0 2024-05-13T22:47:53.551 INFO:tasks.workunit.client.0.smithi102.stdout:7/34: unlink f5 0 2024-05-13T22:47:53.559 INFO:tasks.workunit.client.0.smithi102.stdout:3/62: rename lc to de/l1a 0 2024-05-13T22:47:53.559 INFO:tasks.workunit.client.0.smithi102.stdout:7/35: dwrite f4 [0,4194304] 0 2024-05-13T22:47:53.559 INFO:tasks.workunit.client.0.smithi102.stdout:9/56: write d1/fa [1420477,42939] 0 2024-05-13T22:47:53.561 INFO:tasks.workunit.client.0.smithi102.stdout:7/36: write d6/fc [41659,122298] 0 2024-05-13T22:47:53.561 INFO:tasks.workunit.client.0.smithi102.stdout:0/71: write d1/f5 [3663712,47316] 0 2024-05-13T22:47:53.562 INFO:tasks.workunit.client.0.smithi102.stdout:3/63: creat de/d10/f1b x:0 0 0 2024-05-13T22:47:53.563 INFO:tasks.workunit.client.0.smithi102.stdout:7/37: stat d6/d9/la 0 2024-05-13T22:47:53.563 INFO:tasks.workunit.client.0.smithi102.stdout:0/72: fdatasync d1/d7/fd 0 2024-05-13T22:47:53.564 INFO:tasks.workunit.client.0.smithi102.stdout:3/64: read - de/d10/f14 zero size 2024-05-13T22:47:53.564 INFO:tasks.workunit.client.0.smithi102.stdout:0/73: read d1/f9 [127271,99285] 0 2024-05-13T22:47:53.568 INFO:tasks.workunit.client.0.smithi102.stdout:9/57: dread d1/fa [0,4194304] 0 2024-05-13T22:47:53.570 INFO:tasks.workunit.client.0.smithi102.stdout:3/65: mkdir de/d1c 0 2024-05-13T22:47:53.573 INFO:tasks.workunit.client.0.smithi102.stdout:7/38: dwrite d6/f8 [0,4194304] 0 2024-05-13T22:47:53.576 INFO:tasks.workunit.client.0.smithi102.stdout:0/74: mknod d1/de/c12 0 2024-05-13T22:47:53.579 INFO:tasks.workunit.client.0.smithi102.stdout:9/58: dwrite d1/d6/fe [0,4194304] 0 2024-05-13T22:47:53.582 INFO:tasks.workunit.client.0.smithi102.stdout:7/39: dread f4 [0,4194304] 0 2024-05-13T22:47:53.582 INFO:tasks.workunit.client.0.smithi102.stdout:5/113: truncate f6 897443 0 2024-05-13T22:47:53.583 INFO:tasks.workunit.client.0.smithi102.stdout:5/114: fdatasync de/d10/f2e 0 2024-05-13T22:47:53.585 INFO:tasks.workunit.client.0.smithi102.stdout:0/75: dwrite d1/f4 [0,4194304] 0 2024-05-13T22:47:53.586 INFO:tasks.workunit.client.0.smithi102.stdout:9/59: creat d1/f17 x:0 0 0 2024-05-13T22:47:53.592 INFO:tasks.workunit.client.0.smithi102.stdout:0/76: creat d1/d7/f13 x:0 0 0 2024-05-13T22:47:53.594 INFO:tasks.workunit.client.0.smithi102.stdout:9/60: symlink d1/d6/db/l18 0 2024-05-13T22:47:53.595 INFO:tasks.workunit.client.0.smithi102.stdout:0/77: unlink d1/l8 0 2024-05-13T22:47:53.597 INFO:tasks.workunit.client.0.smithi102.stdout:0/78: dread d1/f9 [0,4194304] 0 2024-05-13T22:47:53.599 INFO:tasks.workunit.client.0.smithi102.stdout:9/61: rename d1/c15 to d1/d6/c19 0 2024-05-13T22:47:53.603 INFO:tasks.workunit.client.0.smithi102.stdout:9/62: symlink d1/l1a 0 2024-05-13T22:47:53.603 INFO:tasks.workunit.client.0.smithi102.stdout:7/40: dread f3 [0,4194304] 0 2024-05-13T22:47:53.606 INFO:tasks.workunit.client.0.smithi102.stdout:7/41: rmdir d6 39 2024-05-13T22:47:53.611 INFO:tasks.workunit.client.0.smithi102.stdout:9/63: dwrite d1/f16 [0,4194304] 0 2024-05-13T22:47:53.614 INFO:tasks.workunit.client.0.smithi102.stdout:7/42: mkdir d6/dd 0 2024-05-13T22:47:53.615 INFO:tasks.workunit.client.0.smithi102.stdout:7/43: read f4 [4151693,5991] 0 2024-05-13T22:47:53.618 INFO:tasks.workunit.client.0.smithi102.stdout:9/64: creat d1/d6/db/f1b x:0 0 0 2024-05-13T22:47:53.620 INFO:tasks.workunit.client.0.smithi102.stdout:7/44: truncate f3 192688 0 2024-05-13T22:47:53.621 INFO:tasks.workunit.client.0.smithi102.stdout:7/45: truncate d6/fc 374991 0 2024-05-13T22:47:53.621 INFO:tasks.workunit.client.0.smithi102.stdout:9/65: creat d1/d6/db/f1c x:0 0 0 2024-05-13T22:47:53.624 INFO:tasks.workunit.client.0.smithi102.stdout:7/46: rename d6/d9 to d6/dd/de 0 2024-05-13T22:47:53.624 INFO:tasks.workunit.client.0.smithi102.stdout:3/66: rmdir de 39 2024-05-13T22:47:53.624 INFO:tasks.workunit.client.0.smithi102.stdout:7/47: read d6/fc [73446,42039] 0 2024-05-13T22:47:53.626 INFO:tasks.workunit.client.0.smithi102.stdout:7/48: chown d6/dd/de/fb 663724431 1 2024-05-13T22:47:53.626 INFO:tasks.workunit.client.0.smithi102.stdout:9/66: creat d1/d6/f1d x:0 0 0 2024-05-13T22:47:53.630 INFO:tasks.workunit.client.0.smithi102.stdout:7/49: mknod d6/cf 0 2024-05-13T22:47:53.634 INFO:tasks.workunit.client.0.smithi102.stdout:9/67: rename d1/d6/fe to d1/f1e 0 2024-05-13T22:47:53.636 INFO:tasks.workunit.client.0.smithi102.stdout:3/67: rmdir de/d10 39 2024-05-13T22:47:53.637 INFO:tasks.workunit.client.0.smithi102.stdout:7/50: fdatasync f4 0 2024-05-13T22:47:53.638 INFO:tasks.workunit.client.0.smithi102.stdout:0/79: fsync d1/d7/f13 0 2024-05-13T22:47:53.639 INFO:tasks.workunit.client.0.smithi102.stdout:7/51: write d6/f7 [109450,14772] 0 2024-05-13T22:47:53.639 INFO:tasks.workunit.client.0.smithi102.stdout:0/80: write d1/d7/f13 [266350,42468] 0 2024-05-13T22:47:53.640 INFO:tasks.workunit.client.0.smithi102.stdout:7/52: write f4 [812192,17695] 0 2024-05-13T22:47:53.644 INFO:tasks.workunit.client.0.smithi102.stdout:0/81: mknod d1/c14 0 2024-05-13T22:47:53.645 INFO:tasks.workunit.client.0.smithi102.stdout:0/82: fdatasync d1/f4 0 2024-05-13T22:47:53.646 INFO:tasks.workunit.client.0.smithi102.stdout:0/83: write d1/f6 [6711554,70358] 0 2024-05-13T22:47:53.647 INFO:tasks.workunit.client.0.smithi102.stdout:3/68: creat de/d10/f1d x:0 0 0 2024-05-13T22:47:53.649 INFO:tasks.workunit.client.0.smithi102.stdout:3/69: write de/d10/f14 [129877,100185] 0 2024-05-13T22:47:53.650 INFO:tasks.workunit.client.0.smithi102.stdout:3/70: write f2 [830902,5116] 0 2024-05-13T22:47:53.651 INFO:tasks.workunit.client.0.smithi102.stdout:0/84: dread d1/d7/fd [0,4194304] 0 2024-05-13T22:47:53.658 INFO:tasks.workunit.client.0.smithi102.stdout:0/85: creat d1/d7/f15 x:0 0 0 2024-05-13T22:47:53.659 INFO:tasks.workunit.client.0.smithi102.stdout:3/71: symlink de/d1c/l1e 0 2024-05-13T22:47:53.661 INFO:tasks.workunit.client.0.smithi102.stdout:3/72: read de/d10/f12 [929512,23899] 0 2024-05-13T22:47:53.667 INFO:tasks.workunit.client.0.smithi102.stdout:9/68: rmdir d1 39 2024-05-13T22:47:53.667 INFO:tasks.workunit.client.0.smithi102.stdout:3/73: dwrite f5 [0,4194304] 0 2024-05-13T22:47:53.674 INFO:tasks.workunit.client.0.smithi102.stdout:3/74: dwrite fa [0,4194304] 0 2024-05-13T22:47:53.675 INFO:tasks.workunit.client.0.smithi102.stdout:9/69: mknod d1/d6/db/c1f 0 2024-05-13T22:47:53.678 INFO:tasks.workunit.client.0.smithi102.stdout:3/75: creat de/d1c/f1f x:0 0 0 2024-05-13T22:47:53.681 INFO:tasks.workunit.client.0.smithi102.stdout:9/70: symlink d1/d6/db/l20 0 2024-05-13T22:47:53.685 INFO:tasks.workunit.client.0.smithi102.stdout:9/71: mkdir d1/d6/d21 0 2024-05-13T22:47:53.687 INFO:tasks.workunit.client.0.smithi102.stdout:9/72: rename d1/c9 to d1/c22 0 2024-05-13T22:47:53.689 INFO:tasks.workunit.client.0.smithi102.stdout:0/86: rmdir d1 39 2024-05-13T22:47:53.707 INFO:tasks.workunit.client.0.smithi102.stdout:9/73: write d1/f7 [3066576,22945] 0 2024-05-13T22:47:53.707 INFO:tasks.workunit.client.0.smithi102.stdout:3/76: dwrite de/d10/f11 [0,4194304] 0 2024-05-13T22:47:53.709 INFO:tasks.workunit.client.0.smithi102.stdout:9/74: rmdir d1/d6 39 2024-05-13T22:47:53.712 INFO:tasks.workunit.client.0.smithi102.stdout:9/75: truncate d1/d6/f12 129649 0 2024-05-13T22:47:53.715 INFO:tasks.workunit.client.0.smithi102.stdout:9/76: symlink d1/d6/d21/l23 0 2024-05-13T22:47:53.716 INFO:tasks.workunit.client.0.smithi102.stdout:9/77: readlink d1/l2 0 2024-05-13T22:47:53.716 INFO:tasks.workunit.client.0.smithi102.stdout:3/77: dwrite de/d10/f1b [0,4194304] 0 2024-05-13T22:47:53.739 INFO:tasks.workunit.client.0.smithi102.stdout:3/78: readlink de/l1a 0 2024-05-13T22:47:53.741 INFO:tasks.workunit.client.0.smithi102.stdout:3/79: unlink de/ff 0 2024-05-13T22:47:53.745 INFO:tasks.workunit.client.0.smithi102.stdout:9/78: link d1/d6/db/f1c d1/f24 0 2024-05-13T22:47:53.746 INFO:tasks.workunit.client.0.smithi102.stdout:3/80: dread de/d10/f1b [0,4194304] 0 2024-05-13T22:47:53.747 INFO:tasks.workunit.client.0.smithi102.stdout:3/81: read de/d10/f11 [1862223,18335] 0 2024-05-13T22:47:53.748 INFO:tasks.workunit.client.0.smithi102.stdout:9/79: creat d1/d6/db/f25 x:0 0 0 2024-05-13T22:47:53.749 INFO:tasks.workunit.client.0.smithi102.stdout:9/80: dread - d1/d6/f1d zero size 2024-05-13T22:47:53.754 INFO:tasks.workunit.client.0.smithi102.stdout:3/82: dwrite de/d10/f1d [0,4194304] 0 2024-05-13T22:47:53.764 INFO:tasks.workunit.client.0.smithi102.stdout:9/81: getdents d1 0 2024-05-13T22:47:53.772 INFO:tasks.workunit.client.0.smithi102.stdout:9/82: dwrite d1/f7 [0,4194304] 0 2024-05-13T22:47:53.772 INFO:tasks.workunit.client.0.smithi102.stdout:9/83: chown d1 18 1 2024-05-13T22:47:53.773 INFO:tasks.workunit.client.0.smithi102.stdout:3/83: truncate de/d10/f14 369297 0 2024-05-13T22:47:53.775 INFO:tasks.workunit.client.0.smithi102.stdout:3/84: rmdir de/d1c 39 2024-05-13T22:47:53.779 INFO:tasks.workunit.client.0.smithi102.stdout:9/84: rename d1/fa to d1/d6/d21/f26 0 2024-05-13T22:47:53.780 INFO:tasks.workunit.client.0.smithi102.stdout:3/85: fdatasync de/d10/f1d 0 2024-05-13T22:47:53.781 INFO:tasks.workunit.client.0.smithi102.stdout:3/86: write fd [286531,37020] 0 2024-05-13T22:47:53.786 INFO:tasks.workunit.client.0.smithi102.stdout:3/87: getdents de/d10 0 2024-05-13T22:47:53.788 INFO:tasks.workunit.client.0.smithi102.stdout:3/88: symlink de/l20 0 2024-05-13T22:47:53.788 INFO:tasks.workunit.client.0.smithi102.stdout:3/89: chown f2 5732 1 2024-05-13T22:47:53.789 INFO:tasks.workunit.client.0.smithi102.stdout:3/90: chown de/d10/c15 108824638 1 2024-05-13T22:47:53.796 INFO:tasks.workunit.client.0.smithi102.stdout:3/91: dwrite f2 [0,4194304] 0 2024-05-13T22:47:53.798 INFO:tasks.workunit.client.0.smithi102.stdout:3/92: write de/f19 [158965,117146] 0 2024-05-13T22:47:53.799 INFO:tasks.workunit.client.0.smithi102.stdout:3/93: write de/f19 [1284049,90271] 0 2024-05-13T22:47:53.800 INFO:tasks.workunit.client.0.smithi102.stdout:9/85: dread d1/f7 [0,4194304] 0 2024-05-13T22:47:53.808 INFO:tasks.workunit.client.0.smithi102.stdout:9/86: fsync d1/f1e 0 2024-05-13T22:47:53.813 INFO:tasks.workunit.client.0.smithi102.stdout:9/87: mkdir d1/d27 0 2024-05-13T22:47:53.815 INFO:tasks.workunit.client.0.smithi102.stdout:3/94: getdents de/d1c 0 2024-05-13T22:47:53.815 INFO:tasks.workunit.client.0.smithi102.stdout:3/95: chown de/l20 5 1 2024-05-13T22:47:53.815 INFO:tasks.workunit.client.0.smithi102.stdout:9/88: chown d1/l2 113312 1 2024-05-13T22:47:53.816 INFO:tasks.workunit.client.0.smithi102.stdout:3/96: mkdir de/d21 0 2024-05-13T22:47:53.818 INFO:tasks.workunit.client.0.smithi102.stdout:9/89: mknod d1/d6/db/c28 0 2024-05-13T22:47:53.821 INFO:tasks.workunit.client.0.smithi102.stdout:3/97: mknod de/c22 0 2024-05-13T22:47:53.822 INFO:tasks.workunit.client.0.smithi102.stdout:3/98: write f5 [840188,26139] 0 2024-05-13T22:47:53.825 INFO:tasks.workunit.client.0.smithi102.stdout:9/90: rmdir d1/d27 0 2024-05-13T22:47:53.828 INFO:tasks.workunit.client.0.smithi102.stdout:3/99: dread de/f19 [0,4194304] 0 2024-05-13T22:47:53.830 INFO:tasks.workunit.client.0.smithi102.stdout:3/100: write f2 [4821245,71103] 0 2024-05-13T22:47:53.832 INFO:tasks.workunit.client.0.smithi102.stdout:3/101: chown de/l13 68 1 2024-05-13T22:47:53.834 INFO:tasks.workunit.client.0.smithi102.stdout:9/91: dwrite d1/d6/f13 [0,4194304] 0 2024-05-13T22:47:53.839 INFO:tasks.workunit.client.0.smithi102.stdout:9/92: creat d1/d6/d21/f29 x:0 0 0 2024-05-13T22:47:53.844 INFO:tasks.workunit.client.0.smithi102.stdout:3/102: dwrite de/d10/f1b [4194304,4194304] 0 2024-05-13T22:47:53.844 INFO:tasks.workunit.client.0.smithi102.stdout:9/93: mknod d1/c2a 0 2024-05-13T22:47:53.851 INFO:tasks.workunit.client.0.smithi102.stdout:3/103: rename de/l17 to de/d1c/l23 0 2024-05-13T22:47:53.856 INFO:tasks.workunit.client.0.smithi102.stdout:3/104: unlink de/c22 0 2024-05-13T22:47:53.861 INFO:tasks.workunit.client.0.smithi102.stdout:9/94: fdatasync d1/d6/f13 0 2024-05-13T22:47:53.867 INFO:tasks.workunit.client.0.smithi102.stdout:9/95: creat d1/f2b x:0 0 0 2024-05-13T22:47:53.867 INFO:tasks.workunit.client.0.smithi102.stdout:9/96: dread - d1/d6/db/f1b zero size 2024-05-13T22:47:53.869 INFO:tasks.workunit.client.0.smithi102.stdout:9/97: mkdir d1/d6/db/d2c 0 2024-05-13T22:47:53.884 INFO:tasks.workunit.client.0.smithi102.stdout:8/34: sync 2024-05-13T22:47:53.884 INFO:tasks.workunit.client.0.smithi102.stdout:6/39: sync 2024-05-13T22:47:53.885 INFO:tasks.workunit.client.0.smithi102.stdout:4/49: sync 2024-05-13T22:47:53.885 INFO:tasks.workunit.client.0.smithi102.stdout:4/50: read - d3/fa zero size 2024-05-13T22:47:53.890 INFO:tasks.workunit.client.0.smithi102.stdout:4/51: write d3/fa [829345,58651] 0 2024-05-13T22:47:53.890 INFO:tasks.workunit.client.0.smithi102.stdout:6/40: mkdir d9 0 2024-05-13T22:47:53.895 INFO:tasks.workunit.client.0.smithi102.stdout:9/98: write d1/f7 [835950,74015] 0 2024-05-13T22:47:53.895 INFO:tasks.workunit.client.0.smithi102.stdout:4/52: write d3/fa [645481,15621] 0 2024-05-13T22:47:53.897 INFO:tasks.workunit.client.0.smithi102.stdout:6/41: rename f4 to d9/fa 0 2024-05-13T22:47:53.902 INFO:tasks.workunit.client.0.smithi102.stdout:4/53: symlink d3/d4/d6/l13 0 2024-05-13T22:47:53.905 INFO:tasks.workunit.client.0.smithi102.stdout:9/99: mkdir d1/d6/db/d2c/d2d 0 2024-05-13T22:47:53.906 INFO:tasks.workunit.client.0.smithi102.stdout:8/35: dwrite d2/f4 [8388608,4194304] 0 2024-05-13T22:47:53.908 INFO:tasks.workunit.client.0.smithi102.stdout:3/105: dwrite de/d10/f12 [0,4194304] 0 2024-05-13T22:47:53.911 INFO:tasks.workunit.client.0.smithi102.stdout:4/54: rename d3/d4/d6/de to d3/d4/d6/de/d14 22 2024-05-13T22:47:53.916 INFO:tasks.workunit.client.0.smithi102.stdout:6/42: dwrite f6 [0,4194304] 0 2024-05-13T22:47:53.923 INFO:tasks.workunit.client.0.smithi102.stdout:9/100: truncate d1/f1e 1856540 0 2024-05-13T22:47:53.924 INFO:tasks.workunit.client.0.smithi102.stdout:6/43: mkdir d9/db 0 2024-05-13T22:47:53.929 INFO:tasks.workunit.client.0.smithi102.stdout:4/55: creat d3/d4/d6/de/f15 x:0 0 0 2024-05-13T22:47:53.931 INFO:tasks.workunit.client.0.smithi102.stdout:9/101: rename d1/d6/f13 to d1/d6/db/d2c/d2d/f2e 0 2024-05-13T22:47:53.931 INFO:tasks.workunit.client.0.smithi102.stdout:9/102: chown d1/d6/f11 53020842 1 2024-05-13T22:47:53.935 INFO:tasks.workunit.client.0.smithi102.stdout:4/56: mknod d3/d4/c16 0 2024-05-13T22:47:53.935 INFO:tasks.workunit.client.0.smithi102.stdout:3/106: truncate f7 3960430 0 2024-05-13T22:47:53.936 INFO:tasks.workunit.client.0.smithi102.stdout:4/57: stat d3/d4/d6/de/f15 0 2024-05-13T22:47:53.937 INFO:tasks.workunit.client.0.smithi102.stdout:6/44: mknod d9/db/cc 0 2024-05-13T22:47:53.937 INFO:tasks.workunit.client.0.smithi102.stdout:4/58: dread - d3/d4/ff zero size 2024-05-13T22:47:53.939 INFO:tasks.workunit.client.0.smithi102.stdout:6/45: creat d9/fd x:0 0 0 2024-05-13T22:47:53.944 INFO:tasks.workunit.client.0.smithi102.stdout:4/59: creat d3/db/f17 x:0 0 0 2024-05-13T22:47:53.947 INFO:tasks.workunit.client.0.smithi102.stdout:4/60: chown d3/d4/l5 11945 1 2024-05-13T22:47:53.948 INFO:tasks.workunit.client.0.smithi102.stdout:6/46: dwrite f6 [4194304,4194304] 0 2024-05-13T22:47:53.948 INFO:tasks.workunit.client.0.smithi102.stdout:6/47: chown f8 10899427 1 2024-05-13T22:47:53.949 INFO:tasks.workunit.client.0.smithi102.stdout:6/48: truncate f5 917046 0 2024-05-13T22:47:53.949 INFO:tasks.workunit.client.0.smithi102.stdout:6/49: fdatasync d9/fd 0 2024-05-13T22:47:53.952 INFO:tasks.workunit.client.0.smithi102.stdout:6/50: creat d9/db/fe x:0 0 0 2024-05-13T22:47:53.954 INFO:tasks.workunit.client.0.smithi102.stdout:6/51: write f8 [814734,15177] 0 2024-05-13T22:47:53.955 INFO:tasks.workunit.client.0.smithi102.stdout:6/52: fdatasync f3 0 2024-05-13T22:47:53.956 INFO:tasks.workunit.client.0.smithi102.stdout:4/61: dwrite d3/d4/d6/de/f15 [0,4194304] 0 2024-05-13T22:47:53.958 INFO:tasks.workunit.client.0.smithi102.stdout:6/53: rename d9/fd to d9/ff 0 2024-05-13T22:47:53.961 INFO:tasks.workunit.client.0.smithi102.stdout:4/62: creat d3/d4/d6/f18 x:0 0 0 2024-05-13T22:47:53.962 INFO:tasks.workunit.client.0.smithi102.stdout:6/54: dread f1 [4194304,4194304] 0 2024-05-13T22:47:53.962 INFO:tasks.workunit.client.0.smithi102.stdout:4/63: write d3/f10 [765077,111452] 0 2024-05-13T22:47:53.963 INFO:tasks.workunit.client.0.smithi102.stdout:6/55: chown d9/fa 377 1 2024-05-13T22:47:53.965 INFO:tasks.workunit.client.0.smithi102.stdout:4/64: mkdir d3/d19 0 2024-05-13T22:47:53.968 INFO:tasks.workunit.client.0.smithi102.stdout:4/65: symlink d3/d4/l1a 0 2024-05-13T22:47:53.969 INFO:tasks.workunit.client.0.smithi102.stdout:4/66: readlink d3/d4/l1a 0 2024-05-13T22:47:53.970 INFO:tasks.workunit.client.0.smithi102.stdout:6/56: dwrite f0 [4194304,4194304] 0 2024-05-13T22:47:53.972 INFO:tasks.workunit.client.0.smithi102.stdout:6/57: creat d9/f10 x:0 0 0 2024-05-13T22:47:53.972 INFO:tasks.workunit.client.0.smithi102.stdout:6/58: dread - f3 zero size 2024-05-13T22:47:53.972 INFO:tasks.workunit.client.0.smithi102.stdout:6/59: truncate f3 94106 0 2024-05-13T22:47:53.974 INFO:tasks.workunit.client.0.smithi102.stdout:6/60: symlink d9/db/l11 0 2024-05-13T22:47:53.975 INFO:tasks.workunit.client.0.smithi102.stdout:6/61: mknod d9/db/c12 0 2024-05-13T22:47:53.979 INFO:tasks.workunit.client.0.smithi102.stdout:6/62: link f6 d9/db/f13 0 2024-05-13T22:47:53.980 INFO:tasks.workunit.client.0.smithi102.stdout:6/63: write f8 [448614,4351] 0 2024-05-13T22:47:53.982 INFO:tasks.workunit.client.0.smithi102.stdout:6/64: mkdir d9/db/d14 0 2024-05-13T22:47:53.983 INFO:tasks.workunit.client.0.smithi102.stdout:6/65: truncate f7 1551691 0 2024-05-13T22:47:53.986 INFO:tasks.workunit.client.0.smithi102.stdout:6/66: dread f0 [4194304,4194304] 0 2024-05-13T22:47:53.987 INFO:tasks.workunit.client.0.smithi102.stdout:6/67: write d9/f10 [566980,111376] 0 2024-05-13T22:47:53.994 INFO:tasks.workunit.client.0.smithi102.stdout:6/68: dwrite d9/f10 [0,4194304] 0 2024-05-13T22:47:53.995 INFO:tasks.workunit.client.0.smithi102.stdout:6/69: chown d9/db/fe 39 1 2024-05-13T22:47:53.995 INFO:tasks.workunit.client.0.smithi102.stdout:6/70: chown d9/db/d14 33585 1 2024-05-13T22:47:53.997 INFO:tasks.workunit.client.0.smithi102.stdout:6/71: truncate d9/db/fe 477309 0 2024-05-13T22:47:54.027 INFO:tasks.workunit.client.0.smithi102.stdout:3/107: truncate f2 1354336 0 2024-05-13T22:47:54.029 INFO:tasks.workunit.client.0.smithi102.stdout:6/72: dread f8 [0,4194304] 0 2024-05-13T22:47:54.030 INFO:tasks.workunit.client.0.smithi102.stdout:3/108: symlink de/d10/l24 0 2024-05-13T22:47:54.030 INFO:tasks.workunit.client.0.smithi102.stdout:6/73: read - d9/ff zero size 2024-05-13T22:47:54.033 INFO:tasks.workunit.client.0.smithi102.stdout:3/109: creat de/d1c/f25 x:0 0 0 2024-05-13T22:47:54.036 INFO:tasks.workunit.client.0.smithi102.stdout:6/74: rename d9/db/c12 to d9/c15 0 2024-05-13T22:47:54.041 INFO:tasks.workunit.client.0.smithi102.stdout:3/110: rename de/l20 to de/d21/l26 0 2024-05-13T22:47:54.042 INFO:tasks.workunit.client.0.smithi102.stdout:4/67: fsync d3/f10 0 2024-05-13T22:47:54.046 INFO:tasks.workunit.client.0.smithi102.stdout:3/111: rename de/d10/f1b to de/d10/f27 0 2024-05-13T22:47:54.046 INFO:tasks.workunit.client.0.smithi102.stdout:6/75: dwrite f5 [0,4194304] 0 2024-05-13T22:47:54.047 INFO:tasks.workunit.client.0.smithi102.stdout:3/112: read de/d10/f11 [43971,44546] 0 2024-05-13T22:47:54.051 INFO:tasks.workunit.client.0.smithi102.stdout:4/68: mknod d3/db/c1b 0 2024-05-13T22:47:54.054 INFO:tasks.workunit.client.0.smithi102.stdout:3/113: mknod de/d1c/c28 0 2024-05-13T22:47:54.055 INFO:tasks.workunit.client.0.smithi102.stdout:3/114: fdatasync de/d1c/f1f 0 2024-05-13T22:47:54.057 INFO:tasks.workunit.client.0.smithi102.stdout:6/76: rename d9/db/l11 to d9/db/d14/l16 0 2024-05-13T22:47:54.059 INFO:tasks.workunit.client.0.smithi102.stdout:4/69: creat d3/d4/d6/f1c x:0 0 0 2024-05-13T22:47:54.060 INFO:tasks.workunit.client.0.smithi102.stdout:3/115: symlink de/d10/l29 0 2024-05-13T22:47:54.062 INFO:tasks.workunit.client.0.smithi102.stdout:6/77: symlink d9/l17 0 2024-05-13T22:47:54.064 INFO:tasks.workunit.client.0.smithi102.stdout:4/70: creat d3/db/f1d x:0 0 0 2024-05-13T22:47:54.071 INFO:tasks.workunit.client.0.smithi102.stdout:4/71: write d3/f10 [1648287,128097] 0 2024-05-13T22:47:54.071 INFO:tasks.workunit.client.0.smithi102.stdout:3/116: readlink de/d1c/l23 0 2024-05-13T22:47:54.073 INFO:tasks.workunit.client.0.smithi102.stdout:3/117: write fa [2910923,69785] 0 2024-05-13T22:47:54.074 INFO:tasks.workunit.client.0.smithi102.stdout:4/72: truncate d3/d4/ff 810587 0 2024-05-13T22:47:54.074 INFO:tasks.workunit.client.0.smithi102.stdout:3/118: chown de/d10/f12 572385 1 2024-05-13T22:47:54.110 INFO:tasks.workunit.client.0.smithi102.stdout:2/41: sync 2024-05-13T22:47:54.110 INFO:tasks.workunit.client.0.smithi102.stdout:1/58: sync 2024-05-13T22:47:54.114 INFO:tasks.workunit.client.0.smithi102.stdout:6/78: dwrite d9/fa [0,4194304] 0 2024-05-13T22:47:54.118 INFO:tasks.workunit.client.0.smithi102.stdout:4/73: dwrite d3/d4/ff [0,4194304] 0 2024-05-13T22:47:54.119 INFO:tasks.workunit.client.0.smithi102.stdout:3/119: dwrite fa [0,4194304] 0 2024-05-13T22:47:54.120 INFO:tasks.workunit.client.0.smithi102.stdout:3/120: stat de/c16 0 2024-05-13T22:47:54.129 INFO:tasks.workunit.client.0.smithi102.stdout:1/59: symlink d2/d10/l14 0 2024-05-13T22:47:54.130 INFO:tasks.workunit.client.0.smithi102.stdout:6/79: symlink d9/db/d14/l18 0 2024-05-13T22:47:54.132 INFO:tasks.workunit.client.0.smithi102.stdout:4/74: creat d3/d4/f1e x:0 0 0 2024-05-13T22:47:54.133 INFO:tasks.workunit.client.0.smithi102.stdout:6/80: read d9/db/fe [411697,19488] 0 2024-05-13T22:47:54.133 INFO:tasks.workunit.client.0.smithi102.stdout:4/75: fsync d3/fa 0 2024-05-13T22:47:54.134 INFO:tasks.workunit.client.0.smithi102.stdout:3/121: mkdir de/d10/d2a 0 2024-05-13T22:47:54.136 INFO:tasks.workunit.client.0.smithi102.stdout:2/42: symlink d7/ld 0 2024-05-13T22:47:54.137 INFO:tasks.workunit.client.0.smithi102.stdout:2/43: chown d7/ld 17025 1 2024-05-13T22:47:54.138 INFO:tasks.workunit.client.0.smithi102.stdout:1/60: chown d2/l3 233 1 2024-05-13T22:47:54.146 INFO:tasks.workunit.client.0.smithi102.stdout:4/76: mkdir d3/d4/d6/de/d1f 0 2024-05-13T22:47:54.147 INFO:tasks.workunit.client.0.smithi102.stdout:3/122: creat de/d21/f2b x:0 0 0 2024-05-13T22:47:54.150 INFO:tasks.workunit.client.0.smithi102.stdout:6/81: dwrite f6 [0,4194304] 0 2024-05-13T22:47:54.150 INFO:tasks.workunit.client.0.smithi102.stdout:6/82: read - d9/ff zero size 2024-05-13T22:47:54.154 INFO:tasks.workunit.client.0.smithi102.stdout:6/83: dread d9/fa [4194304,4194304] 0 2024-05-13T22:47:54.155 INFO:tasks.workunit.client.0.smithi102.stdout:6/84: chown f8 7734 1 2024-05-13T22:47:54.155 INFO:tasks.workunit.client.0.smithi102.stdout:4/77: unlink d3/d4/d6/f9 0 2024-05-13T22:47:54.157 INFO:tasks.workunit.client.0.smithi102.stdout:3/123: mknod de/d21/c2c 0 2024-05-13T22:47:54.157 INFO:tasks.workunit.client.0.smithi102.stdout:3/124: stat de/d1c/l23 0 2024-05-13T22:47:54.165 INFO:tasks.workunit.client.0.smithi102.stdout:1/61: rename d2/fe to d2/d4/d5/f15 0 2024-05-13T22:47:54.168 INFO:tasks.workunit.client.0.smithi102.stdout:6/85: write f8 [1353604,106785] 0 2024-05-13T22:47:54.172 INFO:tasks.workunit.client.0.smithi102.stdout:2/44: link l5 d7/d9/le 0 2024-05-13T22:47:54.174 INFO:tasks.workunit.client.0.smithi102.stdout:2/45: write d7/d9/fb [4391441,93964] 0 2024-05-13T22:47:54.175 INFO:tasks.workunit.client.0.smithi102.stdout:2/46: stat d7/d9/lc 0 2024-05-13T22:47:54.177 INFO:tasks.workunit.client.0.smithi102.stdout:6/86: dwrite f3 [0,4194304] 0 2024-05-13T22:47:54.179 INFO:tasks.workunit.client.0.smithi102.stdout:4/78: rename d3/db/c1b to d3/d4/d6/de/c20 0 2024-05-13T22:47:54.181 INFO:tasks.workunit.client.0.smithi102.stdout:4/79: write d3/d4/ff [2698161,82304] 0 2024-05-13T22:47:54.182 INFO:tasks.workunit.client.0.smithi102.stdout:4/80: chown d3/d4/ff 0 1 2024-05-13T22:47:54.185 INFO:tasks.workunit.client.0.smithi102.stdout:3/125: mknod de/c2d 0 2024-05-13T22:47:54.195 INFO:tasks.workunit.client.0.smithi102.stdout:6/87: rename d9/db/f13 to d9/db/d14/f19 0 2024-05-13T22:47:54.198 INFO:tasks.workunit.client.0.smithi102.stdout:3/126: unlink de/d1c/f25 0 2024-05-13T22:47:54.208 INFO:tasks.workunit.client.0.smithi102.stdout:4/81: symlink d3/l21 0 2024-05-13T22:47:54.209 INFO:tasks.workunit.client.0.smithi102.stdout:5/115: sync 2024-05-13T22:47:54.210 INFO:tasks.workunit.client.0.smithi102.stdout:3/127: dwrite de/d10/f14 [0,4194304] 0 2024-05-13T22:47:54.211 INFO:tasks.workunit.client.0.smithi102.stdout:4/82: dread - d3/d4/d6/f1c zero size 2024-05-13T22:47:54.215 INFO:tasks.workunit.client.0.smithi102.stdout:6/88: dwrite f1 [0,4194304] 0 2024-05-13T22:47:54.217 INFO:tasks.workunit.client.0.smithi102.stdout:2/47: creat d7/ff x:0 0 0 2024-05-13T22:47:54.218 INFO:tasks.workunit.client.0.smithi102.stdout:2/48: chown d7/d9/fb 1764362412 1 2024-05-13T22:47:54.222 INFO:tasks.workunit.client.0.smithi102.stdout:5/116: mknod de/d10/d28/c30 0 2024-05-13T22:47:54.222 INFO:tasks.workunit.client.0.smithi102.stdout:5/117: readlink de/d25/d2c/l2f 0 2024-05-13T22:47:54.222 INFO:tasks.workunit.client.0.smithi102.stdout:5/118: fdatasync de/f11 0 2024-05-13T22:47:54.222 INFO:tasks.workunit.client.0.smithi102.stdout:5/119: fsync de/d10/f1f 0 2024-05-13T22:47:54.224 INFO:tasks.workunit.client.0.smithi102.stdout:2/49: dread d7/d9/fb [0,4194304] 0 2024-05-13T22:47:54.227 INFO:tasks.workunit.client.0.smithi102.stdout:6/89: creat d9/f1a x:0 0 0 2024-05-13T22:47:54.233 INFO:tasks.workunit.client.0.smithi102.stdout:3/128: creat de/d10/d2a/f2e x:0 0 0 2024-05-13T22:47:54.238 INFO:tasks.workunit.client.0.smithi102.stdout:5/120: creat de/d25/d2c/f31 x:0 0 0 2024-05-13T22:47:54.239 INFO:tasks.workunit.client.0.smithi102.stdout:5/121: truncate de/f13 240682 0 2024-05-13T22:47:54.242 INFO:tasks.workunit.client.0.smithi102.stdout:2/50: mkdir d7/d9/d10 0 2024-05-13T22:47:54.243 INFO:tasks.workunit.client.0.smithi102.stdout:6/90: mknod d9/db/d14/c1b 0 2024-05-13T22:47:54.250 INFO:tasks.workunit.client.0.smithi102.stdout:1/62: dread - d2/d10/f12 zero size 2024-05-13T22:47:54.252 INFO:tasks.workunit.client.0.smithi102.stdout:7/53: sync 2024-05-13T22:47:54.252 INFO:tasks.workunit.client.0.smithi102.stdout:5/122: dwrite de/f13 [0,4194304] 0 2024-05-13T22:47:54.253 INFO:tasks.workunit.client.0.smithi102.stdout:7/54: chown d6 31946 1 2024-05-13T22:47:54.255 INFO:tasks.workunit.client.0.smithi102.stdout:6/91: dwrite d9/f1a [0,4194304] 0 2024-05-13T22:47:54.255 INFO:tasks.workunit.client.0.smithi102.stdout:1/63: truncate d2/ff 780657 0 2024-05-13T22:47:54.258 INFO:tasks.workunit.client.0.smithi102.stdout:5/123: fdatasync de/d10/f1f 0 2024-05-13T22:47:54.260 INFO:tasks.workunit.client.0.smithi102.stdout:2/51: write d7/d9/fb [5161361,22004] 0 2024-05-13T22:47:54.263 INFO:tasks.workunit.client.0.smithi102.stdout:3/129: fdatasync f2 0 2024-05-13T22:47:54.268 INFO:tasks.workunit.client.0.smithi102.stdout:2/52: dwrite d7/d9/fb [0,4194304] 0 2024-05-13T22:47:54.269 INFO:tasks.workunit.client.0.smithi102.stdout:4/83: getdents d3/d4/d6/de 0 2024-05-13T22:47:54.275 INFO:tasks.workunit.client.0.smithi102.stdout:1/64: rmdir d2/d4/d5 39 2024-05-13T22:47:54.278 INFO:tasks.workunit.client.0.smithi102.stdout:7/55: rmdir d6/dd/de 39 2024-05-13T22:47:54.284 INFO:tasks.workunit.client.0.smithi102.stdout:5/124: rmdir de/d10/d28 39 2024-05-13T22:47:54.284 INFO:tasks.workunit.client.0.smithi102.stdout:4/84: dwrite d3/d4/f1e [0,4194304] 0 2024-05-13T22:47:54.286 INFO:tasks.workunit.client.0.smithi102.stdout:2/53: dwrite d7/d9/fb [4194304,4194304] 0 2024-05-13T22:47:54.289 INFO:tasks.workunit.client.0.smithi102.stdout:4/85: write d3/d4/f1e [2340997,85356] 0 2024-05-13T22:47:54.289 INFO:tasks.workunit.client.0.smithi102.stdout:3/130: creat de/d10/d2a/f2f x:0 0 0 2024-05-13T22:47:54.296 INFO:tasks.workunit.client.0.smithi102.stdout:7/56: dwrite d6/f8 [0,4194304] 0 2024-05-13T22:47:54.297 INFO:tasks.workunit.client.0.smithi102.stdout:2/54: dread d7/d9/fb [0,4194304] 0 2024-05-13T22:47:54.297 INFO:tasks.workunit.client.0.smithi102.stdout:7/57: write d6/f7 [811851,113423] 0 2024-05-13T22:47:54.299 INFO:tasks.workunit.client.0.smithi102.stdout:7/58: chown l2 236661 1 2024-05-13T22:47:54.307 INFO:tasks.workunit.client.0.smithi102.stdout:5/125: readlink de/d25/d2c/l2d 0 2024-05-13T22:47:54.308 INFO:tasks.workunit.client.0.smithi102.stdout:5/126: write de/f14 [447286,92830] 0 2024-05-13T22:47:54.315 INFO:tasks.workunit.client.0.smithi102.stdout:1/65: fsync d2/d4/d5/fa 0 2024-05-13T22:47:54.316 INFO:tasks.workunit.client.0.smithi102.stdout:1/66: read - d2/d10/f12 zero size 2024-05-13T22:47:54.318 INFO:tasks.workunit.client.0.smithi102.stdout:1/67: dread d2/d4/d5/fa [0,4194304] 0 2024-05-13T22:47:54.320 INFO:tasks.workunit.client.0.smithi102.stdout:2/55: fdatasync d7/d9/fb 0 2024-05-13T22:47:54.324 INFO:tasks.workunit.client.0.smithi102.stdout:4/86: mkdir d3/d22 0 2024-05-13T22:47:54.327 INFO:tasks.workunit.client.0.smithi102.stdout:6/92: getdents d9/db/d14 0 2024-05-13T22:47:54.330 INFO:tasks.workunit.client.0.smithi102.stdout:7/59: fsync d6/dd/de/fb 0 2024-05-13T22:47:54.330 INFO:tasks.workunit.client.0.smithi102.stdout:7/60: readlink l2 0 2024-05-13T22:47:54.331 INFO:tasks.workunit.client.0.smithi102.stdout:2/56: mkdir d7/d9/d11 0 2024-05-13T22:47:54.332 INFO:tasks.workunit.client.0.smithi102.stdout:7/61: write f4 [2613919,83531] 0 2024-05-13T22:47:54.332 INFO:tasks.workunit.client.0.smithi102.stdout:2/57: chown d7/d9 933 1 2024-05-13T22:47:54.335 INFO:tasks.workunit.client.0.smithi102.stdout:6/93: symlink d9/db/l1c 0 2024-05-13T22:47:54.335 INFO:tasks.workunit.client.0.smithi102.stdout:6/94: write f5 [3813046,57401] 0 2024-05-13T22:47:54.336 INFO:tasks.workunit.client.0.smithi102.stdout:1/68: truncate d2/d4/d5/f15 4056450 0 2024-05-13T22:47:54.341 INFO:tasks.workunit.client.0.smithi102.stdout:4/87: mkdir d3/d19/d23 0 2024-05-13T22:47:54.342 INFO:tasks.workunit.client.0.smithi102.stdout:4/88: read d3/d4/d6/de/f15 [575132,114893] 0 2024-05-13T22:47:54.356 INFO:tasks.workunit.client.0.smithi102.stdout:4/89: symlink d3/d19/l24 0 2024-05-13T22:47:54.364 INFO:tasks.workunit.client.0.smithi102.stdout:2/58: truncate d7/d9/fb 1742413 0 2024-05-13T22:47:54.365 INFO:tasks.workunit.client.0.smithi102.stdout:2/59: chown l1 493255 1 2024-05-13T22:47:54.366 INFO:tasks.workunit.client.0.smithi102.stdout:4/90: symlink d3/d4/d6/l25 0 2024-05-13T22:47:54.367 INFO:tasks.workunit.client.0.smithi102.stdout:4/91: read - d3/d4/fd zero size 2024-05-13T22:47:54.369 INFO:tasks.workunit.client.0.smithi102.stdout:1/69: creat d2/f16 x:0 0 0 2024-05-13T22:47:54.371 INFO:tasks.workunit.client.0.smithi102.stdout:4/92: mknod d3/d4/c26 0 2024-05-13T22:47:54.371 INFO:tasks.workunit.client.0.smithi102.stdout:2/60: dwrite d7/ff [0,4194304] 0 2024-05-13T22:47:54.380 INFO:tasks.workunit.client.0.smithi102.stdout:4/93: rename d3/d4/d6/de/c20 to d3/d19/c27 0 2024-05-13T22:47:54.381 INFO:tasks.workunit.client.0.smithi102.stdout:7/62: dwrite f3 [0,4194304] 0 2024-05-13T22:47:54.386 INFO:tasks.workunit.client.0.smithi102.stdout:6/95: truncate f1 5899976 0 2024-05-13T22:47:54.390 INFO:tasks.workunit.client.0.smithi102.stdout:1/70: link d2/d4/d5/fa d2/d4/d5/f17 0 2024-05-13T22:47:54.397 INFO:tasks.workunit.client.0.smithi102.stdout:6/96: creat d9/db/d14/f1d x:0 0 0 2024-05-13T22:47:54.398 INFO:tasks.workunit.client.0.smithi102.stdout:7/63: mknod d6/dd/c10 0 2024-05-13T22:47:54.399 INFO:tasks.workunit.client.0.smithi102.stdout:4/94: getdents d3/d19/d23 0 2024-05-13T22:47:54.400 INFO:tasks.workunit.client.0.smithi102.stdout:2/61: write d7/ff [5176006,66893] 0 2024-05-13T22:47:54.400 INFO:tasks.workunit.client.0.smithi102.stdout:4/95: readlink d3/l11 0 2024-05-13T22:47:54.403 INFO:tasks.workunit.client.0.smithi102.stdout:1/71: rmdir d2/d4 39 2024-05-13T22:47:54.405 INFO:tasks.workunit.client.0.smithi102.stdout:1/72: truncate d2/f13 975907 0 2024-05-13T22:47:54.407 INFO:tasks.workunit.client.0.smithi102.stdout:4/96: dwrite d3/fa [0,4194304] 0 2024-05-13T22:47:54.407 INFO:tasks.workunit.client.0.smithi102.stdout:6/97: mknod d9/c1e 0 2024-05-13T22:47:54.426 INFO:tasks.workunit.client.0.smithi102.stdout:4/97: creat d3/d4/d6/de/d1f/f28 x:0 0 0 2024-05-13T22:47:54.426 INFO:tasks.workunit.client.0.smithi102.stdout:7/64: link d6/fc d6/dd/de/f11 0 2024-05-13T22:47:54.426 INFO:tasks.workunit.client.0.smithi102.stdout:7/65: fdatasync f4 0 2024-05-13T22:47:54.426 INFO:tasks.workunit.client.0.smithi102.stdout:2/62: truncate d7/d9/fb 2778914 0 2024-05-13T22:47:54.428 INFO:tasks.workunit.client.0.smithi102.stdout:4/98: fdatasync d3/d4/d6/de/f15 0 2024-05-13T22:47:54.433 INFO:tasks.workunit.client.0.smithi102.stdout:7/66: creat d6/dd/de/f12 x:0 0 0 2024-05-13T22:47:54.435 INFO:tasks.workunit.client.0.smithi102.stdout:1/73: getdents d2 0 2024-05-13T22:47:54.435 INFO:tasks.workunit.client.0.smithi102.stdout:7/67: dread - d6/dd/de/f12 zero size 2024-05-13T22:47:54.436 INFO:tasks.workunit.client.0.smithi102.stdout:4/99: dwrite d3/f10 [0,4194304] 0 2024-05-13T22:47:54.437 INFO:tasks.workunit.client.0.smithi102.stdout:4/100: fsync d3/db/f1d 0 2024-05-13T22:47:54.446 INFO:tasks.workunit.client.0.smithi102.stdout:4/101: symlink d3/d19/l29 0 2024-05-13T22:47:54.448 INFO:tasks.workunit.client.0.smithi102.stdout:4/102: creat d3/d4/d6/f2a x:0 0 0 2024-05-13T22:47:54.453 INFO:tasks.workunit.client.0.smithi102.stdout:7/68: rename d6/dd to d6/d13 0 2024-05-13T22:47:54.455 INFO:tasks.workunit.client.0.smithi102.stdout:4/103: dwrite d3/d4/d6/de/d1f/f28 [0,4194304] 0 2024-05-13T22:47:54.457 INFO:tasks.workunit.client.0.smithi102.stdout:7/69: rmdir d6 39 2024-05-13T22:47:54.460 INFO:tasks.workunit.client.0.smithi102.stdout:2/63: write d7/d9/fb [1349894,3819] 0 2024-05-13T22:47:54.461 INFO:tasks.workunit.client.0.smithi102.stdout:4/104: dread d3/d4/d6/de/d1f/f28 [0,4194304] 0 2024-05-13T22:47:54.461 INFO:tasks.workunit.client.0.smithi102.stdout:2/64: read d7/ff [3289542,100926] 0 2024-05-13T22:47:54.462 INFO:tasks.workunit.client.0.smithi102.stdout:2/65: fsync d7/ff 0 2024-05-13T22:47:54.462 INFO:tasks.workunit.client.0.smithi102.stdout:6/98: truncate f1 4098894 0 2024-05-13T22:47:54.468 INFO:tasks.workunit.client.0.smithi102.stdout:2/66: dread d7/ff [4194304,4194304] 0 2024-05-13T22:47:54.469 INFO:tasks.workunit.client.0.smithi102.stdout:4/105: rename d3/db to d3/d22/d2b 0 2024-05-13T22:47:54.472 INFO:tasks.workunit.client.0.smithi102.stdout:6/99: symlink d9/db/l1f 0 2024-05-13T22:47:54.474 INFO:tasks.workunit.client.0.smithi102.stdout:7/70: truncate d6/d13/de/f11 687648 0 2024-05-13T22:47:54.474 INFO:tasks.workunit.client.0.smithi102.stdout:7/71: truncate f3 4382883 0 2024-05-13T22:47:54.478 INFO:tasks.workunit.client.0.smithi102.stdout:6/100: mknod d9/c20 0 2024-05-13T22:47:54.478 INFO:tasks.workunit.client.0.smithi102.stdout:9/103: sync 2024-05-13T22:47:54.478 INFO:tasks.workunit.client.0.smithi102.stdout:0/87: sync 2024-05-13T22:47:54.478 INFO:tasks.workunit.client.0.smithi102.stdout:8/36: sync 2024-05-13T22:47:54.478 INFO:tasks.workunit.client.0.smithi102.stdout:6/101: chown f5 0 1 2024-05-13T22:47:54.479 INFO:tasks.workunit.client.0.smithi102.stdout:9/104: fdatasync d1/d6/f11 0 2024-05-13T22:47:54.479 INFO:tasks.workunit.client.0.smithi102.stdout:8/37: write d2/f4 [11608077,62352] 0 2024-05-13T22:47:54.484 INFO:tasks.workunit.client.0.smithi102.stdout:4/106: link d3/d4/d6/f18 d3/d22/d2b/f2c 0 2024-05-13T22:47:54.485 INFO:tasks.workunit.client.0.smithi102.stdout:4/107: dread - d3/d22/d2b/f17 zero size 2024-05-13T22:47:54.489 INFO:tasks.workunit.client.0.smithi102.stdout:6/102: creat d9/f21 x:0 0 0 2024-05-13T22:47:54.490 INFO:tasks.workunit.client.0.smithi102.stdout:6/103: fsync d9/db/d14/f1d 0 2024-05-13T22:47:54.490 INFO:tasks.workunit.client.0.smithi102.stdout:9/105: symlink d1/d6/db/d2c/l2f 0 2024-05-13T22:47:54.490 INFO:tasks.workunit.client.0.smithi102.stdout:6/104: chown d9/db/d14/l18 596 1 2024-05-13T22:47:54.492 INFO:tasks.workunit.client.0.smithi102.stdout:2/67: truncate d7/ff 642849 0 2024-05-13T22:47:54.493 INFO:tasks.workunit.client.0.smithi102.stdout:2/68: write d7/d9/fb [3483004,34990] 0 2024-05-13T22:47:54.494 INFO:tasks.workunit.client.0.smithi102.stdout:4/108: mknod d3/d22/c2d 0 2024-05-13T22:47:54.495 INFO:tasks.workunit.client.0.smithi102.stdout:2/69: fsync d7/d9/fb 0 2024-05-13T22:47:54.496 INFO:tasks.workunit.client.0.smithi102.stdout:0/88: rmdir d1 39 2024-05-13T22:47:54.499 INFO:tasks.workunit.client.0.smithi102.stdout:9/106: write d1/f17 [20634,9790] 0 2024-05-13T22:47:54.503 INFO:tasks.workunit.client.0.smithi102.stdout:7/72: dread d6/d13/de/f11 [0,4194304] 0 2024-05-13T22:47:54.504 INFO:tasks.workunit.client.0.smithi102.stdout:4/109: rename d3/fa to d3/d4/f2e 0 2024-05-13T22:47:54.505 INFO:tasks.workunit.client.0.smithi102.stdout:0/89: readlink d1/l10 0 2024-05-13T22:47:54.508 INFO:tasks.workunit.client.0.smithi102.stdout:6/105: link f6 d9/db/f22 0 2024-05-13T22:47:54.512 INFO:tasks.workunit.client.0.smithi102.stdout:0/90: unlink d1/d7/l11 0 2024-05-13T22:47:54.513 INFO:tasks.workunit.client.0.smithi102.stdout:0/91: readlink d1/l10 0 2024-05-13T22:47:54.515 INFO:tasks.workunit.client.0.smithi102.stdout:9/107: unlink d1/l1a 0 2024-05-13T22:47:54.516 INFO:tasks.workunit.client.0.smithi102.stdout:9/108: dread - d1/d6/db/f1b zero size 2024-05-13T22:47:54.516 INFO:tasks.workunit.client.0.smithi102.stdout:4/110: dwrite d3/d4/f2e [0,4194304] 0 2024-05-13T22:47:54.521 INFO:tasks.workunit.client.0.smithi102.stdout:6/106: unlink f6 0 2024-05-13T22:47:54.527 INFO:tasks.workunit.client.0.smithi102.stdout:6/107: dwrite f8 [0,4194304] 0 2024-05-13T22:47:54.528 INFO:tasks.workunit.client.0.smithi102.stdout:4/111: unlink d3/d4/c26 0 2024-05-13T22:47:54.529 INFO:tasks.workunit.client.0.smithi102.stdout:9/109: mkdir d1/d6/db/d30 0 2024-05-13T22:47:54.532 INFO:tasks.workunit.client.0.smithi102.stdout:2/70: link c2 d7/d9/d10/c12 0 2024-05-13T22:47:54.535 INFO:tasks.workunit.client.0.smithi102.stdout:0/92: link d1/d7/fb d1/f16 0 2024-05-13T22:47:54.535 INFO:tasks.workunit.client.0.smithi102.stdout:0/93: chown d1/d7/f13 1587 1 2024-05-13T22:47:54.536 INFO:tasks.workunit.client.0.smithi102.stdout:0/94: chown d1/d7/f15 363493214 1 2024-05-13T22:47:54.542 INFO:tasks.workunit.client.0.smithi102.stdout:2/71: creat d7/d9/f13 x:0 0 0 2024-05-13T22:47:54.542 INFO:tasks.workunit.client.0.smithi102.stdout:7/73: link d6/d13/de/f11 d6/d13/f14 0 2024-05-13T22:47:54.544 INFO:tasks.workunit.client.0.smithi102.stdout:7/74: dread - d6/d13/de/f12 zero size 2024-05-13T22:47:54.544 INFO:tasks.workunit.client.0.smithi102.stdout:0/95: write d1/f6 [5135976,85818] 0 2024-05-13T22:47:54.545 INFO:tasks.workunit.client.0.smithi102.stdout:2/72: truncate d7/d9/f13 562966 0 2024-05-13T22:47:54.547 INFO:tasks.workunit.client.0.smithi102.stdout:2/73: fdatasync d7/d9/fb 0 2024-05-13T22:47:54.547 INFO:tasks.workunit.client.0.smithi102.stdout:2/74: chown d7/d9 226284 1 2024-05-13T22:47:54.551 INFO:tasks.workunit.client.0.smithi102.stdout:0/96: creat d1/d7/f17 x:0 0 0 2024-05-13T22:47:54.555 INFO:tasks.workunit.client.0.smithi102.stdout:9/110: creat d1/f31 x:0 0 0 2024-05-13T22:47:54.555 INFO:tasks.workunit.client.0.smithi102.stdout:9/111: read - d1/d6/f1d zero size 2024-05-13T22:47:54.556 INFO:tasks.workunit.client.0.smithi102.stdout:9/112: dread - d1/d6/f1d zero size 2024-05-13T22:47:54.560 INFO:tasks.workunit.client.0.smithi102.stdout:9/113: dread d1/d6/db/d2c/d2d/f2e [0,4194304] 0 2024-05-13T22:47:54.564 INFO:tasks.workunit.client.0.smithi102.stdout:0/97: rename d1/f6 to d1/d7/f18 0 2024-05-13T22:47:54.570 INFO:tasks.workunit.client.0.smithi102.stdout:6/108: dwrite d9/db/f22 [4194304,4194304] 0 2024-05-13T22:47:54.573 INFO:tasks.workunit.client.0.smithi102.stdout:9/114: mkdir d1/d6/d32 0 2024-05-13T22:47:54.577 INFO:tasks.workunit.client.0.smithi102.stdout:2/75: fdatasync d7/ff 0 2024-05-13T22:47:54.577 INFO:tasks.workunit.client.0.smithi102.stdout:2/76: chown d7/d9/fb 5520914 1 2024-05-13T22:47:54.584 INFO:tasks.workunit.client.0.smithi102.stdout:7/75: getdents d6/d13/de 0 2024-05-13T22:47:54.587 INFO:tasks.workunit.client.0.smithi102.stdout:9/115: symlink d1/d6/db/d2c/d2d/l33 0 2024-05-13T22:47:54.588 INFO:tasks.workunit.client.0.smithi102.stdout:0/98: link d1/d7/f17 d1/de/f19 0 2024-05-13T22:47:54.589 INFO:tasks.workunit.client.0.smithi102.stdout:0/99: write d1/f4 [3801114,88166] 0 2024-05-13T22:47:54.590 INFO:tasks.workunit.client.0.smithi102.stdout:2/77: rmdir d7/d9/d10 39 2024-05-13T22:47:54.592 INFO:tasks.workunit.client.0.smithi102.stdout:7/76: mkdir d6/d15 0 2024-05-13T22:47:54.593 INFO:tasks.workunit.client.0.smithi102.stdout:9/116: creat d1/d6/db/d2c/d2d/f34 x:0 0 0 2024-05-13T22:47:54.594 INFO:tasks.workunit.client.0.smithi102.stdout:9/117: dread - d1/d6/db/d2c/d2d/f34 zero size 2024-05-13T22:47:54.596 INFO:tasks.workunit.client.0.smithi102.stdout:0/100: creat d1/d7/f1a x:0 0 0 2024-05-13T22:47:54.597 INFO:tasks.workunit.client.0.smithi102.stdout:0/101: write d1/d7/f15 [1037983,15729] 0 2024-05-13T22:47:54.597 INFO:tasks.workunit.client.0.smithi102.stdout:0/102: dread - d1/d7/f1a zero size 2024-05-13T22:47:54.599 INFO:tasks.workunit.client.0.smithi102.stdout:7/77: dread d6/fc [0,4194304] 0 2024-05-13T22:47:54.599 INFO:tasks.workunit.client.0.smithi102.stdout:7/78: chown d6/f7 16466292 1 2024-05-13T22:47:54.600 INFO:tasks.workunit.client.0.smithi102.stdout:9/118: rename d1/d6/ld to d1/d6/db/l35 0 2024-05-13T22:47:54.601 INFO:tasks.workunit.client.0.smithi102.stdout:2/78: mkdir d7/d9/d14 0 2024-05-13T22:47:54.603 INFO:tasks.workunit.client.0.smithi102.stdout:0/103: creat d1/f1b x:0 0 0 2024-05-13T22:47:54.604 INFO:tasks.workunit.client.0.smithi102.stdout:0/104: dread - d1/de/f19 zero size 2024-05-13T22:47:54.606 INFO:tasks.workunit.client.0.smithi102.stdout:9/119: rename d1/d6/db/f25 to d1/d6/db/d30/f36 0 2024-05-13T22:47:54.607 INFO:tasks.workunit.client.0.smithi102.stdout:6/109: creat d9/db/d14/f23 x:0 0 0 2024-05-13T22:47:54.610 INFO:tasks.workunit.client.0.smithi102.stdout:2/79: mkdir d7/d15 0 2024-05-13T22:47:54.612 INFO:tasks.workunit.client.0.smithi102.stdout:9/120: dwrite d1/f31 [0,4194304] 0 2024-05-13T22:47:54.619 INFO:tasks.workunit.client.0.smithi102.stdout:2/80: unlink l1 0 2024-05-13T22:47:54.621 INFO:tasks.workunit.client.0.smithi102.stdout:0/105: dwrite d1/d7/f18 [0,4194304] 0 2024-05-13T22:47:54.621 INFO:tasks.workunit.client.0.smithi102.stdout:9/121: write d1/f16 [4449293,24358] 0 2024-05-13T22:47:54.623 INFO:tasks.workunit.client.0.smithi102.stdout:2/81: dread d7/ff [0,4194304] 0 2024-05-13T22:47:54.624 INFO:tasks.workunit.client.0.smithi102.stdout:2/82: stat d7/l8 0 2024-05-13T22:47:54.625 INFO:tasks.workunit.client.0.smithi102.stdout:2/83: stat d7/d9/d14 0 2024-05-13T22:47:54.630 INFO:tasks.workunit.client.0.smithi102.stdout:9/122: dwrite d1/d6/f1d [0,4194304] 0 2024-05-13T22:47:54.634 INFO:tasks.workunit.client.0.smithi102.stdout:0/106: link d1/f1b d1/de/f1c 0 2024-05-13T22:47:54.634 INFO:tasks.workunit.client.0.smithi102.stdout:0/107: chown d1/d7/f13 7792 1 2024-05-13T22:47:54.635 INFO:tasks.workunit.client.0.smithi102.stdout:0/108: chown d1/c14 1701666189 1 2024-05-13T22:47:54.636 INFO:tasks.workunit.client.0.smithi102.stdout:9/123: mknod d1/d6/db/d30/c37 0 2024-05-13T22:47:54.637 INFO:tasks.workunit.client.0.smithi102.stdout:9/124: read - d1/d6/db/f1c zero size 2024-05-13T22:47:54.638 INFO:tasks.workunit.client.0.smithi102.stdout:9/125: fdatasync d1/d6/f11 0 2024-05-13T22:47:54.643 INFO:tasks.workunit.client.0.smithi102.stdout:9/126: dread d1/f16 [0,4194304] 0 2024-05-13T22:47:54.644 INFO:tasks.workunit.client.0.smithi102.stdout:9/127: read - d1/d6/db/d2c/d2d/f34 zero size 2024-05-13T22:47:54.650 INFO:tasks.workunit.client.0.smithi102.stdout:2/84: link d7/d9/ca d7/d9/d10/c16 0 2024-05-13T22:47:54.653 INFO:tasks.workunit.client.0.smithi102.stdout:2/85: fdatasync d7/ff 0 2024-05-13T22:47:54.657 INFO:tasks.workunit.client.0.smithi102.stdout:2/86: mkdir d7/d9/d11/d17 0 2024-05-13T22:47:54.661 INFO:tasks.workunit.client.0.smithi102.stdout:0/109: dwrite d1/f9 [0,4194304] 0 2024-05-13T22:47:54.661 INFO:tasks.workunit.client.0.smithi102.stdout:2/87: dread d7/d9/f13 [0,4194304] 0 2024-05-13T22:47:54.661 INFO:tasks.workunit.client.0.smithi102.stdout:0/110: chown d1 13455 1 2024-05-13T22:47:54.662 INFO:tasks.workunit.client.0.smithi102.stdout:2/88: stat d7/d9/f13 0 2024-05-13T22:47:54.666 INFO:tasks.workunit.client.0.smithi102.stdout:0/111: rename d1/f16 to d1/de/f1d 0 2024-05-13T22:47:54.666 INFO:tasks.workunit.client.0.smithi102.stdout:0/112: chown d1/cc 15 1 2024-05-13T22:47:54.667 INFO:tasks.workunit.client.0.smithi102.stdout:0/113: stat d1/d7/fb 0 2024-05-13T22:47:54.700 INFO:tasks.workunit.client.0.smithi102.stdout:2/89: unlink d7/d9/d10/c16 0 2024-05-13T22:47:54.700 INFO:tasks.workunit.client.0.smithi102.stdout:2/90: chown d7/d9 0 1 2024-05-13T22:47:54.701 INFO:tasks.workunit.client.0.smithi102.stdout:0/114: read - d1/f1b zero size 2024-05-13T22:47:54.702 INFO:tasks.workunit.client.0.smithi102.stdout:2/91: creat d7/d9/d11/d17/f18 x:0 0 0 2024-05-13T22:47:54.705 INFO:tasks.workunit.client.0.smithi102.stdout:2/92: getdents d7/d9/d11 0 2024-05-13T22:47:54.706 INFO:tasks.workunit.client.0.smithi102.stdout:2/93: dread d7/ff [0,4194304] 0 2024-05-13T22:47:54.712 INFO:tasks.workunit.client.0.smithi102.stdout:2/94: rmdir d7 39 2024-05-13T22:47:54.715 INFO:tasks.workunit.client.0.smithi102.stdout:2/95: creat d7/d15/f19 x:0 0 0 2024-05-13T22:47:54.720 INFO:tasks.workunit.client.0.smithi102.stdout:2/96: write d7/ff [1414252,122879] 0 2024-05-13T22:47:54.722 INFO:tasks.workunit.client.0.smithi102.stdout:2/97: fdatasync d7/ff 0 2024-05-13T22:47:54.725 INFO:tasks.workunit.client.0.smithi102.stdout:2/98: dread d7/d9/f13 [0,4194304] 0 2024-05-13T22:47:54.727 INFO:tasks.workunit.client.0.smithi102.stdout:2/99: creat d7/d9/d14/f1a x:0 0 0 2024-05-13T22:47:54.730 INFO:tasks.workunit.client.0.smithi102.stdout:2/100: rename d7/d9/d11/d17 to d7/d9/d10/d1b 0 2024-05-13T22:47:54.732 INFO:tasks.workunit.client.0.smithi102.stdout:2/101: mknod d7/d9/d10/d1b/c1c 0 2024-05-13T22:47:54.734 INFO:tasks.workunit.client.0.smithi102.stdout:2/102: symlink d7/d9/d10/d1b/l1d 0 2024-05-13T22:47:54.744 INFO:tasks.workunit.client.0.smithi102.stdout:2/103: write d7/d9/f13 [132850,108824] 0 2024-05-13T22:47:54.746 INFO:tasks.workunit.client.0.smithi102.stdout:2/104: mkdir d7/d15/d1e 0 2024-05-13T22:47:54.748 INFO:tasks.workunit.client.0.smithi102.stdout:2/105: unlink d7/d9/d10/c12 0 2024-05-13T22:47:54.750 INFO:tasks.workunit.client.0.smithi102.stdout:2/106: creat d7/d15/f1f x:0 0 0 2024-05-13T22:47:54.751 INFO:tasks.workunit.client.0.smithi102.stdout:2/107: write d7/d9/d14/f1a [904007,3716] 0 2024-05-13T22:47:54.754 INFO:tasks.workunit.client.0.smithi102.stdout:2/108: truncate d7/d9/fb 3110010 0 2024-05-13T22:47:54.756 INFO:tasks.workunit.client.0.smithi102.stdout:2/109: symlink d7/d9/l20 0 2024-05-13T22:47:54.756 INFO:tasks.workunit.client.0.smithi102.stdout:2/110: fsync d7/d9/f13 0 2024-05-13T22:47:54.763 INFO:tasks.workunit.client.0.smithi102.stdout:2/111: dwrite d7/d15/f19 [0,4194304] 0 2024-05-13T22:47:54.766 INFO:tasks.workunit.client.0.smithi102.stdout:2/112: read d7/d9/d14/f1a [843419,75894] 0 2024-05-13T22:47:54.767 INFO:tasks.workunit.client.0.smithi102.stdout:2/113: rename d7 to d7/d9/d10/d21 22 2024-05-13T22:47:54.771 INFO:tasks.workunit.client.0.smithi102.stdout:2/114: dread d7/d15/f19 [0,4194304] 0 2024-05-13T22:47:54.779 INFO:tasks.workunit.client.0.smithi102.stdout:2/115: dread d7/d9/d14/f1a [0,4194304] 0 2024-05-13T22:47:54.787 INFO:tasks.workunit.client.0.smithi102.stdout:2/116: dwrite d7/d9/f13 [0,4194304] 0 2024-05-13T22:47:54.789 INFO:tasks.workunit.client.0.smithi102.stdout:2/117: symlink d7/d9/d14/l22 0 2024-05-13T22:47:54.789 INFO:tasks.workunit.client.0.smithi102.stdout:2/118: chown d7/d9/f13 336 1 2024-05-13T22:47:54.793 INFO:tasks.workunit.client.0.smithi102.stdout:2/119: unlink d7/d9/l20 0 2024-05-13T22:47:54.793 INFO:tasks.workunit.client.0.smithi102.stdout:2/120: read d7/d9/f13 [550580,17564] 0 2024-05-13T22:47:54.803 INFO:tasks.workunit.client.0.smithi102.stdout:2/121: link d7/d9/ca d7/d9/c23 0 2024-05-13T22:47:54.807 INFO:tasks.workunit.client.0.smithi102.stdout:2/122: read d7/d9/fb [951688,34347] 0 2024-05-13T22:47:54.807 INFO:tasks.workunit.client.0.smithi102.stdout:2/123: chown d7/d9/fb 16451762 1 2024-05-13T22:47:54.809 INFO:tasks.workunit.client.0.smithi102.stdout:2/124: write d7/d9/f13 [3215075,22762] 0 2024-05-13T22:47:54.814 INFO:tasks.workunit.client.0.smithi102.stdout:2/125: rename d7/d9/le to d7/l24 0 2024-05-13T22:47:54.816 INFO:tasks.workunit.client.0.smithi102.stdout:2/126: rmdir d7/d9/d14 39 2024-05-13T22:47:54.824 INFO:tasks.workunit.client.0.smithi102.stdout:2/127: write d7/d9/fb [1058789,83888] 0 2024-05-13T22:47:54.825 INFO:tasks.workunit.client.0.smithi102.stdout:2/128: stat d7/ff 0 2024-05-13T22:47:54.827 INFO:tasks.workunit.client.0.smithi102.stdout:2/129: mkdir d7/d9/d10/d25 0 2024-05-13T22:47:54.833 INFO:tasks.workunit.client.0.smithi102.stdout:2/130: dwrite d7/d9/fb [0,4194304] 0 2024-05-13T22:47:54.836 INFO:tasks.workunit.client.0.smithi102.stdout:2/131: unlink d7/d9/ca 0 2024-05-13T22:47:54.837 INFO:tasks.workunit.client.0.smithi102.stdout:2/132: write d7/d9/fb [3644317,15388] 0 2024-05-13T22:47:54.838 INFO:tasks.workunit.client.0.smithi102.stdout:2/133: read d7/d9/d14/f1a [135040,4429] 0 2024-05-13T22:47:54.840 INFO:tasks.workunit.client.0.smithi102.stdout:2/134: fsync d7/d15/f19 0 2024-05-13T22:47:54.844 INFO:tasks.workunit.client.0.smithi102.stdout:3/131: sync 2024-05-13T22:47:54.845 INFO:tasks.workunit.client.0.smithi102.stdout:3/132: chown de/d10/c15 116 1 2024-05-13T22:47:54.850 INFO:tasks.workunit.client.0.smithi102.stdout:2/135: truncate d7/d9/d14/f1a 118145 0 2024-05-13T22:47:54.854 INFO:tasks.workunit.client.0.smithi102.stdout:2/136: creat d7/d15/d1e/f26 x:0 0 0 2024-05-13T22:47:54.856 INFO:tasks.workunit.client.0.smithi102.stdout:2/137: truncate d7/d15/d1e/f26 213992 0 2024-05-13T22:47:54.858 INFO:tasks.workunit.client.0.smithi102.stdout:3/133: dwrite f2 [0,4194304] 0 2024-05-13T22:47:54.860 INFO:tasks.workunit.client.0.smithi102.stdout:2/138: mkdir d7/d9/d10/d25/d27 0 2024-05-13T22:47:54.860 INFO:tasks.workunit.client.0.smithi102.stdout:2/139: chown d7 0 1 2024-05-13T22:47:54.861 INFO:tasks.workunit.client.0.smithi102.stdout:3/134: creat de/d10/d2a/f30 x:0 0 0 2024-05-13T22:47:54.861 INFO:tasks.workunit.client.0.smithi102.stdout:2/140: chown d7/d9/d10/d25/d27 2879246 1 2024-05-13T22:47:54.866 INFO:tasks.workunit.client.0.smithi102.stdout:2/141: symlink d7/d9/d10/d25/l28 0 2024-05-13T22:47:54.869 INFO:tasks.workunit.client.0.smithi102.stdout:2/142: readlink l5 0 2024-05-13T22:47:54.870 INFO:tasks.workunit.client.0.smithi102.stdout:2/143: fsync d7/d15/f19 0 2024-05-13T22:47:54.873 INFO:tasks.workunit.client.0.smithi102.stdout:2/144: creat d7/d9/f29 x:0 0 0 2024-05-13T22:47:54.877 INFO:tasks.workunit.client.0.smithi102.stdout:2/145: dread d7/d9/fb [0,4194304] 0 2024-05-13T22:47:54.878 INFO:tasks.workunit.client.0.smithi102.stdout:3/135: write f7 [3495071,17444] 0 2024-05-13T22:47:54.879 INFO:tasks.workunit.client.0.smithi102.stdout:2/146: symlink d7/d9/d10/d25/l2a 0 2024-05-13T22:47:54.881 INFO:tasks.workunit.client.0.smithi102.stdout:2/147: readlink d7/d9/d10/d25/l2a 0 2024-05-13T22:47:54.881 INFO:tasks.workunit.client.0.smithi102.stdout:3/136: unlink de/d1c/l1e 0 2024-05-13T22:47:54.884 INFO:tasks.workunit.client.0.smithi102.stdout:3/137: rename de/d1c/f1f to de/d10/f31 0 2024-05-13T22:47:54.884 INFO:tasks.workunit.client.0.smithi102.stdout:2/148: dread d7/d15/f19 [0,4194304] 0 2024-05-13T22:47:54.885 INFO:tasks.workunit.client.0.smithi102.stdout:2/149: chown d7/d9/d10/d1b 9 1 2024-05-13T22:47:54.886 INFO:tasks.workunit.client.0.smithi102.stdout:3/138: unlink de/d10/d2a/f30 0 2024-05-13T22:47:54.887 INFO:tasks.workunit.client.0.smithi102.stdout:3/139: truncate de/d10/f31 928166 0 2024-05-13T22:47:54.888 INFO:tasks.workunit.client.0.smithi102.stdout:3/140: fdatasync de/f19 0 2024-05-13T22:47:54.890 INFO:tasks.workunit.client.0.smithi102.stdout:2/150: rename d7/ld to d7/l2b 0 2024-05-13T22:47:54.890 INFO:tasks.workunit.client.0.smithi102.stdout:2/151: fsync d7/d15/f19 0 2024-05-13T22:47:54.891 INFO:tasks.workunit.client.0.smithi102.stdout:2/152: dread - d7/d9/f29 zero size 2024-05-13T22:47:54.892 INFO:tasks.workunit.client.0.smithi102.stdout:3/141: dread de/d10/f11 [0,4194304] 0 2024-05-13T22:47:54.893 INFO:tasks.workunit.client.0.smithi102.stdout:2/153: symlink d7/d9/d10/l2c 0 2024-05-13T22:47:54.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:54 smithi130 ceph-mon[36591]: pgmap v14: 65 pgs: 65 active+clean; 232 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 12 MiB/s wr, 430 op/s 2024-05-13T22:47:54.901 INFO:tasks.workunit.client.0.smithi102.stdout:2/154: truncate d7/ff 431751 0 2024-05-13T22:47:54.902 INFO:tasks.workunit.client.0.smithi102.stdout:2/155: write d7/d15/f19 [3296135,63334] 0 2024-05-13T22:47:54.908 INFO:tasks.workunit.client.0.smithi102.stdout:3/142: rmdir de 39 2024-05-13T22:47:54.911 INFO:tasks.workunit.client.0.smithi102.stdout:3/143: dread fa [0,4194304] 0 2024-05-13T22:47:54.912 INFO:tasks.workunit.client.0.smithi102.stdout:5/127: sync 2024-05-13T22:47:54.916 INFO:tasks.workunit.client.0.smithi102.stdout:3/144: mknod de/c32 0 2024-05-13T22:47:54.922 INFO:tasks.workunit.client.0.smithi102.stdout:3/145: dwrite de/d10/d2a/f2e [0,4194304] 0 2024-05-13T22:47:54.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:54 smithi102 ceph-mon[28599]: pgmap v14: 65 pgs: 65 active+clean; 232 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 12 MiB/s wr, 430 op/s 2024-05-13T22:47:54.972 INFO:tasks.workunit.client.0.smithi102.stdout:3/146: dwrite f7 [0,4194304] 0 2024-05-13T22:47:54.975 INFO:tasks.workunit.client.0.smithi102.stdout:3/147: mkdir de/d10/d2a/d33 0 2024-05-13T22:47:54.980 INFO:tasks.workunit.client.0.smithi102.stdout:3/148: mkdir de/d10/d34 0 2024-05-13T22:47:54.981 INFO:tasks.workunit.client.0.smithi102.stdout:5/128: dread f9 [0,4194304] 0 2024-05-13T22:47:54.987 INFO:tasks.workunit.client.0.smithi102.stdout:3/149: dread de/d10/d2a/f2e [0,4194304] 0 2024-05-13T22:47:54.989 INFO:tasks.workunit.client.0.smithi102.stdout:5/129: dwrite de/d10/f2e [0,4194304] 0 2024-05-13T22:47:54.999 INFO:tasks.workunit.client.0.smithi102.stdout:3/150: dwrite de/d10/f27 [0,4194304] 0 2024-05-13T22:47:55.000 INFO:tasks.workunit.client.0.smithi102.stdout:5/130: link de/f13 de/d10/f32 0 2024-05-13T22:47:55.002 INFO:tasks.workunit.client.0.smithi102.stdout:5/131: mkdir de/d33 0 2024-05-13T22:47:55.003 INFO:tasks.workunit.client.0.smithi102.stdout:5/132: fdatasync de/f11 0 2024-05-13T22:47:55.004 INFO:tasks.workunit.client.0.smithi102.stdout:3/151: rename de/d10/l29 to de/d1c/l35 0 2024-05-13T22:47:55.004 INFO:tasks.workunit.client.0.smithi102.stdout:3/152: stat de/c16 0 2024-05-13T22:47:55.005 INFO:tasks.workunit.client.0.smithi102.stdout:3/153: write de/d10/f14 [10742,24940] 0 2024-05-13T22:47:55.011 INFO:tasks.workunit.client.0.smithi102.stdout:5/133: mkdir de/d33/d34 0 2024-05-13T22:47:55.014 INFO:tasks.workunit.client.0.smithi102.stdout:5/134: creat de/f35 x:0 0 0 2024-05-13T22:47:55.016 INFO:tasks.workunit.client.0.smithi102.stdout:3/154: dwrite fa [4194304,4194304] 0 2024-05-13T22:47:55.023 INFO:tasks.workunit.client.0.smithi102.stdout:5/135: rename de/l27 to de/d33/l36 0 2024-05-13T22:47:55.026 INFO:tasks.workunit.client.0.smithi102.stdout:5/136: rename de/d10/l1a to de/l37 0 2024-05-13T22:47:55.028 INFO:tasks.workunit.client.0.smithi102.stdout:5/137: symlink de/d25/l38 0 2024-05-13T22:47:55.034 INFO:tasks.workunit.client.0.smithi102.stdout:5/138: getdents de/d25/d2a 0 2024-05-13T22:47:55.037 INFO:tasks.workunit.client.0.smithi102.stdout:5/139: rename de/l23 to de/d10/l39 0 2024-05-13T22:47:55.037 INFO:tasks.workunit.client.0.smithi102.stdout:5/140: fdatasync de/f21 0 2024-05-13T22:47:55.045 INFO:tasks.workunit.client.0.smithi102.stdout:5/141: dwrite f9 [4194304,4194304] 0 2024-05-13T22:47:55.047 INFO:tasks.workunit.client.0.smithi102.stdout:5/142: write de/d10/f18 [877061,39296] 0 2024-05-13T22:47:55.052 INFO:tasks.workunit.client.0.smithi102.stdout:5/143: rename de/d10 to de/d33/d34/d3a 0 2024-05-13T22:47:55.054 INFO:tasks.workunit.client.0.smithi102.stdout:5/144: mknod de/d33/d34/c3b 0 2024-05-13T22:47:55.057 INFO:tasks.workunit.client.0.smithi102.stdout:5/145: creat de/d25/d2a/f3c x:0 0 0 2024-05-13T22:47:55.057 INFO:tasks.workunit.client.0.smithi102.stdout:5/146: dread - de/f1d zero size 2024-05-13T22:47:55.058 INFO:tasks.workunit.client.0.smithi102.stdout:5/147: read de/f14 [1272955,57906] 0 2024-05-13T22:47:55.071 INFO:tasks.workunit.client.0.smithi102.stdout:5/148: dwrite f2 [0,4194304] 0 2024-05-13T22:47:55.073 INFO:tasks.workunit.client.0.smithi102.stdout:5/149: write de/d33/d34/d3a/f2e [3404002,70742] 0 2024-05-13T22:47:55.076 INFO:tasks.workunit.client.0.smithi102.stdout:5/150: readlink de/l37 0 2024-05-13T22:47:55.079 INFO:tasks.workunit.client.0.smithi102.stdout:5/151: mknod de/d33/d34/d3a/c3d 0 2024-05-13T22:47:55.082 INFO:tasks.workunit.client.0.smithi102.stdout:5/152: mkdir de/d25/d2c/d3e 0 2024-05-13T22:47:55.089 INFO:tasks.workunit.client.0.smithi102.stdout:5/153: dwrite de/f1e [0,4194304] 0 2024-05-13T22:47:55.091 INFO:tasks.workunit.client.0.smithi102.stdout:5/154: readlink de/d33/l36 0 2024-05-13T22:47:55.096 INFO:tasks.workunit.client.0.smithi102.stdout:5/155: creat de/d25/d2c/d3e/f3f x:0 0 0 2024-05-13T22:47:55.103 INFO:tasks.workunit.client.0.smithi102.stdout:5/156: dwrite de/f11 [0,4194304] 0 2024-05-13T22:47:55.113 INFO:tasks.workunit.client.0.smithi102.stdout:5/157: rename de/d25/d2c/f31 to de/d33/d34/d3a/f40 0 2024-05-13T22:47:55.116 INFO:tasks.workunit.client.0.smithi102.stdout:5/158: dread de/f14 [4194304,4194304] 0 2024-05-13T22:47:55.116 INFO:tasks.workunit.client.0.smithi102.stdout:5/159: stat de/d33/d34 0 2024-05-13T22:47:55.119 INFO:tasks.workunit.client.0.smithi102.stdout:5/160: rmdir de/d33/d34/d3a 39 2024-05-13T22:47:55.121 INFO:tasks.workunit.client.0.smithi102.stdout:5/161: readlink lc 0 2024-05-13T22:47:55.124 INFO:tasks.workunit.client.0.smithi102.stdout:5/162: mkdir de/d41 0 2024-05-13T22:47:55.126 INFO:tasks.workunit.client.0.smithi102.stdout:5/163: read de/d33/d34/d3a/f2e [2137546,70273] 0 2024-05-13T22:47:55.130 INFO:tasks.workunit.client.0.smithi102.stdout:5/164: mknod de/d33/d34/d3a/c42 0 2024-05-13T22:47:55.159 INFO:tasks.workunit.client.0.smithi102.stdout:5/165: write de/d33/d34/d3a/f32 [1909608,76] 0 2024-05-13T22:47:55.167 INFO:tasks.workunit.client.0.smithi102.stdout:5/166: getdents de/d33/d34/d3a 0 2024-05-13T22:47:55.178 INFO:tasks.workunit.client.0.smithi102.stdout:5/167: dread fd [0,4194304] 0 2024-05-13T22:47:55.181 INFO:tasks.workunit.client.0.smithi102.stdout:5/168: mkdir de/d25/d43 0 2024-05-13T22:47:55.182 INFO:tasks.workunit.client.0.smithi102.stdout:5/169: truncate de/d25/f26 633785 0 2024-05-13T22:47:55.219 INFO:tasks.workunit.client.0.smithi102.stdout:5/170: write f0 [1407068,53165] 0 2024-05-13T22:47:55.224 INFO:tasks.workunit.client.0.smithi102.stdout:5/171: rmdir de/d41 0 2024-05-13T22:47:55.226 INFO:tasks.workunit.client.0.smithi102.stdout:5/172: creat de/d33/d34/d3a/f44 x:0 0 0 2024-05-13T22:47:55.228 INFO:tasks.workunit.client.0.smithi102.stdout:5/173: symlink de/d25/d2a/l45 0 2024-05-13T22:47:55.231 INFO:tasks.workunit.client.0.smithi102.stdout:5/174: dread f1 [0,4194304] 0 2024-05-13T22:47:55.538 INFO:tasks.workunit.client.0.smithi102.stdout:1/74: sync 2024-05-13T22:47:55.539 INFO:tasks.workunit.client.0.smithi102.stdout:1/75: chown d2/d4/d5/l8 30918 1 2024-05-13T22:47:55.539 INFO:tasks.workunit.client.0.smithi102.stdout:1/76: write d2/ff [419011,28531] 0 2024-05-13T22:47:55.542 INFO:tasks.workunit.client.0.smithi102.stdout:1/77: symlink d2/d4/l18 0 2024-05-13T22:47:55.548 INFO:tasks.workunit.client.0.smithi102.stdout:1/78: dwrite d2/f16 [0,4194304] 0 2024-05-13T22:47:55.555 INFO:tasks.workunit.client.0.smithi102.stdout:1/79: dwrite d2/ff [0,4194304] 0 2024-05-13T22:47:55.558 INFO:tasks.workunit.client.0.smithi102.stdout:1/80: mkdir d2/d4/d5/d19 0 2024-05-13T22:47:55.559 INFO:tasks.workunit.client.0.smithi102.stdout:1/81: fdatasync d2/f13 0 2024-05-13T22:47:55.561 INFO:tasks.workunit.client.0.smithi102.stdout:1/82: creat d2/d10/f1a x:0 0 0 2024-05-13T22:47:55.583 INFO:tasks.workunit.client.0.smithi102.stdout:1/83: dread d2/d4/d5/f15 [0,4194304] 0 2024-05-13T22:47:55.587 INFO:tasks.workunit.client.0.smithi102.stdout:1/84: rename d2/d4/d5/l8 to d2/d10/l1b 0 2024-05-13T22:47:55.588 INFO:tasks.workunit.client.0.smithi102.stdout:4/112: sync 2024-05-13T22:47:55.589 INFO:tasks.workunit.client.0.smithi102.stdout:7/79: sync 2024-05-13T22:47:55.589 INFO:tasks.workunit.client.0.smithi102.stdout:6/110: sync 2024-05-13T22:47:55.589 INFO:tasks.workunit.client.0.smithi102.stdout:0/115: sync 2024-05-13T22:47:55.589 INFO:tasks.workunit.client.0.smithi102.stdout:2/156: sync 2024-05-13T22:47:55.589 INFO:tasks.workunit.client.0.smithi102.stdout:8/38: sync 2024-05-13T22:47:55.589 INFO:tasks.workunit.client.0.smithi102.stdout:9/128: sync 2024-05-13T22:47:55.589 INFO:tasks.workunit.client.0.smithi102.stdout:3/155: sync 2024-05-13T22:47:55.589 INFO:tasks.workunit.client.0.smithi102.stdout:5/175: sync 2024-05-13T22:47:55.595 INFO:tasks.workunit.client.0.smithi102.stdout:3/156: dread f7 [0,4194304] 0 2024-05-13T22:47:55.596 INFO:tasks.workunit.client.0.smithi102.stdout:7/80: chown f4 723 1 2024-05-13T22:47:55.596 INFO:tasks.workunit.client.0.smithi102.stdout:9/129: read d1/f16 [2722839,130817] 0 2024-05-13T22:47:55.597 INFO:tasks.workunit.client.0.smithi102.stdout:6/111: dread - d9/ff zero size 2024-05-13T22:47:55.597 INFO:tasks.workunit.client.0.smithi102.stdout:5/176: creat de/d25/d2c/d3e/f46 x:0 0 0 2024-05-13T22:47:55.600 INFO:tasks.workunit.client.0.smithi102.stdout:7/81: write f3 [2727107,59558] 0 2024-05-13T22:47:55.600 INFO:tasks.workunit.client.0.smithi102.stdout:0/116: fdatasync d1/de/f19 0 2024-05-13T22:47:55.600 INFO:tasks.workunit.client.0.smithi102.stdout:1/85: dread d2/d4/d5/fa [0,4194304] 0 2024-05-13T22:47:55.601 INFO:tasks.workunit.client.0.smithi102.stdout:3/157: stat de/d21/l26 0 2024-05-13T22:47:55.609 INFO:tasks.workunit.client.0.smithi102.stdout:4/113: dwrite d3/d4/f1e [4194304,4194304] 0 2024-05-13T22:47:55.611 INFO:tasks.workunit.client.0.smithi102.stdout:4/114: write d3/d4/f2e [459977,55120] 0 2024-05-13T22:47:55.611 INFO:tasks.workunit.client.0.smithi102.stdout:5/177: creat de/d25/f47 x:0 0 0 2024-05-13T22:47:55.612 INFO:tasks.workunit.client.0.smithi102.stdout:4/115: fsync d3/d22/d2b/f17 0 2024-05-13T22:47:55.614 INFO:tasks.workunit.client.0.smithi102.stdout:5/178: write de/f13 [946874,94354] 0 2024-05-13T22:47:55.617 INFO:tasks.workunit.client.0.smithi102.stdout:8/39: rename d2/l6 to d2/d7/l8 0 2024-05-13T22:47:55.622 INFO:tasks.workunit.client.0.smithi102.stdout:3/158: mkdir de/d10/d2a/d36 0 2024-05-13T22:47:55.628 INFO:tasks.workunit.client.0.smithi102.stdout:0/117: rename d1/d7/f15 to d1/d7/f1e 0 2024-05-13T22:47:55.630 INFO:tasks.workunit.client.0.smithi102.stdout:6/112: write d9/db/d14/f1d [514095,102768] 0 2024-05-13T22:47:55.637 INFO:tasks.workunit.client.0.smithi102.stdout:5/179: mknod de/d33/c48 0 2024-05-13T22:47:55.641 INFO:tasks.workunit.client.0.smithi102.stdout:8/40: creat d2/d7/f9 x:0 0 0 2024-05-13T22:47:55.641 INFO:tasks.workunit.client.0.smithi102.stdout:8/41: dread - d2/d7/f9 zero size 2024-05-13T22:47:55.657 INFO:tasks.workunit.client.0.smithi102.stdout:8/42: creat d2/fa x:0 0 0 2024-05-13T22:47:55.661 INFO:tasks.workunit.client.0.smithi102.stdout:9/130: getdents d1 0 2024-05-13T22:47:55.662 INFO:tasks.workunit.client.0.smithi102.stdout:8/43: dread d2/f4 [8388608,4194304] 0 2024-05-13T22:47:55.662 INFO:tasks.workunit.client.0.smithi102.stdout:6/113: fsync f7 0 2024-05-13T22:47:55.663 INFO:tasks.workunit.client.0.smithi102.stdout:8/44: dread - d2/fa zero size 2024-05-13T22:47:55.665 INFO:tasks.workunit.client.0.smithi102.stdout:4/116: link d3/c12 d3/d19/c2f 0 2024-05-13T22:47:55.667 INFO:tasks.workunit.client.0.smithi102.stdout:3/159: creat de/f37 x:0 0 0 2024-05-13T22:47:55.668 INFO:tasks.workunit.client.0.smithi102.stdout:3/160: fsync de/d10/f12 0 2024-05-13T22:47:55.671 INFO:tasks.workunit.client.0.smithi102.stdout:9/131: write d1/d6/db/d2c/d2d/f2e [2317201,67330] 0 2024-05-13T22:47:55.671 INFO:tasks.workunit.client.0.smithi102.stdout:8/45: creat d2/d7/fb x:0 0 0 2024-05-13T22:47:55.682 INFO:tasks.workunit.client.0.smithi102.stdout:8/46: rename d2/c5 to d2/cc 0 2024-05-13T22:47:55.686 INFO:tasks.workunit.client.0.smithi102.stdout:9/132: dwrite d1/d6/db/d30/f36 [0,4194304] 0 2024-05-13T22:47:55.687 INFO:tasks.workunit.client.0.smithi102.stdout:5/180: creat de/d33/d34/d3a/f49 x:0 0 0 2024-05-13T22:47:55.688 INFO:tasks.workunit.client.0.smithi102.stdout:5/181: write de/f35 [638314,33937] 0 2024-05-13T22:47:55.689 INFO:tasks.workunit.client.0.smithi102.stdout:5/182: write de/f1d [849664,46264] 0 2024-05-13T22:47:55.691 INFO:tasks.workunit.client.0.smithi102.stdout:3/161: creat de/d10/d34/f38 x:0 0 0 2024-05-13T22:47:55.694 INFO:tasks.workunit.client.0.smithi102.stdout:8/47: symlink d2/d7/ld 0 2024-05-13T22:47:55.695 INFO:tasks.workunit.client.0.smithi102.stdout:1/86: write d2/d4/d5/f17 [846971,71430] 0 2024-05-13T22:47:55.697 INFO:tasks.workunit.client.0.smithi102.stdout:0/118: rmdir d1/d7 39 2024-05-13T22:47:55.697 INFO:tasks.workunit.client.0.smithi102.stdout:7/82: dwrite d6/fc [0,4194304] 0 2024-05-13T22:47:55.700 INFO:tasks.workunit.client.0.smithi102.stdout:1/87: truncate d2/d10/f12 180815 0 2024-05-13T22:47:55.701 INFO:tasks.workunit.client.0.smithi102.stdout:1/88: write d2/f16 [780353,128138] 0 2024-05-13T22:47:55.705 INFO:tasks.workunit.client.0.smithi102.stdout:3/162: symlink de/d10/d34/l39 0 2024-05-13T22:47:55.706 INFO:tasks.workunit.client.0.smithi102.stdout:3/163: chown de/d21/c2c 84588 1 2024-05-13T22:47:55.707 INFO:tasks.workunit.client.0.smithi102.stdout:3/164: fdatasync fd 0 2024-05-13T22:47:55.711 INFO:tasks.workunit.client.0.smithi102.stdout:9/133: creat d1/d6/d32/f38 x:0 0 0 2024-05-13T22:47:55.712 INFO:tasks.workunit.client.0.smithi102.stdout:1/89: dwrite d2/d10/f1a [0,4194304] 0 2024-05-13T22:47:55.715 INFO:tasks.workunit.client.0.smithi102.stdout:4/117: getdents d3/d4/d6 0 2024-05-13T22:47:55.719 INFO:tasks.workunit.client.0.smithi102.stdout:8/48: dwrite d2/f4 [0,4194304] 0 2024-05-13T22:47:55.731 INFO:tasks.workunit.client.0.smithi102.stdout:3/165: readlink de/d1c/l35 0 2024-05-13T22:47:55.731 INFO:tasks.workunit.client.0.smithi102.stdout:0/119: mknod d1/d7/c1f 0 2024-05-13T22:47:55.731 INFO:tasks.workunit.client.0.smithi102.stdout:8/49: dwrite d2/f4 [4194304,4194304] 0 2024-05-13T22:47:55.731 INFO:tasks.workunit.client.0.smithi102.stdout:0/120: chown d1/d7/f13 4783602 1 2024-05-13T22:47:55.734 INFO:tasks.workunit.client.0.smithi102.stdout:1/90: mknod d2/c1c 0 2024-05-13T22:47:55.739 INFO:tasks.workunit.client.0.smithi102.stdout:5/183: rename de/d25/d2c/d3e/f3f to de/d33/d34/f4a 0 2024-05-13T22:47:55.742 INFO:tasks.workunit.client.0.smithi102.stdout:3/166: creat de/d10/d2a/f3a x:0 0 0 2024-05-13T22:47:55.743 INFO:tasks.workunit.client.0.smithi102.stdout:3/167: write de/f37 [1028443,40329] 0 2024-05-13T22:47:55.745 INFO:tasks.workunit.client.0.smithi102.stdout:6/114: truncate d9/db/d14/f1d 403354 0 2024-05-13T22:47:55.748 INFO:tasks.workunit.client.0.smithi102.stdout:3/168: dread f2 [0,4194304] 0 2024-05-13T22:47:55.751 INFO:tasks.workunit.client.0.smithi102.stdout:8/50: fsync d2/f4 0 2024-05-13T22:47:55.752 INFO:tasks.workunit.client.0.smithi102.stdout:8/51: read d2/f4 [3499714,80139] 0 2024-05-13T22:47:55.753 INFO:tasks.workunit.client.0.smithi102.stdout:4/118: rename d3/d19/l29 to d3/d22/d2b/l30 0 2024-05-13T22:47:55.754 INFO:tasks.workunit.client.0.smithi102.stdout:8/52: fsync d2/fa 0 2024-05-13T22:47:55.757 INFO:tasks.workunit.client.0.smithi102.stdout:5/184: mkdir de/d25/d2a/d4b 0 2024-05-13T22:47:55.757 INFO:tasks.workunit.client.0.smithi102.stdout:3/169: dwrite de/d10/d2a/f2f [0,4194304] 0 2024-05-13T22:47:55.757 INFO:tasks.workunit.client.0.smithi102.stdout:8/53: write d2/d7/fb [855532,887] 0 2024-05-13T22:47:55.763 INFO:tasks.workunit.client.0.smithi102.stdout:4/119: unlink d3/d4/d6/f2a 0 2024-05-13T22:47:55.766 INFO:tasks.workunit.client.0.smithi102.stdout:5/185: unlink f1 0 2024-05-13T22:47:55.770 INFO:tasks.workunit.client.0.smithi102.stdout:6/115: mknod d9/db/c24 0 2024-05-13T22:47:55.772 INFO:tasks.workunit.client.0.smithi102.stdout:0/121: rmdir d1 39 2024-05-13T22:47:55.777 INFO:tasks.workunit.client.0.smithi102.stdout:4/120: creat d3/d4/f31 x:0 0 0 2024-05-13T22:47:55.780 INFO:tasks.workunit.client.0.smithi102.stdout:5/186: mknod de/d33/c4c 0 2024-05-13T22:47:55.781 INFO:tasks.workunit.client.0.smithi102.stdout:6/116: readlink d9/db/l1f 0 2024-05-13T22:47:55.782 INFO:tasks.workunit.client.0.smithi102.stdout:5/187: stat de/d25/d2c/d3e 0 2024-05-13T22:47:55.783 INFO:tasks.workunit.client.0.smithi102.stdout:9/134: dwrite d1/d6/db/f1c [0,4194304] 0 2024-05-13T22:47:55.783 INFO:tasks.workunit.client.0.smithi102.stdout:3/170: fsync de/d10/d2a/f3a 0 2024-05-13T22:47:55.786 INFO:tasks.workunit.client.0.smithi102.stdout:8/54: rename d2/cc to d2/d7/ce 0 2024-05-13T22:47:55.786 INFO:tasks.workunit.client.0.smithi102.stdout:8/55: chown d2/f4 74 1 2024-05-13T22:47:55.787 INFO:tasks.workunit.client.0.smithi102.stdout:0/122: write d1/d7/f18 [2720565,86149] 0 2024-05-13T22:47:55.788 INFO:tasks.workunit.client.0.smithi102.stdout:0/123: dread - d1/f1b zero size 2024-05-13T22:47:55.792 INFO:tasks.workunit.client.0.smithi102.stdout:1/91: dread d2/d10/f12 [0,4194304] 0 2024-05-13T22:47:55.793 INFO:tasks.workunit.client.0.smithi102.stdout:1/92: truncate d2/d10/f12 309874 0 2024-05-13T22:47:55.794 INFO:tasks.workunit.client.0.smithi102.stdout:4/121: creat d3/d4/f32 x:0 0 0 2024-05-13T22:47:55.797 INFO:tasks.workunit.client.0.smithi102.stdout:6/117: mkdir d9/d25 0 2024-05-13T22:47:55.804 INFO:tasks.workunit.client.0.smithi102.stdout:9/135: mknod d1/d6/db/d2c/c39 0 2024-05-13T22:47:55.805 INFO:tasks.workunit.client.0.smithi102.stdout:8/56: symlink d2/d7/lf 0 2024-05-13T22:47:55.814 INFO:tasks.workunit.client.0.smithi102.stdout:1/93: unlink d2/f13 0 2024-05-13T22:47:55.815 INFO:tasks.workunit.client.0.smithi102.stdout:4/122: mknod d3/d4/d6/c33 0 2024-05-13T22:47:55.819 INFO:tasks.workunit.client.0.smithi102.stdout:3/171: mkdir de/d10/d2a/d33/d3b 0 2024-05-13T22:47:55.820 INFO:tasks.workunit.client.0.smithi102.stdout:5/188: mkdir de/d25/d2a/d4b/d4d 0 2024-05-13T22:47:55.821 INFO:tasks.workunit.client.0.smithi102.stdout:6/118: dread f0 [4194304,4194304] 0 2024-05-13T22:47:55.823 INFO:tasks.workunit.client.0.smithi102.stdout:8/57: mkdir d2/d7/d10 0 2024-05-13T22:47:55.824 INFO:tasks.workunit.client.0.smithi102.stdout:8/58: write d2/d7/fb [1097056,8451] 0 2024-05-13T22:47:55.831 INFO:tasks.workunit.client.0.smithi102.stdout:8/59: dwrite d2/fa [0,4194304] 0 2024-05-13T22:47:55.836 INFO:tasks.workunit.client.0.smithi102.stdout:8/60: dread d2/d7/fb [0,4194304] 0 2024-05-13T22:47:55.893 INFO:tasks.workunit.client.0.smithi102.stdout:2/157: sync 2024-05-13T22:47:55.893 INFO:tasks.workunit.client.0.smithi102.stdout:7/83: sync 2024-05-13T22:47:55.945 INFO:tasks.workunit.client.0.smithi102.stdout:3/172: symlink de/d10/d2a/l3c 0 2024-05-13T22:47:55.950 INFO:tasks.workunit.client.0.smithi102.stdout:1/94: mknod d2/d10/c1d 0 2024-05-13T22:47:55.951 INFO:tasks.workunit.client.0.smithi102.stdout:9/136: truncate d1/d6/d21/f26 1222773 0 2024-05-13T22:47:55.954 INFO:tasks.workunit.client.0.smithi102.stdout:8/61: creat d2/f11 x:0 0 0 2024-05-13T22:47:55.954 INFO:tasks.workunit.client.0.smithi102.stdout:0/124: dwrite d1/d7/fb [0,4194304] 0 2024-05-13T22:47:55.957 INFO:tasks.workunit.client.0.smithi102.stdout:5/189: dread de/f11 [0,4194304] 0 2024-05-13T22:47:55.958 INFO:tasks.workunit.client.0.smithi102.stdout:7/84: dread d6/d13/de/f11 [0,4194304] 0 2024-05-13T22:47:55.959 INFO:tasks.workunit.client.0.smithi102.stdout:7/85: truncate f3 4997350 0 2024-05-13T22:47:55.962 INFO:tasks.workunit.client.0.smithi102.stdout:1/95: mkdir d2/d10/d1e 0 2024-05-13T22:47:55.966 INFO:tasks.workunit.client.0.smithi102.stdout:1/96: dread d2/ff [0,4194304] 0 2024-05-13T22:47:55.967 INFO:tasks.workunit.client.0.smithi102.stdout:8/62: chown d2/d7/ce 66101907 1 2024-05-13T22:47:55.968 INFO:tasks.workunit.client.0.smithi102.stdout:8/63: write d2/d7/f9 [882723,6335] 0 2024-05-13T22:47:55.969 INFO:tasks.workunit.client.0.smithi102.stdout:8/64: fdatasync d2/fa 0 2024-05-13T22:47:55.972 INFO:tasks.workunit.client.0.smithi102.stdout:5/190: mknod de/d25/c4e 0 2024-05-13T22:47:55.977 INFO:tasks.workunit.client.0.smithi102.stdout:8/65: dwrite d2/f11 [0,4194304] 0 2024-05-13T22:47:55.978 INFO:tasks.workunit.client.0.smithi102.stdout:2/158: creat d7/f2d x:0 0 0 2024-05-13T22:47:55.980 INFO:tasks.workunit.client.0.smithi102.stdout:9/137: link d1/f17 d1/f3a 0 2024-05-13T22:47:55.982 INFO:tasks.workunit.client.0.smithi102.stdout:1/97: creat d2/d10/f1f x:0 0 0 2024-05-13T22:47:55.983 INFO:tasks.workunit.client.0.smithi102.stdout:7/86: dwrite d6/d13/de/f11 [4194304,4194304] 0 2024-05-13T22:47:55.985 INFO:tasks.workunit.client.0.smithi102.stdout:6/119: dread d9/db/f22 [0,4194304] 0 2024-05-13T22:47:55.988 INFO:tasks.workunit.client.0.smithi102.stdout:7/87: dread d6/d13/de/f11 [4194304,4194304] 0 2024-05-13T22:47:55.997 INFO:tasks.workunit.client.0.smithi102.stdout:8/66: mkdir d2/d7/d12 0 2024-05-13T22:47:56.002 INFO:tasks.workunit.client.0.smithi102.stdout:8/67: dread d2/d7/f9 [0,4194304] 0 2024-05-13T22:47:56.002 INFO:tasks.workunit.client.0.smithi102.stdout:1/98: unlink d2/d10/f1f 0 2024-05-13T22:47:56.003 INFO:tasks.workunit.client.0.smithi102.stdout:8/68: chown d2/d7/f9 112 1 2024-05-13T22:47:56.007 INFO:tasks.workunit.client.0.smithi102.stdout:2/159: symlink d7/l2e 0 2024-05-13T22:47:56.022 INFO:tasks.workunit.client.0.smithi102.stdout:8/69: truncate d2/f4 10146773 0 2024-05-13T22:47:56.024 INFO:tasks.workunit.client.0.smithi102.stdout:6/120: getdents d9/db/d14 0 2024-05-13T22:47:56.025 INFO:tasks.workunit.client.0.smithi102.stdout:6/121: chown d9/db/f22 198 1 2024-05-13T22:47:56.033 INFO:tasks.workunit.client.0.smithi102.stdout:1/99: creat d2/d4/f20 x:0 0 0 2024-05-13T22:47:56.035 INFO:tasks.workunit.client.0.smithi102.stdout:8/70: creat d2/d7/d12/f13 x:0 0 0 2024-05-13T22:47:56.036 INFO:tasks.workunit.client.0.smithi102.stdout:6/122: mknod d9/db/c26 0 2024-05-13T22:47:56.038 INFO:tasks.workunit.client.0.smithi102.stdout:3/173: write f2 [671709,11203] 0 2024-05-13T22:47:56.044 INFO:tasks.workunit.client.0.smithi102.stdout:3/174: unlink de/c2d 0 2024-05-13T22:47:56.048 INFO:tasks.workunit.client.0.smithi102.stdout:8/71: rmdir d2/d7/d10 0 2024-05-13T22:47:56.050 INFO:tasks.workunit.client.0.smithi102.stdout:3/175: creat de/d10/d2a/d33/f3d x:0 0 0 2024-05-13T22:47:56.051 INFO:tasks.workunit.client.0.smithi102.stdout:3/176: truncate de/f37 1414849 0 2024-05-13T22:47:56.062 INFO:tasks.workunit.client.0.smithi102.stdout:9/138: rename d1/d6/d32 to d1/d6/d3b 0 2024-05-13T22:47:56.063 INFO:tasks.workunit.client.0.smithi102.stdout:0/125: truncate d1/f4 2259964 0 2024-05-13T22:47:56.066 INFO:tasks.workunit.client.0.smithi102.stdout:0/126: truncate d1/f1b 409277 0 2024-05-13T22:47:56.069 INFO:tasks.workunit.client.0.smithi102.stdout:9/139: truncate d1/f1e 1351726 0 2024-05-13T22:47:56.074 INFO:tasks.workunit.client.0.smithi102.stdout:0/127: dwrite d1/d7/f1a [0,4194304] 0 2024-05-13T22:47:56.079 INFO:tasks.workunit.client.0.smithi102.stdout:0/128: dread d1/d7/f13 [0,4194304] 0 2024-05-13T22:47:56.082 INFO:tasks.workunit.client.0.smithi102.stdout:0/129: unlink d1/d7/fd 0 2024-05-13T22:47:56.083 INFO:tasks.workunit.client.0.smithi102.stdout:0/130: chown d1/d7/f13 1003482286 1 2024-05-13T22:47:56.088 INFO:tasks.workunit.client.0.smithi102.stdout:8/72: truncate d2/d7/f9 321737 0 2024-05-13T22:47:56.091 INFO:tasks.workunit.client.0.smithi102.stdout:8/73: readlink d2/d7/l8 0 2024-05-13T22:47:56.095 INFO:tasks.workunit.client.0.smithi102.stdout:5/191: creat de/f4f x:0 0 0 2024-05-13T22:47:56.102 INFO:tasks.workunit.client.0.smithi102.stdout:1/100: fsync d2/d4/f20 0 2024-05-13T22:47:56.104 INFO:tasks.workunit.client.0.smithi102.stdout:2/160: dwrite d7/ff [0,4194304] 0 2024-05-13T22:47:56.107 INFO:tasks.workunit.client.0.smithi102.stdout:1/101: readlink d2/l7 0 2024-05-13T22:47:56.107 INFO:tasks.workunit.client.0.smithi102.stdout:1/102: chown d2/d4/d5 35369 1 2024-05-13T22:47:56.108 INFO:tasks.workunit.client.0.smithi102.stdout:1/103: chown d2/ff 115436 1 2024-05-13T22:47:56.111 INFO:tasks.workunit.client.0.smithi102.stdout:2/161: unlink d7/d9/d10/d1b/f18 0 2024-05-13T22:47:56.112 INFO:tasks.workunit.client.0.smithi102.stdout:2/162: dread - d7/d15/f1f zero size 2024-05-13T22:47:56.114 INFO:tasks.workunit.client.0.smithi102.stdout:6/123: dwrite f7 [0,4194304] 0 2024-05-13T22:47:56.119 INFO:tasks.workunit.client.0.smithi102.stdout:1/104: unlink d2/l3 0 2024-05-13T22:47:56.119 INFO:tasks.workunit.client.0.smithi102.stdout:3/177: truncate de/d10/f31 108688 0 2024-05-13T22:47:56.121 INFO:tasks.workunit.client.0.smithi102.stdout:2/163: mknod d7/d15/d1e/c2f 0 2024-05-13T22:47:56.122 INFO:tasks.workunit.client.0.smithi102.stdout:2/164: fsync d7/d9/f13 0 2024-05-13T22:47:56.127 INFO:tasks.workunit.client.0.smithi102.stdout:1/105: creat d2/d10/f21 x:0 0 0 2024-05-13T22:47:56.129 INFO:tasks.workunit.client.0.smithi102.stdout:1/106: chown d2/d4/d5/d19 2125544329 1 2024-05-13T22:47:56.130 INFO:tasks.workunit.client.0.smithi102.stdout:3/178: symlink de/d21/l3e 0 2024-05-13T22:47:56.130 INFO:tasks.workunit.client.0.smithi102.stdout:6/124: dwrite d9/db/f22 [0,4194304] 0 2024-05-13T22:47:56.135 INFO:tasks.workunit.client.0.smithi102.stdout:2/165: mkdir d7/d9/d10/d30 0 2024-05-13T22:47:56.139 INFO:tasks.workunit.client.0.smithi102.stdout:6/125: dwrite d9/db/d14/f23 [0,4194304] 0 2024-05-13T22:47:56.140 INFO:tasks.workunit.client.0.smithi102.stdout:6/126: write f3 [2488443,62249] 0 2024-05-13T22:47:56.141 INFO:tasks.workunit.client.0.smithi102.stdout:3/179: creat de/d10/d2a/d33/f3f x:0 0 0 2024-05-13T22:47:56.150 INFO:tasks.workunit.client.0.smithi102.stdout:3/180: dwrite de/f37 [0,4194304] 0 2024-05-13T22:47:56.151 INFO:tasks.workunit.client.0.smithi102.stdout:5/192: fsync de/f4f 0 2024-05-13T22:47:56.152 INFO:tasks.workunit.client.0.smithi102.stdout:9/140: dread d1/f1e [0,4194304] 0 2024-05-13T22:47:56.153 INFO:tasks.workunit.client.0.smithi102.stdout:8/74: write d2/d7/fb [1055911,84484] 0 2024-05-13T22:47:56.156 INFO:tasks.workunit.client.0.smithi102.stdout:9/141: dread d1/f3a [0,4194304] 0 2024-05-13T22:47:56.168 INFO:tasks.workunit.client.0.smithi102.stdout:1/107: rename d2/d10/l14 to d2/l22 0 2024-05-13T22:47:56.170 INFO:tasks.workunit.client.0.smithi102.stdout:3/181: creat de/d10/d2a/d33/f40 x:0 0 0 2024-05-13T22:47:56.177 INFO:tasks.workunit.client.0.smithi102.stdout:8/75: mknod d2/d7/c14 0 2024-05-13T22:47:56.180 INFO:tasks.workunit.client.0.smithi102.stdout:6/127: unlink f1 0 2024-05-13T22:47:56.184 INFO:tasks.workunit.client.0.smithi102.stdout:9/142: symlink d1/d6/d21/l3c 0 2024-05-13T22:47:56.189 INFO:tasks.workunit.client.0.smithi102.stdout:3/182: mknod de/d21/c41 0 2024-05-13T22:47:56.192 INFO:tasks.workunit.client.0.smithi102.stdout:5/193: mkdir de/d25/d2a/d4b/d4d/d50 0 2024-05-13T22:47:56.192 INFO:tasks.workunit.client.0.smithi102.stdout:3/183: dread f7 [0,4194304] 0 2024-05-13T22:47:56.194 INFO:tasks.workunit.client.0.smithi102.stdout:8/76: mknod d2/d7/c15 0 2024-05-13T22:47:56.195 INFO:tasks.workunit.client.0.smithi102.stdout:8/77: fsync d2/d7/d12/f13 0 2024-05-13T22:47:56.195 INFO:tasks.workunit.client.0.smithi102.stdout:8/78: fdatasync d2/d7/fb 0 2024-05-13T22:47:56.209 INFO:tasks.workunit.client.0.smithi102.stdout:8/79: dread d2/f4 [4194304,4194304] 0 2024-05-13T22:47:56.210 INFO:tasks.workunit.client.0.smithi102.stdout:6/128: rename d9/db/cc to d9/d25/c27 0 2024-05-13T22:47:56.217 INFO:tasks.workunit.client.0.smithi102.stdout:1/108: creat d2/d4/f23 x:0 0 0 2024-05-13T22:47:56.220 INFO:tasks.workunit.client.0.smithi102.stdout:2/166: dwrite d7/d9/d14/f1a [0,4194304] 0 2024-05-13T22:47:56.222 INFO:tasks.workunit.client.0.smithi102.stdout:2/167: write d7/d9/f13 [3800625,8651] 0 2024-05-13T22:47:56.223 INFO:tasks.workunit.client.0.smithi102.stdout:9/143: dwrite d1/f16 [0,4194304] 0 2024-05-13T22:47:56.225 INFO:tasks.workunit.client.0.smithi102.stdout:8/80: symlink d2/l16 0 2024-05-13T22:47:56.228 INFO:tasks.workunit.client.0.smithi102.stdout:2/168: dread d7/ff [0,4194304] 0 2024-05-13T22:47:56.231 INFO:tasks.workunit.client.0.smithi102.stdout:3/184: rename de/d1c/l35 to de/d10/l42 0 2024-05-13T22:47:56.238 INFO:tasks.workunit.client.0.smithi102.stdout:2/169: dwrite d7/ff [4194304,4194304] 0 2024-05-13T22:47:56.247 INFO:tasks.workunit.client.0.smithi102.stdout:1/109: rename d2/d4/d5/f17 to d2/d10/d1e/f24 0 2024-05-13T22:47:56.249 INFO:tasks.workunit.client.0.smithi102.stdout:1/110: dread d2/d10/d1e/f24 [0,4194304] 0 2024-05-13T22:47:56.249 INFO:tasks.workunit.client.0.smithi102.stdout:1/111: rename d2 to d2/d4/d5/d19/d25 22 2024-05-13T22:47:56.254 INFO:tasks.workunit.client.0.smithi102.stdout:3/185: dread fd [0,4194304] 0 2024-05-13T22:47:56.256 INFO:tasks.workunit.client.0.smithi102.stdout:2/170: write d7/d9/fb [176955,13134] 0 2024-05-13T22:47:56.257 INFO:tasks.workunit.client.0.smithi102.stdout:5/194: getdents de/d25/d2a/d4b/d4d 0 2024-05-13T22:47:56.258 INFO:tasks.workunit.client.0.smithi102.stdout:3/186: dread f7 [0,4194304] 0 2024-05-13T22:47:56.258 INFO:tasks.workunit.client.0.smithi102.stdout:3/187: readlink de/l13 0 2024-05-13T22:47:56.263 INFO:tasks.workunit.client.0.smithi102.stdout:8/81: truncate d2/fa 4061817 0 2024-05-13T22:47:56.272 INFO:tasks.workunit.client.0.smithi102.stdout:3/188: rename fd to de/d10/f43 0 2024-05-13T22:47:56.274 INFO:tasks.workunit.client.0.smithi102.stdout:8/82: mkdir d2/d17 0 2024-05-13T22:47:56.278 INFO:tasks.workunit.client.0.smithi102.stdout:6/129: getdents d9/db 0 2024-05-13T22:47:56.281 INFO:tasks.workunit.client.0.smithi102.stdout:8/83: dwrite d2/d7/fb [0,4194304] 0 2024-05-13T22:47:56.300 INFO:tasks.workunit.client.0.smithi102.stdout:3/189: getdents de/d10/d2a/d33 0 2024-05-13T22:47:56.302 INFO:tasks.workunit.client.0.smithi102.stdout:6/130: unlink d9/db/d14/l16 0 2024-05-13T22:47:56.307 INFO:tasks.workunit.client.0.smithi102.stdout:3/190: dwrite de/d10/d2a/f3a [0,4194304] 0 2024-05-13T22:47:56.308 INFO:tasks.workunit.client.0.smithi102.stdout:3/191: chown de/d10 28192 1 2024-05-13T22:47:56.309 INFO:tasks.workunit.client.0.smithi102.stdout:3/192: stat f7 0 2024-05-13T22:47:56.311 INFO:tasks.workunit.client.0.smithi102.stdout:1/112: dwrite d2/d4/d5/f15 [4194304,4194304] 0 2024-05-13T22:47:56.311 INFO:tasks.workunit.client.0.smithi102.stdout:3/193: fdatasync de/d10/d2a/d33/f3d 0 2024-05-13T22:47:56.320 INFO:tasks.workunit.client.0.smithi102.stdout:1/113: write d2/d4/d5/fa [1387325,119259] 0 2024-05-13T22:47:56.320 INFO:tasks.workunit.client.0.smithi102.stdout:1/114: dread - d2/d4/fd zero size 2024-05-13T22:47:56.322 INFO:tasks.workunit.client.0.smithi102.stdout:6/131: mkdir d9/d25/d28 0 2024-05-13T22:47:56.324 INFO:tasks.workunit.client.0.smithi102.stdout:3/194: rename de/d10/d2a/d33/f3d to de/d10/f44 0 2024-05-13T22:47:56.325 INFO:tasks.workunit.client.0.smithi102.stdout:3/195: chown l3 1955907 1 2024-05-13T22:47:56.327 INFO:tasks.workunit.client.0.smithi102.stdout:6/132: creat d9/db/f29 x:0 0 0 2024-05-13T22:47:56.329 INFO:tasks.workunit.client.0.smithi102.stdout:8/84: dwrite d2/f4 [0,4194304] 0 2024-05-13T22:47:56.331 INFO:tasks.workunit.client.0.smithi102.stdout:1/115: mkdir d2/d4/d26 0 2024-05-13T22:47:56.335 INFO:tasks.workunit.client.0.smithi102.stdout:3/196: dwrite de/d21/f2b [0,4194304] 0 2024-05-13T22:47:56.338 INFO:tasks.workunit.client.0.smithi102.stdout:8/85: mknod d2/d7/c18 0 2024-05-13T22:47:56.345 INFO:tasks.workunit.client.0.smithi102.stdout:3/197: unlink de/d10/f27 0 2024-05-13T22:47:56.347 INFO:tasks.workunit.client.0.smithi102.stdout:8/86: creat d2/f19 x:0 0 0 2024-05-13T22:47:56.351 INFO:tasks.workunit.client.0.smithi102.stdout:3/198: dwrite de/d10/f1d [0,4194304] 0 2024-05-13T22:47:56.351 INFO:tasks.workunit.client.0.smithi102.stdout:3/199: readlink de/l1a 0 2024-05-13T22:47:56.352 INFO:tasks.workunit.client.0.smithi102.stdout:3/200: write fa [3843266,29029] 0 2024-05-13T22:47:56.353 INFO:tasks.workunit.client.0.smithi102.stdout:8/87: rename d2/d7/lf to d2/d17/l1a 0 2024-05-13T22:47:56.355 INFO:tasks.workunit.client.0.smithi102.stdout:1/116: link d2/l7 d2/d4/d5/l27 0 2024-05-13T22:47:56.356 INFO:tasks.workunit.client.0.smithi102.stdout:1/117: chown d2 222338 1 2024-05-13T22:47:56.359 INFO:tasks.workunit.client.0.smithi102.stdout:3/201: dwrite de/d21/f2b [0,4194304] 0 2024-05-13T22:47:56.366 INFO:tasks.workunit.client.0.smithi102.stdout:1/118: rename d2/d10/d1e to d2/d4/d5/d19/d28 0 2024-05-13T22:47:56.367 INFO:tasks.workunit.client.0.smithi102.stdout:1/119: write d2/d4/d5/d19/d28/f24 [1519908,33290] 0 2024-05-13T22:47:56.368 INFO:tasks.workunit.client.0.smithi102.stdout:6/133: chown d9/db/l1c 257890031 1 2024-05-13T22:47:56.369 INFO:tasks.workunit.client.0.smithi102.stdout:1/120: write d2/d4/f23 [274521,28966] 0 2024-05-13T22:47:56.371 INFO:tasks.workunit.client.0.smithi102.stdout:8/88: truncate d2/d7/f9 859904 0 2024-05-13T22:47:56.377 INFO:tasks.workunit.client.0.smithi102.stdout:3/202: creat de/d10/d2a/d33/d3b/f45 x:0 0 0 2024-05-13T22:47:56.381 INFO:tasks.workunit.client.0.smithi102.stdout:8/89: rmdir d2/d17 39 2024-05-13T22:47:56.383 INFO:tasks.workunit.client.0.smithi102.stdout:3/203: mkdir de/d10/d46 0 2024-05-13T22:47:56.384 INFO:tasks.workunit.client.0.smithi102.stdout:6/134: creat d9/d25/d28/f2a x:0 0 0 2024-05-13T22:47:56.386 INFO:tasks.workunit.client.0.smithi102.stdout:1/121: link d2/d10/f1a d2/d4/d26/f29 0 2024-05-13T22:47:56.388 INFO:tasks.workunit.client.0.smithi102.stdout:3/204: creat de/d10/d2a/d33/d3b/f47 x:0 0 0 2024-05-13T22:47:56.390 INFO:tasks.workunit.client.0.smithi102.stdout:1/122: rmdir d2/d4/d5 39 2024-05-13T22:47:56.395 INFO:tasks.workunit.client.0.smithi102.stdout:3/205: mkdir de/d10/d2a/d48 0 2024-05-13T22:47:56.395 INFO:tasks.workunit.client.0.smithi102.stdout:6/135: dwrite d9/db/d14/f19 [0,4194304] 0 2024-05-13T22:47:56.396 INFO:tasks.workunit.client.0.smithi102.stdout:6/136: write f5 [2447532,53324] 0 2024-05-13T22:47:56.398 INFO:tasks.workunit.client.0.smithi102.stdout:1/123: read d2/d4/d5/f15 [6161899,68142] 0 2024-05-13T22:47:56.411 INFO:tasks.workunit.client.0.smithi102.stdout:3/206: unlink de/d10/f31 0 2024-05-13T22:47:56.411 INFO:tasks.workunit.client.0.smithi102.stdout:3/207: stat de/d10/d34 0 2024-05-13T22:47:56.412 INFO:tasks.workunit.client.0.smithi102.stdout:3/208: dread - de/d10/d2a/d33/f3f zero size 2024-05-13T22:47:56.412 INFO:tasks.workunit.client.0.smithi102.stdout:3/209: fsync de/d10/d2a/f2f 0 2024-05-13T22:47:56.417 INFO:tasks.workunit.client.0.smithi102.stdout:3/210: dwrite de/d10/d2a/f3a [0,4194304] 0 2024-05-13T22:47:56.429 INFO:tasks.workunit.client.0.smithi102.stdout:4/123: sync 2024-05-13T22:47:56.431 INFO:tasks.workunit.client.0.smithi102.stdout:1/124: mknod d2/d4/d26/c2a 0 2024-05-13T22:47:56.437 INFO:tasks.workunit.client.0.smithi102.stdout:1/125: dwrite d2/d4/d5/fa [0,4194304] 0 2024-05-13T22:47:56.438 INFO:tasks.workunit.client.0.smithi102.stdout:1/126: chown d2/d4/d5/d19/d28 13 1 2024-05-13T22:47:56.439 INFO:tasks.workunit.client.0.smithi102.stdout:4/124: unlink d3/d22/d2b/f17 0 2024-05-13T22:47:56.444 INFO:tasks.workunit.client.0.smithi102.stdout:1/127: rmdir d2/d4/d5/d19/d28 39 2024-05-13T22:47:56.446 INFO:tasks.workunit.client.0.smithi102.stdout:4/125: creat d3/d4/d6/de/f34 x:0 0 0 2024-05-13T22:47:56.451 INFO:tasks.workunit.client.0.smithi102.stdout:8/90: dwrite d2/d7/f9 [0,4194304] 0 2024-05-13T22:47:56.456 INFO:tasks.workunit.client.0.smithi102.stdout:6/137: dwrite f8 [4194304,4194304] 0 2024-05-13T22:47:56.462 INFO:tasks.workunit.client.0.smithi102.stdout:4/126: creat d3/f35 x:0 0 0 2024-05-13T22:47:56.469 INFO:tasks.workunit.client.0.smithi102.stdout:8/91: symlink d2/l1b 0 2024-05-13T22:47:56.472 INFO:tasks.workunit.client.0.smithi102.stdout:8/92: creat d2/d7/d12/f1c x:0 0 0 2024-05-13T22:47:56.475 INFO:tasks.workunit.client.0.smithi102.stdout:8/93: creat d2/d7/f1d x:0 0 0 2024-05-13T22:47:56.476 INFO:tasks.workunit.client.0.smithi102.stdout:4/127: dwrite d3/d4/d6/de/d1f/f28 [0,4194304] 0 2024-05-13T22:47:56.480 INFO:tasks.workunit.client.0.smithi102.stdout:4/128: creat d3/d22/f36 x:0 0 0 2024-05-13T22:47:56.480 INFO:tasks.workunit.client.0.smithi102.stdout:4/129: chown d3 0 1 2024-05-13T22:47:56.481 INFO:tasks.workunit.client.0.smithi102.stdout:4/130: fdatasync d3/f10 0 2024-05-13T22:47:56.482 INFO:tasks.workunit.client.0.smithi102.stdout:4/131: write d3/d4/d6/de/d1f/f28 [2835527,435] 0 2024-05-13T22:47:56.487 INFO:tasks.workunit.client.0.smithi102.stdout:6/138: dread f5 [0,4194304] 0 2024-05-13T22:47:56.488 INFO:tasks.workunit.client.0.smithi102.stdout:4/132: rename d3/d4/ff to d3/d19/d23/f37 0 2024-05-13T22:47:56.492 INFO:tasks.workunit.client.0.smithi102.stdout:6/139: dread - d9/f21 zero size 2024-05-13T22:47:56.493 INFO:tasks.workunit.client.0.smithi102.stdout:6/140: dread - d9/db/f29 zero size 2024-05-13T22:47:56.494 INFO:tasks.workunit.client.0.smithi102.stdout:4/133: write d3/d19/d23/f37 [3312965,36103] 0 2024-05-13T22:47:56.498 INFO:tasks.workunit.client.0.smithi102.stdout:4/134: mknod d3/d4/d6/de/c38 0 2024-05-13T22:47:56.504 INFO:tasks.workunit.client.0.smithi102.stdout:4/135: dwrite d3/d4/f2e [0,4194304] 0 2024-05-13T22:47:56.515 INFO:tasks.workunit.client.0.smithi102.stdout:6/141: dread d9/fa [0,4194304] 0 2024-05-13T22:47:56.520 INFO:tasks.workunit.client.0.smithi102.stdout:6/142: getdents d9/d25/d28 0 2024-05-13T22:47:56.523 INFO:tasks.workunit.client.0.smithi102.stdout:6/143: write d9/ff [129676,59555] 0 2024-05-13T22:47:56.527 INFO:tasks.workunit.client.0.smithi102.stdout:6/144: mkdir d9/db/d2b 0 2024-05-13T22:47:56.530 INFO:tasks.workunit.client.0.smithi102.stdout:6/145: creat d9/db/d14/f2c x:0 0 0 2024-05-13T22:47:56.531 INFO:tasks.workunit.client.0.smithi102.stdout:6/146: write d9/db/f29 [837888,89165] 0 2024-05-13T22:47:56.533 INFO:tasks.workunit.client.0.smithi102.stdout:4/136: rmdir d3/d4/d6/de 39 2024-05-13T22:47:56.535 INFO:tasks.workunit.client.0.smithi102.stdout:6/147: rename d9/db/d14/l18 to d9/d25/l2d 0 2024-05-13T22:47:56.536 INFO:tasks.workunit.client.0.smithi102.stdout:4/137: mkdir d3/d19/d23/d39 0 2024-05-13T22:47:56.537 INFO:tasks.workunit.client.0.smithi102.stdout:4/138: write d3/d22/f36 [994053,41943] 0 2024-05-13T22:47:56.541 INFO:tasks.workunit.client.0.smithi102.stdout:4/139: write d3/d4/d6/de/d1f/f28 [1991240,117323] 0 2024-05-13T22:47:56.541 INFO:tasks.workunit.client.0.smithi102.stdout:6/148: creat d9/db/d2b/f2e x:0 0 0 2024-05-13T22:47:56.543 INFO:tasks.workunit.client.0.smithi102.stdout:6/149: fsync d9/db/f22 0 2024-05-13T22:47:56.544 INFO:tasks.workunit.client.0.smithi102.stdout:4/140: mkdir d3/d19/d23/d3a 0 2024-05-13T22:47:56.545 INFO:tasks.workunit.client.0.smithi102.stdout:4/141: fsync d3/d19/d23/f37 0 2024-05-13T22:47:56.546 INFO:tasks.workunit.client.0.smithi102.stdout:6/150: chown d9/d25/c27 7886954 1 2024-05-13T22:47:56.547 INFO:tasks.workunit.client.0.smithi102.stdout:4/142: symlink d3/d4/l3b 0 2024-05-13T22:47:56.548 INFO:tasks.workunit.client.0.smithi102.stdout:4/143: fsync d3/f10 0 2024-05-13T22:47:56.548 INFO:tasks.workunit.client.0.smithi102.stdout:4/144: fsync d3/d4/fd 0 2024-05-13T22:47:56.549 INFO:tasks.workunit.client.0.smithi102.stdout:6/151: mkdir d9/db/d2f 0 2024-05-13T22:47:56.550 INFO:tasks.workunit.client.0.smithi102.stdout:4/145: symlink d3/d19/l3c 0 2024-05-13T22:47:56.552 INFO:tasks.workunit.client.0.smithi102.stdout:6/152: rename d9/db/d2b/f2e to d9/d25/f30 0 2024-05-13T22:47:56.554 INFO:tasks.workunit.client.0.smithi102.stdout:6/153: read d9/db/d14/f1d [329121,5959] 0 2024-05-13T22:47:56.556 INFO:tasks.workunit.client.0.smithi102.stdout:4/146: getdents d3 0 2024-05-13T22:47:56.559 INFO:tasks.workunit.client.0.smithi102.stdout:4/147: creat d3/d19/d23/f3d x:0 0 0 2024-05-13T22:47:56.560 INFO:tasks.workunit.client.0.smithi102.stdout:6/154: rename d9/db/l1c to d9/db/d2b/l31 0 2024-05-13T22:47:56.562 INFO:tasks.workunit.client.0.smithi102.stdout:4/148: creat d3/d22/f3e x:0 0 0 2024-05-13T22:47:56.563 INFO:tasks.workunit.client.0.smithi102.stdout:4/149: write d3/d22/f3e [687357,6224] 0 2024-05-13T22:47:56.565 INFO:tasks.workunit.client.0.smithi102.stdout:6/155: symlink d9/db/d14/l32 0 2024-05-13T22:47:56.566 INFO:tasks.workunit.client.0.smithi102.stdout:4/150: creat d3/d4/d6/de/f3f x:0 0 0 2024-05-13T22:47:56.571 INFO:tasks.workunit.client.0.smithi102.stdout:6/156: mknod d9/c33 0 2024-05-13T22:47:56.571 INFO:tasks.workunit.client.0.smithi102.stdout:6/157: fdatasync f3 0 2024-05-13T22:47:56.571 INFO:tasks.workunit.client.0.smithi102.stdout:4/151: dread d3/d22/f36 [0,4194304] 0 2024-05-13T22:47:56.573 INFO:tasks.workunit.client.0.smithi102.stdout:6/158: truncate d9/d25/d28/f2a 844106 0 2024-05-13T22:47:56.574 INFO:tasks.workunit.client.0.smithi102.stdout:4/152: symlink d3/d19/l40 0 2024-05-13T22:47:56.575 INFO:tasks.workunit.client.0.smithi102.stdout:6/159: mknod d9/db/d14/c34 0 2024-05-13T22:47:56.578 INFO:tasks.workunit.client.0.smithi102.stdout:4/153: mkdir d3/d4/d6/de/d1f/d41 0 2024-05-13T22:47:56.579 INFO:tasks.workunit.client.0.smithi102.stdout:4/154: fdatasync d3/d4/d6/de/f34 0 2024-05-13T22:47:56.581 INFO:tasks.workunit.client.0.smithi102.stdout:6/160: unlink d9/l17 0 2024-05-13T22:47:56.585 INFO:tasks.workunit.client.0.smithi102.stdout:6/161: mkdir d9/db/d14/d35 0 2024-05-13T22:47:56.588 INFO:tasks.workunit.client.0.smithi102.stdout:4/155: creat d3/d4/d6/f42 x:0 0 0 2024-05-13T22:47:56.590 INFO:tasks.workunit.client.0.smithi102.stdout:4/156: chown d3/d4/d6/f42 1 1 2024-05-13T22:47:56.592 INFO:tasks.workunit.client.0.smithi102.stdout:6/162: dwrite d9/f21 [0,4194304] 0 2024-05-13T22:47:56.597 INFO:tasks.workunit.client.0.smithi102.stdout:4/157: dwrite d3/f10 [0,4194304] 0 2024-05-13T22:47:56.600 INFO:tasks.workunit.client.0.smithi102.stdout:6/163: creat d9/db/f36 x:0 0 0 2024-05-13T22:47:56.601 INFO:tasks.workunit.client.0.smithi102.stdout:4/158: chown d3/c12 82 1 2024-05-13T22:47:56.605 INFO:tasks.workunit.client.0.smithi102.stdout:4/159: fsync d3/d4/d6/de/f15 0 2024-05-13T22:47:56.614 INFO:tasks.workunit.client.0.smithi102.stdout:4/160: dwrite d3/d19/d23/f37 [0,4194304] 0 2024-05-13T22:47:56.622 INFO:tasks.workunit.client.0.smithi102.stdout:4/161: link d3/d22/d2b/f2c d3/d19/d23/f43 0 2024-05-13T22:47:56.625 INFO:tasks.workunit.client.0.smithi102.stdout:6/164: getdents d9/db 0 2024-05-13T22:47:56.626 INFO:tasks.workunit.client.0.smithi102.stdout:4/162: mknod d3/d4/d6/c44 0 2024-05-13T22:47:56.627 INFO:tasks.workunit.client.0.smithi102.stdout:4/163: write d3/d4/d6/f42 [760603,79150] 0 2024-05-13T22:47:56.628 INFO:tasks.workunit.client.0.smithi102.stdout:4/164: read - d3/d4/f31 zero size 2024-05-13T22:47:56.628 INFO:tasks.workunit.client.0.smithi102.stdout:6/165: mkdir d9/db/d14/d37 0 2024-05-13T22:47:56.635 INFO:tasks.workunit.client.0.smithi102.stdout:6/166: dread d9/db/d14/f23 [0,4194304] 0 2024-05-13T22:47:56.635 INFO:tasks.workunit.client.0.smithi102.stdout:4/165: dwrite d3/d4/f1e [0,4194304] 0 2024-05-13T22:47:56.635 INFO:tasks.workunit.client.0.smithi102.stdout:6/167: readlink d9/db/l1f 0 2024-05-13T22:47:56.636 INFO:tasks.workunit.client.0.smithi102.stdout:4/166: stat d3/d19/l3c 0 2024-05-13T22:47:56.637 INFO:tasks.workunit.client.0.smithi102.stdout:0/131: sync 2024-05-13T22:47:56.638 INFO:tasks.workunit.client.0.smithi102.stdout:7/88: sync 2024-05-13T22:47:56.638 INFO:tasks.workunit.client.0.smithi102.stdout:6/168: write f5 [185567,40675] 0 2024-05-13T22:47:56.639 INFO:tasks.workunit.client.0.smithi102.stdout:4/167: truncate d3/d4/d6/de/f34 1034493 0 2024-05-13T22:47:56.640 INFO:tasks.workunit.client.0.smithi102.stdout:0/132: read d1/f5 [1708061,41392] 0 2024-05-13T22:47:56.644 INFO:tasks.workunit.client.0.smithi102.stdout:4/168: dread d3/d22/f3e [0,4194304] 0 2024-05-13T22:47:56.651 INFO:tasks.workunit.client.0.smithi102.stdout:4/169: dwrite d3/d4/f1e [0,4194304] 0 2024-05-13T22:47:56.653 INFO:tasks.workunit.client.0.smithi102.stdout:4/170: truncate d3/d4/fd 873301 0 2024-05-13T22:47:56.654 INFO:tasks.workunit.client.0.smithi102.stdout:6/169: rename d9/c20 to d9/db/c38 0 2024-05-13T22:47:56.656 INFO:tasks.workunit.client.0.smithi102.stdout:0/133: truncate d1/d7/f13 878415 0 2024-05-13T22:47:56.657 INFO:tasks.workunit.client.0.smithi102.stdout:7/89: creat d6/d13/f16 x:0 0 0 2024-05-13T22:47:56.658 INFO:tasks.workunit.client.0.smithi102.stdout:4/171: rmdir d3/d19 39 2024-05-13T22:47:56.662 INFO:tasks.workunit.client.0.smithi102.stdout:6/170: mknod d9/db/d14/c39 0 2024-05-13T22:47:56.663 INFO:tasks.workunit.client.0.smithi102.stdout:6/171: stat d9/f21 0 2024-05-13T22:47:56.665 INFO:tasks.workunit.client.0.smithi102.stdout:0/134: rename d1/d7/c1f to d1/de/c20 0 2024-05-13T22:47:56.666 INFO:tasks.workunit.client.0.smithi102.stdout:7/90: dwrite d6/d13/f16 [0,4194304] 0 2024-05-13T22:47:56.666 INFO:tasks.workunit.client.0.smithi102.stdout:4/172: dread d3/d4/f2e [0,4194304] 0 2024-05-13T22:47:56.669 INFO:tasks.workunit.client.0.smithi102.stdout:4/173: chown d3/d4/d6/de/f34 13 1 2024-05-13T22:47:56.669 INFO:tasks.workunit.client.0.smithi102.stdout:0/135: dread d1/d7/f13 [0,4194304] 0 2024-05-13T22:47:56.669 INFO:tasks.workunit.client.0.smithi102.stdout:7/91: write d6/d13/de/f12 [856503,116992] 0 2024-05-13T22:47:56.670 INFO:tasks.workunit.client.0.smithi102.stdout:0/136: chown d1/de/f19 40651845 1 2024-05-13T22:47:56.672 INFO:tasks.workunit.client.0.smithi102.stdout:6/172: mknod d9/db/d2b/c3a 0 2024-05-13T22:47:56.672 INFO:tasks.workunit.client.0.smithi102.stdout:6/173: truncate d9/f21 4500591 0 2024-05-13T22:47:56.674 INFO:tasks.workunit.client.0.smithi102.stdout:4/174: chown d3/d4/d6/f18 79 1 2024-05-13T22:47:56.674 INFO:tasks.workunit.client.0.smithi102.stdout:7/92: read d6/f8 [1756632,111936] 0 2024-05-13T22:47:56.675 INFO:tasks.workunit.client.0.smithi102.stdout:7/93: write d6/d13/de/f12 [7410,126598] 0 2024-05-13T22:47:56.677 INFO:tasks.workunit.client.0.smithi102.stdout:0/137: read d1/de/f1c [140239,95568] 0 2024-05-13T22:47:56.678 INFO:tasks.workunit.client.0.smithi102.stdout:6/174: mknod d9/d25/c3b 0 2024-05-13T22:47:56.679 INFO:tasks.workunit.client.0.smithi102.stdout:6/175: write f3 [714717,15388] 0 2024-05-13T22:47:56.680 INFO:tasks.workunit.client.0.smithi102.stdout:6/176: write d9/db/f29 [1741902,99239] 0 2024-05-13T22:47:56.682 INFO:tasks.workunit.client.0.smithi102.stdout:7/94: symlink d6/d13/de/l17 0 2024-05-13T22:47:56.685 INFO:tasks.workunit.client.0.smithi102.stdout:6/177: chown d9/db/d2b/l31 547870 1 2024-05-13T22:47:56.688 INFO:tasks.workunit.client.0.smithi102.stdout:7/95: rename d6/d13/de/f11 to d6/d15/f18 0 2024-05-13T22:47:56.707 INFO:tasks.workunit.client.0.smithi102.stdout:0/138: dwrite d1/d7/f17 [0,4194304] 0 2024-05-13T22:47:56.707 INFO:tasks.workunit.client.0.smithi102.stdout:7/96: dread d6/d13/de/fb [0,4194304] 0 2024-05-13T22:47:56.713 INFO:tasks.workunit.client.0.smithi102.stdout:6/178: dread d9/f1a [0,4194304] 0 2024-05-13T22:47:56.714 INFO:tasks.workunit.client.0.smithi102.stdout:7/97: getdents d6/d15 0 2024-05-13T22:47:56.716 INFO:tasks.workunit.client.0.smithi102.stdout:7/98: mkdir d6/d13/de/d19 0 2024-05-13T22:47:56.718 INFO:tasks.workunit.client.0.smithi102.stdout:6/179: creat d9/db/f3c x:0 0 0 2024-05-13T22:47:56.721 INFO:tasks.workunit.client.0.smithi102.stdout:6/180: read f3 [329544,130024] 0 2024-05-13T22:47:56.725 INFO:tasks.workunit.client.0.smithi102.stdout:6/181: rename d9/db/d14/l32 to d9/db/d2f/l3d 0 2024-05-13T22:47:56.726 INFO:tasks.workunit.client.0.smithi102.stdout:6/182: write d9/db/f29 [803526,75538] 0 2024-05-13T22:47:56.730 INFO:tasks.workunit.client.0.smithi102.stdout:6/183: symlink d9/db/d2b/l3e 0 2024-05-13T22:47:56.731 INFO:tasks.workunit.client.0.smithi102.stdout:7/99: getdents d6/d13/de 0 2024-05-13T22:47:56.731 INFO:tasks.workunit.client.0.smithi102.stdout:6/184: chown d9/ff 63282556 1 2024-05-13T22:47:56.733 INFO:tasks.workunit.client.0.smithi102.stdout:7/100: rmdir d6/d15 39 2024-05-13T22:47:56.735 INFO:tasks.workunit.client.0.smithi102.stdout:0/139: dread d1/d7/f18 [0,4194304] 0 2024-05-13T22:47:56.735 INFO:tasks.workunit.client.0.smithi102.stdout:6/185: rename f8 to d9/db/d14/f3f 0 2024-05-13T22:47:56.736 INFO:tasks.workunit.client.0.smithi102.stdout:6/186: read f7 [211507,15123] 0 2024-05-13T22:47:56.741 INFO:tasks.workunit.client.0.smithi102.stdout:6/187: unlink f5 0 2024-05-13T22:47:56.743 INFO:tasks.workunit.client.0.smithi102.stdout:7/101: dwrite d6/fc [4194304,4194304] 0 2024-05-13T22:47:56.749 INFO:tasks.workunit.client.0.smithi102.stdout:0/140: rename d1/cc to d1/c21 0 2024-05-13T22:47:56.749 INFO:tasks.workunit.client.0.smithi102.stdout:6/188: getdents d9/db/d14/d35 0 2024-05-13T22:47:56.751 INFO:tasks.workunit.client.0.smithi102.stdout:0/141: mknod d1/de/c22 0 2024-05-13T22:47:56.754 INFO:tasks.workunit.client.0.smithi102.stdout:0/142: symlink d1/de/l23 0 2024-05-13T22:47:56.756 INFO:tasks.workunit.client.0.smithi102.stdout:6/189: dread f7 [0,4194304] 0 2024-05-13T22:47:56.757 INFO:tasks.workunit.client.0.smithi102.stdout:0/143: write d1/d7/f13 [288579,36329] 0 2024-05-13T22:47:56.758 INFO:tasks.workunit.client.0.smithi102.stdout:6/190: chown d9/db/d14/f3f 1330 1 2024-05-13T22:47:56.760 INFO:tasks.workunit.client.0.smithi102.stdout:7/102: dread d6/f8 [0,4194304] 0 2024-05-13T22:47:56.761 INFO:tasks.workunit.client.0.smithi102.stdout:0/144: mkdir d1/d7/d24 0 2024-05-13T22:47:56.763 INFO:tasks.workunit.client.0.smithi102.stdout:7/103: symlink d6/l1a 0 2024-05-13T22:47:56.764 INFO:tasks.workunit.client.0.smithi102.stdout:0/145: fsync d1/d7/f13 0 2024-05-13T22:47:56.765 INFO:tasks.workunit.client.0.smithi102.stdout:6/191: dwrite d9/db/d14/f19 [0,4194304] 0 2024-05-13T22:47:56.768 INFO:tasks.workunit.client.0.smithi102.stdout:7/104: creat d6/d15/f1b x:0 0 0 2024-05-13T22:47:56.772 INFO:tasks.workunit.client.0.smithi102.stdout:7/105: symlink d6/l1c 0 2024-05-13T22:47:56.774 INFO:tasks.workunit.client.0.smithi102.stdout:0/146: dwrite d1/d7/f13 [0,4194304] 0 2024-05-13T22:47:56.774 INFO:tasks.workunit.client.0.smithi102.stdout:7/106: rename d6 to d6/d1d 22 2024-05-13T22:47:56.775 INFO:tasks.workunit.client.0.smithi102.stdout:6/192: dwrite d9/db/f29 [0,4194304] 0 2024-05-13T22:47:56.778 INFO:tasks.workunit.client.0.smithi102.stdout:7/107: rmdir d6 39 2024-05-13T22:47:56.782 INFO:tasks.workunit.client.0.smithi102.stdout:6/193: unlink d9/db/d2b/c3a 0 2024-05-13T22:47:56.784 INFO:tasks.workunit.client.0.smithi102.stdout:0/147: dwrite d1/de/f19 [0,4194304] 0 2024-05-13T22:47:56.785 INFO:tasks.workunit.client.0.smithi102.stdout:7/108: creat d6/d15/f1e x:0 0 0 2024-05-13T22:47:56.787 INFO:tasks.workunit.client.0.smithi102.stdout:6/194: write d9/f10 [3841454,72842] 0 2024-05-13T22:47:56.789 INFO:tasks.workunit.client.0.smithi102.stdout:6/195: chown d9/d25/d28/f2a 1 1 2024-05-13T22:47:56.791 INFO:tasks.workunit.client.0.smithi102.stdout:7/109: dread d6/fc [4194304,4194304] 0 2024-05-13T22:47:56.798 INFO:tasks.workunit.client.0.smithi102.stdout:7/110: dwrite d6/f7 [0,4194304] 0 2024-05-13T22:47:56.804 INFO:tasks.workunit.client.0.smithi102.stdout:6/196: creat d9/d25/f40 x:0 0 0 2024-05-13T22:47:56.805 INFO:tasks.workunit.client.0.smithi102.stdout:7/111: dread d6/d13/de/f12 [0,4194304] 0 2024-05-13T22:47:56.810 INFO:tasks.workunit.client.0.smithi102.stdout:6/197: creat d9/db/d2f/f41 x:0 0 0 2024-05-13T22:47:56.814 INFO:tasks.workunit.client.0.smithi102.stdout:7/112: dwrite d6/d13/de/f12 [0,4194304] 0 2024-05-13T22:47:56.815 INFO:tasks.workunit.client.0.smithi102.stdout:6/198: mknod d9/db/d14/d35/c42 0 2024-05-13T22:47:56.816 INFO:tasks.workunit.client.0.smithi102.stdout:7/113: read d6/d15/f18 [6612675,26600] 0 2024-05-13T22:47:56.817 INFO:tasks.workunit.client.0.smithi102.stdout:7/114: chown d6/d13/de/fb 416444 1 2024-05-13T22:47:56.817 INFO:tasks.workunit.client.0.smithi102.stdout:7/115: stat d6/d13/c10 0 2024-05-13T22:47:56.825 INFO:tasks.workunit.client.0.smithi102.stdout:7/116: dwrite d6/d15/f1e [0,4194304] 0 2024-05-13T22:47:56.826 INFO:tasks.workunit.client.0.smithi102.stdout:0/148: write d1/f5 [1119840,18841] 0 2024-05-13T22:47:56.829 INFO:tasks.workunit.client.0.smithi102.stdout:7/117: symlink d6/l1f 0 2024-05-13T22:47:56.832 INFO:tasks.workunit.client.0.smithi102.stdout:0/149: dwrite d1/f5 [0,4194304] 0 2024-05-13T22:47:56.833 INFO:tasks.workunit.client.0.smithi102.stdout:0/150: truncate d1/f5 4578807 0 2024-05-13T22:47:56.834 INFO:tasks.workunit.client.0.smithi102.stdout:6/199: write d9/db/d14/f1d [208251,86095] 0 2024-05-13T22:47:56.836 INFO:tasks.workunit.client.0.smithi102.stdout:0/151: mknod d1/de/c25 0 2024-05-13T22:47:56.838 INFO:tasks.workunit.client.0.smithi102.stdout:6/200: creat d9/db/d14/d37/f43 x:0 0 0 2024-05-13T22:47:56.838 INFO:tasks.workunit.client.0.smithi102.stdout:6/201: fdatasync d9/db/d14/f2c 0 2024-05-13T22:47:56.839 INFO:tasks.workunit.client.0.smithi102.stdout:0/152: mknod d1/de/c26 0 2024-05-13T22:47:56.840 INFO:tasks.workunit.client.0.smithi102.stdout:6/202: write f3 [2710045,36575] 0 2024-05-13T22:47:56.844 INFO:tasks.workunit.client.0.smithi102.stdout:6/203: dread d9/db/fe [0,4194304] 0 2024-05-13T22:47:56.853 INFO:tasks.workunit.client.0.smithi102.stdout:6/204: dwrite d9/db/f36 [0,4194304] 0 2024-05-13T22:47:56.862 INFO:tasks.workunit.client.0.smithi102.stdout:6/205: dwrite d9/db/d14/f1d [0,4194304] 0 2024-05-13T22:47:56.865 INFO:tasks.workunit.client.0.smithi102.stdout:0/153: dwrite d1/f1b [0,4194304] 0 2024-05-13T22:47:56.867 INFO:tasks.workunit.client.0.smithi102.stdout:6/206: mknod d9/db/c44 0 2024-05-13T22:47:56.869 INFO:tasks.workunit.client.0.smithi102.stdout:6/207: write d9/db/d14/d37/f43 [853252,31354] 0 2024-05-13T22:47:56.870 INFO:tasks.workunit.client.0.smithi102.stdout:0/154: creat d1/de/f27 x:0 0 0 2024-05-13T22:47:56.870 INFO:tasks.workunit.client.0.smithi102.stdout:0/155: stat d1 0 2024-05-13T22:47:56.882 INFO:tasks.workunit.client.0.smithi102.stdout:6/208: dwrite d9/db/d14/f3f [4194304,4194304] 0 2024-05-13T22:47:56.882 INFO:tasks.workunit.client.0.smithi102.stdout:0/156: dwrite d1/de/f1d [0,4194304] 0 2024-05-13T22:47:56.883 INFO:tasks.workunit.client.0.smithi102.stdout:6/209: chown d9/c33 2029946 1 2024-05-13T22:47:56.885 INFO:tasks.workunit.client.0.smithi102.stdout:6/210: fsync d9/db/f3c 0 2024-05-13T22:47:56.889 INFO:tasks.workunit.client.0.smithi102.stdout:0/157: symlink d1/d7/d24/l28 0 2024-05-13T22:47:56.913 INFO:tasks.workunit.client.0.smithi102.stdout:0/158: truncate d1/d7/f1a 1579191 0 2024-05-13T22:47:56.918 INFO:tasks.workunit.client.0.smithi102.stdout:0/159: link d1/de/c20 d1/c29 0 2024-05-13T22:47:56.920 INFO:tasks.workunit.client.0.smithi102.stdout:0/160: write d1/d7/f1e [1863306,13154] 0 2024-05-13T22:47:56.921 INFO:tasks.workunit.client.0.smithi102.stdout:0/161: fdatasync d1/d7/fb 0 2024-05-13T22:47:56.924 INFO:tasks.workunit.client.0.smithi102.stdout:0/162: symlink d1/l2a 0 2024-05-13T22:47:56.926 INFO:tasks.workunit.client.0.smithi102.stdout:0/163: creat d1/f2b x:0 0 0 2024-05-13T22:47:56.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:56 smithi102 ceph-mon[28599]: pgmap v15: 65 pgs: 65 active+clean; 232 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 10 MiB/s wr, 329 op/s 2024-05-13T22:47:56.930 INFO:tasks.workunit.client.0.smithi102.stdout:0/164: link d1/de/f1d d1/f2c 0 2024-05-13T22:47:56.930 INFO:tasks.workunit.client.0.smithi102.stdout:0/165: readlink d1/l10 0 2024-05-13T22:47:56.932 INFO:tasks.workunit.client.0.smithi102.stdout:0/166: mkdir d1/d7/d24/d2d 0 2024-05-13T22:47:56.934 INFO:tasks.workunit.client.0.smithi102.stdout:0/167: unlink d1/d7/fb 0 2024-05-13T22:47:56.948 INFO:tasks.workunit.client.0.smithi102.stdout:0/168: fsync d1/de/f1d 0 2024-05-13T22:47:56.948 INFO:tasks.workunit.client.0.smithi102.stdout:0/169: readlink d1/l10 0 2024-05-13T22:47:56.949 INFO:tasks.workunit.client.0.smithi102.stdout:0/170: write d1/d7/f1e [979116,43459] 0 2024-05-13T22:47:56.950 INFO:tasks.workunit.client.0.smithi102.stdout:0/171: chown d1/lf 149 1 2024-05-13T22:47:56.951 INFO:tasks.workunit.client.0.smithi102.stdout:0/172: dread - d1/de/f27 zero size 2024-05-13T22:47:56.962 INFO:tasks.workunit.client.0.smithi102.stdout:0/173: link d1/de/f1d d1/d7/d24/f2e 0 2024-05-13T22:47:56.964 INFO:tasks.workunit.client.0.smithi102.stdout:9/144: sync 2024-05-13T22:47:56.966 INFO:tasks.workunit.client.0.smithi102.stdout:9/145: rmdir d1/d6/db/d2c/d2d 39 2024-05-13T22:47:56.975 INFO:tasks.workunit.client.0.smithi102.stdout:9/146: dread d1/f17 [0,4194304] 0 2024-05-13T22:47:56.976 INFO:tasks.workunit.client.0.smithi102.stdout:0/174: mknod d1/d7/d24/d2d/c2f 0 2024-05-13T22:47:56.981 INFO:tasks.workunit.client.0.smithi102.stdout:0/175: fsync d1/f9 0 2024-05-13T22:47:56.982 INFO:tasks.workunit.client.0.smithi102.stdout:0/176: chown d1/d7/d24/d2d/c2f 20 1 2024-05-13T22:47:56.983 INFO:tasks.workunit.client.0.smithi102.stdout:0/177: stat d1/d7/d24 0 2024-05-13T22:47:56.988 INFO:tasks.workunit.client.0.smithi102.stdout:0/178: dwrite d1/d7/f17 [0,4194304] 0 2024-05-13T22:47:56.992 INFO:tasks.workunit.client.0.smithi102.stdout:9/147: dread d1/d6/db/f1c [0,4194304] 0 2024-05-13T22:47:56.997 INFO:tasks.workunit.client.0.smithi102.stdout:9/148: mknod d1/d6/db/c3d 0 2024-05-13T22:47:57.000 INFO:tasks.workunit.client.0.smithi102.stdout:9/149: mkdir d1/d6/db/d2c/d3e 0 2024-05-13T22:47:57.001 INFO:tasks.workunit.client.0.smithi102.stdout:9/150: readlink d1/d6/db/d2c/l2f 0 2024-05-13T22:47:57.001 INFO:tasks.workunit.client.0.smithi102.stdout:9/151: readlink d1/d6/db/d2c/l2f 0 2024-05-13T22:47:57.004 INFO:tasks.workunit.client.0.smithi102.stdout:9/152: write d1/d6/db/d2c/d2d/f34 [434122,65364] 0 2024-05-13T22:47:57.006 INFO:tasks.workunit.client.0.smithi102.stdout:9/153: creat d1/d6/db/d2c/d2d/f3f x:0 0 0 2024-05-13T22:47:57.012 INFO:tasks.workunit.client.0.smithi102.stdout:0/179: truncate d1/d7/f13 1830711 0 2024-05-13T22:47:57.014 INFO:tasks.workunit.client.0.smithi102.stdout:9/154: dwrite d1/d6/db/d30/f36 [0,4194304] 0 2024-05-13T22:47:57.015 INFO:tasks.workunit.client.0.smithi102.stdout:0/180: write d1/f2b [871984,68256] 0 2024-05-13T22:47:57.020 INFO:tasks.workunit.client.0.smithi102.stdout:0/181: dread d1/d7/f18 [0,4194304] 0 2024-05-13T22:47:57.022 INFO:tasks.workunit.client.0.smithi102.stdout:9/155: dread d1/f24 [0,4194304] 0 2024-05-13T22:47:57.023 INFO:tasks.workunit.client.0.smithi102.stdout:9/156: dread - d1/d6/d3b/f38 zero size 2024-05-13T22:47:57.023 INFO:tasks.workunit.client.0.smithi102.stdout:0/182: stat d1/de/l23 0 2024-05-13T22:47:57.029 INFO:tasks.workunit.client.0.smithi102.stdout:0/183: dread d1/f1b [0,4194304] 0 2024-05-13T22:47:57.029 INFO:tasks.workunit.client.0.smithi102.stdout:9/157: truncate d1/d6/d21/f26 1724876 0 2024-05-13T22:47:57.030 INFO:tasks.workunit.client.0.smithi102.stdout:0/184: stat d1/d7/f17 0 2024-05-13T22:47:57.030 INFO:tasks.workunit.client.0.smithi102.stdout:9/158: chown d1/d6/db/f1b 25871 1 2024-05-13T22:47:57.043 INFO:tasks.workunit.client.0.smithi102.stdout:0/185: rename d1/d7/f13 to d1/d7/d24/f30 0 2024-05-13T22:47:57.052 INFO:tasks.workunit.client.0.smithi102.stdout:9/159: write d1/d6/d21/f26 [640213,26384] 0 2024-05-13T22:47:57.059 INFO:tasks.workunit.client.0.smithi102.stdout:9/160: rmdir d1/d6/db/d2c/d2d 39 2024-05-13T22:47:57.059 INFO:tasks.workunit.client.0.smithi102.stdout:9/161: rename d1/d6/f12 to d1/d6/f40 0 2024-05-13T22:47:57.059 INFO:tasks.workunit.client.0.smithi102.stdout:0/186: write d1/d7/d24/f2e [3132139,96957] 0 2024-05-13T22:47:57.060 INFO:tasks.workunit.client.0.smithi102.stdout:9/162: chown d1/l4 87 1 2024-05-13T22:47:57.065 INFO:tasks.workunit.client.0.smithi102.stdout:0/187: link d1/d7/f1e d1/d7/d24/f31 0 2024-05-13T22:47:57.067 INFO:tasks.workunit.client.0.smithi102.stdout:9/163: rename d1/d6/db/l35 to d1/d6/db/d2c/d3e/l41 0 2024-05-13T22:47:57.068 INFO:tasks.workunit.client.0.smithi102.stdout:9/164: chown d1/d6/db/f1b 3 1 2024-05-13T22:47:57.069 INFO:tasks.workunit.client.0.smithi102.stdout:9/165: creat d1/d6/d21/f42 x:0 0 0 2024-05-13T22:47:57.074 INFO:tasks.workunit.client.0.smithi102.stdout:0/188: rename d1/d7 to d1/de/d32 0 2024-05-13T22:47:57.077 INFO:tasks.workunit.client.0.smithi102.stdout:9/166: truncate d1/d6/db/f1c 2904539 0 2024-05-13T22:47:57.077 INFO:tasks.workunit.client.0.smithi102.stdout:9/167: dread - d1/d6/db/f1b zero size 2024-05-13T22:47:57.081 INFO:tasks.workunit.client.0.smithi102.stdout:9/168: rename d1/d6/db/d2c/d2d/f34 to d1/d6/db/d2c/d3e/f43 0 2024-05-13T22:47:57.083 INFO:tasks.workunit.client.0.smithi102.stdout:9/169: fsync d1/f7 0 2024-05-13T22:47:57.084 INFO:tasks.workunit.client.0.smithi102.stdout:0/189: dread d1/f4 [0,4194304] 0 2024-05-13T22:47:57.087 INFO:tasks.workunit.client.0.smithi102.stdout:0/190: creat d1/de/f33 x:0 0 0 2024-05-13T22:47:57.090 INFO:tasks.workunit.client.0.smithi102.stdout:9/170: dwrite d1/d6/db/d2c/d2d/f3f [0,4194304] 0 2024-05-13T22:47:57.093 INFO:tasks.workunit.client.0.smithi102.stdout:9/171: chown d1/f2b 37 1 2024-05-13T22:47:57.093 INFO:tasks.workunit.client.0.smithi102.stdout:0/191: dread d1/f2c [0,4194304] 0 2024-05-13T22:47:57.095 INFO:tasks.workunit.client.0.smithi102.stdout:0/192: truncate d1/f2b 1414607 0 2024-05-13T22:47:57.097 INFO:tasks.workunit.client.0.smithi102.stdout:9/172: mknod d1/d6/db/c44 0 2024-05-13T22:47:57.098 INFO:tasks.workunit.client.0.smithi102.stdout:9/173: write d1/f2b [90135,13785] 0 2024-05-13T22:47:57.102 INFO:tasks.workunit.client.0.smithi102.stdout:0/193: dread d1/f5 [0,4194304] 0 2024-05-13T22:47:57.103 INFO:tasks.workunit.client.0.smithi102.stdout:9/174: rename d1/d6/db/d2c to d1/d6/d45 0 2024-05-13T22:47:57.108 INFO:tasks.workunit.client.0.smithi102.stdout:9/175: mknod d1/d6/d21/c46 0 2024-05-13T22:47:57.111 INFO:tasks.workunit.client.0.smithi102.stdout:9/176: mkdir d1/d6/db/d30/d47 0 2024-05-13T22:47:57.117 INFO:tasks.workunit.client.0.smithi102.stdout:0/194: write d1/f1b [5062803,37281] 0 2024-05-13T22:47:57.124 INFO:tasks.workunit.client.0.smithi102.stdout:0/195: symlink d1/de/d32/d24/l34 0 2024-05-13T22:47:57.141 INFO:tasks.workunit.client.0.smithi102.stdout:0/196: truncate d1/de/f1c 1824469 0 2024-05-13T22:47:57.142 INFO:tasks.workunit.client.0.smithi102.stdout:9/177: dwrite d1/f24 [0,4194304] 0 2024-05-13T22:47:57.146 INFO:tasks.workunit.client.0.smithi102.stdout:9/178: creat d1/d6/d21/f48 x:0 0 0 2024-05-13T22:47:57.146 INFO:tasks.workunit.client.0.smithi102.stdout:2/171: sync 2024-05-13T22:47:57.147 INFO:tasks.workunit.client.0.smithi102.stdout:0/197: symlink d1/de/l35 0 2024-05-13T22:47:57.149 INFO:tasks.workunit.client.0.smithi102.stdout:2/172: mkdir d7/d9/d14/d31 0 2024-05-13T22:47:57.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:56 smithi130 ceph-mon[36591]: pgmap v15: 65 pgs: 65 active+clean; 232 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 10 MiB/s wr, 329 op/s 2024-05-13T22:47:57.151 INFO:tasks.workunit.client.0.smithi102.stdout:0/198: read d1/de/f19 [3513612,24233] 0 2024-05-13T22:47:57.158 INFO:tasks.workunit.client.0.smithi102.stdout:0/199: dwrite d1/de/f19 [0,4194304] 0 2024-05-13T22:47:57.167 INFO:tasks.workunit.client.0.smithi102.stdout:2/173: rename d7/f2d to d7/f32 0 2024-05-13T22:47:57.178 INFO:tasks.workunit.client.0.smithi102.stdout:0/200: link d1/de/d32/f18 d1/de/d32/f36 0 2024-05-13T22:47:57.178 INFO:tasks.workunit.client.0.smithi102.stdout:0/201: readlink d1/de/l23 0 2024-05-13T22:47:57.185 INFO:tasks.workunit.client.0.smithi102.stdout:9/179: truncate d1/d6/db/f1c 1755770 0 2024-05-13T22:47:57.190 INFO:tasks.workunit.client.0.smithi102.stdout:9/180: creat d1/d6/db/d30/d47/f49 x:0 0 0 2024-05-13T22:47:57.193 INFO:tasks.workunit.client.0.smithi102.stdout:9/181: mkdir d1/d6/db/d4a 0 2024-05-13T22:47:57.199 INFO:tasks.workunit.client.0.smithi102.stdout:9/182: dwrite d1/d6/d21/f26 [0,4194304] 0 2024-05-13T22:47:57.202 INFO:tasks.workunit.client.0.smithi102.stdout:9/183: dread d1/f2b [0,4194304] 0 2024-05-13T22:47:57.208 INFO:tasks.workunit.client.0.smithi102.stdout:9/184: dwrite d1/d6/db/d30/f36 [0,4194304] 0 2024-05-13T22:47:57.210 INFO:tasks.workunit.client.0.smithi102.stdout:9/185: chown d1/d6/db/f1c 273775593 1 2024-05-13T22:47:57.216 INFO:tasks.workunit.client.0.smithi102.stdout:9/186: dwrite d1/d6/d45/d2d/f3f [0,4194304] 0 2024-05-13T22:47:57.228 INFO:tasks.workunit.client.0.smithi102.stdout:9/187: dwrite d1/d6/d45/d3e/f43 [0,4194304] 0 2024-05-13T22:47:57.229 INFO:tasks.workunit.client.0.smithi102.stdout:0/202: stat d1/f1b 0 2024-05-13T22:47:57.240 INFO:tasks.workunit.client.0.smithi102.stdout:9/188: link d1/d6/f40 d1/d6/d45/d2d/f4b 0 2024-05-13T22:47:57.246 INFO:tasks.workunit.client.0.smithi102.stdout:9/189: dwrite d1/f16 [4194304,4194304] 0 2024-05-13T22:47:57.251 INFO:tasks.workunit.client.0.smithi102.stdout:9/190: rename d1/f24 to d1/d6/d45/d3e/f4c 0 2024-05-13T22:47:57.265 INFO:tasks.workunit.client.0.smithi102.stdout:0/203: write d1/de/d32/d24/f31 [180996,122326] 0 2024-05-13T22:47:57.268 INFO:tasks.workunit.client.0.smithi102.stdout:0/204: truncate d1/f5 1893812 0 2024-05-13T22:47:57.272 INFO:tasks.workunit.client.0.smithi102.stdout:0/205: dread d1/de/d32/d24/f30 [0,4194304] 0 2024-05-13T22:47:57.274 INFO:tasks.workunit.client.0.smithi102.stdout:0/206: fdatasync d1/f9 0 2024-05-13T22:47:57.280 INFO:tasks.workunit.client.0.smithi102.stdout:0/207: getdents d1 0 2024-05-13T22:47:57.285 INFO:tasks.workunit.client.0.smithi102.stdout:0/208: dwrite d1/f2b [0,4194304] 0 2024-05-13T22:47:57.290 INFO:tasks.workunit.client.0.smithi102.stdout:0/209: chown d1/de/d32/d24/f2e 6 1 2024-05-13T22:47:57.297 INFO:tasks.workunit.client.0.smithi102.stdout:0/210: dwrite d1/f2c [4194304,4194304] 0 2024-05-13T22:47:57.304 INFO:tasks.workunit.client.0.smithi102.stdout:0/211: dwrite d1/f2c [4194304,4194304] 0 2024-05-13T22:47:57.312 INFO:tasks.workunit.client.0.smithi102.stdout:0/212: creat d1/de/d32/f37 x:0 0 0 2024-05-13T22:47:57.314 INFO:tasks.workunit.client.0.smithi102.stdout:0/213: rmdir d1 39 2024-05-13T22:47:57.324 INFO:tasks.workunit.client.0.smithi102.stdout:0/214: mkdir d1/de/d32/d24/d2d/d38 0 2024-05-13T22:47:57.376 INFO:tasks.workunit.client.0.smithi102.stdout:0/215: dwrite d1/f4 [0,4194304] 0 2024-05-13T22:47:57.378 INFO:tasks.workunit.client.0.smithi102.stdout:0/216: truncate d1/de/d32/f1e 2652940 0 2024-05-13T22:47:57.381 INFO:tasks.workunit.client.0.smithi102.stdout:0/217: mknod d1/de/d32/d24/c39 0 2024-05-13T22:47:57.382 INFO:tasks.workunit.client.0.smithi102.stdout:0/218: read - d1/de/d32/f37 zero size 2024-05-13T22:47:57.398 INFO:tasks.workunit.client.0.smithi102.stdout:0/219: dwrite d1/de/d32/f36 [0,4194304] 0 2024-05-13T22:47:57.402 INFO:tasks.workunit.client.0.smithi102.stdout:0/220: dread d1/f2b [0,4194304] 0 2024-05-13T22:47:57.404 INFO:tasks.workunit.client.0.smithi102.stdout:0/221: rename d1 to d1/de/d3a 22 2024-05-13T22:47:57.406 INFO:tasks.workunit.client.0.smithi102.stdout:0/222: getdents d1/de/d32/d24/d2d/d38 0 2024-05-13T22:47:57.409 INFO:tasks.workunit.client.0.smithi102.stdout:0/223: creat d1/de/d32/f3b x:0 0 0 2024-05-13T22:47:57.429 INFO:tasks.workunit.client.0.smithi102.stdout:0/224: dwrite d1/f1b [0,4194304] 0 2024-05-13T22:47:57.430 INFO:tasks.workunit.client.0.smithi102.stdout:0/225: fsync d1/de/d32/f18 0 2024-05-13T22:47:57.445 INFO:tasks.workunit.client.0.smithi102.stdout:0/226: dread d1/de/d32/f1a [0,4194304] 0 2024-05-13T22:47:57.450 INFO:tasks.workunit.client.0.smithi102.stdout:0/227: dwrite d1/de/f19 [0,4194304] 0 2024-05-13T22:47:57.467 INFO:tasks.workunit.client.0.smithi102.stdout:0/228: dwrite d1/f5 [0,4194304] 0 2024-05-13T22:47:57.476 INFO:tasks.workunit.client.0.smithi102.stdout:0/229: truncate d1/de/d32/f1e 684898 0 2024-05-13T22:47:57.479 INFO:tasks.workunit.client.0.smithi102.stdout:0/230: rename d1/f2b to d1/de/d32/d24/f3c 0 2024-05-13T22:47:57.551 INFO:tasks.workunit.client.0.smithi102.stdout:5/195: sync 2024-05-13T22:47:57.554 INFO:tasks.workunit.client.0.smithi102.stdout:5/196: mknod de/d25/d2c/c51 0 2024-05-13T22:47:57.570 INFO:tasks.workunit.client.0.smithi102.stdout:5/197: dread de/d33/d34/d3a/f24 [0,4194304] 0 2024-05-13T22:47:57.572 INFO:tasks.workunit.client.0.smithi102.stdout:5/198: symlink de/d25/l52 0 2024-05-13T22:47:57.573 INFO:tasks.workunit.client.0.smithi102.stdout:5/199: mkdir de/d33/d53 0 2024-05-13T22:47:57.574 INFO:tasks.workunit.client.0.smithi102.stdout:5/200: truncate fd 4974365 0 2024-05-13T22:47:57.577 INFO:tasks.workunit.client.0.smithi102.stdout:5/201: mknod de/d25/d2a/d4b/d4d/d50/c54 0 2024-05-13T22:47:57.580 INFO:tasks.workunit.client.0.smithi102.stdout:5/202: creat de/d33/d34/d3a/d28/f55 x:0 0 0 2024-05-13T22:47:57.583 INFO:tasks.workunit.client.0.smithi102.stdout:5/203: unlink de/d33/d34/d3a/l22 0 2024-05-13T22:47:57.584 INFO:tasks.workunit.client.0.smithi102.stdout:5/204: creat de/d25/d2a/f56 x:0 0 0 2024-05-13T22:47:57.585 INFO:tasks.workunit.client.0.smithi102.stdout:5/205: fdatasync de/f1d 0 2024-05-13T22:47:57.618 INFO:tasks.workunit.client.0.smithi102.stdout:5/206: truncate f0 371213 0 2024-05-13T22:47:57.621 INFO:tasks.workunit.client.0.smithi102.stdout:5/207: creat de/d33/d53/f57 x:0 0 0 2024-05-13T22:47:57.622 INFO:tasks.workunit.client.0.smithi102.stdout:5/208: truncate de/d33/d34/d3a/f49 995347 0 2024-05-13T22:47:57.623 INFO:tasks.workunit.client.0.smithi102.stdout:5/209: write de/d25/d2a/f56 [846585,108901] 0 2024-05-13T22:47:57.625 INFO:tasks.workunit.client.0.smithi102.stdout:5/210: mkdir de/d25/d58 0 2024-05-13T22:47:57.628 INFO:tasks.workunit.client.0.smithi102.stdout:5/211: rename de/d33/d34/d3a to de/d25/d59 0 2024-05-13T22:47:57.630 INFO:tasks.workunit.client.0.smithi102.stdout:5/212: rmdir de/d25/d2a 39 2024-05-13T22:47:57.632 INFO:tasks.workunit.client.0.smithi102.stdout:5/213: creat de/f5a x:0 0 0 2024-05-13T22:47:57.634 INFO:tasks.workunit.client.0.smithi102.stdout:5/214: mknod de/d25/c5b 0 2024-05-13T22:47:57.634 INFO:tasks.workunit.client.0.smithi102.stdout:5/215: dread - de/d25/f47 zero size 2024-05-13T22:47:57.650 INFO:tasks.workunit.client.0.smithi102.stdout:5/216: dread de/f1e [0,4194304] 0 2024-05-13T22:47:57.652 INFO:tasks.workunit.client.0.smithi102.stdout:5/217: dread de/d25/f26 [0,4194304] 0 2024-05-13T22:47:57.656 INFO:tasks.workunit.client.0.smithi102.stdout:5/218: link de/d25/d59/f40 de/d25/d59/d28/f5c 0 2024-05-13T22:47:57.658 INFO:tasks.workunit.client.0.smithi102.stdout:5/219: creat de/d25/d59/f5d x:0 0 0 2024-05-13T22:47:57.659 INFO:tasks.workunit.client.0.smithi102.stdout:5/220: write de/d25/d59/f49 [1605897,98307] 0 2024-05-13T22:47:57.662 INFO:tasks.workunit.client.0.smithi102.stdout:5/221: creat de/d25/d43/f5e x:0 0 0 2024-05-13T22:47:57.663 INFO:tasks.workunit.client.0.smithi102.stdout:5/222: fdatasync de/d25/d59/f1c 0 2024-05-13T22:47:57.664 INFO:tasks.workunit.client.0.smithi102.stdout:5/223: chown l3 182 1 2024-05-13T22:47:57.667 INFO:tasks.workunit.client.0.smithi102.stdout:5/224: link de/d25/d59/f1c de/d25/d2c/f5f 0 2024-05-13T22:47:57.670 INFO:tasks.workunit.client.0.smithi102.stdout:5/225: link de/d25/d2c/l2d de/d25/d2c/l60 0 2024-05-13T22:47:57.671 INFO:tasks.workunit.client.0.smithi102.stdout:5/226: chown de/l19 2369439 1 2024-05-13T22:47:57.671 INFO:tasks.workunit.client.0.smithi102.stdout:5/227: write de/d25/d59/f5d [124004,1575] 0 2024-05-13T22:47:57.677 INFO:tasks.workunit.client.0.smithi102.stdout:3/211: sync 2024-05-13T22:47:57.679 INFO:tasks.workunit.client.0.smithi102.stdout:5/228: read de/d25/d59/f32 [2073417,51926] 0 2024-05-13T22:47:57.685 INFO:tasks.workunit.client.0.smithi102.stdout:3/212: creat de/d10/d2a/d36/f49 x:0 0 0 2024-05-13T22:47:57.687 INFO:tasks.workunit.client.0.smithi102.stdout:5/229: dwrite de/d25/d59/f1c [0,4194304] 0 2024-05-13T22:47:57.690 INFO:tasks.workunit.client.0.smithi102.stdout:3/213: dread de/d10/f43 [0,4194304] 0 2024-05-13T22:47:57.691 INFO:tasks.workunit.client.0.smithi102.stdout:3/214: chown de/d10/f1d 250368 1 2024-05-13T22:47:57.698 INFO:tasks.workunit.client.0.smithi102.stdout:3/215: getdents de 0 2024-05-13T22:47:57.698 INFO:tasks.workunit.client.0.smithi102.stdout:3/216: dread - de/d10/d2a/d33/f40 zero size 2024-05-13T22:47:57.699 INFO:tasks.workunit.client.0.smithi102.stdout:3/217: fsync de/d10/f12 0 2024-05-13T22:47:57.785 INFO:tasks.workunit.client.0.smithi102.stdout:5/230: dwrite f0 [0,4194304] 0 2024-05-13T22:47:57.787 INFO:tasks.workunit.client.0.smithi102.stdout:3/218: dwrite de/f37 [4194304,4194304] 0 2024-05-13T22:47:57.795 INFO:tasks.workunit.client.0.smithi102.stdout:3/219: dwrite de/d10/f44 [0,4194304] 0 2024-05-13T22:47:57.814 INFO:tasks.workunit.client.0.smithi102.stdout:5/231: truncate de/d25/d59/f1c 4056817 0 2024-05-13T22:47:57.816 INFO:tasks.workunit.client.0.smithi102.stdout:3/220: truncate f5 494016 0 2024-05-13T22:47:57.817 INFO:tasks.workunit.client.0.smithi102.stdout:1/128: sync 2024-05-13T22:47:57.819 INFO:tasks.workunit.client.0.smithi102.stdout:3/221: symlink de/d10/l4a 0 2024-05-13T22:47:57.822 INFO:tasks.workunit.client.0.smithi102.stdout:5/232: dread de/d25/d59/f1f [0,4194304] 0 2024-05-13T22:47:57.824 INFO:tasks.workunit.client.0.smithi102.stdout:1/129: dwrite d2/d4/d5/fa [0,4194304] 0 2024-05-13T22:47:57.832 INFO:tasks.workunit.client.0.smithi102.stdout:5/233: creat de/f61 x:0 0 0 2024-05-13T22:47:57.838 INFO:tasks.workunit.client.0.smithi102.stdout:5/234: symlink de/d25/d58/l62 0 2024-05-13T22:47:57.839 INFO:tasks.workunit.client.0.smithi102.stdout:1/130: dread d2/d4/d5/f15 [4194304,4194304] 0 2024-05-13T22:47:57.842 INFO:tasks.workunit.client.0.smithi102.stdout:3/222: write de/d10/f11 [4944587,14076] 0 2024-05-13T22:47:57.843 INFO:tasks.workunit.client.0.smithi102.stdout:1/131: unlink d2/d10/f12 0 2024-05-13T22:47:57.844 INFO:tasks.workunit.client.0.smithi102.stdout:1/132: fdatasync d2/d4/f20 0 2024-05-13T22:47:57.847 INFO:tasks.workunit.client.0.smithi102.stdout:3/223: mkdir de/d10/d2a/d4b 0 2024-05-13T22:47:57.852 INFO:tasks.workunit.client.0.smithi102.stdout:1/133: creat d2/f2b x:0 0 0 2024-05-13T22:47:57.856 INFO:tasks.workunit.client.0.smithi102.stdout:3/224: creat de/d10/d2a/d48/f4c x:0 0 0 2024-05-13T22:47:57.857 INFO:tasks.workunit.client.0.smithi102.stdout:5/235: truncate de/d25/f26 194127 0 2024-05-13T22:47:57.858 INFO:tasks.workunit.client.0.smithi102.stdout:3/225: creat de/d10/d2a/d33/f4d x:0 0 0 2024-05-13T22:47:57.860 INFO:tasks.workunit.client.0.smithi102.stdout:1/134: rename d2/d4/d5/l27 to d2/d4/d26/l2c 0 2024-05-13T22:47:57.864 INFO:tasks.workunit.client.0.smithi102.stdout:5/236: rmdir de/d25/d2a/d4b/d4d/d50 39 2024-05-13T22:47:57.865 INFO:tasks.workunit.client.0.smithi102.stdout:3/226: dwrite de/d10/d2a/f3a [0,4194304] 0 2024-05-13T22:47:57.866 INFO:tasks.workunit.client.0.smithi102.stdout:1/135: unlink d2/f16 0 2024-05-13T22:47:57.867 INFO:tasks.workunit.client.0.smithi102.stdout:1/136: chown d2/d4/cc 44 1 2024-05-13T22:47:57.871 INFO:tasks.workunit.client.0.smithi102.stdout:3/227: rename de/d21/c2c to de/d10/d2a/c4e 0 2024-05-13T22:47:57.871 INFO:tasks.workunit.client.0.smithi102.stdout:5/237: mknod de/d25/d2a/d4b/d4d/c63 0 2024-05-13T22:47:57.872 INFO:tasks.workunit.client.0.smithi102.stdout:3/228: write de/d10/d2a/f3a [647310,28191] 0 2024-05-13T22:47:57.873 INFO:tasks.workunit.client.0.smithi102.stdout:1/137: symlink d2/d4/d26/l2d 0 2024-05-13T22:47:57.875 INFO:tasks.workunit.client.0.smithi102.stdout:5/238: symlink de/d25/d58/l64 0 2024-05-13T22:47:57.876 INFO:tasks.workunit.client.0.smithi102.stdout:1/138: rmdir d2/d10 39 2024-05-13T22:47:57.877 INFO:tasks.workunit.client.0.smithi102.stdout:3/229: dread de/d10/f43 [0,4194304] 0 2024-05-13T22:47:57.878 INFO:tasks.workunit.client.0.smithi102.stdout:5/239: mkdir de/d33/d65 0 2024-05-13T22:47:57.881 INFO:tasks.workunit.client.0.smithi102.stdout:3/230: creat de/d1c/f4f x:0 0 0 2024-05-13T22:47:57.881 INFO:tasks.workunit.client.0.smithi102.stdout:5/240: rmdir de/d33 39 2024-05-13T22:47:57.884 INFO:tasks.workunit.client.0.smithi102.stdout:1/139: mknod d2/d4/d5/d19/d28/c2e 0 2024-05-13T22:47:57.886 INFO:tasks.workunit.client.0.smithi102.stdout:1/140: write d2/d4/f20 [895252,117093] 0 2024-05-13T22:47:57.887 INFO:tasks.workunit.client.0.smithi102.stdout:3/231: dwrite de/d21/f2b [0,4194304] 0 2024-05-13T22:47:57.894 INFO:tasks.workunit.client.0.smithi102.stdout:3/232: rename de/d10/d2a/d33/d3b to de/d10/d50 0 2024-05-13T22:47:57.895 INFO:tasks.workunit.client.0.smithi102.stdout:5/241: getdents de/d25/d2a 0 2024-05-13T22:47:57.901 INFO:tasks.workunit.client.0.smithi102.stdout:3/233: dwrite de/d10/f44 [4194304,4194304] 0 2024-05-13T22:47:57.903 INFO:tasks.workunit.client.0.smithi102.stdout:1/141: getdents d2/d4/d26 0 2024-05-13T22:47:57.907 INFO:tasks.workunit.client.0.smithi102.stdout:3/234: fsync de/d1c/f4f 0 2024-05-13T22:47:57.911 INFO:tasks.workunit.client.0.smithi102.stdout:3/235: dread de/d10/f43 [0,4194304] 0 2024-05-13T22:47:57.913 INFO:tasks.workunit.client.0.smithi102.stdout:3/236: write fa [2290760,50277] 0 2024-05-13T22:47:57.916 INFO:tasks.workunit.client.0.smithi102.stdout:3/237: symlink de/d10/d2a/d36/l51 0 2024-05-13T22:47:57.917 INFO:tasks.workunit.client.0.smithi102.stdout:5/242: truncate de/f1e 917254 0 2024-05-13T22:47:57.918 INFO:tasks.workunit.client.0.smithi102.stdout:1/142: dread d2/d4/f23 [0,4194304] 0 2024-05-13T22:47:57.921 INFO:tasks.workunit.client.0.smithi102.stdout:3/238: creat de/d1c/f52 x:0 0 0 2024-05-13T22:47:57.921 INFO:tasks.workunit.client.0.smithi102.stdout:5/243: stat de/d25/d59/f2e 0 2024-05-13T22:47:57.923 INFO:tasks.workunit.client.0.smithi102.stdout:3/239: symlink de/l53 0 2024-05-13T22:47:57.928 INFO:tasks.workunit.client.0.smithi102.stdout:5/244: dwrite de/d25/d59/f44 [0,4194304] 0 2024-05-13T22:47:57.933 INFO:tasks.workunit.client.0.smithi102.stdout:5/245: creat de/d25/d59/d28/f66 x:0 0 0 2024-05-13T22:47:57.934 INFO:tasks.workunit.client.0.smithi102.stdout:5/246: read fd [4621737,100498] 0 2024-05-13T22:47:57.938 INFO:tasks.workunit.client.0.smithi102.stdout:5/247: creat de/d25/d59/d28/f67 x:0 0 0 2024-05-13T22:47:57.940 INFO:tasks.workunit.client.0.smithi102.stdout:1/143: dread d2/d4/d5/f15 [0,4194304] 0 2024-05-13T22:47:57.941 INFO:tasks.workunit.client.0.smithi102.stdout:5/248: dread - de/d33/d53/f57 zero size 2024-05-13T22:47:57.943 INFO:tasks.workunit.client.0.smithi102.stdout:1/144: mkdir d2/d4/d5/d19/d2f 0 2024-05-13T22:47:57.944 INFO:tasks.workunit.client.0.smithi102.stdout:8/94: sync 2024-05-13T22:47:57.944 INFO:tasks.workunit.client.0.smithi102.stdout:4/175: sync 2024-05-13T22:47:57.948 INFO:tasks.workunit.client.0.smithi102.stdout:1/145: readlink d2/l7 0 2024-05-13T22:47:57.949 INFO:tasks.workunit.client.0.smithi102.stdout:3/240: write de/d10/d2a/f2f [646661,103540] 0 2024-05-13T22:47:57.949 INFO:tasks.workunit.client.0.smithi102.stdout:1/146: chown d2/l7 467554088 1 2024-05-13T22:47:57.952 INFO:tasks.workunit.client.0.smithi102.stdout:1/147: chown d2/d4/d5/d19 8362 1 2024-05-13T22:47:57.952 INFO:tasks.workunit.client.0.smithi102.stdout:8/95: dwrite d2/d7/d12/f1c [0,4194304] 0 2024-05-13T22:47:57.958 INFO:tasks.workunit.client.0.smithi102.stdout:4/176: rename d3/d4/c16 to d3/d19/d23/c45 0 2024-05-13T22:47:57.959 INFO:tasks.workunit.client.0.smithi102.stdout:5/249: rmdir de/d33/d65 0 2024-05-13T22:47:57.962 INFO:tasks.workunit.client.0.smithi102.stdout:4/177: write d3/d4/d6/f42 [809408,61501] 0 2024-05-13T22:47:57.963 INFO:tasks.workunit.client.0.smithi102.stdout:1/148: dwrite d2/d4/d5/fa [0,4194304] 0 2024-05-13T22:47:57.964 INFO:tasks.workunit.client.0.smithi102.stdout:4/178: fsync d3/d19/d23/f3d 0 2024-05-13T22:47:57.967 INFO:tasks.workunit.client.0.smithi102.stdout:5/250: mknod de/d33/d53/c68 0 2024-05-13T22:47:57.968 INFO:tasks.workunit.client.0.smithi102.stdout:8/96: dread d2/d7/fb [0,4194304] 0 2024-05-13T22:47:57.969 INFO:tasks.workunit.client.0.smithi102.stdout:8/97: readlink d2/d7/l8 0 2024-05-13T22:47:57.969 INFO:tasks.workunit.client.0.smithi102.stdout:1/149: mknod d2/d4/d5/c30 0 2024-05-13T22:47:57.970 INFO:tasks.workunit.client.0.smithi102.stdout:8/98: chown d2/f4 811102059 1 2024-05-13T22:47:57.972 INFO:tasks.workunit.client.0.smithi102.stdout:3/241: dread de/f19 [0,4194304] 0 2024-05-13T22:47:57.972 INFO:tasks.workunit.client.0.smithi102.stdout:1/150: write d2/f2b [940174,44361] 0 2024-05-13T22:47:57.974 INFO:tasks.workunit.client.0.smithi102.stdout:4/179: dwrite d3/d19/d23/f3d [0,4194304] 0 2024-05-13T22:47:57.976 INFO:tasks.workunit.client.0.smithi102.stdout:4/180: write d3/d4/f1e [5943590,48096] 0 2024-05-13T22:47:57.976 INFO:tasks.workunit.client.0.smithi102.stdout:3/242: read de/d10/d2a/f2f [21640,26281] 0 2024-05-13T22:47:57.980 INFO:tasks.workunit.client.0.smithi102.stdout:4/181: write d3/d4/d6/de/f34 [146870,43724] 0 2024-05-13T22:47:57.983 INFO:tasks.workunit.client.0.smithi102.stdout:8/99: fdatasync d2/d7/fb 0 2024-05-13T22:47:57.985 INFO:tasks.workunit.client.0.smithi102.stdout:6/211: sync 2024-05-13T22:47:57.985 INFO:tasks.workunit.client.0.smithi102.stdout:2/174: sync 2024-05-13T22:47:57.985 INFO:tasks.workunit.client.0.smithi102.stdout:7/118: sync 2024-05-13T22:47:57.985 INFO:tasks.workunit.client.0.smithi102.stdout:9/191: sync 2024-05-13T22:47:57.985 INFO:tasks.workunit.client.0.smithi102.stdout:0/231: sync 2024-05-13T22:47:57.988 INFO:tasks.workunit.client.0.smithi102.stdout:2/175: write d7/d15/f1f [298696,10314] 0 2024-05-13T22:47:57.989 INFO:tasks.workunit.client.0.smithi102.stdout:4/182: creat d3/d4/d6/de/f46 x:0 0 0 2024-05-13T22:47:57.991 INFO:tasks.workunit.client.0.smithi102.stdout:3/243: dwrite de/d10/d2a/f3a [0,4194304] 0 2024-05-13T22:47:57.995 INFO:tasks.workunit.client.0.smithi102.stdout:1/151: creat d2/d10/f31 x:0 0 0 2024-05-13T22:47:57.999 INFO:tasks.workunit.client.0.smithi102.stdout:7/119: dread d6/d13/f16 [0,4194304] 0 2024-05-13T22:47:58.002 INFO:tasks.workunit.client.0.smithi102.stdout:8/100: unlink d2/d7/ld 0 2024-05-13T22:47:58.004 INFO:tasks.workunit.client.0.smithi102.stdout:5/251: dwrite de/d25/d59/f1c [0,4194304] 0 2024-05-13T22:47:58.004 INFO:tasks.workunit.client.0.smithi102.stdout:4/183: rmdir d3/d4 39 2024-05-13T22:47:58.008 INFO:tasks.workunit.client.0.smithi102.stdout:2/176: creat d7/d9/d10/f33 x:0 0 0 2024-05-13T22:47:58.008 INFO:tasks.workunit.client.0.smithi102.stdout:2/177: dread - d7/d9/d10/f33 zero size 2024-05-13T22:47:58.015 INFO:tasks.workunit.client.0.smithi102.stdout:0/232: mkdir d1/de/d32/d24/d2d/d38/d3d 0 2024-05-13T22:47:58.019 INFO:tasks.workunit.client.0.smithi102.stdout:5/252: symlink de/d25/d2a/l69 0 2024-05-13T22:47:58.020 INFO:tasks.workunit.client.0.smithi102.stdout:5/253: truncate de/f4f 672121 0 2024-05-13T22:47:58.021 INFO:tasks.workunit.client.0.smithi102.stdout:3/244: unlink c8 0 2024-05-13T22:47:58.021 INFO:tasks.workunit.client.0.smithi102.stdout:5/254: fdatasync de/d25/d2a/f3c 0 2024-05-13T22:47:58.021 INFO:tasks.workunit.client.0.smithi102.stdout:3/245: chown de/d10/d34 759860903 1 2024-05-13T22:47:58.022 INFO:tasks.workunit.client.0.smithi102.stdout:2/178: creat d7/d9/d14/f34 x:0 0 0 2024-05-13T22:47:58.023 INFO:tasks.workunit.client.0.smithi102.stdout:2/179: chown d7/d15/d1e 989 1 2024-05-13T22:47:58.026 INFO:tasks.workunit.client.0.smithi102.stdout:7/120: mkdir d6/d13/de/d19/d20 0 2024-05-13T22:47:58.030 INFO:tasks.workunit.client.0.smithi102.stdout:6/212: rename d9/db/d2b to d9/d45 0 2024-05-13T22:47:58.030 INFO:tasks.workunit.client.0.smithi102.stdout:2/180: dwrite d7/d9/d10/f33 [0,4194304] 0 2024-05-13T22:47:58.032 INFO:tasks.workunit.client.0.smithi102.stdout:8/101: mknod d2/d17/c1e 0 2024-05-13T22:47:58.034 INFO:tasks.workunit.client.0.smithi102.stdout:8/102: write d2/d7/d12/f13 [629120,29532] 0 2024-05-13T22:47:58.034 INFO:tasks.workunit.client.0.smithi102.stdout:0/233: mkdir d1/de/d32/d24/d2d/d3e 0 2024-05-13T22:47:58.039 INFO:tasks.workunit.client.0.smithi102.stdout:9/192: dwrite d1/f1e [0,4194304] 0 2024-05-13T22:47:58.040 INFO:tasks.workunit.client.0.smithi102.stdout:1/152: rmdir d2/d4/d5/d19/d2f 0 2024-05-13T22:47:58.042 INFO:tasks.workunit.client.0.smithi102.stdout:7/121: rmdir d6/d13 39 2024-05-13T22:47:58.047 INFO:tasks.workunit.client.0.smithi102.stdout:5/255: dwrite de/d25/d59/f32 [4194304,4194304] 0 2024-05-13T22:47:58.049 INFO:tasks.workunit.client.0.smithi102.stdout:5/256: fsync de/f35 0 2024-05-13T22:47:58.049 INFO:tasks.workunit.client.0.smithi102.stdout:4/184: rename d3/d22/d2b/f1d to d3/d4/f47 0 2024-05-13T22:47:58.051 INFO:tasks.workunit.client.0.smithi102.stdout:6/213: mknod d9/db/d14/d35/c46 0 2024-05-13T22:47:58.053 INFO:tasks.workunit.client.0.smithi102.stdout:7/122: dwrite d6/d15/f1b [0,4194304] 0 2024-05-13T22:47:58.055 INFO:tasks.workunit.client.0.smithi102.stdout:2/181: creat d7/d9/d10/d25/f35 x:0 0 0 2024-05-13T22:47:58.058 INFO:tasks.workunit.client.0.smithi102.stdout:8/103: mknod d2/d7/d12/c1f 0 2024-05-13T22:47:58.060 INFO:tasks.workunit.client.0.smithi102.stdout:8/104: fdatasync d2/d7/d12/f13 0 2024-05-13T22:47:58.065 INFO:tasks.workunit.client.0.smithi102.stdout:3/246: link f2 de/d10/d2a/f54 0 2024-05-13T22:47:58.067 INFO:tasks.workunit.client.0.smithi102.stdout:3/247: read - de/d10/d2a/d36/f49 zero size 2024-05-13T22:47:58.069 INFO:tasks.workunit.client.0.smithi102.stdout:8/105: dwrite d2/d7/d12/f1c [4194304,4194304] 0 2024-05-13T22:47:58.071 INFO:tasks.workunit.client.0.smithi102.stdout:3/248: readlink de/d10/d2a/d36/l51 0 2024-05-13T22:47:58.073 INFO:tasks.workunit.client.0.smithi102.stdout:8/106: chown d2/d7/c15 32203138 1 2024-05-13T22:47:58.077 INFO:tasks.workunit.client.0.smithi102.stdout:0/234: unlink d1/de/d32/d24/l28 0 2024-05-13T22:47:58.078 INFO:tasks.workunit.client.0.smithi102.stdout:9/193: mknod d1/d6/db/d30/c4d 0 2024-05-13T22:47:58.081 INFO:tasks.workunit.client.0.smithi102.stdout:5/257: symlink de/d25/d2a/d4b/d4d/l6a 0 2024-05-13T22:47:58.083 INFO:tasks.workunit.client.0.smithi102.stdout:3/249: rmdir de 39 2024-05-13T22:47:58.083 INFO:tasks.workunit.client.0.smithi102.stdout:0/235: dwrite d1/de/d32/f18 [4194304,4194304] 0 2024-05-13T22:47:58.085 INFO:tasks.workunit.client.0.smithi102.stdout:6/214: dread d9/f21 [0,4194304] 0 2024-05-13T22:47:58.086 INFO:tasks.workunit.client.0.smithi102.stdout:6/215: fdatasync d9/db/d14/f19 0 2024-05-13T22:47:58.086 INFO:tasks.workunit.client.0.smithi102.stdout:6/216: chown d9/db/d2f/f41 170843 1 2024-05-13T22:47:58.089 INFO:tasks.workunit.client.0.smithi102.stdout:2/182: link d7/d9/f29 d7/d9/d14/f36 0 2024-05-13T22:47:58.092 INFO:tasks.workunit.client.0.smithi102.stdout:7/123: rename d6/d13/de/d19 to d6/d15/d21 0 2024-05-13T22:47:58.093 INFO:tasks.workunit.client.0.smithi102.stdout:9/194: chown d1/c22 941 1 2024-05-13T22:47:58.093 INFO:tasks.workunit.client.0.smithi102.stdout:7/124: read d6/d13/de/f12 [519578,60774] 0 2024-05-13T22:47:58.102 INFO:tasks.workunit.client.0.smithi102.stdout:5/258: rmdir de/d25 39 2024-05-13T22:47:58.102 INFO:tasks.workunit.client.0.smithi102.stdout:0/236: write d1/de/f1d [4962543,112191] 0 2024-05-13T22:47:58.103 INFO:tasks.workunit.client.0.smithi102.stdout:5/259: fsync de/f21 0 2024-05-13T22:47:58.104 INFO:tasks.workunit.client.0.smithi102.stdout:6/217: mkdir d9/db/d14/d35/d47 0 2024-05-13T22:47:58.105 INFO:tasks.workunit.client.0.smithi102.stdout:4/185: truncate d3/d19/d23/f37 4039404 0 2024-05-13T22:47:58.109 INFO:tasks.workunit.client.0.smithi102.stdout:8/107: rename d2/d7/c15 to d2/d17/c20 0 2024-05-13T22:47:58.110 INFO:tasks.workunit.client.0.smithi102.stdout:4/186: dwrite d3/f10 [0,4194304] 0 2024-05-13T22:47:58.115 INFO:tasks.workunit.client.0.smithi102.stdout:1/153: getdents d2/d4/d5 0 2024-05-13T22:47:58.117 INFO:tasks.workunit.client.0.smithi102.stdout:1/154: write d2/d4/d5/d19/d28/f24 [4365341,79849] 0 2024-05-13T22:47:58.117 INFO:tasks.workunit.client.0.smithi102.stdout:9/195: chown d1/f31 5066956 1 2024-05-13T22:47:58.117 INFO:tasks.workunit.client.0.smithi102.stdout:1/155: dread - d2/d10/f31 zero size 2024-05-13T22:47:58.119 INFO:tasks.workunit.client.0.smithi102.stdout:1/156: dread d2/d4/f23 [0,4194304] 0 2024-05-13T22:47:58.122 INFO:tasks.workunit.client.0.smithi102.stdout:3/250: mkdir de/d10/d2a/d33/d55 0 2024-05-13T22:47:58.125 INFO:tasks.workunit.client.0.smithi102.stdout:5/260: readlink lc 0 2024-05-13T22:47:58.126 INFO:tasks.workunit.client.0.smithi102.stdout:1/157: dwrite d2/d10/f21 [0,4194304] 0 2024-05-13T22:47:58.136 INFO:tasks.workunit.client.0.smithi102.stdout:9/196: creat d1/d6/f4e x:0 0 0 2024-05-13T22:47:58.138 INFO:tasks.workunit.client.0.smithi102.stdout:7/125: mkdir d6/d22 0 2024-05-13T22:47:58.139 INFO:tasks.workunit.client.0.smithi102.stdout:7/126: chown l2 265229778 1 2024-05-13T22:47:58.144 INFO:tasks.workunit.client.0.smithi102.stdout:9/197: dwrite d1/d6/d45/d2d/f3f [0,4194304] 0 2024-05-13T22:47:58.144 INFO:tasks.workunit.client.0.smithi102.stdout:1/158: readlink d2/l22 0 2024-05-13T22:47:58.149 INFO:tasks.workunit.client.0.smithi102.stdout:1/159: read d2/f2b [541095,52917] 0 2024-05-13T22:47:58.151 INFO:tasks.workunit.client.0.smithi102.stdout:1/160: write d2/d10/f31 [51169,112348] 0 2024-05-13T22:47:58.153 INFO:tasks.workunit.client.0.smithi102.stdout:5/261: dwrite de/d25/d59/d28/f66 [0,4194304] 0 2024-05-13T22:47:58.153 INFO:tasks.workunit.client.0.smithi102.stdout:1/161: chown d2/c1c 15 1 2024-05-13T22:47:58.154 INFO:tasks.workunit.client.0.smithi102.stdout:4/187: mknod d3/d4/d6/c48 0 2024-05-13T22:47:58.155 INFO:tasks.workunit.client.0.smithi102.stdout:4/188: chown d3/d19/l3c 771417 1 2024-05-13T22:47:58.156 INFO:tasks.workunit.client.0.smithi102.stdout:5/262: truncate de/d25/d59/d28/f55 322066 0 2024-05-13T22:47:58.156 INFO:tasks.workunit.client.0.smithi102.stdout:4/189: dread - d3/d4/d6/de/f46 zero size 2024-05-13T22:47:58.164 INFO:tasks.workunit.client.0.smithi102.stdout:2/183: getdents d7/d9/d14 0 2024-05-13T22:47:58.164 INFO:tasks.workunit.client.0.smithi102.stdout:2/184: chown d7/d9/d10/d30 38 1 2024-05-13T22:47:58.170 INFO:tasks.workunit.client.0.smithi102.stdout:0/237: rename d1/f1b to d1/de/d32/f3f 0 2024-05-13T22:47:58.172 INFO:tasks.workunit.client.0.smithi102.stdout:2/185: dwrite d7/d9/d14/f34 [0,4194304] 0 2024-05-13T22:47:58.175 INFO:tasks.workunit.client.0.smithi102.stdout:2/186: truncate d7/d9/d14/f36 890931 0 2024-05-13T22:47:58.179 INFO:tasks.workunit.client.0.smithi102.stdout:1/162: mknod d2/d10/c32 0 2024-05-13T22:47:58.188 INFO:tasks.workunit.client.0.smithi102.stdout:7/127: creat d6/d22/f23 x:0 0 0 2024-05-13T22:47:58.188 INFO:tasks.workunit.client.0.smithi102.stdout:1/163: dwrite d2/d4/d5/d19/d28/f24 [0,4194304] 0 2024-05-13T22:47:58.189 INFO:tasks.workunit.client.0.smithi102.stdout:7/128: stat d6/d13/f16 0 2024-05-13T22:47:58.190 INFO:tasks.workunit.client.0.smithi102.stdout:5/263: dwrite fd [4194304,4194304] 0 2024-05-13T22:47:58.190 INFO:tasks.workunit.client.0.smithi102.stdout:1/164: chown d2/d10/c32 3 1 2024-05-13T22:47:58.193 INFO:tasks.workunit.client.0.smithi102.stdout:3/251: truncate de/d10/f11 2269122 0 2024-05-13T22:47:58.199 INFO:tasks.workunit.client.0.smithi102.stdout:6/218: rename d9/d25/d28/f2a to d9/db/d14/d37/f48 0 2024-05-13T22:47:58.205 INFO:tasks.workunit.client.0.smithi102.stdout:8/108: getdents d2/d17 0 2024-05-13T22:47:58.214 INFO:tasks.workunit.client.0.smithi102.stdout:1/165: creat d2/d4/d26/f33 x:0 0 0 2024-05-13T22:47:58.216 INFO:tasks.workunit.client.0.smithi102.stdout:4/190: dwrite d3/d4/d6/de/f15 [0,4194304] 0 2024-05-13T22:47:58.219 INFO:tasks.workunit.client.0.smithi102.stdout:7/129: creat d6/d22/f24 x:0 0 0 2024-05-13T22:47:58.223 INFO:tasks.workunit.client.0.smithi102.stdout:5/264: rmdir de/d25/d58 39 2024-05-13T22:47:58.235 INFO:tasks.workunit.client.0.smithi102.stdout:3/252: dread de/d10/d2a/f2e [0,4194304] 0 2024-05-13T22:47:58.244 INFO:tasks.workunit.client.0.smithi102.stdout:5/265: symlink de/d25/d2c/d3e/l6b 0 2024-05-13T22:47:58.245 INFO:tasks.workunit.client.0.smithi102.stdout:5/266: write de/f35 [1006643,66023] 0 2024-05-13T22:47:58.250 INFO:tasks.workunit.client.0.smithi102.stdout:0/238: link d1/de/d32/f18 d1/de/d32/d24/f40 0 2024-05-13T22:47:58.253 INFO:tasks.workunit.client.0.smithi102.stdout:8/109: fdatasync d2/fa 0 2024-05-13T22:47:58.255 INFO:tasks.workunit.client.0.smithi102.stdout:3/253: creat de/d10/d2a/d48/f56 x:0 0 0 2024-05-13T22:47:58.257 INFO:tasks.workunit.client.0.smithi102.stdout:0/239: dwrite d1/f4 [0,4194304] 0 2024-05-13T22:47:58.260 INFO:tasks.workunit.client.0.smithi102.stdout:4/191: mknod d3/d19/d23/d39/c49 0 2024-05-13T22:47:58.265 INFO:tasks.workunit.client.0.smithi102.stdout:7/130: truncate d6/d13/de/fb 1592482 0 2024-05-13T22:47:58.268 INFO:tasks.workunit.client.0.smithi102.stdout:9/198: rename d1/d6/d45/d2d/f4b to d1/d6/d45/f4f 0 2024-05-13T22:47:58.269 INFO:tasks.workunit.client.0.smithi102.stdout:9/199: truncate d1/d6/d45/d3e/f4c 1907881 0 2024-05-13T22:47:58.271 INFO:tasks.workunit.client.0.smithi102.stdout:9/200: chown d1/d6/db/c3d 1635 1 2024-05-13T22:47:58.273 INFO:tasks.workunit.client.0.smithi102.stdout:8/110: dread d2/d7/f9 [0,4194304] 0 2024-05-13T22:47:58.274 INFO:tasks.workunit.client.0.smithi102.stdout:8/111: rename d2/d7/d12 to d2/d7/d12/d21 22 2024-05-13T22:47:58.277 INFO:tasks.workunit.client.0.smithi102.stdout:9/201: dwrite d1/d6/f1d [0,4194304] 0 2024-05-13T22:47:58.285 INFO:tasks.workunit.client.0.smithi102.stdout:2/187: getdents d7/d9/d11 0 2024-05-13T22:47:58.291 INFO:tasks.workunit.client.0.smithi102.stdout:1/166: link d2/d4/fd d2/f34 0 2024-05-13T22:47:58.299 INFO:tasks.workunit.client.0.smithi102.stdout:7/131: write d6/f8 [3579028,41516] 0 2024-05-13T22:47:58.301 INFO:tasks.workunit.client.0.smithi102.stdout:5/267: mkdir de/d25/d2a/d4b/d4d/d50/d6c 0 2024-05-13T22:47:58.304 INFO:tasks.workunit.client.0.smithi102.stdout:6/219: getdents d9/db 0 2024-05-13T22:47:58.307 INFO:tasks.workunit.client.0.smithi102.stdout:9/202: unlink d1/d6/d45/d3e/f43 0 2024-05-13T22:47:58.314 INFO:tasks.workunit.client.0.smithi102.stdout:3/254: truncate de/d10/f12 875476 0 2024-05-13T22:47:58.315 INFO:tasks.workunit.client.0.smithi102.stdout:3/255: chown l3 6883609 1 2024-05-13T22:47:58.315 INFO:tasks.workunit.client.0.smithi102.stdout:3/256: stat de/d10/d2a/f54 0 2024-05-13T22:47:58.317 INFO:tasks.workunit.client.0.smithi102.stdout:1/167: creat d2/d4/d5/d19/d28/f35 x:0 0 0 2024-05-13T22:47:58.320 INFO:tasks.workunit.client.0.smithi102.stdout:0/240: link d1/de/f19 d1/de/d32/d24/d2d/d3e/f41 0 2024-05-13T22:47:58.322 INFO:tasks.workunit.client.0.smithi102.stdout:4/192: creat d3/d19/d23/d3a/f4a x:0 0 0 2024-05-13T22:47:58.326 INFO:tasks.workunit.client.0.smithi102.stdout:5/268: mknod de/d25/d2a/d4b/d4d/d50/c6d 0 2024-05-13T22:47:58.329 INFO:tasks.workunit.client.0.smithi102.stdout:6/220: rmdir d9/db/d14 39 2024-05-13T22:47:58.333 INFO:tasks.workunit.client.0.smithi102.stdout:2/188: dwrite d7/d9/d14/f36 [0,4194304] 0 2024-05-13T22:47:58.335 INFO:tasks.workunit.client.0.smithi102.stdout:6/221: dread d9/ff [0,4194304] 0 2024-05-13T22:47:58.341 INFO:tasks.workunit.client.0.smithi102.stdout:5/269: dread fd [0,4194304] 0 2024-05-13T22:47:58.343 INFO:tasks.workunit.client.0.smithi102.stdout:3/257: readlink de/d10/l42 0 2024-05-13T22:47:58.345 INFO:tasks.workunit.client.0.smithi102.stdout:1/168: creat d2/d4/d5/f36 x:0 0 0 2024-05-13T22:47:58.349 INFO:tasks.workunit.client.0.smithi102.stdout:5/270: dwrite de/d25/d2a/f2b [0,4194304] 0 2024-05-13T22:47:58.350 INFO:tasks.workunit.client.0.smithi102.stdout:4/193: mkdir d3/d19/d23/d4b 0 2024-05-13T22:47:58.352 INFO:tasks.workunit.client.0.smithi102.stdout:1/169: dread d2/d10/f21 [0,4194304] 0 2024-05-13T22:47:58.353 INFO:tasks.workunit.client.0.smithi102.stdout:7/132: symlink d6/l25 0 2024-05-13T22:47:58.358 INFO:tasks.workunit.client.0.smithi102.stdout:2/189: dread d7/d9/f13 [0,4194304] 0 2024-05-13T22:47:58.359 INFO:tasks.workunit.client.0.smithi102.stdout:8/112: creat d2/f22 x:0 0 0 2024-05-13T22:47:58.366 INFO:tasks.workunit.client.0.smithi102.stdout:3/258: unlink de/f19 0 2024-05-13T22:47:58.372 INFO:tasks.workunit.client.0.smithi102.stdout:7/133: dread d6/d15/f18 [4194304,4194304] 0 2024-05-13T22:47:58.372 INFO:tasks.workunit.client.0.smithi102.stdout:0/241: mkdir d1/de/d42 0 2024-05-13T22:47:58.373 INFO:tasks.workunit.client.0.smithi102.stdout:0/242: chown d1/de/d32/d24/d2d 177390632 1 2024-05-13T22:47:58.384 INFO:tasks.workunit.client.0.smithi102.stdout:8/113: creat d2/d7/d12/f23 x:0 0 0 2024-05-13T22:47:58.388 INFO:tasks.workunit.client.0.smithi102.stdout:6/222: chown d9/db/d14/c1b 1367150 1 2024-05-13T22:47:58.391 INFO:tasks.workunit.client.0.smithi102.stdout:9/203: creat d1/f50 x:0 0 0 2024-05-13T22:47:58.392 INFO:tasks.workunit.client.0.smithi102.stdout:9/204: readlink d1/d6/db/l20 0 2024-05-13T22:47:58.395 INFO:tasks.workunit.client.0.smithi102.stdout:7/134: creat d6/d15/d21/f26 x:0 0 0 2024-05-13T22:47:58.397 INFO:tasks.workunit.client.0.smithi102.stdout:0/243: rmdir d1/de 39 2024-05-13T22:47:58.399 INFO:tasks.workunit.client.0.smithi102.stdout:5/271: link f0 de/d25/d2a/d4b/f6e 0 2024-05-13T22:47:58.400 INFO:tasks.workunit.client.0.smithi102.stdout:5/272: readlink de/d25/d59/d28/l29 0 2024-05-13T22:47:58.431 INFO:tasks.workunit.client.0.smithi102.stdout:2/190: link d7/d9/d14/f36 d7/d9/d10/f37 0 2024-05-13T22:47:58.434 INFO:tasks.workunit.client.0.smithi102.stdout:6/223: write d9/db/d14/f23 [2850452,5883] 0 2024-05-13T22:47:58.437 INFO:tasks.workunit.client.0.smithi102.stdout:9/205: rmdir d1/d6/d3b 39 2024-05-13T22:47:58.438 INFO:tasks.workunit.client.0.smithi102.stdout:9/206: fdatasync d1/d6/d45/d2d/f3f 0 2024-05-13T22:47:58.439 INFO:tasks.workunit.client.0.smithi102.stdout:9/207: chown d1/d6/db/l20 15 1 2024-05-13T22:47:58.442 INFO:tasks.workunit.client.0.smithi102.stdout:4/194: truncate d3/d19/d23/f3d 1158362 0 2024-05-13T22:47:58.444 INFO:tasks.workunit.client.0.smithi102.stdout:1/170: creat d2/d4/f37 x:0 0 0 2024-05-13T22:47:58.447 INFO:tasks.workunit.client.0.smithi102.stdout:8/114: link d2/f4 d2/d7/f24 0 2024-05-13T22:47:58.448 INFO:tasks.workunit.client.0.smithi102.stdout:6/224: rmdir d9/d45 39 2024-05-13T22:47:58.449 INFO:tasks.workunit.client.0.smithi102.stdout:5/273: dwrite de/f14 [0,4194304] 0 2024-05-13T22:47:58.450 INFO:tasks.workunit.client.0.smithi102.stdout:3/259: symlink de/d10/d2a/d48/l57 0 2024-05-13T22:47:58.451 INFO:tasks.workunit.client.0.smithi102.stdout:4/195: dwrite d3/f10 [0,4194304] 0 2024-05-13T22:47:58.452 INFO:tasks.workunit.client.0.smithi102.stdout:4/196: readlink d3/d19/l40 0 2024-05-13T22:47:58.453 INFO:tasks.workunit.client.0.smithi102.stdout:4/197: chown d3/d4/d6/de/f34 0 1 2024-05-13T22:47:58.453 INFO:tasks.workunit.client.0.smithi102.stdout:9/208: rmdir d1/d6/d45 39 2024-05-13T22:47:58.463 INFO:tasks.workunit.client.0.smithi102.stdout:8/115: mkdir d2/d7/d25 0 2024-05-13T22:47:58.463 INFO:tasks.workunit.client.0.smithi102.stdout:5/274: dwrite de/d25/d59/f5d [0,4194304] 0 2024-05-13T22:47:58.469 INFO:tasks.workunit.client.0.smithi102.stdout:9/209: dwrite d1/d6/db/d30/d47/f49 [0,4194304] 0 2024-05-13T22:47:58.471 INFO:tasks.workunit.client.0.smithi102.stdout:4/198: dwrite d3/d4/d6/de/f34 [0,4194304] 0 2024-05-13T22:47:58.472 INFO:tasks.workunit.client.0.smithi102.stdout:0/244: symlink d1/de/d32/l43 0 2024-05-13T22:47:58.474 INFO:tasks.workunit.client.0.smithi102.stdout:4/199: chown d3/d4/d6/de 883287 1 2024-05-13T22:47:58.475 INFO:tasks.workunit.client.0.smithi102.stdout:9/210: write d1/d6/f11 [149212,8538] 0 2024-05-13T22:47:58.476 INFO:tasks.workunit.client.0.smithi102.stdout:0/245: chown d1/de/f33 49 1 2024-05-13T22:47:58.477 INFO:tasks.workunit.client.0.smithi102.stdout:8/116: dwrite d2/d7/fb [0,4194304] 0 2024-05-13T22:47:58.478 INFO:tasks.workunit.client.0.smithi102.stdout:4/200: dread - d3/d19/d23/d3a/f4a zero size 2024-05-13T22:47:58.479 INFO:tasks.workunit.client.0.smithi102.stdout:0/246: chown d1/de/f19 468 1 2024-05-13T22:47:58.480 INFO:tasks.workunit.client.0.smithi102.stdout:4/201: dread - d3/d4/d6/f1c zero size 2024-05-13T22:47:58.481 INFO:tasks.workunit.client.0.smithi102.stdout:4/202: dread - d3/d4/f47 zero size 2024-05-13T22:47:58.488 INFO:tasks.workunit.client.0.smithi102.stdout:8/117: creat d2/d17/f26 x:0 0 0 2024-05-13T22:47:58.490 INFO:tasks.workunit.client.0.smithi102.stdout:9/211: write d1/d6/db/f1c [869668,34340] 0 2024-05-13T22:47:58.498 INFO:tasks.workunit.client.0.smithi102.stdout:4/203: symlink d3/d19/l4c 0 2024-05-13T22:47:58.500 INFO:tasks.workunit.client.0.smithi102.stdout:1/171: getdents d2/d4/d5/d19 0 2024-05-13T22:47:58.501 INFO:tasks.workunit.client.0.smithi102.stdout:6/225: link d9/fa d9/db/f49 0 2024-05-13T22:47:58.503 INFO:tasks.workunit.client.0.smithi102.stdout:8/118: fsync d2/fa 0 2024-05-13T22:47:58.504 INFO:tasks.workunit.client.0.smithi102.stdout:3/260: rmdir de/d10/d2a 39 2024-05-13T22:47:58.505 INFO:tasks.workunit.client.0.smithi102.stdout:9/212: rmdir d1/d6/db/d30 39 2024-05-13T22:47:58.506 INFO:tasks.workunit.client.0.smithi102.stdout:9/213: chown d1/d6/d21/c46 13022 1 2024-05-13T22:47:58.507 INFO:tasks.workunit.client.0.smithi102.stdout:9/214: write d1/d6/f11 [1166260,8669] 0 2024-05-13T22:47:58.508 INFO:tasks.workunit.client.0.smithi102.stdout:5/275: rename de/d25/d2c/d3e to de/d25/d59/d6f 0 2024-05-13T22:47:58.510 INFO:tasks.workunit.client.0.smithi102.stdout:0/247: symlink d1/de/d32/d24/d2d/d3e/l44 0 2024-05-13T22:47:58.511 INFO:tasks.workunit.client.0.smithi102.stdout:2/191: truncate d7/d9/d14/f34 2029213 0 2024-05-13T22:47:58.513 INFO:tasks.workunit.client.0.smithi102.stdout:4/204: creat d3/d19/d23/d3a/f4d x:0 0 0 2024-05-13T22:47:58.516 INFO:tasks.workunit.client.0.smithi102.stdout:2/192: dwrite d7/d15/f19 [0,4194304] 0 2024-05-13T22:47:58.526 INFO:tasks.workunit.client.0.smithi102.stdout:4/205: mkdir d3/d19/d23/d4e 0 2024-05-13T22:47:58.530 INFO:tasks.workunit.client.0.smithi102.stdout:2/193: creat d7/d9/d11/f38 x:0 0 0 2024-05-13T22:47:58.532 INFO:tasks.workunit.client.0.smithi102.stdout:8/119: getdents d2/d7/d25 0 2024-05-13T22:47:58.533 INFO:tasks.workunit.client.0.smithi102.stdout:3/261: truncate f5 1524260 0 2024-05-13T22:47:58.533 INFO:tasks.workunit.client.0.smithi102.stdout:3/262: chown de/d10/c15 865404498 1 2024-05-13T22:47:58.541 INFO:tasks.workunit.client.0.smithi102.stdout:2/194: fsync d7/d9/d10/f37 0 2024-05-13T22:47:58.543 INFO:tasks.workunit.client.0.smithi102.stdout:4/206: dwrite d3/d4/f47 [0,4194304] 0 2024-05-13T22:47:58.545 INFO:tasks.workunit.client.0.smithi102.stdout:8/120: unlink d2/fa 0 2024-05-13T22:47:58.554 INFO:tasks.workunit.client.0.smithi102.stdout:1/172: getdents d2/d4/d26 0 2024-05-13T22:47:58.554 INFO:tasks.workunit.client.0.smithi102.stdout:6/226: link d9/db/c24 d9/db/d14/c4a 0 2024-05-13T22:47:58.559 INFO:tasks.workunit.client.0.smithi102.stdout:4/207: unlink d3/d19/c2f 0 2024-05-13T22:47:58.560 INFO:tasks.workunit.client.0.smithi102.stdout:4/208: chown d3/f35 7202626 1 2024-05-13T22:47:58.561 INFO:tasks.workunit.client.0.smithi102.stdout:8/121: mkdir d2/d7/d27 0 2024-05-13T22:47:58.563 INFO:tasks.workunit.client.0.smithi102.stdout:5/276: getdents de/d25/d2a/d4b 0 2024-05-13T22:47:58.565 INFO:tasks.workunit.client.0.smithi102.stdout:3/263: creat de/d10/d2a/d36/f58 x:0 0 0 2024-05-13T22:47:58.565 INFO:tasks.workunit.client.0.smithi102.stdout:6/227: rmdir d9/db/d14/d35 39 2024-05-13T22:47:58.566 INFO:tasks.workunit.client.0.smithi102.stdout:4/209: dwrite d3/d4/d6/de/f3f [0,4194304] 0 2024-05-13T22:47:58.567 INFO:tasks.workunit.client.0.smithi102.stdout:3/264: dread - de/d10/d50/f47 zero size 2024-05-13T22:47:58.570 INFO:tasks.workunit.client.0.smithi102.stdout:5/277: dread f0 [0,4194304] 0 2024-05-13T22:47:58.571 INFO:tasks.workunit.client.0.smithi102.stdout:2/195: link d7/d9/fb d7/d9/d11/f39 0 2024-05-13T22:47:58.574 INFO:tasks.workunit.client.0.smithi102.stdout:2/196: write d7/d15/f19 [4847677,17662] 0 2024-05-13T22:47:58.575 INFO:tasks.workunit.client.0.smithi102.stdout:0/248: write d1/de/d32/f1e [31649,55719] 0 2024-05-13T22:47:58.578 INFO:tasks.workunit.client.0.smithi102.stdout:4/210: mknod d3/d19/c4f 0 2024-05-13T22:47:58.580 INFO:tasks.workunit.client.0.smithi102.stdout:3/265: dwrite de/d10/d2a/d33/f40 [0,4194304] 0 2024-05-13T22:47:58.584 INFO:tasks.workunit.client.0.smithi102.stdout:5/278: dwrite de/d25/d59/f44 [4194304,4194304] 0 2024-05-13T22:47:58.591 INFO:tasks.workunit.client.0.smithi102.stdout:2/197: write d7/f32 [899514,89630] 0 2024-05-13T22:47:58.595 INFO:tasks.workunit.client.0.smithi102.stdout:6/228: truncate d9/db/d14/f19 8160444 0 2024-05-13T22:47:58.596 INFO:tasks.workunit.client.0.smithi102.stdout:0/249: mknod d1/de/d32/d24/d2d/c45 0 2024-05-13T22:47:58.600 INFO:tasks.workunit.client.0.smithi102.stdout:4/211: rename d3/d4/d6/de/f34 to d3/d4/f50 0 2024-05-13T22:47:58.600 INFO:tasks.workunit.client.0.smithi102.stdout:4/212: stat d3/d19/d23/d39/c49 0 2024-05-13T22:47:58.602 INFO:tasks.workunit.client.0.smithi102.stdout:4/213: write d3/d4/f32 [681398,5178] 0 2024-05-13T22:47:58.604 INFO:tasks.workunit.client.0.smithi102.stdout:5/279: write de/d25/d59/f18 [723881,83698] 0 2024-05-13T22:47:58.606 INFO:tasks.workunit.client.0.smithi102.stdout:6/229: readlink d9/d45/l31 0 2024-05-13T22:47:58.608 INFO:tasks.workunit.client.0.smithi102.stdout:3/266: mknod de/d10/d2a/d4b/c59 0 2024-05-13T22:47:58.609 INFO:tasks.workunit.client.0.smithi102.stdout:3/267: chown de/d10/d2a/d36 210728266 1 2024-05-13T22:47:58.610 INFO:tasks.workunit.client.0.smithi102.stdout:3/268: fsync de/d10/d2a/d33/f3f 0 2024-05-13T22:47:58.612 INFO:tasks.workunit.client.0.smithi102.stdout:8/122: getdents d2/d17 0 2024-05-13T22:47:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:4/214: rmdir d3/d22/d2b 39 2024-05-13T22:47:58.613 INFO:tasks.workunit.client.0.smithi102.stdout:4/215: stat d3/d4/d6/f42 0 2024-05-13T22:47:58.617 INFO:tasks.workunit.client.0.smithi102.stdout:5/280: creat de/d25/d2a/d4b/f70 x:0 0 0 2024-05-13T22:47:58.618 INFO:tasks.workunit.client.0.smithi102.stdout:5/281: write de/f35 [2052967,95033] 0 2024-05-13T22:47:58.619 INFO:tasks.workunit.client.0.smithi102.stdout:0/250: mknod d1/de/d32/c46 0 2024-05-13T22:47:58.622 INFO:tasks.workunit.client.0.smithi102.stdout:2/198: dread d7/d9/fb [0,4194304] 0 2024-05-13T22:47:58.622 INFO:tasks.workunit.client.0.smithi102.stdout:3/269: symlink de/d10/d2a/d48/l5a 0 2024-05-13T22:47:58.625 INFO:tasks.workunit.client.0.smithi102.stdout:8/123: mkdir d2/d7/d12/d28 0 2024-05-13T22:47:58.626 INFO:tasks.workunit.client.0.smithi102.stdout:4/216: creat d3/d19/d23/d3a/f51 x:0 0 0 2024-05-13T22:47:58.626 INFO:tasks.workunit.client.0.smithi102.stdout:6/230: dwrite d9/db/f36 [4194304,4194304] 0 2024-05-13T22:47:58.635 INFO:tasks.workunit.client.0.smithi102.stdout:3/270: write de/d1c/f4f [548255,98795] 0 2024-05-13T22:47:58.636 INFO:tasks.workunit.client.0.smithi102.stdout:2/199: symlink d7/d9/d14/l3a 0 2024-05-13T22:47:58.640 INFO:tasks.workunit.client.0.smithi102.stdout:8/124: rename d2/l16 to d2/d7/d12/l29 0 2024-05-13T22:47:58.641 INFO:tasks.workunit.client.0.smithi102.stdout:4/217: write d3/d4/f2e [1703119,5656] 0 2024-05-13T22:47:58.643 INFO:tasks.workunit.client.0.smithi102.stdout:6/231: dread d9/ff [0,4194304] 0 2024-05-13T22:47:58.645 INFO:tasks.workunit.client.0.smithi102.stdout:5/282: unlink de/d25/d58/l64 0 2024-05-13T22:47:58.646 INFO:tasks.workunit.client.0.smithi102.stdout:5/283: chown de/d33/c48 1484661592 1 2024-05-13T22:47:58.646 INFO:tasks.workunit.client.0.smithi102.stdout:0/251: creat d1/de/d42/f47 x:0 0 0 2024-05-13T22:47:58.647 INFO:tasks.workunit.client.0.smithi102.stdout:5/284: dread - de/d25/d59/d28/f5c zero size 2024-05-13T22:47:58.648 INFO:tasks.workunit.client.0.smithi102.stdout:3/271: readlink de/l13 0 2024-05-13T22:47:58.648 INFO:tasks.workunit.client.0.smithi102.stdout:3/272: readlink de/l53 0 2024-05-13T22:47:58.652 INFO:tasks.workunit.client.0.smithi102.stdout:0/252: fsync d1/de/d32/d24/f40 0 2024-05-13T22:47:58.653 INFO:tasks.workunit.client.0.smithi102.stdout:0/253: read d1/f4 [2808411,115305] 0 2024-05-13T22:47:58.654 INFO:tasks.workunit.client.0.smithi102.stdout:8/125: mkdir d2/d7/d12/d28/d2a 0 2024-05-13T22:47:58.657 INFO:tasks.workunit.client.0.smithi102.stdout:2/200: rename d7/d9/d14/f34 to d7/d9/d10/d25/d27/f3b 0 2024-05-13T22:47:58.660 INFO:tasks.workunit.client.0.smithi102.stdout:3/273: write de/d21/f2b [907069,105564] 0 2024-05-13T22:47:58.662 INFO:tasks.workunit.client.0.smithi102.stdout:8/126: dwrite d2/f19 [0,4194304] 0 2024-05-13T22:47:58.662 INFO:tasks.workunit.client.0.smithi102.stdout:6/232: link f3 d9/db/d14/d37/f4b 0 2024-05-13T22:47:58.663 INFO:tasks.workunit.client.0.smithi102.stdout:6/233: stat d9/d25/c27 0 2024-05-13T22:47:58.663 INFO:tasks.workunit.client.0.smithi102.stdout:8/127: chown d2/d7/f1d 1651 1 2024-05-13T22:47:58.671 INFO:tasks.workunit.client.0.smithi102.stdout:5/285: rename de/d25/d43/f5e to de/d25/d2c/f71 0 2024-05-13T22:47:58.672 INFO:tasks.workunit.client.0.smithi102.stdout:8/128: dread d2/f4 [8388608,4194304] 0 2024-05-13T22:47:58.676 INFO:tasks.workunit.client.0.smithi102.stdout:2/201: symlink d7/d9/d14/l3c 0 2024-05-13T22:47:58.678 INFO:tasks.workunit.client.0.smithi102.stdout:3/274: write de/d10/d2a/f2e [4486797,111305] 0 2024-05-13T22:47:58.680 INFO:tasks.workunit.client.0.smithi102.stdout:6/234: symlink d9/d45/l4c 0 2024-05-13T22:47:58.686 INFO:tasks.workunit.client.0.smithi102.stdout:5/286: creat de/d25/d59/d6f/f72 x:0 0 0 2024-05-13T22:47:58.688 INFO:tasks.workunit.client.0.smithi102.stdout:0/254: dwrite d1/de/f19 [0,4194304] 0 2024-05-13T22:47:58.689 INFO:tasks.workunit.client.0.smithi102.stdout:8/129: fdatasync d2/d7/f9 0 2024-05-13T22:47:58.690 INFO:tasks.workunit.client.0.smithi102.stdout:0/255: fdatasync d1/de/d32/d24/f31 0 2024-05-13T22:47:58.691 INFO:tasks.workunit.client.0.smithi102.stdout:2/202: mknod d7/d9/d10/d25/c3d 0 2024-05-13T22:47:58.693 INFO:tasks.workunit.client.0.smithi102.stdout:8/130: read d2/d7/d12/f13 [75966,67052] 0 2024-05-13T22:47:58.695 INFO:tasks.workunit.client.0.smithi102.stdout:5/287: dwrite de/f5a [0,4194304] 0 2024-05-13T22:47:58.696 INFO:tasks.workunit.client.0.smithi102.stdout:3/275: symlink de/d10/l5b 0 2024-05-13T22:47:58.697 INFO:tasks.workunit.client.0.smithi102.stdout:5/288: truncate de/d25/f47 691278 0 2024-05-13T22:47:58.702 INFO:tasks.workunit.client.0.smithi102.stdout:8/131: mkdir d2/d7/d12/d2b 0 2024-05-13T22:47:58.703 INFO:tasks.workunit.client.0.smithi102.stdout:8/132: dread - d2/d7/d12/f23 zero size 2024-05-13T22:47:58.703 INFO:tasks.workunit.client.0.smithi102.stdout:8/133: dread - d2/d7/d12/f23 zero size 2024-05-13T22:47:58.704 INFO:tasks.workunit.client.0.smithi102.stdout:0/256: symlink d1/de/d32/l48 0 2024-05-13T22:47:58.706 INFO:tasks.workunit.client.0.smithi102.stdout:5/289: chown de/c20 2955587 1 2024-05-13T22:47:58.711 INFO:tasks.workunit.client.0.smithi102.stdout:6/235: rename d9/d25/l2d to d9/db/d14/d35/d47/l4d 0 2024-05-13T22:47:58.717 INFO:tasks.workunit.client.0.smithi102.stdout:8/134: rename d2/d7/f1d to d2/d17/f2c 0 2024-05-13T22:47:58.717 INFO:tasks.workunit.client.0.smithi102.stdout:3/276: link de/d1c/l23 de/d10/d2a/d33/l5c 0 2024-05-13T22:47:58.719 INFO:tasks.workunit.client.0.smithi102.stdout:2/203: getdents d7/d9/d10/d25 0 2024-05-13T22:47:58.722 INFO:tasks.workunit.client.0.smithi102.stdout:8/135: symlink d2/d7/l2d 0 2024-05-13T22:47:58.723 INFO:tasks.workunit.client.0.smithi102.stdout:0/257: dread d1/de/d32/d24/f3c [0,4194304] 0 2024-05-13T22:47:58.723 INFO:tasks.workunit.client.0.smithi102.stdout:8/136: chown d2/d7/f9 203548 1 2024-05-13T22:47:58.723 INFO:tasks.workunit.client.0.smithi102.stdout:3/277: mkdir de/d10/d34/d5d 0 2024-05-13T22:47:58.724 INFO:tasks.workunit.client.0.smithi102.stdout:0/258: chown d1/de/d32/d24/d2d/c2f 28 1 2024-05-13T22:47:58.727 INFO:tasks.workunit.client.0.smithi102.stdout:3/278: dread de/d1c/f4f [0,4194304] 0 2024-05-13T22:47:58.731 INFO:tasks.workunit.client.0.smithi102.stdout:2/204: getdents d7/d9/d10/d30 0 2024-05-13T22:47:58.734 INFO:tasks.workunit.client.0.smithi102.stdout:5/290: rename de/d25/d2a/f3c to de/d25/f73 0 2024-05-13T22:47:58.735 INFO:tasks.workunit.client.0.smithi102.stdout:3/279: fsync de/d10/d2a/f2f 0 2024-05-13T22:47:58.737 INFO:tasks.workunit.client.0.smithi102.stdout:6/236: write d9/db/f22 [1717351,47473] 0 2024-05-13T22:47:58.738 INFO:tasks.workunit.client.0.smithi102.stdout:8/137: dread d2/f4 [4194304,4194304] 0 2024-05-13T22:47:58.739 INFO:tasks.workunit.client.0.smithi102.stdout:8/138: write d2/d7/d12/f13 [1194666,65119] 0 2024-05-13T22:47:58.740 INFO:tasks.workunit.client.0.smithi102.stdout:2/205: unlink d7/d9/fb 0 2024-05-13T22:47:58.741 INFO:tasks.workunit.client.0.smithi102.stdout:2/206: chown c3 1 1 2024-05-13T22:47:58.744 INFO:tasks.workunit.client.0.smithi102.stdout:5/291: rmdir de/d25/d59/d6f 39 2024-05-13T22:47:58.744 INFO:tasks.workunit.client.0.smithi102.stdout:8/139: dread d2/d7/d12/f1c [0,4194304] 0 2024-05-13T22:47:58.744 INFO:tasks.workunit.client.0.smithi102.stdout:5/292: stat de/f21 0 2024-05-13T22:47:58.747 INFO:tasks.workunit.client.0.smithi102.stdout:3/280: creat de/d10/d2a/d48/f5e x:0 0 0 2024-05-13T22:47:58.750 INFO:tasks.workunit.client.0.smithi102.stdout:2/207: symlink d7/d15/l3e 0 2024-05-13T22:47:58.757 INFO:tasks.workunit.client.0.smithi102.stdout:6/237: dread d9/db/f49 [0,4194304] 0 2024-05-13T22:47:58.758 INFO:tasks.workunit.client.0.smithi102.stdout:6/238: fsync d9/db/d14/d37/f43 0 2024-05-13T22:47:58.798 INFO:tasks.workunit.client.0.smithi102.stdout:3/281: creat de/d10/d34/f5f x:0 0 0 2024-05-13T22:47:58.801 INFO:tasks.workunit.client.0.smithi102.stdout:2/208: mkdir d7/d15/d1e/d3f 0 2024-05-13T22:47:58.802 INFO:tasks.workunit.client.0.smithi102.stdout:6/239: mknod d9/db/d14/d37/c4e 0 2024-05-13T22:47:58.804 INFO:tasks.workunit.client.0.smithi102.stdout:8/140: dwrite d2/f4 [0,4194304] 0 2024-05-13T22:47:58.806 INFO:tasks.workunit.client.0.smithi102.stdout:5/293: symlink de/l74 0 2024-05-13T22:47:58.818 INFO:tasks.workunit.client.0.smithi102.stdout:3/282: creat de/d10/d2a/d4b/f60 x:0 0 0 2024-05-13T22:47:58.818 INFO:tasks.workunit.client.0.smithi102.stdout:8/141: dread d2/f19 [0,4194304] 0 2024-05-13T22:47:58.819 INFO:tasks.workunit.client.0.smithi102.stdout:2/209: rmdir d7/d15/d1e 39 2024-05-13T22:47:58.822 INFO:tasks.workunit.client.0.smithi102.stdout:3/283: write de/d10/d2a/d33/f40 [2489707,115602] 0 2024-05-13T22:47:58.822 INFO:tasks.workunit.client.0.smithi102.stdout:8/142: truncate d2/d7/d12/f23 560631 0 2024-05-13T22:47:58.825 INFO:tasks.workunit.client.0.smithi102.stdout:8/143: write d2/f22 [236186,92566] 0 2024-05-13T22:47:58.827 INFO:tasks.workunit.client.0.smithi102.stdout:5/294: truncate de/f1d 322536 0 2024-05-13T22:47:58.828 INFO:tasks.workunit.client.0.smithi102.stdout:6/240: dwrite d9/db/d2f/f41 [0,4194304] 0 2024-05-13T22:47:58.829 INFO:tasks.workunit.client.0.smithi102.stdout:6/241: read d9/ff [29373,1272] 0 2024-05-13T22:47:58.830 INFO:tasks.workunit.client.0.smithi102.stdout:2/210: stat d7/d9/d11 0 2024-05-13T22:47:58.831 INFO:tasks.workunit.client.0.smithi102.stdout:2/211: chown c3 0 1 2024-05-13T22:47:58.836 INFO:tasks.workunit.client.0.smithi102.stdout:8/144: read d2/f19 [733190,103633] 0 2024-05-13T22:47:58.838 INFO:tasks.workunit.client.0.smithi102.stdout:3/284: dwrite de/d10/f43 [0,4194304] 0 2024-05-13T22:47:58.840 INFO:tasks.workunit.client.0.smithi102.stdout:6/242: mknod d9/db/d14/d37/c4f 0 2024-05-13T22:47:58.841 INFO:tasks.workunit.client.0.smithi102.stdout:2/212: truncate d7/d9/d10/f37 1967738 0 2024-05-13T22:47:58.842 INFO:tasks.workunit.client.0.smithi102.stdout:8/145: dread d2/d7/f9 [0,4194304] 0 2024-05-13T22:47:58.843 INFO:tasks.workunit.client.0.smithi102.stdout:8/146: chown d2/d7/l8 0 1 2024-05-13T22:47:58.844 INFO:tasks.workunit.client.0.smithi102.stdout:5/295: mkdir de/d75 0 2024-05-13T22:47:58.844 INFO:tasks.workunit.client.0.smithi102.stdout:8/147: stat d2/d17/l1a 0 2024-05-13T22:47:58.848 INFO:tasks.workunit.client.0.smithi102.stdout:5/296: chown de/d25/d2c/l60 490020 1 2024-05-13T22:47:58.848 INFO:tasks.workunit.client.0.smithi102.stdout:6/243: dwrite d9/f10 [0,4194304] 0 2024-05-13T22:47:58.849 INFO:tasks.workunit.client.0.smithi102.stdout:8/148: symlink d2/d17/l2e 0 2024-05-13T22:47:58.852 INFO:tasks.workunit.client.0.smithi102.stdout:8/149: dread d2/f22 [0,4194304] 0 2024-05-13T22:47:58.853 INFO:tasks.workunit.client.0.smithi102.stdout:2/213: getdents d7/d9/d10/d25 0 2024-05-13T22:47:58.859 INFO:tasks.workunit.client.0.smithi102.stdout:6/244: getdents d9/d25 0 2024-05-13T22:47:58.860 INFO:tasks.workunit.client.0.smithi102.stdout:5/297: dwrite f2 [0,4194304] 0 2024-05-13T22:47:58.863 INFO:tasks.workunit.client.0.smithi102.stdout:5/298: creat de/d25/d2a/d4b/f76 x:0 0 0 2024-05-13T22:47:58.865 INFO:tasks.workunit.client.0.smithi102.stdout:5/299: truncate de/f61 1037780 0 2024-05-13T22:47:58.865 INFO:tasks.workunit.client.0.smithi102.stdout:6/245: rename d9/ff to d9/db/f50 0 2024-05-13T22:47:58.873 INFO:tasks.workunit.client.0.smithi102.stdout:5/300: link de/d25/l38 de/d75/l77 0 2024-05-13T22:47:58.875 INFO:tasks.workunit.client.0.smithi102.stdout:6/246: rename d9/db/c24 to d9/db/d14/d35/d47/c51 0 2024-05-13T22:47:58.876 INFO:tasks.workunit.client.0.smithi102.stdout:5/301: creat de/d25/d2c/f78 x:0 0 0 2024-05-13T22:47:58.877 INFO:tasks.workunit.client.0.smithi102.stdout:5/302: chown de/d33/d34/f4a 458 1 2024-05-13T22:47:58.879 INFO:tasks.workunit.client.0.smithi102.stdout:5/303: mknod de/d25/d2c/c79 0 2024-05-13T22:47:58.881 INFO:tasks.workunit.client.0.smithi102.stdout:5/304: rmdir de/d25 39 2024-05-13T22:47:58.881 INFO:tasks.workunit.client.0.smithi102.stdout:6/247: dread d9/f21 [0,4194304] 0 2024-05-13T22:47:58.885 INFO:tasks.workunit.client.0.smithi102.stdout:6/248: creat d9/db/d14/d35/f52 x:0 0 0 2024-05-13T22:47:58.885 INFO:tasks.workunit.client.0.smithi102.stdout:2/214: write d7/d9/d10/f37 [2720189,32230] 0 2024-05-13T22:47:58.886 INFO:tasks.workunit.client.0.smithi102.stdout:5/305: write de/d25/f73 [212037,111292] 0 2024-05-13T22:47:58.887 INFO:tasks.workunit.client.0.smithi102.stdout:2/215: fsync d7/d15/f19 0 2024-05-13T22:47:58.888 INFO:tasks.workunit.client.0.smithi102.stdout:2/216: rename d7/d9/d10/d25 to d7/d9/d10/d25/d40 22 2024-05-13T22:47:58.890 INFO:tasks.workunit.client.0.smithi102.stdout:5/306: mkdir de/d25/d43/d7a 0 2024-05-13T22:47:58.891 INFO:tasks.workunit.client.0.smithi102.stdout:5/307: fdatasync de/d25/d2a/d4b/f70 0 2024-05-13T22:47:58.893 INFO:tasks.workunit.client.0.smithi102.stdout:2/217: symlink d7/d9/d10/d25/l41 0 2024-05-13T22:47:58.894 INFO:tasks.workunit.client.0.smithi102.stdout:5/308: creat de/d33/f7b x:0 0 0 2024-05-13T22:47:58.896 INFO:tasks.workunit.client.0.smithi102.stdout:6/249: creat d9/f53 x:0 0 0 2024-05-13T22:47:58.898 INFO:tasks.workunit.client.0.smithi102.stdout:2/218: mknod d7/c42 0 2024-05-13T22:47:58.899 INFO:tasks.workunit.client.0.smithi102.stdout:5/309: dread de/d25/d2a/d4b/f6e [0,4194304] 0 2024-05-13T22:47:58.900 INFO:tasks.workunit.client.0.smithi102.stdout:6/250: dread d9/db/f50 [0,4194304] 0 2024-05-13T22:47:58.901 INFO:tasks.workunit.client.0.smithi102.stdout:2/219: truncate d7/d9/d10/d25/f35 293969 0 2024-05-13T22:47:58.904 INFO:tasks.workunit.client.0.smithi102.stdout:2/220: symlink d7/d9/d10/d25/l43 0 2024-05-13T22:47:58.905 INFO:tasks.workunit.client.0.smithi102.stdout:5/310: mknod de/d25/d2a/d4b/d4d/d50/d6c/c7c 0 2024-05-13T22:47:58.906 INFO:tasks.workunit.client.0.smithi102.stdout:5/311: chown de/d25/c5b 103056 1 2024-05-13T22:47:58.907 INFO:tasks.workunit.client.0.smithi102.stdout:6/251: dwrite d9/f10 [0,4194304] 0 2024-05-13T22:47:58.909 INFO:tasks.workunit.client.0.smithi102.stdout:5/312: mknod de/d25/d58/c7d 0 2024-05-13T22:47:58.910 INFO:tasks.workunit.client.0.smithi102.stdout:6/252: fsync d9/db/f50 0 2024-05-13T22:47:58.912 INFO:tasks.workunit.client.0.smithi102.stdout:6/253: truncate d9/db/d14/d35/f52 280215 0 2024-05-13T22:47:58.914 INFO:tasks.workunit.client.0.smithi102.stdout:5/313: symlink de/l7e 0 2024-05-13T22:47:58.915 INFO:tasks.workunit.client.0.smithi102.stdout:6/254: unlink d9/db/d14/f1d 0 2024-05-13T22:47:58.919 INFO:tasks.workunit.client.0.smithi102.stdout:6/255: mkdir d9/db/d14/d54 0 2024-05-13T22:47:58.920 INFO:tasks.workunit.client.0.smithi102.stdout:6/256: chown d9/f10 10 1 2024-05-13T22:47:58.920 INFO:tasks.workunit.client.0.smithi102.stdout:5/314: dwrite de/d25/d59/d28/f55 [0,4194304] 0 2024-05-13T22:47:58.926 INFO:tasks.workunit.client.0.smithi102.stdout:5/315: mkdir de/d25/d59/d7f 0 2024-05-13T22:47:58.926 INFO:tasks.workunit.client.0.smithi102.stdout:6/257: dread d9/db/f49 [0,4194304] 0 2024-05-13T22:47:58.926 INFO:tasks.workunit.client.0.smithi102.stdout:5/316: fsync de/f35 0 2024-05-13T22:47:58.926 INFO:tasks.workunit.client.0.smithi102.stdout:6/258: fdatasync d9/db/d14/f3f 0 2024-05-13T22:47:58.926 INFO:tasks.workunit.client.0.smithi102.stdout:6/259: read d9/fa [1147935,42624] 0 2024-05-13T22:47:58.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:47:58 smithi102 ceph-mon[28599]: pgmap v16: 65 pgs: 65 active+clean; 232 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 10 MiB/s wr, 329 op/s 2024-05-13T22:47:58.930 INFO:tasks.workunit.client.0.smithi102.stdout:5/317: dread de/d25/d2c/f5f [0,4194304] 0 2024-05-13T22:47:58.931 INFO:tasks.workunit.client.0.smithi102.stdout:5/318: fdatasync de/d25/d59/f18 0 2024-05-13T22:47:58.935 INFO:tasks.workunit.client.0.smithi102.stdout:8/150: read d2/d7/d12/f13 [797386,121244] 0 2024-05-13T22:47:58.939 INFO:tasks.workunit.client.0.smithi102.stdout:5/319: dread de/d25/d2c/f5f [0,4194304] 0 2024-05-13T22:47:58.944 INFO:tasks.workunit.client.0.smithi102.stdout:2/221: write d7/d9/d10/d25/d27/f3b [99490,109159] 0 2024-05-13T22:47:58.948 INFO:tasks.workunit.client.0.smithi102.stdout:5/320: dwrite de/d25/d2a/f56 [0,4194304] 0 2024-05-13T22:47:58.950 INFO:tasks.workunit.client.0.smithi102.stdout:2/222: dread d7/d9/d10/d25/d27/f3b [0,4194304] 0 2024-05-13T22:47:58.951 INFO:tasks.workunit.client.0.smithi102.stdout:2/223: readlink d7/d9/lc 0 2024-05-13T22:47:58.952 INFO:tasks.workunit.client.0.smithi102.stdout:6/260: dread d9/db/d14/f3f [4194304,4194304] 0 2024-05-13T22:47:58.952 INFO:tasks.workunit.client.0.smithi102.stdout:2/224: rename d7/d9/d10/d30 to d7/d9/d10/d30/d44 22 2024-05-13T22:47:58.960 INFO:tasks.workunit.client.0.smithi102.stdout:2/225: creat d7/d9/d10/d25/d27/f45 x:0 0 0 2024-05-13T22:47:58.961 INFO:tasks.workunit.client.0.smithi102.stdout:6/261: dread d9/db/f36 [4194304,4194304] 0 2024-05-13T22:47:58.962 INFO:tasks.workunit.client.0.smithi102.stdout:5/321: dwrite de/d33/d53/f57 [0,4194304] 0 2024-05-13T22:47:58.962 INFO:tasks.workunit.client.0.smithi102.stdout:8/151: write d2/f22 [632499,60399] 0 2024-05-13T22:47:58.964 INFO:tasks.workunit.client.0.smithi102.stdout:8/152: write d2/d7/fb [3082232,12356] 0 2024-05-13T22:47:58.965 INFO:tasks.workunit.client.0.smithi102.stdout:8/153: readlink d2/d7/l2d 0 2024-05-13T22:47:58.966 INFO:tasks.workunit.client.0.smithi102.stdout:8/154: chown d2/d7/d12/f1c 1057 1 2024-05-13T22:47:58.966 INFO:tasks.workunit.client.0.smithi102.stdout:2/226: symlink d7/d9/d14/d31/l46 0 2024-05-13T22:47:58.968 INFO:tasks.workunit.client.0.smithi102.stdout:6/262: readlink d9/db/d14/d35/d47/l4d 0 2024-05-13T22:47:58.971 INFO:tasks.workunit.client.0.smithi102.stdout:5/322: symlink de/d25/d2a/l80 0 2024-05-13T22:47:58.973 INFO:tasks.workunit.client.0.smithi102.stdout:2/227: dwrite d7/d9/d10/f37 [0,4194304] 0 2024-05-13T22:47:58.975 INFO:tasks.workunit.client.0.smithi102.stdout:2/228: chown d7/d9/d10/d25/l43 457237831 1 2024-05-13T22:47:58.976 INFO:tasks.workunit.client.0.smithi102.stdout:8/155: mknod d2/d7/d25/c2f 0 2024-05-13T22:47:58.978 INFO:tasks.workunit.client.0.smithi102.stdout:2/229: read d7/d15/d1e/f26 [190529,129899] 0 2024-05-13T22:47:58.982 INFO:tasks.workunit.client.0.smithi102.stdout:6/263: rename d9/d45/l4c to d9/db/d14/l55 0 2024-05-13T22:47:58.985 INFO:tasks.workunit.client.0.smithi102.stdout:5/323: link f2 de/d25/d59/f81 0 2024-05-13T22:47:58.987 INFO:tasks.workunit.client.0.smithi102.stdout:2/230: dwrite d7/d9/d10/d25/d27/f45 [0,4194304] 0 2024-05-13T22:47:58.990 INFO:tasks.workunit.client.0.smithi102.stdout:6/264: rmdir d9/db/d14/d37 39 2024-05-13T22:47:58.992 INFO:tasks.workunit.client.0.smithi102.stdout:5/324: truncate f0 2302014 0 2024-05-13T22:47:58.993 INFO:tasks.workunit.client.0.smithi102.stdout:5/325: readlink de/d25/d2c/l60 0 2024-05-13T22:47:58.994 INFO:tasks.workunit.client.0.smithi102.stdout:6/265: mkdir d9/d45/d56 0 2024-05-13T22:47:58.995 INFO:tasks.workunit.client.0.smithi102.stdout:8/156: rmdir d2/d7/d12/d28/d2a 0 2024-05-13T22:47:58.998 INFO:tasks.workunit.client.0.smithi102.stdout:5/326: chown f9 99052153 1 2024-05-13T22:47:59.002 INFO:tasks.workunit.client.0.smithi102.stdout:2/231: link d7/l8 d7/d9/d10/d30/l47 0 2024-05-13T22:47:59.005 INFO:tasks.workunit.client.0.smithi102.stdout:5/327: rename fd to de/d25/d43/f82 0 2024-05-13T22:47:59.006 INFO:tasks.workunit.client.0.smithi102.stdout:5/328: fsync de/f13 0 2024-05-13T22:47:59.008 INFO:tasks.workunit.client.0.smithi102.stdout:6/266: getdents d9/d25 0 2024-05-13T22:47:59.014 INFO:tasks.workunit.client.0.smithi102.stdout:5/329: creat de/d33/f83 x:0 0 0 2024-05-13T22:47:59.015 INFO:tasks.workunit.client.0.smithi102.stdout:6/267: chown d9/db/d14/d37/c4e 11899014 1 2024-05-13T22:47:59.018 INFO:tasks.workunit.client.0.smithi102.stdout:5/330: chown c4 1 1 2024-05-13T22:47:59.019 INFO:tasks.workunit.client.0.smithi102.stdout:8/157: truncate d2/d7/f24 4845728 0 2024-05-13T22:47:59.026 INFO:tasks.workunit.client.0.smithi102.stdout:2/232: rename d7/l2b to d7/d15/l48 0 2024-05-13T22:47:59.030 INFO:tasks.workunit.client.0.smithi102.stdout:6/268: write f3 [3168331,7403] 0 2024-05-13T22:47:59.036 INFO:tasks.workunit.client.0.smithi102.stdout:8/158: rmdir d2/d7/d12/d28 0 2024-05-13T22:47:59.037 INFO:tasks.workunit.client.0.smithi102.stdout:6/269: creat d9/db/d14/d37/f57 x:0 0 0 2024-05-13T22:47:59.038 INFO:tasks.workunit.client.0.smithi102.stdout:2/233: write d7/d15/d1e/f26 [262579,31156] 0 2024-05-13T22:47:59.043 INFO:tasks.workunit.client.0.smithi102.stdout:8/159: mknod d2/d7/d12/d2b/c30 0 2024-05-13T22:47:59.044 INFO:tasks.workunit.client.0.smithi102.stdout:5/331: dread f0 [0,4194304] 0 2024-05-13T22:47:59.045 INFO:tasks.workunit.client.0.smithi102.stdout:2/234: creat d7/d9/f49 x:0 0 0 2024-05-13T22:47:59.047 INFO:tasks.workunit.client.0.smithi102.stdout:8/160: chown d2/d7/d12/l29 3 1 2024-05-13T22:47:59.048 INFO:tasks.workunit.client.0.smithi102.stdout:8/161: chown d2/d7/d12/f13 7 1 2024-05-13T22:47:59.049 INFO:tasks.workunit.client.0.smithi102.stdout:8/162: chown d2/f22 17813247 1 2024-05-13T22:47:59.049 INFO:tasks.workunit.client.0.smithi102.stdout:8/163: chown d2/d17/f26 3 1 2024-05-13T22:47:59.052 INFO:tasks.workunit.client.0.smithi102.stdout:2/235: rmdir d7/d15/d1e/d3f 0 2024-05-13T22:47:59.053 INFO:tasks.workunit.client.0.smithi102.stdout:5/332: getdents de/d33/d34 0 2024-05-13T22:47:59.053 INFO:tasks.workunit.client.0.smithi102.stdout:6/270: dread d9/f1a [0,4194304] 0 2024-05-13T22:47:59.054 INFO:tasks.workunit.client.0.smithi102.stdout:2/236: chown d7/d9/d10/d25/l2a 214 1 2024-05-13T22:47:59.054 INFO:tasks.workunit.client.0.smithi102.stdout:6/271: chown d9/db/d14/d35 2762500 1 2024-05-13T22:47:59.055 INFO:tasks.workunit.client.0.smithi102.stdout:2/237: read d7/d9/d14/f36 [3821397,73196] 0 2024-05-13T22:47:59.061 INFO:tasks.workunit.client.0.smithi102.stdout:6/272: rename d9/db/f49 to d9/d25/f58 0 2024-05-13T22:47:59.065 INFO:tasks.workunit.client.0.smithi102.stdout:2/238: truncate d7/d9/d11/f39 4645360 0 2024-05-13T22:47:59.067 INFO:tasks.workunit.client.0.smithi102.stdout:6/273: mkdir d9/db/d2f/d59 0 2024-05-13T22:47:59.070 INFO:tasks.workunit.client.0.smithi102.stdout:6/274: mkdir d9/d25/d5a 0 2024-05-13T22:47:59.073 INFO:tasks.workunit.client.0.smithi102.stdout:2/239: creat d7/f4a x:0 0 0 2024-05-13T22:47:59.073 INFO:tasks.workunit.client.0.smithi102.stdout:5/333: truncate de/d25/d59/f1c 2804041 0 2024-05-13T22:47:59.075 INFO:tasks.workunit.client.0.smithi102.stdout:6/275: rename d9/db/d14/d37/c4e to d9/db/d14/d54/c5b 0 2024-05-13T22:47:59.079 INFO:tasks.workunit.client.0.smithi102.stdout:6/276: creat d9/db/d14/d35/d47/f5c x:0 0 0 2024-05-13T22:47:59.080 INFO:tasks.workunit.client.0.smithi102.stdout:8/164: dwrite d2/d7/f24 [0,4194304] 0 2024-05-13T22:47:59.081 INFO:tasks.workunit.client.0.smithi102.stdout:5/334: fdatasync de/d25/d2c/f5f 0 2024-05-13T22:47:59.082 INFO:tasks.workunit.client.0.smithi102.stdout:6/277: chown d9/db/d14/d35/d47/c51 18605 1 2024-05-13T22:47:59.083 INFO:tasks.workunit.client.0.smithi102.stdout:6/278: stat d9/db/d14/d35/c42 0 2024-05-13T22:47:59.088 INFO:tasks.workunit.client.0.smithi102.stdout:8/165: rename d2/d7/fb to d2/d7/d27/f31 0 2024-05-13T22:47:59.094 INFO:tasks.workunit.client.0.smithi102.stdout:8/166: dwrite d2/d7/d12/f23 [0,4194304] 0 2024-05-13T22:47:59.099 INFO:tasks.workunit.client.0.smithi102.stdout:2/240: write d7/d9/d11/f39 [5321233,7116] 0 2024-05-13T22:47:59.102 INFO:tasks.workunit.client.0.smithi102.stdout:5/335: getdents de/d75 0 2024-05-13T22:47:59.108 INFO:tasks.workunit.client.0.smithi102.stdout:8/167: rename d2/f19 to d2/f32 0 2024-05-13T22:47:59.109 INFO:tasks.workunit.client.0.smithi102.stdout:8/168: write d2/f11 [1264570,16773] 0 2024-05-13T22:47:59.113 INFO:tasks.workunit.client.0.smithi102.stdout:5/336: link de/d25/d59/f24 de/d25/d2a/d4b/d4d/f84 0 2024-05-13T22:47:59.115 INFO:tasks.workunit.client.0.smithi102.stdout:5/337: rmdir de/d25/d2a 39 2024-05-13T22:47:59.116 INFO:tasks.workunit.client.0.smithi102.stdout:2/241: write d7/d9/f29 [43120,115017] 0 2024-05-13T22:47:59.120 INFO:tasks.workunit.client.0.smithi102.stdout:2/242: fsync d7/ff 0 2024-05-13T22:47:59.121 INFO:tasks.workunit.client.0.smithi102.stdout:8/169: dread d2/f32 [0,4194304] 0 2024-05-13T22:47:59.125 INFO:tasks.workunit.client.0.smithi102.stdout:2/243: unlink d7/d15/f1f 0 2024-05-13T22:47:59.125 INFO:tasks.workunit.client.0.smithi102.stdout:5/338: write de/d25/d59/f81 [3770568,84298] 0 2024-05-13T22:47:59.128 INFO:tasks.workunit.client.0.smithi102.stdout:8/170: dwrite d2/d7/f24 [4194304,4194304] 0 2024-05-13T22:47:59.129 INFO:tasks.workunit.client.0.smithi102.stdout:8/171: readlink d2/d7/l2d 0 2024-05-13T22:47:59.130 INFO:tasks.workunit.client.0.smithi102.stdout:2/244: mkdir d7/d9/d10/d25/d4b 0 2024-05-13T22:47:59.136 INFO:tasks.workunit.client.0.smithi102.stdout:8/172: creat d2/d17/f33 x:0 0 0 2024-05-13T22:47:59.136 INFO:tasks.workunit.client.0.smithi102.stdout:5/339: dwrite de/d25/d59/d28/f66 [0,4194304] 0 2024-05-13T22:47:59.142 INFO:tasks.workunit.client.0.smithi102.stdout:2/245: dread d7/d9/f13 [0,4194304] 0 2024-05-13T22:47:59.142 INFO:tasks.workunit.client.0.smithi102.stdout:8/173: creat d2/d7/d12/d2b/f34 x:0 0 0 2024-05-13T22:47:59.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:47:58 smithi130 ceph-mon[36591]: pgmap v16: 65 pgs: 65 active+clean; 232 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 10 MiB/s wr, 329 op/s 2024-05-13T22:47:59.151 INFO:tasks.workunit.client.0.smithi102.stdout:8/174: rename d2/d7/f24 to d2/d7/d12/d2b/f35 0 2024-05-13T22:47:59.152 INFO:tasks.workunit.client.0.smithi102.stdout:8/175: write d2/d17/f33 [306095,70581] 0 2024-05-13T22:47:59.153 INFO:tasks.workunit.client.0.smithi102.stdout:2/246: link d7/ff d7/d9/d10/d1b/f4c 0 2024-05-13T22:47:59.158 INFO:tasks.workunit.client.0.smithi102.stdout:2/247: symlink d7/d9/d14/l4d 0 2024-05-13T22:47:59.160 INFO:tasks.workunit.client.0.smithi102.stdout:8/176: mkdir d2/d36 0 2024-05-13T22:47:59.163 INFO:tasks.workunit.client.0.smithi102.stdout:5/340: truncate de/d25/d43/f82 2397842 0 2024-05-13T22:47:59.163 INFO:tasks.workunit.client.0.smithi102.stdout:8/177: fsync d2/d7/d12/f1c 0 2024-05-13T22:47:59.164 INFO:tasks.workunit.client.0.smithi102.stdout:5/341: chown de/d25/d59/d28/f5c 1414 1 2024-05-13T22:47:59.166 INFO:tasks.workunit.client.0.smithi102.stdout:8/178: symlink d2/d17/l37 0 2024-05-13T22:47:59.168 INFO:tasks.workunit.client.0.smithi102.stdout:5/342: chown de/d25/d59/c3d 60 1 2024-05-13T22:47:59.169 INFO:tasks.workunit.client.0.smithi102.stdout:8/179: symlink d2/d7/d12/l38 0 2024-05-13T22:47:59.170 INFO:tasks.workunit.client.0.smithi102.stdout:5/343: creat de/d75/f85 x:0 0 0 2024-05-13T22:47:59.173 INFO:tasks.workunit.client.0.smithi102.stdout:5/344: write de/d25/d59/d6f/f72 [286492,66741] 0 2024-05-13T22:47:59.174 INFO:tasks.workunit.client.0.smithi102.stdout:5/345: write de/f21 [1960478,126949] 0 2024-05-13T22:47:59.178 INFO:tasks.workunit.client.0.smithi102.stdout:5/346: mkdir de/d25/d59/d7f/d86 0 2024-05-13T22:47:59.180 INFO:tasks.workunit.client.0.smithi102.stdout:5/347: mknod de/d33/d34/c87 0 2024-05-13T22:47:59.180 INFO:tasks.workunit.client.0.smithi102.stdout:5/348: read - de/d25/d59/d28/f67 zero size 2024-05-13T22:47:59.182 INFO:tasks.workunit.client.0.smithi102.stdout:5/349: fdatasync f0 0 2024-05-13T22:47:59.187 INFO:tasks.workunit.client.0.smithi102.stdout:5/350: creat de/d25/d43/d7a/f88 x:0 0 0 2024-05-13T22:47:59.188 INFO:tasks.workunit.client.0.smithi102.stdout:5/351: write de/d25/d59/f44 [8557607,99368] 0 2024-05-13T22:47:59.212 INFO:tasks.workunit.client.0.smithi102.stdout:5/352: write de/d33/d34/f4a [706062,118405] 0 2024-05-13T22:47:59.218 INFO:tasks.workunit.client.0.smithi102.stdout:5/353: dwrite de/d25/d59/f32 [8388608,4194304] 0 2024-05-13T22:47:59.260 INFO:tasks.workunit.client.0.smithi102.stdout:5/354: write de/d25/d59/f2e [3125066,60175] 0 2024-05-13T22:47:59.262 INFO:tasks.workunit.client.0.smithi102.stdout:5/355: write de/d75/f85 [533295,3384] 0 2024-05-13T22:47:59.266 INFO:tasks.workunit.client.0.smithi102.stdout:5/356: dread de/d25/d59/d28/f55 [0,4194304] 0 2024-05-13T22:47:59.275 INFO:tasks.workunit.client.0.smithi102.stdout:5/357: getdents de/d25/d43/d7a 0 2024-05-13T22:47:59.276 INFO:tasks.workunit.client.0.smithi102.stdout:5/358: write de/d25/d59/f44 [7796351,114062] 0 2024-05-13T22:47:59.277 INFO:tasks.workunit.client.0.smithi102.stdout:5/359: write de/d25/d59/f81 [4799474,84924] 0 2024-05-13T22:47:59.286 INFO:tasks.workunit.client.0.smithi102.stdout:5/360: link de/d25/d59/c3d de/d25/d59/d28/c89 0 2024-05-13T22:47:59.286 INFO:tasks.workunit.client.0.smithi102.stdout:5/361: stat de/d25/d59/f32 0 2024-05-13T22:47:59.287 INFO:tasks.workunit.client.0.smithi102.stdout:5/362: fdatasync de/d25/f47 0 2024-05-13T22:47:59.291 INFO:tasks.workunit.client.0.smithi102.stdout:5/363: creat de/d25/d2c/f8a x:0 0 0 2024-05-13T22:47:59.298 INFO:tasks.workunit.client.0.smithi102.stdout:5/364: fdatasync f6 0 2024-05-13T22:47:59.331 INFO:tasks.workunit.client.0.smithi102.stdout:5/365: truncate de/f14 3317236 0 2024-05-13T22:47:59.334 INFO:tasks.workunit.client.0.smithi102.stdout:5/366: mknod de/d25/d59/d28/c8b 0 2024-05-13T22:47:59.334 INFO:tasks.workunit.client.0.smithi102.stdout:5/367: mknod de/d75/c8c 0 2024-05-13T22:47:59.338 INFO:tasks.workunit.client.0.smithi102.stdout:5/368: symlink de/d25/d2a/d4b/l8d 0 2024-05-13T22:47:59.340 INFO:tasks.workunit.client.0.smithi102.stdout:5/369: creat de/d25/d59/d7f/f8e x:0 0 0 2024-05-13T22:47:59.343 INFO:tasks.workunit.client.0.smithi102.stdout:5/370: creat de/d25/d59/f8f x:0 0 0 2024-05-13T22:47:59.344 INFO:tasks.workunit.client.0.smithi102.stdout:5/371: write de/d25/d59/f44 [5510456,109428] 0 2024-05-13T22:47:59.345 INFO:tasks.workunit.client.0.smithi102.stdout:5/372: dread - de/d25/d59/f8f zero size 2024-05-13T22:47:59.352 INFO:tasks.workunit.client.0.smithi102.stdout:5/373: dread de/f35 [0,4194304] 0 2024-05-13T22:47:59.355 INFO:tasks.workunit.client.0.smithi102.stdout:5/374: rmdir de/d25/d58 39 2024-05-13T22:47:59.358 INFO:tasks.workunit.client.0.smithi102.stdout:5/375: symlink de/d25/d58/l90 0 2024-05-13T22:47:59.359 INFO:tasks.workunit.client.0.smithi102.stdout:5/376: fsync de/d33/d34/f4a 0 2024-05-13T22:47:59.360 INFO:tasks.workunit.client.0.smithi102.stdout:5/377: dread - de/d25/d59/d7f/f8e zero size 2024-05-13T22:47:59.363 INFO:tasks.workunit.client.0.smithi102.stdout:5/378: rmdir de/d25/d59/d6f 39 2024-05-13T22:47:59.364 INFO:tasks.workunit.client.0.smithi102.stdout:5/379: write de/d25/d59/f49 [1498617,63475] 0 2024-05-13T22:47:59.371 INFO:tasks.workunit.client.0.smithi102.stdout:5/380: rename de/d25/d59/l39 to de/d75/l91 0 2024-05-13T22:47:59.373 INFO:tasks.workunit.client.0.smithi102.stdout:5/381: read de/d25/f26 [69215,61372] 0 2024-05-13T22:47:59.378 INFO:tasks.workunit.client.0.smithi102.stdout:5/382: creat de/d25/d59/d6f/f92 x:0 0 0 2024-05-13T22:47:59.385 INFO:tasks.workunit.client.0.smithi102.stdout:5/383: getdents de/d25/d2c 0 2024-05-13T22:47:59.387 INFO:tasks.workunit.client.0.smithi102.stdout:5/384: mkdir de/d25/d2a/d4b/d4d/d93 0 2024-05-13T22:47:59.390 INFO:tasks.workunit.client.0.smithi102.stdout:5/385: creat de/d25/d43/f94 x:0 0 0 2024-05-13T22:47:59.396 INFO:tasks.workunit.client.0.smithi102.stdout:5/386: creat de/f95 x:0 0 0 2024-05-13T22:47:59.397 INFO:tasks.workunit.client.0.smithi102.stdout:5/387: creat de/d25/d2c/f96 x:0 0 0 2024-05-13T22:47:59.399 INFO:tasks.workunit.client.0.smithi102.stdout:5/388: write de/d25/d2c/f71 [321260,112013] 0 2024-05-13T22:47:59.400 INFO:tasks.workunit.client.0.smithi102.stdout:5/389: write de/f61 [713104,50966] 0 2024-05-13T22:47:59.402 INFO:tasks.workunit.client.0.smithi102.stdout:5/390: creat de/d25/d43/f97 x:0 0 0 2024-05-13T22:47:59.403 INFO:tasks.workunit.client.0.smithi102.stdout:5/391: creat de/d25/d59/d28/f98 x:0 0 0 2024-05-13T22:47:59.404 INFO:tasks.workunit.client.0.smithi102.stdout:5/392: readlink de/d25/d58/l90 0 2024-05-13T22:47:59.405 INFO:tasks.workunit.client.0.smithi102.stdout:5/393: dread - de/d25/d2a/d4b/f70 zero size 2024-05-13T22:47:59.406 INFO:tasks.workunit.client.0.smithi102.stdout:5/394: mkdir de/d25/d2a/d4b/d4d/d50/d99 0 2024-05-13T22:47:59.407 INFO:tasks.workunit.client.0.smithi102.stdout:5/395: chown de/d25/d2c/l2d 336631 1 2024-05-13T22:47:59.410 INFO:tasks.workunit.client.0.smithi102.stdout:5/396: dread de/d25/d59/f81 [4194304,4194304] 0 2024-05-13T22:47:59.416 INFO:tasks.workunit.client.0.smithi102.stdout:5/397: dwrite de/d25/d59/d6f/f46 [0,4194304] 0 2024-05-13T22:47:59.421 INFO:tasks.workunit.client.0.smithi102.stdout:5/398: dread de/d25/d2a/f2b [0,4194304] 0 2024-05-13T22:47:59.426 INFO:tasks.workunit.client.0.smithi102.stdout:5/399: rename de/d25/d2c/l60 to de/d25/d59/d6f/l9a 0 2024-05-13T22:47:59.428 INFO:tasks.workunit.client.0.smithi102.stdout:5/400: rmdir de/d75 39 2024-05-13T22:47:59.429 INFO:tasks.workunit.client.0.smithi102.stdout:5/401: fdatasync de/d25/d59/f18 0 2024-05-13T22:47:59.431 INFO:tasks.workunit.client.0.smithi102.stdout:5/402: getdents de/d25/d43/d7a 0 2024-05-13T22:47:59.434 INFO:tasks.workunit.client.0.smithi102.stdout:5/403: fsync de/d25/d59/f40 0 2024-05-13T22:47:59.438 INFO:tasks.workunit.client.0.smithi102.stdout:5/404: symlink de/d33/l9b 0 2024-05-13T22:47:59.438 INFO:tasks.workunit.client.0.smithi102.stdout:5/405: readlink de/d25/d58/l90 0 2024-05-13T22:47:59.440 INFO:tasks.workunit.client.0.smithi102.stdout:5/406: symlink de/d25/d59/d7f/d86/l9c 0 2024-05-13T22:47:59.442 INFO:tasks.workunit.client.0.smithi102.stdout:5/407: symlink de/d25/d2a/l9d 0 2024-05-13T22:47:59.443 INFO:tasks.workunit.client.0.smithi102.stdout:5/408: write de/d25/d59/d6f/f92 [984412,79011] 0 2024-05-13T22:47:59.444 INFO:tasks.workunit.client.0.smithi102.stdout:5/409: dread - de/d25/d59/d28/f98 zero size 2024-05-13T22:47:59.445 INFO:tasks.workunit.client.0.smithi102.stdout:5/410: chown de/d33/d34/c87 15488370 1 2024-05-13T22:47:59.447 INFO:tasks.workunit.client.0.smithi102.stdout:5/411: mkdir de/d33/d53/d9e 0 2024-05-13T22:47:59.453 INFO:tasks.workunit.client.0.smithi102.stdout:5/412: dread de/d25/d59/f49 [0,4194304] 0 2024-05-13T22:47:59.459 INFO:tasks.workunit.client.0.smithi102.stdout:5/413: dwrite de/d25/d59/d28/f98 [0,4194304] 0 2024-05-13T22:47:59.460 INFO:tasks.workunit.client.0.smithi102.stdout:5/414: chown de/d25/d59/d28 6 1 2024-05-13T22:47:59.829 INFO:tasks.workunit.client.0.smithi102.stdout:4/218: sync 2024-05-13T22:47:59.829 INFO:tasks.workunit.client.0.smithi102.stdout:9/215: sync 2024-05-13T22:47:59.829 INFO:tasks.workunit.client.0.smithi102.stdout:2/248: sync 2024-05-13T22:47:59.829 INFO:tasks.workunit.client.0.smithi102.stdout:6/279: sync 2024-05-13T22:47:59.829 INFO:tasks.workunit.client.0.smithi102.stdout:3/285: sync 2024-05-13T22:47:59.829 INFO:tasks.workunit.client.0.smithi102.stdout:1/173: sync 2024-05-13T22:47:59.830 INFO:tasks.workunit.client.0.smithi102.stdout:7/135: sync 2024-05-13T22:47:59.830 INFO:tasks.workunit.client.0.smithi102.stdout:8/180: sync 2024-05-13T22:47:59.830 INFO:tasks.workunit.client.0.smithi102.stdout:5/415: sync 2024-05-13T22:47:59.830 INFO:tasks.workunit.client.0.smithi102.stdout:0/259: sync 2024-05-13T22:47:59.830 INFO:tasks.workunit.client.0.smithi102.stdout:8/181: readlink d2/d7/l2d 0 2024-05-13T22:47:59.832 INFO:tasks.workunit.client.0.smithi102.stdout:7/136: read d6/f8 [1982896,61874] 0 2024-05-13T22:47:59.835 INFO:tasks.workunit.client.0.smithi102.stdout:1/174: write d2/d4/f23 [548059,119973] 0 2024-05-13T22:47:59.838 INFO:tasks.workunit.client.0.smithi102.stdout:6/280: dread d9/f1a [0,4194304] 0 2024-05-13T22:47:59.838 INFO:tasks.workunit.client.0.smithi102.stdout:5/416: fdatasync de/d25/d59/f40 0 2024-05-13T22:47:59.838 INFO:tasks.workunit.client.0.smithi102.stdout:2/249: dread d7/d9/d11/f39 [0,4194304] 0 2024-05-13T22:47:59.840 INFO:tasks.workunit.client.0.smithi102.stdout:0/260: dread d1/de/d32/f1e [0,4194304] 0 2024-05-13T22:47:59.843 INFO:tasks.workunit.client.0.smithi102.stdout:9/216: mknod d1/c51 0 2024-05-13T22:47:59.844 INFO:tasks.workunit.client.0.smithi102.stdout:8/182: creat d2/d7/d12/d2b/f39 x:0 0 0 2024-05-13T22:47:59.847 INFO:tasks.workunit.client.0.smithi102.stdout:7/137: unlink d6/l1a 0 2024-05-13T22:47:59.852 INFO:tasks.workunit.client.0.smithi102.stdout:5/417: creat de/d25/d2a/f9f x:0 0 0 2024-05-13T22:47:59.854 INFO:tasks.workunit.client.0.smithi102.stdout:6/281: dread f0 [0,4194304] 0 2024-05-13T22:47:59.859 INFO:tasks.workunit.client.0.smithi102.stdout:5/418: dwrite de/d25/d2a/d4b/f76 [0,4194304] 0 2024-05-13T22:47:59.863 INFO:tasks.workunit.client.0.smithi102.stdout:1/175: mkdir d2/d4/d38 0 2024-05-13T22:47:59.869 INFO:tasks.workunit.client.0.smithi102.stdout:2/250: rename d7/d15/l48 to d7/d9/d10/d25/l4e 0 2024-05-13T22:47:59.870 INFO:tasks.workunit.client.0.smithi102.stdout:0/261: symlink d1/de/d32/l49 0 2024-05-13T22:47:59.871 INFO:tasks.workunit.client.0.smithi102.stdout:0/262: write d1/f2c [1699608,62830] 0 2024-05-13T22:47:59.872 INFO:tasks.workunit.client.0.smithi102.stdout:9/217: creat d1/d6/db/d4a/f52 x:0 0 0 2024-05-13T22:47:59.876 INFO:tasks.workunit.client.0.smithi102.stdout:7/138: symlink d6/d15/d21/d20/l27 0 2024-05-13T22:47:59.878 INFO:tasks.workunit.client.0.smithi102.stdout:5/419: creat de/d25/d2a/fa0 x:0 0 0 2024-05-13T22:47:59.878 INFO:tasks.workunit.client.0.smithi102.stdout:9/218: dwrite d1/f1e [0,4194304] 0 2024-05-13T22:47:59.886 INFO:tasks.workunit.client.0.smithi102.stdout:5/420: dwrite de/d25/d43/f97 [0,4194304] 0 2024-05-13T22:47:59.886 INFO:tasks.workunit.client.0.smithi102.stdout:1/176: rename d2/d10/f1a to d2/d4/d5/d19/f39 0 2024-05-13T22:47:59.890 INFO:tasks.workunit.client.0.smithi102.stdout:0/263: mknod d1/de/d42/c4a 0 2024-05-13T22:47:59.891 INFO:tasks.workunit.client.0.smithi102.stdout:0/264: write d1/de/d32/f17 [4994676,110854] 0 2024-05-13T22:47:59.898 INFO:tasks.workunit.client.0.smithi102.stdout:2/251: symlink d7/l4f 0 2024-05-13T22:47:59.900 INFO:tasks.workunit.client.0.smithi102.stdout:7/139: dwrite d6/d13/f14 [4194304,4194304] 0 2024-05-13T22:47:59.901 INFO:tasks.workunit.client.0.smithi102.stdout:0/265: dwrite d1/f9 [0,4194304] 0 2024-05-13T22:47:59.903 INFO:tasks.workunit.client.0.smithi102.stdout:5/421: readlink lb 0 2024-05-13T22:47:59.904 INFO:tasks.workunit.client.0.smithi102.stdout:1/177: mknod d2/d4/d26/c3a 0 2024-05-13T22:47:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:5/422: write de/f13 [10626042,26083] 0 2024-05-13T22:47:59.905 INFO:tasks.workunit.client.0.smithi102.stdout:2/252: dread d7/f32 [0,4194304] 0 2024-05-13T22:47:59.908 INFO:tasks.workunit.client.0.smithi102.stdout:2/253: write d7/d9/d14/f36 [1101314,60473] 0 2024-05-13T22:47:59.911 INFO:tasks.workunit.client.0.smithi102.stdout:9/219: creat d1/d6/d45/d3e/f53 x:0 0 0 2024-05-13T22:47:59.914 INFO:tasks.workunit.client.0.smithi102.stdout:7/140: creat d6/d15/f28 x:0 0 0 2024-05-13T22:47:59.922 INFO:tasks.workunit.client.0.smithi102.stdout:0/266: dwrite d1/de/d32/f3f [4194304,4194304] 0 2024-05-13T22:47:59.922 INFO:tasks.workunit.client.0.smithi102.stdout:5/423: mknod de/d25/d2a/d4b/d4d/d50/ca1 0 2024-05-13T22:47:59.924 INFO:tasks.workunit.client.0.smithi102.stdout:2/254: symlink d7/d9/d10/d1b/l50 0 2024-05-13T22:47:59.925 INFO:tasks.workunit.client.0.smithi102.stdout:2/255: chown d7/d9/d10/d25 34826482 1 2024-05-13T22:47:59.928 INFO:tasks.workunit.client.0.smithi102.stdout:5/424: dread de/d25/d59/d28/f98 [0,4194304] 0 2024-05-13T22:47:59.929 INFO:tasks.workunit.client.0.smithi102.stdout:5/425: truncate de/d33/f83 506167 0 2024-05-13T22:47:59.930 INFO:tasks.workunit.client.0.smithi102.stdout:5/426: chown de/d25 605353506 1 2024-05-13T22:47:59.930 INFO:tasks.workunit.client.0.smithi102.stdout:9/220: unlink d1/d6/d45/c39 0 2024-05-13T22:47:59.937 INFO:tasks.workunit.client.0.smithi102.stdout:2/256: symlink d7/d9/d10/d1b/l51 0 2024-05-13T22:47:59.938 INFO:tasks.workunit.client.0.smithi102.stdout:2/257: write d7/d9/f29 [643375,69912] 0 2024-05-13T22:47:59.943 INFO:tasks.workunit.client.0.smithi102.stdout:9/221: rmdir d1/d6/db/d30 39 2024-05-13T22:47:59.945 INFO:tasks.workunit.client.0.smithi102.stdout:2/258: rmdir d7/d9/d10/d30 39 2024-05-13T22:47:59.945 INFO:tasks.workunit.client.0.smithi102.stdout:2/259: chown d7/d9/d10/d1b/f4c 711 1 2024-05-13T22:47:59.949 INFO:tasks.workunit.client.0.smithi102.stdout:7/141: getdents d6/d15/d21 0 2024-05-13T22:47:59.949 INFO:tasks.workunit.client.0.smithi102.stdout:7/142: readlink d6/d13/de/la 0 2024-05-13T22:47:59.949 INFO:tasks.workunit.client.0.smithi102.stdout:7/143: chown d6/d15 5934 1 2024-05-13T22:47:59.951 INFO:tasks.workunit.client.0.smithi102.stdout:9/222: mknod d1/c54 0 2024-05-13T22:47:59.951 INFO:tasks.workunit.client.0.smithi102.stdout:9/223: stat d1/d6/f1d 0 2024-05-13T22:47:59.953 INFO:tasks.workunit.client.0.smithi102.stdout:7/144: creat d6/d13/de/f29 x:0 0 0 2024-05-13T22:47:59.954 INFO:tasks.workunit.client.0.smithi102.stdout:9/224: fdatasync d1/f16 0 2024-05-13T22:47:59.956 INFO:tasks.workunit.client.0.smithi102.stdout:2/260: creat d7/d15/f52 x:0 0 0 2024-05-13T22:48:00.013 INFO:tasks.workunit.client.0.smithi102.stdout:3/286: write f5 [1936546,93064] 0 2024-05-13T22:48:00.013 INFO:tasks.workunit.client.0.smithi102.stdout:4/219: write d3/d19/d23/f43 [592361,32805] 0 2024-05-13T22:48:00.015 INFO:tasks.workunit.client.0.smithi102.stdout:3/287: fsync de/d10/d34/f5f 0 2024-05-13T22:48:00.022 INFO:tasks.workunit.client.0.smithi102.stdout:8/183: dwrite d2/f32 [0,4194304] 0 2024-05-13T22:48:00.025 INFO:tasks.workunit.client.0.smithi102.stdout:2/261: rmdir d7 39 2024-05-13T22:48:00.025 INFO:tasks.workunit.client.0.smithi102.stdout:6/282: dwrite d9/fa [0,4194304] 0 2024-05-13T22:48:00.029 INFO:tasks.workunit.client.0.smithi102.stdout:4/220: chown d3/d19/c27 5 1 2024-05-13T22:48:00.029 INFO:tasks.workunit.client.0.smithi102.stdout:7/145: creat d6/f2a x:0 0 0 2024-05-13T22:48:00.032 INFO:tasks.workunit.client.0.smithi102.stdout:6/283: fsync f0 0 2024-05-13T22:48:00.037 INFO:tasks.workunit.client.0.smithi102.stdout:2/262: mkdir d7/d9/d10/d30/d53 0 2024-05-13T22:48:00.038 INFO:tasks.workunit.client.0.smithi102.stdout:9/225: getdents d1/d6/db/d30 0 2024-05-13T22:48:00.040 INFO:tasks.workunit.client.0.smithi102.stdout:8/184: dwrite d2/d7/f9 [0,4194304] 0 2024-05-13T22:48:00.043 INFO:tasks.workunit.client.0.smithi102.stdout:8/185: readlink d2/d17/l37 0 2024-05-13T22:48:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:1/178: dwrite d2/d4/d5/d19/d28/f35 [0,4194304] 0 2024-05-13T22:48:00.044 INFO:tasks.workunit.client.0.smithi102.stdout:4/221: rename d3/d4/l1a to d3/d4/d6/de/l52 0 2024-05-13T22:48:00.049 INFO:tasks.workunit.client.0.smithi102.stdout:4/222: write d3/d4/d6/f18 [396289,52284] 0 2024-05-13T22:48:00.051 INFO:tasks.workunit.client.0.smithi102.stdout:6/284: dwrite d9/db/f22 [4194304,4194304] 0 2024-05-13T22:48:00.052 INFO:tasks.workunit.client.0.smithi102.stdout:2/263: mkdir d7/d9/d10/d1b/d54 0 2024-05-13T22:48:00.053 INFO:tasks.workunit.client.0.smithi102.stdout:6/285: chown d9/db/d14/c1b 13 1 2024-05-13T22:48:00.055 INFO:tasks.workunit.client.0.smithi102.stdout:6/286: chown d9/db/f50 85765050 1 2024-05-13T22:48:00.055 INFO:tasks.workunit.client.0.smithi102.stdout:8/186: mknod d2/d7/d12/d2b/c3a 0 2024-05-13T22:48:00.057 INFO:tasks.workunit.client.0.smithi102.stdout:1/179: creat d2/d4/d5/d19/d28/f3b x:0 0 0 2024-05-13T22:48:00.059 INFO:tasks.workunit.client.0.smithi102.stdout:1/180: chown d2/d4/d5/f36 149413149 1 2024-05-13T22:48:00.059 INFO:tasks.workunit.client.0.smithi102.stdout:8/187: fdatasync d2/d7/f9 0 2024-05-13T22:48:00.060 INFO:tasks.workunit.client.0.smithi102.stdout:1/181: fdatasync d2/d4/d5/fa 0 2024-05-13T22:48:00.060 INFO:tasks.workunit.client.0.smithi102.stdout:8/188: chown d2/d17/f2c 243684 1 2024-05-13T22:48:00.061 INFO:tasks.workunit.client.0.smithi102.stdout:9/226: symlink d1/d6/d45/l55 0 2024-05-13T22:48:00.062 INFO:tasks.workunit.client.0.smithi102.stdout:7/146: dread d6/f7 [0,4194304] 0 2024-05-13T22:48:00.063 INFO:tasks.workunit.client.0.smithi102.stdout:7/147: dread - d6/d15/d21/f26 zero size 2024-05-13T22:48:00.064 INFO:tasks.workunit.client.0.smithi102.stdout:7/148: write d6/d15/f1e [4910043,32089] 0 2024-05-13T22:48:00.065 INFO:tasks.workunit.client.0.smithi102.stdout:7/149: dread - d6/d22/f24 zero size 2024-05-13T22:48:00.069 INFO:tasks.workunit.client.0.smithi102.stdout:1/182: read d2/f2b [64732,15953] 0 2024-05-13T22:48:00.070 INFO:tasks.workunit.client.0.smithi102.stdout:1/183: stat d2/d4/d26/l2d 0 2024-05-13T22:48:00.071 INFO:tasks.workunit.client.0.smithi102.stdout:1/184: stat d2/d4/d26/l2c 0 2024-05-13T22:48:00.073 INFO:tasks.workunit.client.0.smithi102.stdout:7/150: dwrite d6/d13/f14 [0,4194304] 0 2024-05-13T22:48:00.077 INFO:tasks.workunit.client.0.smithi102.stdout:6/287: mkdir d9/d45/d56/d5d 0 2024-05-13T22:48:00.080 INFO:tasks.workunit.client.0.smithi102.stdout:8/189: dwrite d2/f4 [4194304,4194304] 0 2024-05-13T22:48:00.081 INFO:tasks.workunit.client.0.smithi102.stdout:8/190: chown d2/d7/d12 64 1 2024-05-13T22:48:00.083 INFO:tasks.workunit.client.0.smithi102.stdout:2/264: rename d7/d9/d10/d1b/f4c to d7/d9/d11/f55 0 2024-05-13T22:48:00.089 INFO:tasks.workunit.client.0.smithi102.stdout:8/191: dwrite d2/d7/d12/f23 [0,4194304] 0 2024-05-13T22:48:00.095 INFO:tasks.workunit.client.0.smithi102.stdout:2/265: mkdir d7/d9/d10/d25/d56 0 2024-05-13T22:48:00.097 INFO:tasks.workunit.client.0.smithi102.stdout:7/151: mkdir d6/d2b 0 2024-05-13T22:48:00.097 INFO:tasks.workunit.client.0.smithi102.stdout:4/223: truncate d3/d4/f50 4064559 0 2024-05-13T22:48:00.099 INFO:tasks.workunit.client.0.smithi102.stdout:9/227: write d1/d6/d21/f29 [329269,95676] 0 2024-05-13T22:48:00.099 INFO:tasks.workunit.client.0.smithi102.stdout:8/192: fsync d2/d17/f2c 0 2024-05-13T22:48:00.101 INFO:tasks.workunit.client.0.smithi102.stdout:2/266: dread d7/d9/d10/d25/d27/f3b [0,4194304] 0 2024-05-13T22:48:00.102 INFO:tasks.workunit.client.0.smithi102.stdout:1/185: rename d2/l11 to d2/d4/d5/d19/d28/l3c 0 2024-05-13T22:48:00.102 INFO:tasks.workunit.client.0.smithi102.stdout:8/193: write d2/d7/d12/d2b/f35 [8579910,116765] 0 2024-05-13T22:48:00.103 INFO:tasks.workunit.client.0.smithi102.stdout:8/194: fdatasync d2/f22 0 2024-05-13T22:48:00.104 INFO:tasks.workunit.client.0.smithi102.stdout:1/186: truncate d2/d4/f23 1472803 0 2024-05-13T22:48:00.106 INFO:tasks.workunit.client.0.smithi102.stdout:4/224: creat d3/d4/d6/de/d1f/f53 x:0 0 0 2024-05-13T22:48:00.109 INFO:tasks.workunit.client.0.smithi102.stdout:4/225: read d3/d4/d6/de/f3f [2530094,67118] 0 2024-05-13T22:48:00.112 INFO:tasks.workunit.client.0.smithi102.stdout:7/152: mkdir d6/d13/d2c 0 2024-05-13T22:48:00.114 INFO:tasks.workunit.client.0.smithi102.stdout:1/187: mknod d2/d4/d26/c3d 0 2024-05-13T22:48:00.115 INFO:tasks.workunit.client.0.smithi102.stdout:6/288: link d9/db/c26 d9/d45/c5e 0 2024-05-13T22:48:00.117 INFO:tasks.workunit.client.0.smithi102.stdout:2/267: symlink d7/d9/d10/d1b/d54/l57 0 2024-05-13T22:48:00.124 INFO:tasks.workunit.client.0.smithi102.stdout:1/188: unlink d2/l7 0 2024-05-13T22:48:00.125 INFO:tasks.workunit.client.0.smithi102.stdout:7/153: dwrite d6/f8 [4194304,4194304] 0 2024-05-13T22:48:00.126 INFO:tasks.workunit.client.0.smithi102.stdout:7/154: write d6/d13/f14 [7620722,9961] 0 2024-05-13T22:48:00.126 INFO:tasks.workunit.client.0.smithi102.stdout:1/189: read d2/d10/f21 [3179446,49031] 0 2024-05-13T22:48:00.127 INFO:tasks.workunit.client.0.smithi102.stdout:7/155: chown d6/d13/de 426 1 2024-05-13T22:48:00.134 INFO:tasks.workunit.client.0.smithi102.stdout:4/226: creat d3/d19/d23/d4e/f54 x:0 0 0 2024-05-13T22:48:00.135 INFO:tasks.workunit.client.0.smithi102.stdout:4/227: stat d3/l11 0 2024-05-13T22:48:00.136 INFO:tasks.workunit.client.0.smithi102.stdout:7/156: dwrite d6/d15/f1b [0,4194304] 0 2024-05-13T22:48:00.144 INFO:tasks.workunit.client.0.smithi102.stdout:1/190: fdatasync d2/ff 0 2024-05-13T22:48:00.147 INFO:tasks.workunit.client.0.smithi102.stdout:6/289: write f7 [2544811,1246] 0 2024-05-13T22:48:00.149 INFO:tasks.workunit.client.0.smithi102.stdout:2/268: write d7/ff [8815221,61055] 0 2024-05-13T22:48:00.150 INFO:tasks.workunit.client.0.smithi102.stdout:8/195: getdents d2/d7/d25 0 2024-05-13T22:48:00.150 INFO:tasks.workunit.client.0.smithi102.stdout:2/269: write d7/d15/d1e/f26 [520019,45833] 0 2024-05-13T22:48:00.152 INFO:tasks.workunit.client.0.smithi102.stdout:2/270: truncate d7/d9/f49 1020019 0 2024-05-13T22:48:00.153 INFO:tasks.workunit.client.0.smithi102.stdout:2/271: write d7/d9/d10/f33 [3125481,18057] 0 2024-05-13T22:48:00.154 INFO:tasks.workunit.client.0.smithi102.stdout:4/228: creat d3/d19/d23/d4e/f55 x:0 0 0 2024-05-13T22:48:00.159 INFO:tasks.workunit.client.0.smithi102.stdout:1/191: mknod d2/d4/d5/d19/d28/c3e 0 2024-05-13T22:48:00.160 INFO:tasks.workunit.client.0.smithi102.stdout:4/229: dwrite d3/d4/d6/f18 [0,4194304] 0 2024-05-13T22:48:00.161 INFO:tasks.workunit.client.0.smithi102.stdout:6/290: mkdir d9/db/d14/d37/d5f 0 2024-05-13T22:48:00.168 INFO:tasks.workunit.client.0.smithi102.stdout:8/196: creat d2/d7/d27/f3b x:0 0 0 2024-05-13T22:48:00.169 INFO:tasks.workunit.client.0.smithi102.stdout:8/197: readlink d2/d7/d12/l29 0 2024-05-13T22:48:00.176 INFO:tasks.workunit.client.0.smithi102.stdout:1/192: chown d2/d4/d5/d19/d28/l3c 2078060 1 2024-05-13T22:48:00.178 INFO:tasks.workunit.client.0.smithi102.stdout:6/291: unlink d9/c33 0 2024-05-13T22:48:00.181 INFO:tasks.workunit.client.0.smithi102.stdout:4/230: creat d3/d19/d23/d3a/f56 x:0 0 0 2024-05-13T22:48:00.182 INFO:tasks.workunit.client.0.smithi102.stdout:7/157: rmdir d6/d2b 0 2024-05-13T22:48:00.195 INFO:tasks.workunit.client.0.smithi102.stdout:6/292: dread d9/db/d14/f3f [0,4194304] 0 2024-05-13T22:48:00.196 INFO:tasks.workunit.client.0.smithi102.stdout:2/272: rmdir d7/d9/d10/d25/d4b 0 2024-05-13T22:48:00.196 INFO:tasks.workunit.client.0.smithi102.stdout:6/293: chown d9/db/d14/c4a 27 1 2024-05-13T22:48:00.200 INFO:tasks.workunit.client.0.smithi102.stdout:8/198: link d2/d7/l2d d2/d7/d12/l3c 0 2024-05-13T22:48:00.201 INFO:tasks.workunit.client.0.smithi102.stdout:8/199: truncate d2/d7/d12/d2b/f39 557244 0 2024-05-13T22:48:00.205 INFO:tasks.workunit.client.0.smithi102.stdout:2/273: mkdir d7/d9/d10/d1b/d54/d58 0 2024-05-13T22:48:00.207 INFO:tasks.workunit.client.0.smithi102.stdout:8/200: creat d2/d7/d27/f3d x:0 0 0 2024-05-13T22:48:00.208 INFO:tasks.workunit.client.0.smithi102.stdout:8/201: readlink d2/d7/d12/l38 0 2024-05-13T22:48:00.208 INFO:tasks.workunit.client.0.smithi102.stdout:1/193: write d2/d4/fd [983427,57219] 0 2024-05-13T22:48:00.209 INFO:tasks.workunit.client.0.smithi102.stdout:8/202: write d2/d7/d12/d2b/f39 [51664,44569] 0 2024-05-13T22:48:00.213 INFO:tasks.workunit.client.0.smithi102.stdout:2/274: write d7/d9/d11/f39 [2493255,67604] 0 2024-05-13T22:48:00.215 INFO:tasks.workunit.client.0.smithi102.stdout:8/203: dread d2/f4 [4194304,4194304] 0 2024-05-13T22:48:00.216 INFO:tasks.workunit.client.0.smithi102.stdout:8/204: chown d2/d7/ce 87096 1 2024-05-13T22:48:00.221 INFO:tasks.workunit.client.0.smithi102.stdout:2/275: dread - d7/d9/d11/f38 zero size 2024-05-13T22:48:00.223 INFO:tasks.workunit.client.0.smithi102.stdout:6/294: write d9/f21 [1843402,13483] 0 2024-05-13T22:48:00.224 INFO:tasks.workunit.client.0.smithi102.stdout:2/276: chown d7/c42 26925 1 2024-05-13T22:48:00.227 INFO:tasks.workunit.client.0.smithi102.stdout:4/231: dwrite d3/d19/d23/f37 [0,4194304] 0 2024-05-13T22:48:00.228 INFO:tasks.workunit.client.0.smithi102.stdout:1/194: dread d2/d10/f21 [0,4194304] 0 2024-05-13T22:48:00.229 INFO:tasks.workunit.client.0.smithi102.stdout:8/205: dwrite d2/d7/d12/d2b/f34 [0,4194304] 0 2024-05-13T22:48:00.230 INFO:tasks.workunit.client.0.smithi102.stdout:6/295: write d9/db/d14/d37/f4b [408209,43745] 0 2024-05-13T22:48:00.231 INFO:tasks.workunit.client.0.smithi102.stdout:8/206: truncate d2/d17/f2c 531831 0 2024-05-13T22:48:00.240 INFO:tasks.workunit.client.0.smithi102.stdout:8/207: dwrite d2/d17/f26 [0,4194304] 0 2024-05-13T22:48:00.244 INFO:tasks.workunit.client.0.smithi102.stdout:6/296: mkdir d9/d25/d60 0 2024-05-13T22:48:00.245 INFO:tasks.workunit.client.0.smithi102.stdout:6/297: chown d9/d45/d56 75225 1 2024-05-13T22:48:00.245 INFO:tasks.workunit.client.0.smithi102.stdout:6/298: write f3 [2343994,64288] 0 2024-05-13T22:48:00.248 INFO:tasks.workunit.client.0.smithi102.stdout:2/277: mkdir d7/d9/d10/d25/d56/d59 0 2024-05-13T22:48:00.252 INFO:tasks.workunit.client.0.smithi102.stdout:8/208: creat d2/d7/d12/f3e x:0 0 0 2024-05-13T22:48:00.254 INFO:tasks.workunit.client.0.smithi102.stdout:1/195: symlink d2/d4/l3f 0 2024-05-13T22:48:00.260 INFO:tasks.workunit.client.0.smithi102.stdout:6/299: unlink d9/db/d14/f23 0 2024-05-13T22:48:00.260 INFO:tasks.workunit.client.0.smithi102.stdout:8/209: dwrite d2/d7/d27/f3d [0,4194304] 0 2024-05-13T22:48:00.262 INFO:tasks.workunit.client.0.smithi102.stdout:2/278: mknod d7/d9/d10/d25/d27/c5a 0 2024-05-13T22:48:00.262 INFO:tasks.workunit.client.0.smithi102.stdout:6/300: write d9/db/d14/d37/f4b [3299583,71501] 0 2024-05-13T22:48:00.267 INFO:tasks.workunit.client.0.smithi102.stdout:6/301: read f3 [1181651,5702] 0 2024-05-13T22:48:00.269 INFO:tasks.workunit.client.0.smithi102.stdout:4/232: link d3/d19/l4c d3/d4/d6/de/d1f/l57 0 2024-05-13T22:48:00.269 INFO:tasks.workunit.client.0.smithi102.stdout:6/302: write d9/db/d14/f3f [3290677,117100] 0 2024-05-13T22:48:00.270 INFO:tasks.workunit.client.0.smithi102.stdout:4/233: read - d3/d19/d23/d3a/f4a zero size 2024-05-13T22:48:00.271 INFO:tasks.workunit.client.0.smithi102.stdout:4/234: chown d3/d19/d23/d4e/f55 152 1 2024-05-13T22:48:00.272 INFO:tasks.workunit.client.0.smithi102.stdout:6/303: read d9/db/d14/d37/f4b [2339863,11931] 0 2024-05-13T22:48:00.272 INFO:tasks.workunit.client.0.smithi102.stdout:6/304: chown d9/d25/f30 87156 1 2024-05-13T22:48:00.273 INFO:tasks.workunit.client.0.smithi102.stdout:6/305: readlink d9/db/d14/d35/d47/l4d 0 2024-05-13T22:48:00.280 INFO:tasks.workunit.client.0.smithi102.stdout:8/210: dwrite d2/d7/d12/f13 [0,4194304] 0 2024-05-13T22:48:00.282 INFO:tasks.workunit.client.0.smithi102.stdout:2/279: creat d7/d9/d10/d25/d27/f5b x:0 0 0 2024-05-13T22:48:00.285 INFO:tasks.workunit.client.0.smithi102.stdout:4/235: mknod d3/d4/d6/de/d1f/c58 0 2024-05-13T22:48:00.288 INFO:tasks.workunit.client.0.smithi102.stdout:1/196: dwrite d2/d4/d26/f33 [0,4194304] 0 2024-05-13T22:48:00.288 INFO:tasks.workunit.client.0.smithi102.stdout:6/306: symlink d9/db/d14/d35/l61 0 2024-05-13T22:48:00.297 INFO:tasks.workunit.client.0.smithi102.stdout:2/280: creat d7/d15/d1e/f5c x:0 0 0 2024-05-13T22:48:00.298 INFO:tasks.workunit.client.0.smithi102.stdout:2/281: write d7/d15/f19 [4940658,30188] 0 2024-05-13T22:48:00.299 INFO:tasks.workunit.client.0.smithi102.stdout:2/282: chown d7/d9/d10/d1b/c1c 1 1 2024-05-13T22:48:00.302 INFO:tasks.workunit.client.0.smithi102.stdout:8/211: dread d2/f11 [0,4194304] 0 2024-05-13T22:48:00.303 INFO:tasks.workunit.client.0.smithi102.stdout:8/212: read - d2/d7/d27/f3b zero size 2024-05-13T22:48:00.305 INFO:tasks.workunit.client.0.smithi102.stdout:6/307: rename d9/db/d14/f2c to d9/db/d2f/f62 0 2024-05-13T22:48:00.305 INFO:tasks.workunit.client.0.smithi102.stdout:8/213: rename d2 to d2/d7/d27/d3f 22 2024-05-13T22:48:00.308 INFO:tasks.workunit.client.0.smithi102.stdout:2/283: symlink d7/d15/d1e/l5d 0 2024-05-13T22:48:00.311 INFO:tasks.workunit.client.0.smithi102.stdout:4/236: unlink d3/d4/d6/de/d1f/l57 0 2024-05-13T22:48:00.314 INFO:tasks.workunit.client.0.smithi102.stdout:1/197: unlink d2/d4/d5/d19/f39 0 2024-05-13T22:48:00.319 INFO:tasks.workunit.client.0.smithi102.stdout:4/237: dwrite d3/f35 [0,4194304] 0 2024-05-13T22:48:00.320 INFO:tasks.workunit.client.0.smithi102.stdout:8/214: mkdir d2/d7/d12/d2b/d40 0 2024-05-13T22:48:00.325 INFO:tasks.workunit.client.0.smithi102.stdout:8/215: dread d2/f22 [0,4194304] 0 2024-05-13T22:48:00.333 INFO:tasks.workunit.client.0.smithi102.stdout:1/198: mkdir d2/d4/d26/d40 0 2024-05-13T22:48:00.333 INFO:tasks.workunit.client.0.smithi102.stdout:1/199: stat d2/ff 0 2024-05-13T22:48:00.335 INFO:tasks.workunit.client.0.smithi102.stdout:1/200: write d2/d4/d5/f36 [731133,15185] 0 2024-05-13T22:48:00.335 INFO:tasks.workunit.client.0.smithi102.stdout:1/201: chown d2/l22 0 1 2024-05-13T22:48:00.337 INFO:tasks.workunit.client.0.smithi102.stdout:6/308: link d9/fa d9/db/f63 0 2024-05-13T22:48:00.339 INFO:tasks.workunit.client.0.smithi102.stdout:4/238: symlink d3/d4/d6/l59 0 2024-05-13T22:48:00.341 INFO:tasks.workunit.client.0.smithi102.stdout:2/284: rmdir d7/d9/d10/d1b/d54/d58 0 2024-05-13T22:48:00.350 INFO:tasks.workunit.client.0.smithi102.stdout:6/309: mknod d9/d25/c64 0 2024-05-13T22:48:00.353 INFO:tasks.workunit.client.0.smithi102.stdout:4/239: creat d3/d4/f5a x:0 0 0 2024-05-13T22:48:00.359 INFO:tasks.workunit.client.0.smithi102.stdout:6/310: creat d9/db/d14/d37/f65 x:0 0 0 2024-05-13T22:48:00.361 INFO:tasks.workunit.client.0.smithi102.stdout:1/202: dread d2/d4/d5/d19/d28/f24 [0,4194304] 0 2024-05-13T22:48:00.362 INFO:tasks.workunit.client.0.smithi102.stdout:4/240: truncate d3/d22/f3e 413001 0 2024-05-13T22:48:00.364 INFO:tasks.workunit.client.0.smithi102.stdout:2/285: creat d7/d9/d10/d30/d53/f5e x:0 0 0 2024-05-13T22:48:00.366 INFO:tasks.workunit.client.0.smithi102.stdout:1/203: dread d2/d4/f23 [0,4194304] 0 2024-05-13T22:48:00.368 INFO:tasks.workunit.client.0.smithi102.stdout:8/216: truncate d2/d7/d27/f3d 2886640 0 2024-05-13T22:48:00.368 INFO:tasks.workunit.client.0.smithi102.stdout:8/217: chown d2/d17 628337412 1 2024-05-13T22:48:00.370 INFO:tasks.workunit.client.0.smithi102.stdout:8/218: read d2/d17/f2c [522430,108317] 0 2024-05-13T22:48:00.370 INFO:tasks.workunit.client.0.smithi102.stdout:8/219: stat d2/d7/f9 0 2024-05-13T22:48:00.376 INFO:tasks.workunit.client.0.smithi102.stdout:8/220: dwrite d2/d7/f9 [0,4194304] 0 2024-05-13T22:48:00.379 INFO:tasks.workunit.client.0.smithi102.stdout:8/221: dread d2/d17/f33 [0,4194304] 0 2024-05-13T22:48:00.381 INFO:tasks.workunit.client.0.smithi102.stdout:8/222: write d2/d7/d12/d2b/f34 [1952691,112725] 0 2024-05-13T22:48:00.382 INFO:tasks.workunit.client.0.smithi102.stdout:4/241: unlink d3/d4/f50 0 2024-05-13T22:48:00.387 INFO:tasks.workunit.client.0.smithi102.stdout:6/311: creat d9/f66 x:0 0 0 2024-05-13T22:48:00.470 INFO:tasks.workunit.client.0.smithi102.stdout:2/286: rename d7/d15/d1e/c2f to d7/d15/c5f 0 2024-05-13T22:48:00.470 INFO:tasks.workunit.client.0.smithi102.stdout:0/267: sync 2024-05-13T22:48:00.470 INFO:tasks.workunit.client.0.smithi102.stdout:2/287: fdatasync d7/f4a 0 2024-05-13T22:48:00.506 INFO:tasks.workunit.client.0.smithi102.stdout:6/312: dread f3 [0,4194304] 0 2024-05-13T22:48:00.598 INFO:tasks.workunit.client.0.smithi102.stdout:1/204: rename d2/d4/l3f to d2/d4/d5/d19/l41 0 2024-05-13T22:48:00.600 INFO:tasks.workunit.client.0.smithi102.stdout:0/268: write d1/de/d32/d24/f30 [970542,128403] 0 2024-05-13T22:48:00.605 INFO:tasks.workunit.client.0.smithi102.stdout:2/288: getdents d7/d9/d10/d25 0 2024-05-13T22:48:00.606 INFO:tasks.workunit.client.0.smithi102.stdout:0/269: rmdir d1/de/d32/d24/d2d/d38 39 2024-05-13T22:48:00.610 INFO:tasks.workunit.client.0.smithi102.stdout:1/205: write d2/d4/d26/f33 [1719572,119471] 0 2024-05-13T22:48:00.613 INFO:tasks.workunit.client.0.smithi102.stdout:4/242: rename d3/d22/d2b/f2c to d3/d22/f5b 0 2024-05-13T22:48:00.614 INFO:tasks.workunit.client.0.smithi102.stdout:6/313: dwrite f3 [0,4194304] 0 2024-05-13T22:48:00.615 INFO:tasks.workunit.client.0.smithi102.stdout:4/243: write d3/d4/f2e [1493277,78364] 0 2024-05-13T22:48:00.615 INFO:tasks.workunit.client.0.smithi102.stdout:6/314: readlink d9/db/d14/d35/d47/l4d 0 2024-05-13T22:48:00.618 INFO:tasks.workunit.client.0.smithi102.stdout:0/270: creat d1/de/d32/d24/d2d/d38/f4b x:0 0 0 2024-05-13T22:48:00.620 INFO:tasks.workunit.client.0.smithi102.stdout:2/289: rename d7/c42 to d7/d9/d10/d25/c60 0 2024-05-13T22:48:00.624 INFO:tasks.workunit.client.0.smithi102.stdout:4/244: mknod d3/d19/d23/c5c 0 2024-05-13T22:48:00.625 INFO:tasks.workunit.client.0.smithi102.stdout:6/315: rmdir d9/db/d14/d54 39 2024-05-13T22:48:00.631 INFO:tasks.workunit.client.0.smithi102.stdout:4/245: dwrite d3/d19/d23/f37 [0,4194304] 0 2024-05-13T22:48:00.632 INFO:tasks.workunit.client.0.smithi102.stdout:6/316: creat d9/db/d14/d35/f67 x:0 0 0 2024-05-13T22:48:00.639 INFO:tasks.workunit.client.0.smithi102.stdout:0/271: creat d1/de/d32/d24/f4c x:0 0 0 2024-05-13T22:48:00.641 INFO:tasks.workunit.client.0.smithi102.stdout:4/246: fsync d3/d22/f3e 0 2024-05-13T22:48:00.642 INFO:tasks.workunit.client.0.smithi102.stdout:4/247: stat d3/d4/d6/l59 0 2024-05-13T22:48:00.643 INFO:tasks.workunit.client.0.smithi102.stdout:2/290: dread d7/d9/d10/f33 [0,4194304] 0 2024-05-13T22:48:00.645 INFO:tasks.workunit.client.0.smithi102.stdout:0/272: mkdir d1/d4d 0 2024-05-13T22:48:00.646 INFO:tasks.workunit.client.0.smithi102.stdout:0/273: chown d1/de/d32/f17 3666 1 2024-05-13T22:48:00.647 INFO:tasks.workunit.client.0.smithi102.stdout:2/291: mkdir d7/d9/d14/d61 0 2024-05-13T22:48:00.648 INFO:tasks.workunit.client.0.smithi102.stdout:2/292: chown d7/d15/d1e/f26 0 1 2024-05-13T22:48:00.653 INFO:tasks.workunit.client.0.smithi102.stdout:0/274: rmdir d1/de/d32/d24 39 2024-05-13T22:48:00.660 INFO:tasks.workunit.client.0.smithi102.stdout:0/275: getdents d1/de/d32/d24/d2d/d38 0 2024-05-13T22:48:00.662 INFO:tasks.workunit.client.0.smithi102.stdout:2/293: dread d7/ff [4194304,4194304] 0 2024-05-13T22:48:00.664 INFO:tasks.workunit.client.0.smithi102.stdout:2/294: creat d7/d9/d14/f62 x:0 0 0 2024-05-13T22:48:00.667 INFO:tasks.workunit.client.0.smithi102.stdout:2/295: symlink d7/d9/d10/d30/d53/l63 0 2024-05-13T22:48:00.668 INFO:tasks.workunit.client.0.smithi102.stdout:0/276: dwrite d1/de/d32/d24/f31 [0,4194304] 0 2024-05-13T22:48:00.674 INFO:tasks.workunit.client.0.smithi102.stdout:0/277: symlink d1/de/d32/d24/d2d/l4e 0 2024-05-13T22:48:00.674 INFO:tasks.workunit.client.0.smithi102.stdout:0/278: read d1/de/d32/f1e [454114,4832] 0 2024-05-13T22:48:00.675 INFO:tasks.workunit.client.0.smithi102.stdout:0/279: stat d1/de/d42 0 2024-05-13T22:48:00.676 INFO:tasks.workunit.client.0.smithi102.stdout:2/296: getdents d7/d9/d10/d25/d56 0 2024-05-13T22:48:00.677 INFO:tasks.workunit.client.0.smithi102.stdout:2/297: chown d7/d9/d14/l3a 0 1 2024-05-13T22:48:00.680 INFO:tasks.workunit.client.0.smithi102.stdout:0/280: mknod d1/de/c4f 0 2024-05-13T22:48:00.680 INFO:tasks.workunit.client.0.smithi102.stdout:2/298: dread d7/f32 [0,4194304] 0 2024-05-13T22:48:00.685 INFO:tasks.workunit.client.0.smithi102.stdout:2/299: creat d7/d9/d14/d31/f64 x:0 0 0 2024-05-13T22:48:00.687 INFO:tasks.workunit.client.0.smithi102.stdout:0/281: dwrite d1/de/f1c [4194304,4194304] 0 2024-05-13T22:48:00.691 INFO:tasks.workunit.client.0.smithi102.stdout:2/300: dread d7/ff [4194304,4194304] 0 2024-05-13T22:48:00.693 INFO:tasks.workunit.client.0.smithi102.stdout:2/301: write d7/ff [234346,23053] 0 2024-05-13T22:48:00.695 INFO:tasks.workunit.client.0.smithi102.stdout:2/302: write d7/d9/d10/d25/d27/f5b [800758,13544] 0 2024-05-13T22:48:00.705 INFO:tasks.workunit.client.0.smithi102.stdout:2/303: write d7/d9/f13 [2571521,111483] 0 2024-05-13T22:48:00.711 INFO:tasks.workunit.client.0.smithi102.stdout:0/282: dread d1/f5 [0,4194304] 0 2024-05-13T22:48:00.717 INFO:tasks.workunit.client.0.smithi102.stdout:0/283: dwrite d1/de/d32/f3b [0,4194304] 0 2024-05-13T22:48:00.726 INFO:tasks.workunit.client.0.smithi102.stdout:0/284: getdents d1/de/d32 0 2024-05-13T22:48:00.728 INFO:tasks.workunit.client.0.smithi102.stdout:0/285: creat d1/de/f50 x:0 0 0 2024-05-13T22:48:00.730 INFO:tasks.workunit.client.0.smithi102.stdout:0/286: fsync d1/de/d32/f36 0 2024-05-13T22:48:00.731 INFO:tasks.workunit.client.0.smithi102.stdout:0/287: write d1/de/f1d [2084675,34325] 0 2024-05-13T22:48:00.737 INFO:tasks.workunit.client.0.smithi102.stdout:0/288: dwrite d1/de/f19 [0,4194304] 0 2024-05-13T22:48:00.740 INFO:tasks.workunit.client.0.smithi102.stdout:0/289: dread d1/de/f19 [0,4194304] 0 2024-05-13T22:48:00.744 INFO:tasks.workunit.client.0.smithi102.stdout:0/290: creat d1/de/d32/d24/d2d/f51 x:0 0 0 2024-05-13T22:48:00.746 INFO:tasks.workunit.client.0.smithi102.stdout:0/291: truncate d1/de/d32/d24/d2d/f51 677615 0 2024-05-13T22:48:00.747 INFO:tasks.workunit.client.0.smithi102.stdout:0/292: stat d1/de/l23 0 2024-05-13T22:48:00.750 INFO:tasks.workunit.client.0.smithi102.stdout:0/293: rename d1/c21 to d1/de/d32/d24/d2d/d38/c52 0 2024-05-13T22:48:00.754 INFO:tasks.workunit.client.0.smithi102.stdout:0/294: dread d1/de/d32/d24/f31 [0,4194304] 0 2024-05-13T22:48:00.756 INFO:tasks.workunit.client.0.smithi102.stdout:0/295: chown d1/de/c25 147 1 2024-05-13T22:48:00.759 INFO:tasks.workunit.client.0.smithi102.stdout:0/296: creat d1/de/d32/d24/d2d/f53 x:0 0 0 2024-05-13T22:48:00.765 INFO:tasks.workunit.client.0.smithi102.stdout:0/297: dwrite d1/de/d42/f47 [0,4194304] 0 2024-05-13T22:48:00.773 INFO:tasks.workunit.client.0.smithi102.stdout:0/298: link d1/de/d42/c4a d1/de/d32/d24/d2d/d3e/c54 0 2024-05-13T22:48:00.774 INFO:tasks.workunit.client.0.smithi102.stdout:0/299: stat d1/de/d32/d24/d2d/d38/f4b 0 2024-05-13T22:48:00.777 INFO:tasks.workunit.client.0.smithi102.stdout:0/300: mknod d1/de/d32/d24/d2d/c55 0 2024-05-13T22:48:00.779 INFO:tasks.workunit.client.0.smithi102.stdout:0/301: mknod d1/d4d/c56 0 2024-05-13T22:48:00.781 INFO:tasks.workunit.client.0.smithi102.stdout:0/302: creat d1/de/d32/d24/d2d/d38/d3d/f57 x:0 0 0 2024-05-13T22:48:00.787 INFO:tasks.workunit.client.0.smithi102.stdout:5/427: sync 2024-05-13T22:48:00.789 INFO:tasks.workunit.client.0.smithi102.stdout:5/428: chown de/d25/d59/c42 11 1 2024-05-13T22:48:00.789 INFO:tasks.workunit.client.0.smithi102.stdout:0/303: link d1/de/c25 d1/de/d32/c58 0 2024-05-13T22:48:00.792 INFO:tasks.workunit.client.0.smithi102.stdout:5/429: mknod de/d75/ca2 0 2024-05-13T22:48:00.796 INFO:tasks.workunit.client.0.smithi102.stdout:5/430: dread de/d25/d2a/f2b [0,4194304] 0 2024-05-13T22:48:00.799 INFO:tasks.workunit.client.0.smithi102.stdout:5/431: mkdir de/d25/d2a/d4b/d4d/d50/da3 0 2024-05-13T22:48:00.803 INFO:tasks.workunit.client.0.smithi102.stdout:5/432: rename de/d25/d59/f40 to de/d75/fa4 0 2024-05-13T22:48:00.803 INFO:tasks.workunit.client.0.smithi102.stdout:5/433: write de/f13 [8293272,122961] 0 2024-05-13T22:48:00.809 INFO:tasks.workunit.client.0.smithi102.stdout:0/304: dread - d1/de/d32/d24/f4c zero size 2024-05-13T22:48:00.812 INFO:tasks.workunit.client.0.smithi102.stdout:5/434: getdents de/d25/d43 0 2024-05-13T22:48:00.815 INFO:tasks.workunit.client.0.smithi102.stdout:5/435: mkdir de/d25/d59/d7f/d86/da5 0 2024-05-13T22:48:00.819 INFO:tasks.workunit.client.0.smithi102.stdout:5/436: rename de/d25/d59/f24 to de/d25/d58/fa6 0 2024-05-13T22:48:00.819 INFO:tasks.workunit.client.0.smithi102.stdout:0/305: dread d1/de/d32/f1a [0,4194304] 0 2024-05-13T22:48:00.821 INFO:tasks.workunit.client.0.smithi102.stdout:0/306: read d1/de/d32/f17 [2546547,32025] 0 2024-05-13T22:48:00.824 INFO:tasks.workunit.client.0.smithi102.stdout:0/307: rmdir d1/d4d 39 2024-05-13T22:48:00.830 INFO:tasks.workunit.client.0.smithi102.stdout:0/308: rename d1/de/f1c to d1/d4d/f59 0 2024-05-13T22:48:00.833 INFO:tasks.workunit.client.0.smithi102.stdout:5/437: truncate de/d25/d2a/d4b/f76 849550 0 2024-05-13T22:48:00.835 INFO:tasks.workunit.client.0.smithi102.stdout:0/309: creat d1/de/f5a x:0 0 0 2024-05-13T22:48:00.836 INFO:tasks.workunit.client.0.smithi102.stdout:5/438: creat de/d33/d53/d9e/fa7 x:0 0 0 2024-05-13T22:48:00.840 INFO:tasks.workunit.client.0.smithi102.stdout:5/439: rename de/l7e to de/d33/d53/la8 0 2024-05-13T22:48:00.843 INFO:tasks.workunit.client.0.smithi102.stdout:5/440: rename de/d25/d2a/l45 to de/d25/d59/d7f/la9 0 2024-05-13T22:48:00.845 INFO:tasks.workunit.client.0.smithi102.stdout:5/441: mknod de/d25/d2a/d4b/d4d/d50/d6c/caa 0 2024-05-13T22:48:00.850 INFO:tasks.workunit.client.0.smithi102.stdout:0/310: truncate d1/de/f1d 315741 0 2024-05-13T22:48:00.851 INFO:tasks.workunit.client.0.smithi102.stdout:5/442: dwrite de/d25/d59/f1f [0,4194304] 0 2024-05-13T22:48:00.856 INFO:tasks.workunit.client.0.smithi102.stdout:0/311: unlink d1/d4d/c56 0 2024-05-13T22:48:00.858 INFO:tasks.workunit.client.0.smithi102.stdout:5/443: mkdir de/d25/d2a/d4b/dab 0 2024-05-13T22:48:00.865 INFO:tasks.workunit.client.0.smithi102.stdout:0/312: dwrite d1/de/d32/f17 [0,4194304] 0 2024-05-13T22:48:00.871 INFO:tasks.workunit.client.0.smithi102.stdout:0/313: getdents d1/de/d32/d24/d2d/d38/d3d 0 2024-05-13T22:48:00.878 INFO:tasks.workunit.client.0.smithi102.stdout:5/444: write de/d25/d2a/d4b/d4d/f84 [447600,111146] 0 2024-05-13T22:48:00.879 INFO:tasks.workunit.client.0.smithi102.stdout:0/314: dwrite d1/de/d32/f3b [0,4194304] 0 2024-05-13T22:48:00.881 INFO:tasks.workunit.client.0.smithi102.stdout:0/315: readlink d1/de/d32/d24/d2d/d3e/l44 0 2024-05-13T22:48:00.884 INFO:tasks.workunit.client.0.smithi102.stdout:0/316: creat d1/d4d/f5b x:0 0 0 2024-05-13T22:48:00.892 INFO:tasks.workunit.client.0.smithi102.stdout:0/317: dwrite d1/de/d32/d24/d2d/d38/d3d/f57 [0,4194304] 0 2024-05-13T22:48:00.901 INFO:tasks.workunit.client.0.smithi102.stdout:0/318: truncate d1/de/d32/f17 4171311 0 2024-05-13T22:48:00.907 INFO:tasks.workunit.client.0.smithi102.stdout:0/319: dwrite d1/de/d32/d24/d2d/f51 [0,4194304] 0 2024-05-13T22:48:00.911 INFO:tasks.workunit.client.0.smithi102.stdout:0/320: dread - d1/de/f27 zero size 2024-05-13T22:48:00.918 INFO:tasks.workunit.client.0.smithi102.stdout:0/321: dwrite d1/de/d32/d24/d2d/f53 [0,4194304] 0 2024-05-13T22:48:00.919 INFO:tasks.workunit.client.0.smithi102.stdout:0/322: fdatasync d1/de/f33 0 2024-05-13T22:48:00.920 INFO:tasks.workunit.client.0.smithi102.stdout:0/323: chown d1/de/d32/d24/d2d/d38/d3d 220 1 2024-05-13T22:48:00.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:00 smithi102 ceph-mon[28599]: pgmap v17: 65 pgs: 65 active+clean; 587 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 7.8 MiB/s rd, 61 MiB/s wr, 471 op/s 2024-05-13T22:48:00.957 INFO:tasks.workunit.client.0.smithi102.stdout:0/324: mknod d1/de/d32/d24/d2d/d3e/c5c 0 2024-05-13T22:48:00.998 INFO:tasks.workunit.client.0.smithi102.stdout:9/228: sync 2024-05-13T22:48:00.998 INFO:tasks.workunit.client.0.smithi102.stdout:3/288: sync 2024-05-13T22:48:00.999 INFO:tasks.workunit.client.0.smithi102.stdout:3/289: stat de/d10/f43 0 2024-05-13T22:48:01.001 INFO:tasks.workunit.client.0.smithi102.stdout:9/229: creat d1/d6/d45/d3e/f56 x:0 0 0 2024-05-13T22:48:01.002 INFO:tasks.workunit.client.0.smithi102.stdout:9/230: chown d1/d6/db/d30/c4d 12 1 2024-05-13T22:48:01.006 INFO:tasks.workunit.client.0.smithi102.stdout:3/290: rename de/d10/d2a/d48 to de/d10/d2a/d33/d61 0 2024-05-13T22:48:01.007 INFO:tasks.workunit.client.0.smithi102.stdout:9/231: dread d1/d6/f11 [0,4194304] 0 2024-05-13T22:48:01.008 INFO:tasks.workunit.client.0.smithi102.stdout:9/232: fdatasync d1/d6/d21/f42 0 2024-05-13T22:48:01.012 INFO:tasks.workunit.client.0.smithi102.stdout:9/233: mknod d1/d6/db/d30/c57 0 2024-05-13T22:48:01.013 INFO:tasks.workunit.client.0.smithi102.stdout:3/291: dwrite de/d10/f44 [0,4194304] 0 2024-05-13T22:48:01.025 INFO:tasks.workunit.client.0.smithi102.stdout:3/292: link de/c16 de/d1c/c62 0 2024-05-13T22:48:01.025 INFO:tasks.workunit.client.0.smithi102.stdout:3/293: readlink de/d10/l24 0 2024-05-13T22:48:01.028 INFO:tasks.workunit.client.0.smithi102.stdout:3/294: mkdir de/d10/d50/d63 0 2024-05-13T22:48:01.038 INFO:tasks.workunit.client.0.smithi102.stdout:0/325: write d1/de/d32/f18 [7992384,30263] 0 2024-05-13T22:48:01.042 INFO:tasks.workunit.client.0.smithi102.stdout:0/326: mknod d1/d4d/c5d 0 2024-05-13T22:48:01.047 INFO:tasks.workunit.client.0.smithi102.stdout:3/295: rmdir de/d21 39 2024-05-13T22:48:01.055 INFO:tasks.workunit.client.0.smithi102.stdout:3/296: dwrite de/d10/f44 [0,4194304] 0 2024-05-13T22:48:01.057 INFO:tasks.workunit.client.0.smithi102.stdout:3/297: truncate de/d10/d2a/d33/d61/f4c 287024 0 2024-05-13T22:48:01.065 INFO:tasks.workunit.client.0.smithi102.stdout:3/298: dwrite de/d10/d2a/d33/d61/f56 [0,4194304] 0 2024-05-13T22:48:01.068 INFO:tasks.workunit.client.0.smithi102.stdout:3/299: creat de/d10/d2a/d33/d61/f64 x:0 0 0 2024-05-13T22:48:01.069 INFO:tasks.workunit.client.0.smithi102.stdout:3/300: chown de/d10/d2a/d33/d61/f56 176312 1 2024-05-13T22:48:01.075 INFO:tasks.workunit.client.0.smithi102.stdout:3/301: dwrite de/d10/f1d [0,4194304] 0 2024-05-13T22:48:01.076 INFO:tasks.workunit.client.0.smithi102.stdout:3/302: fdatasync de/d10/d50/f47 0 2024-05-13T22:48:01.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:00 smithi130 ceph-mon[36591]: pgmap v17: 65 pgs: 65 active+clean; 587 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 7.8 MiB/s rd, 61 MiB/s wr, 471 op/s 2024-05-13T22:48:01.190 INFO:tasks.workunit.client.0.smithi102.stdout:7/158: sync 2024-05-13T22:48:01.193 INFO:tasks.workunit.client.0.smithi102.stdout:7/159: mknod d6/d15/d21/c2d 0 2024-05-13T22:48:01.199 INFO:tasks.workunit.client.0.smithi102.stdout:7/160: dwrite d6/d15/f1e [4194304,4194304] 0 2024-05-13T22:48:01.200 INFO:tasks.workunit.client.0.smithi102.stdout:7/161: write d6/f2a [16230,51501] 0 2024-05-13T22:48:01.200 INFO:tasks.workunit.client.0.smithi102.stdout:7/162: chown d6/l1c 11049887 1 2024-05-13T22:48:01.204 INFO:tasks.workunit.client.0.smithi102.stdout:7/163: write d6/d13/de/f12 [566769,4146] 0 2024-05-13T22:48:01.204 INFO:tasks.workunit.client.0.smithi102.stdout:7/164: chown d6/d13/c10 59999466 1 2024-05-13T22:48:01.308 INFO:tasks.workunit.client.0.smithi102.stdout:7/165: truncate f3 2432560 0 2024-05-13T22:48:01.312 INFO:tasks.workunit.client.0.smithi102.stdout:7/166: dread d6/fc [0,4194304] 0 2024-05-13T22:48:01.316 INFO:tasks.workunit.client.0.smithi102.stdout:7/167: link d6/d13/f16 d6/f2e 0 2024-05-13T22:48:01.391 INFO:tasks.workunit.client.0.smithi102.stdout:8/223: sync 2024-05-13T22:48:01.391 INFO:tasks.workunit.client.0.smithi102.stdout:7/168: creat d6/d13/d2c/f2f x:0 0 0 2024-05-13T22:48:01.392 INFO:tasks.workunit.client.0.smithi102.stdout:7/169: write d6/f8 [5799172,31451] 0 2024-05-13T22:48:01.393 INFO:tasks.workunit.client.0.smithi102.stdout:7/170: dread - d6/d13/d2c/f2f zero size 2024-05-13T22:48:01.395 INFO:tasks.workunit.client.0.smithi102.stdout:8/224: mkdir d2/d7/d25/d41 0 2024-05-13T22:48:01.395 INFO:tasks.workunit.client.0.smithi102.stdout:7/171: symlink d6/d15/l30 0 2024-05-13T22:48:01.396 INFO:tasks.workunit.client.0.smithi102.stdout:7/172: read d6/f2a [38068,61240] 0 2024-05-13T22:48:01.411 INFO:tasks.workunit.client.0.smithi102.stdout:8/225: dread d2/d7/d12/f1c [0,4194304] 0 2024-05-13T22:48:01.469 INFO:tasks.workunit.client.0.smithi102.stdout:7/173: mknod d6/d22/c31 0 2024-05-13T22:48:01.471 INFO:tasks.workunit.client.0.smithi102.stdout:7/174: read d6/d13/de/f12 [2162454,111462] 0 2024-05-13T22:48:01.472 INFO:tasks.workunit.client.0.smithi102.stdout:8/226: creat d2/d36/f42 x:0 0 0 2024-05-13T22:48:01.473 INFO:tasks.workunit.client.0.smithi102.stdout:7/175: read - d6/d13/d2c/f2f zero size 2024-05-13T22:48:01.478 INFO:tasks.workunit.client.0.smithi102.stdout:7/176: unlink d6/d15/l30 0 2024-05-13T22:48:01.480 INFO:tasks.workunit.client.0.smithi102.stdout:8/227: getdents d2/d7/d12/d2b/d40 0 2024-05-13T22:48:01.486 INFO:tasks.workunit.client.0.smithi102.stdout:8/228: dwrite d2/d36/f42 [0,4194304] 0 2024-05-13T22:48:01.493 INFO:tasks.workunit.client.0.smithi102.stdout:8/229: rename d2/d17/l37 to d2/d7/d25/d41/l43 0 2024-05-13T22:48:01.496 INFO:tasks.workunit.client.0.smithi102.stdout:8/230: truncate d2/d7/d12/d2b/f35 5664129 0 2024-05-13T22:48:01.535 INFO:tasks.workunit.client.0.smithi102.stdout:7/177: truncate d6/fc 5038542 0 2024-05-13T22:48:01.535 INFO:tasks.workunit.client.0.smithi102.stdout:7/178: readlink d6/l1f 0 2024-05-13T22:48:01.538 INFO:tasks.workunit.client.0.smithi102.stdout:7/179: fdatasync d6/d13/de/f12 0 2024-05-13T22:48:01.538 INFO:tasks.workunit.client.0.smithi102.stdout:7/180: dread - d6/d13/de/f29 zero size 2024-05-13T22:48:01.544 INFO:tasks.workunit.client.0.smithi102.stdout:8/231: write d2/d7/d27/f3d [3438839,6766] 0 2024-05-13T22:48:01.546 INFO:tasks.workunit.client.0.smithi102.stdout:8/232: fsync d2/d17/f2c 0 2024-05-13T22:48:01.563 INFO:tasks.workunit.client.0.smithi102.stdout:7/181: dwrite f4 [0,4194304] 0 2024-05-13T22:48:01.568 INFO:tasks.workunit.client.0.smithi102.stdout:7/182: unlink d6/d15/f28 0 2024-05-13T22:48:01.568 INFO:tasks.workunit.client.0.smithi102.stdout:8/233: dwrite d2/d17/f33 [0,4194304] 0 2024-05-13T22:48:01.569 INFO:tasks.workunit.client.0.smithi102.stdout:8/234: fdatasync d2/f22 0 2024-05-13T22:48:01.572 INFO:tasks.workunit.client.0.smithi102.stdout:7/183: rmdir d6/d13/d2c 39 2024-05-13T22:48:01.574 INFO:tasks.workunit.client.0.smithi102.stdout:8/235: dread d2/d7/d12/f23 [0,4194304] 0 2024-05-13T22:48:01.574 INFO:tasks.workunit.client.0.smithi102.stdout:8/236: fdatasync d2/d36/f42 0 2024-05-13T22:48:01.585 INFO:tasks.workunit.client.0.smithi102.stdout:7/184: mkdir d6/d13/d2c/d32 0 2024-05-13T22:48:01.585 INFO:tasks.workunit.client.0.smithi102.stdout:7/185: chown d6/l25 362 1 2024-05-13T22:48:01.586 INFO:tasks.workunit.client.0.smithi102.stdout:8/237: rename d2/d7/l2d to d2/d7/d25/l44 0 2024-05-13T22:48:01.587 INFO:tasks.workunit.client.0.smithi102.stdout:8/238: chown d2/d7 436102836 1 2024-05-13T22:48:01.588 INFO:tasks.workunit.client.0.smithi102.stdout:7/186: creat d6/d22/f33 x:0 0 0 2024-05-13T22:48:01.594 INFO:tasks.workunit.client.0.smithi102.stdout:8/239: dwrite d2/d7/d27/f3b [0,4194304] 0 2024-05-13T22:48:01.596 INFO:tasks.workunit.client.0.smithi102.stdout:7/187: dwrite d6/f8 [4194304,4194304] 0 2024-05-13T22:48:01.596 INFO:tasks.workunit.client.0.smithi102.stdout:8/240: chown d2/d7/f9 24650 1 2024-05-13T22:48:01.600 INFO:tasks.workunit.client.0.smithi102.stdout:8/241: rename d2/f11 to d2/d17/f45 0 2024-05-13T22:48:01.608 INFO:tasks.workunit.client.0.smithi102.stdout:7/188: link d6/d15/f1b d6/d15/d21/f34 0 2024-05-13T22:48:01.609 INFO:tasks.workunit.client.0.smithi102.stdout:8/242: dwrite d2/d7/d27/f3d [0,4194304] 0 2024-05-13T22:48:01.610 INFO:tasks.workunit.client.0.smithi102.stdout:8/243: chown d2/d7/d27/f31 3 1 2024-05-13T22:48:01.612 INFO:tasks.workunit.client.0.smithi102.stdout:8/244: creat d2/d7/d27/f46 x:0 0 0 2024-05-13T22:48:01.737 INFO:tasks.workunit.client.0.smithi102.stdout:1/206: sync 2024-05-13T22:48:01.738 INFO:tasks.workunit.client.0.smithi102.stdout:1/207: truncate d2/d4/f20 1196102 0 2024-05-13T22:48:01.738 INFO:tasks.workunit.client.0.smithi102.stdout:1/208: read d2/d10/f21 [795181,12405] 0 2024-05-13T22:48:01.742 INFO:tasks.workunit.client.0.smithi102.stdout:1/209: symlink d2/d4/d26/d40/l42 0 2024-05-13T22:48:01.745 INFO:tasks.workunit.client.0.smithi102.stdout:1/210: dread d2/d4/f23 [0,4194304] 0 2024-05-13T22:48:01.745 INFO:tasks.workunit.client.0.smithi102.stdout:1/211: fdatasync d2/d4/d5/f36 0 2024-05-13T22:48:01.750 INFO:tasks.workunit.client.0.smithi102.stdout:1/212: link d2/d4/d5/d19/d28/c3e d2/d4/c43 0 2024-05-13T22:48:01.751 INFO:tasks.workunit.client.0.smithi102.stdout:1/213: creat d2/d10/f44 x:0 0 0 2024-05-13T22:48:01.753 INFO:tasks.workunit.client.0.smithi102.stdout:1/214: mkdir d2/d45 0 2024-05-13T22:48:01.781 INFO:tasks.workunit.client.0.smithi102.stdout:5/445: sync 2024-05-13T22:48:01.781 INFO:tasks.workunit.client.0.smithi102.stdout:4/248: sync 2024-05-13T22:48:01.781 INFO:tasks.workunit.client.0.smithi102.stdout:0/327: sync 2024-05-13T22:48:01.781 INFO:tasks.workunit.client.0.smithi102.stdout:3/303: sync 2024-05-13T22:48:01.782 INFO:tasks.workunit.client.0.smithi102.stdout:2/304: sync 2024-05-13T22:48:01.782 INFO:tasks.workunit.client.0.smithi102.stdout:6/317: sync 2024-05-13T22:48:01.782 INFO:tasks.workunit.client.0.smithi102.stdout:9/234: sync 2024-05-13T22:48:01.783 INFO:tasks.workunit.client.0.smithi102.stdout:5/446: fsync de/d25/d43/f94 0 2024-05-13T22:48:01.783 INFO:tasks.workunit.client.0.smithi102.stdout:0/328: readlink d1/de/d32/d24/l34 0 2024-05-13T22:48:01.786 INFO:tasks.workunit.client.0.smithi102.stdout:6/318: chown d9/db/c38 5093 1 2024-05-13T22:48:01.787 INFO:tasks.workunit.client.0.smithi102.stdout:3/304: mknod de/d10/d2a/d4b/c65 0 2024-05-13T22:48:01.787 INFO:tasks.workunit.client.0.smithi102.stdout:5/447: truncate de/d25/d2a/f56 4612012 0 2024-05-13T22:48:01.788 INFO:tasks.workunit.client.0.smithi102.stdout:1/215: rmdir d2 39 2024-05-13T22:48:01.790 INFO:tasks.workunit.client.0.smithi102.stdout:9/235: symlink d1/d6/d3b/l58 0 2024-05-13T22:48:01.791 INFO:tasks.workunit.client.0.smithi102.stdout:5/448: read de/d25/d2a/f56 [2798096,23904] 0 2024-05-13T22:48:01.793 INFO:tasks.workunit.client.0.smithi102.stdout:4/249: dwrite d3/d19/d23/d4e/f55 [0,4194304] 0 2024-05-13T22:48:01.793 INFO:tasks.workunit.client.0.smithi102.stdout:2/305: rename d7/d9/d14/l4d to d7/d9/d10/d30/l65 0 2024-05-13T22:48:01.795 INFO:tasks.workunit.client.0.smithi102.stdout:5/449: read - de/d25/d2c/f96 zero size 2024-05-13T22:48:01.798 INFO:tasks.workunit.client.0.smithi102.stdout:6/319: dwrite f0 [0,4194304] 0 2024-05-13T22:48:01.801 INFO:tasks.workunit.client.0.smithi102.stdout:4/250: readlink d3/d22/d2b/l30 0 2024-05-13T22:48:01.802 INFO:tasks.workunit.client.0.smithi102.stdout:0/329: creat d1/de/d32/f5e x:0 0 0 2024-05-13T22:48:01.804 INFO:tasks.workunit.client.0.smithi102.stdout:1/216: readlink d2/d4/d26/l2c 0 2024-05-13T22:48:01.807 INFO:tasks.workunit.client.0.smithi102.stdout:2/306: dwrite d7/d9/d11/f55 [8388608,4194304] 0 2024-05-13T22:48:01.808 INFO:tasks.workunit.client.0.smithi102.stdout:2/307: chown d7/d9/d14/l3c 1118295 1 2024-05-13T22:48:01.810 INFO:tasks.workunit.client.0.smithi102.stdout:2/308: write d7/d15/d1e/f5c [396719,11316] 0 2024-05-13T22:48:01.810 INFO:tasks.workunit.client.0.smithi102.stdout:5/450: symlink de/d25/d59/d7f/lac 0 2024-05-13T22:48:01.811 INFO:tasks.workunit.client.0.smithi102.stdout:5/451: write de/d25/d43/f94 [79836,98412] 0 2024-05-13T22:48:01.813 INFO:tasks.workunit.client.0.smithi102.stdout:4/251: mkdir d3/d19/d23/d39/d5d 0 2024-05-13T22:48:01.820 INFO:tasks.workunit.client.0.smithi102.stdout:5/452: truncate de/f11 2171573 0 2024-05-13T22:48:01.822 INFO:tasks.workunit.client.0.smithi102.stdout:2/309: rename d7/d9/d10/d1b/c1c to d7/d9/d14/d31/c66 0 2024-05-13T22:48:01.825 INFO:tasks.workunit.client.0.smithi102.stdout:2/310: write d7/d15/d1e/f5c [709393,113758] 0 2024-05-13T22:48:01.829 INFO:tasks.workunit.client.0.smithi102.stdout:5/453: dwrite de/d25/d2a/fa0 [0,4194304] 0 2024-05-13T22:48:01.830 INFO:tasks.workunit.client.0.smithi102.stdout:6/320: rmdir d9/d25/d28 0 2024-05-13T22:48:01.831 INFO:tasks.workunit.client.0.smithi102.stdout:6/321: readlink d9/d45/l3e 0 2024-05-13T22:48:01.832 INFO:tasks.workunit.client.0.smithi102.stdout:5/454: write de/d25/d2c/f71 [585760,95552] 0 2024-05-13T22:48:01.834 INFO:tasks.workunit.client.0.smithi102.stdout:0/330: dwrite d1/de/d32/d24/f3c [4194304,4194304] 0 2024-05-13T22:48:01.837 INFO:tasks.workunit.client.0.smithi102.stdout:1/217: creat d2/f46 x:0 0 0 2024-05-13T22:48:01.843 INFO:tasks.workunit.client.0.smithi102.stdout:9/236: dwrite d1/f31 [0,4194304] 0 2024-05-13T22:48:01.846 INFO:tasks.workunit.client.0.smithi102.stdout:2/311: mkdir d7/d15/d67 0 2024-05-13T22:48:01.847 INFO:tasks.workunit.client.0.smithi102.stdout:2/312: write d7/d15/f52 [423880,31554] 0 2024-05-13T22:48:01.849 INFO:tasks.workunit.client.0.smithi102.stdout:2/313: write d7/d9/f29 [3438954,10394] 0 2024-05-13T22:48:01.851 INFO:tasks.workunit.client.0.smithi102.stdout:2/314: read d7/d15/d1e/f26 [96898,45743] 0 2024-05-13T22:48:01.851 INFO:tasks.workunit.client.0.smithi102.stdout:5/455: fsync de/f1e 0 2024-05-13T22:48:01.857 INFO:tasks.workunit.client.0.smithi102.stdout:1/218: rename d2/d4/d26/f29 to d2/d45/f47 0 2024-05-13T22:48:01.865 INFO:tasks.workunit.client.0.smithi102.stdout:9/237: symlink d1/d6/l59 0 2024-05-13T22:48:01.865 INFO:tasks.workunit.client.0.smithi102.stdout:9/238: stat d1/d6/db/f1c 0 2024-05-13T22:48:01.867 INFO:tasks.workunit.client.0.smithi102.stdout:0/331: creat d1/de/d32/d24/d2d/f5f x:0 0 0 2024-05-13T22:48:01.873 INFO:tasks.workunit.client.0.smithi102.stdout:5/456: symlink de/d25/d2a/d4b/d4d/d50/da3/lad 0 2024-05-13T22:48:01.874 INFO:tasks.workunit.client.0.smithi102.stdout:5/457: chown de/d25/d59/f49 362 1 2024-05-13T22:48:01.875 INFO:tasks.workunit.client.0.smithi102.stdout:9/239: symlink d1/d6/db/d30/l5a 0 2024-05-13T22:48:01.879 INFO:tasks.workunit.client.0.smithi102.stdout:6/322: getdents d9/db/d14/d54 0 2024-05-13T22:48:01.882 INFO:tasks.workunit.client.0.smithi102.stdout:5/458: dwrite de/d33/f7b [0,4194304] 0 2024-05-13T22:48:01.882 INFO:tasks.workunit.client.0.smithi102.stdout:9/240: mkdir d1/d6/d3b/d5b 0 2024-05-13T22:48:01.882 INFO:tasks.workunit.client.0.smithi102.stdout:1/219: write d2/d45/f47 [1804807,14272] 0 2024-05-13T22:48:01.884 INFO:tasks.workunit.client.0.smithi102.stdout:5/459: chown de/d25/d2a/d4b/l8d 6078711 1 2024-05-13T22:48:01.887 INFO:tasks.workunit.client.0.smithi102.stdout:6/323: mknod d9/db/d14/d37/c68 0 2024-05-13T22:48:01.889 INFO:tasks.workunit.client.0.smithi102.stdout:1/220: dread d2/d10/f31 [0,4194304] 0 2024-05-13T22:48:01.895 INFO:tasks.workunit.client.0.smithi102.stdout:5/460: dread de/d25/d59/d6f/f72 [0,4194304] 0 2024-05-13T22:48:01.895 INFO:tasks.workunit.client.0.smithi102.stdout:6/324: unlink d9/db/d14/c34 0 2024-05-13T22:48:01.897 INFO:tasks.workunit.client.0.smithi102.stdout:5/461: dread - de/d25/d59/f8f zero size 2024-05-13T22:48:01.897 INFO:tasks.workunit.client.0.smithi102.stdout:1/221: dwrite d2/d4/d5/fa [0,4194304] 0 2024-05-13T22:48:01.898 INFO:tasks.workunit.client.0.smithi102.stdout:5/462: chown de/f35 403909 1 2024-05-13T22:48:01.899 INFO:tasks.workunit.client.0.smithi102.stdout:5/463: write de/f61 [1729681,108978] 0 2024-05-13T22:48:01.900 INFO:tasks.workunit.client.0.smithi102.stdout:9/241: creat d1/f5c x:0 0 0 2024-05-13T22:48:01.901 INFO:tasks.workunit.client.0.smithi102.stdout:6/325: mkdir d9/db/d14/d54/d69 0 2024-05-13T22:48:01.904 INFO:tasks.workunit.client.0.smithi102.stdout:5/464: creat de/d25/d43/d7a/fae x:0 0 0 2024-05-13T22:48:01.905 INFO:tasks.workunit.client.0.smithi102.stdout:0/332: dwrite d1/f2c [0,4194304] 0 2024-05-13T22:48:01.907 INFO:tasks.workunit.client.0.smithi102.stdout:9/242: mkdir d1/d5d 0 2024-05-13T22:48:01.908 INFO:tasks.workunit.client.0.smithi102.stdout:1/222: link d2/f2b d2/d45/f48 0 2024-05-13T22:48:01.909 INFO:tasks.workunit.client.0.smithi102.stdout:6/326: mkdir d9/d25/d60/d6a 0 2024-05-13T22:48:01.909 INFO:tasks.workunit.client.0.smithi102.stdout:1/223: read d2/d4/f23 [1039740,114202] 0 2024-05-13T22:48:01.910 INFO:tasks.workunit.client.0.smithi102.stdout:5/465: rmdir de/d33/d53 39 2024-05-13T22:48:01.912 INFO:tasks.workunit.client.0.smithi102.stdout:1/224: dread d2/d4/f23 [0,4194304] 0 2024-05-13T22:48:01.915 INFO:tasks.workunit.client.0.smithi102.stdout:5/466: symlink de/d75/laf 0 2024-05-13T22:48:01.917 INFO:tasks.workunit.client.0.smithi102.stdout:5/467: read de/f4f [655834,40994] 0 2024-05-13T22:48:01.919 INFO:tasks.workunit.client.0.smithi102.stdout:1/225: mknod d2/d4/d5/d19/d28/c49 0 2024-05-13T22:48:01.919 INFO:tasks.workunit.client.0.smithi102.stdout:5/468: fdatasync de/d25/d2a/d4b/d4d/f84 0 2024-05-13T22:48:01.924 INFO:tasks.workunit.client.0.smithi102.stdout:6/327: symlink d9/db/l6b 0 2024-05-13T22:48:01.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:01 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:48:01.927 INFO:tasks.workunit.client.0.smithi102.stdout:1/226: dwrite d2/d10/f44 [0,4194304] 0 2024-05-13T22:48:01.929 INFO:tasks.workunit.client.0.smithi102.stdout:0/333: creat d1/f60 x:0 0 0 2024-05-13T22:48:01.930 INFO:tasks.workunit.client.0.smithi102.stdout:0/334: write d1/de/d32/d24/f3c [6190170,4588] 0 2024-05-13T22:48:01.958 INFO:tasks.workunit.client.0.smithi102.stdout:7/189: sync 2024-05-13T22:48:01.958 INFO:tasks.workunit.client.0.smithi102.stdout:8/245: sync 2024-05-13T22:48:01.959 INFO:tasks.workunit.client.0.smithi102.stdout:5/469: fsync de/f14 0 2024-05-13T22:48:01.962 INFO:tasks.workunit.client.0.smithi102.stdout:9/243: getdents d1/d6/db 0 2024-05-13T22:48:01.964 INFO:tasks.workunit.client.0.smithi102.stdout:9/244: chown d1/d6/d3b 654069954 1 2024-05-13T22:48:01.964 INFO:tasks.workunit.client.0.smithi102.stdout:7/190: mknod d6/d22/c35 0 2024-05-13T22:48:01.966 INFO:tasks.workunit.client.0.smithi102.stdout:9/245: dread - d1/d6/d3b/f38 zero size 2024-05-13T22:48:01.968 INFO:tasks.workunit.client.0.smithi102.stdout:1/227: dread d2/f2b [0,4194304] 0 2024-05-13T22:48:01.969 INFO:tasks.workunit.client.0.smithi102.stdout:5/470: symlink de/d25/d59/d7f/d86/da5/lb0 0 2024-05-13T22:48:01.970 INFO:tasks.workunit.client.0.smithi102.stdout:0/335: dread d1/de/d32/f17 [0,4194304] 0 2024-05-13T22:48:01.970 INFO:tasks.workunit.client.0.smithi102.stdout:7/191: truncate d6/d22/f24 740627 0 2024-05-13T22:48:01.971 INFO:tasks.workunit.client.0.smithi102.stdout:8/246: dwrite d2/d7/d27/f46 [0,4194304] 0 2024-05-13T22:48:01.974 INFO:tasks.workunit.client.0.smithi102.stdout:1/228: write d2/d4/d5/f36 [837135,69626] 0 2024-05-13T22:48:01.974 INFO:tasks.workunit.client.0.smithi102.stdout:5/471: stat de/d25/d2a/d4b/d4d/d50/d6c 0 2024-05-13T22:48:01.974 INFO:tasks.workunit.client.0.smithi102.stdout:9/246: chown d1/d6/c19 2747140 1 2024-05-13T22:48:01.976 INFO:tasks.workunit.client.0.smithi102.stdout:0/336: mkdir d1/de/d42/d61 0 2024-05-13T22:48:01.979 INFO:tasks.workunit.client.0.smithi102.stdout:8/247: creat d2/d7/d12/f47 x:0 0 0 2024-05-13T22:48:01.979 INFO:tasks.workunit.client.0.smithi102.stdout:9/247: chown d1/f7 24 1 2024-05-13T22:48:01.981 INFO:tasks.workunit.client.0.smithi102.stdout:7/192: rename d6/d13/de/l17 to d6/d13/d2c/l36 0 2024-05-13T22:48:01.983 INFO:tasks.workunit.client.0.smithi102.stdout:1/229: symlink d2/d10/l4a 0 2024-05-13T22:48:01.984 INFO:tasks.workunit.client.0.smithi102.stdout:5/472: mkdir de/d25/d43/d7a/db1 0 2024-05-13T22:48:01.985 INFO:tasks.workunit.client.0.smithi102.stdout:0/337: truncate d1/f4 4642519 0 2024-05-13T22:48:01.985 INFO:tasks.workunit.client.0.smithi102.stdout:5/473: write de/d25/d2c/f78 [887686,34585] 0 2024-05-13T22:48:01.986 INFO:tasks.workunit.client.0.smithi102.stdout:0/338: fdatasync d1/f9 0 2024-05-13T22:48:01.988 INFO:tasks.workunit.client.0.smithi102.stdout:9/248: rmdir d1/d6/d45 39 2024-05-13T22:48:01.989 INFO:tasks.workunit.client.0.smithi102.stdout:9/249: write d1/d6/d3b/f38 [881255,17477] 0 2024-05-13T22:48:01.993 INFO:tasks.workunit.client.0.smithi102.stdout:5/474: readlink de/d25/d59/d6f/l9a 0 2024-05-13T22:48:01.995 INFO:tasks.workunit.client.0.smithi102.stdout:8/248: fdatasync d2/f4 0 2024-05-13T22:48:01.996 INFO:tasks.workunit.client.0.smithi102.stdout:8/249: chown d2/l1b 2 1 2024-05-13T22:48:02.000 INFO:tasks.workunit.client.0.smithi102.stdout:7/193: dread d6/d13/de/fb [0,4194304] 0 2024-05-13T22:48:02.001 INFO:tasks.workunit.client.0.smithi102.stdout:5/475: dwrite de/f61 [0,4194304] 0 2024-05-13T22:48:02.007 INFO:tasks.workunit.client.0.smithi102.stdout:1/230: creat d2/f4b x:0 0 0 2024-05-13T22:48:02.007 INFO:tasks.workunit.client.0.smithi102.stdout:8/250: dwrite d2/d7/d12/f3e [0,4194304] 0 2024-05-13T22:48:02.011 INFO:tasks.workunit.client.0.smithi102.stdout:7/194: dwrite d6/d15/f1e [4194304,4194304] 0 2024-05-13T22:48:02.020 INFO:tasks.workunit.client.0.smithi102.stdout:0/339: creat d1/de/d32/d24/f62 x:0 0 0 2024-05-13T22:48:02.020 INFO:tasks.workunit.client.0.smithi102.stdout:0/340: readlink d1/de/d32/d24/l34 0 2024-05-13T22:48:02.023 INFO:tasks.workunit.client.0.smithi102.stdout:8/251: creat d2/d7/d25/d41/f48 x:0 0 0 2024-05-13T22:48:02.025 INFO:tasks.workunit.client.0.smithi102.stdout:7/195: creat d6/d15/f37 x:0 0 0 2024-05-13T22:48:02.027 INFO:tasks.workunit.client.0.smithi102.stdout:5/476: symlink de/d25/d2a/d4b/dab/lb2 0 2024-05-13T22:48:02.028 INFO:tasks.workunit.client.0.smithi102.stdout:7/196: write d6/d15/d21/f26 [521477,25039] 0 2024-05-13T22:48:02.031 INFO:tasks.workunit.client.0.smithi102.stdout:9/250: link d1/d6/d45/d3e/f4c d1/d6/d45/f5e 0 2024-05-13T22:48:02.032 INFO:tasks.workunit.client.0.smithi102.stdout:0/341: dwrite d1/de/f33 [0,4194304] 0 2024-05-13T22:48:02.033 INFO:tasks.workunit.client.0.smithi102.stdout:5/477: chown de/d25/d59/d28/c30 27343612 1 2024-05-13T22:48:02.036 INFO:tasks.workunit.client.0.smithi102.stdout:9/251: stat d1/d6/db/d4a/f52 0 2024-05-13T22:48:02.037 INFO:tasks.workunit.client.0.smithi102.stdout:5/478: chown de/d25/f47 31081 1 2024-05-13T22:48:02.041 INFO:tasks.workunit.client.0.smithi102.stdout:8/252: dwrite d2/d7/d27/f3b [4194304,4194304] 0 2024-05-13T22:48:02.041 INFO:tasks.workunit.client.0.smithi102.stdout:9/252: chown d1/d6/d45/d3e/f56 24108 1 2024-05-13T22:48:02.045 INFO:tasks.workunit.client.0.smithi102.stdout:0/342: fsync d1/de/d32/f3f 0 2024-05-13T22:48:02.045 INFO:tasks.workunit.client.0.smithi102.stdout:9/253: chown d1/d6/d45/d3e 552 1 2024-05-13T22:48:02.047 INFO:tasks.workunit.client.0.smithi102.stdout:7/197: dwrite f4 [4194304,4194304] 0 2024-05-13T22:48:02.047 INFO:tasks.workunit.client.0.smithi102.stdout:9/254: chown d1/d6/db 6785120 1 2024-05-13T22:48:02.049 INFO:tasks.workunit.client.0.smithi102.stdout:8/253: mkdir d2/d36/d49 0 2024-05-13T22:48:02.051 INFO:tasks.workunit.client.0.smithi102.stdout:1/231: dwrite d2/f2b [0,4194304] 0 2024-05-13T22:48:02.055 INFO:tasks.workunit.client.0.smithi102.stdout:5/479: write f9 [4208959,130919] 0 2024-05-13T22:48:02.059 INFO:tasks.workunit.client.0.smithi102.stdout:1/232: chown d2/d4/d26/l2d 966751 1 2024-05-13T22:48:02.060 INFO:tasks.workunit.client.0.smithi102.stdout:0/343: mkdir d1/de/d32/d63 0 2024-05-13T22:48:02.064 INFO:tasks.workunit.client.0.smithi102.stdout:8/254: mknod d2/d36/d49/c4a 0 2024-05-13T22:48:02.065 INFO:tasks.workunit.client.0.smithi102.stdout:8/255: fdatasync d2/d7/d12/d2b/f34 0 2024-05-13T22:48:02.066 INFO:tasks.workunit.client.0.smithi102.stdout:5/480: mknod de/d25/d2a/d4b/d4d/d50/d99/cb3 0 2024-05-13T22:48:02.067 INFO:tasks.workunit.client.0.smithi102.stdout:1/233: mkdir d2/d4/d26/d4c 0 2024-05-13T22:48:02.071 INFO:tasks.workunit.client.0.smithi102.stdout:7/198: getdents d6/d13 0 2024-05-13T22:48:02.073 INFO:tasks.workunit.client.0.smithi102.stdout:1/234: creat d2/d4/d5/d19/d28/f4d x:0 0 0 2024-05-13T22:48:02.074 INFO:tasks.workunit.client.0.smithi102.stdout:0/344: creat d1/de/d32/d24/d2d/f64 x:0 0 0 2024-05-13T22:48:02.074 INFO:tasks.workunit.client.0.smithi102.stdout:7/199: chown d6/d15/f37 881506349 1 2024-05-13T22:48:02.076 INFO:tasks.workunit.client.0.smithi102.stdout:9/255: dwrite d1/f2b [0,4194304] 0 2024-05-13T22:48:02.076 INFO:tasks.workunit.client.0.smithi102.stdout:5/481: creat de/d25/d59/fb4 x:0 0 0 2024-05-13T22:48:02.083 INFO:tasks.workunit.client.0.smithi102.stdout:9/256: dread d1/f2b [0,4194304] 0 2024-05-13T22:48:02.085 INFO:tasks.workunit.client.0.smithi102.stdout:9/257: fsync d1/d6/d21/f48 0 2024-05-13T22:48:02.086 INFO:tasks.workunit.client.0.smithi102.stdout:5/482: dwrite de/d25/d59/f5d [0,4194304] 0 2024-05-13T22:48:02.086 INFO:tasks.workunit.client.0.smithi102.stdout:1/235: creat d2/d4/d5/d19/f4e x:0 0 0 2024-05-13T22:48:02.087 INFO:tasks.workunit.client.0.smithi102.stdout:9/258: dread - d1/d6/db/f1b zero size 2024-05-13T22:48:02.095 INFO:tasks.workunit.client.0.smithi102.stdout:9/259: dread d1/f17 [0,4194304] 0 2024-05-13T22:48:02.097 INFO:tasks.workunit.client.0.smithi102.stdout:0/345: link d1/de/d32/d24/d2d/d38/c52 d1/c65 0 2024-05-13T22:48:02.098 INFO:tasks.workunit.client.0.smithi102.stdout:5/483: dwrite de/d25/d59/d6f/f46 [0,4194304] 0 2024-05-13T22:48:02.098 INFO:tasks.workunit.client.0.smithi102.stdout:9/260: stat d1/d6/db 0 2024-05-13T22:48:02.098 INFO:tasks.workunit.client.0.smithi102.stdout:1/236: dwrite d2/d4/d5/d19/d28/f3b [0,4194304] 0 2024-05-13T22:48:02.105 INFO:tasks.workunit.client.0.smithi102.stdout:7/200: truncate d6/f8 3446567 0 2024-05-13T22:48:02.108 INFO:tasks.workunit.client.0.smithi102.stdout:1/237: dwrite d2/d45/f48 [0,4194304] 0 2024-05-13T22:48:02.109 INFO:tasks.workunit.client.0.smithi102.stdout:9/261: creat d1/d6/d3b/d5b/f5f x:0 0 0 2024-05-13T22:48:02.116 INFO:tasks.workunit.client.0.smithi102.stdout:0/346: rename d1/c29 to d1/de/d32/d24/d2d/d3e/c66 0 2024-05-13T22:48:02.120 INFO:tasks.workunit.client.0.smithi102.stdout:5/484: creat de/d25/d2a/d4b/d4d/fb5 x:0 0 0 2024-05-13T22:48:02.123 INFO:tasks.workunit.client.0.smithi102.stdout:1/238: symlink d2/d4/d26/l4f 0 2024-05-13T22:48:02.123 INFO:tasks.workunit.client.0.smithi102.stdout:9/262: getdents d1/d5d 0 2024-05-13T22:48:02.124 INFO:tasks.workunit.client.0.smithi102.stdout:5/485: mknod de/d75/cb6 0 2024-05-13T22:48:02.126 INFO:tasks.workunit.client.0.smithi102.stdout:0/347: creat d1/de/d42/d61/f67 x:0 0 0 2024-05-13T22:48:02.127 INFO:tasks.workunit.client.0.smithi102.stdout:0/348: fdatasync d1/de/d32/d24/f4c 0 2024-05-13T22:48:02.127 INFO:tasks.workunit.client.0.smithi102.stdout:1/239: symlink d2/d4/d26/d40/l50 0 2024-05-13T22:48:02.130 INFO:tasks.workunit.client.0.smithi102.stdout:5/486: unlink de/d25/d2c/l2f 0 2024-05-13T22:48:02.135 INFO:tasks.workunit.client.0.smithi102.stdout:9/263: creat d1/d6/f60 x:0 0 0 2024-05-13T22:48:02.136 INFO:tasks.workunit.client.0.smithi102.stdout:5/487: symlink de/d75/lb7 0 2024-05-13T22:48:02.138 INFO:tasks.workunit.client.0.smithi102.stdout:0/349: mkdir d1/d68 0 2024-05-13T22:48:02.139 INFO:tasks.workunit.client.0.smithi102.stdout:0/350: chown d1/de/f5a 5762 1 2024-05-13T22:48:02.140 INFO:tasks.workunit.client.0.smithi102.stdout:9/264: fdatasync d1/f2b 0 2024-05-13T22:48:02.142 INFO:tasks.workunit.client.0.smithi102.stdout:7/201: dwrite d6/d13/f14 [0,4194304] 0 2024-05-13T22:48:02.142 INFO:tasks.workunit.client.0.smithi102.stdout:5/488: creat de/d75/fb8 x:0 0 0 2024-05-13T22:48:02.144 INFO:tasks.workunit.client.0.smithi102.stdout:7/202: truncate d6/d22/f23 672860 0 2024-05-13T22:48:02.145 INFO:tasks.workunit.client.0.smithi102.stdout:5/489: write de/d25/d2a/fa0 [2994084,88175] 0 2024-05-13T22:48:02.145 INFO:tasks.workunit.client.0.smithi102.stdout:0/351: chown d1/de/c20 0 1 2024-05-13T22:48:02.147 INFO:tasks.workunit.client.0.smithi102.stdout:9/265: dread d1/d6/d21/f29 [0,4194304] 0 2024-05-13T22:48:02.147 INFO:tasks.workunit.client.0.smithi102.stdout:0/352: read d1/f9 [4134115,98367] 0 2024-05-13T22:48:02.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:01 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:48:02.152 INFO:tasks.workunit.client.0.smithi102.stdout:9/266: write d1/d6/f11 [741059,98262] 0 2024-05-13T22:48:02.154 INFO:tasks.workunit.client.0.smithi102.stdout:0/353: creat d1/de/d32/d24/d2d/d38/f69 x:0 0 0 2024-05-13T22:48:02.160 INFO:tasks.workunit.client.0.smithi102.stdout:0/354: creat d1/d4d/f6a x:0 0 0 2024-05-13T22:48:02.163 INFO:tasks.workunit.client.0.smithi102.stdout:9/267: symlink d1/d5d/l61 0 2024-05-13T22:48:02.164 INFO:tasks.workunit.client.0.smithi102.stdout:9/268: truncate d1/d6/db/f1b 608554 0 2024-05-13T22:48:02.168 INFO:tasks.workunit.client.0.smithi102.stdout:9/269: dread d1/d6/d21/f29 [0,4194304] 0 2024-05-13T22:48:02.173 INFO:tasks.workunit.client.0.smithi102.stdout:0/355: rename d1/f4 to d1/de/d32/d24/d2d/f6b 0 2024-05-13T22:48:02.175 INFO:tasks.workunit.client.0.smithi102.stdout:5/490: dwrite de/f11 [0,4194304] 0 2024-05-13T22:48:02.176 INFO:tasks.workunit.client.0.smithi102.stdout:5/491: chown de/d25/d2c/f96 1834101 1 2024-05-13T22:48:02.176 INFO:tasks.workunit.client.0.smithi102.stdout:0/356: mknod d1/de/d42/c6c 0 2024-05-13T22:48:02.177 INFO:tasks.workunit.client.0.smithi102.stdout:9/270: dread d1/d6/db/f1c [0,4194304] 0 2024-05-13T22:48:02.188 INFO:tasks.workunit.client.0.smithi102.stdout:0/357: symlink d1/l6d 0 2024-05-13T22:48:02.197 INFO:tasks.workunit.client.0.smithi102.stdout:0/358: mknod d1/d68/c6e 0 2024-05-13T22:48:02.198 INFO:tasks.workunit.client.0.smithi102.stdout:5/492: getdents de/d25/d2a/d4b/d4d/d50/d99 0 2024-05-13T22:48:02.201 INFO:tasks.workunit.client.0.smithi102.stdout:0/359: mknod d1/d68/c6f 0 2024-05-13T22:48:02.202 INFO:tasks.workunit.client.0.smithi102.stdout:5/493: mkdir de/d25/d2a/d4b/d4d/d50/d6c/db9 0 2024-05-13T22:48:02.204 INFO:tasks.workunit.client.0.smithi102.stdout:0/360: mkdir d1/de/d32/d24/d2d/d3e/d70 0 2024-05-13T22:48:02.206 INFO:tasks.workunit.client.0.smithi102.stdout:5/494: symlink de/d25/d43/d7a/lba 0 2024-05-13T22:48:02.210 INFO:tasks.workunit.client.0.smithi102.stdout:0/361: dwrite d1/de/d32/d24/d2d/d38/d3d/f57 [0,4194304] 0 2024-05-13T22:48:02.211 INFO:tasks.workunit.client.0.smithi102.stdout:5/495: read - de/d75/fa4 zero size 2024-05-13T22:48:02.215 INFO:tasks.workunit.client.0.smithi102.stdout:5/496: mkdir de/d25/d2a/d4b/d4d/d50/d99/dbb 0 2024-05-13T22:48:02.218 INFO:tasks.workunit.client.0.smithi102.stdout:9/271: truncate d1/d6/db/d30/d47/f49 2161233 0 2024-05-13T22:48:02.219 INFO:tasks.workunit.client.0.smithi102.stdout:9/272: dread - d1/d6/d45/d3e/f56 zero size 2024-05-13T22:48:02.221 INFO:tasks.workunit.client.0.smithi102.stdout:5/497: dwrite de/d25/d2c/f5f [0,4194304] 0 2024-05-13T22:48:02.225 INFO:tasks.workunit.client.0.smithi102.stdout:9/273: creat d1/d6/d3b/d5b/f62 x:0 0 0 2024-05-13T22:48:02.225 INFO:tasks.workunit.client.0.smithi102.stdout:9/274: chown d1/d6/db/l18 299448921 1 2024-05-13T22:48:02.227 INFO:tasks.workunit.client.0.smithi102.stdout:0/362: getdents d1/de 0 2024-05-13T22:48:02.229 INFO:tasks.workunit.client.0.smithi102.stdout:5/498: write de/d25/d59/f81 [4309429,126412] 0 2024-05-13T22:48:02.230 INFO:tasks.workunit.client.0.smithi102.stdout:5/499: write de/d25/d59/d28/f67 [514406,80097] 0 2024-05-13T22:48:02.238 INFO:tasks.workunit.client.0.smithi102.stdout:5/500: dwrite de/d75/fb8 [0,4194304] 0 2024-05-13T22:48:02.239 INFO:tasks.workunit.client.0.smithi102.stdout:5/501: dread de/d25/d59/d6f/f72 [0,4194304] 0 2024-05-13T22:48:02.241 INFO:tasks.workunit.client.0.smithi102.stdout:0/363: rename d1/de/d32/d24/d2d/d3e/c54 to d1/de/d42/c71 0 2024-05-13T22:48:02.243 INFO:tasks.workunit.client.0.smithi102.stdout:0/364: read - d1/de/d32/d24/d2d/d38/f69 zero size 2024-05-13T22:48:02.246 INFO:tasks.workunit.client.0.smithi102.stdout:5/502: mkdir de/d25/d59/d7f/d86/da5/dbc 0 2024-05-13T22:48:02.252 INFO:tasks.workunit.client.0.smithi102.stdout:5/503: creat de/d75/fbd x:0 0 0 2024-05-13T22:48:02.253 INFO:tasks.workunit.client.0.smithi102.stdout:5/504: truncate de/d25/d59/f18 1339472 0 2024-05-13T22:48:02.256 INFO:tasks.workunit.client.0.smithi102.stdout:0/365: creat d1/de/d32/f72 x:0 0 0 2024-05-13T22:48:02.258 INFO:tasks.workunit.client.0.smithi102.stdout:0/366: write d1/de/d32/f3b [1910759,78156] 0 2024-05-13T22:48:02.265 INFO:tasks.workunit.client.0.smithi102.stdout:9/275: dwrite d1/d6/f40 [0,4194304] 0 2024-05-13T22:48:02.266 INFO:tasks.workunit.client.0.smithi102.stdout:9/276: readlink d1/d6/d21/l23 0 2024-05-13T22:48:02.267 INFO:tasks.workunit.client.0.smithi102.stdout:0/367: dwrite d1/de/f33 [0,4194304] 0 2024-05-13T22:48:02.273 INFO:tasks.workunit.client.0.smithi102.stdout:5/505: getdents de/d25/d59/d6f 0 2024-05-13T22:48:02.275 INFO:tasks.workunit.client.0.smithi102.stdout:9/277: dwrite d1/d6/d3b/f38 [0,4194304] 0 2024-05-13T22:48:02.281 INFO:tasks.workunit.client.0.smithi102.stdout:5/506: readlink de/d75/l91 0 2024-05-13T22:48:02.282 INFO:tasks.workunit.client.0.smithi102.stdout:5/507: write de/d25/d59/f1c [4489957,80621] 0 2024-05-13T22:48:02.283 INFO:tasks.workunit.client.0.smithi102.stdout:9/278: rmdir d1/d6/d45 39 2024-05-13T22:48:02.288 INFO:tasks.workunit.client.0.smithi102.stdout:0/368: truncate d1/de/d32/f3f 1621 0 2024-05-13T22:48:02.292 INFO:tasks.workunit.client.0.smithi102.stdout:5/508: mknod de/d25/d59/d6f/cbe 0 2024-05-13T22:48:02.296 INFO:tasks.workunit.client.0.smithi102.stdout:0/369: chown d1/de/d32/l49 11 1 2024-05-13T22:48:02.300 INFO:tasks.workunit.client.0.smithi102.stdout:9/279: creat d1/d6/d45/d3e/f63 x:0 0 0 2024-05-13T22:48:02.302 INFO:tasks.workunit.client.0.smithi102.stdout:0/370: creat d1/de/d32/d24/d2d/d3e/f73 x:0 0 0 2024-05-13T22:48:02.304 INFO:tasks.workunit.client.0.smithi102.stdout:5/509: truncate de/f1d 1083935 0 2024-05-13T22:48:02.306 INFO:tasks.workunit.client.0.smithi102.stdout:9/280: rmdir d1/d6/d3b 39 2024-05-13T22:48:02.309 INFO:tasks.workunit.client.0.smithi102.stdout:5/510: write de/d25/d59/d6f/f72 [748274,94996] 0 2024-05-13T22:48:02.311 INFO:tasks.workunit.client.0.smithi102.stdout:9/281: mkdir d1/d6/db/d4a/d64 0 2024-05-13T22:48:02.313 INFO:tasks.workunit.client.0.smithi102.stdout:0/371: creat d1/de/d32/d24/f74 x:0 0 0 2024-05-13T22:48:02.315 INFO:tasks.workunit.client.0.smithi102.stdout:9/282: mknod d1/d6/db/d4a/d64/c65 0 2024-05-13T22:48:02.315 INFO:tasks.workunit.client.0.smithi102.stdout:9/283: read - d1/d6/d21/f48 zero size 2024-05-13T22:48:02.317 INFO:tasks.workunit.client.0.smithi102.stdout:0/372: rename d1/l10 to d1/de/d32/d24/d2d/d38/l75 0 2024-05-13T22:48:02.318 INFO:tasks.workunit.client.0.smithi102.stdout:9/284: creat d1/d6/db/f66 x:0 0 0 2024-05-13T22:48:02.320 INFO:tasks.workunit.client.0.smithi102.stdout:0/373: rename d1/d4d/c5d to d1/de/d42/c76 0 2024-05-13T22:48:02.323 INFO:tasks.workunit.client.0.smithi102.stdout:9/285: creat d1/d6/d45/f67 x:0 0 0 2024-05-13T22:48:02.518 INFO:tasks.workunit.client.0.smithi102.stdout:9/286: dwrite d1/d6/db/f1b [0,4194304] 0 2024-05-13T22:48:02.519 INFO:tasks.workunit.client.0.smithi102.stdout:9/287: dread - d1/d6/d45/d3e/f56 zero size 2024-05-13T22:48:02.522 INFO:tasks.workunit.client.0.smithi102.stdout:9/288: truncate d1/d6/d21/f29 510422 0 2024-05-13T22:48:02.524 INFO:tasks.workunit.client.0.smithi102.stdout:9/289: dread d1/d6/d45/f5e [0,4194304] 0 2024-05-13T22:48:02.526 INFO:tasks.workunit.client.0.smithi102.stdout:9/290: readlink d1/d6/d45/d3e/l41 0 2024-05-13T22:48:02.528 INFO:tasks.workunit.client.0.smithi102.stdout:9/291: creat d1/d6/d45/d3e/f68 x:0 0 0 2024-05-13T22:48:02.534 INFO:tasks.workunit.client.0.smithi102.stdout:9/292: read d1/f1e [876799,50391] 0 2024-05-13T22:48:02.534 INFO:tasks.workunit.client.0.smithi102.stdout:9/293: readlink d1/d6/d45/d2d/l33 0 2024-05-13T22:48:02.535 INFO:tasks.workunit.client.0.smithi102.stdout:9/294: fdatasync d1/d6/f40 0 2024-05-13T22:48:02.540 INFO:tasks.workunit.client.0.smithi102.stdout:9/295: mknod d1/d6/c69 0 2024-05-13T22:48:02.546 INFO:tasks.workunit.client.0.smithi102.stdout:9/296: creat d1/d6/db/d30/f6a x:0 0 0 2024-05-13T22:48:02.553 INFO:tasks.workunit.client.0.smithi102.stdout:9/297: dwrite d1/d6/f4e [0,4194304] 0 2024-05-13T22:48:02.559 INFO:tasks.workunit.client.0.smithi102.stdout:9/298: dread d1/d6/d3b/f38 [0,4194304] 0 2024-05-13T22:48:02.564 INFO:tasks.workunit.client.0.smithi102.stdout:9/299: mkdir d1/d6b 0 2024-05-13T22:48:02.566 INFO:tasks.workunit.client.0.smithi102.stdout:9/300: mkdir d1/d5d/d6c 0 2024-05-13T22:48:02.571 INFO:tasks.workunit.client.0.smithi102.stdout:9/301: creat d1/d6/db/d30/d47/f6d x:0 0 0 2024-05-13T22:48:02.576 INFO:tasks.workunit.client.0.smithi102.stdout:9/302: dread d1/d6/db/f1b [0,4194304] 0 2024-05-13T22:48:02.580 INFO:tasks.workunit.client.0.smithi102.stdout:9/303: symlink d1/d6/db/d30/d47/l6e 0 2024-05-13T22:48:02.582 INFO:tasks.workunit.client.0.smithi102.stdout:9/304: unlink d1/d6/db/d30/c37 0 2024-05-13T22:48:02.593 INFO:tasks.workunit.client.0.smithi102.stdout:9/305: write d1/d6/db/d30/f36 [2031088,105907] 0 2024-05-13T22:48:02.598 INFO:tasks.workunit.client.0.smithi102.stdout:9/306: write d1/d6/d21/f26 [5059264,64474] 0 2024-05-13T22:48:02.602 INFO:tasks.workunit.client.0.smithi102.stdout:9/307: mknod d1/d6/db/d4a/d64/c6f 0 2024-05-13T22:48:02.603 INFO:tasks.workunit.client.0.smithi102.stdout:9/308: mknod d1/d6/d3b/c70 0 2024-05-13T22:48:02.610 INFO:tasks.workunit.client.0.smithi102.stdout:9/309: truncate d1/d6/f1d 1829788 0 2024-05-13T22:48:02.611 INFO:tasks.workunit.client.0.smithi102.stdout:9/310: write d1/d6/d45/d3e/f63 [256156,78231] 0 2024-05-13T22:48:02.612 INFO:tasks.workunit.client.0.smithi102.stdout:9/311: write d1/d6/db/d30/f36 [662953,104792] 0 2024-05-13T22:48:02.614 INFO:tasks.workunit.client.0.smithi102.stdout:9/312: mkdir d1/d6/d21/d71 0 2024-05-13T22:48:02.617 INFO:tasks.workunit.client.0.smithi102.stdout:9/313: getdents d1/d5d/d6c 0 2024-05-13T22:48:02.624 INFO:tasks.workunit.client.0.smithi102.stdout:9/314: fsync d1/d6/f1d 0 2024-05-13T22:48:02.625 INFO:tasks.workunit.client.0.smithi102.stdout:9/315: chown d1/d6/d21/f42 750 1 2024-05-13T22:48:02.629 INFO:tasks.workunit.client.0.smithi102.stdout:9/316: symlink d1/d6b/l72 0 2024-05-13T22:48:02.630 INFO:tasks.workunit.client.0.smithi102.stdout:9/317: chown d1/d6/db/d30/f36 3592344 1 2024-05-13T22:48:02.632 INFO:tasks.workunit.client.0.smithi102.stdout:3/305: sync 2024-05-13T22:48:02.632 INFO:tasks.workunit.client.0.smithi102.stdout:4/252: sync 2024-05-13T22:48:02.635 INFO:tasks.workunit.client.0.smithi102.stdout:3/306: creat de/d10/d50/f66 x:0 0 0 2024-05-13T22:48:02.636 INFO:tasks.workunit.client.0.smithi102.stdout:3/307: stat de/d10/d34/d5d 0 2024-05-13T22:48:02.636 INFO:tasks.workunit.client.0.smithi102.stdout:4/253: mknod d3/d4/d6/de/c5e 0 2024-05-13T22:48:02.639 INFO:tasks.workunit.client.0.smithi102.stdout:4/254: mkdir d3/d4/d5f 0 2024-05-13T22:48:02.642 INFO:tasks.workunit.client.0.smithi102.stdout:4/255: fdatasync d3/d4/f31 0 2024-05-13T22:48:02.644 INFO:tasks.workunit.client.0.smithi102.stdout:9/318: dwrite d1/d6/d21/f29 [0,4194304] 0 2024-05-13T22:48:02.646 INFO:tasks.workunit.client.0.smithi102.stdout:9/319: stat d1/d6/db 0 2024-05-13T22:48:02.648 INFO:tasks.workunit.client.0.smithi102.stdout:3/308: write de/d10/f11 [3154725,17406] 0 2024-05-13T22:48:02.649 INFO:tasks.workunit.client.0.smithi102.stdout:9/320: creat d1/d6/d45/d3e/f73 x:0 0 0 2024-05-13T22:48:02.650 INFO:tasks.workunit.client.0.smithi102.stdout:9/321: fdatasync d1/d6/f60 0 2024-05-13T22:48:02.651 INFO:tasks.workunit.client.0.smithi102.stdout:4/256: dwrite d3/f10 [0,4194304] 0 2024-05-13T22:48:02.653 INFO:tasks.workunit.client.0.smithi102.stdout:4/257: truncate d3/d19/d23/d3a/f4d 800573 0 2024-05-13T22:48:02.654 INFO:tasks.workunit.client.0.smithi102.stdout:9/322: creat d1/d5d/f74 x:0 0 0 2024-05-13T22:48:02.654 INFO:tasks.workunit.client.0.smithi102.stdout:3/309: readlink de/d1c/l23 0 2024-05-13T22:48:02.655 INFO:tasks.workunit.client.0.smithi102.stdout:4/258: read - d3/d19/d23/d3a/f51 zero size 2024-05-13T22:48:02.656 INFO:tasks.workunit.client.0.smithi102.stdout:4/259: read - d3/d19/d23/d3a/f56 zero size 2024-05-13T22:48:02.661 INFO:tasks.workunit.client.0.smithi102.stdout:4/260: creat d3/d19/d23/d4b/f60 x:0 0 0 2024-05-13T22:48:02.662 INFO:tasks.workunit.client.0.smithi102.stdout:9/323: dwrite d1/d6/d21/f26 [4194304,4194304] 0 2024-05-13T22:48:02.664 INFO:tasks.workunit.client.0.smithi102.stdout:9/324: mkdir d1/d6/db/d4a/d75 0 2024-05-13T22:48:02.666 INFO:tasks.workunit.client.0.smithi102.stdout:4/261: truncate d3/d4/d6/de/d1f/f28 3525298 0 2024-05-13T22:48:02.666 INFO:tasks.workunit.client.0.smithi102.stdout:4/262: chown d3/d4/l5 26398906 1 2024-05-13T22:48:02.667 INFO:tasks.workunit.client.0.smithi102.stdout:9/325: mknod d1/d6/db/d4a/c76 0 2024-05-13T22:48:02.672 INFO:tasks.workunit.client.0.smithi102.stdout:9/326: mkdir d1/d6/db/d77 0 2024-05-13T22:48:02.673 INFO:tasks.workunit.client.0.smithi102.stdout:9/327: fsync d1/f3a 0 2024-05-13T22:48:02.675 INFO:tasks.workunit.client.0.smithi102.stdout:9/328: readlink d1/l2 0 2024-05-13T22:48:02.679 INFO:tasks.workunit.client.0.smithi102.stdout:9/329: dread d1/d6/d21/f29 [0,4194304] 0 2024-05-13T22:48:02.680 INFO:tasks.workunit.client.0.smithi102.stdout:9/330: mknod d1/d6b/c78 0 2024-05-13T22:48:02.682 INFO:tasks.workunit.client.0.smithi102.stdout:9/331: mkdir d1/d6/d45/d3e/d79 0 2024-05-13T22:48:02.687 INFO:tasks.workunit.client.0.smithi102.stdout:9/332: creat d1/d6/db/f7a x:0 0 0 2024-05-13T22:48:02.687 INFO:tasks.workunit.client.0.smithi102.stdout:4/263: write d3/d19/d23/f3d [1630908,64242] 0 2024-05-13T22:48:02.688 INFO:tasks.workunit.client.0.smithi102.stdout:4/264: chown d3/d19 186678 1 2024-05-13T22:48:02.689 INFO:tasks.workunit.client.0.smithi102.stdout:9/333: truncate d1/d6/d45/d3e/f63 688407 0 2024-05-13T22:48:02.691 INFO:tasks.workunit.client.0.smithi102.stdout:4/265: mknod d3/d4/c61 0 2024-05-13T22:48:02.696 INFO:tasks.workunit.client.0.smithi102.stdout:9/334: dwrite d1/d6/f11 [0,4194304] 0 2024-05-13T22:48:02.707 INFO:tasks.workunit.client.0.smithi102.stdout:4/266: write d3/d22/f36 [977817,66380] 0 2024-05-13T22:48:02.708 INFO:tasks.workunit.client.0.smithi102.stdout:4/267: chown d3/d22/f3e 12342408 1 2024-05-13T22:48:02.716 INFO:tasks.workunit.client.0.smithi102.stdout:9/335: dread d1/d6/db/d30/d47/f49 [0,4194304] 0 2024-05-13T22:48:02.717 INFO:tasks.workunit.client.0.smithi102.stdout:9/336: write d1/d6/db/f7a [169219,29425] 0 2024-05-13T22:48:02.718 INFO:tasks.workunit.client.0.smithi102.stdout:9/337: readlink d1/d6/db/l18 0 2024-05-13T22:48:02.719 INFO:tasks.workunit.client.0.smithi102.stdout:9/338: read d1/f17 [17789,6999] 0 2024-05-13T22:48:02.720 INFO:tasks.workunit.client.0.smithi102.stdout:9/339: truncate d1/f5c 306614 0 2024-05-13T22:48:02.721 INFO:tasks.workunit.client.0.smithi102.stdout:9/340: write d1/d6/d45/f67 [795779,69593] 0 2024-05-13T22:48:02.724 INFO:tasks.workunit.client.0.smithi102.stdout:9/341: fsync d1/d6/d45/d2d/f2e 0 2024-05-13T22:48:02.726 INFO:tasks.workunit.client.0.smithi102.stdout:4/268: dread d3/d4/d6/f18 [0,4194304] 0 2024-05-13T22:48:02.727 INFO:tasks.workunit.client.0.smithi102.stdout:9/342: creat d1/d6/db/d77/f7b x:0 0 0 2024-05-13T22:48:02.728 INFO:tasks.workunit.client.0.smithi102.stdout:4/269: mknod d3/d19/d23/d39/c62 0 2024-05-13T22:48:02.729 INFO:tasks.workunit.client.0.smithi102.stdout:4/270: fdatasync d3/d19/d23/d4e/f54 0 2024-05-13T22:48:02.738 INFO:tasks.workunit.client.0.smithi102.stdout:9/343: write d1/f3a [180187,30103] 0 2024-05-13T22:48:02.739 INFO:tasks.workunit.client.0.smithi102.stdout:9/344: read - d1/d6/d45/d3e/f73 zero size 2024-05-13T22:48:02.743 INFO:tasks.workunit.client.0.smithi102.stdout:9/345: truncate d1/d6/d45/f5e 6759 0 2024-05-13T22:48:02.743 INFO:tasks.workunit.client.0.smithi102.stdout:4/271: dwrite d3/d4/f31 [0,4194304] 0 2024-05-13T22:48:02.746 INFO:tasks.workunit.client.0.smithi102.stdout:4/272: mknod d3/d19/d23/d4e/c63 0 2024-05-13T22:48:02.747 INFO:tasks.workunit.client.0.smithi102.stdout:9/346: creat d1/d6/f7c x:0 0 0 2024-05-13T22:48:02.748 INFO:tasks.workunit.client.0.smithi102.stdout:4/273: rmdir d3/d4/d6 39 2024-05-13T22:48:02.749 INFO:tasks.workunit.client.0.smithi102.stdout:9/347: truncate d1/d6/db/d30/f6a 460019 0 2024-05-13T22:48:02.749 INFO:tasks.workunit.client.0.smithi102.stdout:4/274: chown d3/l21 667904 1 2024-05-13T22:48:02.749 INFO:tasks.workunit.client.0.smithi102.stdout:9/348: stat d1 0 2024-05-13T22:48:02.750 INFO:tasks.workunit.client.0.smithi102.stdout:4/275: fsync d3/d19/d23/f43 0 2024-05-13T22:48:02.751 INFO:tasks.workunit.client.0.smithi102.stdout:9/349: symlink d1/d6/db/d30/l7d 0 2024-05-13T22:48:02.752 INFO:tasks.workunit.client.0.smithi102.stdout:9/350: stat d1/d6/d45/d3e/d79 0 2024-05-13T22:48:02.756 INFO:tasks.workunit.client.0.smithi102.stdout:4/276: rename d3/d4/d6/de/c5e to d3/d4/d5f/c64 0 2024-05-13T22:48:02.757 INFO:tasks.workunit.client.0.smithi102.stdout:9/351: link d1/d6/cf d1/d6b/c7e 0 2024-05-13T22:48:02.757 INFO:tasks.workunit.client.0.smithi102.stdout:4/277: fdatasync d3/d19/d23/d3a/f4d 0 2024-05-13T22:48:02.760 INFO:tasks.workunit.client.0.smithi102.stdout:9/352: unlink d1/f50 0 2024-05-13T22:48:02.763 INFO:tasks.workunit.client.0.smithi102.stdout:9/353: symlink d1/d6/d3b/l7f 0 2024-05-13T22:48:02.764 INFO:tasks.workunit.client.0.smithi102.stdout:4/278: creat d3/d4/d6/de/d1f/d41/f65 x:0 0 0 2024-05-13T22:48:02.768 INFO:tasks.workunit.client.0.smithi102.stdout:9/354: symlink d1/d6/d45/l80 0 2024-05-13T22:48:02.771 INFO:tasks.workunit.client.0.smithi102.stdout:4/279: rename d3/d19/d23/d4e/f54 to d3/d4/f66 0 2024-05-13T22:48:02.774 INFO:tasks.workunit.client.0.smithi102.stdout:2/315: sync 2024-05-13T22:48:02.774 INFO:tasks.workunit.client.0.smithi102.stdout:4/280: creat d3/d4/d6/de/d1f/f67 x:0 0 0 2024-05-13T22:48:02.775 INFO:tasks.workunit.client.0.smithi102.stdout:9/355: mkdir d1/d6/db/d4a/d81 0 2024-05-13T22:48:02.776 INFO:tasks.workunit.client.0.smithi102.stdout:2/316: dread - d7/d9/d10/d30/d53/f5e zero size 2024-05-13T22:48:02.777 INFO:tasks.workunit.client.0.smithi102.stdout:2/317: write d7/d9/d14/f62 [647651,117485] 0 2024-05-13T22:48:02.784 INFO:tasks.workunit.client.0.smithi102.stdout:9/356: dwrite d1/d6/f60 [0,4194304] 0 2024-05-13T22:48:02.787 INFO:tasks.workunit.client.0.smithi102.stdout:2/318: rename d7/d9/d14/f1a to d7/d9/d10/d25/f68 0 2024-05-13T22:48:02.790 INFO:tasks.workunit.client.0.smithi102.stdout:2/319: mknod d7/d9/d10/d25/d56/c69 0 2024-05-13T22:48:02.792 INFO:tasks.workunit.client.0.smithi102.stdout:2/320: write d7/d9/d14/f36 [5025375,121803] 0 2024-05-13T22:48:02.792 INFO:tasks.workunit.client.0.smithi102.stdout:9/357: creat d1/d6/db/d4a/d75/f82 x:0 0 0 2024-05-13T22:48:02.808 INFO:tasks.workunit.client.0.smithi102.stdout:4/281: rmdir d3/d4 39 2024-05-13T22:48:02.815 INFO:tasks.workunit.client.0.smithi102.stdout:2/321: dread d7/d9/d10/f33 [0,4194304] 0 2024-05-13T22:48:02.817 INFO:tasks.workunit.client.0.smithi102.stdout:2/322: write d7/d9/d14/f62 [1809396,106008] 0 2024-05-13T22:48:02.818 INFO:tasks.workunit.client.0.smithi102.stdout:4/282: getdents d3/d4/d5f 0 2024-05-13T22:48:02.820 INFO:tasks.workunit.client.0.smithi102.stdout:2/323: chown d7/d9/d10/d30/l65 375159349 1 2024-05-13T22:48:02.821 INFO:tasks.workunit.client.0.smithi102.stdout:2/324: stat d7/d9/d14/d31/l46 0 2024-05-13T22:48:02.821 INFO:tasks.workunit.client.0.smithi102.stdout:4/283: mkdir d3/d19/d23/d4e/d68 0 2024-05-13T22:48:02.822 INFO:tasks.workunit.client.0.smithi102.stdout:9/358: truncate d1/d6/d45/f67 700916 0 2024-05-13T22:48:02.824 INFO:tasks.workunit.client.0.smithi102.stdout:4/284: fsync d3/d22/f3e 0 2024-05-13T22:48:02.824 INFO:tasks.workunit.client.0.smithi102.stdout:8/256: sync 2024-05-13T22:48:02.824 INFO:tasks.workunit.client.0.smithi102.stdout:5/511: sync 2024-05-13T22:48:02.824 INFO:tasks.workunit.client.0.smithi102.stdout:6/328: sync 2024-05-13T22:48:02.825 INFO:tasks.workunit.client.0.smithi102.stdout:0/374: sync 2024-05-13T22:48:02.825 INFO:tasks.workunit.client.0.smithi102.stdout:1/240: sync 2024-05-13T22:48:02.825 INFO:tasks.workunit.client.0.smithi102.stdout:7/203: sync 2024-05-13T22:48:02.826 INFO:tasks.workunit.client.0.smithi102.stdout:4/285: dread - d3/d4/f5a zero size 2024-05-13T22:48:02.827 INFO:tasks.workunit.client.0.smithi102.stdout:7/204: fsync d6/d13/d2c/f2f 0 2024-05-13T22:48:02.827 INFO:tasks.workunit.client.0.smithi102.stdout:8/257: read d2/d17/f33 [3131627,44067] 0 2024-05-13T22:48:02.828 INFO:tasks.workunit.client.0.smithi102.stdout:0/375: read - d1/de/d32/d24/d2d/f64 zero size 2024-05-13T22:48:02.828 INFO:tasks.workunit.client.0.smithi102.stdout:4/286: chown d3/d4/d6/f42 1 1 2024-05-13T22:48:02.829 INFO:tasks.workunit.client.0.smithi102.stdout:2/325: getdents d7/d9/d14/d61 0 2024-05-13T22:48:02.830 INFO:tasks.workunit.client.0.smithi102.stdout:4/287: fdatasync d3/d4/d6/de/d1f/d41/f65 0 2024-05-13T22:48:02.831 INFO:tasks.workunit.client.0.smithi102.stdout:6/329: unlink d9/db/f63 0 2024-05-13T22:48:02.832 INFO:tasks.workunit.client.0.smithi102.stdout:4/288: fdatasync d3/f10 0 2024-05-13T22:48:02.833 INFO:tasks.workunit.client.0.smithi102.stdout:8/258: truncate d2/d7/d27/f3d 4791069 0 2024-05-13T22:48:02.837 INFO:tasks.workunit.client.0.smithi102.stdout:7/205: symlink d6/d15/l38 0 2024-05-13T22:48:02.842 INFO:tasks.workunit.client.0.smithi102.stdout:9/359: rename d1/d6/d21/f26 to d1/d6/db/f83 0 2024-05-13T22:48:02.847 INFO:tasks.workunit.client.0.smithi102.stdout:8/259: mknod d2/d36/c4b 0 2024-05-13T22:48:02.849 INFO:tasks.workunit.client.0.smithi102.stdout:0/376: dread d1/f5 [0,4194304] 0 2024-05-13T22:48:02.849 INFO:tasks.workunit.client.0.smithi102.stdout:5/512: creat de/d25/d2a/d4b/d4d/d93/fbf x:0 0 0 2024-05-13T22:48:02.851 INFO:tasks.workunit.client.0.smithi102.stdout:2/326: mkdir d7/d15/d67/d6a 0 2024-05-13T22:48:02.852 INFO:tasks.workunit.client.0.smithi102.stdout:0/377: truncate d1/de/d32/d24/f62 326476 0 2024-05-13T22:48:02.852 INFO:tasks.workunit.client.0.smithi102.stdout:2/327: fsync d7/d15/f19 0 2024-05-13T22:48:02.853 INFO:tasks.workunit.client.0.smithi102.stdout:9/360: symlink d1/d6/d3b/d5b/l84 0 2024-05-13T22:48:02.855 INFO:tasks.workunit.client.0.smithi102.stdout:4/289: symlink d3/d4/l69 0 2024-05-13T22:48:02.857 INFO:tasks.workunit.client.0.smithi102.stdout:1/241: creat d2/d4/f51 x:0 0 0 2024-05-13T22:48:02.858 INFO:tasks.workunit.client.0.smithi102.stdout:1/242: readlink d2/d4/d5/d19/d28/l3c 0 2024-05-13T22:48:02.858 INFO:tasks.workunit.client.0.smithi102.stdout:1/243: stat l1 0 2024-05-13T22:48:02.861 INFO:tasks.workunit.client.0.smithi102.stdout:2/328: dread d7/d9/d10/d25/d27/f3b [0,4194304] 0 2024-05-13T22:48:02.861 INFO:tasks.workunit.client.0.smithi102.stdout:8/260: rename d2/d7/d12/f3e to d2/d7/d27/f4c 0 2024-05-13T22:48:02.866 INFO:tasks.workunit.client.0.smithi102.stdout:0/378: chown d1/de/d42/c71 62073841 1 2024-05-13T22:48:02.867 INFO:tasks.workunit.client.0.smithi102.stdout:0/379: write d1/d4d/f5b [927042,43942] 0 2024-05-13T22:48:02.869 INFO:tasks.workunit.client.0.smithi102.stdout:6/330: write d9/db/fe [7616,127572] 0 2024-05-13T22:48:02.870 INFO:tasks.workunit.client.0.smithi102.stdout:1/244: readlink d2/d10/l1b 0 2024-05-13T22:48:02.874 INFO:tasks.workunit.client.0.smithi102.stdout:2/329: rmdir d7/d9/d10 39 2024-05-13T22:48:02.876 INFO:tasks.workunit.client.0.smithi102.stdout:9/361: read d1/d6/d45/f67 [653176,31892] 0 2024-05-13T22:48:02.877 INFO:tasks.workunit.client.0.smithi102.stdout:8/261: unlink d2/d7/d12/l29 0 2024-05-13T22:48:02.877 INFO:tasks.workunit.client.0.smithi102.stdout:1/245: dwrite d2/d4/d5/d19/d28/f4d [0,4194304] 0 2024-05-13T22:48:02.878 INFO:tasks.workunit.client.0.smithi102.stdout:8/262: chown d2/d17/f26 823 1 2024-05-13T22:48:02.879 INFO:tasks.workunit.client.0.smithi102.stdout:1/246: chown d2/d4/d5 29 1 2024-05-13T22:48:02.879 INFO:tasks.workunit.client.0.smithi102.stdout:5/513: write de/d25/f26 [320906,124866] 0 2024-05-13T22:48:02.886 INFO:tasks.workunit.client.0.smithi102.stdout:9/362: creat d1/d6/db/d4a/f85 x:0 0 0 2024-05-13T22:48:02.889 INFO:tasks.workunit.client.0.smithi102.stdout:8/263: rename d2/d7/c18 to d2/d36/c4d 0 2024-05-13T22:48:02.891 INFO:tasks.workunit.client.0.smithi102.stdout:5/514: creat de/d25/d2a/d4b/d4d/d50/d99/fc0 x:0 0 0 2024-05-13T22:48:02.891 INFO:tasks.workunit.client.0.smithi102.stdout:0/380: dwrite d1/de/f19 [4194304,4194304] 0 2024-05-13T22:48:02.893 INFO:tasks.workunit.client.0.smithi102.stdout:0/381: write d1/de/d42/d61/f67 [236426,30019] 0 2024-05-13T22:48:02.895 INFO:tasks.workunit.client.0.smithi102.stdout:6/331: mknod d9/c6c 0 2024-05-13T22:48:02.896 INFO:tasks.workunit.client.0.smithi102.stdout:1/247: creat d2/d4/d38/f52 x:0 0 0 2024-05-13T22:48:02.899 INFO:tasks.workunit.client.0.smithi102.stdout:8/264: unlink d2/d7/ce 0 2024-05-13T22:48:02.902 INFO:tasks.workunit.client.0.smithi102.stdout:6/332: write d9/f1a [824121,89928] 0 2024-05-13T22:48:02.902 INFO:tasks.workunit.client.0.smithi102.stdout:5/515: dwrite de/d25/f47 [0,4194304] 0 2024-05-13T22:48:02.907 INFO:tasks.workunit.client.0.smithi102.stdout:6/333: readlink d9/db/d14/d35/l61 0 2024-05-13T22:48:02.914 INFO:tasks.workunit.client.0.smithi102.stdout:0/382: dwrite d1/f5 [0,4194304] 0 2024-05-13T22:48:02.914 INFO:tasks.workunit.client.0.smithi102.stdout:9/363: getdents d1/d6 0 2024-05-13T22:48:02.916 INFO:tasks.workunit.client.0.smithi102.stdout:8/265: dwrite d2/d7/d12/f47 [0,4194304] 0 2024-05-13T22:48:02.926 INFO:tasks.workunit.client.0.smithi102.stdout:6/334: creat d9/d25/d5a/f6d x:0 0 0 2024-05-13T22:48:02.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:02 smithi102 ceph-mon[28599]: pgmap v18: 65 pgs: 65 active+clean; 587 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 7.2 MiB/s rd, 59 MiB/s wr, 243 op/s 2024-05-13T22:48:02.927 INFO:tasks.workunit.client.0.smithi102.stdout:1/248: link d2/d4/d5/f15 d2/d4/d5/d19/f53 0 2024-05-13T22:48:02.928 INFO:tasks.workunit.client.0.smithi102.stdout:0/383: dwrite d1/de/d32/f5e [0,4194304] 0 2024-05-13T22:48:02.929 INFO:tasks.workunit.client.0.smithi102.stdout:6/335: chown d9/d25/d5a/f6d 77 1 2024-05-13T22:48:02.929 INFO:tasks.workunit.client.0.smithi102.stdout:1/249: readlink d2/d10/l4a 0 2024-05-13T22:48:02.929 INFO:tasks.workunit.client.0.smithi102.stdout:6/336: readlink d9/db/l6b 0 2024-05-13T22:48:02.930 INFO:tasks.workunit.client.0.smithi102.stdout:1/250: chown d2/f46 65181260 1 2024-05-13T22:48:02.932 INFO:tasks.workunit.client.0.smithi102.stdout:8/266: rename d2/d7/d12/f47 to d2/d7/f4e 0 2024-05-13T22:48:02.934 INFO:tasks.workunit.client.0.smithi102.stdout:0/384: rmdir d1/de/d32/d24/d2d 39 2024-05-13T22:48:02.937 INFO:tasks.workunit.client.0.smithi102.stdout:8/267: creat d2/d7/d12/f4f x:0 0 0 2024-05-13T22:48:02.939 INFO:tasks.workunit.client.0.smithi102.stdout:6/337: unlink d9/db/d2f/l3d 0 2024-05-13T22:48:02.943 INFO:tasks.workunit.client.0.smithi102.stdout:0/385: mkdir d1/d77 0 2024-05-13T22:48:02.944 INFO:tasks.workunit.client.0.smithi102.stdout:5/516: truncate de/f13 4903295 0 2024-05-13T22:48:02.948 INFO:tasks.workunit.client.0.smithi102.stdout:6/338: symlink d9/db/d2f/l6e 0 2024-05-13T22:48:02.948 INFO:tasks.workunit.client.0.smithi102.stdout:5/517: dread de/d33/f7b [0,4194304] 0 2024-05-13T22:48:02.949 INFO:tasks.workunit.client.0.smithi102.stdout:0/386: mknod d1/de/d42/d61/c78 0 2024-05-13T22:48:02.950 INFO:tasks.workunit.client.0.smithi102.stdout:0/387: read - d1/de/d32/d24/f74 zero size 2024-05-13T22:48:02.954 INFO:tasks.workunit.client.0.smithi102.stdout:1/251: getdents d2 0 2024-05-13T22:48:02.958 INFO:tasks.workunit.client.0.smithi102.stdout:5/518: unlink de/d25/d59/f81 0 2024-05-13T22:48:02.960 INFO:tasks.workunit.client.0.smithi102.stdout:0/388: mkdir d1/d4d/d79 0 2024-05-13T22:48:02.961 INFO:tasks.workunit.client.0.smithi102.stdout:0/389: stat d1/f60 0 2024-05-13T22:48:02.962 INFO:tasks.workunit.client.0.smithi102.stdout:1/252: dwrite d2/d4/d5/d19/f4e [0,4194304] 0 2024-05-13T22:48:02.967 INFO:tasks.workunit.client.0.smithi102.stdout:6/339: dread f7 [0,4194304] 0 2024-05-13T22:48:02.968 INFO:tasks.workunit.client.0.smithi102.stdout:6/340: read d9/db/d14/d37/f43 [443410,24548] 0 2024-05-13T22:48:02.972 INFO:tasks.workunit.client.0.smithi102.stdout:5/519: creat de/d25/d2a/d4b/fc1 x:0 0 0 2024-05-13T22:48:02.975 INFO:tasks.workunit.client.0.smithi102.stdout:0/390: unlink d1/de/d42/f47 0 2024-05-13T22:48:02.980 INFO:tasks.workunit.client.0.smithi102.stdout:5/520: dwrite de/f11 [0,4194304] 0 2024-05-13T22:48:02.981 INFO:tasks.workunit.client.0.smithi102.stdout:8/268: getdents d2/d7/d27 0 2024-05-13T22:48:02.987 INFO:tasks.workunit.client.0.smithi102.stdout:0/391: mknod d1/d68/c7a 0 2024-05-13T22:48:02.988 INFO:tasks.workunit.client.0.smithi102.stdout:0/392: read - d1/f60 zero size 2024-05-13T22:48:02.991 INFO:tasks.workunit.client.0.smithi102.stdout:6/341: dread d9/db/d14/f3f [0,4194304] 0 2024-05-13T22:48:02.991 INFO:tasks.workunit.client.0.smithi102.stdout:5/521: write de/d33/d53/d9e/fa7 [409679,114644] 0 2024-05-13T22:48:02.992 INFO:tasks.workunit.client.0.smithi102.stdout:8/269: rename d2/d7/d12/d2b to d2/d7/d25/d41/d50 0 2024-05-13T22:48:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:5/522: write de/d25/d2c/f78 [1761378,45712] 0 2024-05-13T22:48:02.993 INFO:tasks.workunit.client.0.smithi102.stdout:6/342: write d9/db/d14/d35/f67 [260134,57017] 0 2024-05-13T22:48:02.997 INFO:tasks.workunit.client.0.smithi102.stdout:5/523: write de/d25/d59/f5d [3884462,41859] 0 2024-05-13T22:48:02.999 INFO:tasks.workunit.client.0.smithi102.stdout:6/343: creat d9/d45/f6f x:0 0 0 2024-05-13T22:48:03.003 INFO:tasks.workunit.client.0.smithi102.stdout:0/393: getdents d1 0 2024-05-13T22:48:03.004 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:02 smithi130 ceph-mon[36591]: pgmap v18: 65 pgs: 65 active+clean; 587 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 7.2 MiB/s rd, 59 MiB/s wr, 243 op/s 2024-05-13T22:48:03.005 INFO:tasks.workunit.client.0.smithi102.stdout:1/253: dread d2/d4/d5/f15 [0,4194304] 0 2024-05-13T22:48:03.005 INFO:tasks.workunit.client.0.smithi102.stdout:6/344: rmdir d9/db/d14/d37 39 2024-05-13T22:48:03.007 INFO:tasks.workunit.client.0.smithi102.stdout:1/254: truncate d2/d4/d38/f52 972883 0 2024-05-13T22:48:03.017 INFO:tasks.workunit.client.0.smithi102.stdout:8/270: dread d2/d17/f26 [0,4194304] 0 2024-05-13T22:48:03.017 INFO:tasks.workunit.client.0.smithi102.stdout:8/271: write d2/d36/f42 [3902149,110716] 0 2024-05-13T22:48:03.045 INFO:tasks.workunit.client.0.smithi102.stdout:3/310: sync 2024-05-13T22:48:03.046 INFO:tasks.workunit.client.0.smithi102.stdout:0/394: unlink d1/de/d32/f5e 0 2024-05-13T22:48:03.056 INFO:tasks.workunit.client.0.smithi102.stdout:5/524: rename de/d25/d59/d7f/d86/da5/dbc to de/d33/d34/dc2 0 2024-05-13T22:48:03.059 INFO:tasks.workunit.client.0.smithi102.stdout:6/345: getdents d9/d45/d56/d5d 0 2024-05-13T22:48:03.060 INFO:tasks.workunit.client.0.smithi102.stdout:0/395: dwrite d1/de/d32/d24/d2d/d38/d3d/f57 [0,4194304] 0 2024-05-13T22:48:03.062 INFO:tasks.workunit.client.0.smithi102.stdout:8/272: unlink d2/d7/d25/d41/l43 0 2024-05-13T22:48:03.062 INFO:tasks.workunit.client.0.smithi102.stdout:0/396: dread - d1/de/d32/d24/d2d/f64 zero size 2024-05-13T22:48:03.064 INFO:tasks.workunit.client.0.smithi102.stdout:3/311: mkdir de/d67 0 2024-05-13T22:48:03.064 INFO:tasks.workunit.client.0.smithi102.stdout:3/312: readlink l3 0 2024-05-13T22:48:03.066 INFO:tasks.workunit.client.0.smithi102.stdout:5/525: creat de/d25/d2a/d4b/d4d/d93/fc3 x:0 0 0 2024-05-13T22:48:03.066 INFO:tasks.workunit.client.0.smithi102.stdout:1/255: dwrite d2/d4/d5/d19/f53 [8388608,4194304] 0 2024-05-13T22:48:03.068 INFO:tasks.workunit.client.0.smithi102.stdout:5/526: write de/f11 [3269404,2788] 0 2024-05-13T22:48:03.068 INFO:tasks.workunit.client.0.smithi102.stdout:6/346: creat d9/db/d14/d35/d47/f70 x:0 0 0 2024-05-13T22:48:03.069 INFO:tasks.workunit.client.0.smithi102.stdout:3/313: truncate de/d10/d50/f66 237715 0 2024-05-13T22:48:03.070 INFO:tasks.workunit.client.0.smithi102.stdout:8/273: creat d2/d17/f51 x:0 0 0 2024-05-13T22:48:03.070 INFO:tasks.workunit.client.0.smithi102.stdout:8/274: readlink d2/d17/l2e 0 2024-05-13T22:48:03.070 INFO:tasks.workunit.client.0.smithi102.stdout:0/397: symlink d1/de/l7b 0 2024-05-13T22:48:03.071 INFO:tasks.workunit.client.0.smithi102.stdout:8/275: readlink d2/d7/l8 0 2024-05-13T22:48:03.072 INFO:tasks.workunit.client.0.smithi102.stdout:1/256: chown d2/d4/d26/c3a 880438 1 2024-05-13T22:48:03.075 INFO:tasks.workunit.client.0.smithi102.stdout:8/276: fdatasync d2/d7/d27/f4c 0 2024-05-13T22:48:03.078 INFO:tasks.workunit.client.0.smithi102.stdout:1/257: creat d2/f54 x:0 0 0 2024-05-13T22:48:03.079 INFO:tasks.workunit.client.0.smithi102.stdout:3/314: mkdir de/d68 0 2024-05-13T22:48:03.080 INFO:tasks.workunit.client.0.smithi102.stdout:3/315: chown de/l53 51373334 1 2024-05-13T22:48:03.082 INFO:tasks.workunit.client.0.smithi102.stdout:5/527: creat de/fc4 x:0 0 0 2024-05-13T22:48:03.083 INFO:tasks.workunit.client.0.smithi102.stdout:8/277: mknod d2/d7/d27/c52 0 2024-05-13T22:48:03.084 INFO:tasks.workunit.client.0.smithi102.stdout:8/278: write d2/f32 [2890738,34563] 0 2024-05-13T22:48:03.085 INFO:tasks.workunit.client.0.smithi102.stdout:6/347: getdents d9/db/d14/d37/d5f 0 2024-05-13T22:48:03.085 INFO:tasks.workunit.client.0.smithi102.stdout:6/348: chown d9/d45/l3e 1423228 1 2024-05-13T22:48:03.087 INFO:tasks.workunit.client.0.smithi102.stdout:0/398: rmdir d1/de/d32/d24/d2d/d3e/d70 0 2024-05-13T22:48:03.087 INFO:tasks.workunit.client.0.smithi102.stdout:6/349: chown d9/db/d14/d37/c4f 24 1 2024-05-13T22:48:03.088 INFO:tasks.workunit.client.0.smithi102.stdout:0/399: chown d1/d4d 34706 1 2024-05-13T22:48:03.091 INFO:tasks.workunit.client.0.smithi102.stdout:8/279: unlink d2/d7/d25/d41/d50/c30 0 2024-05-13T22:48:03.095 INFO:tasks.workunit.client.0.smithi102.stdout:5/528: rename de/d33/d34/dc2 to de/d25/d43/d7a/db1/dc5 0 2024-05-13T22:48:03.099 INFO:tasks.workunit.client.0.smithi102.stdout:8/280: dwrite d2/d7/d12/f1c [4194304,4194304] 0 2024-05-13T22:48:03.099 INFO:tasks.workunit.client.0.smithi102.stdout:0/400: rmdir d1/de/d32/d24/d2d/d38/d3d 39 2024-05-13T22:48:03.100 INFO:tasks.workunit.client.0.smithi102.stdout:6/350: dwrite f7 [0,4194304] 0 2024-05-13T22:48:03.101 INFO:tasks.workunit.client.0.smithi102.stdout:8/281: write d2/d7/d27/f3b [3467143,55488] 0 2024-05-13T22:48:03.105 INFO:tasks.workunit.client.0.smithi102.stdout:3/316: dread de/d10/d2a/f3a [0,4194304] 0 2024-05-13T22:48:03.108 INFO:tasks.workunit.client.0.smithi102.stdout:5/529: dread de/d33/f7b [0,4194304] 0 2024-05-13T22:48:03.112 INFO:tasks.workunit.client.0.smithi102.stdout:3/317: mkdir de/d69 0 2024-05-13T22:48:03.116 INFO:tasks.workunit.client.0.smithi102.stdout:0/401: rename d1/de/d42/c4a to d1/de/d32/d24/d2d/c7c 0 2024-05-13T22:48:03.117 INFO:tasks.workunit.client.0.smithi102.stdout:0/402: dread - d1/d4d/f6a zero size 2024-05-13T22:48:03.120 INFO:tasks.workunit.client.0.smithi102.stdout:3/318: symlink de/d10/d2a/d33/d61/l6a 0 2024-05-13T22:48:03.120 INFO:tasks.workunit.client.0.smithi102.stdout:3/319: readlink de/l53 0 2024-05-13T22:48:03.121 INFO:tasks.workunit.client.0.smithi102.stdout:3/320: stat de/d10/d2a/f54 0 2024-05-13T22:48:03.121 INFO:tasks.workunit.client.0.smithi102.stdout:1/258: write d2/d4/f20 [1432048,12546] 0 2024-05-13T22:48:03.122 INFO:tasks.workunit.client.0.smithi102.stdout:1/259: fdatasync d2/f54 0 2024-05-13T22:48:03.123 INFO:tasks.workunit.client.0.smithi102.stdout:1/260: chown d2/d4/f23 685 1 2024-05-13T22:48:03.123 INFO:tasks.workunit.client.0.smithi102.stdout:1/261: readlink d2/d4/d26/l4f 0 2024-05-13T22:48:03.125 INFO:tasks.workunit.client.0.smithi102.stdout:5/530: link de/d25/d2a/f56 de/d25/d2a/d4b/d4d/d50/d6c/db9/fc6 0 2024-05-13T22:48:03.125 INFO:tasks.workunit.client.0.smithi102.stdout:1/262: write d2/d4/fd [471827,105552] 0 2024-05-13T22:48:03.126 INFO:tasks.workunit.client.0.smithi102.stdout:1/263: truncate d2/d4/d5/fa 5288852 0 2024-05-13T22:48:03.136 INFO:tasks.workunit.client.0.smithi102.stdout:0/403: dwrite d1/de/d32/f1e [0,4194304] 0 2024-05-13T22:48:03.136 INFO:tasks.workunit.client.0.smithi102.stdout:1/264: mkdir d2/d4/d55 0 2024-05-13T22:48:03.139 INFO:tasks.workunit.client.0.smithi102.stdout:1/265: rmdir d2/d4/d38 39 2024-05-13T22:48:03.142 INFO:tasks.workunit.client.0.smithi102.stdout:5/531: getdents de/d25/d59/d7f/d86 0 2024-05-13T22:48:03.144 INFO:tasks.workunit.client.0.smithi102.stdout:3/321: dread de/f37 [0,4194304] 0 2024-05-13T22:48:03.144 INFO:tasks.workunit.client.0.smithi102.stdout:0/404: dwrite d1/de/d32/f1e [0,4194304] 0 2024-05-13T22:48:03.145 INFO:tasks.workunit.client.0.smithi102.stdout:3/322: chown de/d10/d2a/d36/f58 27632177 1 2024-05-13T22:48:03.147 INFO:tasks.workunit.client.0.smithi102.stdout:3/323: write de/d10/d34/f5f [177721,85852] 0 2024-05-13T22:48:03.149 INFO:tasks.workunit.client.0.smithi102.stdout:0/405: creat d1/de/d32/d63/f7d x:0 0 0 2024-05-13T22:48:03.153 INFO:tasks.workunit.client.0.smithi102.stdout:0/406: rename d1/de/d32/d24/d2d/d3e/c5c to d1/d4d/d79/c7e 0 2024-05-13T22:48:03.155 INFO:tasks.workunit.client.0.smithi102.stdout:3/324: mknod de/d21/c6b 0 2024-05-13T22:48:03.158 INFO:tasks.workunit.client.0.smithi102.stdout:0/407: creat d1/d4d/f7f x:0 0 0 2024-05-13T22:48:03.160 INFO:tasks.workunit.client.0.smithi102.stdout:3/325: creat de/d10/d46/f6c x:0 0 0 2024-05-13T22:48:03.162 INFO:tasks.workunit.client.0.smithi102.stdout:0/408: creat d1/d4d/d79/f80 x:0 0 0 2024-05-13T22:48:03.164 INFO:tasks.workunit.client.0.smithi102.stdout:0/409: write d1/d4d/f5b [1914632,87457] 0 2024-05-13T22:48:03.169 INFO:tasks.workunit.client.0.smithi102.stdout:0/410: mkdir d1/de/d32/d24/d2d/d38/d3d/d81 0 2024-05-13T22:48:03.170 INFO:tasks.workunit.client.0.smithi102.stdout:0/411: write d1/de/d32/d63/f7d [607675,12419] 0 2024-05-13T22:48:03.174 INFO:tasks.workunit.client.0.smithi102.stdout:0/412: creat d1/f82 x:0 0 0 2024-05-13T22:48:03.186 INFO:tasks.workunit.client.0.smithi102.stdout:3/326: write de/d1c/f4f [403139,67482] 0 2024-05-13T22:48:03.194 INFO:tasks.workunit.client.0.smithi102.stdout:3/327: dwrite de/f37 [4194304,4194304] 0 2024-05-13T22:48:03.197 INFO:tasks.workunit.client.0.smithi102.stdout:3/328: mknod de/d1c/c6d 0 2024-05-13T22:48:03.201 INFO:tasks.workunit.client.0.smithi102.stdout:3/329: mknod de/d10/d46/c6e 0 2024-05-13T22:48:03.206 INFO:tasks.workunit.client.0.smithi102.stdout:0/413: creat d1/de/d32/d24/f83 x:0 0 0 2024-05-13T22:48:03.208 INFO:tasks.workunit.client.0.smithi102.stdout:3/330: dwrite de/d10/d2a/d33/f3f [0,4194304] 0 2024-05-13T22:48:03.209 INFO:tasks.workunit.client.0.smithi102.stdout:3/331: chown de/d10/d34/d5d 1676272 1 2024-05-13T22:48:03.211 INFO:tasks.workunit.client.0.smithi102.stdout:3/332: creat de/d10/d46/f6f x:0 0 0 2024-05-13T22:48:03.255 INFO:tasks.workunit.client.0.smithi102.stdout:0/414: rename d1/de/d32/l49 to d1/d4d/l84 0 2024-05-13T22:48:03.258 INFO:tasks.workunit.client.0.smithi102.stdout:3/333: chown de/d10/f12 0 1 2024-05-13T22:48:03.259 INFO:tasks.workunit.client.0.smithi102.stdout:3/334: write de/d10/d2a/d33/d61/f4c [213393,5959] 0 2024-05-13T22:48:03.260 INFO:tasks.workunit.client.0.smithi102.stdout:7/206: sync 2024-05-13T22:48:03.260 INFO:tasks.workunit.client.0.smithi102.stdout:0/415: write d1/de/d32/f18 [8751833,52072] 0 2024-05-13T22:48:03.261 INFO:tasks.workunit.client.0.smithi102.stdout:0/416: fsync d1/de/d32/d24/f3c 0 2024-05-13T22:48:03.269 INFO:tasks.workunit.client.0.smithi102.stdout:0/417: rmdir d1/de/d32/d63 39 2024-05-13T22:48:03.271 INFO:tasks.workunit.client.0.smithi102.stdout:3/335: symlink de/d68/l70 0 2024-05-13T22:48:03.273 INFO:tasks.workunit.client.0.smithi102.stdout:0/418: write d1/f2c [1973200,36161] 0 2024-05-13T22:48:03.274 INFO:tasks.workunit.client.0.smithi102.stdout:3/336: symlink de/d10/d2a/d36/l71 0 2024-05-13T22:48:03.274 INFO:tasks.workunit.client.0.smithi102.stdout:3/337: fdatasync de/d10/d50/f47 0 2024-05-13T22:48:03.278 INFO:tasks.workunit.client.0.smithi102.stdout:0/419: creat d1/d4d/f85 x:0 0 0 2024-05-13T22:48:03.289 INFO:tasks.workunit.client.0.smithi102.stdout:3/338: dread de/d10/d2a/f2f [0,4194304] 0 2024-05-13T22:48:03.300 INFO:tasks.workunit.client.0.smithi102.stdout:3/339: getdents de/d68 0 2024-05-13T22:48:03.305 INFO:tasks.workunit.client.0.smithi102.stdout:3/340: mknod de/d10/d34/d5d/c72 0 2024-05-13T22:48:03.310 INFO:tasks.workunit.client.0.smithi102.stdout:3/341: dwrite de/d10/d2a/d36/f58 [0,4194304] 0 2024-05-13T22:48:03.316 INFO:tasks.workunit.client.0.smithi102.stdout:0/420: dwrite d1/de/d32/d24/d2d/f6b [0,4194304] 0 2024-05-13T22:48:03.318 INFO:tasks.workunit.client.0.smithi102.stdout:3/342: unlink de/d10/d2a/c4e 0 2024-05-13T22:48:03.326 INFO:tasks.workunit.client.0.smithi102.stdout:0/421: creat d1/d68/f86 x:0 0 0 2024-05-13T22:48:03.332 INFO:tasks.workunit.client.0.smithi102.stdout:0/422: rename d1/de/d32/f1a to d1/d4d/d79/f87 0 2024-05-13T22:48:03.339 INFO:tasks.workunit.client.0.smithi102.stdout:0/423: symlink d1/de/d32/l88 0 2024-05-13T22:48:03.339 INFO:tasks.workunit.client.0.smithi102.stdout:0/424: dread - d1/de/f5a zero size 2024-05-13T22:48:03.347 INFO:tasks.workunit.client.0.smithi102.stdout:0/425: rename d1/de/d32/d24/d2d/d38/f4b to d1/de/d32/f89 0 2024-05-13T22:48:03.352 INFO:tasks.workunit.client.0.smithi102.stdout:0/426: mkdir d1/de/d32/d8a 0 2024-05-13T22:48:03.356 INFO:tasks.workunit.client.0.smithi102.stdout:3/343: dwrite de/d10/d50/f45 [0,4194304] 0 2024-05-13T22:48:03.361 INFO:tasks.workunit.client.0.smithi102.stdout:0/427: mknod d1/de/d32/d24/d2d/c8b 0 2024-05-13T22:48:03.363 INFO:tasks.workunit.client.0.smithi102.stdout:3/344: symlink de/d67/l73 0 2024-05-13T22:48:03.365 INFO:tasks.workunit.client.0.smithi102.stdout:0/428: mknod d1/d68/c8c 0 2024-05-13T22:48:03.365 INFO:tasks.workunit.client.0.smithi102.stdout:0/429: chown d1/d4d 2163 1 2024-05-13T22:48:03.366 INFO:tasks.workunit.client.0.smithi102.stdout:0/430: write d1/de/d32/d24/d2d/f6b [5364830,45473] 0 2024-05-13T22:48:03.367 INFO:tasks.workunit.client.0.smithi102.stdout:3/345: mknod de/d10/d2a/d33/d61/c74 0 2024-05-13T22:48:03.371 INFO:tasks.workunit.client.0.smithi102.stdout:0/431: stat d1/l6d 0 2024-05-13T22:48:03.373 INFO:tasks.workunit.client.0.smithi102.stdout:3/346: symlink de/d10/d34/d5d/l75 0 2024-05-13T22:48:03.380 INFO:tasks.workunit.client.0.smithi102.stdout:3/347: creat de/d10/d50/d63/f76 x:0 0 0 2024-05-13T22:48:03.391 INFO:tasks.workunit.client.0.smithi102.stdout:3/348: getdents de/d10/d2a 0 2024-05-13T22:48:03.403 INFO:tasks.workunit.client.0.smithi102.stdout:3/349: getdents de/d10/d46 0 2024-05-13T22:48:03.403 INFO:tasks.workunit.client.0.smithi102.stdout:3/350: stat de/d21/c6b 0 2024-05-13T22:48:03.406 INFO:tasks.workunit.client.0.smithi102.stdout:0/432: truncate d1/de/d32/f36 1365109 0 2024-05-13T22:48:03.415 INFO:tasks.workunit.client.0.smithi102.stdout:3/351: creat de/d67/f77 x:0 0 0 2024-05-13T22:48:03.419 INFO:tasks.workunit.client.0.smithi102.stdout:3/352: creat de/f78 x:0 0 0 2024-05-13T22:48:03.424 INFO:tasks.workunit.client.0.smithi102.stdout:3/353: rmdir de/d69 0 2024-05-13T22:48:03.427 INFO:tasks.workunit.client.0.smithi102.stdout:3/354: rename de/d21 to de/d1c/d79 0 2024-05-13T22:48:03.427 INFO:tasks.workunit.client.0.smithi102.stdout:3/355: chown de/d10/d2a 28286 1 2024-05-13T22:48:03.430 INFO:tasks.workunit.client.0.smithi102.stdout:3/356: mknod de/d10/c7a 0 2024-05-13T22:48:03.432 INFO:tasks.workunit.client.0.smithi102.stdout:3/357: truncate f7 162206 0 2024-05-13T22:48:03.434 INFO:tasks.workunit.client.0.smithi102.stdout:3/358: creat de/d10/d2a/f7b x:0 0 0 2024-05-13T22:48:03.521 INFO:tasks.workunit.client.0.smithi102.stdout:3/359: truncate de/d1c/f4f 490387 0 2024-05-13T22:48:03.522 INFO:tasks.workunit.client.0.smithi102.stdout:3/360: chown de/d10/d34/d5d/c72 11 1 2024-05-13T22:48:03.524 INFO:tasks.workunit.client.0.smithi102.stdout:3/361: mknod de/d10/d2a/d33/c7c 0 2024-05-13T22:48:03.527 INFO:tasks.workunit.client.0.smithi102.stdout:3/362: rename de/d10/d34/l39 to de/d1c/l7d 0 2024-05-13T22:48:03.527 INFO:tasks.workunit.client.0.smithi102.stdout:3/363: truncate f5 2339429 0 2024-05-13T22:48:03.529 INFO:tasks.workunit.client.0.smithi102.stdout:3/364: creat de/d10/d2a/f7e x:0 0 0 2024-05-13T22:48:03.530 INFO:tasks.workunit.client.0.smithi102.stdout:3/365: dread - de/d10/d50/d63/f76 zero size 2024-05-13T22:48:03.532 INFO:tasks.workunit.client.0.smithi102.stdout:3/366: mkdir de/d10/d2a/d33/d55/d7f 0 2024-05-13T22:48:03.533 INFO:tasks.workunit.client.0.smithi102.stdout:3/367: write de/d10/d50/f66 [74719,14517] 0 2024-05-13T22:48:03.551 INFO:tasks.workunit.client.0.smithi102.stdout:3/368: write de/d1c/f52 [4444,54366] 0 2024-05-13T22:48:03.559 INFO:tasks.workunit.client.0.smithi102.stdout:3/369: link de/l13 de/d1c/d79/l80 0 2024-05-13T22:48:03.564 INFO:tasks.workunit.client.0.smithi102.stdout:3/370: dwrite de/d10/d2a/f3a [0,4194304] 0 2024-05-13T22:48:03.569 INFO:tasks.workunit.client.0.smithi102.stdout:3/371: dread de/f37 [4194304,4194304] 0 2024-05-13T22:48:03.576 INFO:tasks.workunit.client.0.smithi102.stdout:3/372: dwrite f5 [0,4194304] 0 2024-05-13T22:48:03.580 INFO:tasks.workunit.client.0.smithi102.stdout:3/373: creat de/d1c/d79/f81 x:0 0 0 2024-05-13T22:48:03.583 INFO:tasks.workunit.client.0.smithi102.stdout:3/374: rmdir de/d10/d2a 39 2024-05-13T22:48:03.587 INFO:tasks.workunit.client.0.smithi102.stdout:3/375: creat de/d68/f82 x:0 0 0 2024-05-13T22:48:03.594 INFO:tasks.workunit.client.0.smithi102.stdout:4/290: sync 2024-05-13T22:48:03.596 INFO:tasks.workunit.client.0.smithi102.stdout:4/291: unlink d3/f10 0 2024-05-13T22:48:03.610 INFO:tasks.workunit.client.0.smithi102.stdout:4/292: unlink d3/d19/l4c 0 2024-05-13T22:48:03.611 INFO:tasks.workunit.client.0.smithi102.stdout:3/376: dread de/d10/f44 [0,4194304] 0 2024-05-13T22:48:03.614 INFO:tasks.workunit.client.0.smithi102.stdout:4/293: read d3/d4/d6/de/d1f/f28 [1456311,30660] 0 2024-05-13T22:48:03.618 INFO:tasks.workunit.client.0.smithi102.stdout:3/377: dwrite f5 [0,4194304] 0 2024-05-13T22:48:03.618 INFO:tasks.workunit.client.0.smithi102.stdout:4/294: dread d3/d4/fd [0,4194304] 0 2024-05-13T22:48:03.622 INFO:tasks.workunit.client.0.smithi102.stdout:3/378: stat de/d10/d2a/d33/d61/l5a 0 2024-05-13T22:48:03.623 INFO:tasks.workunit.client.0.smithi102.stdout:4/295: dread d3/d22/f5b [0,4194304] 0 2024-05-13T22:48:03.625 INFO:tasks.workunit.client.0.smithi102.stdout:3/379: unlink f5 0 2024-05-13T22:48:03.628 INFO:tasks.workunit.client.0.smithi102.stdout:4/296: mkdir d3/d4/d6/de/d1f/d41/d6a 0 2024-05-13T22:48:03.629 INFO:tasks.workunit.client.0.smithi102.stdout:3/380: rename de/d10/d34/f5f to de/d10/d2a/d36/f83 0 2024-05-13T22:48:03.631 INFO:tasks.workunit.client.0.smithi102.stdout:4/297: rmdir d3/d19/d23/d39 39 2024-05-13T22:48:03.632 INFO:tasks.workunit.client.0.smithi102.stdout:4/298: chown d3/d19/d23/d3a/f51 1515855 1 2024-05-13T22:48:03.633 INFO:tasks.workunit.client.0.smithi102.stdout:3/381: rename de/d10/d2a/d33/d61/l6a to de/d68/l84 0 2024-05-13T22:48:03.634 INFO:tasks.workunit.client.0.smithi102.stdout:4/299: truncate d3/d19/d23/f3d 1827529 0 2024-05-13T22:48:03.639 INFO:tasks.workunit.client.0.smithi102.stdout:4/300: mkdir d3/d6b 0 2024-05-13T22:48:03.641 INFO:tasks.workunit.client.0.smithi102.stdout:4/301: mknod d3/d4/d6/de/d1f/d41/c6c 0 2024-05-13T22:48:03.645 INFO:tasks.workunit.client.0.smithi102.stdout:4/302: dread d3/d19/d23/f43 [0,4194304] 0 2024-05-13T22:48:03.647 INFO:tasks.workunit.client.0.smithi102.stdout:4/303: chown d3/d4/d6/de/l52 5565658 1 2024-05-13T22:48:03.648 INFO:tasks.workunit.client.0.smithi102.stdout:4/304: chown d3/d4/d6/f42 257083 1 2024-05-13T22:48:03.650 INFO:tasks.workunit.client.0.smithi102.stdout:4/305: mknod d3/d4/d6/de/c6d 0 2024-05-13T22:48:03.651 INFO:tasks.workunit.client.0.smithi102.stdout:4/306: truncate d3/f35 4734470 0 2024-05-13T22:48:03.652 INFO:tasks.workunit.client.0.smithi102.stdout:3/382: write de/d10/f14 [518263,86843] 0 2024-05-13T22:48:03.655 INFO:tasks.workunit.client.0.smithi102.stdout:4/307: rename d3/c12 to d3/d19/d23/d4e/d68/c6e 0 2024-05-13T22:48:03.659 INFO:tasks.workunit.client.0.smithi102.stdout:4/308: dread d3/d4/f32 [0,4194304] 0 2024-05-13T22:48:03.662 INFO:tasks.workunit.client.0.smithi102.stdout:4/309: creat d3/d6b/f6f x:0 0 0 2024-05-13T22:48:03.663 INFO:tasks.workunit.client.0.smithi102.stdout:3/383: write f2 [3416582,51028] 0 2024-05-13T22:48:03.664 INFO:tasks.workunit.client.0.smithi102.stdout:4/310: creat d3/d22/d2b/f70 x:0 0 0 2024-05-13T22:48:03.665 INFO:tasks.workunit.client.0.smithi102.stdout:3/384: chown de/d1c/l7d 113 1 2024-05-13T22:48:03.667 INFO:tasks.workunit.client.0.smithi102.stdout:3/385: write de/d10/d2a/d4b/f60 [927699,46779] 0 2024-05-13T22:48:03.670 INFO:tasks.workunit.client.0.smithi102.stdout:3/386: creat de/d10/d50/d63/f85 x:0 0 0 2024-05-13T22:48:03.672 INFO:tasks.workunit.client.0.smithi102.stdout:4/311: dwrite d3/d19/d23/f3d [0,4194304] 0 2024-05-13T22:48:03.674 INFO:tasks.workunit.client.0.smithi102.stdout:3/387: symlink de/d1c/d79/l86 0 2024-05-13T22:48:03.677 INFO:tasks.workunit.client.0.smithi102.stdout:3/388: symlink de/d68/l87 0 2024-05-13T22:48:03.679 INFO:tasks.workunit.client.0.smithi102.stdout:3/389: write de/d10/d2a/d33/d61/f64 [873431,115640] 0 2024-05-13T22:48:03.679 INFO:tasks.workunit.client.0.smithi102.stdout:3/390: chown de/d10/l4a 2075694 1 2024-05-13T22:48:03.683 INFO:tasks.workunit.client.0.smithi102.stdout:3/391: mkdir de/d10/d2a/d36/d88 0 2024-05-13T22:48:03.685 INFO:tasks.workunit.client.0.smithi102.stdout:4/312: link d3/d4/d6/de/f3f d3/d22/f71 0 2024-05-13T22:48:03.775 INFO:tasks.workunit.client.0.smithi102.stdout:3/392: dread de/d10/d2a/d33/f40 [0,4194304] 0 2024-05-13T22:48:03.777 INFO:tasks.workunit.client.0.smithi102.stdout:3/393: creat de/d10/d50/f89 x:0 0 0 2024-05-13T22:48:03.781 INFO:tasks.workunit.client.0.smithi102.stdout:3/394: link de/d10/f43 de/d1c/f8a 0 2024-05-13T22:48:03.795 INFO:tasks.workunit.client.0.smithi102.stdout:4/313: write d3/d4/f1e [3427539,57998] 0 2024-05-13T22:48:03.798 INFO:tasks.workunit.client.0.smithi102.stdout:4/314: creat d3/d19/d23/d4e/f72 x:0 0 0 2024-05-13T22:48:03.799 INFO:tasks.workunit.client.0.smithi102.stdout:4/315: read d3/d19/d23/f43 [489871,18514] 0 2024-05-13T22:48:03.801 INFO:tasks.workunit.client.0.smithi102.stdout:4/316: unlink d3/d4/d6/de/f46 0 2024-05-13T22:48:03.802 INFO:tasks.workunit.client.0.smithi102.stdout:3/395: fsync de/d10/f43 0 2024-05-13T22:48:03.808 INFO:tasks.workunit.client.0.smithi102.stdout:4/317: dwrite d3/d19/d23/d3a/f4a [0,4194304] 0 2024-05-13T22:48:03.810 INFO:tasks.workunit.client.0.smithi102.stdout:4/318: write d3/d4/d6/f1c [208276,29977] 0 2024-05-13T22:48:03.811 INFO:tasks.workunit.client.0.smithi102.stdout:3/396: link de/l1a de/d10/l8b 0 2024-05-13T22:48:03.819 INFO:tasks.workunit.client.0.smithi102.stdout:3/397: dwrite de/d10/d2a/f7b [0,4194304] 0 2024-05-13T22:48:03.821 INFO:tasks.workunit.client.0.smithi102.stdout:4/319: dread d3/d4/d6/f42 [0,4194304] 0 2024-05-13T22:48:03.821 INFO:tasks.workunit.client.0.smithi102.stdout:3/398: rmdir de/d10/d2a/d33/d55 39 2024-05-13T22:48:03.825 INFO:tasks.workunit.client.0.smithi102.stdout:4/320: fdatasync d3/d4/f32 0 2024-05-13T22:48:03.830 INFO:tasks.workunit.client.0.smithi102.stdout:3/399: rename de/d10/d2a/d33/d55 to de/d10/d2a/d8c 0 2024-05-13T22:48:03.833 INFO:tasks.workunit.client.0.smithi102.stdout:3/400: rename de/d10/d2a/d33/c7c to de/d10/d2a/d36/d88/c8d 0 2024-05-13T22:48:03.834 INFO:tasks.workunit.client.0.smithi102.stdout:3/401: truncate de/d10/d50/f89 844778 0 2024-05-13T22:48:03.838 INFO:tasks.workunit.client.0.smithi102.stdout:3/402: link de/d10/l4a de/d10/d2a/d36/l8e 0 2024-05-13T22:48:03.839 INFO:tasks.workunit.client.0.smithi102.stdout:3/403: stat de/d10/c18 0 2024-05-13T22:48:03.840 INFO:tasks.workunit.client.0.smithi102.stdout:3/404: dread - de/d10/d50/f47 zero size 2024-05-13T22:48:03.841 INFO:tasks.workunit.client.0.smithi102.stdout:4/321: dwrite d3/d22/f71 [0,4194304] 0 2024-05-13T22:48:03.843 INFO:tasks.workunit.client.0.smithi102.stdout:4/322: read - d3/d19/d23/d3a/f56 zero size 2024-05-13T22:48:03.844 INFO:tasks.workunit.client.0.smithi102.stdout:4/323: truncate d3/d4/f5a 246399 0 2024-05-13T22:48:03.845 INFO:tasks.workunit.client.0.smithi102.stdout:3/405: rename de/d10/d2a/d4b to de/d10/d50/d8f 0 2024-05-13T22:48:03.847 INFO:tasks.workunit.client.0.smithi102.stdout:4/324: mknod d3/d4/d6/de/c73 0 2024-05-13T22:48:03.851 INFO:tasks.workunit.client.0.smithi102.stdout:3/406: getdents de/d10/d2a/d8c 0 2024-05-13T22:48:03.851 INFO:tasks.workunit.client.0.smithi102.stdout:4/325: creat d3/d22/f74 x:0 0 0 2024-05-13T22:48:03.854 INFO:tasks.workunit.client.0.smithi102.stdout:4/326: creat d3/d19/d23/d4b/f75 x:0 0 0 2024-05-13T22:48:03.857 INFO:tasks.workunit.client.0.smithi102.stdout:3/407: dwrite de/d68/f82 [0,4194304] 0 2024-05-13T22:48:03.858 INFO:tasks.workunit.client.0.smithi102.stdout:3/408: chown de/d10/d2a/d8c 0 1 2024-05-13T22:48:03.860 INFO:tasks.workunit.client.0.smithi102.stdout:4/327: getdents d3/d4/d6/de/d1f/d41/d6a 0 2024-05-13T22:48:03.862 INFO:tasks.workunit.client.0.smithi102.stdout:3/409: mkdir de/d1c/d90 0 2024-05-13T22:48:03.864 INFO:tasks.workunit.client.0.smithi102.stdout:4/328: creat d3/d19/d23/d4e/f76 x:0 0 0 2024-05-13T22:48:03.866 INFO:tasks.workunit.client.0.smithi102.stdout:4/329: write d3/d4/d6/f1c [1140977,76628] 0 2024-05-13T22:48:03.868 INFO:tasks.workunit.client.0.smithi102.stdout:9/364: sync 2024-05-13T22:48:03.868 INFO:tasks.workunit.client.0.smithi102.stdout:2/330: sync 2024-05-13T22:48:03.872 INFO:tasks.workunit.client.0.smithi102.stdout:9/365: dread d1/d6/db/d30/d47/f49 [0,4194304] 0 2024-05-13T22:48:03.884 INFO:tasks.workunit.client.0.smithi102.stdout:3/410: dread de/d10/f43 [0,4194304] 0 2024-05-13T22:48:03.887 INFO:tasks.workunit.client.0.smithi102.stdout:3/411: unlink de/d10/c7a 0 2024-05-13T22:48:03.887 INFO:tasks.workunit.client.0.smithi102.stdout:4/330: write d3/d4/d6/de/f15 [890433,38268] 0 2024-05-13T22:48:03.889 INFO:tasks.workunit.client.0.smithi102.stdout:2/331: write d7/d15/d1e/f26 [779901,4206] 0 2024-05-13T22:48:03.894 INFO:tasks.workunit.client.0.smithi102.stdout:4/331: mkdir d3/d19/d23/d39/d77 0 2024-05-13T22:48:03.894 INFO:tasks.workunit.client.0.smithi102.stdout:3/412: dwrite de/d1c/d79/f2b [0,4194304] 0 2024-05-13T22:48:03.897 INFO:tasks.workunit.client.0.smithi102.stdout:2/332: dwrite d7/d15/d1e/f5c [0,4194304] 0 2024-05-13T22:48:03.898 INFO:tasks.workunit.client.0.smithi102.stdout:3/413: creat de/d1c/f91 x:0 0 0 2024-05-13T22:48:03.902 INFO:tasks.workunit.client.0.smithi102.stdout:3/414: fdatasync de/d1c/f8a 0 2024-05-13T22:48:03.903 INFO:tasks.workunit.client.0.smithi102.stdout:2/333: creat d7/d9/d10/d1b/f6b x:0 0 0 2024-05-13T22:48:03.904 INFO:tasks.workunit.client.0.smithi102.stdout:2/334: chown d7/d9/d10/d30/d53 381509 1 2024-05-13T22:48:03.906 INFO:tasks.workunit.client.0.smithi102.stdout:3/415: write de/d10/d2a/d33/f40 [4263527,18450] 0 2024-05-13T22:48:03.908 INFO:tasks.workunit.client.0.smithi102.stdout:2/335: symlink d7/d9/l6c 0 2024-05-13T22:48:03.910 INFO:tasks.workunit.client.0.smithi102.stdout:3/416: rename de/d10/d2a/d33/d61/f5e to de/d1c/d90/f92 0 2024-05-13T22:48:03.911 INFO:tasks.workunit.client.0.smithi102.stdout:3/417: chown de/d10/d50/f47 55667 1 2024-05-13T22:48:03.916 INFO:tasks.workunit.client.0.smithi102.stdout:2/336: mknod d7/d9/d10/d25/d56/d59/c6d 0 2024-05-13T22:48:03.916 INFO:tasks.workunit.client.0.smithi102.stdout:4/332: dwrite d3/d4/d6/f42 [0,4194304] 0 2024-05-13T22:48:03.918 INFO:tasks.workunit.client.0.smithi102.stdout:3/418: creat de/d10/d34/f93 x:0 0 0 2024-05-13T22:48:03.921 INFO:tasks.workunit.client.0.smithi102.stdout:2/337: dread d7/d15/f52 [0,4194304] 0 2024-05-13T22:48:03.921 INFO:tasks.workunit.client.0.smithi102.stdout:3/419: rmdir de/d67 39 2024-05-13T22:48:03.922 INFO:tasks.workunit.client.0.smithi102.stdout:3/420: chown de/d10/d2a/d36/f49 3306134 1 2024-05-13T22:48:03.923 INFO:tasks.workunit.client.0.smithi102.stdout:2/338: creat d7/d15/d67/f6e x:0 0 0 2024-05-13T22:48:03.926 INFO:tasks.workunit.client.0.smithi102.stdout:3/421: read f7 [29309,9354] 0 2024-05-13T22:48:03.926 INFO:tasks.workunit.client.0.smithi102.stdout:4/333: dread d3/d4/f5a [0,4194304] 0 2024-05-13T22:48:03.929 INFO:tasks.workunit.client.0.smithi102.stdout:2/339: rename d7/l8 to d7/d9/d10/d25/d56/l6f 0 2024-05-13T22:48:03.932 INFO:tasks.workunit.client.0.smithi102.stdout:3/422: creat de/d68/f94 x:0 0 0 2024-05-13T22:48:03.935 INFO:tasks.workunit.client.0.smithi102.stdout:4/334: creat d3/d22/f78 x:0 0 0 2024-05-13T22:48:03.936 INFO:tasks.workunit.client.0.smithi102.stdout:2/340: dwrite d7/d9/d10/d1b/f6b [0,4194304] 0 2024-05-13T22:48:03.939 INFO:tasks.workunit.client.0.smithi102.stdout:3/423: read - de/d10/d2a/d36/f49 zero size 2024-05-13T22:48:03.941 INFO:tasks.workunit.client.0.smithi102.stdout:4/335: creat d3/d19/d23/d4e/f79 x:0 0 0 2024-05-13T22:48:03.943 INFO:tasks.workunit.client.0.smithi102.stdout:2/341: creat d7/d15/d67/f70 x:0 0 0 2024-05-13T22:48:03.947 INFO:tasks.workunit.client.0.smithi102.stdout:4/336: creat d3/d4/d6/de/d1f/d41/d6a/f7a x:0 0 0 2024-05-13T22:48:03.948 INFO:tasks.workunit.client.0.smithi102.stdout:4/337: stat d3/d4/d6/f18 0 2024-05-13T22:48:03.955 INFO:tasks.workunit.client.0.smithi102.stdout:4/338: link d3/d4/d6/de/d1f/d41/d6a/f7a d3/d4/f7b 0 2024-05-13T22:48:03.962 INFO:tasks.workunit.client.0.smithi102.stdout:4/339: dwrite d3/d22/f78 [0,4194304] 0 2024-05-13T22:48:03.962 INFO:tasks.workunit.client.0.smithi102.stdout:3/424: dwrite de/d1c/f8a [0,4194304] 0 2024-05-13T22:48:03.963 INFO:tasks.workunit.client.0.smithi102.stdout:3/425: write de/d10/d34/f93 [724967,1137] 0 2024-05-13T22:48:03.966 INFO:tasks.workunit.client.0.smithi102.stdout:4/340: rmdir d3/d6b 39 2024-05-13T22:48:03.968 INFO:tasks.workunit.client.0.smithi102.stdout:3/426: symlink de/d10/d2a/l95 0 2024-05-13T22:48:03.971 INFO:tasks.workunit.client.0.smithi102.stdout:4/341: creat d3/d19/d23/d39/d77/f7c x:0 0 0 2024-05-13T22:48:03.973 INFO:tasks.workunit.client.0.smithi102.stdout:3/427: rename de/d67/f77 to de/d10/d2a/d33/d61/f96 0 2024-05-13T22:48:03.977 INFO:tasks.workunit.client.0.smithi102.stdout:4/342: getdents d3/d19/d23/d3a 0 2024-05-13T22:48:03.979 INFO:tasks.workunit.client.0.smithi102.stdout:4/343: chown d3/d19/d23/d4e/f76 4225888 1 2024-05-13T22:48:03.979 INFO:tasks.workunit.client.0.smithi102.stdout:3/428: creat de/d10/f97 x:0 0 0 2024-05-13T22:48:03.982 INFO:tasks.workunit.client.0.smithi102.stdout:3/429: mkdir de/d10/d50/d63/d98 0 2024-05-13T22:48:03.986 INFO:tasks.workunit.client.0.smithi102.stdout:3/430: creat de/d10/d50/d63/d98/f99 x:0 0 0 2024-05-13T22:48:03.988 INFO:tasks.workunit.client.0.smithi102.stdout:3/431: creat de/d10/d34/f9a x:0 0 0 2024-05-13T22:48:03.994 INFO:tasks.workunit.client.0.smithi102.stdout:4/344: write d3/d22/f3e [1428032,29202] 0 2024-05-13T22:48:03.995 INFO:tasks.workunit.client.0.smithi102.stdout:4/345: readlink d3/l21 0 2024-05-13T22:48:03.996 INFO:tasks.workunit.client.0.smithi102.stdout:3/432: dwrite de/d10/d2a/d33/f40 [4194304,4194304] 0 2024-05-13T22:48:03.997 INFO:tasks.workunit.client.0.smithi102.stdout:3/433: write de/d10/d2a/d33/d61/f64 [85177,12086] 0 2024-05-13T22:48:04.001 INFO:tasks.workunit.client.0.smithi102.stdout:3/434: mknod de/d10/d34/c9b 0 2024-05-13T22:48:04.003 INFO:tasks.workunit.client.0.smithi102.stdout:4/346: dwrite d3/d4/f1e [4194304,4194304] 0 2024-05-13T22:48:04.005 INFO:tasks.workunit.client.0.smithi102.stdout:3/435: creat de/d10/d2a/f9c x:0 0 0 2024-05-13T22:48:04.008 INFO:tasks.workunit.client.0.smithi102.stdout:3/436: readlink de/d10/l8b 0 2024-05-13T22:48:04.012 INFO:tasks.workunit.client.0.smithi102.stdout:4/347: dwrite d3/d19/d23/d4e/f76 [0,4194304] 0 2024-05-13T22:48:04.017 INFO:tasks.workunit.client.0.smithi102.stdout:4/348: rename d3/d22/d2b to d3/d19/d23/d4b/d7d 0 2024-05-13T22:48:04.018 INFO:tasks.workunit.client.0.smithi102.stdout:4/349: write d3/d19/d23/f37 [2455137,79933] 0 2024-05-13T22:48:04.022 INFO:tasks.workunit.client.0.smithi102.stdout:4/350: mkdir d3/d19/d23/d7e 0 2024-05-13T22:48:04.028 INFO:tasks.workunit.client.0.smithi102.stdout:4/351: dread d3/d19/d23/d4e/f76 [0,4194304] 0 2024-05-13T22:48:04.029 INFO:tasks.workunit.client.0.smithi102.stdout:4/352: write d3/d4/f2e [3509196,107842] 0 2024-05-13T22:48:04.036 INFO:tasks.workunit.client.0.smithi102.stdout:4/353: dwrite d3/d4/f31 [0,4194304] 0 2024-05-13T22:48:04.037 INFO:tasks.workunit.client.0.smithi102.stdout:3/437: dread de/d10/d2a/f2e [0,4194304] 0 2024-05-13T22:48:04.039 INFO:tasks.workunit.client.0.smithi102.stdout:3/438: creat de/d68/f9d x:0 0 0 2024-05-13T22:48:04.041 INFO:tasks.workunit.client.0.smithi102.stdout:4/354: rename d3/d19/d23/d4b/d7d/l30 to d3/d4/d6/de/d1f/d41/l7f 0 2024-05-13T22:48:04.045 INFO:tasks.workunit.client.0.smithi102.stdout:6/351: sync 2024-05-13T22:48:04.045 INFO:tasks.workunit.client.0.smithi102.stdout:7/207: sync 2024-05-13T22:48:04.045 INFO:tasks.workunit.client.0.smithi102.stdout:1/266: sync 2024-05-13T22:48:04.045 INFO:tasks.workunit.client.0.smithi102.stdout:0/433: sync 2024-05-13T22:48:04.045 INFO:tasks.workunit.client.0.smithi102.stdout:5/532: sync 2024-05-13T22:48:04.045 INFO:tasks.workunit.client.0.smithi102.stdout:8/282: sync 2024-05-13T22:48:04.046 INFO:tasks.workunit.client.0.smithi102.stdout:0/434: chown d1/de/c12 1342141 1 2024-05-13T22:48:04.047 INFO:tasks.workunit.client.0.smithi102.stdout:8/283: read d2/d17/f2c [496541,3837] 0 2024-05-13T22:48:04.047 INFO:tasks.workunit.client.0.smithi102.stdout:5/533: write de/d25/d2a/d4b/fc1 [914022,104297] 0 2024-05-13T22:48:04.052 INFO:tasks.workunit.client.0.smithi102.stdout:0/435: write d1/de/d42/d61/f67 [942640,42311] 0 2024-05-13T22:48:04.053 INFO:tasks.workunit.client.0.smithi102.stdout:5/534: chown de/d25/d59/d28/f66 1607451 1 2024-05-13T22:48:04.055 INFO:tasks.workunit.client.0.smithi102.stdout:5/535: write de/d25/d43/d7a/fae [133041,65370] 0 2024-05-13T22:48:04.056 INFO:tasks.workunit.client.0.smithi102.stdout:5/536: read - de/d25/d43/d7a/f88 zero size 2024-05-13T22:48:04.056 INFO:tasks.workunit.client.0.smithi102.stdout:3/439: mknod de/c9e 0 2024-05-13T22:48:04.059 INFO:tasks.workunit.client.0.smithi102.stdout:6/352: mkdir d9/d25/d71 0 2024-05-13T22:48:04.061 INFO:tasks.workunit.client.0.smithi102.stdout:7/208: mkdir d6/d13/d2c/d39 0 2024-05-13T22:48:04.061 INFO:tasks.workunit.client.0.smithi102.stdout:7/209: stat d6/f2a 0 2024-05-13T22:48:04.064 INFO:tasks.workunit.client.0.smithi102.stdout:8/284: write d2/d7/d27/f31 [902873,81299] 0 2024-05-13T22:48:04.065 INFO:tasks.workunit.client.0.smithi102.stdout:1/267: mkdir d2/d4/d26/d56 0 2024-05-13T22:48:04.067 INFO:tasks.workunit.client.0.smithi102.stdout:3/440: symlink de/d10/d2a/d8c/l9f 0 2024-05-13T22:48:04.068 INFO:tasks.workunit.client.0.smithi102.stdout:5/537: mknod de/d25/d2a/d4b/cc7 0 2024-05-13T22:48:04.070 INFO:tasks.workunit.client.0.smithi102.stdout:0/436: mkdir d1/de/d32/d8d 0 2024-05-13T22:48:04.072 INFO:tasks.workunit.client.0.smithi102.stdout:4/355: creat d3/d4/d6/f80 x:0 0 0 2024-05-13T22:48:04.072 INFO:tasks.workunit.client.0.smithi102.stdout:7/210: dread d6/d13/de/fb [0,4194304] 0 2024-05-13T22:48:04.074 INFO:tasks.workunit.client.0.smithi102.stdout:3/441: mkdir de/d10/d2a/d36/da0 0 2024-05-13T22:48:04.075 INFO:tasks.workunit.client.0.smithi102.stdout:1/268: rmdir d2/d4/d38 39 2024-05-13T22:48:04.076 INFO:tasks.workunit.client.0.smithi102.stdout:6/353: dwrite d9/db/d14/d37/f48 [0,4194304] 0 2024-05-13T22:48:04.077 INFO:tasks.workunit.client.0.smithi102.stdout:1/269: chown d2/d4/d5/d19/d28/c49 0 1 2024-05-13T22:48:04.080 INFO:tasks.workunit.client.0.smithi102.stdout:0/437: rename d1/de/d32/d24/f2e to d1/de/d42/f8e 0 2024-05-13T22:48:04.082 INFO:tasks.workunit.client.0.smithi102.stdout:0/438: fdatasync d1/de/d42/d61/f67 0 2024-05-13T22:48:04.082 INFO:tasks.workunit.client.0.smithi102.stdout:3/442: dread de/d10/d50/d8f/f60 [0,4194304] 0 2024-05-13T22:48:04.083 INFO:tasks.workunit.client.0.smithi102.stdout:1/270: dread d2/d4/f20 [0,4194304] 0 2024-05-13T22:48:04.085 INFO:tasks.workunit.client.0.smithi102.stdout:1/271: truncate d2/d4/d26/f33 4852584 0 2024-05-13T22:48:04.085 INFO:tasks.workunit.client.0.smithi102.stdout:1/272: chown d2/d4/d5/d19 25171118 1 2024-05-13T22:48:04.087 INFO:tasks.workunit.client.0.smithi102.stdout:6/354: dread - d9/db/f3c zero size 2024-05-13T22:48:04.096 INFO:tasks.workunit.client.0.smithi102.stdout:4/356: rename d3/d19/d23/d4b to d3/d6b/d81 0 2024-05-13T22:48:04.096 INFO:tasks.workunit.client.0.smithi102.stdout:0/439: chown d1/c65 519512412 1 2024-05-13T22:48:04.098 INFO:tasks.workunit.client.0.smithi102.stdout:5/538: creat de/fc8 x:0 0 0 2024-05-13T22:48:04.100 INFO:tasks.workunit.client.0.smithi102.stdout:3/443: mknod de/d1c/ca1 0 2024-05-13T22:48:04.100 INFO:tasks.workunit.client.0.smithi102.stdout:5/539: write de/d75/f85 [875717,32918] 0 2024-05-13T22:48:04.101 INFO:tasks.workunit.client.0.smithi102.stdout:1/273: chown d2/d4/d38 969 1 2024-05-13T22:48:04.111 INFO:tasks.workunit.client.0.smithi102.stdout:3/444: mkdir de/d10/d50/d63/da2 0 2024-05-13T22:48:04.112 INFO:tasks.workunit.client.0.smithi102.stdout:3/445: chown de/d1c/d90/f92 112979 1 2024-05-13T22:48:04.114 INFO:tasks.workunit.client.0.smithi102.stdout:5/540: truncate de/d33/f7b 2336912 0 2024-05-13T22:48:04.115 INFO:tasks.workunit.client.0.smithi102.stdout:1/274: symlink d2/d4/d26/d40/l57 0 2024-05-13T22:48:04.116 INFO:tasks.workunit.client.0.smithi102.stdout:6/355: rename d9/c1e to d9/d25/d60/d6a/c72 0 2024-05-13T22:48:04.118 INFO:tasks.workunit.client.0.smithi102.stdout:0/440: truncate d1/de/d32/f3f 1017545 0 2024-05-13T22:48:04.119 INFO:tasks.workunit.client.0.smithi102.stdout:3/446: unlink de/d10/l5b 0 2024-05-13T22:48:04.121 INFO:tasks.workunit.client.0.smithi102.stdout:1/275: chown d2/f34 194518254 1 2024-05-13T22:48:04.123 INFO:tasks.workunit.client.0.smithi102.stdout:1/276: chown d2/d4/d5/d19/d28 186750690 1 2024-05-13T22:48:04.123 INFO:tasks.workunit.client.0.smithi102.stdout:4/357: creat d3/d19/d23/f82 x:0 0 0 2024-05-13T22:48:04.125 INFO:tasks.workunit.client.0.smithi102.stdout:3/447: mknod de/d10/d2a/d36/da0/ca3 0 2024-05-13T22:48:04.127 INFO:tasks.workunit.client.0.smithi102.stdout:3/448: write de/d10/d2a/f54 [1882282,110096] 0 2024-05-13T22:48:04.129 INFO:tasks.workunit.client.0.smithi102.stdout:5/541: dread de/f21 [0,4194304] 0 2024-05-13T22:48:04.130 INFO:tasks.workunit.client.0.smithi102.stdout:3/449: creat de/d10/d2a/fa4 x:0 0 0 2024-05-13T22:48:04.131 INFO:tasks.workunit.client.0.smithi102.stdout:1/277: dwrite d2/d4/d5/d19/d28/f24 [0,4194304] 0 2024-05-13T22:48:04.132 INFO:tasks.workunit.client.0.smithi102.stdout:3/450: chown de/d10/d50/d8f/c59 16 1 2024-05-13T22:48:04.132 INFO:tasks.workunit.client.0.smithi102.stdout:1/278: fsync d2/d4/d38/f52 0 2024-05-13T22:48:04.133 INFO:tasks.workunit.client.0.smithi102.stdout:5/542: creat de/d25/d59/d7f/fc9 x:0 0 0 2024-05-13T22:48:04.138 INFO:tasks.workunit.client.0.smithi102.stdout:1/279: rename d2/d4/d38/f52 to d2/d45/f58 0 2024-05-13T22:48:04.139 INFO:tasks.workunit.client.0.smithi102.stdout:5/543: dwrite de/d25/d59/d7f/f8e [0,4194304] 0 2024-05-13T22:48:04.142 INFO:tasks.workunit.client.0.smithi102.stdout:3/451: link de/d10/d50/d8f/c65 de/d1c/d90/ca5 0 2024-05-13T22:48:04.144 INFO:tasks.workunit.client.0.smithi102.stdout:3/452: stat de/d1c/d79/l3e 0 2024-05-13T22:48:04.147 INFO:tasks.workunit.client.0.smithi102.stdout:3/453: symlink de/d1c/d79/la6 0 2024-05-13T22:48:04.149 INFO:tasks.workunit.client.0.smithi102.stdout:3/454: chown de/d10/d34/c9b 478981 1 2024-05-13T22:48:04.152 INFO:tasks.workunit.client.0.smithi102.stdout:1/280: dwrite d2/d10/f21 [4194304,4194304] 0 2024-05-13T22:48:04.155 INFO:tasks.workunit.client.0.smithi102.stdout:3/455: symlink de/d10/d2a/d8c/d7f/la7 0 2024-05-13T22:48:04.156 INFO:tasks.workunit.client.0.smithi102.stdout:3/456: fsync de/d10/d2a/d33/d61/f56 0 2024-05-13T22:48:04.160 INFO:tasks.workunit.client.0.smithi102.stdout:3/457: mkdir de/d10/d2a/d8c/da8 0 2024-05-13T22:48:04.160 INFO:tasks.workunit.client.0.smithi102.stdout:1/281: dread d2/d10/f31 [0,4194304] 0 2024-05-13T22:48:04.163 INFO:tasks.workunit.client.0.smithi102.stdout:3/458: chown de/l13 47334720 1 2024-05-13T22:48:04.169 INFO:tasks.workunit.client.0.smithi102.stdout:3/459: symlink de/d10/d2a/d36/la9 0 2024-05-13T22:48:04.176 INFO:tasks.workunit.client.0.smithi102.stdout:3/460: mkdir de/d10/d2a/d33/daa 0 2024-05-13T22:48:04.182 INFO:tasks.workunit.client.0.smithi102.stdout:1/282: dread d2/d4/d5/d19/d28/f3b [0,4194304] 0 2024-05-13T22:48:04.183 INFO:tasks.workunit.client.0.smithi102.stdout:1/283: fdatasync d2/f54 0 2024-05-13T22:48:04.184 INFO:tasks.workunit.client.0.smithi102.stdout:3/461: dwrite de/d10/d2a/f2e [0,4194304] 0 2024-05-13T22:48:04.189 INFO:tasks.workunit.client.0.smithi102.stdout:3/462: readlink de/d10/d2a/d36/l8e 0 2024-05-13T22:48:04.191 INFO:tasks.workunit.client.0.smithi102.stdout:1/284: creat d2/d4/d26/d4c/f59 x:0 0 0 2024-05-13T22:48:04.194 INFO:tasks.workunit.client.0.smithi102.stdout:1/285: stat d2/f4b 0 2024-05-13T22:48:04.195 INFO:tasks.workunit.client.0.smithi102.stdout:1/286: write d2/d4/d5/fa [213882,101216] 0 2024-05-13T22:48:04.207 INFO:tasks.workunit.client.0.smithi102.stdout:1/287: link d2/d4/f20 d2/d45/f5a 0 2024-05-13T22:48:04.211 INFO:tasks.workunit.client.0.smithi102.stdout:1/288: symlink d2/d4/d5/d19/d28/l5b 0 2024-05-13T22:48:04.214 INFO:tasks.workunit.client.0.smithi102.stdout:1/289: mknod d2/d10/c5c 0 2024-05-13T22:48:04.230 INFO:tasks.workunit.client.0.smithi102.stdout:3/463: truncate de/d10/d2a/f2e 4569584 0 2024-05-13T22:48:04.234 INFO:tasks.workunit.client.0.smithi102.stdout:3/464: creat de/d10/d50/fab x:0 0 0 2024-05-13T22:48:04.236 INFO:tasks.workunit.client.0.smithi102.stdout:3/465: creat de/d10/d46/fac x:0 0 0 2024-05-13T22:48:04.237 INFO:tasks.workunit.client.0.smithi102.stdout:3/466: write de/d10/d46/f6c [557427,11927] 0 2024-05-13T22:48:04.245 INFO:tasks.workunit.client.0.smithi102.stdout:9/366: sync 2024-05-13T22:48:04.250 INFO:tasks.workunit.client.0.smithi102.stdout:9/367: creat d1/d5d/d6c/f86 x:0 0 0 2024-05-13T22:48:04.251 INFO:tasks.workunit.client.0.smithi102.stdout:1/290: write d2/d45/f5a [2175166,83259] 0 2024-05-13T22:48:04.255 INFO:tasks.workunit.client.0.smithi102.stdout:1/291: creat d2/d10/f5d x:0 0 0 2024-05-13T22:48:04.257 INFO:tasks.workunit.client.0.smithi102.stdout:9/368: creat d1/d6/db/d4a/d64/f87 x:0 0 0 2024-05-13T22:48:04.260 INFO:tasks.workunit.client.0.smithi102.stdout:1/292: write d2/f2b [85438,5618] 0 2024-05-13T22:48:04.262 INFO:tasks.workunit.client.0.smithi102.stdout:9/369: mknod d1/d6/db/d4a/c88 0 2024-05-13T22:48:04.263 INFO:tasks.workunit.client.0.smithi102.stdout:9/370: dread - d1/d6/db/d4a/f85 zero size 2024-05-13T22:48:04.277 INFO:tasks.workunit.client.0.smithi102.stdout:3/467: dwrite de/d10/d2a/f2f [0,4194304] 0 2024-05-13T22:48:04.278 INFO:tasks.workunit.client.0.smithi102.stdout:3/468: chown de/d10/f1d 4 1 2024-05-13T22:48:04.281 INFO:tasks.workunit.client.0.smithi102.stdout:1/293: dread d2/f2b [0,4194304] 0 2024-05-13T22:48:04.283 INFO:tasks.workunit.client.0.smithi102.stdout:3/469: truncate de/d10/d50/d8f/f60 551236 0 2024-05-13T22:48:04.283 INFO:tasks.workunit.client.0.smithi102.stdout:3/470: stat de/d1c/d79 0 2024-05-13T22:48:04.284 INFO:tasks.workunit.client.0.smithi102.stdout:3/471: write de/d10/d50/d63/f85 [168587,17294] 0 2024-05-13T22:48:04.285 INFO:tasks.workunit.client.0.smithi102.stdout:1/294: unlink d2/f2b 0 2024-05-13T22:48:04.288 INFO:tasks.workunit.client.0.smithi102.stdout:3/472: rename de/d10/l4a to de/d10/d2a/lad 0 2024-05-13T22:48:04.296 INFO:tasks.workunit.client.0.smithi102.stdout:1/295: dwrite d2/d4/d26/f33 [0,4194304] 0 2024-05-13T22:48:04.299 INFO:tasks.workunit.client.0.smithi102.stdout:1/296: mkdir d2/d4/d26/d40/d5e 0 2024-05-13T22:48:04.308 INFO:tasks.workunit.client.0.smithi102.stdout:9/371: dwrite d1/d6/db/f1c [0,4194304] 0 2024-05-13T22:48:04.309 INFO:tasks.workunit.client.0.smithi102.stdout:9/372: chown d1/d6/d45/d3e/f63 28921183 1 2024-05-13T22:48:04.316 INFO:tasks.workunit.client.0.smithi102.stdout:9/373: symlink d1/d6/d21/d71/l89 0 2024-05-13T22:48:04.317 INFO:tasks.workunit.client.0.smithi102.stdout:9/374: write d1/d6/db/f7a [767499,51049] 0 2024-05-13T22:48:04.325 INFO:tasks.workunit.client.0.smithi102.stdout:9/375: dread d1/d6/d45/d3e/f63 [0,4194304] 0 2024-05-13T22:48:04.340 INFO:tasks.workunit.client.0.smithi102.stdout:9/376: dread d1/d6/f60 [0,4194304] 0 2024-05-13T22:48:04.342 INFO:tasks.workunit.client.0.smithi102.stdout:1/297: truncate d2/d4/d5/f15 11730577 0 2024-05-13T22:48:04.345 INFO:tasks.workunit.client.0.smithi102.stdout:9/377: write d1/d6/db/f1c [4496028,22789] 0 2024-05-13T22:48:04.346 INFO:tasks.workunit.client.0.smithi102.stdout:9/378: write d1/d6/db/d30/f36 [1873039,58308] 0 2024-05-13T22:48:04.352 INFO:tasks.workunit.client.0.smithi102.stdout:1/298: getdents d2/d4/d26/d56 0 2024-05-13T22:48:04.355 INFO:tasks.workunit.client.0.smithi102.stdout:1/299: symlink d2/d4/d26/l5f 0 2024-05-13T22:48:04.357 INFO:tasks.workunit.client.0.smithi102.stdout:1/300: mknod d2/d4/c60 0 2024-05-13T22:48:04.367 INFO:tasks.workunit.client.0.smithi102.stdout:1/301: rmdir d2/d4/d55 0 2024-05-13T22:48:04.373 INFO:tasks.workunit.client.0.smithi102.stdout:1/302: dwrite d2/d4/fd [0,4194304] 0 2024-05-13T22:48:04.374 INFO:tasks.workunit.client.0.smithi102.stdout:1/303: readlink d2/d4/d26/l5f 0 2024-05-13T22:48:04.383 INFO:tasks.workunit.client.0.smithi102.stdout:1/304: getdents d2/d4/d26 0 2024-05-13T22:48:04.387 INFO:tasks.workunit.client.0.smithi102.stdout:1/305: stat d2/d10/f21 0 2024-05-13T22:48:04.387 INFO:tasks.workunit.client.0.smithi102.stdout:9/379: dwrite d1/f7 [0,4194304] 0 2024-05-13T22:48:04.388 INFO:tasks.workunit.client.0.smithi102.stdout:9/380: dread - d1/d5d/d6c/f86 zero size 2024-05-13T22:48:04.389 INFO:tasks.workunit.client.0.smithi102.stdout:9/381: readlink d1/d6/d21/d71/l89 0 2024-05-13T22:48:04.390 INFO:tasks.workunit.client.0.smithi102.stdout:1/306: unlink d2/d10/f21 0 2024-05-13T22:48:04.396 INFO:tasks.workunit.client.0.smithi102.stdout:9/382: mkdir d1/d6/db/d30/d8a 0 2024-05-13T22:48:04.397 INFO:tasks.workunit.client.0.smithi102.stdout:9/383: write d1/f2b [2618442,55265] 0 2024-05-13T22:48:04.408 INFO:tasks.workunit.client.0.smithi102.stdout:9/384: dwrite d1/d6/d3b/d5b/f62 [0,4194304] 0 2024-05-13T22:48:04.409 INFO:tasks.workunit.client.0.smithi102.stdout:9/385: write d1/f17 [328419,86212] 0 2024-05-13T22:48:04.524 INFO:tasks.workunit.client.0.smithi102.stdout:9/386: mknod d1/d6/db/d30/c8b 0 2024-05-13T22:48:04.531 INFO:tasks.workunit.client.0.smithi102.stdout:9/387: dwrite d1/d6/db/d4a/d64/f87 [0,4194304] 0 2024-05-13T22:48:04.536 INFO:tasks.workunit.client.0.smithi102.stdout:9/388: rename d1/d6/d45/f5e to d1/d6/d21/d71/f8c 0 2024-05-13T22:48:04.539 INFO:tasks.workunit.client.0.smithi102.stdout:9/389: mknod d1/d6/d3b/d5b/c8d 0 2024-05-13T22:48:04.540 INFO:tasks.workunit.client.0.smithi102.stdout:9/390: truncate d1/d5d/f74 680626 0 2024-05-13T22:48:04.543 INFO:tasks.workunit.client.0.smithi102.stdout:9/391: mknod d1/c8e 0 2024-05-13T22:48:04.544 INFO:tasks.workunit.client.0.smithi102.stdout:9/392: write d1/d6/d45/d3e/f56 [1019644,121103] 0 2024-05-13T22:48:04.558 INFO:tasks.workunit.client.0.smithi102.stdout:9/393: dwrite d1/d6/d45/f67 [0,4194304] 0 2024-05-13T22:48:04.565 INFO:tasks.workunit.client.0.smithi102.stdout:9/394: write d1/d6/f1d [622333,80905] 0 2024-05-13T22:48:04.568 INFO:tasks.workunit.client.0.smithi102.stdout:9/395: rename d1/d6/db/d4a/d64/f87 to d1/d6/d45/d2d/f8f 0 2024-05-13T22:48:04.570 INFO:tasks.workunit.client.0.smithi102.stdout:9/396: creat d1/d6/db/d30/d47/f90 x:0 0 0 2024-05-13T22:48:04.573 INFO:tasks.workunit.client.0.smithi102.stdout:9/397: mkdir d1/d6/d21/d71/d91 0 2024-05-13T22:48:04.594 INFO:tasks.workunit.client.0.smithi102.stdout:2/342: sync 2024-05-13T22:48:04.595 INFO:tasks.workunit.client.0.smithi102.stdout:2/343: truncate d7/d9/d11/f38 148100 0 2024-05-13T22:48:04.596 INFO:tasks.workunit.client.0.smithi102.stdout:2/344: chown d7/d15/d67/f6e 119 1 2024-05-13T22:48:04.598 INFO:tasks.workunit.client.0.smithi102.stdout:2/345: symlink d7/d9/l71 0 2024-05-13T22:48:04.601 INFO:tasks.workunit.client.0.smithi102.stdout:2/346: symlink d7/d9/d10/d30/d53/l72 0 2024-05-13T22:48:04.603 INFO:tasks.workunit.client.0.smithi102.stdout:2/347: creat d7/d9/d14/f73 x:0 0 0 2024-05-13T22:48:04.616 INFO:tasks.workunit.client.0.smithi102.stdout:2/348: truncate d7/d9/d11/f39 772188 0 2024-05-13T22:48:04.624 INFO:tasks.workunit.client.0.smithi102.stdout:2/349: truncate d7/d15/d1e/f5c 3919280 0 2024-05-13T22:48:04.627 INFO:tasks.workunit.client.0.smithi102.stdout:2/350: mknod d7/d9/d10/d30/c74 0 2024-05-13T22:48:04.629 INFO:tasks.workunit.client.0.smithi102.stdout:2/351: creat d7/f75 x:0 0 0 2024-05-13T22:48:04.636 INFO:tasks.workunit.client.0.smithi102.stdout:2/352: dwrite d7/d9/d11/f38 [0,4194304] 0 2024-05-13T22:48:04.656 INFO:tasks.workunit.client.0.smithi102.stdout:2/353: dread d7/d9/f13 [0,4194304] 0 2024-05-13T22:48:04.658 INFO:tasks.workunit.client.0.smithi102.stdout:2/354: write d7/d9/d11/f55 [3613135,121042] 0 2024-05-13T22:48:04.685 INFO:tasks.workunit.client.0.smithi102.stdout:2/355: dread d7/d9/d10/d25/f68 [0,4194304] 0 2024-05-13T22:48:04.687 INFO:tasks.workunit.client.0.smithi102.stdout:2/356: mknod d7/d15/d67/d6a/c76 0 2024-05-13T22:48:04.688 INFO:tasks.workunit.client.0.smithi102.stdout:2/357: creat d7/d9/d14/d31/f77 x:0 0 0 2024-05-13T22:48:04.690 INFO:tasks.workunit.client.0.smithi102.stdout:2/358: mknod d7/d9/d10/d1b/c78 0 2024-05-13T22:48:04.691 INFO:tasks.workunit.client.0.smithi102.stdout:2/359: mknod d7/d9/d10/d25/c79 0 2024-05-13T22:48:04.697 INFO:tasks.workunit.client.0.smithi102.stdout:2/360: dwrite d7/ff [4194304,4194304] 0 2024-05-13T22:48:04.699 INFO:tasks.workunit.client.0.smithi102.stdout:2/361: mknod d7/d9/d14/c7a 0 2024-05-13T22:48:04.701 INFO:tasks.workunit.client.0.smithi102.stdout:2/362: truncate d7/d9/d14/d31/f64 30592 0 2024-05-13T22:48:04.702 INFO:tasks.workunit.client.0.smithi102.stdout:2/363: chown d7/d9/f13 32594 1 2024-05-13T22:48:04.704 INFO:tasks.workunit.client.0.smithi102.stdout:2/364: mknod d7/d9/d10/d25/d56/c7b 0 2024-05-13T22:48:04.720 INFO:tasks.workunit.client.0.smithi102.stdout:2/365: truncate d7/d9/d11/f38 2130033 0 2024-05-13T22:48:04.722 INFO:tasks.workunit.client.0.smithi102.stdout:2/366: write d7/d15/d67/f6e [608506,61888] 0 2024-05-13T22:48:04.724 INFO:tasks.workunit.client.0.smithi102.stdout:2/367: mkdir d7/d9/d7c 0 2024-05-13T22:48:04.730 INFO:tasks.workunit.client.0.smithi102.stdout:2/368: dwrite d7/d9/d11/f55 [12582912,4194304] 0 2024-05-13T22:48:04.752 INFO:tasks.workunit.client.0.smithi102.stdout:8/285: sync 2024-05-13T22:48:04.792 INFO:tasks.workunit.client.0.smithi102.stdout:8/286: creat d2/d7/f53 x:0 0 0 2024-05-13T22:48:04.795 INFO:tasks.workunit.client.0.smithi102.stdout:2/369: rename d7/d9/d10/d1b/c78 to d7/d15/c7d 0 2024-05-13T22:48:04.797 INFO:tasks.workunit.client.0.smithi102.stdout:2/370: write d7/d15/f52 [1275281,3249] 0 2024-05-13T22:48:04.799 INFO:tasks.workunit.client.0.smithi102.stdout:2/371: mknod d7/c7e 0 2024-05-13T22:48:04.802 INFO:tasks.workunit.client.0.smithi102.stdout:2/372: mkdir d7/d9/d10/d30/d7f 0 2024-05-13T22:48:04.802 INFO:tasks.workunit.client.0.smithi102.stdout:2/373: chown d7/d9/d10/d30/d53 103 1 2024-05-13T22:48:04.859 INFO:tasks.workunit.client.0.smithi102.stdout:2/374: readlink d7/d9/d10/d25/l2a 0 2024-05-13T22:48:04.859 INFO:tasks.workunit.client.0.smithi102.stdout:2/375: chown c3 1 1 2024-05-13T22:48:04.859 INFO:tasks.workunit.client.0.smithi102.stdout:8/287: dread d2/d17/f26 [0,4194304] 0 2024-05-13T22:48:04.859 INFO:tasks.workunit.client.0.smithi102.stdout:2/376: chown d7/d9/d10/d30/c74 7532 1 2024-05-13T22:48:04.865 INFO:tasks.workunit.client.0.smithi102.stdout:8/288: rename d2/d7/d12/l3c to d2/d7/d27/l54 0 2024-05-13T22:48:04.866 INFO:tasks.workunit.client.0.smithi102.stdout:2/377: dwrite d7/d9/d11/f55 [0,4194304] 0 2024-05-13T22:48:04.868 INFO:tasks.workunit.client.0.smithi102.stdout:8/289: symlink d2/d7/d27/l55 0 2024-05-13T22:48:04.870 INFO:tasks.workunit.client.0.smithi102.stdout:2/378: read d7/d9/d10/d25/f35 [56987,81095] 0 2024-05-13T22:48:04.871 INFO:tasks.workunit.client.0.smithi102.stdout:2/379: chown d7/d15/f52 6 1 2024-05-13T22:48:04.879 INFO:tasks.workunit.client.0.smithi102.stdout:8/290: creat d2/f56 x:0 0 0 2024-05-13T22:48:04.881 INFO:tasks.workunit.client.0.smithi102.stdout:8/291: mknod d2/d36/d49/c57 0 2024-05-13T22:48:04.883 INFO:tasks.workunit.client.0.smithi102.stdout:8/292: write d2/d17/f45 [2765343,107604] 0 2024-05-13T22:48:04.886 INFO:tasks.workunit.client.0.smithi102.stdout:8/293: creat d2/d7/d12/f58 x:0 0 0 2024-05-13T22:48:04.887 INFO:tasks.workunit.client.0.smithi102.stdout:8/294: fdatasync d2/d7/d25/d41/d50/f34 0 2024-05-13T22:48:04.888 INFO:tasks.workunit.client.0.smithi102.stdout:8/295: stat d2/d7/d12/f58 0 2024-05-13T22:48:04.889 INFO:tasks.workunit.client.0.smithi102.stdout:8/296: write d2/d7/d12/f1c [1536146,88797] 0 2024-05-13T22:48:04.896 INFO:tasks.workunit.client.0.smithi102.stdout:8/297: getdents d2/d7/d25/d41/d50 0 2024-05-13T22:48:04.897 INFO:tasks.workunit.client.0.smithi102.stdout:8/298: chown d2/d7/d25/d41 98518 1 2024-05-13T22:48:04.898 INFO:tasks.workunit.client.0.smithi102.stdout:8/299: write d2/d17/f45 [3942909,53704] 0 2024-05-13T22:48:04.898 INFO:tasks.workunit.client.0.smithi102.stdout:8/300: chown d2/d17/f2c 781544 1 2024-05-13T22:48:04.899 INFO:tasks.workunit.client.0.smithi102.stdout:8/301: fdatasync d2/d7/d12/f4f 0 2024-05-13T22:48:04.919 INFO:tasks.workunit.client.0.smithi102.stdout:6/356: sync 2024-05-13T22:48:04.919 INFO:tasks.workunit.client.0.smithi102.stdout:7/211: sync 2024-05-13T22:48:04.920 INFO:tasks.workunit.client.0.smithi102.stdout:7/212: chown d6/d15/d21 776 1 2024-05-13T22:48:04.923 INFO:tasks.workunit.client.0.smithi102.stdout:2/380: dwrite d7/f32 [0,4194304] 0 2024-05-13T22:48:04.925 INFO:tasks.workunit.client.0.smithi102.stdout:6/357: symlink d9/d25/d71/l73 0 2024-05-13T22:48:04.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:04 smithi102 ceph-mon[28599]: pgmap v19: 65 pgs: 65 active+clean; 843 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 MiB/s rd, 109 MiB/s wr, 403 op/s 2024-05-13T22:48:04.928 INFO:tasks.workunit.client.0.smithi102.stdout:7/213: creat d6/d13/d2c/f3a x:0 0 0 2024-05-13T22:48:04.929 INFO:tasks.workunit.client.0.smithi102.stdout:2/381: dread d7/d9/d10/d25/d27/f3b [0,4194304] 0 2024-05-13T22:48:04.932 INFO:tasks.workunit.client.0.smithi102.stdout:7/214: creat d6/d15/f3b x:0 0 0 2024-05-13T22:48:04.934 INFO:tasks.workunit.client.0.smithi102.stdout:7/215: symlink d6/d13/d2c/l3c 0 2024-05-13T22:48:04.937 INFO:tasks.workunit.client.0.smithi102.stdout:7/216: dread d6/d22/f23 [0,4194304] 0 2024-05-13T22:48:04.937 INFO:tasks.workunit.client.0.smithi102.stdout:8/302: truncate d2/d7/d25/d41/d50/f39 459707 0 2024-05-13T22:48:04.938 INFO:tasks.workunit.client.0.smithi102.stdout:7/217: truncate d6/d13/d2c/f3a 1024382 0 2024-05-13T22:48:04.940 INFO:tasks.workunit.client.0.smithi102.stdout:8/303: symlink d2/d7/d25/d41/d50/l59 0 2024-05-13T22:48:04.947 INFO:tasks.workunit.client.0.smithi102.stdout:7/218: dwrite d6/f2a [0,4194304] 0 2024-05-13T22:48:04.949 INFO:tasks.workunit.client.0.smithi102.stdout:2/382: dread d7/d9/d11/f38 [0,4194304] 0 2024-05-13T22:48:04.950 INFO:tasks.workunit.client.0.smithi102.stdout:7/219: mknod d6/d22/c3d 0 2024-05-13T22:48:04.951 INFO:tasks.workunit.client.0.smithi102.stdout:7/220: fsync d6/d15/f1e 0 2024-05-13T22:48:04.953 INFO:tasks.workunit.client.0.smithi102.stdout:6/358: write d9/db/f50 [510180,40889] 0 2024-05-13T22:48:04.955 INFO:tasks.workunit.client.0.smithi102.stdout:2/383: creat d7/d9/d10/d30/f80 x:0 0 0 2024-05-13T22:48:04.959 INFO:tasks.workunit.client.0.smithi102.stdout:8/304: dread d2/d7/d27/f46 [0,4194304] 0 2024-05-13T22:48:04.959 INFO:tasks.workunit.client.0.smithi102.stdout:6/359: mknod d9/d25/d60/d6a/c74 0 2024-05-13T22:48:04.961 INFO:tasks.workunit.client.0.smithi102.stdout:7/221: dwrite d6/d15/f3b [0,4194304] 0 2024-05-13T22:48:04.964 INFO:tasks.workunit.client.0.smithi102.stdout:2/384: mkdir d7/d9/d10/d25/d56/d59/d81 0 2024-05-13T22:48:04.970 INFO:tasks.workunit.client.0.smithi102.stdout:8/305: dwrite d2/d7/d27/f31 [4194304,4194304] 0 2024-05-13T22:48:04.973 INFO:tasks.workunit.client.0.smithi102.stdout:7/222: fsync d6/d15/f1b 0 2024-05-13T22:48:04.975 INFO:tasks.workunit.client.0.smithi102.stdout:7/223: chown d6/d13/d2c/f3a 21782033 1 2024-05-13T22:48:04.975 INFO:tasks.workunit.client.0.smithi102.stdout:2/385: dwrite d7/d15/d67/f70 [0,4194304] 0 2024-05-13T22:48:04.979 INFO:tasks.workunit.client.0.smithi102.stdout:7/224: dread d6/d13/de/fb [0,4194304] 0 2024-05-13T22:48:04.980 INFO:tasks.workunit.client.0.smithi102.stdout:7/225: chown d6/d15/d21/d20/l27 0 1 2024-05-13T22:48:04.983 INFO:tasks.workunit.client.0.smithi102.stdout:8/306: creat d2/d36/f5a x:0 0 0 2024-05-13T22:48:04.986 INFO:tasks.workunit.client.0.smithi102.stdout:6/360: rmdir d9/db/d14/d54/d69 0 2024-05-13T22:48:04.988 INFO:tasks.workunit.client.0.smithi102.stdout:6/361: fsync d9/f53 0 2024-05-13T22:48:04.991 INFO:tasks.workunit.client.0.smithi102.stdout:8/307: creat d2/d7/d25/d41/d50/d40/f5b x:0 0 0 2024-05-13T22:48:04.992 INFO:tasks.workunit.client.0.smithi102.stdout:6/362: creat d9/d45/f75 x:0 0 0 2024-05-13T22:48:04.994 INFO:tasks.workunit.client.0.smithi102.stdout:8/308: mknod d2/d7/d27/c5c 0 2024-05-13T22:48:04.997 INFO:tasks.workunit.client.0.smithi102.stdout:6/363: symlink d9/d45/d56/l76 0 2024-05-13T22:48:04.998 INFO:tasks.workunit.client.0.smithi102.stdout:6/364: truncate d9/db/d14/d37/f65 190646 0 2024-05-13T22:48:05.000 INFO:tasks.workunit.client.0.smithi102.stdout:8/309: rename d2/d17 to d2/d36/d49/d5d 0 2024-05-13T22:48:05.000 INFO:tasks.workunit.client.0.smithi102.stdout:8/310: fsync d2/d36/f42 0 2024-05-13T22:48:05.001 INFO:tasks.workunit.client.0.smithi102.stdout:7/226: dread d6/f7 [0,4194304] 0 2024-05-13T22:48:05.008 INFO:tasks.workunit.client.0.smithi102.stdout:7/227: link d6/d15/f37 d6/d15/f3e 0 2024-05-13T22:48:05.010 INFO:tasks.workunit.client.0.smithi102.stdout:6/365: creat d9/db/f77 x:0 0 0 2024-05-13T22:48:05.010 INFO:tasks.workunit.client.0.smithi102.stdout:6/366: stat d9/d45/l31 0 2024-05-13T22:48:05.012 INFO:tasks.workunit.client.0.smithi102.stdout:7/228: rmdir d6/d15/d21/d20 39 2024-05-13T22:48:05.013 INFO:tasks.workunit.client.0.smithi102.stdout:7/229: chown d6/l25 3624599 1 2024-05-13T22:48:05.013 INFO:tasks.workunit.client.0.smithi102.stdout:7/230: stat d6/d22 0 2024-05-13T22:48:05.013 INFO:tasks.workunit.client.0.smithi102.stdout:6/367: creat d9/db/d14/d37/f78 x:0 0 0 2024-05-13T22:48:05.014 INFO:tasks.workunit.client.0.smithi102.stdout:7/231: dread - d6/d13/de/f29 zero size 2024-05-13T22:48:05.014 INFO:tasks.workunit.client.0.smithi102.stdout:7/232: stat d6/d13/de 0 2024-05-13T22:48:05.015 INFO:tasks.workunit.client.0.smithi102.stdout:7/233: chown d6/d15/l38 5 1 2024-05-13T22:48:05.016 INFO:tasks.workunit.client.0.smithi102.stdout:8/311: link d2/d36/c4d d2/c5e 0 2024-05-13T22:48:05.022 INFO:tasks.workunit.client.0.smithi102.stdout:6/368: dwrite d9/db/d14/f3f [4194304,4194304] 0 2024-05-13T22:48:05.023 INFO:tasks.workunit.client.0.smithi102.stdout:8/312: dwrite d2/d7/d12/f1c [0,4194304] 0 2024-05-13T22:48:05.029 INFO:tasks.workunit.client.0.smithi102.stdout:8/313: rename d2/d7/d27/l55 to d2/d36/d49/l5f 0 2024-05-13T22:48:05.030 INFO:tasks.workunit.client.0.smithi102.stdout:7/234: write d6/d22/f23 [610385,12298] 0 2024-05-13T22:48:05.031 INFO:tasks.workunit.client.0.smithi102.stdout:7/235: stat d6/f7 0 2024-05-13T22:48:05.031 INFO:tasks.workunit.client.0.smithi102.stdout:7/236: readlink l2 0 2024-05-13T22:48:05.035 INFO:tasks.workunit.client.0.smithi102.stdout:6/369: link d9/db/d14/d35/f52 d9/d25/d60/d6a/f79 0 2024-05-13T22:48:05.038 INFO:tasks.workunit.client.0.smithi102.stdout:7/237: link d6/f2a d6/d13/d2c/d39/f3f 0 2024-05-13T22:48:05.039 INFO:tasks.workunit.client.0.smithi102.stdout:6/370: creat d9/d25/d71/f7a x:0 0 0 2024-05-13T22:48:05.040 INFO:tasks.workunit.client.0.smithi102.stdout:8/314: getdents d2/d7 0 2024-05-13T22:48:05.042 INFO:tasks.workunit.client.0.smithi102.stdout:6/371: creat d9/db/d14/d37/f7b x:0 0 0 2024-05-13T22:48:05.044 INFO:tasks.workunit.client.0.smithi102.stdout:7/238: rename d6/d13/de/la to d6/d15/d21/d20/l40 0 2024-05-13T22:48:05.049 INFO:tasks.workunit.client.0.smithi102.stdout:8/315: getdents d2/d7/d25/d41/d50 0 2024-05-13T22:48:05.055 INFO:tasks.workunit.client.0.smithi102.stdout:8/316: chown d2/d36/c4d 1532830 1 2024-05-13T22:48:05.057 INFO:tasks.workunit.client.0.smithi102.stdout:6/372: getdents d9/db 0 2024-05-13T22:48:05.057 INFO:tasks.workunit.client.0.smithi102.stdout:6/373: readlink d9/d45/d56/l76 0 2024-05-13T22:48:05.059 INFO:tasks.workunit.client.0.smithi102.stdout:8/317: creat d2/d36/d49/f60 x:0 0 0 2024-05-13T22:48:05.061 INFO:tasks.workunit.client.0.smithi102.stdout:6/374: mknod d9/d25/d71/c7c 0 2024-05-13T22:48:05.065 INFO:tasks.workunit.client.0.smithi102.stdout:6/375: unlink d9/db/d14/d35/c46 0 2024-05-13T22:48:05.067 INFO:tasks.workunit.client.0.smithi102.stdout:8/318: rmdir d2/d36 39 2024-05-13T22:48:05.069 INFO:tasks.workunit.client.0.smithi102.stdout:7/239: dwrite f3 [0,4194304] 0 2024-05-13T22:48:05.071 INFO:tasks.workunit.client.0.smithi102.stdout:8/319: symlink d2/d7/d25/d41/l61 0 2024-05-13T22:48:05.073 INFO:tasks.workunit.client.0.smithi102.stdout:7/240: symlink d6/d13/de/l41 0 2024-05-13T22:48:05.075 INFO:tasks.workunit.client.0.smithi102.stdout:7/241: write d6/d13/de/f29 [978694,9244] 0 2024-05-13T22:48:05.080 INFO:tasks.workunit.client.0.smithi102.stdout:8/320: creat d2/d36/f62 x:0 0 0 2024-05-13T22:48:05.081 INFO:tasks.workunit.client.0.smithi102.stdout:6/376: dread d9/d25/f58 [0,4194304] 0 2024-05-13T22:48:05.083 INFO:tasks.workunit.client.0.smithi102.stdout:8/321: mknod d2/d36/c63 0 2024-05-13T22:48:05.085 INFO:tasks.workunit.client.0.smithi102.stdout:6/377: rename d9/f53 to d9/db/f7d 0 2024-05-13T22:48:05.088 INFO:tasks.workunit.client.0.smithi102.stdout:6/378: creat d9/d25/d60/f7e x:0 0 0 2024-05-13T22:48:05.090 INFO:tasks.workunit.client.0.smithi102.stdout:7/242: dwrite d6/f2a [4194304,4194304] 0 2024-05-13T22:48:05.092 INFO:tasks.workunit.client.0.smithi102.stdout:6/379: symlink d9/d25/d60/d6a/l7f 0 2024-05-13T22:48:05.095 INFO:tasks.workunit.client.0.smithi102.stdout:7/243: symlink d6/d13/d2c/d32/l42 0 2024-05-13T22:48:05.099 INFO:tasks.workunit.client.0.smithi102.stdout:6/380: dwrite d9/f21 [0,4194304] 0 2024-05-13T22:48:05.105 INFO:tasks.workunit.client.0.smithi102.stdout:6/381: creat d9/db/d2f/d59/f80 x:0 0 0 2024-05-13T22:48:05.109 INFO:tasks.workunit.client.0.smithi102.stdout:6/382: fdatasync d9/d25/d60/d6a/f79 0 2024-05-13T22:48:05.110 INFO:tasks.workunit.client.0.smithi102.stdout:6/383: chown f7 1102900553 1 2024-05-13T22:48:05.117 INFO:tasks.workunit.client.0.smithi102.stdout:6/384: rename d9/db/c26 to d9/db/d14/d37/c81 0 2024-05-13T22:48:05.122 INFO:tasks.workunit.client.0.smithi102.stdout:6/385: dread d9/f21 [0,4194304] 0 2024-05-13T22:48:05.128 INFO:tasks.workunit.client.0.smithi102.stdout:6/386: unlink d9/db/l6b 0 2024-05-13T22:48:05.128 INFO:tasks.workunit.client.0.smithi102.stdout:5/544: sync 2024-05-13T22:48:05.128 INFO:tasks.workunit.client.0.smithi102.stdout:0/441: sync 2024-05-13T22:48:05.129 INFO:tasks.workunit.client.0.smithi102.stdout:6/387: write d9/db/d14/d35/f67 [162354,104020] 0 2024-05-13T22:48:05.132 INFO:tasks.workunit.client.0.smithi102.stdout:5/545: mkdir de/d25/d43/d7a/db1/dc5/dca 0 2024-05-13T22:48:05.132 INFO:tasks.workunit.client.0.smithi102.stdout:0/442: truncate d1/f60 579501 0 2024-05-13T22:48:05.133 INFO:tasks.workunit.client.0.smithi102.stdout:7/244: dread d6/d13/f16 [0,4194304] 0 2024-05-13T22:48:05.139 INFO:tasks.workunit.client.0.smithi102.stdout:7/245: write f4 [8430157,39522] 0 2024-05-13T22:48:05.143 INFO:tasks.workunit.client.0.smithi102.stdout:6/388: dwrite d9/db/d14/d37/f43 [0,4194304] 0 2024-05-13T22:48:05.143 INFO:tasks.workunit.client.0.smithi102.stdout:0/443: unlink d1/de/f5a 0 2024-05-13T22:48:05.148 INFO:tasks.workunit.client.0.smithi102.stdout:7/246: rename d6/d13/f14 to d6/d22/f43 0 2024-05-13T22:48:05.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:04 smithi130 ceph-mon[36591]: pgmap v19: 65 pgs: 65 active+clean; 843 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 MiB/s rd, 109 MiB/s wr, 403 op/s 2024-05-13T22:48:05.151 INFO:tasks.workunit.client.0.smithi102.stdout:6/389: write d9/db/f50 [1558287,40979] 0 2024-05-13T22:48:05.154 INFO:tasks.workunit.client.0.smithi102.stdout:5/546: mknod de/d25/d43/d7a/db1/dc5/dca/ccb 0 2024-05-13T22:48:05.158 INFO:tasks.workunit.client.0.smithi102.stdout:7/247: symlink d6/d13/d2c/d32/l44 0 2024-05-13T22:48:05.163 INFO:tasks.workunit.client.0.smithi102.stdout:6/390: unlink d9/db/d2f/f41 0 2024-05-13T22:48:05.163 INFO:tasks.workunit.client.0.smithi102.stdout:7/248: dread d6/d15/d21/f26 [0,4194304] 0 2024-05-13T22:48:05.163 INFO:tasks.workunit.client.0.smithi102.stdout:6/391: stat d9/db/d14/d37 0 2024-05-13T22:48:05.164 INFO:tasks.workunit.client.0.smithi102.stdout:5/547: mknod de/d25/d59/d28/ccc 0 2024-05-13T22:48:05.165 INFO:tasks.workunit.client.0.smithi102.stdout:0/444: dread d1/de/d32/f36 [0,4194304] 0 2024-05-13T22:48:05.165 INFO:tasks.workunit.client.0.smithi102.stdout:7/249: read d6/d15/f1b [2206037,42878] 0 2024-05-13T22:48:05.166 INFO:tasks.workunit.client.0.smithi102.stdout:5/548: dread - de/d25/d59/f8f zero size 2024-05-13T22:48:05.167 INFO:tasks.workunit.client.0.smithi102.stdout:5/549: read de/d33/d53/d9e/fa7 [36710,95479] 0 2024-05-13T22:48:05.168 INFO:tasks.workunit.client.0.smithi102.stdout:5/550: chown de/d33/d34 1661876908 1 2024-05-13T22:48:05.168 INFO:tasks.workunit.client.0.smithi102.stdout:5/551: fsync de/d25/d59/d7f/fc9 0 2024-05-13T22:48:05.173 INFO:tasks.workunit.client.0.smithi102.stdout:0/445: symlink d1/de/d32/d24/l8f 0 2024-05-13T22:48:05.178 INFO:tasks.workunit.client.0.smithi102.stdout:5/552: unlink de/d25/d2a/f56 0 2024-05-13T22:48:05.180 INFO:tasks.workunit.client.0.smithi102.stdout:0/446: dwrite d1/de/d32/d24/d2d/f6b [0,4194304] 0 2024-05-13T22:48:05.180 INFO:tasks.workunit.client.0.smithi102.stdout:7/250: link d6/d13/de/l41 d6/d15/d21/d20/l45 0 2024-05-13T22:48:05.181 INFO:tasks.workunit.client.0.smithi102.stdout:7/251: fsync d6/f2a 0 2024-05-13T22:48:05.182 INFO:tasks.workunit.client.0.smithi102.stdout:7/252: chown d6/d15/d21/f34 892695 1 2024-05-13T22:48:05.184 INFO:tasks.workunit.client.0.smithi102.stdout:5/553: creat de/d33/d53/d9e/fcd x:0 0 0 2024-05-13T22:48:05.189 INFO:tasks.workunit.client.0.smithi102.stdout:5/554: creat de/d25/d43/d7a/db1/dc5/dca/fce x:0 0 0 2024-05-13T22:48:05.193 INFO:tasks.workunit.client.0.smithi102.stdout:5/555: mkdir de/d25/d2c/dcf 0 2024-05-13T22:48:05.193 INFO:tasks.workunit.client.0.smithi102.stdout:5/556: fsync de/d75/fbd 0 2024-05-13T22:48:05.200 INFO:tasks.workunit.client.0.smithi102.stdout:5/557: mknod de/d25/d2a/d4b/d4d/cd0 0 2024-05-13T22:48:05.201 INFO:tasks.workunit.client.0.smithi102.stdout:0/447: dread d1/f5 [0,4194304] 0 2024-05-13T22:48:05.204 INFO:tasks.workunit.client.0.smithi102.stdout:5/558: mkdir de/d25/d43/dd1 0 2024-05-13T22:48:05.207 INFO:tasks.workunit.client.0.smithi102.stdout:0/448: dwrite d1/de/d32/f17 [4194304,4194304] 0 2024-05-13T22:48:05.225 INFO:tasks.workunit.client.0.smithi102.stdout:0/449: fsync d1/de/d32/d24/d2d/d3e/f41 0 2024-05-13T22:48:05.229 INFO:tasks.workunit.client.0.smithi102.stdout:7/253: dwrite d6/d15/f37 [0,4194304] 0 2024-05-13T22:48:05.232 INFO:tasks.workunit.client.0.smithi102.stdout:0/450: dwrite d1/de/d32/f17 [4194304,4194304] 0 2024-05-13T22:48:05.234 INFO:tasks.workunit.client.0.smithi102.stdout:7/254: symlink d6/d13/de/l46 0 2024-05-13T22:48:05.239 INFO:tasks.workunit.client.0.smithi102.stdout:0/451: mknod d1/d77/c90 0 2024-05-13T22:48:05.242 INFO:tasks.workunit.client.0.smithi102.stdout:5/559: write de/d75/fa4 [15817,127724] 0 2024-05-13T22:48:05.243 INFO:tasks.workunit.client.0.smithi102.stdout:0/452: rmdir d1/de/d32 39 2024-05-13T22:48:05.281 INFO:tasks.workunit.client.0.smithi102.stdout:0/453: dwrite d1/de/d32/d24/f40 [0,4194304] 0 2024-05-13T22:48:05.286 INFO:tasks.workunit.client.0.smithi102.stdout:0/454: mkdir d1/de/d32/d24/d2d/d91 0 2024-05-13T22:48:05.294 INFO:tasks.workunit.client.0.smithi102.stdout:0/455: dwrite d1/f5 [4194304,4194304] 0 2024-05-13T22:48:05.299 INFO:tasks.workunit.client.0.smithi102.stdout:0/456: chown d1/de/d32/c46 41567 1 2024-05-13T22:48:05.300 INFO:tasks.workunit.client.0.smithi102.stdout:0/457: dread - d1/de/f50 zero size 2024-05-13T22:48:05.316 INFO:tasks.workunit.client.0.smithi102.stdout:0/458: dread d1/de/d32/d24/d2d/d38/d3d/f57 [0,4194304] 0 2024-05-13T22:48:05.317 INFO:tasks.workunit.client.0.smithi102.stdout:0/459: stat d1/de/d32/d24/d2d/d3e/c66 0 2024-05-13T22:48:05.317 INFO:tasks.workunit.client.0.smithi102.stdout:0/460: dread - d1/de/d32/d24/d2d/f64 zero size 2024-05-13T22:48:05.318 INFO:tasks.workunit.client.0.smithi102.stdout:0/461: write d1/de/d32/d24/d2d/d38/f69 [647515,82138] 0 2024-05-13T22:48:05.322 INFO:tasks.workunit.client.0.smithi102.stdout:0/462: chown d1/de/d32/f37 37777497 1 2024-05-13T22:48:05.329 INFO:tasks.workunit.client.0.smithi102.stdout:4/358: sync 2024-05-13T22:48:05.329 INFO:tasks.workunit.client.0.smithi102.stdout:4/359: fsync d3/d22/f78 0 2024-05-13T22:48:05.366 INFO:tasks.workunit.client.0.smithi102.stdout:4/360: creat d3/d19/d23/d3a/f83 x:0 0 0 2024-05-13T22:48:05.367 INFO:tasks.workunit.client.0.smithi102.stdout:4/361: dread - d3/d4/f7b zero size 2024-05-13T22:48:05.367 INFO:tasks.workunit.client.0.smithi102.stdout:4/362: chown d3/d19/d23/d3a/f51 110 1 2024-05-13T22:48:05.410 INFO:tasks.workunit.client.0.smithi102.stdout:4/363: rename d3/d19/d23/d39/c62 to d3/d4/d6/c84 0 2024-05-13T22:48:05.438 INFO:tasks.workunit.client.0.smithi102.stdout:4/364: getdents d3 0 2024-05-13T22:48:05.441 INFO:tasks.workunit.client.0.smithi102.stdout:4/365: mkdir d3/d19/d23/d4e/d68/d85 0 2024-05-13T22:48:05.478 INFO:tasks.workunit.client.0.smithi102.stdout:4/366: dwrite d3/d4/f47 [0,4194304] 0 2024-05-13T22:48:05.480 INFO:tasks.workunit.client.0.smithi102.stdout:4/367: mknod d3/d4/d6/de/c86 0 2024-05-13T22:48:05.484 INFO:tasks.workunit.client.0.smithi102.stdout:4/368: creat d3/d19/d23/d4e/d68/f87 x:0 0 0 2024-05-13T22:48:05.489 INFO:tasks.workunit.client.0.smithi102.stdout:4/369: creat d3/d19/d23/d39/d5d/f88 x:0 0 0 2024-05-13T22:48:05.492 INFO:tasks.workunit.client.0.smithi102.stdout:4/370: symlink d3/d4/d6/l89 0 2024-05-13T22:48:05.495 INFO:tasks.workunit.client.0.smithi102.stdout:4/371: unlink d3/d22/f71 0 2024-05-13T22:48:05.497 INFO:tasks.workunit.client.0.smithi102.stdout:4/372: rmdir d3/d4/d5f 39 2024-05-13T22:48:05.499 INFO:tasks.workunit.client.0.smithi102.stdout:4/373: unlink d3/d4/d6/de/c73 0 2024-05-13T22:48:05.502 INFO:tasks.workunit.client.0.smithi102.stdout:4/374: creat d3/d4/d6/de/d1f/d41/f8a x:0 0 0 2024-05-13T22:48:05.502 INFO:tasks.workunit.client.0.smithi102.stdout:4/375: readlink d3/d4/d6/l89 0 2024-05-13T22:48:05.505 INFO:tasks.workunit.client.0.smithi102.stdout:4/376: link d3/f35 d3/d19/d23/d39/f8b 0 2024-05-13T22:48:05.506 INFO:tasks.workunit.client.0.smithi102.stdout:4/377: readlink d3/l11 0 2024-05-13T22:48:05.510 INFO:tasks.workunit.client.0.smithi102.stdout:4/378: link d3/d4/d6/de/d1f/f28 d3/d4/d6/de/d1f/f8c 0 2024-05-13T22:48:05.541 INFO:tasks.workunit.client.0.smithi102.stdout:4/379: fsync d3/d4/d6/de/d1f/d41/f8a 0 2024-05-13T22:48:05.547 INFO:tasks.workunit.client.0.smithi102.stdout:4/380: dwrite d3/d4/d6/f42 [0,4194304] 0 2024-05-13T22:48:05.549 INFO:tasks.workunit.client.0.smithi102.stdout:4/381: dread - d3/d19/d23/d3a/f56 zero size 2024-05-13T22:48:05.549 INFO:tasks.workunit.client.0.smithi102.stdout:4/382: chown d3/d19/d23/d4e/d68 359124 1 2024-05-13T22:48:05.550 INFO:tasks.workunit.client.0.smithi102.stdout:4/383: readlink d3/d19/l40 0 2024-05-13T22:48:05.557 INFO:tasks.workunit.client.0.smithi102.stdout:4/384: dwrite d3/d19/d23/d4e/f76 [0,4194304] 0 2024-05-13T22:48:05.601 INFO:tasks.workunit.client.0.smithi102.stdout:4/385: dread d3/d19/d23/f37 [0,4194304] 0 2024-05-13T22:48:05.606 INFO:tasks.workunit.client.0.smithi102.stdout:4/386: dwrite d3/d4/f2e [0,4194304] 0 2024-05-13T22:48:05.610 INFO:tasks.workunit.client.0.smithi102.stdout:4/387: mkdir d3/d4/d6/de/d8d 0 2024-05-13T22:48:05.611 INFO:tasks.workunit.client.0.smithi102.stdout:4/388: chown d3/d22/f3e 9678858 1 2024-05-13T22:48:05.614 INFO:tasks.workunit.client.0.smithi102.stdout:4/389: rename d3/d19/l3c to d3/d4/d6/de/d1f/l8e 0 2024-05-13T22:48:05.620 INFO:tasks.workunit.client.0.smithi102.stdout:4/390: dwrite d3/d22/f74 [0,4194304] 0 2024-05-13T22:48:05.631 INFO:tasks.workunit.client.0.smithi102.stdout:4/391: readlink d3/d4/d6/de/d1f/l8e 0 2024-05-13T22:48:05.632 INFO:tasks.workunit.client.0.smithi102.stdout:4/392: dread - d3/d19/d23/d4e/f72 zero size 2024-05-13T22:48:05.633 INFO:tasks.workunit.client.0.smithi102.stdout:4/393: readlink d3/d4/d6/l25 0 2024-05-13T22:48:05.634 INFO:tasks.workunit.client.0.smithi102.stdout:4/394: write d3/d19/d23/d4e/f79 [683840,51626] 0 2024-05-13T22:48:05.636 INFO:tasks.workunit.client.0.smithi102.stdout:4/395: dread - d3/d4/d6/de/d1f/f53 zero size 2024-05-13T22:48:05.637 INFO:tasks.workunit.client.0.smithi102.stdout:4/396: write d3/d22/f74 [3555457,97465] 0 2024-05-13T22:48:05.638 INFO:tasks.workunit.client.0.smithi102.stdout:4/397: fdatasync d3/d19/d23/d4e/f76 0 2024-05-13T22:48:05.654 INFO:tasks.workunit.client.0.smithi102.stdout:4/398: dread d3/d4/d6/de/d1f/f28 [0,4194304] 0 2024-05-13T22:48:05.657 INFO:tasks.workunit.client.0.smithi102.stdout:4/399: getdents d3/d19/d23/d4e/d68/d85 0 2024-05-13T22:48:05.657 INFO:tasks.workunit.client.0.smithi102.stdout:4/400: stat d3/d19/l24 0 2024-05-13T22:48:05.658 INFO:tasks.workunit.client.0.smithi102.stdout:4/401: write d3/d22/f5b [2941963,1678] 0 2024-05-13T22:48:05.660 INFO:tasks.workunit.client.0.smithi102.stdout:4/402: creat d3/d6b/d81/d7d/f8f x:0 0 0 2024-05-13T22:48:05.665 INFO:tasks.workunit.client.0.smithi102.stdout:4/403: dwrite d3/d4/f1e [0,4194304] 0 2024-05-13T22:48:05.668 INFO:tasks.workunit.client.0.smithi102.stdout:4/404: symlink d3/l90 0 2024-05-13T22:48:05.684 INFO:tasks.workunit.client.0.smithi102.stdout:4/405: link d3/d4/d6/f80 d3/d4/d6/de/d8d/f91 0 2024-05-13T22:48:05.686 INFO:tasks.workunit.client.0.smithi102.stdout:4/406: read d3/d22/f74 [2431816,33509] 0 2024-05-13T22:48:05.688 INFO:tasks.workunit.client.0.smithi102.stdout:4/407: chown d3/d6b 20 1 2024-05-13T22:48:05.691 INFO:tasks.workunit.client.0.smithi102.stdout:4/408: truncate d3/d4/d6/de/d1f/f28 105813 0 2024-05-13T22:48:05.693 INFO:tasks.workunit.client.0.smithi102.stdout:4/409: truncate d3/d19/d23/d3a/f56 156625 0 2024-05-13T22:48:05.695 INFO:tasks.workunit.client.0.smithi102.stdout:4/410: creat d3/d4/d6/de/d8d/f92 x:0 0 0 2024-05-13T22:48:05.708 INFO:tasks.workunit.client.0.smithi102.stdout:4/411: dread d3/d4/d6/de/f3f [0,4194304] 0 2024-05-13T22:48:05.709 INFO:tasks.workunit.client.0.smithi102.stdout:4/412: write d3/d19/d23/d39/d5d/f88 [330231,73054] 0 2024-05-13T22:48:05.713 INFO:tasks.workunit.client.0.smithi102.stdout:4/413: creat d3/d4/f93 x:0 0 0 2024-05-13T22:48:05.714 INFO:tasks.workunit.client.0.smithi102.stdout:4/414: readlink d3/d4/d6/de/l52 0 2024-05-13T22:48:05.715 INFO:tasks.workunit.client.0.smithi102.stdout:4/415: creat d3/d19/d23/d4e/d68/f94 x:0 0 0 2024-05-13T22:48:05.716 INFO:tasks.workunit.client.0.smithi102.stdout:4/416: stat d3/d4/f5a 0 2024-05-13T22:48:05.717 INFO:tasks.workunit.client.0.smithi102.stdout:3/473: sync 2024-05-13T22:48:05.721 INFO:tasks.workunit.client.0.smithi102.stdout:4/417: dread d3/d4/f32 [0,4194304] 0 2024-05-13T22:48:05.723 INFO:tasks.workunit.client.0.smithi102.stdout:3/474: dwrite de/d1c/f52 [0,4194304] 0 2024-05-13T22:48:05.724 INFO:tasks.workunit.client.0.smithi102.stdout:3/475: dread - de/d68/f94 zero size 2024-05-13T22:48:05.730 INFO:tasks.workunit.client.0.smithi102.stdout:3/476: symlink de/d10/d2a/d33/lae 0 2024-05-13T22:48:05.732 INFO:tasks.workunit.client.0.smithi102.stdout:3/477: creat de/d68/faf x:0 0 0 2024-05-13T22:48:05.750 INFO:tasks.workunit.client.0.smithi102.stdout:3/478: truncate de/d10/d2a/f3a 1226201 0 2024-05-13T22:48:05.753 INFO:tasks.workunit.client.0.smithi102.stdout:3/479: creat de/d10/d50/d63/fb0 x:0 0 0 2024-05-13T22:48:05.756 INFO:tasks.workunit.client.0.smithi102.stdout:4/418: dwrite d3/d4/d6/de/d1f/f8c [0,4194304] 0 2024-05-13T22:48:05.759 INFO:tasks.workunit.client.0.smithi102.stdout:4/419: mkdir d3/d6b/d81/d7d/d95 0 2024-05-13T22:48:05.761 INFO:tasks.workunit.client.0.smithi102.stdout:3/480: dwrite de/d68/f82 [0,4194304] 0 2024-05-13T22:48:05.763 INFO:tasks.workunit.client.0.smithi102.stdout:4/420: creat d3/d19/d23/d39/d5d/f96 x:0 0 0 2024-05-13T22:48:05.768 INFO:tasks.workunit.client.0.smithi102.stdout:3/481: mknod de/d10/d2a/d8c/da8/cb1 0 2024-05-13T22:48:05.772 INFO:tasks.workunit.client.0.smithi102.stdout:4/421: dwrite d3/d4/d6/de/d1f/f28 [0,4194304] 0 2024-05-13T22:48:05.773 INFO:tasks.workunit.client.0.smithi102.stdout:3/482: mkdir de/d10/d2a/d36/db2 0 2024-05-13T22:48:05.775 INFO:tasks.workunit.client.0.smithi102.stdout:4/422: unlink d3/d4/f66 0 2024-05-13T22:48:05.778 INFO:tasks.workunit.client.0.smithi102.stdout:4/423: mknod d3/d6b/d81/d7d/d95/c97 0 2024-05-13T22:48:05.780 INFO:tasks.workunit.client.0.smithi102.stdout:4/424: read d3/d19/d23/d3a/f4d [223876,113782] 0 2024-05-13T22:48:05.780 INFO:tasks.workunit.client.0.smithi102.stdout:4/425: chown d3/d4/d6/de/f3f 3147 1 2024-05-13T22:48:05.791 INFO:tasks.workunit.client.0.smithi102.stdout:3/483: dread de/f37 [0,4194304] 0 2024-05-13T22:48:05.792 INFO:tasks.workunit.client.0.smithi102.stdout:3/484: dread - de/d10/d2a/f9c zero size 2024-05-13T22:48:05.796 INFO:tasks.workunit.client.0.smithi102.stdout:3/485: getdents de 0 2024-05-13T22:48:05.798 INFO:tasks.workunit.client.0.smithi102.stdout:3/486: unlink de/d10/d50/f45 0 2024-05-13T22:48:05.798 INFO:tasks.workunit.client.0.smithi102.stdout:3/487: fsync de/d10/d50/f66 0 2024-05-13T22:48:05.799 INFO:tasks.workunit.client.0.smithi102.stdout:3/488: dread - de/d10/d46/fac zero size 2024-05-13T22:48:05.857 INFO:tasks.workunit.client.0.smithi102.stdout:5/560: sync 2024-05-13T22:48:05.857 INFO:tasks.workunit.client.0.smithi102.stdout:7/255: sync 2024-05-13T22:48:05.857 INFO:tasks.workunit.client.0.smithi102.stdout:6/392: sync 2024-05-13T22:48:05.858 INFO:tasks.workunit.client.0.smithi102.stdout:1/307: sync 2024-05-13T22:48:05.858 INFO:tasks.workunit.client.0.smithi102.stdout:0/463: sync 2024-05-13T22:48:05.858 INFO:tasks.workunit.client.0.smithi102.stdout:9/398: sync 2024-05-13T22:48:05.858 INFO:tasks.workunit.client.0.smithi102.stdout:2/386: sync 2024-05-13T22:48:05.858 INFO:tasks.workunit.client.0.smithi102.stdout:8/322: sync 2024-05-13T22:48:05.858 INFO:tasks.workunit.client.0.smithi102.stdout:3/489: rename de/d10/f11 to de/d10/d2a/d33/daa/fb3 0 2024-05-13T22:48:05.858 INFO:tasks.workunit.client.0.smithi102.stdout:5/561: readlink de/d75/lb7 0 2024-05-13T22:48:05.861 INFO:tasks.workunit.client.0.smithi102.stdout:3/490: fdatasync de/d10/d46/f6c 0 2024-05-13T22:48:05.862 INFO:tasks.workunit.client.0.smithi102.stdout:5/562: write de/d25/d59/f8f [729146,38640] 0 2024-05-13T22:48:05.866 INFO:tasks.workunit.client.0.smithi102.stdout:9/399: symlink d1/d6/l92 0 2024-05-13T22:48:05.868 INFO:tasks.workunit.client.0.smithi102.stdout:2/387: dwrite d7/d15/d1e/f26 [0,4194304] 0 2024-05-13T22:48:05.873 INFO:tasks.workunit.client.0.smithi102.stdout:0/464: mknod d1/de/d32/d24/d2d/d38/d3d/c92 0 2024-05-13T22:48:05.873 INFO:tasks.workunit.client.0.smithi102.stdout:0/465: readlink d1/l6d 0 2024-05-13T22:48:05.875 INFO:tasks.workunit.client.0.smithi102.stdout:6/393: creat d9/d45/d56/d5d/f82 x:0 0 0 2024-05-13T22:48:05.877 INFO:tasks.workunit.client.0.smithi102.stdout:3/491: fsync de/d10/d2a/d33/f4d 0 2024-05-13T22:48:05.878 INFO:tasks.workunit.client.0.smithi102.stdout:5/563: symlink de/d33/d53/ld2 0 2024-05-13T22:48:05.882 INFO:tasks.workunit.client.0.smithi102.stdout:2/388: fdatasync d7/d9/d10/d25/f68 0 2024-05-13T22:48:05.883 INFO:tasks.workunit.client.0.smithi102.stdout:1/308: getdents d2/d4/d26/d40/d5e 0 2024-05-13T22:48:05.883 INFO:tasks.workunit.client.0.smithi102.stdout:6/394: truncate d9/d25/d60/d6a/f79 80943 0 2024-05-13T22:48:05.886 INFO:tasks.workunit.client.0.smithi102.stdout:5/564: dwrite de/d25/d59/d6f/f72 [0,4194304] 0 2024-05-13T22:48:05.887 INFO:tasks.workunit.client.0.smithi102.stdout:3/492: symlink de/d10/d2a/d36/d88/lb4 0 2024-05-13T22:48:05.887 INFO:tasks.workunit.client.0.smithi102.stdout:5/565: dread - de/fc8 zero size 2024-05-13T22:48:05.890 INFO:tasks.workunit.client.0.smithi102.stdout:2/389: chown d7/d15/c7d 20 1 2024-05-13T22:48:05.895 INFO:tasks.workunit.client.0.smithi102.stdout:0/466: link d1/de/d32/d24/d2d/f51 d1/de/d42/f93 0 2024-05-13T22:48:05.897 INFO:tasks.workunit.client.0.smithi102.stdout:6/395: write d9/f21 [2544582,55324] 0 2024-05-13T22:48:05.903 INFO:tasks.workunit.client.0.smithi102.stdout:2/390: fsync d7/d9/d10/f33 0 2024-05-13T22:48:05.907 INFO:tasks.workunit.client.0.smithi102.stdout:1/309: rename d2/f34 to d2/d4/d26/d56/f61 0 2024-05-13T22:48:05.911 INFO:tasks.workunit.client.0.smithi102.stdout:3/493: symlink de/d10/d2a/d36/db2/lb5 0 2024-05-13T22:48:05.914 INFO:tasks.workunit.client.0.smithi102.stdout:0/467: mknod d1/de/d32/d24/d2d/d38/c94 0 2024-05-13T22:48:05.915 INFO:tasks.workunit.client.0.smithi102.stdout:0/468: truncate d1/d4d/f6a 635493 0 2024-05-13T22:48:05.917 INFO:tasks.workunit.client.0.smithi102.stdout:1/310: mkdir d2/d10/d62 0 2024-05-13T22:48:05.925 INFO:tasks.workunit.client.0.smithi102.stdout:3/494: dread de/d10/f14 [0,4194304] 0 2024-05-13T22:48:05.976 INFO:tasks.workunit.client.0.smithi102.stdout:5/566: creat de/d25/fd3 x:0 0 0 2024-05-13T22:48:05.983 INFO:tasks.workunit.client.0.smithi102.stdout:5/567: dwrite de/d75/f85 [0,4194304] 0 2024-05-13T22:48:05.983 INFO:tasks.workunit.client.0.smithi102.stdout:3/495: symlink de/d10/d2a/d33/daa/lb6 0 2024-05-13T22:48:05.998 INFO:tasks.workunit.client.0.smithi102.stdout:2/391: getdents d7/d9/d10 0 2024-05-13T22:48:06.002 INFO:tasks.workunit.client.0.smithi102.stdout:5/568: getdents de/d25/d2a/d4b/d4d/d50/d99/dbb 0 2024-05-13T22:48:06.002 INFO:tasks.workunit.client.0.smithi102.stdout:5/569: fdatasync de/d25/d59/d28/f67 0 2024-05-13T22:48:06.003 INFO:tasks.workunit.client.0.smithi102.stdout:9/400: getdents d1/d6/d21 0 2024-05-13T22:48:06.008 INFO:tasks.workunit.client.0.smithi102.stdout:9/401: chown d1/d6/f4e 42 1 2024-05-13T22:48:06.008 INFO:tasks.workunit.client.0.smithi102.stdout:5/570: truncate de/d25/d43/f97 3627330 0 2024-05-13T22:48:06.019 INFO:tasks.workunit.client.0.smithi102.stdout:2/392: getdents d7/d15 0 2024-05-13T22:48:06.023 INFO:tasks.workunit.client.0.smithi102.stdout:2/393: mkdir d7/d9/d14/d61/d82 0 2024-05-13T22:48:06.027 INFO:tasks.workunit.client.0.smithi102.stdout:2/394: unlink d7/d9/d10/d25/d27/c5a 0 2024-05-13T22:48:06.028 INFO:tasks.workunit.client.0.smithi102.stdout:8/323: sync 2024-05-13T22:48:06.028 INFO:tasks.workunit.client.0.smithi102.stdout:7/256: sync 2024-05-13T22:48:06.028 INFO:tasks.workunit.client.0.smithi102.stdout:4/426: sync 2024-05-13T22:48:06.028 INFO:tasks.workunit.client.0.smithi102.stdout:7/257: chown d6/d13/de/fb 11 1 2024-05-13T22:48:06.030 INFO:tasks.workunit.client.0.smithi102.stdout:9/402: dread d1/f16 [0,4194304] 0 2024-05-13T22:48:06.030 INFO:tasks.workunit.client.0.smithi102.stdout:5/571: dread de/d33/f7b [0,4194304] 0 2024-05-13T22:48:06.035 INFO:tasks.workunit.client.0.smithi102.stdout:5/572: unlink de/d25/d2c/f96 0 2024-05-13T22:48:06.043 INFO:tasks.workunit.client.0.smithi102.stdout:2/395: getdents d7/d15/d67 0 2024-05-13T22:48:06.044 INFO:tasks.workunit.client.0.smithi102.stdout:2/396: write d7/d9/d14/f36 [613307,27008] 0 2024-05-13T22:48:06.045 INFO:tasks.workunit.client.0.smithi102.stdout:5/573: creat de/d25/d2a/d4b/dab/fd4 x:0 0 0 2024-05-13T22:48:06.047 INFO:tasks.workunit.client.0.smithi102.stdout:5/574: truncate de/d25/d2a/d4b/d4d/d93/fc3 583516 0 2024-05-13T22:48:06.050 INFO:tasks.workunit.client.0.smithi102.stdout:2/397: mkdir d7/d9/d10/d1b/d54/d83 0 2024-05-13T22:48:06.052 INFO:tasks.workunit.client.0.smithi102.stdout:5/575: mkdir de/d25/d2c/dd5 0 2024-05-13T22:48:06.055 INFO:tasks.workunit.client.0.smithi102.stdout:2/398: symlink d7/d9/d14/d31/l84 0 2024-05-13T22:48:06.060 INFO:tasks.workunit.client.0.smithi102.stdout:2/399: symlink d7/d9/l85 0 2024-05-13T22:48:06.062 INFO:tasks.workunit.client.0.smithi102.stdout:7/258: creat d6/d15/f47 x:0 0 0 2024-05-13T22:48:06.062 INFO:tasks.workunit.client.0.smithi102.stdout:7/259: chown d6/l1c 0 1 2024-05-13T22:48:06.064 INFO:tasks.workunit.client.0.smithi102.stdout:2/400: dread d7/d15/d67/f70 [0,4194304] 0 2024-05-13T22:48:06.065 INFO:tasks.workunit.client.0.smithi102.stdout:2/401: write d7/d9/d14/f36 [5748925,69196] 0 2024-05-13T22:48:06.070 INFO:tasks.workunit.client.0.smithi102.stdout:2/402: symlink d7/d9/d14/d61/l86 0 2024-05-13T22:48:06.074 INFO:tasks.workunit.client.0.smithi102.stdout:2/403: unlink d7/d15/d1e/l5d 0 2024-05-13T22:48:06.079 INFO:tasks.workunit.client.0.smithi102.stdout:7/260: link d6/d15/d21/d20/l40 d6/d15/d21/l48 0 2024-05-13T22:48:06.105 INFO:tasks.workunit.client.0.smithi102.stdout:0/469: rmdir d1/de/d32/d63 39 2024-05-13T22:48:06.107 INFO:tasks.workunit.client.0.smithi102.stdout:0/470: truncate d1/de/d32/d24/d2d/d3e/f73 195079 0 2024-05-13T22:48:06.110 INFO:tasks.workunit.client.0.smithi102.stdout:0/471: creat d1/de/d32/d24/d2d/d38/d3d/f95 x:0 0 0 2024-05-13T22:48:06.111 INFO:tasks.workunit.client.0.smithi102.stdout:0/472: chown d1/de/d32/d24/d2d/d38/d3d/d81 1 1 2024-05-13T22:48:06.114 INFO:tasks.workunit.client.0.smithi102.stdout:9/403: dwrite d1/f16 [4194304,4194304] 0 2024-05-13T22:48:06.116 INFO:tasks.workunit.client.0.smithi102.stdout:0/473: creat d1/de/d32/d63/f96 x:0 0 0 2024-05-13T22:48:06.117 INFO:tasks.workunit.client.0.smithi102.stdout:0/474: write d1/de/d32/d63/f96 [307371,31201] 0 2024-05-13T22:48:06.122 INFO:tasks.workunit.client.0.smithi102.stdout:6/396: rename d9/db/d14/c1b to d9/db/d14/d54/c83 0 2024-05-13T22:48:06.123 INFO:tasks.workunit.client.0.smithi102.stdout:0/475: unlink d1/de/d42/d61/c78 0 2024-05-13T22:48:06.127 INFO:tasks.workunit.client.0.smithi102.stdout:3/496: rename de/d10/f1d to de/d10/d2a/d8c/d7f/fb7 0 2024-05-13T22:48:06.129 INFO:tasks.workunit.client.0.smithi102.stdout:6/397: read d9/db/d14/d35/f52 [49863,40003] 0 2024-05-13T22:48:06.136 INFO:tasks.workunit.client.0.smithi102.stdout:6/398: symlink d9/d25/d71/l84 0 2024-05-13T22:48:06.138 INFO:tasks.workunit.client.0.smithi102.stdout:5/576: write de/f1e [777090,64838] 0 2024-05-13T22:48:06.140 INFO:tasks.workunit.client.0.smithi102.stdout:0/476: unlink d1/de/d32/f3f 0 2024-05-13T22:48:06.140 INFO:tasks.workunit.client.0.smithi102.stdout:3/497: dwrite de/d10/f44 [4194304,4194304] 0 2024-05-13T22:48:06.141 INFO:tasks.workunit.client.0.smithi102.stdout:3/498: stat de/d68/f94 0 2024-05-13T22:48:06.142 INFO:tasks.workunit.client.0.smithi102.stdout:3/499: chown de/d10/d2a/d8c/d7f/la7 65 1 2024-05-13T22:48:06.143 INFO:tasks.workunit.client.0.smithi102.stdout:6/399: mknod d9/d25/d60/d6a/c85 0 2024-05-13T22:48:06.145 INFO:tasks.workunit.client.0.smithi102.stdout:5/577: creat de/d25/d58/fd6 x:0 0 0 2024-05-13T22:48:06.147 INFO:tasks.workunit.client.0.smithi102.stdout:3/500: dread de/d10/f44 [4194304,4194304] 0 2024-05-13T22:48:06.152 INFO:tasks.workunit.client.0.smithi102.stdout:6/400: mknod d9/db/d14/d54/c86 0 2024-05-13T22:48:06.153 INFO:tasks.workunit.client.0.smithi102.stdout:5/578: dwrite de/d25/fd3 [0,4194304] 0 2024-05-13T22:48:06.154 INFO:tasks.workunit.client.0.smithi102.stdout:6/401: write d9/d45/f6f [397061,6355] 0 2024-05-13T22:48:06.161 INFO:tasks.workunit.client.0.smithi102.stdout:0/477: creat d1/de/d32/d24/d2d/d38/f97 x:0 0 0 2024-05-13T22:48:06.168 INFO:tasks.workunit.client.0.smithi102.stdout:0/478: dwrite d1/de/d32/d24/f40 [0,4194304] 0 2024-05-13T22:48:06.170 INFO:tasks.workunit.client.0.smithi102.stdout:6/402: rename d9/d25/d60/d6a/f79 to d9/d45/d56/d5d/f87 0 2024-05-13T22:48:06.171 INFO:tasks.workunit.client.0.smithi102.stdout:6/403: write d9/db/d14/d37/f78 [891840,48584] 0 2024-05-13T22:48:06.175 INFO:tasks.workunit.client.0.smithi102.stdout:3/501: creat de/d10/d2a/d33/fb8 x:0 0 0 2024-05-13T22:48:06.175 INFO:tasks.workunit.client.0.smithi102.stdout:7/261: write d6/f8 [2792196,81893] 0 2024-05-13T22:48:06.176 INFO:tasks.workunit.client.0.smithi102.stdout:0/479: creat d1/de/d32/d63/f98 x:0 0 0 2024-05-13T22:48:06.177 INFO:tasks.workunit.client.0.smithi102.stdout:7/262: chown d6/d13/d2c/d32/l44 202 1 2024-05-13T22:48:06.178 INFO:tasks.workunit.client.0.smithi102.stdout:0/480: truncate d1/d4d/d79/f80 638290 0 2024-05-13T22:48:06.179 INFO:tasks.workunit.client.0.smithi102.stdout:6/404: rmdir d9/db/d14/d35 39 2024-05-13T22:48:06.180 INFO:tasks.workunit.client.0.smithi102.stdout:0/481: chown d1/de/d32/f36 931346723 1 2024-05-13T22:48:06.184 INFO:tasks.workunit.client.0.smithi102.stdout:7/263: mkdir d6/d13/d2c/d32/d49 0 2024-05-13T22:48:06.185 INFO:tasks.workunit.client.0.smithi102.stdout:7/264: truncate d6/d22/f33 449540 0 2024-05-13T22:48:06.186 INFO:tasks.workunit.client.0.smithi102.stdout:6/405: creat d9/d25/f88 x:0 0 0 2024-05-13T22:48:06.188 INFO:tasks.workunit.client.0.smithi102.stdout:0/482: chown d1/de/d32/d24/d2d/c7c 298976 1 2024-05-13T22:48:06.190 INFO:tasks.workunit.client.0.smithi102.stdout:5/579: dread de/d25/d59/f44 [0,4194304] 0 2024-05-13T22:48:06.192 INFO:tasks.workunit.client.0.smithi102.stdout:5/580: read de/d25/d59/d6f/f72 [1285665,100113] 0 2024-05-13T22:48:06.192 INFO:tasks.workunit.client.0.smithi102.stdout:6/406: creat d9/d25/d60/d6a/f89 x:0 0 0 2024-05-13T22:48:06.196 INFO:tasks.workunit.client.0.smithi102.stdout:7/265: dwrite d6/fc [4194304,4194304] 0 2024-05-13T22:48:06.200 INFO:tasks.workunit.client.0.smithi102.stdout:6/407: fdatasync d9/fa 0 2024-05-13T22:48:06.202 INFO:tasks.workunit.client.0.smithi102.stdout:5/581: creat de/d25/d59/d7f/fd7 x:0 0 0 2024-05-13T22:48:06.203 INFO:tasks.workunit.client.0.smithi102.stdout:1/311: link d2/d4/d26/c3a d2/d4/c63 0 2024-05-13T22:48:06.204 INFO:tasks.workunit.client.0.smithi102.stdout:1/312: dread - d2/d4/d26/d4c/f59 zero size 2024-05-13T22:48:06.208 INFO:tasks.workunit.client.0.smithi102.stdout:0/483: symlink d1/l99 0 2024-05-13T22:48:06.209 INFO:tasks.workunit.client.0.smithi102.stdout:9/404: write d1/d6/db/d30/d47/f49 [701529,2766] 0 2024-05-13T22:48:06.211 INFO:tasks.workunit.client.0.smithi102.stdout:7/266: rmdir d6/d13/d2c/d39 39 2024-05-13T22:48:06.215 INFO:tasks.workunit.client.0.smithi102.stdout:3/502: unlink de/d10/d2a/d8c/d7f/fb7 0 2024-05-13T22:48:06.221 INFO:tasks.workunit.client.0.smithi102.stdout:5/582: creat de/d25/d59/d7f/d86/da5/fd8 x:0 0 0 2024-05-13T22:48:06.223 INFO:tasks.workunit.client.0.smithi102.stdout:1/313: truncate d2/d4/f23 2210198 0 2024-05-13T22:48:06.228 INFO:tasks.workunit.client.0.smithi102.stdout:9/405: readlink d1/d6/db/l20 0 2024-05-13T22:48:06.231 INFO:tasks.workunit.client.0.smithi102.stdout:7/267: creat d6/d15/d21/d20/f4a x:0 0 0 2024-05-13T22:48:06.237 INFO:tasks.workunit.client.0.smithi102.stdout:7/268: dwrite d6/d13/de/f29 [0,4194304] 0 2024-05-13T22:48:06.239 INFO:tasks.workunit.client.0.smithi102.stdout:5/583: creat de/d25/d59/d7f/d86/da5/fd9 x:0 0 0 2024-05-13T22:48:06.246 INFO:tasks.workunit.client.0.smithi102.stdout:5/584: dwrite de/d33/d53/d9e/fcd [0,4194304] 0 2024-05-13T22:48:06.250 INFO:tasks.workunit.client.0.smithi102.stdout:0/484: truncate d1/de/d32/f37 225032 0 2024-05-13T22:48:06.252 INFO:tasks.workunit.client.0.smithi102.stdout:0/485: write d1/de/f50 [837253,118238] 0 2024-05-13T22:48:06.253 INFO:tasks.workunit.client.0.smithi102.stdout:0/486: read - d1/de/d32/d24/f83 zero size 2024-05-13T22:48:06.260 INFO:tasks.workunit.client.0.smithi102.stdout:1/314: write d2/d45/f5a [2907187,88022] 0 2024-05-13T22:48:06.262 INFO:tasks.workunit.client.0.smithi102.stdout:7/269: dread d6/d13/de/f12 [0,4194304] 0 2024-05-13T22:48:06.262 INFO:tasks.workunit.client.0.smithi102.stdout:7/270: stat d6/d15/d21 0 2024-05-13T22:48:06.276 INFO:tasks.workunit.client.0.smithi102.stdout:6/408: dwrite d9/f66 [0,4194304] 0 2024-05-13T22:48:06.278 INFO:tasks.workunit.client.0.smithi102.stdout:1/315: creat d2/d4/d26/d56/f64 x:0 0 0 2024-05-13T22:48:06.282 INFO:tasks.workunit.client.0.smithi102.stdout:7/271: mkdir d6/d15/d21/d20/d4b 0 2024-05-13T22:48:06.283 INFO:tasks.workunit.client.0.smithi102.stdout:7/272: fdatasync d6/d13/d2c/d39/f3f 0 2024-05-13T22:48:06.285 INFO:tasks.workunit.client.0.smithi102.stdout:5/585: mkdir de/d25/d2a/d4b/d4d/d50/d99/dbb/dda 0 2024-05-13T22:48:06.287 INFO:tasks.workunit.client.0.smithi102.stdout:0/487: link d1/de/d32/d24/d2d/d3e/f73 d1/de/d32/d24/d2d/d38/d3d/f9a 0 2024-05-13T22:48:06.289 INFO:tasks.workunit.client.0.smithi102.stdout:3/503: link de/d1c/c62 de/d10/d2a/d8c/da8/cb9 0 2024-05-13T22:48:06.293 INFO:tasks.workunit.client.0.smithi102.stdout:6/409: symlink d9/d45/d56/l8a 0 2024-05-13T22:48:06.298 INFO:tasks.workunit.client.0.smithi102.stdout:1/316: mknod d2/d4/d5/d19/d28/c65 0 2024-05-13T22:48:06.300 INFO:tasks.workunit.client.0.smithi102.stdout:7/273: mkdir d6/d15/d4c 0 2024-05-13T22:48:06.301 INFO:tasks.workunit.client.0.smithi102.stdout:7/274: fsync d6/d13/d2c/d39/f3f 0 2024-05-13T22:48:06.303 INFO:tasks.workunit.client.0.smithi102.stdout:5/586: mkdir de/d25/d59/d7f/d86/da5/ddb 0 2024-05-13T22:48:06.303 INFO:tasks.workunit.client.0.smithi102.stdout:5/587: write de/d75/fb8 [273332,84414] 0 2024-05-13T22:48:06.309 INFO:tasks.workunit.client.0.smithi102.stdout:9/406: truncate d1/d6/d21/f29 3945476 0 2024-05-13T22:48:06.313 INFO:tasks.workunit.client.0.smithi102.stdout:7/275: mkdir d6/d15/d4d 0 2024-05-13T22:48:06.315 INFO:tasks.workunit.client.0.smithi102.stdout:6/410: dwrite d9/db/d2f/f62 [0,4194304] 0 2024-05-13T22:48:06.317 INFO:tasks.workunit.client.0.smithi102.stdout:6/411: write d9/db/d14/d37/f48 [2091047,80736] 0 2024-05-13T22:48:06.324 INFO:tasks.workunit.client.0.smithi102.stdout:1/317: fdatasync d2/d4/d5/f15 0 2024-05-13T22:48:06.423 INFO:tasks.workunit.client.0.smithi102.stdout:1/318: mknod d2/d4/d26/c66 0 2024-05-13T22:48:06.425 INFO:tasks.workunit.client.0.smithi102.stdout:1/319: write d2/d4/d5/f36 [789754,118309] 0 2024-05-13T22:48:06.426 INFO:tasks.workunit.client.0.smithi102.stdout:3/504: write de/d10/f14 [2259667,12353] 0 2024-05-13T22:48:06.428 INFO:tasks.workunit.client.0.smithi102.stdout:7/276: creat d6/d13/d2c/d32/d49/f4e x:0 0 0 2024-05-13T22:48:06.431 INFO:tasks.workunit.client.0.smithi102.stdout:0/488: dwrite d1/de/d42/f93 [0,4194304] 0 2024-05-13T22:48:06.432 INFO:tasks.workunit.client.0.smithi102.stdout:0/489: fsync d1/de/d32/f72 0 2024-05-13T22:48:06.449 INFO:tasks.workunit.client.0.smithi102.stdout:5/588: dwrite de/d25/d2a/d4b/f76 [0,4194304] 0 2024-05-13T22:48:06.451 INFO:tasks.workunit.client.0.smithi102.stdout:1/320: unlink d2/d10/l1b 0 2024-05-13T22:48:06.455 INFO:tasks.workunit.client.0.smithi102.stdout:9/407: write d1/d6/d21/f29 [2939138,9623] 0 2024-05-13T22:48:06.455 INFO:tasks.workunit.client.0.smithi102.stdout:3/505: chown de/d1c/d90/ca5 8 1 2024-05-13T22:48:06.456 INFO:tasks.workunit.client.0.smithi102.stdout:3/506: write de/d10/f43 [1325107,9852] 0 2024-05-13T22:48:06.457 INFO:tasks.workunit.client.0.smithi102.stdout:6/412: write d9/d45/d56/d5d/f87 [158910,43996] 0 2024-05-13T22:48:06.459 INFO:tasks.workunit.client.0.smithi102.stdout:7/277: rename d6/d13/d2c/l3c to d6/d15/d4c/l4f 0 2024-05-13T22:48:06.462 INFO:tasks.workunit.client.0.smithi102.stdout:0/490: mknod d1/de/d32/d24/d2d/d3e/c9b 0 2024-05-13T22:48:06.466 INFO:tasks.workunit.client.0.smithi102.stdout:5/589: mknod de/d33/d53/cdc 0 2024-05-13T22:48:06.467 INFO:tasks.workunit.client.0.smithi102.stdout:7/278: dwrite d6/d22/f23 [0,4194304] 0 2024-05-13T22:48:06.468 INFO:tasks.workunit.client.0.smithi102.stdout:7/279: write d6/d15/f1e [3321916,31162] 0 2024-05-13T22:48:06.469 INFO:tasks.workunit.client.0.smithi102.stdout:7/280: dread - d6/d15/d21/d20/f4a zero size 2024-05-13T22:48:06.471 INFO:tasks.workunit.client.0.smithi102.stdout:7/281: write d6/d13/d2c/d39/f3f [8047687,29311] 0 2024-05-13T22:48:06.471 INFO:tasks.workunit.client.0.smithi102.stdout:3/507: mknod de/cba 0 2024-05-13T22:48:06.474 INFO:tasks.workunit.client.0.smithi102.stdout:1/321: dread d2/d4/d5/f15 [8388608,4194304] 0 2024-05-13T22:48:06.480 INFO:tasks.workunit.client.0.smithi102.stdout:9/408: creat d1/d6/d45/d3e/f93 x:0 0 0 2024-05-13T22:48:06.480 INFO:tasks.workunit.client.0.smithi102.stdout:9/409: fdatasync d1/d6/db/d30/f36 0 2024-05-13T22:48:06.485 INFO:tasks.workunit.client.0.smithi102.stdout:7/282: creat d6/d15/d21/d20/f50 x:0 0 0 2024-05-13T22:48:06.496 INFO:tasks.workunit.client.0.smithi102.stdout:0/491: creat d1/de/d32/f9c x:0 0 0 2024-05-13T22:48:06.507 INFO:tasks.workunit.client.0.smithi102.stdout:6/413: rename d9/db/fe to d9/f8b 0 2024-05-13T22:48:06.510 INFO:tasks.workunit.client.0.smithi102.stdout:9/410: creat d1/f94 x:0 0 0 2024-05-13T22:48:06.512 INFO:tasks.workunit.client.0.smithi102.stdout:5/590: getdents de/d25/d43 0 2024-05-13T22:48:06.513 INFO:tasks.workunit.client.0.smithi102.stdout:5/591: write de/d25/d2a/d4b/d4d/fb5 [525209,130404] 0 2024-05-13T22:48:06.514 INFO:tasks.workunit.client.0.smithi102.stdout:3/508: chown de/d10/d50 2079977 1 2024-05-13T22:48:06.515 INFO:tasks.workunit.client.0.smithi102.stdout:5/592: read de/d25/d43/f97 [741107,7155] 0 2024-05-13T22:48:06.516 INFO:tasks.workunit.client.0.smithi102.stdout:5/593: chown de/d33/d53/d9e 242908172 1 2024-05-13T22:48:06.516 INFO:tasks.workunit.client.0.smithi102.stdout:5/594: write de/d25/d2a/d4b/dab/fd4 [229336,118525] 0 2024-05-13T22:48:06.523 INFO:tasks.workunit.client.0.smithi102.stdout:3/509: creat de/d10/d2a/d36/db2/fbb x:0 0 0 2024-05-13T22:48:06.525 INFO:tasks.workunit.client.0.smithi102.stdout:7/283: truncate d6/f2a 576067 0 2024-05-13T22:48:06.527 INFO:tasks.workunit.client.0.smithi102.stdout:6/414: symlink d9/db/l8c 0 2024-05-13T22:48:06.530 INFO:tasks.workunit.client.0.smithi102.stdout:9/411: write d1/d6/db/f1b [4775641,33748] 0 2024-05-13T22:48:06.532 INFO:tasks.workunit.client.0.smithi102.stdout:3/510: write de/d10/d2a/f2e [3765009,8519] 0 2024-05-13T22:48:06.535 INFO:tasks.workunit.client.0.smithi102.stdout:5/595: unlink l3 0 2024-05-13T22:48:06.537 INFO:tasks.workunit.client.0.smithi102.stdout:1/322: rename d2/d4/d5/c30 to d2/d4/d5/d19/c67 0 2024-05-13T22:48:06.539 INFO:tasks.workunit.client.0.smithi102.stdout:7/284: dread d6/d15/f1b [0,4194304] 0 2024-05-13T22:48:06.540 INFO:tasks.workunit.client.0.smithi102.stdout:9/412: truncate d1/d6/db/d4a/f52 772787 0 2024-05-13T22:48:06.542 INFO:tasks.workunit.client.0.smithi102.stdout:3/511: symlink de/d10/d50/lbc 0 2024-05-13T22:48:06.543 INFO:tasks.workunit.client.0.smithi102.stdout:3/512: chown de/d10/d2a/d36/d88/lb4 0 1 2024-05-13T22:48:06.544 INFO:tasks.workunit.client.0.smithi102.stdout:5/596: creat de/d25/d58/fdd x:0 0 0 2024-05-13T22:48:06.546 INFO:tasks.workunit.client.0.smithi102.stdout:1/323: write d2/d45/f47 [3934428,13090] 0 2024-05-13T22:48:06.550 INFO:tasks.workunit.client.0.smithi102.stdout:3/513: dwrite de/d10/d50/f47 [0,4194304] 0 2024-05-13T22:48:06.552 INFO:tasks.workunit.client.0.smithi102.stdout:7/285: creat d6/d13/d2c/d32/f51 x:0 0 0 2024-05-13T22:48:06.556 INFO:tasks.workunit.client.0.smithi102.stdout:0/492: truncate d1/f2c 3629004 0 2024-05-13T22:48:06.560 INFO:tasks.workunit.client.0.smithi102.stdout:7/286: dwrite d6/f8 [0,4194304] 0 2024-05-13T22:48:06.562 INFO:tasks.workunit.client.0.smithi102.stdout:9/413: dwrite d1/f1e [0,4194304] 0 2024-05-13T22:48:06.565 INFO:tasks.workunit.client.0.smithi102.stdout:6/415: creat d9/db/d14/f8d x:0 0 0 2024-05-13T22:48:06.565 INFO:tasks.workunit.client.0.smithi102.stdout:6/416: fdatasync f0 0 2024-05-13T22:48:06.566 INFO:tasks.workunit.client.0.smithi102.stdout:1/324: rmdir d2/d4/d5/d19 39 2024-05-13T22:48:06.567 INFO:tasks.workunit.client.0.smithi102.stdout:1/325: chown d2/f54 155454 1 2024-05-13T22:48:06.571 INFO:tasks.workunit.client.0.smithi102.stdout:0/493: dread - d1/d68/f86 zero size 2024-05-13T22:48:06.575 INFO:tasks.workunit.client.0.smithi102.stdout:9/414: creat d1/d6/db/d30/f95 x:0 0 0 2024-05-13T22:48:06.580 INFO:tasks.workunit.client.0.smithi102.stdout:1/326: dread - d2/d4/f37 zero size 2024-05-13T22:48:06.584 INFO:tasks.workunit.client.0.smithi102.stdout:5/597: rename de/d25/d2a/d4b/d4d/d50/d6c/caa to de/d33/cde 0 2024-05-13T22:48:06.588 INFO:tasks.workunit.client.0.smithi102.stdout:9/415: mkdir d1/d6/db/d4a/d75/d96 0 2024-05-13T22:48:06.591 INFO:tasks.workunit.client.0.smithi102.stdout:5/598: dwrite de/d25/d59/d7f/fd7 [0,4194304] 0 2024-05-13T22:48:06.592 INFO:tasks.workunit.client.0.smithi102.stdout:1/327: creat d2/d4/d38/f68 x:0 0 0 2024-05-13T22:48:06.596 INFO:tasks.workunit.client.0.smithi102.stdout:7/287: creat d6/d15/d21/f52 x:0 0 0 2024-05-13T22:48:06.597 INFO:tasks.workunit.client.0.smithi102.stdout:9/416: dwrite d1/d6/d45/d3e/f56 [0,4194304] 0 2024-05-13T22:48:06.598 INFO:tasks.workunit.client.0.smithi102.stdout:7/288: chown d6/d22/f33 0 1 2024-05-13T22:48:06.599 INFO:tasks.workunit.client.0.smithi102.stdout:9/417: truncate d1/d6/d45/d3e/f4c 4673690 0 2024-05-13T22:48:06.600 INFO:tasks.workunit.client.0.smithi102.stdout:9/418: stat d1/d6/db/f1b 0 2024-05-13T22:48:06.600 INFO:tasks.workunit.client.0.smithi102.stdout:9/419: stat d1/d6/d45/l80 0 2024-05-13T22:48:06.601 INFO:tasks.workunit.client.0.smithi102.stdout:5/599: fdatasync de/f5a 0 2024-05-13T22:48:06.604 INFO:tasks.workunit.client.0.smithi102.stdout:1/328: write d2/d4/d26/d56/f64 [989972,37442] 0 2024-05-13T22:48:06.604 INFO:tasks.workunit.client.0.smithi102.stdout:3/514: getdents de/d10/d2a/d8c/da8 0 2024-05-13T22:48:06.607 INFO:tasks.workunit.client.0.smithi102.stdout:1/329: write d2/d4/d5/f36 [631355,115641] 0 2024-05-13T22:48:06.607 INFO:tasks.workunit.client.0.smithi102.stdout:3/515: readlink de/d10/d2a/d36/l71 0 2024-05-13T22:48:06.609 INFO:tasks.workunit.client.0.smithi102.stdout:0/494: getdents d1/de/d42 0 2024-05-13T22:48:06.613 INFO:tasks.workunit.client.0.smithi102.stdout:0/495: dread - d1/de/d32/d24/f83 zero size 2024-05-13T22:48:06.613 INFO:tasks.workunit.client.0.smithi102.stdout:6/417: dwrite d9/db/f29 [0,4194304] 0 2024-05-13T22:48:06.616 INFO:tasks.workunit.client.0.smithi102.stdout:5/600: truncate de/d25/d59/f49 1148642 0 2024-05-13T22:48:06.618 INFO:tasks.workunit.client.0.smithi102.stdout:3/516: dread de/d10/d2a/d36/f83 [0,4194304] 0 2024-05-13T22:48:06.619 INFO:tasks.workunit.client.0.smithi102.stdout:9/420: dwrite d1/d6/d21/d71/f8c [0,4194304] 0 2024-05-13T22:48:06.620 INFO:tasks.workunit.client.0.smithi102.stdout:7/289: link d6/l1c d6/d13/d2c/d32/d49/l53 0 2024-05-13T22:48:06.623 INFO:tasks.workunit.client.0.smithi102.stdout:1/330: creat d2/d4/d26/d4c/f69 x:0 0 0 2024-05-13T22:48:06.630 INFO:tasks.workunit.client.0.smithi102.stdout:6/418: dwrite d9/db/d14/d37/f7b [0,4194304] 0 2024-05-13T22:48:06.632 INFO:tasks.workunit.client.0.smithi102.stdout:5/601: dwrite de/d25/d2c/f5f [0,4194304] 0 2024-05-13T22:48:06.633 INFO:tasks.workunit.client.0.smithi102.stdout:9/421: dread - d1/d6/d3b/d5b/f5f zero size 2024-05-13T22:48:06.636 INFO:tasks.workunit.client.0.smithi102.stdout:7/290: dread d6/fc [4194304,4194304] 0 2024-05-13T22:48:06.637 INFO:tasks.workunit.client.0.smithi102.stdout:3/517: mkdir de/d1c/d79/dbd 0 2024-05-13T22:48:06.640 INFO:tasks.workunit.client.0.smithi102.stdout:0/496: creat d1/de/d32/d24/d2d/f9d x:0 0 0 2024-05-13T22:48:06.643 INFO:tasks.workunit.client.0.smithi102.stdout:5/602: rename de/d25/d58/l62 to de/d25/d43/dd1/ldf 0 2024-05-13T22:48:06.646 INFO:tasks.workunit.client.0.smithi102.stdout:3/518: unlink de/d10/d2a/l3c 0 2024-05-13T22:48:06.654 INFO:tasks.workunit.client.0.smithi102.stdout:0/497: creat d1/de/d32/d24/d2d/d3e/f9e x:0 0 0 2024-05-13T22:48:06.656 INFO:tasks.workunit.client.0.smithi102.stdout:1/331: rename d2/d4/d26/d4c to d2/d4/d5/d19/d28/d6a 0 2024-05-13T22:48:06.659 INFO:tasks.workunit.client.0.smithi102.stdout:3/519: chown de/d10/d2a/d36/l8e 123 1 2024-05-13T22:48:06.660 INFO:tasks.workunit.client.0.smithi102.stdout:3/520: stat de/d10/l42 0 2024-05-13T22:48:06.663 INFO:tasks.workunit.client.0.smithi102.stdout:9/422: rename d1/d6/d3b/d5b/f5f to d1/d6/db/d4a/d81/f97 0 2024-05-13T22:48:06.663 INFO:tasks.workunit.client.0.smithi102.stdout:3/521: rename de/d10 to de/d10/d2a/d36/db2/dbe 22 2024-05-13T22:48:06.666 INFO:tasks.workunit.client.0.smithi102.stdout:7/291: creat d6/d13/d2c/f54 x:0 0 0 2024-05-13T22:48:06.668 INFO:tasks.workunit.client.0.smithi102.stdout:6/419: truncate d9/db/d2f/f62 1159878 0 2024-05-13T22:48:06.669 INFO:tasks.workunit.client.0.smithi102.stdout:6/420: readlink d9/db/d2f/l6e 0 2024-05-13T22:48:06.673 INFO:tasks.workunit.client.0.smithi102.stdout:7/292: dwrite d6/d15/d21/d20/f50 [0,4194304] 0 2024-05-13T22:48:06.676 INFO:tasks.workunit.client.0.smithi102.stdout:1/332: symlink d2/d4/d5/d19/l6b 0 2024-05-13T22:48:06.677 INFO:tasks.workunit.client.0.smithi102.stdout:0/498: mkdir d1/d4d/d79/d9f 0 2024-05-13T22:48:06.678 INFO:tasks.workunit.client.0.smithi102.stdout:0/499: readlink d1/de/d32/d24/d2d/l4e 0 2024-05-13T22:48:06.689 INFO:tasks.workunit.client.0.smithi102.stdout:6/421: symlink d9/db/d14/l8e 0 2024-05-13T22:48:06.690 INFO:tasks.workunit.client.0.smithi102.stdout:6/422: chown d9/d25 53 1 2024-05-13T22:48:06.691 INFO:tasks.workunit.client.0.smithi102.stdout:6/423: dread - d9/d25/d60/f7e zero size 2024-05-13T22:48:06.692 INFO:tasks.workunit.client.0.smithi102.stdout:9/423: creat d1/d6/d45/f98 x:0 0 0 2024-05-13T22:48:06.693 INFO:tasks.workunit.client.0.smithi102.stdout:9/424: chown d1/d6/db/d30/f95 26317789 1 2024-05-13T22:48:06.694 INFO:tasks.workunit.client.0.smithi102.stdout:9/425: read d1/f3a [67282,100301] 0 2024-05-13T22:48:06.695 INFO:tasks.workunit.client.0.smithi102.stdout:7/293: mkdir d6/d15/d4d/d55 0 2024-05-13T22:48:06.695 INFO:tasks.workunit.client.0.smithi102.stdout:9/426: read d1/d6/d45/d3e/f56 [1894352,75408] 0 2024-05-13T22:48:06.696 INFO:tasks.workunit.client.0.smithi102.stdout:3/522: rmdir de/d1c/d79/dbd 0 2024-05-13T22:48:06.698 INFO:tasks.workunit.client.0.smithi102.stdout:1/333: write d2/d45/f48 [1356996,120967] 0 2024-05-13T22:48:06.705 INFO:tasks.workunit.client.0.smithi102.stdout:5/603: dwrite de/d25/d59/f49 [0,4194304] 0 2024-05-13T22:48:06.730 INFO:tasks.workunit.client.0.smithi102.stdout:2/404: sync 2024-05-13T22:48:06.730 INFO:tasks.workunit.client.0.smithi102.stdout:8/324: sync 2024-05-13T22:48:06.730 INFO:tasks.workunit.client.0.smithi102.stdout:4/427: sync 2024-05-13T22:48:06.732 INFO:tasks.workunit.client.0.smithi102.stdout:4/428: truncate d3/d6b/f6f 655698 0 2024-05-13T22:48:06.733 INFO:tasks.workunit.client.0.smithi102.stdout:7/294: chown d6/d13/d2c/d32/d49/l53 122543 1 2024-05-13T22:48:06.735 INFO:tasks.workunit.client.0.smithi102.stdout:4/429: truncate d3/d19/d23/d4e/d68/f94 858134 0 2024-05-13T22:48:06.735 INFO:tasks.workunit.client.0.smithi102.stdout:4/430: dread - d3/d4/f7b zero size 2024-05-13T22:48:06.735 INFO:tasks.workunit.client.0.smithi102.stdout:4/431: chown d3 223 1 2024-05-13T22:48:06.737 INFO:tasks.workunit.client.0.smithi102.stdout:0/500: write d1/f2c [1521567,44325] 0 2024-05-13T22:48:06.738 INFO:tasks.workunit.client.0.smithi102.stdout:3/523: fdatasync de/d10/f44 0 2024-05-13T22:48:06.738 INFO:tasks.workunit.client.0.smithi102.stdout:0/501: chown d1/de/d42 5595417 1 2024-05-13T22:48:06.740 INFO:tasks.workunit.client.0.smithi102.stdout:6/424: creat d9/db/d14/d35/d47/f8f x:0 0 0 2024-05-13T22:48:06.745 INFO:tasks.workunit.client.0.smithi102.stdout:8/325: mkdir d2/d7/d27/d64 0 2024-05-13T22:48:06.747 INFO:tasks.workunit.client.0.smithi102.stdout:2/405: mkdir d7/d15/d1e/d87 0 2024-05-13T22:48:06.756 INFO:tasks.workunit.client.0.smithi102.stdout:9/427: creat d1/d6/db/d30/d8a/f99 x:0 0 0 2024-05-13T22:48:06.757 INFO:tasks.workunit.client.0.smithi102.stdout:3/524: rmdir de/d1c/d90 39 2024-05-13T22:48:06.757 INFO:tasks.workunit.client.0.smithi102.stdout:6/425: symlink d9/d45/d56/l90 0 2024-05-13T22:48:06.758 INFO:tasks.workunit.client.0.smithi102.stdout:4/432: creat d3/d19/d23/d39/d5d/f98 x:0 0 0 2024-05-13T22:48:06.758 INFO:tasks.workunit.client.0.smithi102.stdout:9/428: chown d1/d6/db/d4a 14 1 2024-05-13T22:48:06.758 INFO:tasks.workunit.client.0.smithi102.stdout:7/295: dread d6/d15/f37 [0,4194304] 0 2024-05-13T22:48:06.759 INFO:tasks.workunit.client.0.smithi102.stdout:8/326: rename d2/d36/d49/d5d/f2c to d2/d7/d25/d41/f65 0 2024-05-13T22:48:06.759 INFO:tasks.workunit.client.0.smithi102.stdout:2/406: symlink d7/d9/d10/d25/d56/l88 0 2024-05-13T22:48:06.763 INFO:tasks.workunit.client.0.smithi102.stdout:3/525: dread - de/d10/d50/d63/fb0 zero size 2024-05-13T22:48:06.763 INFO:tasks.workunit.client.0.smithi102.stdout:4/433: dread d3/d4/f32 [0,4194304] 0 2024-05-13T22:48:06.764 INFO:tasks.workunit.client.0.smithi102.stdout:6/426: symlink d9/db/d14/d35/l91 0 2024-05-13T22:48:06.764 INFO:tasks.workunit.client.0.smithi102.stdout:4/434: chown d3/d22/f36 160 1 2024-05-13T22:48:06.770 INFO:tasks.workunit.client.0.smithi102.stdout:9/429: creat d1/d6/db/d4a/d64/f9a x:0 0 0 2024-05-13T22:48:06.772 INFO:tasks.workunit.client.0.smithi102.stdout:7/296: fdatasync d6/d13/de/fb 0 2024-05-13T22:48:06.773 INFO:tasks.workunit.client.0.smithi102.stdout:5/604: write de/d25/d2a/d4b/d4d/d50/d6c/db9/fc6 [5303032,117586] 0 2024-05-13T22:48:06.777 INFO:tasks.workunit.client.0.smithi102.stdout:4/435: truncate d3/d19/d23/d39/f8b 5399209 0 2024-05-13T22:48:06.778 INFO:tasks.workunit.client.0.smithi102.stdout:8/327: write d2/d7/d12/f23 [2707571,3536] 0 2024-05-13T22:48:06.779 INFO:tasks.workunit.client.0.smithi102.stdout:4/436: read - d3/d19/d23/d4e/f72 zero size 2024-05-13T22:48:06.782 INFO:tasks.workunit.client.0.smithi102.stdout:4/437: truncate d3/d19/d23/d3a/f56 922250 0 2024-05-13T22:48:06.782 INFO:tasks.workunit.client.0.smithi102.stdout:3/526: dread de/d1c/f8a [0,4194304] 0 2024-05-13T22:48:06.783 INFO:tasks.workunit.client.0.smithi102.stdout:9/430: mknod d1/d6/d3b/d5b/c9b 0 2024-05-13T22:48:06.784 INFO:tasks.workunit.client.0.smithi102.stdout:3/527: truncate de/d10/d50/d63/fb0 693659 0 2024-05-13T22:48:06.785 INFO:tasks.workunit.client.0.smithi102.stdout:7/297: readlink d6/d15/d21/d20/l40 0 2024-05-13T22:48:06.785 INFO:tasks.workunit.client.0.smithi102.stdout:3/528: chown de/d10/d46/f6c 185838219 1 2024-05-13T22:48:06.786 INFO:tasks.workunit.client.0.smithi102.stdout:7/298: write d6/d15/d21/d20/f50 [4211632,87932] 0 2024-05-13T22:48:06.791 INFO:tasks.workunit.client.0.smithi102.stdout:8/328: rename d2/d7/d25/c2f to d2/d7/d27/c66 0 2024-05-13T22:48:06.792 INFO:tasks.workunit.client.0.smithi102.stdout:2/407: creat d7/d9/f89 x:0 0 0 2024-05-13T22:48:06.793 INFO:tasks.workunit.client.0.smithi102.stdout:8/329: stat d2/d36/d49/d5d/f26 0 2024-05-13T22:48:06.794 INFO:tasks.workunit.client.0.smithi102.stdout:4/438: rmdir d3/d19/d23 39 2024-05-13T22:48:06.795 INFO:tasks.workunit.client.0.smithi102.stdout:7/299: dwrite d6/d15/f1e [8388608,4194304] 0 2024-05-13T22:48:06.803 INFO:tasks.workunit.client.0.smithi102.stdout:3/529: symlink de/d10/d50/lbf 0 2024-05-13T22:48:06.804 INFO:tasks.workunit.client.0.smithi102.stdout:4/439: dread d3/d22/f3e [0,4194304] 0 2024-05-13T22:48:06.805 INFO:tasks.workunit.client.0.smithi102.stdout:3/530: read de/d10/f14 [2200437,7340] 0 2024-05-13T22:48:06.807 INFO:tasks.workunit.client.0.smithi102.stdout:8/330: rmdir d2/d36 39 2024-05-13T22:48:06.811 INFO:tasks.workunit.client.0.smithi102.stdout:2/408: creat d7/d9/d10/d1b/f8a x:0 0 0 2024-05-13T22:48:06.812 INFO:tasks.workunit.client.0.smithi102.stdout:6/427: dwrite d9/db/f3c [0,4194304] 0 2024-05-13T22:48:06.816 INFO:tasks.workunit.client.0.smithi102.stdout:9/431: symlink d1/d6/d21/d71/d91/l9c 0 2024-05-13T22:48:06.818 INFO:tasks.workunit.client.0.smithi102.stdout:9/432: chown d1/d6/d45/d3e 6112324 1 2024-05-13T22:48:06.818 INFO:tasks.workunit.client.0.smithi102.stdout:9/433: chown d1/d6/db/d4a/d75/d96 30579444 1 2024-05-13T22:48:06.820 INFO:tasks.workunit.client.0.smithi102.stdout:9/434: write d1/f94 [110253,119032] 0 2024-05-13T22:48:06.821 INFO:tasks.workunit.client.0.smithi102.stdout:5/605: creat de/d25/fe0 x:0 0 0 2024-05-13T22:48:06.823 INFO:tasks.workunit.client.0.smithi102.stdout:3/531: dwrite de/d10/d2a/d33/f40 [0,4194304] 0 2024-05-13T22:48:06.825 INFO:tasks.workunit.client.0.smithi102.stdout:7/300: dread d6/d15/f18 [0,4194304] 0 2024-05-13T22:48:06.827 INFO:tasks.workunit.client.0.smithi102.stdout:2/409: mkdir d7/d15/d67/d6a/d8b 0 2024-05-13T22:48:06.833 INFO:tasks.workunit.client.0.smithi102.stdout:9/435: creat d1/d6/d21/d71/f9d x:0 0 0 2024-05-13T22:48:06.836 INFO:tasks.workunit.client.0.smithi102.stdout:2/410: dwrite d7/d9/d14/f36 [0,4194304] 0 2024-05-13T22:48:06.839 INFO:tasks.workunit.client.0.smithi102.stdout:7/301: symlink d6/d13/d2c/d32/d49/l56 0 2024-05-13T22:48:06.842 INFO:tasks.workunit.client.0.smithi102.stdout:4/440: write d3/d4/f5a [208910,99042] 0 2024-05-13T22:48:06.844 INFO:tasks.workunit.client.0.smithi102.stdout:3/532: dwrite de/d10/f43 [0,4194304] 0 2024-05-13T22:48:06.844 INFO:tasks.workunit.client.0.smithi102.stdout:5/606: symlink de/d25/d2c/dd5/le1 0 2024-05-13T22:48:06.845 INFO:tasks.workunit.client.0.smithi102.stdout:8/331: dread d2/f4 [0,4194304] 0 2024-05-13T22:48:06.846 INFO:tasks.workunit.client.0.smithi102.stdout:9/436: creat d1/d6b/f9e x:0 0 0 2024-05-13T22:48:06.846 INFO:tasks.workunit.client.0.smithi102.stdout:5/607: dread - de/d25/d59/d7f/fc9 zero size 2024-05-13T22:48:06.850 INFO:tasks.workunit.client.0.smithi102.stdout:2/411: truncate d7/f4a 828499 0 2024-05-13T22:48:06.851 INFO:tasks.workunit.client.0.smithi102.stdout:7/302: symlink d6/d13/d2c/d32/d49/l57 0 2024-05-13T22:48:06.857 INFO:tasks.workunit.client.0.smithi102.stdout:3/533: dwrite de/d10/f97 [0,4194304] 0 2024-05-13T22:48:06.857 INFO:tasks.workunit.client.0.smithi102.stdout:5/608: mknod de/d25/d43/d7a/db1/dc5/dca/ce2 0 2024-05-13T22:48:06.861 INFO:tasks.workunit.client.0.smithi102.stdout:5/609: readlink de/d75/laf 0 2024-05-13T22:48:06.862 INFO:tasks.workunit.client.0.smithi102.stdout:9/437: mknod d1/d6/c9f 0 2024-05-13T22:48:06.864 INFO:tasks.workunit.client.0.smithi102.stdout:8/332: creat d2/d36/d49/d5d/f67 x:0 0 0 2024-05-13T22:48:06.865 INFO:tasks.workunit.client.0.smithi102.stdout:6/428: dwrite d9/d25/f40 [0,4194304] 0 2024-05-13T22:48:06.868 INFO:tasks.workunit.client.0.smithi102.stdout:8/333: write d2/d36/f5a [433521,117845] 0 2024-05-13T22:48:06.869 INFO:tasks.workunit.client.0.smithi102.stdout:3/534: dread - de/d68/f94 zero size 2024-05-13T22:48:06.873 INFO:tasks.workunit.client.0.smithi102.stdout:3/535: fsync de/d10/d2a/d36/db2/fbb 0 2024-05-13T22:48:06.873 INFO:tasks.workunit.client.0.smithi102.stdout:4/441: rename d3/d19/d23/d3a/f4d to d3/d4/d6/de/d1f/d41/d6a/f99 0 2024-05-13T22:48:06.875 INFO:tasks.workunit.client.0.smithi102.stdout:9/438: dwrite d1/f2b [4194304,4194304] 0 2024-05-13T22:48:06.876 INFO:tasks.workunit.client.0.smithi102.stdout:4/442: dread - d3/d4/d6/de/d1f/d41/f8a zero size 2024-05-13T22:48:06.878 INFO:tasks.workunit.client.0.smithi102.stdout:5/610: chown de/d33/d34/c3b 481 1 2024-05-13T22:48:06.884 INFO:tasks.workunit.client.0.smithi102.stdout:2/412: getdents d7/d15/d1e 0 2024-05-13T22:48:06.888 INFO:tasks.workunit.client.0.smithi102.stdout:4/443: dwrite d3/d19/d23/f3d [0,4194304] 0 2024-05-13T22:48:06.889 INFO:tasks.workunit.client.0.smithi102.stdout:9/439: dwrite d1/f31 [4194304,4194304] 0 2024-05-13T22:48:06.890 INFO:tasks.workunit.client.0.smithi102.stdout:3/536: creat de/d10/d50/d8f/fc0 x:0 0 0 2024-05-13T22:48:06.890 INFO:tasks.workunit.client.0.smithi102.stdout:3/537: readlink l4 0 2024-05-13T22:48:06.892 INFO:tasks.workunit.client.0.smithi102.stdout:3/538: write de/d10/d2a/f2f [853313,130590] 0 2024-05-13T22:48:06.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:06 smithi130 ceph-mon[36591]: pgmap v20: 65 pgs: 65 active+clean; 843 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 MiB/s rd, 101 MiB/s wr, 302 op/s 2024-05-13T22:48:06.903 INFO:tasks.workunit.client.0.smithi102.stdout:7/303: truncate d6/d15/d21/d20/f50 3986506 0 2024-05-13T22:48:06.904 INFO:tasks.workunit.client.0.smithi102.stdout:6/429: mknod d9/db/d14/c92 0 2024-05-13T22:48:06.907 INFO:tasks.workunit.client.0.smithi102.stdout:5/611: dread de/d33/d53/d9e/fa7 [0,4194304] 0 2024-05-13T22:48:06.908 INFO:tasks.workunit.client.0.smithi102.stdout:5/612: readlink de/d75/laf 0 2024-05-13T22:48:06.910 INFO:tasks.workunit.client.0.smithi102.stdout:4/444: dread d3/d22/f3e [0,4194304] 0 2024-05-13T22:48:06.911 INFO:tasks.workunit.client.0.smithi102.stdout:9/440: truncate d1/d6/db/f66 244557 0 2024-05-13T22:48:06.912 INFO:tasks.workunit.client.0.smithi102.stdout:3/539: symlink de/d10/d34/d5d/lc1 0 2024-05-13T22:48:06.914 INFO:tasks.workunit.client.0.smithi102.stdout:6/430: read f3 [188131,118240] 0 2024-05-13T22:48:06.917 INFO:tasks.workunit.client.0.smithi102.stdout:2/413: rename d7/f4a to d7/d9/d10/d25/d56/f8c 0 2024-05-13T22:48:06.917 INFO:tasks.workunit.client.0.smithi102.stdout:6/431: write d9/db/d14/d37/f7b [4947701,65198] 0 2024-05-13T22:48:06.919 INFO:tasks.workunit.client.0.smithi102.stdout:6/432: chown d9/d45/d56/l76 16 1 2024-05-13T22:48:06.920 INFO:tasks.workunit.client.0.smithi102.stdout:9/441: mknod d1/d6b/ca0 0 2024-05-13T22:48:06.922 INFO:tasks.workunit.client.0.smithi102.stdout:6/433: fsync d9/db/f29 0 2024-05-13T22:48:06.922 INFO:tasks.workunit.client.0.smithi102.stdout:4/445: dwrite d3/d4/d6/de/d1f/d41/f8a [0,4194304] 0 2024-05-13T22:48:06.923 INFO:tasks.workunit.client.0.smithi102.stdout:3/540: chown de/d10/d2a/f3a 358001584 1 2024-05-13T22:48:06.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:06 smithi102 ceph-mon[28599]: pgmap v20: 65 pgs: 65 active+clean; 843 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 MiB/s rd, 101 MiB/s wr, 302 op/s 2024-05-13T22:48:06.929 INFO:tasks.workunit.client.0.smithi102.stdout:8/334: getdents d2/d7/d27 0 2024-05-13T22:48:06.934 INFO:tasks.workunit.client.0.smithi102.stdout:3/541: symlink de/d10/d50/d63/lc2 0 2024-05-13T22:48:06.935 INFO:tasks.workunit.client.0.smithi102.stdout:6/434: dwrite d9/db/f29 [0,4194304] 0 2024-05-13T22:48:06.940 INFO:tasks.workunit.client.0.smithi102.stdout:7/304: rename d6/d15/d21 to d6/d13/d58 0 2024-05-13T22:48:06.941 INFO:tasks.workunit.client.0.smithi102.stdout:6/435: read - d9/db/d2f/d59/f80 zero size 2024-05-13T22:48:06.941 INFO:tasks.workunit.client.0.smithi102.stdout:4/446: dwrite d3/d4/d6/de/d1f/f28 [0,4194304] 0 2024-05-13T22:48:06.948 INFO:tasks.workunit.client.0.smithi102.stdout:8/335: dwrite d2/d7/d12/f23 [0,4194304] 0 2024-05-13T22:48:06.952 INFO:tasks.workunit.client.0.smithi102.stdout:9/442: dwrite d1/d6/d21/f29 [0,4194304] 0 2024-05-13T22:48:06.955 INFO:tasks.workunit.client.0.smithi102.stdout:7/305: dread d6/d22/f23 [0,4194304] 0 2024-05-13T22:48:06.956 INFO:tasks.workunit.client.0.smithi102.stdout:8/336: chown d2/d7/d12/f13 13 1 2024-05-13T22:48:06.960 INFO:tasks.workunit.client.0.smithi102.stdout:5/613: rename de/d25/d2c/f5f to de/d25/d2a/d4b/d4d/d50/d99/dbb/dda/fe3 0 2024-05-13T22:48:06.961 INFO:tasks.workunit.client.0.smithi102.stdout:5/614: readlink de/d75/lb7 0 2024-05-13T22:48:06.969 INFO:tasks.workunit.client.0.smithi102.stdout:8/337: dwrite d2/d7/d12/f58 [0,4194304] 0 2024-05-13T22:48:06.969 INFO:tasks.workunit.client.0.smithi102.stdout:4/447: unlink d3/d19/c27 0 2024-05-13T22:48:06.971 INFO:tasks.workunit.client.0.smithi102.stdout:2/414: getdents d7/d9/d14 0 2024-05-13T22:48:06.975 INFO:tasks.workunit.client.0.smithi102.stdout:7/306: creat d6/d13/de/f59 x:0 0 0 2024-05-13T22:48:06.976 INFO:tasks.workunit.client.0.smithi102.stdout:4/448: dread d3/d4/d6/de/d1f/d41/f8a [0,4194304] 0 2024-05-13T22:48:06.976 INFO:tasks.workunit.client.0.smithi102.stdout:7/307: write d6/f8 [4270974,120267] 0 2024-05-13T22:48:06.977 INFO:tasks.workunit.client.0.smithi102.stdout:4/449: write d3/d4/f2e [3750133,97908] 0 2024-05-13T22:48:06.980 INFO:tasks.workunit.client.0.smithi102.stdout:5/615: mkdir de/d25/d59/d28/de4 0 2024-05-13T22:48:06.985 INFO:tasks.workunit.client.0.smithi102.stdout:5/616: dread de/d25/d2a/d4b/d4d/d50/d99/dbb/dda/fe3 [4194304,4194304] 0 2024-05-13T22:48:06.985 INFO:tasks.workunit.client.0.smithi102.stdout:5/617: readlink de/d75/laf 0 2024-05-13T22:48:07.007 INFO:tasks.workunit.client.0.smithi102.stdout:3/542: write f7 [172130,130929] 0 2024-05-13T22:48:07.008 INFO:tasks.workunit.client.0.smithi102.stdout:8/338: symlink d2/d36/d49/l68 0 2024-05-13T22:48:07.009 INFO:tasks.workunit.client.0.smithi102.stdout:8/339: write d2/d7/d27/f3d [1245984,48114] 0 2024-05-13T22:48:07.020 INFO:tasks.workunit.client.0.smithi102.stdout:7/308: fdatasync d6/d22/f23 0 2024-05-13T22:48:07.022 INFO:tasks.workunit.client.0.smithi102.stdout:7/309: write d6/d13/d2c/f2f [765071,104011] 0 2024-05-13T22:48:07.030 INFO:tasks.workunit.client.0.smithi102.stdout:3/543: stat de/f78 0 2024-05-13T22:48:07.032 INFO:tasks.workunit.client.0.smithi102.stdout:4/450: dwrite d3/d4/d6/de/d1f/d41/d6a/f99 [0,4194304] 0 2024-05-13T22:48:07.033 INFO:tasks.workunit.client.0.smithi102.stdout:6/436: dwrite d9/db/d2f/f62 [0,4194304] 0 2024-05-13T22:48:07.038 INFO:tasks.workunit.client.0.smithi102.stdout:9/443: mkdir d1/d6/d45/da1 0 2024-05-13T22:48:07.045 INFO:tasks.workunit.client.0.smithi102.stdout:6/437: mknod d9/d45/d56/d5d/c93 0 2024-05-13T22:48:07.048 INFO:tasks.workunit.client.0.smithi102.stdout:8/340: write d2/f4 [5075127,56578] 0 2024-05-13T22:48:07.048 INFO:tasks.workunit.client.0.smithi102.stdout:8/341: chown d2/d7 3130915 1 2024-05-13T22:48:07.049 INFO:tasks.workunit.client.0.smithi102.stdout:2/415: truncate d7/d9/d14/f36 3490144 0 2024-05-13T22:48:07.054 INFO:tasks.workunit.client.0.smithi102.stdout:7/310: rename d6/f2a to d6/d15/d4c/f5a 0 2024-05-13T22:48:07.056 INFO:tasks.workunit.client.0.smithi102.stdout:5/618: dwrite de/d25/d59/f32 [4194304,4194304] 0 2024-05-13T22:48:07.060 INFO:tasks.workunit.client.0.smithi102.stdout:6/438: creat d9/db/d14/d35/f94 x:0 0 0 2024-05-13T22:48:07.062 INFO:tasks.workunit.client.0.smithi102.stdout:4/451: creat d3/d19/d23/d4e/d68/d85/f9a x:0 0 0 2024-05-13T22:48:07.068 INFO:tasks.workunit.client.0.smithi102.stdout:8/342: creat d2/d7/f69 x:0 0 0 2024-05-13T22:48:07.070 INFO:tasks.workunit.client.0.smithi102.stdout:4/452: dwrite d3/d4/f47 [0,4194304] 0 2024-05-13T22:48:07.071 INFO:tasks.workunit.client.0.smithi102.stdout:4/453: readlink d3/d4/d6/de/l52 0 2024-05-13T22:48:07.072 INFO:tasks.workunit.client.0.smithi102.stdout:2/416: symlink d7/d9/d14/l8d 0 2024-05-13T22:48:07.072 INFO:tasks.workunit.client.0.smithi102.stdout:3/544: truncate de/d1c/d79/f2b 3494611 0 2024-05-13T22:48:07.074 INFO:tasks.workunit.client.0.smithi102.stdout:2/417: write d7/d9/f89 [625594,62789] 0 2024-05-13T22:48:07.074 INFO:tasks.workunit.client.0.smithi102.stdout:3/545: chown de/d10/d34/f93 3369 1 2024-05-13T22:48:07.079 INFO:tasks.workunit.client.0.smithi102.stdout:7/311: mknod d6/d15/d4c/c5b 0 2024-05-13T22:48:07.080 INFO:tasks.workunit.client.0.smithi102.stdout:9/444: creat d1/d6/d3b/d5b/fa2 x:0 0 0 2024-05-13T22:48:07.081 INFO:tasks.workunit.client.0.smithi102.stdout:5/619: mknod de/d33/d53/d9e/ce5 0 2024-05-13T22:48:07.087 INFO:tasks.workunit.client.0.smithi102.stdout:3/546: dwrite de/d1c/f52 [0,4194304] 0 2024-05-13T22:48:07.093 INFO:tasks.workunit.client.0.smithi102.stdout:6/439: mknod d9/db/c95 0 2024-05-13T22:48:07.093 INFO:tasks.workunit.client.0.smithi102.stdout:7/312: dwrite d6/d15/f1e [12582912,4194304] 0 2024-05-13T22:48:07.096 INFO:tasks.workunit.client.0.smithi102.stdout:7/313: chown d6/d22/f33 127 1 2024-05-13T22:48:07.099 INFO:tasks.workunit.client.0.smithi102.stdout:5/620: mknod de/d25/d2a/ce6 0 2024-05-13T22:48:07.103 INFO:tasks.workunit.client.0.smithi102.stdout:8/343: dread d2/d7/f4e [0,4194304] 0 2024-05-13T22:48:07.103 INFO:tasks.workunit.client.0.smithi102.stdout:9/445: symlink d1/d6/db/d4a/d81/la3 0 2024-05-13T22:48:07.106 INFO:tasks.workunit.client.0.smithi102.stdout:8/344: chown d2/d7/d25/d41/d50/f34 13 1 2024-05-13T22:48:07.110 INFO:tasks.workunit.client.0.smithi102.stdout:7/314: dwrite d6/d22/f24 [0,4194304] 0 2024-05-13T22:48:07.112 INFO:tasks.workunit.client.0.smithi102.stdout:7/315: chown d6/d13/d58/f26 104803 1 2024-05-13T22:48:07.114 INFO:tasks.workunit.client.0.smithi102.stdout:5/621: rename de/l19 to de/d25/d59/d7f/d86/da5/le7 0 2024-05-13T22:48:07.116 INFO:tasks.workunit.client.0.smithi102.stdout:4/454: getdents d3/d4/d6/de/d1f 0 2024-05-13T22:48:07.119 INFO:tasks.workunit.client.0.smithi102.stdout:2/418: dwrite d7/d9/d11/f38 [0,4194304] 0 2024-05-13T22:48:07.121 INFO:tasks.workunit.client.0.smithi102.stdout:6/440: symlink d9/db/d14/l96 0 2024-05-13T22:48:07.121 INFO:tasks.workunit.client.0.smithi102.stdout:6/441: stat d9/d25/c3b 0 2024-05-13T22:48:07.122 INFO:tasks.workunit.client.0.smithi102.stdout:2/419: read d7/d9/f89 [466879,37815] 0 2024-05-13T22:48:07.122 INFO:tasks.workunit.client.0.smithi102.stdout:6/442: truncate d9/db/f50 2103137 0 2024-05-13T22:48:07.130 INFO:tasks.workunit.client.0.smithi102.stdout:5/622: truncate de/d33/f83 680214 0 2024-05-13T22:48:07.131 INFO:tasks.workunit.client.0.smithi102.stdout:4/455: symlink d3/d19/d23/d4e/l9b 0 2024-05-13T22:48:07.138 INFO:tasks.workunit.client.0.smithi102.stdout:2/420: creat d7/d9/d10/d30/d53/f8e x:0 0 0 2024-05-13T22:48:07.138 INFO:tasks.workunit.client.0.smithi102.stdout:6/443: creat d9/d25/d60/d6a/f97 x:0 0 0 2024-05-13T22:48:07.138 INFO:tasks.workunit.client.0.smithi102.stdout:4/456: dwrite d3/d4/d6/de/d8d/f92 [0,4194304] 0 2024-05-13T22:48:07.140 INFO:tasks.workunit.client.0.smithi102.stdout:2/421: fsync d7/d15/d1e/f26 0 2024-05-13T22:48:07.145 INFO:tasks.workunit.client.0.smithi102.stdout:6/444: chown d9/db/l1f 1 1 2024-05-13T22:48:07.148 INFO:tasks.workunit.client.0.smithi102.stdout:7/316: creat d6/d13/d2c/f5c x:0 0 0 2024-05-13T22:48:07.150 INFO:tasks.workunit.client.0.smithi102.stdout:2/422: dwrite d7/d9/d14/f62 [0,4194304] 0 2024-05-13T22:48:07.153 INFO:tasks.workunit.client.0.smithi102.stdout:9/446: getdents d1/d6/d3b 0 2024-05-13T22:48:07.158 INFO:tasks.workunit.client.0.smithi102.stdout:6/445: dwrite d9/db/d14/d37/f7b [0,4194304] 0 2024-05-13T22:48:07.160 INFO:tasks.workunit.client.0.smithi102.stdout:4/457: symlink d3/l9c 0 2024-05-13T22:48:07.168 INFO:tasks.workunit.client.0.smithi102.stdout:8/345: truncate d2/d36/d49/d5d/f26 1784004 0 2024-05-13T22:48:07.171 INFO:tasks.workunit.client.0.smithi102.stdout:5/623: creat de/d25/fe8 x:0 0 0 2024-05-13T22:48:07.172 INFO:tasks.workunit.client.0.smithi102.stdout:8/346: dread d2/d7/d12/f23 [0,4194304] 0 2024-05-13T22:48:07.174 INFO:tasks.workunit.client.0.smithi102.stdout:9/447: symlink d1/d6/db/la4 0 2024-05-13T22:48:07.175 INFO:tasks.workunit.client.0.smithi102.stdout:9/448: write d1/d6/d45/f98 [1000928,52594] 0 2024-05-13T22:48:07.176 INFO:tasks.workunit.client.0.smithi102.stdout:6/446: symlink d9/db/l98 0 2024-05-13T22:48:07.176 INFO:tasks.workunit.client.0.smithi102.stdout:9/449: stat d1/d6/db/d4a/d64/c65 0 2024-05-13T22:48:07.178 INFO:tasks.workunit.client.0.smithi102.stdout:6/447: write d9/db/d14/d35/d47/f70 [586625,87799] 0 2024-05-13T22:48:07.179 INFO:tasks.workunit.client.0.smithi102.stdout:5/624: unlink de/d25/d2a/d4b/d4d/l6a 0 2024-05-13T22:48:07.180 INFO:tasks.workunit.client.0.smithi102.stdout:4/458: creat d3/d19/d23/d3a/f9d x:0 0 0 2024-05-13T22:48:07.187 INFO:tasks.workunit.client.0.smithi102.stdout:2/423: write d7/d9/d10/d25/d27/f45 [3595356,90421] 0 2024-05-13T22:48:07.190 INFO:tasks.workunit.client.0.smithi102.stdout:8/347: truncate d2/d7/d27/f4c 283819 0 2024-05-13T22:48:07.191 INFO:tasks.workunit.client.0.smithi102.stdout:4/459: dwrite d3/d19/d23/d3a/f83 [0,4194304] 0 2024-05-13T22:48:07.191 INFO:tasks.workunit.client.0.smithi102.stdout:2/424: chown d7/d9/d14/l3c 6658606 1 2024-05-13T22:48:07.192 INFO:tasks.workunit.client.0.smithi102.stdout:8/348: fsync d2/d36/f5a 0 2024-05-13T22:48:07.195 INFO:tasks.workunit.client.0.smithi102.stdout:2/425: stat d7/d9/d10/d30/d7f 0 2024-05-13T22:48:07.196 INFO:tasks.workunit.client.0.smithi102.stdout:5/625: dread de/d33/d34/f4a [0,4194304] 0 2024-05-13T22:48:07.200 INFO:tasks.workunit.client.0.smithi102.stdout:6/448: dread d9/db/d14/d37/f48 [0,4194304] 0 2024-05-13T22:48:07.202 INFO:tasks.workunit.client.0.smithi102.stdout:9/450: dread d1/d6/f1d [0,4194304] 0 2024-05-13T22:48:07.204 INFO:tasks.workunit.client.0.smithi102.stdout:6/449: dread d9/db/d2f/f62 [0,4194304] 0 2024-05-13T22:48:07.206 INFO:tasks.workunit.client.0.smithi102.stdout:6/450: chown d9/db/d14/f8d 13 1 2024-05-13T22:48:07.210 INFO:tasks.workunit.client.0.smithi102.stdout:4/460: mknod d3/d19/d23/d4e/d68/d85/c9e 0 2024-05-13T22:48:07.215 INFO:tasks.workunit.client.0.smithi102.stdout:2/426: rename d7/d9/d10/d30/c74 to d7/d9/d7c/c8f 0 2024-05-13T22:48:07.224 INFO:tasks.workunit.client.0.smithi102.stdout:6/451: dread d9/fa [0,4194304] 0 2024-05-13T22:48:07.226 INFO:tasks.workunit.client.0.smithi102.stdout:2/427: creat d7/d9/d10/d1b/f90 x:0 0 0 2024-05-13T22:48:07.227 INFO:tasks.workunit.client.0.smithi102.stdout:6/452: dread d9/d45/d56/d5d/f87 [0,4194304] 0 2024-05-13T22:48:07.227 INFO:tasks.workunit.client.0.smithi102.stdout:2/428: chown d7/d15/d1e/d87 1526103 1 2024-05-13T22:48:07.229 INFO:tasks.workunit.client.0.smithi102.stdout:5/626: symlink de/d33/le9 0 2024-05-13T22:48:07.234 INFO:tasks.workunit.client.0.smithi102.stdout:4/461: fsync d3/d22/f74 0 2024-05-13T22:48:07.238 INFO:tasks.workunit.client.0.smithi102.stdout:6/453: mknod d9/db/d2f/d59/c99 0 2024-05-13T22:48:07.239 INFO:tasks.workunit.client.0.smithi102.stdout:6/454: read d9/db/f3c [611797,123967] 0 2024-05-13T22:48:07.242 INFO:tasks.workunit.client.0.smithi102.stdout:8/349: getdents d2/d7/d25/d41/d50/d40 0 2024-05-13T22:48:07.243 INFO:tasks.workunit.client.0.smithi102.stdout:8/350: write d2/d7/d25/d41/d50/f35 [5154966,19805] 0 2024-05-13T22:48:07.247 INFO:tasks.workunit.client.0.smithi102.stdout:4/462: rmdir d3/d19/d23/d4e/d68 39 2024-05-13T22:48:07.254 INFO:tasks.workunit.client.0.smithi102.stdout:6/455: mknod d9/d25/d5a/c9a 0 2024-05-13T22:48:07.256 INFO:tasks.workunit.client.0.smithi102.stdout:2/429: mknod d7/d9/d10/d30/d7f/c91 0 2024-05-13T22:48:07.257 INFO:tasks.workunit.client.0.smithi102.stdout:2/430: write d7/d9/d10/d30/d53/f8e [165456,127930] 0 2024-05-13T22:48:07.258 INFO:tasks.workunit.client.0.smithi102.stdout:2/431: fsync d7/d9/d11/f55 0 2024-05-13T22:48:07.260 INFO:tasks.workunit.client.0.smithi102.stdout:8/351: creat d2/d36/d49/f6a x:0 0 0 2024-05-13T22:48:07.266 INFO:tasks.workunit.client.0.smithi102.stdout:6/456: rmdir d9/db/d2f/d59 39 2024-05-13T22:48:07.270 INFO:tasks.workunit.client.0.smithi102.stdout:2/432: creat d7/d9/d14/d61/f92 x:0 0 0 2024-05-13T22:48:07.289 INFO:tasks.workunit.client.0.smithi102.stdout:4/463: creat d3/d19/d23/d7e/f9f x:0 0 0 2024-05-13T22:48:07.289 INFO:tasks.workunit.client.0.smithi102.stdout:2/433: chown d7/d9/d10/d25 178234746 1 2024-05-13T22:48:07.291 INFO:tasks.workunit.client.0.smithi102.stdout:6/457: read d9/db/f3c [604553,338] 0 2024-05-13T22:48:07.293 INFO:tasks.workunit.client.0.smithi102.stdout:2/434: creat d7/d9/d14/d61/f93 x:0 0 0 2024-05-13T22:48:07.298 INFO:tasks.workunit.client.0.smithi102.stdout:4/464: creat d3/d19/d23/d39/fa0 x:0 0 0 2024-05-13T22:48:07.300 INFO:tasks.workunit.client.0.smithi102.stdout:8/352: link d2/l1b d2/d7/d25/d41/d50/l6b 0 2024-05-13T22:48:07.303 INFO:tasks.workunit.client.0.smithi102.stdout:8/353: dread d2/d36/f5a [0,4194304] 0 2024-05-13T22:48:07.303 INFO:tasks.workunit.client.0.smithi102.stdout:6/458: creat d9/d45/d56/f9b x:0 0 0 2024-05-13T22:48:07.311 INFO:tasks.workunit.client.0.smithi102.stdout:6/459: unlink d9/db/l8c 0 2024-05-13T22:48:07.314 INFO:tasks.workunit.client.0.smithi102.stdout:8/354: truncate d2/d7/d12/f23 2033189 0 2024-05-13T22:48:07.314 INFO:tasks.workunit.client.0.smithi102.stdout:4/465: dwrite d3/d4/f2e [0,4194304] 0 2024-05-13T22:48:07.316 INFO:tasks.workunit.client.0.smithi102.stdout:2/435: mknod d7/c94 0 2024-05-13T22:48:07.321 INFO:tasks.workunit.client.0.smithi102.stdout:4/466: creat d3/d4/d6/de/d1f/d41/d6a/fa1 x:0 0 0 2024-05-13T22:48:07.322 INFO:tasks.workunit.client.0.smithi102.stdout:2/436: mknod d7/d9/d10/d30/c95 0 2024-05-13T22:48:07.324 INFO:tasks.workunit.client.0.smithi102.stdout:6/460: symlink d9/db/d14/d37/d5f/l9c 0 2024-05-13T22:48:07.325 INFO:tasks.workunit.client.0.smithi102.stdout:8/355: dwrite d2/d7/d25/d41/d50/d40/f5b [0,4194304] 0 2024-05-13T22:48:07.327 INFO:tasks.workunit.client.0.smithi102.stdout:6/461: dread - d9/d45/d56/f9b zero size 2024-05-13T22:48:07.329 INFO:tasks.workunit.client.0.smithi102.stdout:6/462: fsync d9/d25/d60/d6a/f97 0 2024-05-13T22:48:07.330 INFO:tasks.workunit.client.0.smithi102.stdout:2/437: write d7/d15/d67/f70 [3477317,109467] 0 2024-05-13T22:48:07.332 INFO:tasks.workunit.client.0.smithi102.stdout:2/438: write d7/d15/f19 [5591298,61227] 0 2024-05-13T22:48:07.333 INFO:tasks.workunit.client.0.smithi102.stdout:2/439: chown d7/l4f 1 1 2024-05-13T22:48:07.334 INFO:tasks.workunit.client.0.smithi102.stdout:2/440: readlink d7/l4f 0 2024-05-13T22:48:07.335 INFO:tasks.workunit.client.0.smithi102.stdout:6/463: creat d9/db/d14/d35/f9d x:0 0 0 2024-05-13T22:48:07.337 INFO:tasks.workunit.client.0.smithi102.stdout:8/356: link d2/d7/d27/f31 d2/d7/d25/d41/d50/d40/f6c 0 2024-05-13T22:48:07.337 INFO:tasks.workunit.client.0.smithi102.stdout:8/357: readlink d2/d7/d12/l38 0 2024-05-13T22:48:07.341 INFO:tasks.workunit.client.0.smithi102.stdout:6/464: creat d9/db/d14/d37/d5f/f9e x:0 0 0 2024-05-13T22:48:07.350 INFO:tasks.workunit.client.0.smithi102.stdout:4/467: dread d3/d19/d23/d3a/f4a [0,4194304] 0 2024-05-13T22:48:07.350 INFO:tasks.workunit.client.0.smithi102.stdout:4/468: stat d3/d4/d6/de/f3f 0 2024-05-13T22:48:07.351 INFO:tasks.workunit.client.0.smithi102.stdout:8/358: creat d2/f6d x:0 0 0 2024-05-13T22:48:07.355 INFO:tasks.workunit.client.0.smithi102.stdout:4/469: rename d3/d4 to d3/d4/d5f/da2 22 2024-05-13T22:48:07.357 INFO:tasks.workunit.client.0.smithi102.stdout:8/359: creat d2/d7/d25/f6e x:0 0 0 2024-05-13T22:48:07.358 INFO:tasks.workunit.client.0.smithi102.stdout:6/465: getdents d9/d25/d5a 0 2024-05-13T22:48:07.362 INFO:tasks.workunit.client.0.smithi102.stdout:8/360: rmdir d2/d7/d12 39 2024-05-13T22:48:07.364 INFO:tasks.workunit.client.0.smithi102.stdout:6/466: creat d9/d45/d56/d5d/f9f x:0 0 0 2024-05-13T22:48:07.365 INFO:tasks.workunit.client.0.smithi102.stdout:6/467: truncate d9/d25/f30 431376 0 2024-05-13T22:48:07.367 INFO:tasks.workunit.client.0.smithi102.stdout:8/361: mkdir d2/d7/d25/d41/d6f 0 2024-05-13T22:48:07.369 INFO:tasks.workunit.client.0.smithi102.stdout:6/468: mkdir d9/d45/da0 0 2024-05-13T22:48:07.370 INFO:tasks.workunit.client.0.smithi102.stdout:6/469: fdatasync d9/db/d14/d37/f43 0 2024-05-13T22:48:07.372 INFO:tasks.workunit.client.0.smithi102.stdout:6/470: chown d9/db/d14/d54/c5b 2036 1 2024-05-13T22:48:07.373 INFO:tasks.workunit.client.0.smithi102.stdout:6/471: stat d9/db/d14/d35/d47 0 2024-05-13T22:48:07.374 INFO:tasks.workunit.client.0.smithi102.stdout:8/362: dwrite d2/d7/d25/d41/d50/d40/f5b [0,4194304] 0 2024-05-13T22:48:07.375 INFO:tasks.workunit.client.0.smithi102.stdout:8/363: fdatasync d2/d36/d49/f60 0 2024-05-13T22:48:07.377 INFO:tasks.workunit.client.0.smithi102.stdout:8/364: chown d2/d36/d49/d5d/f67 3775031 1 2024-05-13T22:48:07.378 INFO:tasks.workunit.client.0.smithi102.stdout:8/365: write d2/d7/d25/d41/d50/d40/f5b [1326403,60085] 0 2024-05-13T22:48:07.385 INFO:tasks.workunit.client.0.smithi102.stdout:8/366: mknod d2/d7/d25/d41/c70 0 2024-05-13T22:48:07.386 INFO:tasks.workunit.client.0.smithi102.stdout:8/367: rename d2/d7 to d2/d7/d27/d64/d71 22 2024-05-13T22:48:07.388 INFO:tasks.workunit.client.0.smithi102.stdout:6/472: creat d9/db/fa1 x:0 0 0 2024-05-13T22:48:07.392 INFO:tasks.workunit.client.0.smithi102.stdout:6/473: mknod d9/d25/d60/d6a/ca2 0 2024-05-13T22:48:07.392 INFO:tasks.workunit.client.0.smithi102.stdout:6/474: dread - d9/d25/d5a/f6d zero size 2024-05-13T22:48:07.399 INFO:tasks.workunit.client.0.smithi102.stdout:8/368: dread d2/d7/d25/d41/d50/f34 [0,4194304] 0 2024-05-13T22:48:07.400 INFO:tasks.workunit.client.0.smithi102.stdout:6/475: truncate d9/db/f7d 889971 0 2024-05-13T22:48:07.400 INFO:tasks.workunit.client.0.smithi102.stdout:8/369: chown d2/c5e 16272881 1 2024-05-13T22:48:07.410 INFO:tasks.workunit.client.0.smithi102.stdout:6/476: dread d9/db/d2f/f62 [0,4194304] 0 2024-05-13T22:48:07.417 INFO:tasks.workunit.client.0.smithi102.stdout:6/477: mkdir d9/da3 0 2024-05-13T22:48:07.425 INFO:tasks.workunit.client.0.smithi102.stdout:2/441: write d7/d15/d1e/f5c [2708939,13092] 0 2024-05-13T22:48:07.437 INFO:tasks.workunit.client.0.smithi102.stdout:4/470: write d3/d4/d6/de/d1f/d41/d6a/f7a [94853,116691] 0 2024-05-13T22:48:07.438 INFO:tasks.workunit.client.0.smithi102.stdout:4/471: dread - d3/d4/d6/de/d1f/f67 zero size 2024-05-13T22:48:07.439 INFO:tasks.workunit.client.0.smithi102.stdout:4/472: read d3/d19/d23/f3d [2467597,104206] 0 2024-05-13T22:48:07.449 INFO:tasks.workunit.client.0.smithi102.stdout:6/478: creat d9/db/d14/d37/d5f/fa4 x:0 0 0 2024-05-13T22:48:07.450 INFO:tasks.workunit.client.0.smithi102.stdout:6/479: chown d9/db/d14/d37/f65 8 1 2024-05-13T22:48:07.455 INFO:tasks.workunit.client.0.smithi102.stdout:2/442: dread d7/d9/d10/d25/d27/f5b [0,4194304] 0 2024-05-13T22:48:07.456 INFO:tasks.workunit.client.0.smithi102.stdout:4/473: creat d3/d4/d5f/fa3 x:0 0 0 2024-05-13T22:48:07.457 INFO:tasks.workunit.client.0.smithi102.stdout:6/480: dwrite d9/db/d14/d37/f43 [0,4194304] 0 2024-05-13T22:48:07.459 INFO:tasks.workunit.client.0.smithi102.stdout:6/481: chown d9/d25/d60/d6a 4318 1 2024-05-13T22:48:07.460 INFO:tasks.workunit.client.0.smithi102.stdout:1/334: sync 2024-05-13T22:48:07.462 INFO:tasks.workunit.client.0.smithi102.stdout:4/474: mkdir d3/d19/d23/d3a/da4 0 2024-05-13T22:48:07.464 INFO:tasks.workunit.client.0.smithi102.stdout:6/482: truncate d9/db/d14/d35/f9d 683682 0 2024-05-13T22:48:07.464 INFO:tasks.workunit.client.0.smithi102.stdout:2/443: dwrite d7/d9/d14/d61/f92 [0,4194304] 0 2024-05-13T22:48:07.471 INFO:tasks.workunit.client.0.smithi102.stdout:6/483: truncate d9/d45/d56/d5d/f82 305562 0 2024-05-13T22:48:07.472 INFO:tasks.workunit.client.0.smithi102.stdout:1/335: dwrite d2/d4/f51 [0,4194304] 0 2024-05-13T22:48:07.476 INFO:tasks.workunit.client.0.smithi102.stdout:2/444: mknod d7/d9/d10/d1b/c96 0 2024-05-13T22:48:07.480 INFO:tasks.workunit.client.0.smithi102.stdout:6/484: dwrite d9/d25/d60/d6a/f97 [0,4194304] 0 2024-05-13T22:48:07.482 INFO:tasks.workunit.client.0.smithi102.stdout:6/485: chown f0 5514933 1 2024-05-13T22:48:07.482 INFO:tasks.workunit.client.0.smithi102.stdout:4/475: rmdir d3/d19/d23/d4e/d68/d85 39 2024-05-13T22:48:07.489 INFO:tasks.workunit.client.0.smithi102.stdout:2/445: unlink d7/d9/d10/d1b/l50 0 2024-05-13T22:48:07.490 INFO:tasks.workunit.client.0.smithi102.stdout:4/476: dwrite d3/d22/f5b [0,4194304] 0 2024-05-13T22:48:07.491 INFO:tasks.workunit.client.0.smithi102.stdout:3/547: sync 2024-05-13T22:48:07.491 INFO:tasks.workunit.client.0.smithi102.stdout:9/451: sync 2024-05-13T22:48:07.491 INFO:tasks.workunit.client.0.smithi102.stdout:0/502: sync 2024-05-13T22:48:07.491 INFO:tasks.workunit.client.0.smithi102.stdout:7/317: sync 2024-05-13T22:48:07.491 INFO:tasks.workunit.client.0.smithi102.stdout:5/627: sync 2024-05-13T22:48:07.494 INFO:tasks.workunit.client.0.smithi102.stdout:4/477: stat d3/d19/d23/d3a/f9d 0 2024-05-13T22:48:07.494 INFO:tasks.workunit.client.0.smithi102.stdout:5/628: chown de/f14 3214199 1 2024-05-13T22:48:07.495 INFO:tasks.workunit.client.0.smithi102.stdout:9/452: write d1/d5d/d6c/f86 [600295,83537] 0 2024-05-13T22:48:07.496 INFO:tasks.workunit.client.0.smithi102.stdout:6/486: unlink f3 0 2024-05-13T22:48:07.508 INFO:tasks.workunit.client.0.smithi102.stdout:4/478: chown d3/d4/d6/c84 6 1 2024-05-13T22:48:07.511 INFO:tasks.workunit.client.0.smithi102.stdout:0/503: truncate d1/de/d32/d24/d2d/d3e/f73 215464 0 2024-05-13T22:48:07.516 INFO:tasks.workunit.client.0.smithi102.stdout:6/487: symlink d9/db/d14/d35/la5 0 2024-05-13T22:48:07.517 INFO:tasks.workunit.client.0.smithi102.stdout:6/488: chown d9/d25/f30 19722 1 2024-05-13T22:48:07.517 INFO:tasks.workunit.client.0.smithi102.stdout:9/453: read d1/d6/d45/d2d/f3f [2050378,9118] 0 2024-05-13T22:48:07.520 INFO:tasks.workunit.client.0.smithi102.stdout:3/548: rename de/d10/d2a/d36/f83 to de/d10/d50/d63/d98/fc3 0 2024-05-13T22:48:07.520 INFO:tasks.workunit.client.0.smithi102.stdout:6/489: dread d9/db/d14/d35/f9d [0,4194304] 0 2024-05-13T22:48:07.524 INFO:tasks.workunit.client.0.smithi102.stdout:4/479: mkdir d3/d19/d23/d39/d5d/da5 0 2024-05-13T22:48:07.530 INFO:tasks.workunit.client.0.smithi102.stdout:9/454: unlink d1/f94 0 2024-05-13T22:48:07.533 INFO:tasks.workunit.client.0.smithi102.stdout:3/549: creat de/d10/d50/d63/d98/fc4 x:0 0 0 2024-05-13T22:48:07.535 INFO:tasks.workunit.client.0.smithi102.stdout:7/318: creat d6/d13/d2c/f5d x:0 0 0 2024-05-13T22:48:07.536 INFO:tasks.workunit.client.0.smithi102.stdout:7/319: dread - d6/d13/de/f59 zero size 2024-05-13T22:48:07.537 INFO:tasks.workunit.client.0.smithi102.stdout:7/320: chown d6/d13/d2c/d32/d49 240248938 1 2024-05-13T22:48:07.540 INFO:tasks.workunit.client.0.smithi102.stdout:0/504: mknod d1/d4d/ca0 0 2024-05-13T22:48:07.547 INFO:tasks.workunit.client.0.smithi102.stdout:3/550: rename de/d10/d2a/d33/l5c to de/d10/d2a/d8c/lc5 0 2024-05-13T22:48:07.549 INFO:tasks.workunit.client.0.smithi102.stdout:5/629: link f6 de/d25/d43/d7a/fea 0 2024-05-13T22:48:07.551 INFO:tasks.workunit.client.0.smithi102.stdout:5/630: truncate de/d25/f26 530844 0 2024-05-13T22:48:07.551 INFO:tasks.workunit.client.0.smithi102.stdout:7/321: creat d6/d15/f5e x:0 0 0 2024-05-13T22:48:07.553 INFO:tasks.workunit.client.0.smithi102.stdout:4/480: rmdir d3/d6b/d81/d7d 39 2024-05-13T22:48:07.554 INFO:tasks.workunit.client.0.smithi102.stdout:4/481: write d3/d19/d23/d39/d5d/f98 [631056,22873] 0 2024-05-13T22:48:07.556 INFO:tasks.workunit.client.0.smithi102.stdout:4/482: write d3/d19/d23/d4e/d68/f94 [1502534,24495] 0 2024-05-13T22:48:07.556 INFO:tasks.workunit.client.0.smithi102.stdout:0/505: mknod d1/d68/ca1 0 2024-05-13T22:48:07.559 INFO:tasks.workunit.client.0.smithi102.stdout:6/490: creat d9/db/d14/fa6 x:0 0 0 2024-05-13T22:48:07.562 INFO:tasks.workunit.client.0.smithi102.stdout:5/631: creat de/d25/d59/d7f/d86/da5/feb x:0 0 0 2024-05-13T22:48:07.567 INFO:tasks.workunit.client.0.smithi102.stdout:9/455: creat d1/d6/fa5 x:0 0 0 2024-05-13T22:48:07.567 INFO:tasks.workunit.client.0.smithi102.stdout:3/551: dread de/d10/d2a/d36/f58 [0,4194304] 0 2024-05-13T22:48:07.573 INFO:tasks.workunit.client.0.smithi102.stdout:7/322: creat d6/d13/d58/d20/d4b/f5f x:0 0 0 2024-05-13T22:48:07.575 INFO:tasks.workunit.client.0.smithi102.stdout:5/632: creat de/d25/d2c/dd5/fec x:0 0 0 2024-05-13T22:48:07.577 INFO:tasks.workunit.client.0.smithi102.stdout:0/506: dread d1/de/d32/d24/f30 [0,4194304] 0 2024-05-13T22:48:07.578 INFO:tasks.workunit.client.0.smithi102.stdout:3/552: creat de/d10/d50/fc6 x:0 0 0 2024-05-13T22:48:07.580 INFO:tasks.workunit.client.0.smithi102.stdout:4/483: mkdir d3/d19/d23/d4e/da6 0 2024-05-13T22:48:07.582 INFO:tasks.workunit.client.0.smithi102.stdout:6/491: rename d9/db/d14/l55 to d9/da3/la7 0 2024-05-13T22:48:07.587 INFO:tasks.workunit.client.0.smithi102.stdout:9/456: creat d1/d6/d45/d3e/d79/fa6 x:0 0 0 2024-05-13T22:48:07.588 INFO:tasks.workunit.client.0.smithi102.stdout:4/484: dwrite d3/d4/d6/f18 [4194304,4194304] 0 2024-05-13T22:48:07.590 INFO:tasks.workunit.client.0.smithi102.stdout:5/633: mkdir de/d25/d2a/d4b/d4d/d50/d6c/ded 0 2024-05-13T22:48:07.592 INFO:tasks.workunit.client.0.smithi102.stdout:0/507: fdatasync d1/f60 0 2024-05-13T22:48:07.592 INFO:tasks.workunit.client.0.smithi102.stdout:3/553: truncate de/d10/d34/f38 150797 0 2024-05-13T22:48:07.597 INFO:tasks.workunit.client.0.smithi102.stdout:4/485: dwrite d3/d4/d6/f1c [0,4194304] 0 2024-05-13T22:48:07.601 INFO:tasks.workunit.client.0.smithi102.stdout:4/486: truncate d3/d19/d23/d4e/d68/f94 2253746 0 2024-05-13T22:48:07.602 INFO:tasks.workunit.client.0.smithi102.stdout:4/487: chown d3/d4/d6/de/d1f/d41/d6a 436 1 2024-05-13T22:48:07.603 INFO:tasks.workunit.client.0.smithi102.stdout:0/508: dwrite d1/de/d32/d63/f96 [0,4194304] 0 2024-05-13T22:48:07.605 INFO:tasks.workunit.client.0.smithi102.stdout:6/492: rmdir d9/d45/d56 39 2024-05-13T22:48:07.605 INFO:tasks.workunit.client.0.smithi102.stdout:6/493: fsync f0 0 2024-05-13T22:48:07.611 INFO:tasks.workunit.client.0.smithi102.stdout:9/457: mknod d1/d6b/ca7 0 2024-05-13T22:48:07.612 INFO:tasks.workunit.client.0.smithi102.stdout:2/446: truncate d7/d9/d10/d25/d27/f3b 1027767 0 2024-05-13T22:48:07.616 INFO:tasks.workunit.client.0.smithi102.stdout:1/336: dwrite d2/f46 [0,4194304] 0 2024-05-13T22:48:07.625 INFO:tasks.workunit.client.0.smithi102.stdout:0/509: symlink d1/d4d/d79/la2 0 2024-05-13T22:48:07.629 INFO:tasks.workunit.client.0.smithi102.stdout:6/494: creat d9/db/d14/d35/d47/fa8 x:0 0 0 2024-05-13T22:48:07.630 INFO:tasks.workunit.client.0.smithi102.stdout:4/488: dread d3/d4/d6/de/f15 [0,4194304] 0 2024-05-13T22:48:07.631 INFO:tasks.workunit.client.0.smithi102.stdout:6/495: fsync d9/db/d14/d35/f94 0 2024-05-13T22:48:07.633 INFO:tasks.workunit.client.0.smithi102.stdout:9/458: mknod d1/d5d/d6c/ca8 0 2024-05-13T22:48:07.635 INFO:tasks.workunit.client.0.smithi102.stdout:2/447: dread d7/d15/d67/f70 [0,4194304] 0 2024-05-13T22:48:07.638 INFO:tasks.workunit.client.0.smithi102.stdout:0/510: dwrite d1/de/d32/d24/d2d/f6b [4194304,4194304] 0 2024-05-13T22:48:07.639 INFO:tasks.workunit.client.0.smithi102.stdout:5/634: mkdir de/d25/d59/d7f/d86/da5/ddb/dee 0 2024-05-13T22:48:07.640 INFO:tasks.workunit.client.0.smithi102.stdout:5/635: dread - de/d25/d2c/dd5/fec zero size 2024-05-13T22:48:07.641 INFO:tasks.workunit.client.0.smithi102.stdout:0/511: chown d1/de/d32/d63/f98 20 1 2024-05-13T22:48:07.644 INFO:tasks.workunit.client.0.smithi102.stdout:3/554: rename de/d1c/l23 to de/d10/d50/d63/da2/lc7 0 2024-05-13T22:48:07.650 INFO:tasks.workunit.client.0.smithi102.stdout:6/496: dread - d9/db/d14/d35/d47/f5c zero size 2024-05-13T22:48:07.651 INFO:tasks.workunit.client.0.smithi102.stdout:9/459: creat d1/d6/db/d4a/fa9 x:0 0 0 2024-05-13T22:48:07.652 INFO:tasks.workunit.client.0.smithi102.stdout:6/497: chown d9/db/fa1 17803 1 2024-05-13T22:48:07.655 INFO:tasks.workunit.client.0.smithi102.stdout:6/498: write d9/db/d14/d35/d47/f70 [1063543,62349] 0 2024-05-13T22:48:07.656 INFO:tasks.workunit.client.0.smithi102.stdout:8/370: dwrite d2/d7/d12/f23 [0,4194304] 0 2024-05-13T22:48:07.656 INFO:tasks.workunit.client.0.smithi102.stdout:1/337: mknod d2/d4/d26/d40/c6c 0 2024-05-13T22:48:07.656 INFO:tasks.workunit.client.0.smithi102.stdout:6/499: stat d9/f66 0 2024-05-13T22:48:07.659 INFO:tasks.workunit.client.0.smithi102.stdout:1/338: write d2/d4/f20 [877202,6470] 0 2024-05-13T22:48:07.662 INFO:tasks.workunit.client.0.smithi102.stdout:1/339: chown d2/d4/d26/d56/f61 3 1 2024-05-13T22:48:07.665 INFO:tasks.workunit.client.0.smithi102.stdout:9/460: symlink d1/d6/db/d77/laa 0 2024-05-13T22:48:07.666 INFO:tasks.workunit.client.0.smithi102.stdout:9/461: chown d1/d6/db/d30/c8b 6 1 2024-05-13T22:48:07.667 INFO:tasks.workunit.client.0.smithi102.stdout:9/462: read - d1/d6/db/d30/d8a/f99 zero size 2024-05-13T22:48:07.674 INFO:tasks.workunit.client.0.smithi102.stdout:8/371: truncate d2/d7/f9 5047431 0 2024-05-13T22:48:07.677 INFO:tasks.workunit.client.0.smithi102.stdout:6/500: rename d9/f66 to d9/db/d14/d54/fa9 0 2024-05-13T22:48:07.679 INFO:tasks.workunit.client.0.smithi102.stdout:6/501: fdatasync d9/d25/f58 0 2024-05-13T22:48:07.680 INFO:tasks.workunit.client.0.smithi102.stdout:7/323: dwrite d6/d15/d4c/f5a [0,4194304] 0 2024-05-13T22:48:07.682 INFO:tasks.workunit.client.0.smithi102.stdout:7/324: chown d6/d15/f3b 48284791 1 2024-05-13T22:48:07.682 INFO:tasks.workunit.client.0.smithi102.stdout:7/325: fdatasync d6/d13/d2c/f3a 0 2024-05-13T22:48:07.683 INFO:tasks.workunit.client.0.smithi102.stdout:7/326: dread - d6/d13/d58/f52 zero size 2024-05-13T22:48:07.683 INFO:tasks.workunit.client.0.smithi102.stdout:1/340: truncate d2/d4/d5/f15 6900318 0 2024-05-13T22:48:07.694 INFO:tasks.workunit.client.0.smithi102.stdout:6/502: creat d9/d25/faa x:0 0 0 2024-05-13T22:48:07.696 INFO:tasks.workunit.client.0.smithi102.stdout:0/512: creat d1/de/d32/d24/d2d/d38/fa3 x:0 0 0 2024-05-13T22:48:07.697 INFO:tasks.workunit.client.0.smithi102.stdout:0/513: dread d1/f60 [0,4194304] 0 2024-05-13T22:48:07.699 INFO:tasks.workunit.client.0.smithi102.stdout:7/327: rmdir d6/d13/d2c/d32/d49 39 2024-05-13T22:48:07.701 INFO:tasks.workunit.client.0.smithi102.stdout:7/328: truncate d6/d13/de/f59 667906 0 2024-05-13T22:48:07.705 INFO:tasks.workunit.client.0.smithi102.stdout:8/372: link d2/d36/d49/d5d/f51 d2/d36/d49/d5d/f72 0 2024-05-13T22:48:07.709 INFO:tasks.workunit.client.0.smithi102.stdout:5/636: write de/d25/d2a/d4b/f70 [126861,28239] 0 2024-05-13T22:48:07.709 INFO:tasks.workunit.client.0.smithi102.stdout:7/329: dwrite d6/d13/d2c/f5c [0,4194304] 0 2024-05-13T22:48:07.709 INFO:tasks.workunit.client.0.smithi102.stdout:9/463: dread d1/d6/f60 [0,4194304] 0 2024-05-13T22:48:07.709 INFO:tasks.workunit.client.0.smithi102.stdout:0/514: mknod d1/de/d42/ca4 0 2024-05-13T22:48:07.711 INFO:tasks.workunit.client.0.smithi102.stdout:6/503: dread d9/d45/d56/d5d/f82 [0,4194304] 0 2024-05-13T22:48:07.711 INFO:tasks.workunit.client.0.smithi102.stdout:4/489: getdents d3/d6b/d81/d7d 0 2024-05-13T22:48:07.714 INFO:tasks.workunit.client.0.smithi102.stdout:0/515: write d1/f82 [530119,130422] 0 2024-05-13T22:48:07.716 INFO:tasks.workunit.client.0.smithi102.stdout:0/516: chown d1/d68/ca1 183624 1 2024-05-13T22:48:07.719 INFO:tasks.workunit.client.0.smithi102.stdout:5/637: mkdir de/d25/d2a/def 0 2024-05-13T22:48:07.723 INFO:tasks.workunit.client.0.smithi102.stdout:5/638: chown de/d25/d59/d7f/d86 41 1 2024-05-13T22:48:07.723 INFO:tasks.workunit.client.0.smithi102.stdout:9/464: dread - d1/d6/f7c zero size 2024-05-13T22:48:07.724 INFO:tasks.workunit.client.0.smithi102.stdout:7/330: dwrite d6/d13/d2c/f3a [0,4194304] 0 2024-05-13T22:48:07.726 INFO:tasks.workunit.client.0.smithi102.stdout:2/448: dwrite d7/d9/f29 [0,4194304] 0 2024-05-13T22:48:07.726 INFO:tasks.workunit.client.0.smithi102.stdout:1/341: link d2/f46 d2/d4/d5/f6d 0 2024-05-13T22:48:07.727 INFO:tasks.workunit.client.0.smithi102.stdout:6/504: write d9/db/f77 [850228,46674] 0 2024-05-13T22:48:07.729 INFO:tasks.workunit.client.0.smithi102.stdout:9/465: fdatasync d1/d5d/f74 0 2024-05-13T22:48:07.730 INFO:tasks.workunit.client.0.smithi102.stdout:6/505: readlink d9/d25/d71/l84 0 2024-05-13T22:48:07.732 INFO:tasks.workunit.client.0.smithi102.stdout:6/506: chown d9/d25/d5a/f6d 393 1 2024-05-13T22:48:07.733 INFO:tasks.workunit.client.0.smithi102.stdout:4/490: rename d3/d6b/d81 to d3/d19/d23/d39/da7 0 2024-05-13T22:48:07.744 INFO:tasks.workunit.client.0.smithi102.stdout:7/331: unlink d6/d13/de/f29 0 2024-05-13T22:48:07.749 INFO:tasks.workunit.client.0.smithi102.stdout:9/466: unlink d1/d6/d21/f29 0 2024-05-13T22:48:07.754 INFO:tasks.workunit.client.0.smithi102.stdout:2/449: rename d7/d9/d10/d1b/f90 to d7/d9/d14/d31/f97 0 2024-05-13T22:48:07.758 INFO:tasks.workunit.client.0.smithi102.stdout:8/373: creat d2/f73 x:0 0 0 2024-05-13T22:48:07.759 INFO:tasks.workunit.client.0.smithi102.stdout:0/517: mkdir d1/de/d32/d24/d2d/d38/d3d/d81/da5 0 2024-05-13T22:48:07.760 INFO:tasks.workunit.client.0.smithi102.stdout:8/374: chown d2/d36/c63 1036520 1 2024-05-13T22:48:07.762 INFO:tasks.workunit.client.0.smithi102.stdout:7/332: dread d6/d22/f43 [0,4194304] 0 2024-05-13T22:48:07.764 INFO:tasks.workunit.client.0.smithi102.stdout:1/342: link d2/d4/d38/f68 d2/d4/d38/f6e 0 2024-05-13T22:48:07.767 INFO:tasks.workunit.client.0.smithi102.stdout:1/343: truncate d2/d4/d5/f6d 5168576 0 2024-05-13T22:48:07.767 INFO:tasks.workunit.client.0.smithi102.stdout:8/375: dwrite d2/d7/d12/f1c [4194304,4194304] 0 2024-05-13T22:48:07.773 INFO:tasks.workunit.client.0.smithi102.stdout:0/518: readlink d1/d4d/l84 0 2024-05-13T22:48:07.782 INFO:tasks.workunit.client.0.smithi102.stdout:7/333: dwrite d6/d15/f18 [0,4194304] 0 2024-05-13T22:48:07.785 INFO:tasks.workunit.client.0.smithi102.stdout:8/376: creat d2/d7/d25/f74 x:0 0 0 2024-05-13T22:48:07.786 INFO:tasks.workunit.client.0.smithi102.stdout:8/377: dread - d2/d36/d49/d5d/f72 zero size 2024-05-13T22:48:07.791 INFO:tasks.workunit.client.0.smithi102.stdout:5/639: getdents de/d25/d2a/d4b/dab 0 2024-05-13T22:48:07.793 INFO:tasks.workunit.client.0.smithi102.stdout:1/344: mknod d2/d10/d62/c6f 0 2024-05-13T22:48:07.796 INFO:tasks.workunit.client.0.smithi102.stdout:2/450: write d7/d9/d10/d25/f68 [111158,60082] 0 2024-05-13T22:48:07.797 INFO:tasks.workunit.client.0.smithi102.stdout:2/451: dread - d7/d9/d14/d31/f77 zero size 2024-05-13T22:48:07.799 INFO:tasks.workunit.client.0.smithi102.stdout:9/467: truncate d1/d6/d45/f67 1758833 0 2024-05-13T22:48:07.801 INFO:tasks.workunit.client.0.smithi102.stdout:4/491: getdents d3/d6b 0 2024-05-13T22:48:07.802 INFO:tasks.workunit.client.0.smithi102.stdout:6/507: write d9/db/d14/d37/f57 [271613,15172] 0 2024-05-13T22:48:07.806 INFO:tasks.workunit.client.0.smithi102.stdout:2/452: dwrite d7/d9/d10/f33 [0,4194304] 0 2024-05-13T22:48:07.811 INFO:tasks.workunit.client.0.smithi102.stdout:0/519: dwrite d1/de/d32/d24/d2d/f5f [0,4194304] 0 2024-05-13T22:48:07.813 INFO:tasks.workunit.client.0.smithi102.stdout:8/378: link d2/d36/d49/f60 d2/d7/d25/d41/f75 0 2024-05-13T22:48:07.816 INFO:tasks.workunit.client.0.smithi102.stdout:8/379: stat d2/d36/d49/d5d/c20 0 2024-05-13T22:48:07.816 INFO:tasks.workunit.client.0.smithi102.stdout:2/453: dwrite d7/d9/d10/d1b/f8a [0,4194304] 0 2024-05-13T22:48:07.819 INFO:tasks.workunit.client.0.smithi102.stdout:2/454: truncate d7/d9/d10/f37 5089284 0 2024-05-13T22:48:07.820 INFO:tasks.workunit.client.0.smithi102.stdout:4/492: symlink d3/d19/d23/d7e/la8 0 2024-05-13T22:48:07.820 INFO:tasks.workunit.client.0.smithi102.stdout:2/455: stat d7/d15/d67/d6a 0 2024-05-13T22:48:07.821 INFO:tasks.workunit.client.0.smithi102.stdout:5/640: write de/f14 [2047138,105148] 0 2024-05-13T22:48:07.825 INFO:tasks.workunit.client.0.smithi102.stdout:6/508: truncate d9/d45/d56/d5d/f87 141292 0 2024-05-13T22:48:07.829 INFO:tasks.workunit.client.0.smithi102.stdout:8/380: dwrite d2/d7/d12/f23 [0,4194304] 0 2024-05-13T22:48:07.830 INFO:tasks.workunit.client.0.smithi102.stdout:8/381: chown d2/d7/d27 164894 1 2024-05-13T22:48:07.833 INFO:tasks.workunit.client.0.smithi102.stdout:7/334: creat d6/d13/d2c/f60 x:0 0 0 2024-05-13T22:48:07.841 INFO:tasks.workunit.client.0.smithi102.stdout:5/641: rmdir de/d25/d59/d7f 39 2024-05-13T22:48:07.842 INFO:tasks.workunit.client.0.smithi102.stdout:6/509: mknod d9/db/d14/d35/d47/cab 0 2024-05-13T22:48:07.843 INFO:tasks.workunit.client.0.smithi102.stdout:6/510: readlink d9/db/d14/l8e 0 2024-05-13T22:48:07.843 INFO:tasks.workunit.client.0.smithi102.stdout:8/382: mkdir d2/d7/d12/d76 0 2024-05-13T22:48:07.843 INFO:tasks.workunit.client.0.smithi102.stdout:6/511: chown f7 27798 1 2024-05-13T22:48:07.844 INFO:tasks.workunit.client.0.smithi102.stdout:6/512: chown d9/d45/l3e 1740356 1 2024-05-13T22:48:07.846 INFO:tasks.workunit.client.0.smithi102.stdout:9/468: dread d1/d6/d3b/d5b/f62 [0,4194304] 0 2024-05-13T22:48:07.846 INFO:tasks.workunit.client.0.smithi102.stdout:0/520: mkdir d1/de/d32/da6 0 2024-05-13T22:48:07.848 INFO:tasks.workunit.client.0.smithi102.stdout:0/521: write d1/de/d32/f72 [922822,110700] 0 2024-05-13T22:48:07.848 INFO:tasks.workunit.client.0.smithi102.stdout:1/345: truncate d2/d4/d5/f6d 3940592 0 2024-05-13T22:48:07.849 INFO:tasks.workunit.client.0.smithi102.stdout:0/522: readlink d1/l99 0 2024-05-13T22:48:07.853 INFO:tasks.workunit.client.0.smithi102.stdout:4/493: mkdir d3/d19/d23/d39/da7/da9 0 2024-05-13T22:48:07.859 INFO:tasks.workunit.client.0.smithi102.stdout:9/469: mkdir d1/d6/d21/dab 0 2024-05-13T22:48:07.861 INFO:tasks.workunit.client.0.smithi102.stdout:7/335: rename d6/d13/d2c/l36 to d6/d13/l61 0 2024-05-13T22:48:07.865 INFO:tasks.workunit.client.0.smithi102.stdout:1/346: unlink d2/d4/d5/d19/d28/f3b 0 2024-05-13T22:48:07.866 INFO:tasks.workunit.client.0.smithi102.stdout:0/523: mkdir d1/de/d42/da7 0 2024-05-13T22:48:07.874 INFO:tasks.workunit.client.0.smithi102.stdout:6/513: symlink d9/d45/lac 0 2024-05-13T22:48:07.875 INFO:tasks.workunit.client.0.smithi102.stdout:9/470: mkdir d1/d6/d45/d2d/dac 0 2024-05-13T22:48:07.876 INFO:tasks.workunit.client.0.smithi102.stdout:8/383: dwrite d2/d7/d25/d41/d50/d40/f6c [0,4194304] 0 2024-05-13T22:48:07.878 INFO:tasks.workunit.client.0.smithi102.stdout:4/494: rename d3/d19/d23/d39/da7/f60 to d3/d4/d6/de/faa 0 2024-05-13T22:48:07.883 INFO:tasks.workunit.client.0.smithi102.stdout:7/336: mkdir d6/d13/d62 0 2024-05-13T22:48:07.887 INFO:tasks.workunit.client.0.smithi102.stdout:8/384: dwrite d2/d7/d25/f74 [0,4194304] 0 2024-05-13T22:48:07.887 INFO:tasks.workunit.client.0.smithi102.stdout:7/337: write d6/d13/d2c/f3a [4722023,74992] 0 2024-05-13T22:48:07.887 INFO:tasks.workunit.client.0.smithi102.stdout:9/471: dwrite d1/d6/db/d4a/d64/f9a [0,4194304] 0 2024-05-13T22:48:07.891 INFO:tasks.workunit.client.0.smithi102.stdout:2/456: getdents d7/d9 0 2024-05-13T22:48:07.891 INFO:tasks.workunit.client.0.smithi102.stdout:9/472: write d1/d6/db/d4a/d64/f9a [681437,128810] 0 2024-05-13T22:48:07.895 INFO:tasks.workunit.client.0.smithi102.stdout:6/514: mkdir d9/d25/d60/d6a/dad 0 2024-05-13T22:48:07.899 INFO:tasks.workunit.client.0.smithi102.stdout:7/338: dwrite d6/d13/d58/d20/f4a [0,4194304] 0 2024-05-13T22:48:07.904 INFO:tasks.workunit.client.0.smithi102.stdout:7/339: dread d6/d13/d2c/f3a [0,4194304] 0 2024-05-13T22:48:07.905 INFO:tasks.workunit.client.0.smithi102.stdout:8/385: creat d2/d7/d27/f77 x:0 0 0 2024-05-13T22:48:07.909 INFO:tasks.workunit.client.0.smithi102.stdout:2/457: mknod d7/d9/d7c/c98 0 2024-05-13T22:48:07.910 INFO:tasks.workunit.client.0.smithi102.stdout:2/458: fsync d7/d9/d14/d31/f64 0 2024-05-13T22:48:07.911 INFO:tasks.workunit.client.0.smithi102.stdout:5/642: getdents de/d25/d59/d7f/d86 0 2024-05-13T22:48:07.911 INFO:tasks.workunit.client.0.smithi102.stdout:2/459: chown d7/d15/c7d 3129 1 2024-05-13T22:48:07.916 INFO:tasks.workunit.client.0.smithi102.stdout:2/460: dread d7/d9/f29 [0,4194304] 0 2024-05-13T22:48:07.916 INFO:tasks.workunit.client.0.smithi102.stdout:6/515: mknod d9/da3/cae 0 2024-05-13T22:48:07.917 INFO:tasks.workunit.client.0.smithi102.stdout:2/461: chown d7/d9/d10/d25/l2a 349115673 1 2024-05-13T22:48:07.918 INFO:tasks.workunit.client.0.smithi102.stdout:2/462: stat d7/d9/d14/d61/l86 0 2024-05-13T22:48:07.918 INFO:tasks.workunit.client.0.smithi102.stdout:2/463: fdatasync d7/ff 0 2024-05-13T22:48:07.920 INFO:tasks.workunit.client.0.smithi102.stdout:4/495: mkdir d3/d22/dab 0 2024-05-13T22:48:07.924 INFO:tasks.workunit.client.0.smithi102.stdout:1/347: truncate d2/d4/d5/fa 4915728 0 2024-05-13T22:48:07.928 INFO:tasks.workunit.client.0.smithi102.stdout:9/473: rename d1/d6/f40 to d1/d6/d45/d3e/fad 0 2024-05-13T22:48:07.929 INFO:tasks.workunit.client.0.smithi102.stdout:6/516: dread - d9/db/d14/d37/d5f/fa4 zero size 2024-05-13T22:48:07.929 INFO:tasks.workunit.client.0.smithi102.stdout:9/474: write d1/d6/db/d4a/f85 [360423,108589] 0 2024-05-13T22:48:07.935 INFO:tasks.workunit.client.0.smithi102.stdout:7/340: symlink d6/d13/d2c/d32/l63 0 2024-05-13T22:48:07.936 INFO:tasks.workunit.client.0.smithi102.stdout:6/517: dread d9/db/d2f/f62 [0,4194304] 0 2024-05-13T22:48:07.939 INFO:tasks.workunit.client.0.smithi102.stdout:7/341: chown d6/l1f 19052524 1 2024-05-13T22:48:07.939 INFO:tasks.workunit.client.0.smithi102.stdout:9/475: dwrite d1/f16 [0,4194304] 0 2024-05-13T22:48:07.942 INFO:tasks.workunit.client.0.smithi102.stdout:4/496: dwrite d3/d19/d23/d3a/f4a [0,4194304] 0 2024-05-13T22:48:07.945 INFO:tasks.workunit.client.0.smithi102.stdout:4/497: write d3/d4/d6/de/d1f/f8c [1325055,83986] 0 2024-05-13T22:48:07.947 INFO:tasks.workunit.client.0.smithi102.stdout:4/498: dread - d3/d4/f93 zero size 2024-05-13T22:48:07.950 INFO:tasks.workunit.client.0.smithi102.stdout:5/643: creat de/d33/d34/ff0 x:0 0 0 2024-05-13T22:48:07.952 INFO:tasks.workunit.client.0.smithi102.stdout:1/348: truncate d2/d45/f58 351934 0 2024-05-13T22:48:07.952 INFO:tasks.workunit.client.0.smithi102.stdout:9/476: dread d1/d6/db/d30/f6a [0,4194304] 0 2024-05-13T22:48:07.956 INFO:tasks.workunit.client.0.smithi102.stdout:4/499: dwrite d3/d19/d23/d3a/f83 [0,4194304] 0 2024-05-13T22:48:07.964 INFO:tasks.workunit.client.0.smithi102.stdout:9/477: dwrite d1/d6/d21/d71/f9d [0,4194304] 0 2024-05-13T22:48:07.967 INFO:tasks.workunit.client.0.smithi102.stdout:6/518: creat d9/d25/d60/d6a/dad/faf x:0 0 0 2024-05-13T22:48:07.968 INFO:tasks.workunit.client.0.smithi102.stdout:7/342: unlink d6/d13/d2c/d32/d49/l53 0 2024-05-13T22:48:07.971 INFO:tasks.workunit.client.0.smithi102.stdout:2/464: getdents d7/d15/d67/d6a 0 2024-05-13T22:48:07.971 INFO:tasks.workunit.client.0.smithi102.stdout:7/343: stat d6/d13/d2c/d32 0 2024-05-13T22:48:07.976 INFO:tasks.workunit.client.0.smithi102.stdout:4/500: mkdir d3/d19/d23/d39/dac 0 2024-05-13T22:48:07.980 INFO:tasks.workunit.client.0.smithi102.stdout:9/478: rename d1/d6/d45/d3e/f53 to d1/d6/d45/d3e/d79/fae 0 2024-05-13T22:48:07.986 INFO:tasks.workunit.client.0.smithi102.stdout:4/501: truncate d3/d22/f74 1117103 0 2024-05-13T22:48:07.987 INFO:tasks.workunit.client.0.smithi102.stdout:9/479: dwrite d1/d6/fa5 [0,4194304] 0 2024-05-13T22:48:07.987 INFO:tasks.workunit.client.0.smithi102.stdout:6/519: unlink d9/db/d14/d37/c4f 0 2024-05-13T22:48:07.988 INFO:tasks.workunit.client.0.smithi102.stdout:5/644: write de/f1d [1684940,79482] 0 2024-05-13T22:48:07.991 INFO:tasks.workunit.client.0.smithi102.stdout:9/480: fdatasync d1/d6/db/d4a/fa9 0 2024-05-13T22:48:07.992 INFO:tasks.workunit.client.0.smithi102.stdout:1/349: getdents d2/d45 0 2024-05-13T22:48:07.994 INFO:tasks.workunit.client.0.smithi102.stdout:6/520: symlink d9/d25/d60/d6a/dad/lb0 0 2024-05-13T22:48:07.995 INFO:tasks.workunit.client.0.smithi102.stdout:7/344: link d6/d13/c10 d6/d13/d62/c64 0 2024-05-13T22:48:07.996 INFO:tasks.workunit.client.0.smithi102.stdout:5/645: symlink de/d25/d2a/d4b/d4d/d50/da3/lf1 0 2024-05-13T22:48:08.003 INFO:tasks.workunit.client.0.smithi102.stdout:4/502: mkdir d3/d19/dad 0 2024-05-13T22:48:08.007 INFO:tasks.workunit.client.0.smithi102.stdout:7/345: rmdir d6/d13/de 39 2024-05-13T22:48:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:5/646: unlink de/d25/d2c/dd5/fec 0 2024-05-13T22:48:08.009 INFO:tasks.workunit.client.0.smithi102.stdout:5/647: read - de/fc8 zero size 2024-05-13T22:48:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:5/648: mknod de/d25/d43/d7a/cf2 0 2024-05-13T22:48:08.017 INFO:tasks.workunit.client.0.smithi102.stdout:5/649: fsync de/fc8 0 2024-05-13T22:48:08.021 INFO:tasks.workunit.client.0.smithi102.stdout:9/481: write d1/d6/d45/f67 [1364124,76199] 0 2024-05-13T22:48:08.022 INFO:tasks.workunit.client.0.smithi102.stdout:7/346: fsync d6/d13/de/f12 0 2024-05-13T22:48:08.062 INFO:tasks.workunit.client.0.smithi102.stdout:5/650: symlink de/d25/d2c/dd5/lf3 0 2024-05-13T22:48:08.066 INFO:tasks.workunit.client.0.smithi102.stdout:9/482: creat d1/d6/db/d30/d8a/faf x:0 0 0 2024-05-13T22:48:08.066 INFO:tasks.workunit.client.0.smithi102.stdout:4/503: truncate d3/d4/d6/f1c 250444 0 2024-05-13T22:48:08.066 INFO:tasks.workunit.client.0.smithi102.stdout:5/651: fsync de/d25/d59/f32 0 2024-05-13T22:48:08.068 INFO:tasks.workunit.client.0.smithi102.stdout:1/350: dwrite d2/d4/d5/fa [4194304,4194304] 0 2024-05-13T22:48:08.069 INFO:tasks.workunit.client.0.smithi102.stdout:7/347: rmdir d6/d13/d58 39 2024-05-13T22:48:08.072 INFO:tasks.workunit.client.0.smithi102.stdout:6/521: dwrite d9/d45/d56/d5d/f87 [0,4194304] 0 2024-05-13T22:48:08.079 INFO:tasks.workunit.client.0.smithi102.stdout:9/483: dwrite d1/d6/db/d4a/d75/f82 [0,4194304] 0 2024-05-13T22:48:08.079 INFO:tasks.workunit.client.0.smithi102.stdout:1/351: mknod d2/d4/d5/d19/d28/d6a/c70 0 2024-05-13T22:48:08.080 INFO:tasks.workunit.client.0.smithi102.stdout:1/352: chown d2/f54 2941 1 2024-05-13T22:48:08.080 INFO:tasks.workunit.client.0.smithi102.stdout:9/484: chown d1/d6/d45/d3e/f56 1 1 2024-05-13T22:48:08.083 INFO:tasks.workunit.client.0.smithi102.stdout:6/522: creat d9/d25/fb1 x:0 0 0 2024-05-13T22:48:08.090 INFO:tasks.workunit.client.0.smithi102.stdout:6/523: dwrite d9/db/d14/f3f [4194304,4194304] 0 2024-05-13T22:48:08.097 INFO:tasks.workunit.client.0.smithi102.stdout:4/504: rename d3/d19/d23/d39/da7/d7d/f8f to d3/d19/fae 0 2024-05-13T22:48:08.098 INFO:tasks.workunit.client.0.smithi102.stdout:4/505: stat d3/d19/d23/d39/d77/f7c 0 2024-05-13T22:48:08.099 INFO:tasks.workunit.client.0.smithi102.stdout:1/353: unlink d2/d4/d26/l2c 0 2024-05-13T22:48:08.101 INFO:tasks.workunit.client.0.smithi102.stdout:9/485: mkdir d1/d6/d45/d2d/dac/db0 0 2024-05-13T22:48:08.103 INFO:tasks.workunit.client.0.smithi102.stdout:6/524: symlink d9/d25/d60/d6a/dad/lb2 0 2024-05-13T22:48:08.104 INFO:tasks.workunit.client.0.smithi102.stdout:7/348: getdents d6 0 2024-05-13T22:48:08.110 INFO:tasks.workunit.client.0.smithi102.stdout:5/652: rename de/d25/d59/d28 to de/d25/d2a/d4b/d4d/d50/da3/df4 0 2024-05-13T22:48:08.117 INFO:tasks.workunit.client.0.smithi102.stdout:1/354: mknod d2/d10/c71 0 2024-05-13T22:48:08.117 INFO:tasks.workunit.client.0.smithi102.stdout:9/486: write d1/d6/d45/f4f [2400261,65649] 0 2024-05-13T22:48:08.124 INFO:tasks.workunit.client.0.smithi102.stdout:5/653: unlink de/d25/d43/d7a/cf2 0 2024-05-13T22:48:08.124 INFO:tasks.workunit.client.0.smithi102.stdout:9/487: dwrite d1/d6/d45/f98 [0,4194304] 0 2024-05-13T22:48:08.125 INFO:tasks.workunit.client.0.smithi102.stdout:9/488: readlink d1/d6/d3b/d5b/l84 0 2024-05-13T22:48:08.130 INFO:tasks.workunit.client.0.smithi102.stdout:4/506: creat d3/d19/dad/faf x:0 0 0 2024-05-13T22:48:08.132 INFO:tasks.workunit.client.0.smithi102.stdout:5/654: dwrite de/f11 [0,4194304] 0 2024-05-13T22:48:08.135 INFO:tasks.workunit.client.0.smithi102.stdout:1/355: creat d2/d4/d26/d40/f72 x:0 0 0 2024-05-13T22:48:08.136 INFO:tasks.workunit.client.0.smithi102.stdout:1/356: chown d2/d4/d5 0 1 2024-05-13T22:48:08.141 INFO:tasks.workunit.client.0.smithi102.stdout:9/489: write d1/d6/f4e [1205944,19274] 0 2024-05-13T22:48:08.148 INFO:tasks.workunit.client.0.smithi102.stdout:6/525: dwrite d9/d45/d56/d5d/f82 [0,4194304] 0 2024-05-13T22:48:08.152 INFO:tasks.workunit.client.0.smithi102.stdout:9/490: rename d1/d5d/l61 to d1/d5d/d6c/lb1 0 2024-05-13T22:48:08.194 INFO:tasks.workunit.client.0.smithi102.stdout:3/555: sync 2024-05-13T22:48:08.204 INFO:tasks.workunit.client.0.smithi102.stdout:4/507: symlink d3/d4/lb0 0 2024-05-13T22:48:08.205 INFO:tasks.workunit.client.0.smithi102.stdout:7/349: write d6/d13/d58/f34 [4238962,114784] 0 2024-05-13T22:48:08.208 INFO:tasks.workunit.client.0.smithi102.stdout:3/556: dread - de/d10/d2a/d36/f49 zero size 2024-05-13T22:48:08.208 INFO:tasks.workunit.client.0.smithi102.stdout:7/350: chown d6/d13/d58/d20 282621 1 2024-05-13T22:48:08.209 INFO:tasks.workunit.client.0.smithi102.stdout:9/491: chown d1/d6/d45/d3e/f4c 4977 1 2024-05-13T22:48:08.211 INFO:tasks.workunit.client.0.smithi102.stdout:7/351: write d6/d22/f24 [2313236,55217] 0 2024-05-13T22:48:08.212 INFO:tasks.workunit.client.0.smithi102.stdout:4/508: unlink d3/f35 0 2024-05-13T22:48:08.212 INFO:tasks.workunit.client.0.smithi102.stdout:5/655: dwrite de/d25/d43/f82 [0,4194304] 0 2024-05-13T22:48:08.213 INFO:tasks.workunit.client.0.smithi102.stdout:9/492: dread - d1/d6/d45/d3e/f68 zero size 2024-05-13T22:48:08.215 INFO:tasks.workunit.client.0.smithi102.stdout:3/557: mkdir de/d10/d46/dc8 0 2024-05-13T22:48:08.226 INFO:tasks.workunit.client.0.smithi102.stdout:5/656: rename de/d25/d2c/dcf to de/d25/d43/df5 0 2024-05-13T22:48:08.231 INFO:tasks.workunit.client.0.smithi102.stdout:4/509: mknod d3/d19/d23/d4e/cb1 0 2024-05-13T22:48:08.232 INFO:tasks.workunit.client.0.smithi102.stdout:4/510: write d3/d19/d23/d39/d77/f7c [604132,48597] 0 2024-05-13T22:48:08.233 INFO:tasks.workunit.client.0.smithi102.stdout:9/493: symlink d1/d6/lb2 0 2024-05-13T22:48:08.234 INFO:tasks.workunit.client.0.smithi102.stdout:9/494: write d1/d6/d45/d3e/f68 [845558,10165] 0 2024-05-13T22:48:08.237 INFO:tasks.workunit.client.0.smithi102.stdout:5/657: mknod de/d25/d59/d7f/d86/cf6 0 2024-05-13T22:48:08.237 INFO:tasks.workunit.client.0.smithi102.stdout:5/658: chown de/d25/d58/l90 41 1 2024-05-13T22:48:08.238 INFO:tasks.workunit.client.0.smithi102.stdout:5/659: chown de/d33/c48 113457 1 2024-05-13T22:48:08.240 INFO:tasks.workunit.client.0.smithi102.stdout:3/558: rename de/d1c/d90/ca5 to de/d10/d2a/d36/db2/cc9 0 2024-05-13T22:48:08.241 INFO:tasks.workunit.client.0.smithi102.stdout:3/559: chown de/d68/f82 554 1 2024-05-13T22:48:08.242 INFO:tasks.workunit.client.0.smithi102.stdout:4/511: creat d3/d4/d6/de/d8d/fb2 x:0 0 0 2024-05-13T22:48:08.245 INFO:tasks.workunit.client.0.smithi102.stdout:5/660: creat de/d25/d2c/ff7 x:0 0 0 2024-05-13T22:48:08.246 INFO:tasks.workunit.client.0.smithi102.stdout:5/661: fdatasync de/d25/d58/fdd 0 2024-05-13T22:48:08.249 INFO:tasks.workunit.client.0.smithi102.stdout:4/512: dwrite d3/d19/d23/d39/d5d/f98 [0,4194304] 0 2024-05-13T22:48:08.249 INFO:tasks.workunit.client.0.smithi102.stdout:9/495: rename d1/f1e to d1/d6/d21/d71/fb3 0 2024-05-13T22:48:08.252 INFO:tasks.workunit.client.0.smithi102.stdout:7/352: write d6/d13/de/f12 [3908746,46277] 0 2024-05-13T22:48:08.255 INFO:tasks.workunit.client.0.smithi102.stdout:3/560: symlink de/d10/d2a/d36/d88/lca 0 2024-05-13T22:48:08.257 INFO:tasks.workunit.client.0.smithi102.stdout:3/561: readlink de/l53 0 2024-05-13T22:48:08.262 INFO:tasks.workunit.client.0.smithi102.stdout:4/513: dwrite d3/d4/f93 [0,4194304] 0 2024-05-13T22:48:08.262 INFO:tasks.workunit.client.0.smithi102.stdout:9/496: dwrite d1/d6/d45/d3e/f93 [0,4194304] 0 2024-05-13T22:48:08.264 INFO:tasks.workunit.client.0.smithi102.stdout:9/497: write d1/d6/db/d4a/d64/f9a [1353009,124077] 0 2024-05-13T22:48:08.283 INFO:tasks.workunit.client.0.smithi102.stdout:4/514: rmdir d3/d4/d5f 39 2024-05-13T22:48:08.287 INFO:tasks.workunit.client.0.smithi102.stdout:9/498: creat d1/d6/d45/d2d/dac/fb4 x:0 0 0 2024-05-13T22:48:08.289 INFO:tasks.workunit.client.0.smithi102.stdout:7/353: mkdir d6/d13/d2c/d32/d65 0 2024-05-13T22:48:08.295 INFO:tasks.workunit.client.0.smithi102.stdout:9/499: fsync d1/d6/d45/d3e/f56 0 2024-05-13T22:48:08.295 INFO:tasks.workunit.client.0.smithi102.stdout:9/500: stat d1/d6/db 0 2024-05-13T22:48:08.297 INFO:tasks.workunit.client.0.smithi102.stdout:7/354: write d6/d13/d2c/f3a [2003434,89332] 0 2024-05-13T22:48:08.298 INFO:tasks.workunit.client.0.smithi102.stdout:9/501: truncate d1/d6/db/d30/d47/f6d 734821 0 2024-05-13T22:48:08.300 INFO:tasks.workunit.client.0.smithi102.stdout:3/562: link de/d10/d50/f47 de/d10/d2a/d33/fcb 0 2024-05-13T22:48:08.308 INFO:tasks.workunit.client.0.smithi102.stdout:4/515: getdents d3/d4/d6/de 0 2024-05-13T22:48:08.312 INFO:tasks.workunit.client.0.smithi102.stdout:3/563: rename de/d10/f12 to de/d1c/fcc 0 2024-05-13T22:48:08.318 INFO:tasks.workunit.client.0.smithi102.stdout:4/516: truncate d3/d19/d23/d4e/d68/d85/f9a 193959 0 2024-05-13T22:48:08.318 INFO:tasks.workunit.client.0.smithi102.stdout:9/502: getdents d1/d6/d45 0 2024-05-13T22:48:08.318 INFO:tasks.workunit.client.0.smithi102.stdout:9/503: dread - d1/d6b/f9e zero size 2024-05-13T22:48:08.318 INFO:tasks.workunit.client.0.smithi102.stdout:9/504: chown d1/d6/db/d4a/f85 488410250 1 2024-05-13T22:48:08.321 INFO:tasks.workunit.client.0.smithi102.stdout:9/505: dread d1/d6/db/f1b [4194304,4194304] 0 2024-05-13T22:48:08.329 INFO:tasks.workunit.client.0.smithi102.stdout:3/564: dread de/d10/f97 [0,4194304] 0 2024-05-13T22:48:08.331 INFO:tasks.workunit.client.0.smithi102.stdout:3/565: write de/d10/d50/d63/d98/fc4 [376119,67897] 0 2024-05-13T22:48:08.335 INFO:tasks.workunit.client.0.smithi102.stdout:4/517: creat d3/d19/d23/d39/da7/fb3 x:0 0 0 2024-05-13T22:48:08.336 INFO:tasks.workunit.client.0.smithi102.stdout:9/506: dwrite d1/d6/d21/d71/fb3 [4194304,4194304] 0 2024-05-13T22:48:08.337 INFO:tasks.workunit.client.0.smithi102.stdout:3/566: creat de/d10/d46/fcd x:0 0 0 2024-05-13T22:48:08.337 INFO:tasks.workunit.client.0.smithi102.stdout:4/518: write d3/d4/d6/f18 [466324,36775] 0 2024-05-13T22:48:08.345 INFO:tasks.workunit.client.0.smithi102.stdout:4/519: chown d3/l11 615761802 1 2024-05-13T22:48:08.347 INFO:tasks.workunit.client.0.smithi102.stdout:4/520: write d3/d19/d23/d39/da7/f75 [187909,85736] 0 2024-05-13T22:48:08.349 INFO:tasks.workunit.client.0.smithi102.stdout:9/507: mknod d1/d6/d45/da1/cb5 0 2024-05-13T22:48:08.350 INFO:tasks.workunit.client.0.smithi102.stdout:9/508: stat d1/d6/d3b/d5b/c9b 0 2024-05-13T22:48:08.350 INFO:tasks.workunit.client.0.smithi102.stdout:9/509: stat d1/d6/db/d30/c4d 0 2024-05-13T22:48:08.351 INFO:tasks.workunit.client.0.smithi102.stdout:9/510: write d1/d6/d45/d3e/f56 [3766948,80378] 0 2024-05-13T22:48:08.352 INFO:tasks.workunit.client.0.smithi102.stdout:9/511: write d1/d6/db/f1c [385451,120900] 0 2024-05-13T22:48:08.358 INFO:tasks.workunit.client.0.smithi102.stdout:4/521: creat d3/d4/fb4 x:0 0 0 2024-05-13T22:48:08.359 INFO:tasks.workunit.client.0.smithi102.stdout:9/512: chown d1/d6/l59 72856356 1 2024-05-13T22:48:08.367 INFO:tasks.workunit.client.0.smithi102.stdout:4/522: dwrite d3/d4/d6/de/d1f/d41/d6a/fa1 [0,4194304] 0 2024-05-13T22:48:08.370 INFO:tasks.workunit.client.0.smithi102.stdout:9/513: dwrite d1/d6/d21/d71/f8c [4194304,4194304] 0 2024-05-13T22:48:08.372 INFO:tasks.workunit.client.0.smithi102.stdout:4/523: write d3/d4/d6/de/d1f/d41/d6a/f7a [1003219,8644] 0 2024-05-13T22:48:08.373 INFO:tasks.workunit.client.0.smithi102.stdout:3/567: dwrite fa [4194304,4194304] 0 2024-05-13T22:48:08.375 INFO:tasks.workunit.client.0.smithi102.stdout:4/524: dread - d3/d19/dad/faf zero size 2024-05-13T22:48:08.377 INFO:tasks.workunit.client.0.smithi102.stdout:4/525: chown d3/d4/d6/l13 517280 1 2024-05-13T22:48:08.378 INFO:tasks.workunit.client.0.smithi102.stdout:3/568: mknod de/d10/d2a/cce 0 2024-05-13T22:48:08.380 INFO:tasks.workunit.client.0.smithi102.stdout:4/526: chown d3/d19/d23/d4e/f76 1455 1 2024-05-13T22:48:08.381 INFO:tasks.workunit.client.0.smithi102.stdout:9/514: dwrite d1/d6/db/d30/d8a/f99 [0,4194304] 0 2024-05-13T22:48:08.381 INFO:tasks.workunit.client.0.smithi102.stdout:3/569: fsync de/d10/d2a/d33/fb8 0 2024-05-13T22:48:08.386 INFO:tasks.workunit.client.0.smithi102.stdout:3/570: rename de/d68/l70 to de/d10/d50/d63/da2/lcf 0 2024-05-13T22:48:08.387 INFO:tasks.workunit.client.0.smithi102.stdout:3/571: dread - de/d68/f9d zero size 2024-05-13T22:48:08.397 INFO:tasks.workunit.client.0.smithi102.stdout:4/527: creat d3/d4/d6/fb5 x:0 0 0 2024-05-13T22:48:08.398 INFO:tasks.workunit.client.0.smithi102.stdout:4/528: write d3/d19/d23/d4e/f79 [891256,11193] 0 2024-05-13T22:48:08.399 INFO:tasks.workunit.client.0.smithi102.stdout:9/515: link d1/d6/d21/c46 d1/d6/db/d4a/d75/d96/cb6 0 2024-05-13T22:48:08.404 INFO:tasks.workunit.client.0.smithi102.stdout:9/516: dread d1/d6/db/d4a/d64/f9a [0,4194304] 0 2024-05-13T22:48:08.409 INFO:tasks.workunit.client.0.smithi102.stdout:9/517: creat d1/d6/d45/da1/fb7 x:0 0 0 2024-05-13T22:48:08.409 INFO:tasks.workunit.client.0.smithi102.stdout:9/518: write d1/d6/fa5 [164822,94887] 0 2024-05-13T22:48:08.411 INFO:tasks.workunit.client.0.smithi102.stdout:9/519: dread d1/d6/db/f66 [0,4194304] 0 2024-05-13T22:48:08.412 INFO:tasks.workunit.client.0.smithi102.stdout:9/520: read - d1/d6b/f9e zero size 2024-05-13T22:48:08.446 INFO:tasks.workunit.client.0.smithi102.stdout:0/524: sync 2024-05-13T22:48:08.448 INFO:tasks.workunit.client.0.smithi102.stdout:4/529: rename d3/d19/d23/d39/c49 to d3/d19/d23/d39/d77/cb6 0 2024-05-13T22:48:08.449 INFO:tasks.workunit.client.0.smithi102.stdout:0/525: chown d1/de/d32/d24/f3c 632677 1 2024-05-13T22:48:08.456 INFO:tasks.workunit.client.0.smithi102.stdout:0/526: dwrite d1/de/d32/d24/d2d/f64 [0,4194304] 0 2024-05-13T22:48:08.475 INFO:tasks.workunit.client.0.smithi102.stdout:0/527: dread d1/f2c [0,4194304] 0 2024-05-13T22:48:08.497 INFO:tasks.workunit.client.0.smithi102.stdout:9/521: mkdir d1/d6/d45/d2d/dac/db0/db8 0 2024-05-13T22:48:08.500 INFO:tasks.workunit.client.0.smithi102.stdout:9/522: mkdir d1/d6/db/d4a/d75/db9 0 2024-05-13T22:48:08.502 INFO:tasks.workunit.client.0.smithi102.stdout:4/530: creat d3/d19/d23/d39/d5d/da5/fb7 x:0 0 0 2024-05-13T22:48:08.503 INFO:tasks.workunit.client.0.smithi102.stdout:9/523: mknod d1/d6/db/d30/cba 0 2024-05-13T22:48:08.507 INFO:tasks.workunit.client.0.smithi102.stdout:9/524: fdatasync d1/d6/d21/f42 0 2024-05-13T22:48:08.508 INFO:tasks.workunit.client.0.smithi102.stdout:9/525: write d1/d6/db/d30/d8a/faf [61481,8498] 0 2024-05-13T22:48:08.509 INFO:tasks.workunit.client.0.smithi102.stdout:4/531: rename d3/d4/d6/de/d1f/c58 to d3/d19/d23/d39/dac/cb8 0 2024-05-13T22:48:08.513 INFO:tasks.workunit.client.0.smithi102.stdout:9/526: creat d1/d6/d45/d2d/dac/db0/fbb x:0 0 0 2024-05-13T22:48:08.517 INFO:tasks.workunit.client.0.smithi102.stdout:4/532: creat d3/d22/fb9 x:0 0 0 2024-05-13T22:48:08.525 INFO:tasks.workunit.client.0.smithi102.stdout:4/533: rename d3/d19/d23/d39/fa0 to d3/fba 0 2024-05-13T22:48:08.529 INFO:tasks.workunit.client.0.smithi102.stdout:4/534: rename d3/d22/fb9 to d3/d22/dab/fbb 0 2024-05-13T22:48:08.530 INFO:tasks.workunit.client.0.smithi102.stdout:4/535: stat d3/d19/d23/d4e/d68/f94 0 2024-05-13T22:48:08.534 INFO:tasks.workunit.client.0.smithi102.stdout:4/536: unlink d3/d4/l3b 0 2024-05-13T22:48:08.537 INFO:tasks.workunit.client.0.smithi102.stdout:4/537: unlink d3/d4/d6/de/c38 0 2024-05-13T22:48:08.538 INFO:tasks.workunit.client.0.smithi102.stdout:4/538: fsync d3/d19/d23/d3a/f56 0 2024-05-13T22:48:08.539 INFO:tasks.workunit.client.0.smithi102.stdout:4/539: truncate d3/d4/fb4 818773 0 2024-05-13T22:48:08.542 INFO:tasks.workunit.client.0.smithi102.stdout:4/540: creat d3/d19/dad/fbc x:0 0 0 2024-05-13T22:48:08.564 INFO:tasks.workunit.client.0.smithi102.stdout:9/527: truncate d1/d6/d21/d71/f8c 7316038 0 2024-05-13T22:48:08.577 INFO:tasks.workunit.client.0.smithi102.stdout:4/541: dwrite d3/d19/d23/d4e/f55 [0,4194304] 0 2024-05-13T22:48:08.580 INFO:tasks.workunit.client.0.smithi102.stdout:9/528: truncate d1/d6/db/f83 335580 0 2024-05-13T22:48:08.584 INFO:tasks.workunit.client.0.smithi102.stdout:9/529: mknod d1/d6b/cbc 0 2024-05-13T22:48:08.592 INFO:tasks.workunit.client.0.smithi102.stdout:2/465: sync 2024-05-13T22:48:08.592 INFO:tasks.workunit.client.0.smithi102.stdout:8/386: sync 2024-05-13T22:48:08.593 INFO:tasks.workunit.client.0.smithi102.stdout:8/387: fdatasync d2/d7/f4e 0 2024-05-13T22:48:08.594 INFO:tasks.workunit.client.0.smithi102.stdout:8/388: stat d2/d7/d27/f46 0 2024-05-13T22:48:08.594 INFO:tasks.workunit.client.0.smithi102.stdout:2/466: symlink d7/d9/d10/d30/l99 0 2024-05-13T22:48:08.595 INFO:tasks.workunit.client.0.smithi102.stdout:8/389: stat d2/d36/d49/d5d/l1a 0 2024-05-13T22:48:08.595 INFO:tasks.workunit.client.0.smithi102.stdout:8/390: stat d2/d36/f5a 0 2024-05-13T22:48:08.602 INFO:tasks.workunit.client.0.smithi102.stdout:8/391: dwrite d2/d7/d25/d41/d50/f35 [4194304,4194304] 0 2024-05-13T22:48:08.607 INFO:tasks.workunit.client.0.smithi102.stdout:8/392: truncate d2/d36/d49/d5d/f51 738170 0 2024-05-13T22:48:08.610 INFO:tasks.workunit.client.0.smithi102.stdout:8/393: write d2/d36/d49/d5d/f33 [4096686,73082] 0 2024-05-13T22:48:08.613 INFO:tasks.workunit.client.0.smithi102.stdout:8/394: mkdir d2/d7/d12/d78 0 2024-05-13T22:48:08.615 INFO:tasks.workunit.client.0.smithi102.stdout:2/467: dwrite d7/d9/d11/f39 [0,4194304] 0 2024-05-13T22:48:08.616 INFO:tasks.workunit.client.0.smithi102.stdout:8/395: truncate d2/d36/d49/d5d/f33 4686614 0 2024-05-13T22:48:08.617 INFO:tasks.workunit.client.0.smithi102.stdout:8/396: readlink d2/d36/d49/d5d/l2e 0 2024-05-13T22:48:08.618 INFO:tasks.workunit.client.0.smithi102.stdout:8/397: chown d2/d7/d25/d41/f75 88702 1 2024-05-13T22:48:08.620 INFO:tasks.workunit.client.0.smithi102.stdout:8/398: fdatasync d2/d7/d12/f58 0 2024-05-13T22:48:08.622 INFO:tasks.workunit.client.0.smithi102.stdout:2/468: symlink d7/d9/d10/d1b/d54/d83/l9a 0 2024-05-13T22:48:08.623 INFO:tasks.workunit.client.0.smithi102.stdout:8/399: dread d2/d7/d25/d41/d50/f39 [0,4194304] 0 2024-05-13T22:48:08.626 INFO:tasks.workunit.client.0.smithi102.stdout:2/469: mknod d7/d15/c9b 0 2024-05-13T22:48:08.630 INFO:tasks.workunit.client.0.smithi102.stdout:8/400: getdents d2/d7/d12 0 2024-05-13T22:48:08.906 INFO:tasks.workunit.client.0.smithi102.stdout:1/357: sync 2024-05-13T22:48:08.906 INFO:tasks.workunit.client.0.smithi102.stdout:6/526: sync 2024-05-13T22:48:08.907 INFO:tasks.workunit.client.0.smithi102.stdout:1/358: rmdir d2/d4/d5/d19 39 2024-05-13T22:48:08.909 INFO:tasks.workunit.client.0.smithi102.stdout:6/527: creat d9/d25/d71/fb3 x:0 0 0 2024-05-13T22:48:08.912 INFO:tasks.workunit.client.0.smithi102.stdout:6/528: dread d9/db/d14/d37/f57 [0,4194304] 0 2024-05-13T22:48:08.914 INFO:tasks.workunit.client.0.smithi102.stdout:6/529: truncate d9/d25/d60/d6a/dad/faf 689243 0 2024-05-13T22:48:08.914 INFO:tasks.workunit.client.0.smithi102.stdout:6/530: chown d9/d45/d56/f9b 0 1 2024-05-13T22:48:08.916 INFO:tasks.workunit.client.0.smithi102.stdout:6/531: stat d9/db/f7d 0 2024-05-13T22:48:08.921 INFO:tasks.workunit.client.0.smithi102.stdout:6/532: creat d9/db/d14/d37/fb4 x:0 0 0 2024-05-13T22:48:08.972 INFO:tasks.workunit.client.0.smithi102.stdout:1/359: write d2/d4/f37 [671275,60529] 0 2024-05-13T22:48:08.976 INFO:tasks.workunit.client.0.smithi102.stdout:6/533: rename d9/db/d14/d37 to d9/d25/db5 0 2024-05-13T22:48:08.976 INFO:tasks.workunit.client.0.smithi102.stdout:6/534: rename d9 to d9/d25/d71/db6 22 2024-05-13T22:48:08.977 INFO:tasks.workunit.client.0.smithi102.stdout:6/535: dread - d9/db/d14/d35/d47/f8f zero size 2024-05-13T22:48:08.984 INFO:tasks.workunit.client.0.smithi102.stdout:6/536: dwrite d9/db/f77 [0,4194304] 0 2024-05-13T22:48:08.987 INFO:tasks.workunit.client.0.smithi102.stdout:6/537: fdatasync d9/d25/db5/f4b 0 2024-05-13T22:48:08.987 INFO:tasks.workunit.client.0.smithi102.stdout:6/538: chown d9/f21 76182134 1 2024-05-13T22:48:08.989 INFO:tasks.workunit.client.0.smithi102.stdout:6/539: read d9/db/f29 [1757790,27395] 0 2024-05-13T22:48:09.008 INFO:tasks.workunit.client.0.smithi102.stdout:6/540: dread d9/d25/db5/f7b [0,4194304] 0 2024-05-13T22:48:09.011 INFO:tasks.workunit.client.0.smithi102.stdout:6/541: dread d9/db/f77 [0,4194304] 0 2024-05-13T22:48:09.025 INFO:tasks.workunit.client.0.smithi102.stdout:6/542: dread d9/db/f3c [0,4194304] 0 2024-05-13T22:48:09.027 INFO:tasks.workunit.client.0.smithi102.stdout:6/543: creat d9/d25/d71/fb7 x:0 0 0 2024-05-13T22:48:09.028 INFO:tasks.workunit.client.0.smithi102.stdout:6/544: chown d9/d45/d56/l90 891 1 2024-05-13T22:48:09.029 INFO:tasks.workunit.client.0.smithi102.stdout:6/545: mknod d9/db/d14/d35/d47/cb8 0 2024-05-13T22:48:09.031 INFO:tasks.workunit.client.0.smithi102.stdout:6/546: unlink d9/db/d14/l8e 0 2024-05-13T22:48:09.033 INFO:tasks.workunit.client.0.smithi102.stdout:6/547: creat d9/d25/d5a/fb9 x:0 0 0 2024-05-13T22:48:09.039 INFO:tasks.workunit.client.0.smithi102.stdout:6/548: dwrite d9/d45/d56/f9b [0,4194304] 0 2024-05-13T22:48:09.042 INFO:tasks.workunit.client.0.smithi102.stdout:6/549: symlink d9/db/d14/d54/lba 0 2024-05-13T22:48:09.044 INFO:tasks.workunit.client.0.smithi102.stdout:6/550: mknod d9/d25/d5a/cbb 0 2024-05-13T22:48:09.045 INFO:tasks.workunit.client.0.smithi102.stdout:6/551: dread - d9/d25/f88 zero size 2024-05-13T22:48:09.048 INFO:tasks.workunit.client.0.smithi102.stdout:6/552: symlink d9/d25/d60/d6a/lbc 0 2024-05-13T22:48:09.053 INFO:tasks.workunit.client.0.smithi102.stdout:6/553: creat d9/db/d2f/d59/fbd x:0 0 0 2024-05-13T22:48:09.057 INFO:tasks.workunit.client.0.smithi102.stdout:6/554: creat d9/fbe x:0 0 0 2024-05-13T22:48:09.060 INFO:tasks.workunit.client.0.smithi102.stdout:6/555: getdents d9/d25/db5/d5f 0 2024-05-13T22:48:09.062 INFO:tasks.workunit.client.0.smithi102.stdout:6/556: mknod d9/d25/db5/cbf 0 2024-05-13T22:48:09.063 INFO:tasks.workunit.client.0.smithi102.stdout:6/557: write d9/d25/d71/fb7 [985323,35423] 0 2024-05-13T22:48:09.108 INFO:tasks.workunit.client.0.smithi102.stdout:5/662: sync 2024-05-13T22:48:09.110 INFO:tasks.workunit.client.0.smithi102.stdout:5/663: fsync f0 0 2024-05-13T22:48:09.114 INFO:tasks.workunit.client.0.smithi102.stdout:5/664: rename de/d33/d53/d9e to de/d25/d2a/d4b/d4d/d50/d6c/df8 0 2024-05-13T22:48:09.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:08 smithi130 ceph-mon[36591]: pgmap v21: 65 pgs: 65 active+clean; 843 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 MiB/s rd, 101 MiB/s wr, 302 op/s 2024-05-13T22:48:09.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:08 smithi102 ceph-mon[28599]: pgmap v21: 65 pgs: 65 active+clean; 843 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 MiB/s rd, 101 MiB/s wr, 302 op/s 2024-05-13T22:48:09.274 INFO:tasks.workunit.client.0.smithi102.stdout:2/470: sync 2024-05-13T22:48:09.275 INFO:tasks.workunit.client.0.smithi102.stdout:0/528: sync 2024-05-13T22:48:09.275 INFO:tasks.workunit.client.0.smithi102.stdout:7/355: sync 2024-05-13T22:48:09.275 INFO:tasks.workunit.client.0.smithi102.stdout:4/542: sync 2024-05-13T22:48:09.275 INFO:tasks.workunit.client.0.smithi102.stdout:3/572: sync 2024-05-13T22:48:09.275 INFO:tasks.workunit.client.0.smithi102.stdout:9/530: sync 2024-05-13T22:48:09.275 INFO:tasks.workunit.client.0.smithi102.stdout:8/401: sync 2024-05-13T22:48:09.275 INFO:tasks.workunit.client.0.smithi102.stdout:1/360: sync 2024-05-13T22:48:09.278 INFO:tasks.workunit.client.0.smithi102.stdout:9/531: dread - d1/d6/db/d30/f95 zero size 2024-05-13T22:48:09.279 INFO:tasks.workunit.client.0.smithi102.stdout:4/543: write d3/d4/d6/f18 [5595180,86118] 0 2024-05-13T22:48:09.283 INFO:tasks.workunit.client.0.smithi102.stdout:8/402: dwrite d2/d7/f69 [0,4194304] 0 2024-05-13T22:48:09.284 INFO:tasks.workunit.client.0.smithi102.stdout:7/356: dread d6/fc [4194304,4194304] 0 2024-05-13T22:48:09.285 INFO:tasks.workunit.client.0.smithi102.stdout:7/357: chown d6/d15 33705718 1 2024-05-13T22:48:09.285 INFO:tasks.workunit.client.0.smithi102.stdout:0/529: fsync d1/d4d/f85 0 2024-05-13T22:48:09.286 INFO:tasks.workunit.client.0.smithi102.stdout:0/530: stat d1/d68/ca1 0 2024-05-13T22:48:09.288 INFO:tasks.workunit.client.0.smithi102.stdout:2/471: dread d7/d9/d14/f36 [0,4194304] 0 2024-05-13T22:48:09.288 INFO:tasks.workunit.client.0.smithi102.stdout:2/472: stat d7/d9/d14/f36 0 2024-05-13T22:48:09.293 INFO:tasks.workunit.client.0.smithi102.stdout:8/403: symlink d2/d36/d49/l79 0 2024-05-13T22:48:09.293 INFO:tasks.workunit.client.0.smithi102.stdout:4/544: dwrite d3/d4/d6/de/f15 [0,4194304] 0 2024-05-13T22:48:09.294 INFO:tasks.workunit.client.0.smithi102.stdout:8/404: write d2/d36/f42 [2491819,83944] 0 2024-05-13T22:48:09.295 INFO:tasks.workunit.client.0.smithi102.stdout:1/361: read d2/d4/d5/d19/d28/f35 [1259703,24282] 0 2024-05-13T22:48:09.298 INFO:tasks.workunit.client.0.smithi102.stdout:7/358: chown d6/d15/d4c/l4f 0 1 2024-05-13T22:48:09.298 INFO:tasks.workunit.client.0.smithi102.stdout:7/359: stat d6/d22/c31 0 2024-05-13T22:48:09.300 INFO:tasks.workunit.client.0.smithi102.stdout:0/531: fdatasync d1/de/d32/d24/f62 0 2024-05-13T22:48:09.301 INFO:tasks.workunit.client.0.smithi102.stdout:0/532: fdatasync d1/de/d32/d24/d2d/f64 0 2024-05-13T22:48:09.303 INFO:tasks.workunit.client.0.smithi102.stdout:2/473: rename d7/d15/d67/d6a/c76 to d7/d9/d10/d25/d27/c9c 0 2024-05-13T22:48:09.304 INFO:tasks.workunit.client.0.smithi102.stdout:2/474: chown d7/d9/d10/d25/d56/d59/d81 59534360 1 2024-05-13T22:48:09.305 INFO:tasks.workunit.client.0.smithi102.stdout:4/545: rmdir d3/d19/d23/d4e/d68 39 2024-05-13T22:48:09.305 INFO:tasks.workunit.client.0.smithi102.stdout:2/475: write d7/d9/d14/d61/f92 [4699053,1352] 0 2024-05-13T22:48:09.306 INFO:tasks.workunit.client.0.smithi102.stdout:2/476: fsync d7/d9/d10/f33 0 2024-05-13T22:48:09.314 INFO:tasks.workunit.client.0.smithi102.stdout:7/360: rename d6/f8 to d6/d13/d58/d20/d4b/f66 0 2024-05-13T22:48:09.319 INFO:tasks.workunit.client.0.smithi102.stdout:8/405: mknod d2/d7/d25/d41/d6f/c7a 0 2024-05-13T22:48:09.320 INFO:tasks.workunit.client.0.smithi102.stdout:2/477: creat d7/d9/d14/d61/f9d x:0 0 0 2024-05-13T22:48:09.327 INFO:tasks.workunit.client.0.smithi102.stdout:8/406: truncate d2/f32 1527491 0 2024-05-13T22:48:09.329 INFO:tasks.workunit.client.0.smithi102.stdout:2/478: rmdir d7/d9/d10/d30 39 2024-05-13T22:48:09.330 INFO:tasks.workunit.client.0.smithi102.stdout:2/479: fdatasync d7/d15/d1e/f26 0 2024-05-13T22:48:09.331 INFO:tasks.workunit.client.0.smithi102.stdout:8/407: creat d2/d7/d12/f7b x:0 0 0 2024-05-13T22:48:09.335 INFO:tasks.workunit.client.0.smithi102.stdout:8/408: symlink d2/d36/d49/l7c 0 2024-05-13T22:48:09.336 INFO:tasks.workunit.client.0.smithi102.stdout:2/480: creat d7/d9/d14/d61/d82/f9e x:0 0 0 2024-05-13T22:48:09.338 INFO:tasks.workunit.client.0.smithi102.stdout:2/481: dread d7/d9/d10/f37 [4194304,4194304] 0 2024-05-13T22:48:09.341 INFO:tasks.workunit.client.0.smithi102.stdout:8/409: creat d2/d7/d25/d41/d50/d40/f7d x:0 0 0 2024-05-13T22:48:09.341 INFO:tasks.workunit.client.0.smithi102.stdout:2/482: write d7/d9/d10/d30/f80 [766764,119605] 0 2024-05-13T22:48:09.343 INFO:tasks.workunit.client.0.smithi102.stdout:2/483: write d7/f32 [1822410,40555] 0 2024-05-13T22:48:09.346 INFO:tasks.workunit.client.0.smithi102.stdout:8/410: dwrite d2/d7/f53 [0,4194304] 0 2024-05-13T22:48:09.346 INFO:tasks.workunit.client.0.smithi102.stdout:8/411: read - d2/f73 zero size 2024-05-13T22:48:09.348 INFO:tasks.workunit.client.0.smithi102.stdout:8/412: write d2/d36/f62 [374442,60056] 0 2024-05-13T22:48:09.350 INFO:tasks.workunit.client.0.smithi102.stdout:2/484: symlink d7/d9/l9f 0 2024-05-13T22:48:09.353 INFO:tasks.workunit.client.0.smithi102.stdout:8/413: mkdir d2/d36/d49/d7e 0 2024-05-13T22:48:09.414 INFO:tasks.workunit.client.0.smithi102.stdout:5/665: sync 2024-05-13T22:48:09.414 INFO:tasks.workunit.client.0.smithi102.stdout:6/558: sync 2024-05-13T22:48:09.415 INFO:tasks.workunit.client.0.smithi102.stdout:5/666: dread - de/fc8 zero size 2024-05-13T22:48:09.510 INFO:tasks.workunit.client.0.smithi102.stdout:6/559: symlink d9/db/d14/lc0 0 2024-05-13T22:48:09.514 INFO:tasks.workunit.client.0.smithi102.stdout:6/560: rmdir d9/db/d2f/d59 39 2024-05-13T22:48:09.527 INFO:tasks.workunit.client.0.smithi102.stdout:7/361: write d6/d22/f33 [206833,124785] 0 2024-05-13T22:48:09.530 INFO:tasks.workunit.client.0.smithi102.stdout:7/362: chown d6/d13/d58/d20 1164 1 2024-05-13T22:48:09.534 INFO:tasks.workunit.client.0.smithi102.stdout:1/362: truncate d2/d4/d5/d19/d28/f24 814615 0 2024-05-13T22:48:09.536 INFO:tasks.workunit.client.0.smithi102.stdout:7/363: getdents d6/d13/d58/d20 0 2024-05-13T22:48:09.542 INFO:tasks.workunit.client.0.smithi102.stdout:7/364: write d6/d15/f3e [4876733,91559] 0 2024-05-13T22:48:09.553 INFO:tasks.workunit.client.0.smithi102.stdout:8/414: getdents d2/d36/d49 0 2024-05-13T22:48:09.556 INFO:tasks.workunit.client.0.smithi102.stdout:8/415: rmdir d2/d36/d49/d5d 39 2024-05-13T22:48:09.556 INFO:tasks.workunit.client.0.smithi102.stdout:8/416: readlink d2/d36/d49/l79 0 2024-05-13T22:48:09.557 INFO:tasks.workunit.client.0.smithi102.stdout:8/417: stat d2/d36/d49/l7c 0 2024-05-13T22:48:09.559 INFO:tasks.workunit.client.0.smithi102.stdout:8/418: write d2/d7/d25/d41/d50/f35 [5567997,65771] 0 2024-05-13T22:48:09.560 INFO:tasks.workunit.client.0.smithi102.stdout:7/365: dread d6/d15/f1e [0,4194304] 0 2024-05-13T22:48:09.561 INFO:tasks.workunit.client.0.smithi102.stdout:7/366: chown d6/d13/d2c/f3a 15 1 2024-05-13T22:48:09.562 INFO:tasks.workunit.client.0.smithi102.stdout:8/419: creat d2/d7/d25/d41/d50/d40/f7f x:0 0 0 2024-05-13T22:48:09.568 INFO:tasks.workunit.client.0.smithi102.stdout:7/367: creat d6/f67 x:0 0 0 2024-05-13T22:48:09.569 INFO:tasks.workunit.client.0.smithi102.stdout:8/420: mkdir d2/d7/d27/d64/d80 0 2024-05-13T22:48:09.569 INFO:tasks.workunit.client.0.smithi102.stdout:8/421: fsync d2/d7/d12/f58 0 2024-05-13T22:48:09.570 INFO:tasks.workunit.client.0.smithi102.stdout:5/667: write de/d25/d59/fb4 [369720,14496] 0 2024-05-13T22:48:09.571 INFO:tasks.workunit.client.0.smithi102.stdout:7/368: fdatasync d6/d13/d2c/d32/f51 0 2024-05-13T22:48:09.573 INFO:tasks.workunit.client.0.smithi102.stdout:8/422: write d2/d36/d49/d5d/f33 [2017013,49508] 0 2024-05-13T22:48:09.573 INFO:tasks.workunit.client.0.smithi102.stdout:7/369: dread - d6/d13/d2c/d32/f51 zero size 2024-05-13T22:48:09.580 INFO:tasks.workunit.client.0.smithi102.stdout:5/668: creat de/d25/d59/d7f/d86/da5/ddb/dee/ff9 x:0 0 0 2024-05-13T22:48:09.585 INFO:tasks.workunit.client.0.smithi102.stdout:7/370: getdents d6/d13/de 0 2024-05-13T22:48:09.586 INFO:tasks.workunit.client.0.smithi102.stdout:7/371: write d6/d13/d58/d20/d4b/f5f [303919,112712] 0 2024-05-13T22:48:09.589 INFO:tasks.workunit.client.0.smithi102.stdout:7/372: creat d6/d13/d58/d20/f68 x:0 0 0 2024-05-13T22:48:09.590 INFO:tasks.workunit.client.0.smithi102.stdout:5/669: link de/d25/d2a/d4b/d4d/d50/da3/lf1 de/d25/d2a/d4b/d4d/lfa 0 2024-05-13T22:48:09.592 INFO:tasks.workunit.client.0.smithi102.stdout:2/485: truncate d7/d9/d10/d1b/f6b 4158209 0 2024-05-13T22:48:09.592 INFO:tasks.workunit.client.0.smithi102.stdout:2/486: chown d7/d15/c7d 1185 1 2024-05-13T22:48:09.593 INFO:tasks.workunit.client.0.smithi102.stdout:7/373: mknod d6/d13/d2c/d32/d49/c69 0 2024-05-13T22:48:09.597 INFO:tasks.workunit.client.0.smithi102.stdout:7/374: stat d6/d15/d4d 0 2024-05-13T22:48:09.599 INFO:tasks.workunit.client.0.smithi102.stdout:2/487: dwrite d7/d9/d10/d1b/f8a [0,4194304] 0 2024-05-13T22:48:09.605 INFO:tasks.workunit.client.0.smithi102.stdout:2/488: fdatasync d7/d9/d10/d30/d53/f5e 0 2024-05-13T22:48:09.607 INFO:tasks.workunit.client.0.smithi102.stdout:7/375: dwrite d6/d13/d2c/d39/f3f [0,4194304] 0 2024-05-13T22:48:09.611 INFO:tasks.workunit.client.0.smithi102.stdout:7/376: symlink d6/d13/d2c/d39/l6a 0 2024-05-13T22:48:09.614 INFO:tasks.workunit.client.0.smithi102.stdout:5/670: dread de/d25/d2a/d4b/d4d/d50/da3/df4/f66 [0,4194304] 0 2024-05-13T22:48:09.617 INFO:tasks.workunit.client.0.smithi102.stdout:2/489: dwrite d7/d9/d14/d61/f93 [0,4194304] 0 2024-05-13T22:48:09.618 INFO:tasks.workunit.client.0.smithi102.stdout:7/377: truncate d6/d13/d58/f26 844231 0 2024-05-13T22:48:09.622 INFO:tasks.workunit.client.0.smithi102.stdout:5/671: mkdir de/d25/d58/dfb 0 2024-05-13T22:48:09.623 INFO:tasks.workunit.client.0.smithi102.stdout:2/490: symlink d7/d9/d10/d1b/la0 0 2024-05-13T22:48:09.627 INFO:tasks.workunit.client.0.smithi102.stdout:5/672: creat de/d25/d2a/d4b/d4d/d50/d99/dbb/dda/ffc x:0 0 0 2024-05-13T22:48:09.629 INFO:tasks.workunit.client.0.smithi102.stdout:7/378: fdatasync d6/d22/f43 0 2024-05-13T22:48:09.631 INFO:tasks.workunit.client.0.smithi102.stdout:2/491: rename d7/d15/d67/f6e to d7/d9/d14/d61/fa1 0 2024-05-13T22:48:09.635 INFO:tasks.workunit.client.0.smithi102.stdout:7/379: dwrite d6/d15/d4c/f5a [0,4194304] 0 2024-05-13T22:48:09.639 INFO:tasks.workunit.client.0.smithi102.stdout:7/380: chown d6/d13/l61 0 1 2024-05-13T22:48:09.642 INFO:tasks.workunit.client.0.smithi102.stdout:3/573: sync 2024-05-13T22:48:09.644 INFO:tasks.workunit.client.0.smithi102.stdout:2/492: link d7/d9/d10/d30/l65 d7/d9/d10/d30/d7f/la2 0 2024-05-13T22:48:09.645 INFO:tasks.workunit.client.0.smithi102.stdout:7/381: creat d6/d13/d2c/d32/d65/f6b x:0 0 0 2024-05-13T22:48:09.646 INFO:tasks.workunit.client.0.smithi102.stdout:7/382: readlink d6/d13/d58/d20/l27 0 2024-05-13T22:48:09.647 INFO:tasks.workunit.client.0.smithi102.stdout:3/574: mknod de/d10/d2a/d8c/d7f/cd0 0 2024-05-13T22:48:09.649 INFO:tasks.workunit.client.0.smithi102.stdout:7/383: creat d6/d13/d58/d20/d4b/f6c x:0 0 0 2024-05-13T22:48:09.652 INFO:tasks.workunit.client.0.smithi102.stdout:2/493: mkdir d7/d9/d10/da3 0 2024-05-13T22:48:09.653 INFO:tasks.workunit.client.0.smithi102.stdout:2/494: chown d7/c7e 248 1 2024-05-13T22:48:09.656 INFO:tasks.workunit.client.0.smithi102.stdout:7/384: dwrite d6/d15/f5e [0,4194304] 0 2024-05-13T22:48:09.656 INFO:tasks.workunit.client.0.smithi102.stdout:7/385: chown d6/d15/d4c 1268 1 2024-05-13T22:48:09.657 INFO:tasks.workunit.client.0.smithi102.stdout:2/495: unlink d7/l2e 0 2024-05-13T22:48:09.660 INFO:tasks.workunit.client.0.smithi102.stdout:7/386: symlink d6/d15/l6d 0 2024-05-13T22:48:09.662 INFO:tasks.workunit.client.0.smithi102.stdout:2/496: mkdir d7/d9/d10/d30/d53/da4 0 2024-05-13T22:48:09.667 INFO:tasks.workunit.client.0.smithi102.stdout:2/497: rename d7/d9/d11/f55 to d7/d15/fa5 0 2024-05-13T22:48:09.674 INFO:tasks.workunit.client.0.smithi102.stdout:2/498: unlink d7/d9/d14/d61/f92 0 2024-05-13T22:48:09.675 INFO:tasks.workunit.client.0.smithi102.stdout:2/499: readlink d7/d9/d10/d25/l43 0 2024-05-13T22:48:09.676 INFO:tasks.workunit.client.0.smithi102.stdout:2/500: stat d7/d15/f19 0 2024-05-13T22:48:09.676 INFO:tasks.workunit.client.0.smithi102.stdout:1/363: dwrite d2/d4/d5/d19/f53 [0,4194304] 0 2024-05-13T22:48:09.683 INFO:tasks.workunit.client.0.smithi102.stdout:1/364: mknod d2/c73 0 2024-05-13T22:48:09.686 INFO:tasks.workunit.client.0.smithi102.stdout:2/501: getdents d7/d9 0 2024-05-13T22:48:09.689 INFO:tasks.workunit.client.0.smithi102.stdout:1/365: mkdir d2/d4/d26/d40/d5e/d74 0 2024-05-13T22:48:09.690 INFO:tasks.workunit.client.0.smithi102.stdout:2/502: dread d7/d9/d10/d1b/f8a [0,4194304] 0 2024-05-13T22:48:09.697 INFO:tasks.workunit.client.0.smithi102.stdout:8/423: dwrite d2/d36/d49/f60 [0,4194304] 0 2024-05-13T22:48:09.700 INFO:tasks.workunit.client.0.smithi102.stdout:1/366: creat d2/d4/f75 x:0 0 0 2024-05-13T22:48:09.701 INFO:tasks.workunit.client.0.smithi102.stdout:9/532: sync 2024-05-13T22:48:09.701 INFO:tasks.workunit.client.0.smithi102.stdout:6/561: sync 2024-05-13T22:48:09.701 INFO:tasks.workunit.client.0.smithi102.stdout:0/533: sync 2024-05-13T22:48:09.701 INFO:tasks.workunit.client.0.smithi102.stdout:4/546: sync 2024-05-13T22:48:09.705 INFO:tasks.workunit.client.0.smithi102.stdout:6/562: write d9/d45/f75 [484527,122978] 0 2024-05-13T22:48:09.705 INFO:tasks.workunit.client.0.smithi102.stdout:9/533: write d1/d6/fa5 [2154157,121925] 0 2024-05-13T22:48:09.707 INFO:tasks.workunit.client.0.smithi102.stdout:7/387: getdents d6/d13/d2c/d32/d49 0 2024-05-13T22:48:09.710 INFO:tasks.workunit.client.0.smithi102.stdout:1/367: creat d2/d10/f76 x:0 0 0 2024-05-13T22:48:09.710 INFO:tasks.workunit.client.0.smithi102.stdout:8/424: dwrite d2/d7/d12/f58 [0,4194304] 0 2024-05-13T22:48:09.710 INFO:tasks.workunit.client.0.smithi102.stdout:7/388: truncate d6/d13/d2c/f2f 891329 0 2024-05-13T22:48:09.712 INFO:tasks.workunit.client.0.smithi102.stdout:1/368: dread - d2/d4/d38/f6e zero size 2024-05-13T22:48:09.712 INFO:tasks.workunit.client.0.smithi102.stdout:4/547: symlink d3/d19/d23/d39/da7/d7d/lbd 0 2024-05-13T22:48:09.715 INFO:tasks.workunit.client.0.smithi102.stdout:6/563: mkdir d9/db/d14/d35/d47/dc1 0 2024-05-13T22:48:09.721 INFO:tasks.workunit.client.0.smithi102.stdout:9/534: dread d1/d6/db/f83 [0,4194304] 0 2024-05-13T22:48:09.723 INFO:tasks.workunit.client.0.smithi102.stdout:1/369: symlink d2/d4/d26/d40/d5e/l77 0 2024-05-13T22:48:09.726 INFO:tasks.workunit.client.0.smithi102.stdout:6/564: creat d9/db/d14/d35/fc2 x:0 0 0 2024-05-13T22:48:09.730 INFO:tasks.workunit.client.0.smithi102.stdout:9/535: mknod d1/d6/d45/d2d/dac/cbd 0 2024-05-13T22:48:09.733 INFO:tasks.workunit.client.0.smithi102.stdout:4/548: symlink d3/d19/d23/lbe 0 2024-05-13T22:48:09.734 INFO:tasks.workunit.client.0.smithi102.stdout:5/673: dwrite de/d25/d59/d6f/f46 [4194304,4194304] 0 2024-05-13T22:48:09.736 INFO:tasks.workunit.client.0.smithi102.stdout:0/534: dread d1/de/d32/f17 [0,4194304] 0 2024-05-13T22:48:09.738 INFO:tasks.workunit.client.0.smithi102.stdout:7/389: dread d6/d13/d58/d20/d4b/f66 [0,4194304] 0 2024-05-13T22:48:09.739 INFO:tasks.workunit.client.0.smithi102.stdout:9/536: mkdir d1/d6/d45/d3e/dbe 0 2024-05-13T22:48:09.741 INFO:tasks.workunit.client.0.smithi102.stdout:7/390: dread - d6/d13/d58/d20/f68 zero size 2024-05-13T22:48:09.742 INFO:tasks.workunit.client.0.smithi102.stdout:7/391: fdatasync d6/d15/f37 0 2024-05-13T22:48:09.744 INFO:tasks.workunit.client.0.smithi102.stdout:8/425: getdents d2/d36 0 2024-05-13T22:48:09.746 INFO:tasks.workunit.client.0.smithi102.stdout:4/549: fsync d3/d19/fae 0 2024-05-13T22:48:09.749 INFO:tasks.workunit.client.0.smithi102.stdout:4/550: chown d3/d22/f5b 793 1 2024-05-13T22:48:09.750 INFO:tasks.workunit.client.0.smithi102.stdout:3/575: dwrite de/d10/d50/fab [0,4194304] 0 2024-05-13T22:48:09.752 INFO:tasks.workunit.client.0.smithi102.stdout:7/392: dread d6/d13/d58/d20/d4b/f5f [0,4194304] 0 2024-05-13T22:48:09.754 INFO:tasks.workunit.client.0.smithi102.stdout:5/674: rename de/d25/d59/d7f/d86/da5/feb to de/d25/d43/d7a/db1/dc5/dca/ffd 0 2024-05-13T22:48:09.756 INFO:tasks.workunit.client.0.smithi102.stdout:1/370: creat d2/d4/d5/d19/f78 x:0 0 0 2024-05-13T22:48:09.759 INFO:tasks.workunit.client.0.smithi102.stdout:4/551: dwrite d3/d6b/f6f [0,4194304] 0 2024-05-13T22:48:09.765 INFO:tasks.workunit.client.0.smithi102.stdout:8/426: unlink d2/d7/d25/d41/d50/d40/f7d 0 2024-05-13T22:48:09.765 INFO:tasks.workunit.client.0.smithi102.stdout:8/427: fdatasync d2/f6d 0 2024-05-13T22:48:09.768 INFO:tasks.workunit.client.0.smithi102.stdout:7/393: rmdir d6/d13/d62 39 2024-05-13T22:48:09.771 INFO:tasks.workunit.client.0.smithi102.stdout:5/675: creat de/d25/d59/d7f/d86/da5/ffe x:0 0 0 2024-05-13T22:48:09.782 INFO:tasks.workunit.client.0.smithi102.stdout:2/503: truncate d7/d9/d11/f39 3112158 0 2024-05-13T22:48:09.785 INFO:tasks.workunit.client.0.smithi102.stdout:9/537: link d1/d6/lb2 d1/d6/d21/d71/d91/lbf 0 2024-05-13T22:48:09.789 INFO:tasks.workunit.client.0.smithi102.stdout:4/552: rename d3/d4/d6/de/d8d/f92 to d3/d19/d23/d4e/d68/d85/fbf 0 2024-05-13T22:48:09.792 INFO:tasks.workunit.client.0.smithi102.stdout:8/428: mknod d2/d7/c81 0 2024-05-13T22:48:09.793 INFO:tasks.workunit.client.0.smithi102.stdout:4/553: dread d3/d4/f32 [0,4194304] 0 2024-05-13T22:48:09.793 INFO:tasks.workunit.client.0.smithi102.stdout:2/504: write d7/d9/f13 [3117299,130241] 0 2024-05-13T22:48:09.796 INFO:tasks.workunit.client.0.smithi102.stdout:5/676: symlink de/d25/d2a/d4b/d4d/d50/da3/df4/de4/lff 0 2024-05-13T22:48:09.799 INFO:tasks.workunit.client.0.smithi102.stdout:9/538: truncate d1/d6/d3b/f38 3739406 0 2024-05-13T22:48:09.799 INFO:tasks.workunit.client.0.smithi102.stdout:8/429: dwrite d2/f4 [0,4194304] 0 2024-05-13T22:48:09.801 INFO:tasks.workunit.client.0.smithi102.stdout:8/430: readlink d2/d36/d49/d5d/l1a 0 2024-05-13T22:48:09.801 INFO:tasks.workunit.client.0.smithi102.stdout:8/431: stat d2/d7/f69 0 2024-05-13T22:48:09.808 INFO:tasks.workunit.client.0.smithi102.stdout:5/677: dread de/d25/d2a/d4b/f6e [0,4194304] 0 2024-05-13T22:48:09.841 INFO:tasks.workunit.client.0.smithi102.stdout:2/505: symlink d7/d9/d10/d25/la6 0 2024-05-13T22:48:09.845 INFO:tasks.workunit.client.0.smithi102.stdout:9/539: mkdir d1/d6/db/d4a/d64/dc0 0 2024-05-13T22:48:09.846 INFO:tasks.workunit.client.0.smithi102.stdout:4/554: mknod d3/d19/cc0 0 2024-05-13T22:48:09.848 INFO:tasks.workunit.client.0.smithi102.stdout:8/432: creat d2/d7/d25/d41/d6f/f82 x:0 0 0 2024-05-13T22:48:09.849 INFO:tasks.workunit.client.0.smithi102.stdout:6/565: truncate d9/d45/d56/f9b 1511327 0 2024-05-13T22:48:09.850 INFO:tasks.workunit.client.0.smithi102.stdout:6/566: read - d9/db/d14/d35/d47/fa8 zero size 2024-05-13T22:48:09.858 INFO:tasks.workunit.client.0.smithi102.stdout:8/433: unlink d2/d36/f62 0 2024-05-13T22:48:09.859 INFO:tasks.workunit.client.0.smithi102.stdout:6/567: rename d9/d25/d71/c7c to d9/d25/d60/cc3 0 2024-05-13T22:48:09.860 INFO:tasks.workunit.client.0.smithi102.stdout:6/568: chown d9/d25/fb1 33 1 2024-05-13T22:48:09.862 INFO:tasks.workunit.client.0.smithi102.stdout:9/540: getdents d1/d6/db/d4a/d75/db9 0 2024-05-13T22:48:09.869 INFO:tasks.workunit.client.0.smithi102.stdout:6/569: chown d9/db/c44 14 1 2024-05-13T22:48:09.871 INFO:tasks.workunit.client.0.smithi102.stdout:9/541: fdatasync d1/d6/db/d4a/d81/f97 0 2024-05-13T22:48:09.872 INFO:tasks.workunit.client.0.smithi102.stdout:1/371: dwrite d2/f54 [0,4194304] 0 2024-05-13T22:48:09.877 INFO:tasks.workunit.client.0.smithi102.stdout:8/434: dread d2/d36/d49/d5d/f45 [0,4194304] 0 2024-05-13T22:48:09.878 INFO:tasks.workunit.client.0.smithi102.stdout:8/435: read - d2/d36/d49/f6a zero size 2024-05-13T22:48:09.879 INFO:tasks.workunit.client.0.smithi102.stdout:4/555: getdents d3/d19/d23/d4e/d68/d85 0 2024-05-13T22:48:09.879 INFO:tasks.workunit.client.0.smithi102.stdout:6/570: dwrite d9/d25/db5/d5f/fa4 [0,4194304] 0 2024-05-13T22:48:09.882 INFO:tasks.workunit.client.0.smithi102.stdout:1/372: mkdir d2/d45/d79 0 2024-05-13T22:48:09.885 INFO:tasks.workunit.client.0.smithi102.stdout:9/542: symlink d1/d6/d45/d2d/dac/db0/lc1 0 2024-05-13T22:48:09.886 INFO:tasks.workunit.client.0.smithi102.stdout:9/543: dread d1/d6/db/f83 [0,4194304] 0 2024-05-13T22:48:09.887 INFO:tasks.workunit.client.0.smithi102.stdout:9/544: fdatasync d1/d6/f7c 0 2024-05-13T22:48:09.889 INFO:tasks.workunit.client.0.smithi102.stdout:6/571: mknod d9/d25/cc4 0 2024-05-13T22:48:09.894 INFO:tasks.workunit.client.0.smithi102.stdout:1/373: readlink d2/d4/d26/l2d 0 2024-05-13T22:48:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.915+0000 7fe331476700 1 -- 172.21.15.102:0/3198691244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe32c100cb0 msgr2=0x7fe32c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:09.894 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.915+0000 7fe331476700 1 --2- 172.21.15.102:0/3198691244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe32c100cb0 0x7fe32c101110 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe320009910 tx=0x7fe320009c20 comp rx=0 tx=0).stop 2024-05-13T22:48:09.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.915+0000 7fe331476700 1 -- 172.21.15.102:0/3198691244 shutdown_connections 2024-05-13T22:48:09.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.915+0000 7fe331476700 1 --2- 172.21.15.102:0/3198691244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe32c100cb0 0x7fe32c101110 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:09.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.915+0000 7fe331476700 1 --2- 172.21.15.102:0/3198691244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe32c0ffab0 0x7fe32c0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:09.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.915+0000 7fe331476700 1 -- 172.21.15.102:0/3198691244 >> 172.21.15.102:0/3198691244 conn(0x7fe32c0fb030 msgr2=0x7fe32c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:09.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.915+0000 7fe331476700 1 -- 172.21.15.102:0/3198691244 shutdown_connections 2024-05-13T22:48:09.895 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.915+0000 7fe331476700 1 -- 172.21.15.102:0/3198691244 wait complete. 2024-05-13T22:48:09.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.916+0000 7fe331476700 1 Processor -- start 2024-05-13T22:48:09.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.916+0000 7fe331476700 1 -- start start 2024-05-13T22:48:09.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.917+0000 7fe331476700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe32c0ffab0 0x7fe32c103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:09.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.917+0000 7fe331476700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe32c100cb0 0x7fe32c1020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:09.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.917+0000 7fe331476700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c199ff0 con 0x7fe32c100cb0 2024-05-13T22:48:09.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.917+0000 7fe331476700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c19a130 con 0x7fe32c0ffab0 2024-05-13T22:48:09.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.917+0000 7fe32a7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe32c100cb0 0x7fe32c1020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:09.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.917+0000 7fe32a7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe32c100cb0 0x7fe32c1020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34794/0 (socket says 172.21.15.102:34794) 2024-05-13T22:48:09.896 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.917+0000 7fe32a7fc700 1 -- 172.21.15.102:0/3775782367 learned_addr learned my addr 172.21.15.102:0/3775782367 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:48:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.918+0000 7fe32a7fc700 1 -- 172.21.15.102:0/3775782367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe32c0ffab0 msgr2=0x7fe32c103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.918+0000 7fe32a7fc700 1 --2- 172.21.15.102:0/3775782367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe32c0ffab0 0x7fe32c103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.918+0000 7fe32a7fc700 1 -- 172.21.15.102:0/3775782367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3200095f0 con 0x7fe32c100cb0 2024-05-13T22:48:09.897 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.918+0000 7fe32a7fc700 1 --2- 172.21.15.102:0/3775782367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe32c100cb0 0x7fe32c1020b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe320000c00 tx=0x7fe32000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:09.897 INFO:tasks.workunit.client.0.smithi102.stdout:9/545: rename d1/d6/db/d4a/d64/c65 to d1/d6/db/d4a/d64/cc2 0 2024-05-13T22:48:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.918+0000 7fe313fff700 1 -- 172.21.15.102:0/3775782367 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe32001d070 con 0x7fe32c100cb0 2024-05-13T22:48:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.918+0000 7fe313fff700 1 -- 172.21.15.102:0/3775782367 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe320022470 con 0x7fe32c100cb0 2024-05-13T22:48:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.918+0000 7fe313fff700 1 -- 172.21.15.102:0/3775782367 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe32000f720 con 0x7fe32c100cb0 2024-05-13T22:48:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.918+0000 7fe331476700 1 -- 172.21.15.102:0/3775782367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c1025f0 con 0x7fe32c100cb0 2024-05-13T22:48:09.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.918+0000 7fe331476700 1 -- 172.21.15.102:0/3775782367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c102b60 con 0x7fe32c100cb0 2024-05-13T22:48:09.899 INFO:tasks.workunit.client.0.smithi102.stdout:9/546: fdatasync d1/d6/d45/d3e/f56 0 2024-05-13T22:48:09.899 INFO:tasks.workunit.client.0.smithi102.stdout:6/572: creat d9/db/d14/d35/fc5 x:0 0 0 2024-05-13T22:48:09.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.919+0000 7fe331476700 1 -- 172.21.15.102:0/3775782367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c102e90 con 0x7fe32c100cb0 2024-05-13T22:48:09.899 INFO:tasks.workunit.client.0.smithi102.stdout:9/547: readlink d1/d6/d45/l80 0 2024-05-13T22:48:09.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.921+0000 7fe313fff700 1 -- 172.21.15.102:0/3775782367 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) v1 ==== 91288+0+0 (secure 0 0 0) 0x7fe320004b30 con 0x7fe32c100cb0 2024-05-13T22:48:09.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.921+0000 7fe313fff700 1 --2- 172.21.15.102:0/3775782367 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fe31406d330 0x7fe31406f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:09.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.921+0000 7fe313fff700 1 -- 172.21.15.102:0/3775782367 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe32008e540 con 0x7fe32c100cb0 2024-05-13T22:48:09.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.922+0000 7fe32affd700 1 --2- 172.21.15.102:0/3775782367 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fe31406d330 0x7fe31406f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:09.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.922+0000 7fe32affd700 1 --2- 172.21.15.102:0/3775782367 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fe31406d330 0x7fe31406f7f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe32c100b10 tx=0x7fe31c006cd0 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:09.903 INFO:tasks.workunit.client.0.smithi102.stdout:1/374: symlink d2/d4/d26/d40/d5e/l7a 0 2024-05-13T22:48:09.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:09.927+0000 7fe313fff700 1 -- 172.21.15.102:0/3775782367 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fe32005b790 con 0x7fe32c100cb0 2024-05-13T22:48:09.907 INFO:tasks.workunit.client.0.smithi102.stdout:9/548: mknod d1/d6/d45/d3e/cc3 0 2024-05-13T22:48:09.909 INFO:tasks.workunit.client.0.smithi102.stdout:4/556: rename d3/d19/d23/d4e/f79 to d3/d19/d23/d39/fc1 0 2024-05-13T22:48:09.913 INFO:tasks.workunit.client.0.smithi102.stdout:5/678: dwrite de/f5a [0,4194304] 0 2024-05-13T22:48:09.913 INFO:tasks.workunit.client.0.smithi102.stdout:1/375: read d2/d4/d26/f33 [4198372,67314] 0 2024-05-13T22:48:09.916 INFO:tasks.workunit.client.0.smithi102.stdout:2/506: dwrite d7/d9/d10/f37 [4194304,4194304] 0 2024-05-13T22:48:09.916 INFO:tasks.workunit.client.0.smithi102.stdout:6/573: creat d9/db/d14/d35/d47/dc1/fc6 x:0 0 0 2024-05-13T22:48:09.924 INFO:tasks.workunit.client.0.smithi102.stdout:4/557: creat d3/d4/d6/de/d1f/d41/fc2 x:0 0 0 2024-05-13T22:48:09.924 INFO:tasks.workunit.client.0.smithi102.stdout:5/679: dwrite de/d25/d58/fdd [0,4194304] 0 2024-05-13T22:48:09.926 INFO:tasks.workunit.client.0.smithi102.stdout:1/376: mkdir d2/d10/d62/d7b 0 2024-05-13T22:48:09.927 INFO:tasks.workunit.client.0.smithi102.stdout:2/507: mkdir d7/d15/d1e/da7 0 2024-05-13T22:48:09.934 INFO:tasks.workunit.client.0.smithi102.stdout:1/377: dread d2/d4/d5/f36 [0,4194304] 0 2024-05-13T22:48:09.937 INFO:tasks.workunit.client.0.smithi102.stdout:2/508: unlink d7/d9/d14/d61/d82/f9e 0 2024-05-13T22:48:09.940 INFO:tasks.workunit.client.0.smithi102.stdout:8/436: dwrite d2/d7/d25/d41/d50/f39 [0,4194304] 0 2024-05-13T22:48:09.942 INFO:tasks.workunit.client.0.smithi102.stdout:5/680: symlink de/d33/d34/l100 0 2024-05-13T22:48:09.954 INFO:tasks.workunit.client.0.smithi102.stdout:9/549: write d1/d6/db/d77/f7b [693681,83613] 0 2024-05-13T22:48:09.956 INFO:tasks.workunit.client.0.smithi102.stdout:9/550: truncate d1/d6/db/d30/f36 4998708 0 2024-05-13T22:48:09.957 INFO:tasks.workunit.client.0.smithi102.stdout:5/681: creat de/d25/d58/f101 x:0 0 0 2024-05-13T22:48:09.958 INFO:tasks.workunit.client.0.smithi102.stdout:8/437: dread d2/d7/d25/d41/f65 [0,4194304] 0 2024-05-13T22:48:09.959 INFO:tasks.workunit.client.0.smithi102.stdout:5/682: write de/d25/d58/fd6 [459027,47190] 0 2024-05-13T22:48:09.962 INFO:tasks.workunit.client.0.smithi102.stdout:9/551: dread d1/d5d/d6c/f86 [0,4194304] 0 2024-05-13T22:48:09.965 INFO:tasks.workunit.client.0.smithi102.stdout:2/509: link d7/f75 d7/d15/d67/fa8 0 2024-05-13T22:48:09.966 INFO:tasks.workunit.client.0.smithi102.stdout:2/510: readlink d7/d9/d14/d31/l84 0 2024-05-13T22:48:09.968 INFO:tasks.workunit.client.0.smithi102.stdout:4/558: getdents d3/d19/d23/d39 0 2024-05-13T22:48:09.968 INFO:tasks.workunit.client.0.smithi102.stdout:6/574: dread d9/d45/d56/d5d/f87 [0,4194304] 0 2024-05-13T22:48:09.969 INFO:tasks.workunit.client.0.smithi102.stdout:4/559: stat d3/d19/d23/d4e/f55 0 2024-05-13T22:48:09.970 INFO:tasks.workunit.client.0.smithi102.stdout:8/438: fdatasync d2/d7/d25/d41/f48 0 2024-05-13T22:48:09.971 INFO:tasks.workunit.client.0.smithi102.stdout:8/439: chown d2/f4 34 1 2024-05-13T22:48:09.972 INFO:tasks.workunit.client.0.smithi102.stdout:8/440: write d2/d7/f69 [2859878,90593] 0 2024-05-13T22:48:09.972 INFO:tasks.workunit.client.0.smithi102.stdout:5/683: fsync de/d25/d2a/d4b/d4d/d50/d99/dbb/dda/fe3 0 2024-05-13T22:48:09.974 INFO:tasks.workunit.client.0.smithi102.stdout:1/378: link d2/ff d2/d10/d62/d7b/f7c 0 2024-05-13T22:48:09.979 INFO:tasks.workunit.client.0.smithi102.stdout:6/575: write d9/d25/f30 [393984,55991] 0 2024-05-13T22:48:09.981 INFO:tasks.workunit.client.0.smithi102.stdout:6/576: write d9/d25/faa [223892,5416] 0 2024-05-13T22:48:09.984 INFO:tasks.workunit.client.0.smithi102.stdout:2/511: dwrite d7/d15/fa5 [0,4194304] 0 2024-05-13T22:48:09.986 INFO:tasks.workunit.client.0.smithi102.stdout:2/512: chown d7/d9/d10/d25/d56/c69 0 1 2024-05-13T22:48:09.994 INFO:tasks.workunit.client.0.smithi102.stdout:4/560: symlink d3/d19/d23/d4e/lc3 0 2024-05-13T22:48:09.995 INFO:tasks.workunit.client.0.smithi102.stdout:8/441: mkdir d2/d83 0 2024-05-13T22:48:09.999 INFO:tasks.workunit.client.0.smithi102.stdout:2/513: creat d7/d9/d10/d30/fa9 x:0 0 0 2024-05-13T22:48:10.000 INFO:tasks.workunit.client.0.smithi102.stdout:5/684: mknod de/d25/c102 0 2024-05-13T22:48:10.000 INFO:tasks.workunit.client.0.smithi102.stdout:6/577: dwrite d9/db/d2f/d59/f80 [0,4194304] 0 2024-05-13T22:48:10.001 INFO:tasks.workunit.client.0.smithi102.stdout:1/379: mknod d2/d4/c7d 0 2024-05-13T22:48:10.011 INFO:tasks.workunit.client.0.smithi102.stdout:6/578: dwrite d9/d45/d56/d5d/f9f [0,4194304] 0 2024-05-13T22:48:10.011 INFO:tasks.workunit.client.0.smithi102.stdout:0/535: sync 2024-05-13T22:48:10.012 INFO:tasks.workunit.client.0.smithi102.stdout:9/552: getdents d1/d6/db/d77 0 2024-05-13T22:48:10.014 INFO:tasks.workunit.client.0.smithi102.stdout:9/553: chown d1/d6/d45/d3e/d79 3053159 1 2024-05-13T22:48:10.021 INFO:tasks.workunit.client.0.smithi102.stdout:1/380: unlink d2/d4/d5/f6d 0 2024-05-13T22:48:10.027 INFO:tasks.workunit.client.0.smithi102.stdout:6/579: write d9/d45/d56/d5d/f87 [2223675,20854] 0 2024-05-13T22:48:10.027 INFO:tasks.workunit.client.0.smithi102.stdout:8/442: write d2/d36/d49/d5d/f26 [1805667,65030] 0 2024-05-13T22:48:10.031 INFO:tasks.workunit.client.0.smithi102.stdout:5/685: write f6 [1302534,28349] 0 2024-05-13T22:48:10.031 INFO:tasks.workunit.client.0.smithi102.stdout:6/580: write d9/d45/d56/d5d/f9f [2102629,40174] 0 2024-05-13T22:48:10.034 INFO:tasks.workunit.client.0.smithi102.stdout:1/381: rename d2/d10/c32 to d2/d4/d5/d19/d28/d6a/c7e 0 2024-05-13T22:48:10.035 INFO:tasks.workunit.client.0.smithi102.stdout:4/561: dwrite d3/d4/fd [0,4194304] 0 2024-05-13T22:48:10.036 INFO:tasks.workunit.client.0.smithi102.stdout:2/514: dread d7/d15/fa5 [4194304,4194304] 0 2024-05-13T22:48:10.040 INFO:tasks.workunit.client.0.smithi102.stdout:0/536: mknod d1/de/d32/d8a/ca8 0 2024-05-13T22:48:10.044 INFO:tasks.workunit.client.0.smithi102.stdout:5/686: mkdir de/d25/d59/d7f/d86/da5/ddb/d103 0 2024-05-13T22:48:10.046 INFO:tasks.workunit.client.0.smithi102.stdout:2/515: symlink d7/d9/laa 0 2024-05-13T22:48:10.048 INFO:tasks.workunit.client.0.smithi102.stdout:6/581: dwrite d9/db/d14/f3f [0,4194304] 0 2024-05-13T22:48:10.051 INFO:tasks.workunit.client.0.smithi102.stdout:4/562: dwrite d3/d4/d6/f18 [4194304,4194304] 0 2024-05-13T22:48:10.056 INFO:tasks.workunit.client.0.smithi102.stdout:9/554: dwrite d1/d6/d45/d3e/f4c [0,4194304] 0 2024-05-13T22:48:10.057 INFO:tasks.workunit.client.0.smithi102.stdout:8/443: getdents d2 0 2024-05-13T22:48:10.058 INFO:tasks.workunit.client.0.smithi102.stdout:6/582: truncate d9/db/d2f/f62 5074075 0 2024-05-13T22:48:10.060 INFO:tasks.workunit.client.0.smithi102.stdout:6/583: dread - d9/db/d14/fa6 zero size 2024-05-13T22:48:10.062 INFO:tasks.workunit.client.0.smithi102.stdout:4/563: fdatasync d3/d4/d6/de/f3f 0 2024-05-13T22:48:10.064 INFO:tasks.workunit.client.0.smithi102.stdout:5/687: write de/d25/d2a/f2b [1244385,86663] 0 2024-05-13T22:48:10.065 INFO:tasks.workunit.client.0.smithi102.stdout:8/444: creat d2/d7/d25/d41/d6f/f84 x:0 0 0 2024-05-13T22:48:10.066 INFO:tasks.workunit.client.0.smithi102.stdout:5/688: read - de/d33/d34/ff0 zero size 2024-05-13T22:48:10.068 INFO:tasks.workunit.client.0.smithi102.stdout:0/537: getdents d1/d4d/d79 0 2024-05-13T22:48:10.070 INFO:tasks.workunit.client.0.smithi102.stdout:0/538: write d1/de/d42/f93 [4253559,56484] 0 2024-05-13T22:48:10.073 INFO:tasks.workunit.client.0.smithi102.stdout:4/564: symlink d3/d22/dab/lc4 0 2024-05-13T22:48:10.076 INFO:tasks.workunit.client.0.smithi102.stdout:8/445: mkdir d2/d7/d25/d41/d50/d40/d85 0 2024-05-13T22:48:10.077 INFO:tasks.workunit.client.0.smithi102.stdout:2/516: mkdir d7/d9/d10/d25/d27/dab 0 2024-05-13T22:48:10.081 INFO:tasks.workunit.client.0.smithi102.stdout:6/584: dwrite d9/db/f77 [4194304,4194304] 0 2024-05-13T22:48:10.082 INFO:tasks.workunit.client.0.smithi102.stdout:0/539: mknod d1/de/d32/d24/d2d/d3e/ca9 0 2024-05-13T22:48:10.085 INFO:tasks.workunit.client.0.smithi102.stdout:9/555: dwrite d1/d6/d45/d3e/f63 [0,4194304] 0 2024-05-13T22:48:10.091 INFO:tasks.workunit.client.0.smithi102.stdout:8/446: symlink d2/d36/d49/l86 0 2024-05-13T22:48:10.092 INFO:tasks.workunit.client.0.smithi102.stdout:9/556: creat d1/d6/db/d4a/fc4 x:0 0 0 2024-05-13T22:48:10.097 INFO:tasks.workunit.client.0.smithi102.stdout:9/557: chown d1/f16 100379 1 2024-05-13T22:48:10.098 INFO:tasks.workunit.client.0.smithi102.stdout:8/447: write d2/d7/d25/f74 [2226391,78599] 0 2024-05-13T22:48:10.099 INFO:tasks.workunit.client.0.smithi102.stdout:0/540: rename d1/f82 to d1/de/faa 0 2024-05-13T22:48:10.104 INFO:tasks.workunit.client.0.smithi102.stdout:9/558: readlink d1/d6/db/d30/d47/l6e 0 2024-05-13T22:48:10.104 INFO:tasks.workunit.client.0.smithi102.stdout:6/585: dwrite d9/d25/d60/f7e [0,4194304] 0 2024-05-13T22:48:10.106 INFO:tasks.workunit.client.0.smithi102.stdout:9/559: stat d1/c8e 0 2024-05-13T22:48:10.106 INFO:tasks.workunit.client.0.smithi102.stdout:4/565: dwrite d3/d19/d23/d39/fc1 [0,4194304] 0 2024-05-13T22:48:10.108 INFO:tasks.workunit.client.0.smithi102.stdout:2/517: dwrite d7/d9/d14/d31/f77 [0,4194304] 0 2024-05-13T22:48:10.110 INFO:tasks.workunit.client.0.smithi102.stdout:8/448: creat d2/f87 x:0 0 0 2024-05-13T22:48:10.111 INFO:tasks.workunit.client.0.smithi102.stdout:2/518: dread - d7/d9/d10/d30/fa9 zero size 2024-05-13T22:48:10.112 INFO:tasks.workunit.client.0.smithi102.stdout:2/519: chown d7/c94 13764 1 2024-05-13T22:48:10.113 INFO:tasks.workunit.client.0.smithi102.stdout:6/586: unlink d9/db/d14/f3f 0 2024-05-13T22:48:10.114 INFO:tasks.workunit.client.0.smithi102.stdout:6/587: write d9/db/f50 [2889139,54445] 0 2024-05-13T22:48:10.118 INFO:tasks.workunit.client.0.smithi102.stdout:6/588: write d9/d25/f58 [367474,115703] 0 2024-05-13T22:48:10.120 INFO:tasks.workunit.client.0.smithi102.stdout:6/589: read - d9/d25/d60/d6a/f89 zero size 2024-05-13T22:48:10.127 INFO:tasks.workunit.client.0.smithi102.stdout:9/560: getdents d1/d6/d45/d2d/dac/db0/db8 0 2024-05-13T22:48:10.128 INFO:tasks.workunit.client.0.smithi102.stdout:4/566: dwrite d3/d22/dab/fbb [0,4194304] 0 2024-05-13T22:48:10.130 INFO:tasks.workunit.client.0.smithi102.stdout:5/689: dread de/d25/d59/f5d [0,4194304] 0 2024-05-13T22:48:10.130 INFO:tasks.workunit.client.0.smithi102.stdout:2/520: dwrite d7/d9/d10/d30/f80 [0,4194304] 0 2024-05-13T22:48:10.131 INFO:tasks.workunit.client.0.smithi102.stdout:4/567: stat d3/d19/d23/d3a/f9d 0 2024-05-13T22:48:10.133 INFO:tasks.workunit.client.0.smithi102.stdout:6/590: symlink d9/db/d14/d54/lc7 0 2024-05-13T22:48:10.137 INFO:tasks.workunit.client.0.smithi102.stdout:9/561: unlink d1/d5d/f74 0 2024-05-13T22:48:10.140 INFO:tasks.workunit.client.0.smithi102.stdout:4/568: creat d3/d19/d23/d39/da7/d7d/fc5 x:0 0 0 2024-05-13T22:48:10.142 INFO:tasks.workunit.client.0.smithi102.stdout:5/690: mkdir de/d25/d43/d7a/db1/d104 0 2024-05-13T22:48:10.144 INFO:tasks.workunit.client.0.smithi102.stdout:5/691: stat de/d25/d43/d7a/db1/dc5/dca/ce2 0 2024-05-13T22:48:10.148 INFO:tasks.workunit.client.0.smithi102.stdout:2/521: symlink d7/lac 0 2024-05-13T22:48:10.149 INFO:tasks.workunit.client.0.smithi102.stdout:2/522: readlink d7/d9/d10/d1b/d54/d83/l9a 0 2024-05-13T22:48:10.150 INFO:tasks.workunit.client.0.smithi102.stdout:2/523: chown d7/d9/d14/f73 526 1 2024-05-13T22:48:10.151 INFO:tasks.workunit.client.0.smithi102.stdout:9/562: read - d1/d6/d21/f48 zero size 2024-05-13T22:48:10.152 INFO:tasks.workunit.client.0.smithi102.stdout:9/563: chown d1/d6/db/d4a/d64/f9a 3 1 2024-05-13T22:48:10.154 INFO:tasks.workunit.client.0.smithi102.stdout:4/569: creat d3/d6b/fc6 x:0 0 0 2024-05-13T22:48:10.155 INFO:tasks.workunit.client.0.smithi102.stdout:4/570: dread - d3/d4/d6/de/faa zero size 2024-05-13T22:48:10.158 INFO:tasks.workunit.client.0.smithi102.stdout:5/692: mkdir de/d25/d59/d7f/d86/da5/d105 0 2024-05-13T22:48:10.159 INFO:tasks.workunit.client.0.smithi102.stdout:5/693: chown de/d25/d58/l90 7 1 2024-05-13T22:48:10.163 INFO:tasks.workunit.client.0.smithi102.stdout:8/449: mknod d2/d7/d25/c88 0 2024-05-13T22:48:10.166 INFO:tasks.workunit.client.0.smithi102.stdout:9/564: unlink d1/d6/d45/d3e/fad 0 2024-05-13T22:48:10.166 INFO:tasks.workunit.client.0.smithi102.stdout:5/694: dwrite de/d25/d2a/d4b/dab/fd4 [0,4194304] 0 2024-05-13T22:48:10.167 INFO:tasks.workunit.client.0.smithi102.stdout:5/695: readlink de/d33/l36 0 2024-05-13T22:48:10.172 INFO:tasks.workunit.client.0.smithi102.stdout:2/524: fdatasync d7/d9/d10/d1b/f6b 0 2024-05-13T22:48:10.178 INFO:tasks.workunit.client.0.smithi102.stdout:6/591: dwrite d9/d45/d56/f9b [0,4194304] 0 2024-05-13T22:48:10.181 INFO:tasks.workunit.client.0.smithi102.stdout:8/450: dread d2/d36/d49/d5d/f45 [0,4194304] 0 2024-05-13T22:48:10.185 INFO:tasks.workunit.client.0.smithi102.stdout:2/525: creat d7/d15/fad x:0 0 0 2024-05-13T22:48:10.188 INFO:tasks.workunit.client.0.smithi102.stdout:6/592: rename d9/db/d14/d35/d47/cab to d9/db/d14/d35/d47/cc8 0 2024-05-13T22:48:10.190 INFO:tasks.workunit.client.0.smithi102.stdout:9/565: creat d1/d6/db/d4a/d75/db9/fc5 x:0 0 0 2024-05-13T22:48:10.191 INFO:tasks.workunit.client.0.smithi102.stdout:9/566: chown d1/d6/db/d30/d47/f6d 243568 1 2024-05-13T22:48:10.191 INFO:tasks.workunit.client.0.smithi102.stdout:9/567: dread - d1/d6/db/d30/f95 zero size 2024-05-13T22:48:10.196 INFO:tasks.workunit.client.0.smithi102.stdout:2/526: creat d7/d15/fae x:0 0 0 2024-05-13T22:48:10.198 INFO:tasks.workunit.client.0.smithi102.stdout:4/571: getdents d3/d4/d6 0 2024-05-13T22:48:10.201 INFO:tasks.workunit.client.0.smithi102.stdout:6/593: mkdir d9/db/d14/d35/dc9 0 2024-05-13T22:48:10.203 INFO:tasks.workunit.client.0.smithi102.stdout:9/568: mkdir d1/d6/db/d4a/d75/d96/dc6 0 2024-05-13T22:48:10.203 INFO:tasks.workunit.client.0.smithi102.stdout:6/594: chown d9/db/d14/c4a 195765910 1 2024-05-13T22:48:10.205 INFO:tasks.workunit.client.0.smithi102.stdout:2/527: dwrite d7/d9/d14/d31/f64 [0,4194304] 0 2024-05-13T22:48:10.207 INFO:tasks.workunit.client.0.smithi102.stdout:5/696: dread de/d25/d2a/d4b/d4d/d50/da3/df4/f55 [0,4194304] 0 2024-05-13T22:48:10.208 INFO:tasks.workunit.client.0.smithi102.stdout:8/451: getdents d2/d36/d49 0 2024-05-13T22:48:10.209 INFO:tasks.workunit.client.0.smithi102.stdout:5/697: chown de/d25/d2a/l69 175967983 1 2024-05-13T22:48:10.215 INFO:tasks.workunit.client.0.smithi102.stdout:2/528: dread d7/d15/f52 [0,4194304] 0 2024-05-13T22:48:10.219 INFO:tasks.workunit.client.0.smithi102.stdout:7/394: sync 2024-05-13T22:48:10.219 INFO:tasks.workunit.client.0.smithi102.stdout:3/576: sync 2024-05-13T22:48:10.226 INFO:tasks.workunit.client.0.smithi102.stdout:8/452: rmdir d2/d7/d25/d41 39 2024-05-13T22:48:10.234 INFO:tasks.workunit.client.0.smithi102.stdout:5/698: dread de/d25/d2c/f78 [0,4194304] 0 2024-05-13T22:48:10.235 INFO:tasks.workunit.client.0.smithi102.stdout:6/595: dwrite f7 [4194304,4194304] 0 2024-05-13T22:48:10.239 INFO:tasks.workunit.client.0.smithi102.stdout:2/529: mknod d7/d9/d10/d25/d27/dab/caf 0 2024-05-13T22:48:10.239 INFO:tasks.workunit.client.0.smithi102.stdout:8/453: truncate d2/d7/d25/d41/d50/d40/f7f 458776 0 2024-05-13T22:48:10.240 INFO:tasks.workunit.client.0.smithi102.stdout:7/395: mkdir d6/d13/d2c/d32/d6e 0 2024-05-13T22:48:10.247 INFO:tasks.workunit.client.0.smithi102.stdout:6/596: fsync d9/f10 0 2024-05-13T22:48:10.250 INFO:tasks.workunit.client.0.smithi102.stdout:2/530: creat d7/d9/d10/d25/d56/d59/fb0 x:0 0 0 2024-05-13T22:48:10.252 INFO:tasks.workunit.client.0.smithi102.stdout:7/396: stat d6/d13/d58/d20/l45 0 2024-05-13T22:48:10.255 INFO:tasks.workunit.client.0.smithi102.stdout:9/569: creat d1/d6/d45/d2d/dac/db0/db8/fc7 x:0 0 0 2024-05-13T22:48:10.256 INFO:tasks.workunit.client.0.smithi102.stdout:9/570: truncate d1/d6b/f9e 783197 0 2024-05-13T22:48:10.261 INFO:tasks.workunit.client.0.smithi102.stdout:6/597: symlink d9/d45/d56/lca 0 2024-05-13T22:48:10.263 INFO:tasks.workunit.client.0.smithi102.stdout:9/571: dwrite d1/d6/db/d30/d47/f49 [0,4194304] 0 2024-05-13T22:48:10.264 INFO:tasks.workunit.client.0.smithi102.stdout:8/454: symlink d2/d7/d25/l89 0 2024-05-13T22:48:10.264 INFO:tasks.workunit.client.0.smithi102.stdout:9/572: stat d1/d6/db/d4a/d75/d96/dc6 0 2024-05-13T22:48:10.267 INFO:tasks.workunit.client.0.smithi102.stdout:3/577: truncate de/d10/d2a/f2e 2299549 0 2024-05-13T22:48:10.268 INFO:tasks.workunit.client.0.smithi102.stdout:9/573: fdatasync d1/d6/db/d30/d47/f6d 0 2024-05-13T22:48:10.272 INFO:tasks.workunit.client.0.smithi102.stdout:7/397: rename d6/d13/d2c/d32 to d6/d15/d4d/d55/d6f 0 2024-05-13T22:48:10.274 INFO:tasks.workunit.client.0.smithi102.stdout:8/455: dwrite d2/d7/d27/f77 [0,4194304] 0 2024-05-13T22:48:10.278 INFO:tasks.workunit.client.0.smithi102.stdout:2/531: dread d7/d15/d1e/f26 [0,4194304] 0 2024-05-13T22:48:10.285 INFO:tasks.workunit.client.0.smithi102.stdout:5/699: dwrite de/fc4 [0,4194304] 0 2024-05-13T22:48:10.285 INFO:tasks.workunit.client.0.smithi102.stdout:3/578: chown de/d68/l84 282 1 2024-05-13T22:48:10.287 INFO:tasks.workunit.client.0.smithi102.stdout:5/700: truncate de/d25/d2a/f2b 4277052 0 2024-05-13T22:48:10.288 INFO:tasks.workunit.client.0.smithi102.stdout:5/701: readlink de/d33/d53/ld2 0 2024-05-13T22:48:10.291 INFO:tasks.workunit.client.0.smithi102.stdout:9/574: mknod d1/d6/db/d4a/cc8 0 2024-05-13T22:48:10.295 INFO:tasks.workunit.client.0.smithi102.stdout:6/598: rename d9/db/l98 to d9/da3/lcb 0 2024-05-13T22:48:10.296 INFO:tasks.workunit.client.0.smithi102.stdout:6/599: readlink d9/db/d14/d54/lc7 0 2024-05-13T22:48:10.297 INFO:tasks.workunit.client.0.smithi102.stdout:6/600: dread - d9/d25/d5a/fb9 zero size 2024-05-13T22:48:10.309 INFO:tasks.workunit.client.0.smithi102.stdout:3/579: dread de/d10/d50/d63/f85 [0,4194304] 0 2024-05-13T22:48:10.314 INFO:tasks.workunit.client.0.smithi102.stdout:5/702: unlink de/d25/d58/c7d 0 2024-05-13T22:48:10.320 INFO:tasks.workunit.client.0.smithi102.stdout:2/532: rename d7/d9/d14/l3c to d7/d9/d10/d25/lb1 0 2024-05-13T22:48:10.321 INFO:tasks.workunit.client.0.smithi102.stdout:2/533: write d7/d9/d14/d31/f77 [2411944,69479] 0 2024-05-13T22:48:10.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.349+0000 7fe331476700 1 -- 172.21.15.102:0/3775782367 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe32c068b20 con 0x7fe31406d330 2024-05-13T22:48:10.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.350+0000 7fe313fff700 1 -- 172.21.15.102:0/3775782367 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fe32c068b20 con 0x7fe31406d330 2024-05-13T22:48:10.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.353+0000 7fe331476700 1 -- 172.21.15.102:0/3775782367 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fe31406d330 msgr2=0x7fe31406f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:10.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.353+0000 7fe331476700 1 --2- 172.21.15.102:0/3775782367 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fe31406d330 0x7fe31406f7f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe32c100b10 tx=0x7fe31c006cd0 comp rx=0 tx=0).stop 2024-05-13T22:48:10.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.353+0000 7fe331476700 1 -- 172.21.15.102:0/3775782367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe32c100cb0 msgr2=0x7fe32c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:10.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.353+0000 7fe331476700 1 --2- 172.21.15.102:0/3775782367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe32c100cb0 0x7fe32c1020b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe320000c00 tx=0x7fe32000be20 comp rx=0 tx=0).stop 2024-05-13T22:48:10.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.353+0000 7fe331476700 1 -- 172.21.15.102:0/3775782367 shutdown_connections 2024-05-13T22:48:10.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.353+0000 7fe331476700 1 --2- 172.21.15.102:0/3775782367 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7fe31406d330 0x7fe31406f7f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:10.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.353+0000 7fe331476700 1 --2- 172.21.15.102:0/3775782367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe32c0ffab0 0x7fe32c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:10.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.353+0000 7fe331476700 1 --2- 172.21.15.102:0/3775782367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe32c100cb0 0x7fe32c1020b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:10.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.353+0000 7fe331476700 1 -- 172.21.15.102:0/3775782367 >> 172.21.15.102:0/3775782367 conn(0x7fe32c0fb030 msgr2=0x7fe32c068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:10.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.353+0000 7fe331476700 1 -- 172.21.15.102:0/3775782367 shutdown_connections 2024-05-13T22:48:10.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.353+0000 7fe331476700 1 -- 172.21.15.102:0/3775782367 wait complete. 2024-05-13T22:48:10.336 INFO:tasks.workunit.client.0.smithi102.stdout:8/456: dwrite d2/d7/d12/f13 [0,4194304] 0 2024-05-13T22:48:10.344 INFO:tasks.workunit.client.0.smithi102.stdout:5/703: creat de/d25/d2a/def/f106 x:0 0 0 2024-05-13T22:48:10.345 INFO:tasks.workunit.client.0.smithi102.stdout:5/704: chown de/d25/d2a/d4b/d4d/d50/d99/dbb 8173353 1 2024-05-13T22:48:10.346 INFO:tasks.workunit.client.0.smithi102.stdout:9/575: link d1/d6/d21/l23 d1/d6/d21/d71/lc9 0 2024-05-13T22:48:10.352 INFO:tasks.workunit.client.0.smithi102.stdout:8/457: chown d2/f4 30 1 2024-05-13T22:48:10.353 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:48:10.353 INFO:tasks.workunit.client.0.smithi102.stdout:7/398: truncate d6/d15/f3b 567986 0 2024-05-13T22:48:10.355 INFO:tasks.workunit.client.0.smithi102.stdout:7/399: write d6/d15/f18 [6846185,28955] 0 2024-05-13T22:48:10.361 INFO:tasks.workunit.client.0.smithi102.stdout:5/705: creat de/d33/d53/f107 x:0 0 0 2024-05-13T22:48:10.361 INFO:tasks.workunit.client.0.smithi102.stdout:3/580: write de/d10/d50/f47 [1487943,31756] 0 2024-05-13T22:48:10.362 INFO:tasks.workunit.client.0.smithi102.stdout:3/581: chown de/d10/d50/d63/da2 823 1 2024-05-13T22:48:10.368 INFO:tasks.workunit.client.0.smithi102.stdout:6/601: rename d9/d25/db5 to d9/db/d14/dcc 0 2024-05-13T22:48:10.369 INFO:tasks.workunit.client.0.smithi102.stdout:6/602: write d9/d25/d60/f7e [3529601,33488] 0 2024-05-13T22:48:10.370 INFO:tasks.workunit.client.0.smithi102.stdout:8/458: readlink d2/l1b 0 2024-05-13T22:48:10.371 INFO:tasks.workunit.client.0.smithi102.stdout:6/603: dread - d9/db/d14/d35/d47/fa8 zero size 2024-05-13T22:48:10.392 INFO:tasks.workunit.client.0.smithi102.stdout:1/382: sync 2024-05-13T22:48:10.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.478+0000 7f82a4241700 1 -- 172.21.15.102:0/550998290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f829c101140 msgr2=0x7f829c101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:10.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.478+0000 7f82a4241700 1 --2- 172.21.15.102:0/550998290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f829c101140 0x7f829c101560 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8298009910 tx=0x7f8298009c20 comp rx=0 tx=0).stop 2024-05-13T22:48:10.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.479+0000 7f82a4241700 1 -- 172.21.15.102:0/550998290 shutdown_connections 2024-05-13T22:48:10.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.479+0000 7f82a4241700 1 --2- 172.21.15.102:0/550998290 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f829c0ffd10 0x7f829c100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:10.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.479+0000 7f82a4241700 1 --2- 172.21.15.102:0/550998290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f829c101140 0x7f829c101560 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:10.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.479+0000 7f82a4241700 1 -- 172.21.15.102:0/550998290 >> 172.21.15.102:0/550998290 conn(0x7f829c0fa7c0 msgr2=0x7f829c0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:10.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.479+0000 7f82a4241700 1 -- 172.21.15.102:0/550998290 shutdown_connections 2024-05-13T22:48:10.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.479+0000 7f82a4241700 1 -- 172.21.15.102:0/550998290 wait complete. 2024-05-13T22:48:10.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.479+0000 7f82a4241700 1 Processor -- start 2024-05-13T22:48:10.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.479+0000 7f82a4241700 1 -- start start 2024-05-13T22:48:10.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.479+0000 7f82a4241700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f829c0ffd10 0x7f829c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:10.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a4241700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f829c101140 0x7f829c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:10.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a4241700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c199130 con 0x7f829c0ffd10 2024-05-13T22:48:10.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a4241700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c199270 con 0x7f829c101140 2024-05-13T22:48:10.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a1fdd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f829c0ffd10 0x7f829c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:10.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a1fdd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f829c0ffd10 0x7f829c1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34810/0 (socket says 172.21.15.102:34810) 2024-05-13T22:48:10.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a1fdd700 1 -- 172.21.15.102:0/3978567901 learned_addr learned my addr 172.21.15.102:0/3978567901 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:48:10.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a17dc700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f829c101140 0x7f829c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:10.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a1fdd700 1 -- 172.21.15.102:0/3978567901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f829c101140 msgr2=0x7f829c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:10.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a1fdd700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f829c101140 0x7f829c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:10.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a1fdd700 1 -- 172.21.15.102:0/3978567901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82980095f0 con 0x7f829c0ffd10 2024-05-13T22:48:10.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a17dc700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f829c101140 0x7f829c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:48:10.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a1fdd700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f829c0ffd10 0x7f829c1985d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8298000c00 tx=0x7f829800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:10.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f8292ffd700 1 -- 172.21.15.102:0/3978567901 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f829801d070 con 0x7f829c0ffd10 2024-05-13T22:48:10.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f82a4241700 1 -- 172.21.15.102:0/3978567901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829c19dc70 con 0x7f829c0ffd10 2024-05-13T22:48:10.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f8292ffd700 1 -- 172.21.15.102:0/3978567901 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8298005dc0 con 0x7f829c0ffd10 2024-05-13T22:48:10.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.480+0000 7f8292ffd700 1 -- 172.21.15.102:0/3978567901 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82980226a0 con 0x7f829c0ffd10 2024-05-13T22:48:10.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.481+0000 7f82a4241700 1 -- 172.21.15.102:0/3978567901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c19e160 con 0x7f829c0ffd10 2024-05-13T22:48:10.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.481+0000 7f82a4241700 1 -- 172.21.15.102:0/3978567901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829c1086e0 con 0x7f829c0ffd10 2024-05-13T22:48:10.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.482+0000 7f8292ffd700 1 -- 172.21.15.102:0/3978567901 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) v1 ==== 91288+0+0 (secure 0 0 0) 0x7f82980038f0 con 0x7f829c0ffd10 2024-05-13T22:48:10.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.483+0000 7f8292ffd700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f828806d2f0 0x7f828806f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:10.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.483+0000 7f8292ffd700 1 -- 172.21.15.102:0/3978567901 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f829808e1d0 con 0x7f829c0ffd10 2024-05-13T22:48:10.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.483+0000 7f82a17dc700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f828806d2f0 0x7f828806f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:10.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.483+0000 7f82a17dc700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f828806d2f0 0x7f828806f7b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f828c011490 tx=0x7f828c009260 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:10.468 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.489+0000 7f8292ffd700 1 -- 172.21.15.102:0/3978567901 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f829805b440 con 0x7f829c0ffd10 2024-05-13T22:48:10.471 INFO:tasks.workunit.client.0.smithi102.stdout:2/534: symlink d7/d9/d11/lb2 0 2024-05-13T22:48:10.472 INFO:tasks.workunit.client.0.smithi102.stdout:3/582: mknod de/d1c/d90/cd1 0 2024-05-13T22:48:10.473 INFO:tasks.workunit.client.0.smithi102.stdout:5/706: fdatasync de/d25/d2a/d4b/d4d/d50/d6c/df8/fa7 0 2024-05-13T22:48:10.473 INFO:tasks.workunit.client.0.smithi102.stdout:6/604: dread - d9/db/d14/d35/d47/dc1/fc6 zero size 2024-05-13T22:48:10.473 INFO:tasks.workunit.client.0.smithi102.stdout:2/535: chown d7/f32 225890015 1 2024-05-13T22:48:10.474 INFO:tasks.workunit.client.0.smithi102.stdout:9/576: mknod d1/d6/db/d4a/d64/dc0/cca 0 2024-05-13T22:48:10.476 INFO:tasks.workunit.client.0.smithi102.stdout:9/577: fsync d1/d6/db/d4a/fc4 0 2024-05-13T22:48:10.476 INFO:tasks.workunit.client.0.smithi102.stdout:2/536: fsync d7/d9/d14/d61/f93 0 2024-05-13T22:48:10.486 INFO:tasks.workunit.client.0.smithi102.stdout:1/383: rmdir d2/d4/d26/d40/d5e 39 2024-05-13T22:48:10.489 INFO:tasks.workunit.client.0.smithi102.stdout:9/578: dread d1/d6/d21/d71/f9d [0,4194304] 0 2024-05-13T22:48:10.490 INFO:tasks.workunit.client.0.smithi102.stdout:3/583: creat de/d10/d2a/d36/db2/fd2 x:0 0 0 2024-05-13T22:48:10.491 INFO:tasks.workunit.client.0.smithi102.stdout:9/579: write d1/d6/fa5 [1085534,70832] 0 2024-05-13T22:48:10.497 INFO:tasks.workunit.client.0.smithi102.stdout:3/584: dwrite de/d10/d50/f47 [0,4194304] 0 2024-05-13T22:48:10.498 INFO:tasks.workunit.client.0.smithi102.stdout:7/400: creat d6/f70 x:0 0 0 2024-05-13T22:48:10.507 INFO:tasks.workunit.client.0.smithi102.stdout:9/580: dread d1/d6/d21/d71/fb3 [0,4194304] 0 2024-05-13T22:48:10.515 INFO:tasks.workunit.client.0.smithi102.stdout:3/585: unlink de/d1c/d90/f92 0 2024-05-13T22:48:10.518 INFO:tasks.workunit.client.0.smithi102.stdout:1/384: truncate d2/d10/d62/d7b/f7c 1719804 0 2024-05-13T22:48:10.520 INFO:tasks.workunit.client.0.smithi102.stdout:6/605: creat d9/db/d2f/fcd x:0 0 0 2024-05-13T22:48:10.520 INFO:tasks.workunit.client.0.smithi102.stdout:3/586: fsync de/d10/d50/f47 0 2024-05-13T22:48:10.525 INFO:tasks.workunit.client.0.smithi102.stdout:5/707: rename de/d25/d43/d7a/db1 to de/d25/d108 0 2024-05-13T22:48:10.525 INFO:tasks.workunit.client.0.smithi102.stdout:9/581: rename d1 to d1/d6/d21/d71/dcb 22 2024-05-13T22:48:10.526 INFO:tasks.workunit.client.0.smithi102.stdout:9/582: fdatasync d1/f16 0 2024-05-13T22:48:10.526 INFO:tasks.workunit.client.0.smithi102.stdout:5/708: write de/d25/fe0 [319410,104429] 0 2024-05-13T22:48:10.526 INFO:tasks.workunit.client.0.smithi102.stdout:2/537: creat d7/d9/d10/fb3 x:0 0 0 2024-05-13T22:48:10.530 INFO:tasks.workunit.client.0.smithi102.stdout:7/401: link d6/d13/d58/d20/d4b/f6c d6/d15/d4d/d55/d6f/d49/f71 0 2024-05-13T22:48:10.531 INFO:tasks.workunit.client.0.smithi102.stdout:7/402: chown d6/f70 4966458 1 2024-05-13T22:48:10.536 INFO:tasks.workunit.client.0.smithi102.stdout:2/538: fsync d7/d9/f29 0 2024-05-13T22:48:10.539 INFO:tasks.workunit.client.0.smithi102.stdout:5/709: rename de/d25/d2a/d4b/dab/fd4 to de/d25/d2a/d4b/d4d/d50/f109 0 2024-05-13T22:48:10.541 INFO:tasks.workunit.client.0.smithi102.stdout:7/403: rmdir d6/d13/d2c 39 2024-05-13T22:48:10.544 INFO:tasks.workunit.client.0.smithi102.stdout:6/606: mknod d9/d45/cce 0 2024-05-13T22:48:10.545 INFO:tasks.workunit.client.0.smithi102.stdout:6/607: chown d9/d25/d5a/f6d 4851 1 2024-05-13T22:48:10.546 INFO:tasks.workunit.client.0.smithi102.stdout:6/608: write d9/f21 [360318,116713] 0 2024-05-13T22:48:10.547 INFO:tasks.workunit.client.0.smithi102.stdout:9/583: dread d1/f7 [0,4194304] 0 2024-05-13T22:48:10.550 INFO:tasks.workunit.client.0.smithi102.stdout:5/710: creat de/d25/d2a/d4b/f10a x:0 0 0 2024-05-13T22:48:10.551 INFO:tasks.workunit.client.0.smithi102.stdout:5/711: stat de/d25/d2a/d4b/d4d/d50/da3/df4/f66 0 2024-05-13T22:48:10.554 INFO:tasks.workunit.client.0.smithi102.stdout:6/609: unlink d9/d25/d60/d6a/ca2 0 2024-05-13T22:48:10.554 INFO:tasks.workunit.client.0.smithi102.stdout:6/610: fdatasync d9/db/d14/fa6 0 2024-05-13T22:48:10.558 INFO:tasks.workunit.client.0.smithi102.stdout:5/712: symlink de/d25/d43/df5/l10b 0 2024-05-13T22:48:10.559 INFO:tasks.workunit.client.0.smithi102.stdout:9/584: symlink d1/d6/db/lcc 0 2024-05-13T22:48:10.562 INFO:tasks.workunit.client.0.smithi102.stdout:7/404: creat d6/f72 x:0 0 0 2024-05-13T22:48:10.562 INFO:tasks.workunit.client.0.smithi102.stdout:6/611: dwrite d9/db/d14/d35/f94 [0,4194304] 0 2024-05-13T22:48:10.565 INFO:tasks.workunit.client.0.smithi102.stdout:6/612: dread - d9/db/d14/d35/d47/f8f zero size 2024-05-13T22:48:10.565 INFO:tasks.workunit.client.0.smithi102.stdout:6/613: chown d9/db/d14/dcc/d5f 261 1 2024-05-13T22:48:10.566 INFO:tasks.workunit.client.0.smithi102.stdout:6/614: fdatasync d9/db/d14/d35/fc2 0 2024-05-13T22:48:10.568 INFO:tasks.workunit.client.0.smithi102.stdout:7/405: truncate d6/d15/f3e 814139 0 2024-05-13T22:48:10.572 INFO:tasks.workunit.client.0.smithi102.stdout:6/615: creat d9/db/d14/d35/d47/fcf x:0 0 0 2024-05-13T22:48:10.576 INFO:tasks.workunit.client.0.smithi102.stdout:6/616: mknod d9/db/d14/d35/dc9/cd0 0 2024-05-13T22:48:10.583 INFO:tasks.workunit.client.0.smithi102.stdout:5/713: dread de/d25/d2a/d4b/d4d/d50/d99/dbb/dda/fe3 [0,4194304] 0 2024-05-13T22:48:10.584 INFO:tasks.workunit.client.0.smithi102.stdout:6/617: dwrite d9/d25/f30 [0,4194304] 0 2024-05-13T22:48:10.589 INFO:tasks.workunit.client.0.smithi102.stdout:5/714: fdatasync de/d25/d59/d6f/f92 0 2024-05-13T22:48:10.589 INFO:tasks.workunit.client.0.smithi102.stdout:5/715: stat de/f4f 0 2024-05-13T22:48:10.595 INFO:tasks.workunit.client.0.smithi102.stdout:5/716: rmdir de/d25/d2a/d4b/d4d/d50/d99 39 2024-05-13T22:48:10.596 INFO:tasks.workunit.client.0.smithi102.stdout:5/717: readlink de/d25/d59/d7f/lac 0 2024-05-13T22:48:10.597 INFO:tasks.workunit.client.0.smithi102.stdout:5/718: chown de/f4f 7 1 2024-05-13T22:48:10.600 INFO:tasks.workunit.client.0.smithi102.stdout:5/719: symlink de/d25/d108/dc5/dca/l10c 0 2024-05-13T22:48:10.601 INFO:tasks.workunit.client.0.smithi102.stdout:5/720: chown de/d25/d2a/d4b/d4d/d50/da3/df4/f55 252674064 1 2024-05-13T22:48:10.602 INFO:tasks.workunit.client.0.smithi102.stdout:5/721: chown de/f5a 1 1 2024-05-13T22:48:10.604 INFO:tasks.workunit.client.0.smithi102.stdout:5/722: creat de/d33/d53/f10d x:0 0 0 2024-05-13T22:48:10.606 INFO:tasks.workunit.client.0.smithi102.stdout:5/723: write de/d25/d2a/def/f106 [130030,20699] 0 2024-05-13T22:48:10.607 INFO:tasks.workunit.client.0.smithi102.stdout:6/618: dread f0 [0,4194304] 0 2024-05-13T22:48:10.611 INFO:tasks.workunit.client.0.smithi102.stdout:6/619: rmdir d9/db/d14/d35/dc9 39 2024-05-13T22:48:10.611 INFO:tasks.workunit.client.0.smithi102.stdout:6/620: stat d9/d25/f40 0 2024-05-13T22:48:10.612 INFO:tasks.workunit.client.0.smithi102.stdout:5/724: rmdir de/d25/d2a/d4b/d4d/d50/d99/dbb 39 2024-05-13T22:48:10.613 INFO:tasks.workunit.client.0.smithi102.stdout:5/725: chown de/d33/d34/c3b 1055948 1 2024-05-13T22:48:10.616 INFO:tasks.workunit.client.0.smithi102.stdout:6/621: mkdir d9/db/d14/d35/dd1 0 2024-05-13T22:48:10.620 INFO:tasks.workunit.client.0.smithi102.stdout:6/622: creat d9/db/d14/d35/dc9/fd2 x:0 0 0 2024-05-13T22:48:10.626 INFO:tasks.workunit.client.0.smithi102.stdout:5/726: dread de/d25/d43/f97 [0,4194304] 0 2024-05-13T22:48:10.630 INFO:tasks.workunit.client.0.smithi102.stdout:5/727: unlink de/d25/f26 0 2024-05-13T22:48:10.633 INFO:tasks.workunit.client.0.smithi102.stdout:5/728: rmdir de/d25/d2a/d4b/d4d/d50/da3 39 2024-05-13T22:48:10.634 INFO:tasks.workunit.client.0.smithi102.stdout:5/729: read de/d25/fe0 [230488,70650] 0 2024-05-13T22:48:10.637 INFO:tasks.workunit.client.0.smithi102.stdout:5/730: creat de/d25/d59/d7f/d86/da5/f10e x:0 0 0 2024-05-13T22:48:10.638 INFO:tasks.workunit.client.0.smithi102.stdout:5/731: chown de/d33 4398 1 2024-05-13T22:48:10.642 INFO:tasks.workunit.client.0.smithi102.stdout:5/732: creat de/d25/d2a/d4b/d4d/d50/d99/dbb/dda/f10f x:0 0 0 2024-05-13T22:48:10.647 INFO:tasks.workunit.client.0.smithi102.stdout:5/733: unlink de/d25/d2a/d4b/d4d/d50/da3/lad 0 2024-05-13T22:48:10.651 INFO:tasks.workunit.client.0.smithi102.stdout:8/459: truncate d2/d7/d12/f4f 5299 0 2024-05-13T22:48:10.653 INFO:tasks.workunit.client.0.smithi102.stdout:8/460: creat d2/d7/d25/d41/d6f/f8a x:0 0 0 2024-05-13T22:48:10.659 INFO:tasks.workunit.client.0.smithi102.stdout:8/461: dwrite d2/d7/d25/d41/d6f/f84 [0,4194304] 0 2024-05-13T22:48:10.662 INFO:tasks.workunit.client.0.smithi102.stdout:8/462: mknod d2/d7/d27/c8b 0 2024-05-13T22:48:10.662 INFO:tasks.workunit.client.0.smithi102.stdout:8/463: chown d2/d7/f53 1352352581 1 2024-05-13T22:48:10.664 INFO:tasks.workunit.client.0.smithi102.stdout:8/464: mknod d2/d7/d12/d78/c8c 0 2024-05-13T22:48:10.666 INFO:tasks.workunit.client.0.smithi102.stdout:8/465: mknod d2/d36/d49/d5d/c8d 0 2024-05-13T22:48:10.671 INFO:tasks.workunit.client.0.smithi102.stdout:8/466: getdents d2/d36 0 2024-05-13T22:48:10.674 INFO:tasks.workunit.client.0.smithi102.stdout:8/467: symlink d2/d83/l8e 0 2024-05-13T22:48:10.675 INFO:tasks.workunit.client.0.smithi102.stdout:8/468: chown d2/d7/d12/f13 32804842 1 2024-05-13T22:48:10.675 INFO:tasks.workunit.client.0.smithi102.stdout:8/469: chown d2/d36/c4d 95781173 1 2024-05-13T22:48:10.679 INFO:tasks.workunit.client.0.smithi102.stdout:8/470: truncate d2/d36/d49/d5d/f72 860332 0 2024-05-13T22:48:10.688 INFO:tasks.workunit.client.0.smithi102.stdout:8/471: rename d2/d7/d25/d41/d50/d40/f7f to d2/d7/f8f 0 2024-05-13T22:48:10.694 INFO:tasks.workunit.client.0.smithi102.stdout:8/472: creat d2/d7/d12/d76/f90 x:0 0 0 2024-05-13T22:48:10.698 INFO:tasks.workunit.client.0.smithi102.stdout:1/385: dwrite d2/d10/f31 [0,4194304] 0 2024-05-13T22:48:10.706 INFO:tasks.workunit.client.0.smithi102.stdout:1/386: chown d2/f46 1934 1 2024-05-13T22:48:10.709 INFO:tasks.workunit.client.0.smithi102.stdout:1/387: fdatasync d2/d4/d5/f36 0 2024-05-13T22:48:10.715 INFO:tasks.workunit.client.0.smithi102.stdout:1/388: dwrite d2/d4/d5/f15 [4194304,4194304] 0 2024-05-13T22:48:10.721 INFO:tasks.workunit.client.0.smithi102.stdout:8/473: dread d2/f4 [4194304,4194304] 0 2024-05-13T22:48:10.722 INFO:tasks.workunit.client.0.smithi102.stdout:8/474: fsync d2/d7/d12/f1c 0 2024-05-13T22:48:10.723 INFO:tasks.workunit.client.0.smithi102.stdout:7/406: write d6/d13/d58/d20/d4b/f5f [1190490,86616] 0 2024-05-13T22:48:10.725 INFO:tasks.workunit.client.0.smithi102.stdout:1/389: rename d2/d4/d26/c3d to d2/d4/d5/d19/d28/d6a/c7f 0 2024-05-13T22:48:10.730 INFO:tasks.workunit.client.0.smithi102.stdout:8/475: symlink d2/d7/d12/d76/l91 0 2024-05-13T22:48:10.734 INFO:tasks.workunit.client.0.smithi102.stdout:1/390: unlink d2/d10/f76 0 2024-05-13T22:48:10.734 INFO:tasks.workunit.client.0.smithi102.stdout:7/407: read d6/d13/d2c/f3a [3569366,54401] 0 2024-05-13T22:48:10.735 INFO:tasks.workunit.client.0.smithi102.stdout:1/391: readlink d2/d4/d26/d40/l57 0 2024-05-13T22:48:10.736 INFO:tasks.workunit.client.0.smithi102.stdout:7/408: write d6/d22/f33 [844376,90619] 0 2024-05-13T22:48:10.744 INFO:tasks.workunit.client.0.smithi102.stdout:8/476: chown d2/d36/d49/d5d/f72 12378025 1 2024-05-13T22:48:10.752 INFO:tasks.workunit.client.0.smithi102.stdout:7/409: dread d6/d13/d58/d20/f4a [0,4194304] 0 2024-05-13T22:48:10.755 INFO:tasks.workunit.client.0.smithi102.stdout:6/623: dwrite d9/db/f36 [4194304,4194304] 0 2024-05-13T22:48:10.758 INFO:tasks.workunit.client.0.smithi102.stdout:6/624: chown d9/db/d14/dcc/d5f/f9e 0 1 2024-05-13T22:48:10.758 INFO:tasks.workunit.client.0.smithi102.stdout:7/410: symlink d6/d15/d4d/d55/d6f/l73 0 2024-05-13T22:48:10.758 INFO:tasks.workunit.client.0.smithi102.stdout:7/411: dread - d6/f70 zero size 2024-05-13T22:48:10.761 INFO:tasks.workunit.client.0.smithi102.stdout:7/412: creat d6/d15/d4d/d55/f74 x:0 0 0 2024-05-13T22:48:10.768 INFO:tasks.workunit.client.0.smithi102.stdout:7/413: dwrite d6/d13/d2c/d39/f3f [0,4194304] 0 2024-05-13T22:48:10.778 INFO:tasks.workunit.client.0.smithi102.stdout:7/414: dread d6/d13/d2c/f3a [4194304,4194304] 0 2024-05-13T22:48:10.781 INFO:tasks.workunit.client.0.smithi102.stdout:7/415: creat d6/d13/d58/d20/f75 x:0 0 0 2024-05-13T22:48:10.783 INFO:tasks.workunit.client.0.smithi102.stdout:6/625: dread f7 [0,4194304] 0 2024-05-13T22:48:10.790 INFO:tasks.workunit.client.0.smithi102.stdout:6/626: dwrite d9/db/d2f/fcd [0,4194304] 0 2024-05-13T22:48:10.794 INFO:tasks.workunit.client.0.smithi102.stdout:7/416: dread d6/d15/f3b [0,4194304] 0 2024-05-13T22:48:10.799 INFO:tasks.workunit.client.0.smithi102.stdout:6/627: mknod d9/db/d14/cd3 0 2024-05-13T22:48:10.804 INFO:tasks.workunit.client.0.smithi102.stdout:6/628: fdatasync d9/db/d14/dcc/f65 0 2024-05-13T22:48:10.808 INFO:tasks.workunit.client.0.smithi102.stdout:6/629: truncate d9/db/d14/d35/d47/f5c 336133 0 2024-05-13T22:48:10.813 INFO:tasks.workunit.client.0.smithi102.stdout:6/630: creat d9/db/d2f/fd4 x:0 0 0 2024-05-13T22:48:10.814 INFO:tasks.workunit.client.0.smithi102.stdout:6/631: read d9/d25/f30 [192343,115382] 0 2024-05-13T22:48:10.817 INFO:tasks.workunit.client.0.smithi102.stdout:6/632: dread d9/db/f7d [0,4194304] 0 2024-05-13T22:48:10.821 INFO:tasks.workunit.client.0.smithi102.stdout:7/417: dread d6/d15/f1e [4194304,4194304] 0 2024-05-13T22:48:10.822 INFO:tasks.workunit.client.0.smithi102.stdout:7/418: chown d6/d13/d58 49 1 2024-05-13T22:48:10.825 INFO:tasks.workunit.client.0.smithi102.stdout:6/633: dwrite d9/db/fa1 [0,4194304] 0 2024-05-13T22:48:10.826 INFO:tasks.workunit.client.0.smithi102.stdout:7/419: mknod d6/d15/d4d/d55/d6f/c76 0 2024-05-13T22:48:10.832 INFO:tasks.workunit.client.0.smithi102.stdout:1/392: dwrite d2/d4/d5/d19/d28/f24 [0,4194304] 0 2024-05-13T22:48:10.836 INFO:tasks.workunit.client.0.smithi102.stdout:6/634: rename d9/db/d2f/d59 to d9/d45/d56/dd5 0 2024-05-13T22:48:10.837 INFO:tasks.workunit.client.0.smithi102.stdout:7/420: dwrite d6/d13/d2c/d39/f3f [0,4194304] 0 2024-05-13T22:48:10.838 INFO:tasks.workunit.client.0.smithi102.stdout:6/635: write d9/d45/f6f [176985,38588] 0 2024-05-13T22:48:10.845 INFO:tasks.workunit.client.0.smithi102.stdout:7/421: dread d6/d15/d4c/f5a [0,4194304] 0 2024-05-13T22:48:10.846 INFO:tasks.workunit.client.0.smithi102.stdout:7/422: dread - d6/d15/d4d/d55/d6f/d65/f6b zero size 2024-05-13T22:48:10.847 INFO:tasks.workunit.client.0.smithi102.stdout:7/423: fsync d6/d15/f18 0 2024-05-13T22:48:10.849 INFO:tasks.workunit.client.0.smithi102.stdout:6/636: mkdir d9/d25/d60/d6a/dd6 0 2024-05-13T22:48:10.856 INFO:tasks.workunit.client.0.smithi102.stdout:6/637: symlink d9/d25/d60/ld7 0 2024-05-13T22:48:10.861 INFO:tasks.workunit.client.0.smithi102.stdout:1/393: getdents d2/d4/d5 0 2024-05-13T22:48:10.864 INFO:tasks.workunit.client.0.smithi102.stdout:7/424: creat d6/f77 x:0 0 0 2024-05-13T22:48:10.865 INFO:tasks.workunit.client.0.smithi102.stdout:7/425: readlink d6/d13/l61 0 2024-05-13T22:48:10.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.888+0000 7f82a4241700 1 -- 172.21.15.102:0/3978567901 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f829c0ff2d0 con 0x7f828806d2f0 2024-05-13T22:48:10.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.890+0000 7f8292ffd700 1 -- 172.21.15.102:0/3978567901 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f829c0ff2d0 con 0x7f828806d2f0 2024-05-13T22:48:10.869 INFO:tasks.workunit.client.0.smithi102.stdout:1/394: dwrite d2/d10/f31 [4194304,4194304] 0 2024-05-13T22:48:10.869 INFO:tasks.workunit.client.0.smithi102.stdout:7/426: chown d6/cf 128812 1 2024-05-13T22:48:10.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.894+0000 7f82a4241700 1 -- 172.21.15.102:0/3978567901 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f828806d2f0 msgr2=0x7f828806f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:10.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.894+0000 7f82a4241700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f828806d2f0 0x7f828806f7b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f828c011490 tx=0x7f828c009260 comp rx=0 tx=0).stop 2024-05-13T22:48:10.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.894+0000 7f82a4241700 1 -- 172.21.15.102:0/3978567901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f829c0ffd10 msgr2=0x7f829c1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:10.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.894+0000 7f82a4241700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f829c0ffd10 0x7f829c1985d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8298000c00 tx=0x7f829800f690 comp rx=0 tx=0).stop 2024-05-13T22:48:10.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.894+0000 7f82a4241700 1 -- 172.21.15.102:0/3978567901 shutdown_connections 2024-05-13T22:48:10.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.894+0000 7f82a4241700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f828806d2f0 0x7f828806f7b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:10.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.894+0000 7f82a4241700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f829c0ffd10 0x7f829c1985d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:10.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.894+0000 7f82a4241700 1 --2- 172.21.15.102:0/3978567901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f829c101140 0x7f829c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:10.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.894+0000 7f82a4241700 1 -- 172.21.15.102:0/3978567901 >> 172.21.15.102:0/3978567901 conn(0x7f829c0fa7c0 msgr2=0x7f829c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:10.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.894+0000 7f82a4241700 1 -- 172.21.15.102:0/3978567901 shutdown_connections 2024-05-13T22:48:10.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:10.894+0000 7f82a4241700 1 -- 172.21.15.102:0/3978567901 wait complete. 2024-05-13T22:48:10.876 INFO:tasks.workunit.client.0.smithi102.stdout:6/638: dread d9/d25/d60/d6a/f97 [0,4194304] 0 2024-05-13T22:48:10.877 INFO:tasks.workunit.client.0.smithi102.stdout:7/427: mkdir d6/d78 0 2024-05-13T22:48:10.878 INFO:tasks.workunit.client.0.smithi102.stdout:7/428: dread - d6/d13/d58/d20/f75 zero size 2024-05-13T22:48:10.880 INFO:tasks.workunit.client.0.smithi102.stdout:7/429: chown d6/d15/d4d/d55 10747268 1 2024-05-13T22:48:10.881 INFO:tasks.workunit.client.0.smithi102.stdout:7/430: write d6/f67 [703829,26271] 0 2024-05-13T22:48:10.884 INFO:tasks.workunit.client.0.smithi102.stdout:7/431: rename d6/d13 to d6/d13/d58/d20/d4b/d79 22 2024-05-13T22:48:10.886 INFO:tasks.workunit.client.0.smithi102.stdout:6/639: dwrite d9/db/d14/d35/d47/dc1/fc6 [0,4194304] 0 2024-05-13T22:48:10.886 INFO:tasks.workunit.client.0.smithi102.stdout:7/432: stat d6/d15/d4d/d55/d6f/c76 0 2024-05-13T22:48:10.896 INFO:tasks.workunit.client.0.smithi102.stdout:1/395: dread d2/d45/f48 [0,4194304] 0 2024-05-13T22:48:10.897 INFO:tasks.workunit.client.0.smithi102.stdout:6/640: creat d9/d25/d60/d6a/dd6/fd8 x:0 0 0 2024-05-13T22:48:10.898 INFO:tasks.workunit.client.0.smithi102.stdout:7/433: dwrite d6/f77 [0,4194304] 0 2024-05-13T22:48:10.904 INFO:tasks.workunit.client.0.smithi102.stdout:6/641: dread d9/d25/f30 [0,4194304] 0 2024-05-13T22:48:10.906 INFO:tasks.workunit.client.0.smithi102.stdout:7/434: rename d6/f72 to d6/d15/f7a 0 2024-05-13T22:48:10.908 INFO:tasks.workunit.client.0.smithi102.stdout:1/396: creat d2/d4/d26/d40/d5e/f80 x:0 0 0 2024-05-13T22:48:10.910 INFO:tasks.workunit.client.0.smithi102.stdout:1/397: stat d2/d4/d26/d40/l50 0 2024-05-13T22:48:10.910 INFO:tasks.workunit.client.0.smithi102.stdout:7/435: dread d6/f67 [0,4194304] 0 2024-05-13T22:48:10.912 INFO:tasks.workunit.client.0.smithi102.stdout:1/398: write d2/d4/d5/d19/f78 [626312,34656] 0 2024-05-13T22:48:10.913 INFO:tasks.workunit.client.0.smithi102.stdout:6/642: mkdir d9/d45/da0/dd9 0 2024-05-13T22:48:10.914 INFO:tasks.workunit.client.0.smithi102.stdout:7/436: creat d6/d22/f7b x:0 0 0 2024-05-13T22:48:10.919 INFO:tasks.workunit.client.0.smithi102.stdout:7/437: write d6/d15/f7a [574582,120979] 0 2024-05-13T22:48:10.925 INFO:tasks.workunit.client.0.smithi102.stdout:7/438: dwrite d6/d13/d2c/f60 [0,4194304] 0 2024-05-13T22:48:10.968 INFO:tasks.workunit.client.0.smithi102.stdout:0/541: sync 2024-05-13T22:48:10.971 INFO:tasks.workunit.client.0.smithi102.stdout:7/439: dwrite d6/d13/d2c/f5c [0,4194304] 0 2024-05-13T22:48:10.991 INFO:tasks.workunit.client.0.smithi102.stdout:0/542: fdatasync d1/de/f33 0 2024-05-13T22:48:10.992 INFO:tasks.workunit.client.0.smithi102.stdout:0/543: write d1/de/d32/f9c [636861,85972] 0 2024-05-13T22:48:10.998 INFO:tasks.workunit.client.0.smithi102.stdout:1/399: write d2/f4b [384458,45170] 0 2024-05-13T22:48:10.999 INFO:tasks.workunit.client.0.smithi102.stdout:0/544: mkdir d1/de/d32/dab 0 2024-05-13T22:48:11.000 INFO:tasks.workunit.client.0.smithi102.stdout:7/440: creat d6/d15/d4d/d55/d6f/f7c x:0 0 0 2024-05-13T22:48:11.000 INFO:tasks.workunit.client.0.smithi102.stdout:0/545: chown d1/de/d32/d24/d2d/d3e/l44 5058386 1 2024-05-13T22:48:11.004 INFO:tasks.workunit.client.0.smithi102.stdout:0/546: creat d1/de/d32/d8a/fac x:0 0 0 2024-05-13T22:48:11.007 INFO:tasks.workunit.client.0.smithi102.stdout:1/400: creat d2/d45/d79/f81 x:0 0 0 2024-05-13T22:48:11.010 INFO:tasks.workunit.client.0.smithi102.stdout:0/547: write d1/d4d/f6a [752047,33754] 0 2024-05-13T22:48:11.018 INFO:tasks.workunit.client.0.smithi102.stdout:1/401: creat d2/d10/d62/d7b/f82 x:0 0 0 2024-05-13T22:48:11.020 INFO:tasks.workunit.client.0.smithi102.stdout:1/402: mknod d2/d10/d62/c83 0 2024-05-13T22:48:11.023 INFO:tasks.workunit.client.0.smithi102.stdout:1/403: rmdir d2/d4/d5/d19/d28/d6a 39 2024-05-13T22:48:11.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.047+0000 7f5a9f7fe700 1 -- 172.21.15.102:0/3548614559 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a8c00baf0 con 0x7f5aa0101e30 2024-05-13T22:48:11.027 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:10 smithi102 ceph-mon[28599]: pgmap v22: 65 pgs: 65 active+clean; 1.1 GiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 37 MiB/s rd, 152 MiB/s wr, 454 op/s 2024-05-13T22:48:11.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.048+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/3548614559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0101e30 msgr2=0x7f5aa0102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:11.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.048+0000 7f5aa6f6a700 1 --2- 172.21.15.102:0/3548614559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0101e30 0x7f5aa0102290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009910 tx=0x7f5a8c009c20 comp rx=0 tx=0).stop 2024-05-13T22:48:11.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.048+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/3548614559 shutdown_connections 2024-05-13T22:48:11.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.048+0000 7f5aa6f6a700 1 --2- 172.21.15.102:0/3548614559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0101e30 0x7f5aa0102290 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.048+0000 7f5aa6f6a700 1 --2- 172.21.15.102:0/3548614559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5aa00fee50 0x7f5aa00ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.048+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/3548614559 >> 172.21.15.102:0/3548614559 conn(0x7f5aa00fa7b0 msgr2=0x7f5aa00fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:11.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.049+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/3548614559 shutdown_connections 2024-05-13T22:48:11.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.049+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/3548614559 wait complete. 2024-05-13T22:48:11.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.049+0000 7f5aa6f6a700 1 Processor -- start 2024-05-13T22:48:11.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.049+0000 7f5aa6f6a700 1 -- start start 2024-05-13T22:48:11.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5aa6f6a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5aa00fee50 0x7f5aa019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:11.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5aa6f6a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0101e30 0x7f5aa019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:11.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5aa6f6a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa019d470 con 0x7f5aa0101e30 2024-05-13T22:48:11.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5aa6f6a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa019d5b0 con 0x7f5aa00fee50 2024-05-13T22:48:11.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5a9ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0101e30 0x7f5aa019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:11.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5aa4d06700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5aa00fee50 0x7f5aa019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:11.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5a9ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0101e30 0x7f5aa019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34842/0 (socket says 172.21.15.102:34842) 2024-05-13T22:48:11.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5a9ffff700 1 -- 172.21.15.102:0/2002238126 learned_addr learned my addr 172.21.15.102:0/2002238126 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:48:11.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5a9ffff700 1 -- 172.21.15.102:0/2002238126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5aa00fee50 msgr2=0x7f5aa019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:11.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5a9ffff700 1 --2- 172.21.15.102:0/2002238126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5aa00fee50 0x7f5aa019c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5a9ffff700 1 -- 172.21.15.102:0/2002238126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c0095f0 con 0x7f5aa0101e30 2024-05-13T22:48:11.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5a9ffff700 1 --2- 172.21.15.102:0/2002238126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0101e30 0x7f5aa019ce50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0092e0 tx=0x7f5a8c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:11.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.050+0000 7f5a9dffb700 1 -- 172.21.15.102:0/2002238126 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a8c01d070 con 0x7f5aa0101e30 2024-05-13T22:48:11.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.051+0000 7f5a9dffb700 1 -- 172.21.15.102:0/2002238126 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5a8c005dc0 con 0x7f5aa0101e30 2024-05-13T22:48:11.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.051+0000 7f5a9dffb700 1 -- 172.21.15.102:0/2002238126 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a8c017630 con 0x7f5aa0101e30 2024-05-13T22:48:11.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.051+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/2002238126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa01a2000 con 0x7f5aa0101e30 2024-05-13T22:48:11.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.051+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/2002238126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa01a24f0 con 0x7f5aa0101e30 2024-05-13T22:48:11.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.052+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/2002238126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa0105f10 con 0x7f5aa0101e30 2024-05-13T22:48:11.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.053+0000 7f5a9dffb700 1 -- 172.21.15.102:0/2002238126 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) v1 ==== 91288+0+0 (secure 0 0 0) 0x7f5a8c00faf0 con 0x7f5aa0101e30 2024-05-13T22:48:11.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.053+0000 7f5a9dffb700 1 --2- 172.21.15.102:0/2002238126 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f5a9006d2f0 0x7f5a9006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:11.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.053+0000 7f5a9dffb700 1 -- 172.21.15.102:0/2002238126 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f5a8c08e120 con 0x7f5aa0101e30 2024-05-13T22:48:11.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.053+0000 7f5aa4d06700 1 --2- 172.21.15.102:0/2002238126 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f5a9006d2f0 0x7f5a9006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:11.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.054+0000 7f5aa4d06700 1 --2- 172.21.15.102:0/2002238126 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f5a9006d2f0 0x7f5a9006f7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0068ae0 tx=0x7f5a94009260 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:11.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.060+0000 7f5a9dffb700 1 -- 172.21.15.102:0/2002238126 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f5a8c05b390 con 0x7f5aa0101e30 2024-05-13T22:48:11.043 INFO:tasks.workunit.client.0.smithi102.stdout:1/404: dread d2/d4/d5/d19/d28/f35 [0,4194304] 0 2024-05-13T22:48:11.043 INFO:tasks.workunit.client.0.smithi102.stdout:1/405: chown d2/f46 0 1 2024-05-13T22:48:11.047 INFO:tasks.workunit.client.0.smithi102.stdout:1/406: creat d2/d45/f84 x:0 0 0 2024-05-13T22:48:11.050 INFO:tasks.workunit.client.0.smithi102.stdout:1/407: creat d2/d10/f85 x:0 0 0 2024-05-13T22:48:11.068 INFO:tasks.workunit.client.0.smithi102.stdout:7/441: rename d6/d13/d2c/f54 to d6/d13/f7d 0 2024-05-13T22:48:11.069 INFO:tasks.workunit.client.0.smithi102.stdout:7/442: chown d6/d13/de/l41 1 1 2024-05-13T22:48:11.070 INFO:tasks.workunit.client.0.smithi102.stdout:7/443: chown d6/d13/d2c/d39/l6a 573527 1 2024-05-13T22:48:11.071 INFO:tasks.workunit.client.0.smithi102.stdout:7/444: dread - d6/d15/d4d/d55/f74 zero size 2024-05-13T22:48:11.072 INFO:tasks.workunit.client.0.smithi102.stdout:7/445: write d6/f70 [793897,8141] 0 2024-05-13T22:48:11.072 INFO:tasks.workunit.client.0.smithi102.stdout:0/548: rename d1/de/d32/d24/f4c to d1/fad 0 2024-05-13T22:48:11.075 INFO:tasks.workunit.client.0.smithi102.stdout:1/408: rename d2/d4/d26/d40/d5e/d74 to d2/d10/d62/d86 0 2024-05-13T22:48:11.078 INFO:tasks.workunit.client.0.smithi102.stdout:0/549: mknod d1/de/d32/d63/cae 0 2024-05-13T22:48:11.079 INFO:tasks.workunit.client.0.smithi102.stdout:1/409: chown d2/d4/d26/l5f 60659261 1 2024-05-13T22:48:11.085 INFO:tasks.workunit.client.0.smithi102.stdout:7/446: dread d6/d13/d2c/f3a [0,4194304] 0 2024-05-13T22:48:11.088 INFO:tasks.workunit.client.0.smithi102.stdout:6/643: link d9/db/d14/d35/d47/dc1/fc6 d9/d45/fda 0 2024-05-13T22:48:11.088 INFO:tasks.workunit.client.0.smithi102.stdout:1/410: dread d2/d4/f23 [0,4194304] 0 2024-05-13T22:48:11.090 INFO:tasks.workunit.client.0.smithi102.stdout:7/447: creat d6/d15/f7e x:0 0 0 2024-05-13T22:48:11.091 INFO:tasks.workunit.client.0.smithi102.stdout:7/448: chown d6/f70 67 1 2024-05-13T22:48:11.091 INFO:tasks.workunit.client.0.smithi102.stdout:7/449: fdatasync d6/d22/f43 0 2024-05-13T22:48:11.095 INFO:tasks.workunit.client.0.smithi102.stdout:7/450: stat d6/l1f 0 2024-05-13T22:48:11.099 INFO:tasks.workunit.client.0.smithi102.stdout:7/451: unlink d6/d13/d58/f52 0 2024-05-13T22:48:11.102 INFO:tasks.workunit.client.0.smithi102.stdout:0/550: dwrite d1/de/d32/d24/f74 [0,4194304] 0 2024-05-13T22:48:11.104 INFO:tasks.workunit.client.0.smithi102.stdout:7/452: mkdir d6/d15/d4d/d55/d7f 0 2024-05-13T22:48:11.106 INFO:tasks.workunit.client.0.smithi102.stdout:7/453: read d6/d13/d58/d20/f4a [1544924,94374] 0 2024-05-13T22:48:11.106 INFO:tasks.workunit.client.0.smithi102.stdout:1/411: write d2/d10/f5d [411420,92625] 0 2024-05-13T22:48:11.108 INFO:tasks.workunit.client.0.smithi102.stdout:7/454: chown d6/d13/d58/d20/f68 0 1 2024-05-13T22:48:11.108 INFO:tasks.workunit.client.0.smithi102.stdout:0/551: dread d1/de/d32/d24/f30 [0,4194304] 0 2024-05-13T22:48:11.112 INFO:tasks.workunit.client.0.smithi102.stdout:7/455: mkdir d6/d13/d58/d20/d80 0 2024-05-13T22:48:11.112 INFO:tasks.workunit.client.0.smithi102.stdout:0/552: mkdir d1/de/d42/daf 0 2024-05-13T22:48:11.113 INFO:tasks.workunit.client.0.smithi102.stdout:0/553: chown d1/l99 122235 1 2024-05-13T22:48:11.113 INFO:tasks.workunit.client.0.smithi102.stdout:1/412: symlink d2/d4/d26/d40/l87 0 2024-05-13T22:48:11.114 INFO:tasks.workunit.client.0.smithi102.stdout:1/413: chown d2/d10/c1d 61 1 2024-05-13T22:48:11.117 INFO:tasks.workunit.client.0.smithi102.stdout:1/414: fdatasync d2/d4/d26/d56/f61 0 2024-05-13T22:48:11.124 INFO:tasks.workunit.client.0.smithi102.stdout:1/415: link d2/d4/d26/d40/f72 d2/d10/d62/f88 0 2024-05-13T22:48:11.125 INFO:tasks.workunit.client.0.smithi102.stdout:1/416: readlink d2/d4/d26/d40/d5e/l77 0 2024-05-13T22:48:11.128 INFO:tasks.workunit.client.0.smithi102.stdout:1/417: read - d2/d4/d5/d19/d28/d6a/f59 zero size 2024-05-13T22:48:11.129 INFO:tasks.workunit.client.0.smithi102.stdout:1/418: chown d2/d4/d5/d19/d28/f4d 64 1 2024-05-13T22:48:11.130 INFO:tasks.workunit.client.0.smithi102.stdout:1/419: write d2/d4/f75 [553827,72599] 0 2024-05-13T22:48:11.131 INFO:tasks.workunit.client.0.smithi102.stdout:1/420: chown d2/d10/d62/d7b 45501 1 2024-05-13T22:48:11.132 INFO:tasks.workunit.client.0.smithi102.stdout:0/554: dwrite d1/de/d32/d24/d2d/d38/d3d/f57 [0,4194304] 0 2024-05-13T22:48:11.132 INFO:tasks.workunit.client.0.smithi102.stdout:1/421: stat d2/d4/d5/d19/d28/c49 0 2024-05-13T22:48:11.134 INFO:tasks.workunit.client.0.smithi102.stdout:0/555: write d1/d68/f86 [949867,84458] 0 2024-05-13T22:48:11.135 INFO:tasks.workunit.client.0.smithi102.stdout:1/422: rmdir d2/d45/d79 39 2024-05-13T22:48:11.135 INFO:tasks.workunit.client.0.smithi102.stdout:0/556: chown d1/d4d/ca0 989771065 1 2024-05-13T22:48:11.138 INFO:tasks.workunit.client.0.smithi102.stdout:0/557: rename d1/de/d32/f1e to d1/de/d32/dab/fb0 0 2024-05-13T22:48:11.139 INFO:tasks.workunit.client.0.smithi102.stdout:0/558: chown d1/d68/ca1 237 1 2024-05-13T22:48:11.140 INFO:tasks.workunit.client.0.smithi102.stdout:0/559: chown d1/de/d32/d24/l8f 109654315 1 2024-05-13T22:48:11.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:10 smithi130 ceph-mon[36591]: pgmap v22: 65 pgs: 65 active+clean; 1.1 GiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 37 MiB/s rd, 152 MiB/s wr, 454 op/s 2024-05-13T22:48:11.168 INFO:tasks.workunit.client.0.smithi102.stdout:0/560: dread d1/de/d32/d24/f31 [0,4194304] 0 2024-05-13T22:48:11.171 INFO:tasks.workunit.client.0.smithi102.stdout:0/561: mknod d1/de/d32/d24/d2d/d3e/cb1 0 2024-05-13T22:48:11.174 INFO:tasks.workunit.client.0.smithi102.stdout:0/562: unlink d1/de/d32/d24/d2d/f6b 0 2024-05-13T22:48:11.189 INFO:tasks.workunit.client.0.smithi102.stdout:0/563: dread d1/de/d32/f18 [0,4194304] 0 2024-05-13T22:48:11.204 INFO:tasks.workunit.client.0.smithi102.stdout:0/564: getdents d1/de/d32/d24/d2d 0 2024-05-13T22:48:11.206 INFO:tasks.workunit.client.0.smithi102.stdout:0/565: fsync d1/de/d32/d24/d2d/d38/f69 0 2024-05-13T22:48:11.209 INFO:tasks.workunit.client.0.smithi102.stdout:0/566: rmdir d1/d4d 39 2024-05-13T22:48:11.215 INFO:tasks.workunit.client.0.smithi102.stdout:0/567: write d1/de/d32/d63/f7d [365167,127619] 0 2024-05-13T22:48:11.217 INFO:tasks.workunit.client.0.smithi102.stdout:0/568: readlink d1/de/d32/d24/d2d/d38/l75 0 2024-05-13T22:48:11.218 INFO:tasks.workunit.client.0.smithi102.stdout:0/569: stat d1/de/d32/d24/d2d/f64 0 2024-05-13T22:48:11.219 INFO:tasks.workunit.client.0.smithi102.stdout:0/570: chown d1/de/d32/d24/d2d/d38/f69 0 1 2024-05-13T22:48:11.220 INFO:tasks.workunit.client.0.smithi102.stdout:0/571: read - d1/de/d32/f89 zero size 2024-05-13T22:48:11.222 INFO:tasks.workunit.client.0.smithi102.stdout:0/572: truncate d1/de/d32/d24/d2d/f53 4798174 0 2024-05-13T22:48:11.222 INFO:tasks.workunit.client.0.smithi102.stdout:0/573: readlink d1/de/d32/d24/l34 0 2024-05-13T22:48:11.224 INFO:tasks.workunit.client.0.smithi102.stdout:0/574: creat d1/de/d32/d8a/fb2 x:0 0 0 2024-05-13T22:48:11.228 INFO:tasks.workunit.client.0.smithi102.stdout:0/575: dread d1/de/d32/d63/f7d [0,4194304] 0 2024-05-13T22:48:11.231 INFO:tasks.workunit.client.0.smithi102.stdout:0/576: creat d1/fb3 x:0 0 0 2024-05-13T22:48:11.233 INFO:tasks.workunit.client.0.smithi102.stdout:0/577: creat d1/de/d32/dab/fb4 x:0 0 0 2024-05-13T22:48:11.236 INFO:tasks.workunit.client.0.smithi102.stdout:0/578: creat d1/de/d42/d61/fb5 x:0 0 0 2024-05-13T22:48:11.241 INFO:tasks.workunit.client.0.smithi102.stdout:0/579: rename d1/de/d42/c76 to d1/d4d/d79/d9f/cb6 0 2024-05-13T22:48:11.247 INFO:tasks.workunit.client.0.smithi102.stdout:0/580: dwrite d1/fb3 [0,4194304] 0 2024-05-13T22:48:11.258 INFO:tasks.workunit.client.0.smithi102.stdout:0/581: rmdir d1/de/d42/daf 0 2024-05-13T22:48:11.259 INFO:tasks.workunit.client.0.smithi102.stdout:0/582: read d1/de/d32/d24/d2d/d3e/f41 [5645439,36171] 0 2024-05-13T22:48:11.267 INFO:tasks.workunit.client.0.smithi102.stdout:0/583: dwrite d1/de/d32/d8a/fac [0,4194304] 0 2024-05-13T22:48:11.275 INFO:tasks.workunit.client.0.smithi102.stdout:0/584: dwrite d1/de/d32/dab/fb4 [0,4194304] 0 2024-05-13T22:48:11.284 INFO:tasks.workunit.client.0.smithi102.stdout:0/585: mkdir d1/de/d32/d24/d2d/d38/db7 0 2024-05-13T22:48:11.288 INFO:tasks.workunit.client.0.smithi102.stdout:0/586: unlink d1/de/d32/d24/d2d/d3e/c66 0 2024-05-13T22:48:11.288 INFO:tasks.workunit.client.0.smithi102.stdout:0/587: dread - d1/de/d42/d61/fb5 zero size 2024-05-13T22:48:11.290 INFO:tasks.workunit.client.0.smithi102.stdout:0/588: readlink d1/de/d32/d24/d2d/d38/l75 0 2024-05-13T22:48:11.296 INFO:tasks.workunit.client.0.smithi102.stdout:0/589: truncate d1/fad 747956 0 2024-05-13T22:48:11.301 INFO:tasks.workunit.client.0.smithi102.stdout:0/590: creat d1/de/d32/d24/d2d/d38/d3d/d81/fb8 x:0 0 0 2024-05-13T22:48:11.304 INFO:tasks.workunit.client.0.smithi102.stdout:0/591: fsync d1/de/f19 0 2024-05-13T22:48:11.321 INFO:tasks.workunit.client.0.smithi102.stdout:0/592: dread d1/f2c [0,4194304] 0 2024-05-13T22:48:11.338 INFO:tasks.workunit.client.0.smithi102.stdout:0/593: dread d1/de/d32/d24/d2d/f53 [0,4194304] 0 2024-05-13T22:48:11.339 INFO:tasks.workunit.client.0.smithi102.stdout:0/594: fdatasync d1/de/d32/f9c 0 2024-05-13T22:48:11.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.364+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/2002238126 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5aa00611d0 con 0x7f5a9006d2f0 2024-05-13T22:48:11.343 INFO:tasks.workunit.client.0.smithi102.stdout:0/595: dread d1/de/d32/d24/d2d/d38/d3d/f9a [0,4194304] 0 2024-05-13T22:48:11.346 INFO:tasks.workunit.client.0.smithi102.stdout:0/596: dread d1/d4d/d79/f80 [0,4194304] 0 2024-05-13T22:48:11.348 INFO:tasks.workunit.client.0.smithi102.stdout:0/597: symlink d1/de/d32/d8a/lb9 0 2024-05-13T22:48:11.349 INFO:tasks.workunit.client.0.smithi102.stdout:0/598: fdatasync d1/de/d32/d8a/fac 0 2024-05-13T22:48:11.351 INFO:tasks.workunit.client.0.smithi102.stdout:0/599: fdatasync d1/de/d32/f18 0 2024-05-13T22:48:11.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.375+0000 7f5a9dffb700 1 -- 172.21.15.102:0/2002238126 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5aa00611d0 con 0x7f5a9006d2f0 2024-05-13T22:48:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:48:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (6m) 25s ago 8m 25.6M - 0.25.0 c8568f914cd2 c096e9e9a228 2024-05-13T22:48:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (8m) 25s ago 8m 7746k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:48:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (7m) 29s ago 7m 7457k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:48:11.354 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (8m) 25s ago 8m 6656k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (7m) 29s ago 7m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (6m) 25s ago 8m 88.4M - 9.4.7 954c08fa6188 bca1790c17aa 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (2m) 25s ago 2m 75.1M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (2m) 25s ago 2m 138M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (2m) 29s ago 2m 13.0M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (2m) 29s ago 2m 16.0M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:9283,8765,8443 running (9m) 25s ago 9m 328M - 18.2.1 5be31c24972a 77cd1ca053e7 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (53s) 29s ago 7m 490M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9a28ef996664 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (9m) 25s ago 9m 48.0M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (7m) 29s ago 7m 41.7M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (8m) 25s ago 8m 16.3M - 1.5.0 0da6a335fe13 af297cf52099 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (7m) 29s ago 7m 15.7M - 1.5.0 0da6a335fe13 cc0f64ee7ecd 2024-05-13T22:48:11.355 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (6m) 25s ago 6m 97.5M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:48:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (6m) 25s ago 6m 105M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:48:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (5m) 25s ago 5m 89.7M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:48:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (5m) 29s ago 5m 108M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:48:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (4m) 29s ago 4m 101M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:48:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (4m) 29s ago 4m 95.0M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:48:11.356 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (6m) 25s ago 7m 62.6M - 2.43.0 a07b618ecd1d a89752444eb6 2024-05-13T22:48:11.356 INFO:tasks.workunit.client.0.smithi102.stdout:0/600: dread d1/de/d32/d24/d2d/f53 [0,4194304] 0 2024-05-13T22:48:11.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.379+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/2002238126 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f5a9006d2f0 msgr2=0x7f5a9006f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:11.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.379+0000 7f5aa6f6a700 1 --2- 172.21.15.102:0/2002238126 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f5a9006d2f0 0x7f5a9006f7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0068ae0 tx=0x7f5a94009260 comp rx=0 tx=0).stop 2024-05-13T22:48:11.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.379+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/2002238126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0101e30 msgr2=0x7f5aa019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:11.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.379+0000 7f5aa6f6a700 1 --2- 172.21.15.102:0/2002238126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0101e30 0x7f5aa019ce50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0092e0 tx=0x7f5a8c00f690 comp rx=0 tx=0).stop 2024-05-13T22:48:11.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.379+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/2002238126 shutdown_connections 2024-05-13T22:48:11.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.379+0000 7f5aa6f6a700 1 --2- 172.21.15.102:0/2002238126 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f5a9006d2f0 0x7f5a9006f7b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.379+0000 7f5aa6f6a700 1 --2- 172.21.15.102:0/2002238126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5aa00fee50 0x7f5aa019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.379+0000 7f5aa6f6a700 1 --2- 172.21.15.102:0/2002238126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5aa0101e30 0x7f5aa019ce50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.379+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/2002238126 >> 172.21.15.102:0/2002238126 conn(0x7f5aa00fa7b0 msgr2=0x7f5aa0103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:11.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.379+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/2002238126 shutdown_connections 2024-05-13T22:48:11.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.379+0000 7f5aa6f6a700 1 -- 172.21.15.102:0/2002238126 wait complete. 2024-05-13T22:48:11.366 INFO:tasks.workunit.client.0.smithi102.stdout:0/601: link d1/de/d42/c6c d1/de/d32/d24/cba 0 2024-05-13T22:48:11.367 INFO:tasks.workunit.client.0.smithi102.stdout:0/602: chown d1/de/d32/d24/d2d/d38 14252895 1 2024-05-13T22:48:11.370 INFO:tasks.workunit.client.0.smithi102.stdout:0/603: symlink d1/de/d32/d8a/lbb 0 2024-05-13T22:48:11.408 INFO:tasks.workunit.client.0.smithi102.stdout:0/604: truncate d1/d4d/f6a 730688 0 2024-05-13T22:48:11.409 INFO:tasks.workunit.client.0.smithi102.stdout:0/605: creat d1/de/d32/dab/fbc x:0 0 0 2024-05-13T22:48:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.505+0000 7f381c7f9700 1 -- 172.21.15.102:0/1828187324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38140fee80 msgr2=0x7f38140ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.505+0000 7f381c7f9700 1 --2- 172.21.15.102:0/1828187324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38140fee80 0x7f38140ff2a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3810009910 tx=0x7f3810009c20 comp rx=0 tx=0).stop 2024-05-13T22:48:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.506+0000 7f381c7f9700 1 -- 172.21.15.102:0/1828187324 shutdown_connections 2024-05-13T22:48:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.506+0000 7f381c7f9700 1 --2- 172.21.15.102:0/1828187324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3814101e60 0x7f38141022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.506+0000 7f381c7f9700 1 --2- 172.21.15.102:0/1828187324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38140fee80 0x7f38140ff2a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.506+0000 7f381c7f9700 1 -- 172.21.15.102:0/1828187324 >> 172.21.15.102:0/1828187324 conn(0x7f38140fa7c0 msgr2=0x7f38140fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.506+0000 7f381c7f9700 1 -- 172.21.15.102:0/1828187324 shutdown_connections 2024-05-13T22:48:11.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.506+0000 7f381c7f9700 1 -- 172.21.15.102:0/1828187324 wait complete. 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.506+0000 7f381c7f9700 1 Processor -- start 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.506+0000 7f381c7f9700 1 -- start start 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381c7f9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38140fee80 0x7f3814196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381c7f9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3814101e60 0x7f3814196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381c7f9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3814196f70 con 0x7f38140fee80 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381c7f9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141970b0 con 0x7f3814101e60 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381a595700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38140fee80 0x7f3814196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381a595700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38140fee80 0x7f3814196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34852/0 (socket says 172.21.15.102:34852) 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381a595700 1 -- 172.21.15.102:0/892473052 learned_addr learned my addr 172.21.15.102:0/892473052 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f3819d94700 1 --2- 172.21.15.102:0/892473052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3814101e60 0x7f3814196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381a595700 1 -- 172.21.15.102:0/892473052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3814101e60 msgr2=0x7f3814196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381a595700 1 --2- 172.21.15.102:0/892473052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3814101e60 0x7f3814196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381a595700 1 -- 172.21.15.102:0/892473052 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38100095f0 con 0x7f38140fee80 2024-05-13T22:48:11.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381a595700 1 --2- 172.21.15.102:0/892473052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38140fee80 0x7f3814196410 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f381000bd70 tx=0x7f381000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:11.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f380b7fe700 1 -- 172.21.15.102:0/892473052 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f381001d070 con 0x7f38140fee80 2024-05-13T22:48:11.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.507+0000 7f381c7f9700 1 -- 172.21.15.102:0/892473052 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381419bb00 con 0x7f38140fee80 2024-05-13T22:48:11.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.508+0000 7f381c7f9700 1 -- 172.21.15.102:0/892473052 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381419c070 con 0x7f38140fee80 2024-05-13T22:48:11.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.508+0000 7f380b7fe700 1 -- 172.21.15.102:0/892473052 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f381000f460 con 0x7f38140fee80 2024-05-13T22:48:11.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.508+0000 7f380b7fe700 1 -- 172.21.15.102:0/892473052 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3810021640 con 0x7f38140fee80 2024-05-13T22:48:11.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.508+0000 7f38097fa700 1 -- 172.21.15.102:0/892473052 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37fc0052f0 con 0x7f38140fee80 2024-05-13T22:48:11.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.510+0000 7f380b7fe700 1 -- 172.21.15.102:0/892473052 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) v1 ==== 91288+0+0 (secure 0 0 0) 0x7f381000fab0 con 0x7f38140fee80 2024-05-13T22:48:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.511+0000 7f380b7fe700 1 --2- 172.21.15.102:0/892473052 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f380006d2e0 0x7f380006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.511+0000 7f380b7fe700 1 -- 172.21.15.102:0/892473052 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f381008e530 con 0x7f38140fee80 2024-05-13T22:48:11.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.511+0000 7f3819d94700 1 --2- 172.21.15.102:0/892473052 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f380006d2e0 0x7f380006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:11.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.512+0000 7f3819d94700 1 --2- 172.21.15.102:0/892473052 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f380006d2e0 0x7f380006f7a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3804006fd0 tx=0x7f3804008040 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:11.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.514+0000 7f380b7fe700 1 -- 172.21.15.102:0/892473052 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f381005b820 con 0x7f38140fee80 2024-05-13T22:48:11.614 INFO:tasks.workunit.client.0.smithi102.stdout:4/572: sync 2024-05-13T22:48:11.615 INFO:tasks.workunit.client.0.smithi102.stdout:4/573: dread - d3/d4/d6/de/faa zero size 2024-05-13T22:48:11.617 INFO:tasks.workunit.client.0.smithi102.stdout:4/574: creat d3/d4/d6/de/d1f/d41/fc7 x:0 0 0 2024-05-13T22:48:11.618 INFO:tasks.workunit.client.0.smithi102.stdout:4/575: read - d3/d4/d6/de/faa zero size 2024-05-13T22:48:11.620 INFO:tasks.workunit.client.0.smithi102.stdout:4/576: write d3/d4/d6/de/faa [347901,37288] 0 2024-05-13T22:48:11.621 INFO:tasks.workunit.client.0.smithi102.stdout:4/577: write d3/d19/d23/d4e/f72 [71908,54248] 0 2024-05-13T22:48:11.721 INFO:tasks.workunit.client.0.smithi102.stdout:4/578: write d3/d19/d23/d39/d5d/f96 [973261,130925] 0 2024-05-13T22:48:11.723 INFO:tasks.workunit.client.0.smithi102.stdout:4/579: mkdir d3/d4/d6/de/d8d/dc8 0 2024-05-13T22:48:11.726 INFO:tasks.workunit.client.0.smithi102.stdout:4/580: mkdir d3/d19/dad/dc9 0 2024-05-13T22:48:11.778 INFO:tasks.workunit.client.0.smithi102.stdout:4/581: truncate d3/d19/d23/d4e/d68/d85/fbf 1205870 0 2024-05-13T22:48:11.781 INFO:tasks.workunit.client.0.smithi102.stdout:4/582: creat d3/d19/d23/d3a/fca x:0 0 0 2024-05-13T22:48:11.785 INFO:tasks.workunit.client.0.smithi102.stdout:4/583: rename d3/d19/d23/d3a/fca to d3/d19/d23/d39/da7/d7d/fcb 0 2024-05-13T22:48:11.786 INFO:tasks.workunit.client.0.smithi102.stdout:4/584: truncate d3/d4/d6/de/d1f/d41/d6a/fa1 4623110 0 2024-05-13T22:48:11.796 INFO:tasks.workunit.client.0.smithi102.stdout:3/587: sync 2024-05-13T22:48:11.805 INFO:tasks.workunit.client.0.smithi102.stdout:3/588: dwrite de/d10/d2a/d36/db2/fd2 [0,4194304] 0 2024-05-13T22:48:11.806 INFO:tasks.workunit.client.0.smithi102.stdout:3/589: read de/d10/d50/f47 [423855,15150] 0 2024-05-13T22:48:11.810 INFO:tasks.workunit.client.0.smithi102.stdout:3/590: rename l4 to de/d10/d50/d63/d98/ld3 0 2024-05-13T22:48:11.812 INFO:tasks.workunit.client.0.smithi102.stdout:4/585: dwrite d3/d22/f74 [0,4194304] 0 2024-05-13T22:48:11.817 INFO:tasks.workunit.client.0.smithi102.stdout:3/591: rename de/d1c/d79/l86 to de/d10/d34/ld4 0 2024-05-13T22:48:11.821 INFO:tasks.workunit.client.0.smithi102.stdout:3/592: dread - de/d10/d34/f9a zero size 2024-05-13T22:48:11.822 INFO:tasks.workunit.client.0.smithi102.stdout:4/586: mknod d3/d19/dad/dc9/ccc 0 2024-05-13T22:48:11.823 INFO:tasks.workunit.client.0.smithi102.stdout:3/593: write de/d10/d50/fc6 [192995,102357] 0 2024-05-13T22:48:11.824 INFO:tasks.workunit.client.0.smithi102.stdout:3/594: readlink de/d10/d2a/d33/lae 0 2024-05-13T22:48:11.825 INFO:tasks.workunit.client.0.smithi102.stdout:3/595: chown de/d10/d2a/d36/da0/ca3 78 1 2024-05-13T22:48:11.828 INFO:tasks.workunit.client.0.smithi102.stdout:3/596: unlink de/d68/f82 0 2024-05-13T22:48:11.829 INFO:tasks.workunit.client.0.smithi102.stdout:3/597: truncate de/d10/d2a/d33/fcb 4702675 0 2024-05-13T22:48:11.833 INFO:tasks.workunit.client.0.smithi102.stdout:3/598: getdents de/d10/d46/dc8 0 2024-05-13T22:48:11.834 INFO:tasks.workunit.client.0.smithi102.stdout:3/599: dread - de/d10/d2a/d36/f49 zero size 2024-05-13T22:48:11.836 INFO:tasks.workunit.client.0.smithi102.stdout:4/587: write d3/d4/d6/f80 [976493,78174] 0 2024-05-13T22:48:11.837 INFO:tasks.workunit.client.0.smithi102.stdout:3/600: chown de/c16 1089 1 2024-05-13T22:48:11.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.863+0000 7f38097fa700 1 -- 172.21.15.102:0/892473052 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f37fc005c90 con 0x7f38140fee80 2024-05-13T22:48:11.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.863+0000 7f380b7fe700 1 -- 172.21.15.102:0/892473052 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7f381005b3b0 con 0x7f38140fee80 2024-05-13T22:48:11.842 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:48:11.842 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:48:11.842 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:48:11.842 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 1 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 1 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:48:11.843 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:48:11.844 INFO:tasks.workunit.client.0.smithi102.stdout:3/601: dwrite de/d10/d50/fc6 [0,4194304] 0 2024-05-13T22:48:11.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.866+0000 7f38097fa700 1 -- 172.21.15.102:0/892473052 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f380006d2e0 msgr2=0x7f380006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:11.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.866+0000 7f38097fa700 1 --2- 172.21.15.102:0/892473052 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f380006d2e0 0x7f380006f7a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3804006fd0 tx=0x7f3804008040 comp rx=0 tx=0).stop 2024-05-13T22:48:11.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.866+0000 7f38097fa700 1 -- 172.21.15.102:0/892473052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38140fee80 msgr2=0x7f3814196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:11.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.866+0000 7f38097fa700 1 --2- 172.21.15.102:0/892473052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38140fee80 0x7f3814196410 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f381000bd70 tx=0x7f381000be50 comp rx=0 tx=0).stop 2024-05-13T22:48:11.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.866+0000 7f38097fa700 1 -- 172.21.15.102:0/892473052 shutdown_connections 2024-05-13T22:48:11.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.866+0000 7f38097fa700 1 --2- 172.21.15.102:0/892473052 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f380006d2e0 0x7f380006f7a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.866+0000 7f38097fa700 1 --2- 172.21.15.102:0/892473052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38140fee80 0x7f3814196410 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.866+0000 7f38097fa700 1 --2- 172.21.15.102:0/892473052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3814101e60 0x7f3814196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:11.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.866+0000 7f38097fa700 1 -- 172.21.15.102:0/892473052 >> 172.21.15.102:0/892473052 conn(0x7f38140fa7c0 msgr2=0x7f3814105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:11.846 INFO:tasks.workunit.client.0.smithi102.stdout:4/588: write d3/d4/f1e [130769,124139] 0 2024-05-13T22:48:11.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.866+0000 7f38097fa700 1 -- 172.21.15.102:0/892473052 shutdown_connections 2024-05-13T22:48:11.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:11.866+0000 7f38097fa700 1 -- 172.21.15.102:0/892473052 wait complete. 2024-05-13T22:48:11.849 INFO:tasks.workunit.client.0.smithi102.stdout:3/602: creat de/d10/d50/d63/fd5 x:0 0 0 2024-05-13T22:48:11.850 INFO:tasks.workunit.client.0.smithi102.stdout:3/603: write de/d10/d50/f47 [938801,14] 0 2024-05-13T22:48:11.850 INFO:tasks.workunit.client.0.smithi102.stdout:4/589: creat d3/d22/fcd x:0 0 0 2024-05-13T22:48:11.853 INFO:tasks.workunit.client.0.smithi102.stdout:3/604: write de/d10/d2a/d36/f58 [4271739,36301] 0 2024-05-13T22:48:11.855 INFO:tasks.workunit.client.0.smithi102.stdout:4/590: creat d3/d22/fce x:0 0 0 2024-05-13T22:48:11.858 INFO:tasks.workunit.client.0.smithi102.stdout:4/591: mkdir d3/d19/d23/d4e/d68/dcf 0 2024-05-13T22:48:11.859 INFO:tasks.workunit.client.0.smithi102.stdout:4/592: dread - d3/d19/dad/fbc zero size 2024-05-13T22:48:11.866 INFO:tasks.workunit.client.0.smithi102.stdout:3/605: write de/d10/d34/f9a [677585,87831] 0 2024-05-13T22:48:11.868 INFO:tasks.workunit.client.0.smithi102.stdout:4/593: dread d3/d19/d23/d39/da7/f75 [0,4194304] 0 2024-05-13T22:48:11.870 INFO:tasks.workunit.client.0.smithi102.stdout:3/606: creat de/d10/d2a/d8c/d7f/fd6 x:0 0 0 2024-05-13T22:48:11.872 INFO:tasks.workunit.client.0.smithi102.stdout:4/594: creat d3/d19/d23/d4e/d68/d85/fd0 x:0 0 0 2024-05-13T22:48:11.874 INFO:tasks.workunit.client.0.smithi102.stdout:3/607: creat de/d67/fd7 x:0 0 0 2024-05-13T22:48:11.875 INFO:tasks.workunit.client.0.smithi102.stdout:3/608: readlink de/d1c/d79/la6 0 2024-05-13T22:48:11.878 INFO:tasks.workunit.client.0.smithi102.stdout:4/595: link d3/d4/d6/c33 d3/d19/d23/d4e/d68/dcf/cd1 0 2024-05-13T22:48:11.881 INFO:tasks.workunit.client.0.smithi102.stdout:3/609: rename de/d10/d2a/lad to de/d10/d2a/d36/db2/ld8 0 2024-05-13T22:48:11.882 INFO:tasks.workunit.client.0.smithi102.stdout:3/610: write de/d10/d50/d8f/fc0 [534667,128400] 0 2024-05-13T22:48:11.883 INFO:tasks.workunit.client.0.smithi102.stdout:4/596: dwrite d3/d19/d23/d39/fc1 [0,4194304] 0 2024-05-13T22:48:11.889 INFO:tasks.workunit.client.0.smithi102.stdout:3/611: fsync de/d10/d2a/f2e 0 2024-05-13T22:48:11.890 INFO:tasks.workunit.client.0.smithi102.stdout:4/597: mkdir d3/d19/d23/dd2 0 2024-05-13T22:48:11.893 INFO:tasks.workunit.client.0.smithi102.stdout:3/612: fsync de/d10/d46/f6f 0 2024-05-13T22:48:11.895 INFO:tasks.workunit.client.0.smithi102.stdout:4/598: chown d3/d4/d6/de/d1f/d41/l7f 36906644 1 2024-05-13T22:48:11.900 INFO:tasks.workunit.client.0.smithi102.stdout:3/613: dread de/d10/d34/f38 [0,4194304] 0 2024-05-13T22:48:11.903 INFO:tasks.workunit.client.0.smithi102.stdout:4/599: dwrite d3/d19/dad/faf [0,4194304] 0 2024-05-13T22:48:11.910 INFO:tasks.workunit.client.0.smithi102.stdout:4/600: getdents d3/d19/d23/d4e/da6 0 2024-05-13T22:48:11.911 INFO:tasks.workunit.client.0.smithi102.stdout:3/614: getdents de/d68 0 2024-05-13T22:48:11.915 INFO:tasks.workunit.client.0.smithi102.stdout:4/601: mkdir d3/d19/d23/d39/da7/da9/dd3 0 2024-05-13T22:48:11.933 INFO:tasks.workunit.client.0.smithi102.stdout:4/602: dread d3/d4/d6/f18 [0,4194304] 0 2024-05-13T22:48:11.934 INFO:tasks.workunit.client.0.smithi102.stdout:4/603: dread - d3/d4/d6/de/d1f/f67 zero size 2024-05-13T22:48:11.944 INFO:tasks.workunit.client.0.smithi102.stdout:4/604: write d3/d19/d23/f37 [1889847,44979] 0 2024-05-13T22:48:11.947 INFO:tasks.workunit.client.0.smithi102.stdout:4/605: mkdir d3/d19/d23/d39/da7/da9/dd3/dd4 0 2024-05-13T22:48:11.949 INFO:tasks.workunit.client.0.smithi102.stdout:4/606: mkdir d3/d19/dad/dc9/dd5 0 2024-05-13T22:48:11.951 INFO:tasks.workunit.client.0.smithi102.stdout:4/607: unlink d3/d4/d6/de/d1f/d41/l7f 0 2024-05-13T22:48:11.953 INFO:tasks.workunit.client.0.smithi102.stdout:4/608: fdatasync d3/d22/f3e 0 2024-05-13T22:48:11.960 INFO:tasks.workunit.client.0.smithi102.stdout:4/609: dwrite d3/d19/d23/d39/d77/f7c [0,4194304] 0 2024-05-13T22:48:11.981 INFO:tasks.workunit.client.0.smithi102.stdout:4/610: dread d3/d4/d6/de/d1f/f8c [0,4194304] 0 2024-05-13T22:48:11.985 INFO:tasks.workunit.client.0.smithi102.stdout:4/611: dread d3/d4/d6/de/d1f/f28 [0,4194304] 0 2024-05-13T22:48:11.992 INFO:tasks.workunit.client.0.smithi102.stdout:4/612: link d3/d4/d6/de/d8d/f91 d3/d19/d23/d39/da7/fd6 0 2024-05-13T22:48:12.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.024+0000 7f430eafc700 1 -- 172.21.15.102:0/2630705919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43080ffab0 msgr2=0x7f43080ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:12.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.024+0000 7f430eafc700 1 --2- 172.21.15.102:0/2630705919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43080ffab0 0x7f43080ffed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009910 tx=0x7f42fc009c20 comp rx=0 tx=0).stop 2024-05-13T22:48:12.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.024+0000 7f430eafc700 1 -- 172.21.15.102:0/2630705919 shutdown_connections 2024-05-13T22:48:12.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.024+0000 7f430eafc700 1 --2- 172.21.15.102:0/2630705919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4308100cb0 0x7f4308101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:12.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.024+0000 7f430eafc700 1 --2- 172.21.15.102:0/2630705919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43080ffab0 0x7f43080ffed0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:12.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.024+0000 7f430eafc700 1 -- 172.21.15.102:0/2630705919 >> 172.21.15.102:0/2630705919 conn(0x7f43080fb050 msgr2=0x7f43080fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:12.004 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.024+0000 7f430eafc700 1 -- 172.21.15.102:0/2630705919 shutdown_connections 2024-05-13T22:48:12.004 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:11 smithi102 ceph-mon[28599]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.025+0000 7f430eafc700 1 -- 172.21.15.102:0/2630705919 wait complete. 2024-05-13T22:48:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.025+0000 7f430eafc700 1 Processor -- start 2024-05-13T22:48:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.025+0000 7f430eafc700 1 -- start start 2024-05-13T22:48:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.025+0000 7f430eafc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43080ffab0 0x7f4308198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.025+0000 7f430eafc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4308100cb0 0x7f4308198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.026+0000 7f430eafc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43081990e0 con 0x7f4308100cb0 2024-05-13T22:48:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.026+0000 7f430eafc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4308199220 con 0x7f43080ffab0 2024-05-13T22:48:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.026+0000 7f4307fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4308100cb0 0x7f4308198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.026+0000 7f430c898700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43080ffab0 0x7f4308198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.026+0000 7f4307fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4308100cb0 0x7f4308198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34878/0 (socket says 172.21.15.102:34878) 2024-05-13T22:48:12.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.026+0000 7f4307fff700 1 -- 172.21.15.102:0/3594688775 learned_addr learned my addr 172.21.15.102:0/3594688775 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:48:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.026+0000 7f430c898700 1 -- 172.21.15.102:0/3594688775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4308100cb0 msgr2=0x7f4308198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.026+0000 7f430c898700 1 --2- 172.21.15.102:0/3594688775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4308100cb0 0x7f4308198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.026+0000 7f430c898700 1 -- 172.21.15.102:0/3594688775 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc0095f0 con 0x7f43080ffab0 2024-05-13T22:48:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.026+0000 7f430c898700 1 --2- 172.21.15.102:0/3594688775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43080ffab0 0x7f4308198580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009910 tx=0x7f42fc00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.027+0000 7f4305ffb700 1 -- 172.21.15.102:0/3594688775 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42fc01d070 con 0x7f43080ffab0 2024-05-13T22:48:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.027+0000 7f4305ffb700 1 -- 172.21.15.102:0/3594688775 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f42fc005dc0 con 0x7f43080ffab0 2024-05-13T22:48:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.027+0000 7f430eafc700 1 -- 172.21.15.102:0/3594688775 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430819dc70 con 0x7f43080ffab0 2024-05-13T22:48:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.027+0000 7f4305ffb700 1 -- 172.21.15.102:0/3594688775 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42fc021410 con 0x7f43080ffab0 2024-05-13T22:48:12.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.027+0000 7f430eafc700 1 -- 172.21.15.102:0/3594688775 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430819e160 con 0x7f43080ffab0 2024-05-13T22:48:12.007 INFO:tasks.workunit.client.0.smithi102.stdout:4/613: dwrite d3/d4/d6/f42 [4194304,4194304] 0 2024-05-13T22:48:12.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.028+0000 7f430eafc700 1 -- 172.21.15.102:0/3594688775 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4308108610 con 0x7f43080ffab0 2024-05-13T22:48:12.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.029+0000 7f4305ffb700 1 -- 172.21.15.102:0/3594688775 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 24) v1 ==== 91288+0+0 (secure 0 0 0) 0x7f42fc00f460 con 0x7f43080ffab0 2024-05-13T22:48:12.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.030+0000 7f4305ffb700 1 --2- 172.21.15.102:0/3594688775 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f42f806d2e0 0x7f42f806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:12.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.030+0000 7f4305ffb700 1 -- 172.21.15.102:0/3594688775 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f42fc08de70 con 0x7f43080ffab0 2024-05-13T22:48:12.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.031+0000 7f4307fff700 1 --2- 172.21.15.102:0/3594688775 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f42f806d2e0 0x7f42f806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:12.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.031+0000 7f4307fff700 1 --2- 172.21.15.102:0/3594688775 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f42f806d2e0 0x7f42f806f7a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005950 tx=0x7f42f400b500 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:12.014 INFO:tasks.workunit.client.0.smithi102.stdout:4/614: write d3/d19/d23/f82 [901895,39156] 0 2024-05-13T22:48:12.015 INFO:tasks.workunit.client.0.smithi102.stdout:4/615: chown d3/d19/d23/d39 30407712 1 2024-05-13T22:48:12.016 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.037+0000 7f4305ffb700 1 -- 172.21.15.102:0/3594688775 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f42fc05b0e0 con 0x7f43080ffab0 2024-05-13T22:48:12.039 INFO:tasks.workunit.client.0.smithi102.stdout:9/585: sync 2024-05-13T22:48:12.039 INFO:tasks.workunit.client.0.smithi102.stdout:2/539: sync 2024-05-13T22:48:12.041 INFO:tasks.workunit.client.0.smithi102.stdout:2/540: stat d7/d15/fae 0 2024-05-13T22:48:12.042 INFO:tasks.workunit.client.0.smithi102.stdout:4/616: link d3/d19/l40 d3/d22/ld7 0 2024-05-13T22:48:12.046 INFO:tasks.workunit.client.0.smithi102.stdout:4/617: unlink d3/d4/d6/de/d1f/f67 0 2024-05-13T22:48:12.048 INFO:tasks.workunit.client.0.smithi102.stdout:9/586: dread d1/d6/db/f7a [0,4194304] 0 2024-05-13T22:48:12.049 INFO:tasks.workunit.client.0.smithi102.stdout:4/618: mkdir d3/dd8 0 2024-05-13T22:48:12.052 INFO:tasks.workunit.client.0.smithi102.stdout:9/587: write d1/d6/d21/d71/fb3 [616569,97216] 0 2024-05-13T22:48:12.053 INFO:tasks.workunit.client.0.smithi102.stdout:4/619: read d3/d19/d23/f3d [1449370,85521] 0 2024-05-13T22:48:12.055 INFO:tasks.workunit.client.0.smithi102.stdout:9/588: mknod d1/d6/d45/d2d/ccd 0 2024-05-13T22:48:12.059 INFO:tasks.workunit.client.0.smithi102.stdout:4/620: dwrite d3/d4/d6/fb5 [0,4194304] 0 2024-05-13T22:48:12.064 INFO:tasks.workunit.client.0.smithi102.stdout:4/621: mknod d3/d19/d23/d3a/cd9 0 2024-05-13T22:48:12.069 INFO:tasks.workunit.client.0.smithi102.stdout:2/541: dread d7/d9/d10/d1b/f8a [0,4194304] 0 2024-05-13T22:48:12.072 INFO:tasks.workunit.client.0.smithi102.stdout:4/622: dwrite d3/d4/d6/de/faa [0,4194304] 0 2024-05-13T22:48:12.074 INFO:tasks.workunit.client.0.smithi102.stdout:4/623: dread - d3/d4/d6/de/d1f/f53 zero size 2024-05-13T22:48:12.075 INFO:tasks.workunit.client.0.smithi102.stdout:2/542: symlink d7/lb4 0 2024-05-13T22:48:12.075 INFO:tasks.workunit.client.0.smithi102.stdout:2/543: dread - d7/d15/fae zero size 2024-05-13T22:48:12.082 INFO:tasks.workunit.client.0.smithi102.stdout:4/624: creat d3/d19/d23/d39/da7/da9/dd3/dd4/fda x:0 0 0 2024-05-13T22:48:12.089 INFO:tasks.workunit.client.0.smithi102.stdout:4/625: link d3/d4/l5 d3/d19/dad/dc9/dd5/ldb 0 2024-05-13T22:48:12.091 INFO:tasks.workunit.client.0.smithi102.stdout:4/626: write d3/d19/d23/d3a/f83 [2249832,30349] 0 2024-05-13T22:48:12.096 INFO:tasks.workunit.client.0.smithi102.stdout:4/627: symlink d3/ldc 0 2024-05-13T22:48:12.099 INFO:tasks.workunit.client.0.smithi102.stdout:4/628: rename d3/d4/d6/de/d8d/dc8 to d3/d19/d23/d39/ddd 0 2024-05-13T22:48:12.101 INFO:tasks.workunit.client.0.smithi102.stdout:2/544: write d7/d9/d10/d25/d56/f8c [1398673,5477] 0 2024-05-13T22:48:12.106 INFO:tasks.workunit.client.0.smithi102.stdout:2/545: unlink d7/d9/d10/d25/lb1 0 2024-05-13T22:48:12.107 INFO:tasks.workunit.client.0.smithi102.stdout:2/546: chown d7/d9/d10/d25/d56/f8c 1017303495 1 2024-05-13T22:48:12.110 INFO:tasks.workunit.client.0.smithi102.stdout:2/547: rmdir d7/d9/d10/d25/d56/d59 39 2024-05-13T22:48:12.111 INFO:tasks.workunit.client.0.smithi102.stdout:2/548: write d7/d9/d14/d61/f93 [3043035,104480] 0 2024-05-13T22:48:12.119 INFO:tasks.workunit.client.0.smithi102.stdout:2/549: dwrite d7/d9/d10/fb3 [0,4194304] 0 2024-05-13T22:48:12.120 INFO:tasks.workunit.client.0.smithi102.stdout:4/629: rename d3/d4/d6/de/d1f/d41/d6a/f7a to d3/d19/d23/d39/d5d/fde 0 2024-05-13T22:48:12.124 INFO:tasks.workunit.client.0.smithi102.stdout:4/630: unlink d3/d22/c2d 0 2024-05-13T22:48:12.126 INFO:tasks.workunit.client.0.smithi102.stdout:2/550: rename d7/d15/d67/d6a/d8b to d7/d15/d1e/da7/db5 0 2024-05-13T22:48:12.132 INFO:tasks.workunit.client.0.smithi102.stdout:2/551: getdents d7/d9/d14/d31 0 2024-05-13T22:48:12.135 INFO:tasks.workunit.client.0.smithi102.stdout:2/552: symlink d7/d9/d10/d1b/lb6 0 2024-05-13T22:48:12.139 INFO:tasks.workunit.client.0.smithi102.stdout:2/553: symlink d7/d9/d10/d1b/d54/d83/lb7 0 2024-05-13T22:48:12.140 INFO:tasks.workunit.client.0.smithi102.stdout:2/554: fsync d7/d9/f29 0 2024-05-13T22:48:12.141 INFO:tasks.workunit.client.0.smithi102.stdout:2/555: readlink d7/d9/d10/d30/l99 0 2024-05-13T22:48:12.146 INFO:tasks.workunit.client.0.smithi102.stdout:4/631: write d3/d4/f31 [1964497,27667] 0 2024-05-13T22:48:12.148 INFO:tasks.workunit.client.0.smithi102.stdout:4/632: creat d3/d6b/fdf x:0 0 0 2024-05-13T22:48:12.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:11 smithi130 ceph-mon[36591]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:12.151 INFO:tasks.workunit.client.0.smithi102.stdout:2/556: write d7/d9/d10/d1b/f6b [2777467,84100] 0 2024-05-13T22:48:12.153 INFO:tasks.workunit.client.0.smithi102.stdout:2/557: chown d7/d9/d14/l22 5771331 1 2024-05-13T22:48:12.156 INFO:tasks.workunit.client.0.smithi102.stdout:4/633: dwrite d3/d4/f93 [4194304,4194304] 0 2024-05-13T22:48:12.156 INFO:tasks.workunit.client.0.smithi102.stdout:2/558: rmdir d7/d15/d1e/da7 39 2024-05-13T22:48:12.157 INFO:tasks.workunit.client.0.smithi102.stdout:2/559: truncate d7/d15/fae 1004764 0 2024-05-13T22:48:12.158 INFO:tasks.workunit.client.0.smithi102.stdout:2/560: read d7/d9/d10/d1b/f8a [3571811,54529] 0 2024-05-13T22:48:12.169 INFO:tasks.workunit.client.0.smithi102.stdout:2/561: dwrite d7/d9/d10/d25/f35 [0,4194304] 0 2024-05-13T22:48:12.178 INFO:tasks.workunit.client.0.smithi102.stdout:2/562: dwrite d7/d15/fa5 [12582912,4194304] 0 2024-05-13T22:48:12.198 INFO:tasks.workunit.client.0.smithi102.stdout:2/563: dread d7/d15/fa5 [0,4194304] 0 2024-05-13T22:48:12.201 INFO:tasks.workunit.client.0.smithi102.stdout:2/564: unlink d7/d15/c5f 0 2024-05-13T22:48:12.204 INFO:tasks.workunit.client.0.smithi102.stdout:2/565: mknod d7/d9/d10/d30/d53/da4/cb8 0 2024-05-13T22:48:12.227 INFO:tasks.workunit.client.0.smithi102.stdout:5/734: sync 2024-05-13T22:48:12.227 INFO:tasks.workunit.client.0.smithi102.stdout:8/477: sync 2024-05-13T22:48:12.228 INFO:tasks.workunit.client.0.smithi102.stdout:8/478: stat d2/d7 0 2024-05-13T22:48:12.231 INFO:tasks.workunit.client.0.smithi102.stdout:5/735: creat de/d25/d2a/d4b/f110 x:0 0 0 2024-05-13T22:48:12.236 INFO:tasks.workunit.client.0.smithi102.stdout:5/736: fsync de/d25/f47 0 2024-05-13T22:48:12.237 INFO:tasks.workunit.client.0.smithi102.stdout:5/737: dread - de/d25/d108/dc5/dca/ffd zero size 2024-05-13T22:48:12.241 INFO:tasks.workunit.client.0.smithi102.stdout:5/738: fsync de/d25/d108/dc5/dca/ffd 0 2024-05-13T22:48:12.242 INFO:tasks.workunit.client.0.smithi102.stdout:8/479: dread d2/d7/d25/d41/d50/d40/f5b [0,4194304] 0 2024-05-13T22:48:12.245 INFO:tasks.workunit.client.0.smithi102.stdout:5/739: readlink de/d25/d59/l16 0 2024-05-13T22:48:12.246 INFO:tasks.workunit.client.0.smithi102.stdout:2/566: dread d7/d9/f13 [0,4194304] 0 2024-05-13T22:48:12.247 INFO:tasks.workunit.client.0.smithi102.stdout:2/567: chown d7/d15/d67/d6a 73547439 1 2024-05-13T22:48:12.247 INFO:tasks.workunit.client.0.smithi102.stdout:2/568: chown d7/d15/f52 29 1 2024-05-13T22:48:12.248 INFO:tasks.workunit.client.0.smithi102.stdout:8/480: rename d2/d7/f53 to d2/d7/d27/d64/d80/f92 0 2024-05-13T22:48:12.249 INFO:tasks.workunit.client.0.smithi102.stdout:8/481: chown d2/d7/d25/d41/f75 38395388 1 2024-05-13T22:48:12.250 INFO:tasks.workunit.client.0.smithi102.stdout:8/482: chown d2/d7/d27/d64/d80/f92 25910602 1 2024-05-13T22:48:12.250 INFO:tasks.workunit.client.0.smithi102.stdout:5/740: mknod de/d75/c111 0 2024-05-13T22:48:12.251 INFO:tasks.workunit.client.0.smithi102.stdout:5/741: chown de/f1d 13 1 2024-05-13T22:48:12.255 INFO:tasks.workunit.client.0.smithi102.stdout:2/569: mknod d7/d15/d67/cb9 0 2024-05-13T22:48:12.256 INFO:tasks.workunit.client.0.smithi102.stdout:2/570: dread - d7/f75 zero size 2024-05-13T22:48:12.257 INFO:tasks.workunit.client.0.smithi102.stdout:8/483: mkdir d2/d7/d25/d41/d50/d93 0 2024-05-13T22:48:12.258 INFO:tasks.workunit.client.0.smithi102.stdout:2/571: read d7/d15/f19 [1320696,72423] 0 2024-05-13T22:48:12.261 INFO:tasks.workunit.client.0.smithi102.stdout:5/742: mkdir de/d25/d43/df5/d112 0 2024-05-13T22:48:12.266 INFO:tasks.workunit.client.0.smithi102.stdout:8/484: dwrite d2/d7/d25/f6e [0,4194304] 0 2024-05-13T22:48:12.267 INFO:tasks.workunit.client.0.smithi102.stdout:2/572: dwrite d7/d9/d10/d25/d27/f45 [0,4194304] 0 2024-05-13T22:48:12.273 INFO:tasks.workunit.client.0.smithi102.stdout:5/743: truncate de/d25/d108/dc5/dca/fce 360895 0 2024-05-13T22:48:12.289 INFO:tasks.workunit.client.0.smithi102.stdout:5/744: mkdir de/d33/d113 0 2024-05-13T22:48:12.293 INFO:tasks.workunit.client.0.smithi102.stdout:2/573: rename c3 to d7/d9/d10/cba 0 2024-05-13T22:48:12.299 INFO:tasks.workunit.client.0.smithi102.stdout:8/485: rename d2/d7/d25/l44 to d2/d7/d25/l94 0 2024-05-13T22:48:12.307 INFO:tasks.workunit.client.0.smithi102.stdout:8/486: symlink d2/d7/d25/d41/d50/d40/d85/l95 0 2024-05-13T22:48:12.310 INFO:tasks.workunit.client.0.smithi102.stdout:8/487: mkdir d2/d7/d27/d64/d96 0 2024-05-13T22:48:12.313 INFO:tasks.workunit.client.0.smithi102.stdout:8/488: mkdir d2/d7/d12/d97 0 2024-05-13T22:48:12.315 INFO:tasks.workunit.client.0.smithi102.stdout:2/574: getdents d7/d15/d1e/da7 0 2024-05-13T22:48:12.315 INFO:tasks.workunit.client.0.smithi102.stdout:2/575: fsync d7/d9/d10/d30/fa9 0 2024-05-13T22:48:12.316 INFO:tasks.workunit.client.0.smithi102.stdout:5/745: write de/d25/d59/f8f [141093,6119] 0 2024-05-13T22:48:12.323 INFO:tasks.workunit.client.0.smithi102.stdout:8/489: link d2/d7/d27/d64/d80/f92 d2/d7/d25/d41/d50/d93/f98 0 2024-05-13T22:48:12.336 INFO:tasks.workunit.client.0.smithi102.stdout:5/746: write de/d25/d43/f97 [1205474,83014] 0 2024-05-13T22:48:12.337 INFO:tasks.workunit.client.0.smithi102.stdout:5/747: chown de/d25/d59/d7f/d86/l9c 719070752 1 2024-05-13T22:48:12.339 INFO:tasks.workunit.client.0.smithi102.stdout:8/490: truncate d2/d7/d27/d64/d80/f92 4028191 0 2024-05-13T22:48:12.341 INFO:tasks.workunit.client.0.smithi102.stdout:5/748: fdatasync de/d25/d59/d6f/f92 0 2024-05-13T22:48:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:2/576: truncate d7/d15/fa5 4811375 0 2024-05-13T22:48:12.344 INFO:tasks.workunit.client.0.smithi102.stdout:8/491: read d2/f22 [665877,30371] 0 2024-05-13T22:48:12.345 INFO:tasks.workunit.client.0.smithi102.stdout:2/577: chown d7/lb4 0 1 2024-05-13T22:48:12.346 INFO:tasks.workunit.client.0.smithi102.stdout:8/492: write d2/d7/d25/d41/d6f/f8a [637566,126368] 0 2024-05-13T22:48:12.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.369+0000 7f430eafc700 1 -- 172.21.15.102:0/3594688775 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4308069390 con 0x7f42f806d2e0 2024-05-13T22:48:12.349 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.370+0000 7f4305ffb700 1 -- 172.21.15.102:0/3594688775 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f4308069390 con 0x7f42f806d2e0 2024-05-13T22:48:12.349 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:48:12.349 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:48:12.349 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:48:12.349 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-13T22:48:12.349 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [], 2024-05-13T22:48:12.349 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "1/2 daemons upgraded", 2024-05-13T22:48:12.350 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "", 2024-05-13T22:48:12.350 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:48:12.350 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:48:12.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.373+0000 7f430eafc700 1 -- 172.21.15.102:0/3594688775 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f42f806d2e0 msgr2=0x7f42f806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:12.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.373+0000 7f430eafc700 1 --2- 172.21.15.102:0/3594688775 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f42f806d2e0 0x7f42f806f7a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005950 tx=0x7f42f400b500 comp rx=0 tx=0).stop 2024-05-13T22:48:12.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.373+0000 7f430eafc700 1 -- 172.21.15.102:0/3594688775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43080ffab0 msgr2=0x7f4308198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:12.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.373+0000 7f430eafc700 1 --2- 172.21.15.102:0/3594688775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43080ffab0 0x7f4308198580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009910 tx=0x7f42fc00bd70 comp rx=0 tx=0).stop 2024-05-13T22:48:12.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.373+0000 7f430eafc700 1 -- 172.21.15.102:0/3594688775 shutdown_connections 2024-05-13T22:48:12.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.373+0000 7f430eafc700 1 --2- 172.21.15.102:0/3594688775 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f42f806d2e0 0x7f42f806f7a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:12.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.373+0000 7f430eafc700 1 --2- 172.21.15.102:0/3594688775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43080ffab0 0x7f4308198580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:12.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.373+0000 7f430eafc700 1 --2- 172.21.15.102:0/3594688775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4308100cb0 0x7f4308198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:12.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.374+0000 7f430eafc700 1 -- 172.21.15.102:0/3594688775 >> 172.21.15.102:0/3594688775 conn(0x7f43080fb050 msgr2=0x7f4308068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:12.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.374+0000 7f430eafc700 1 -- 172.21.15.102:0/3594688775 shutdown_connections 2024-05-13T22:48:12.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:12.374+0000 7f430eafc700 1 -- 172.21.15.102:0/3594688775 wait complete. 2024-05-13T22:48:12.359 INFO:tasks.workunit.client.0.smithi102.stdout:2/578: dread d7/d9/d10/f37 [4194304,4194304] 0 2024-05-13T22:48:12.360 INFO:tasks.workunit.client.0.smithi102.stdout:2/579: write d7/d9/d14/d31/f64 [2375253,22192] 0 2024-05-13T22:48:12.365 INFO:tasks.workunit.client.0.smithi102.stdout:2/580: mkdir d7/d9/d14/d31/dbb 0 2024-05-13T22:48:12.370 INFO:tasks.workunit.client.0.smithi102.stdout:2/581: chown d7/d9/d10/d30/d7f/la2 28842 1 2024-05-13T22:48:12.374 INFO:tasks.workunit.client.0.smithi102.stdout:8/493: write d2/d36/d49/d5d/f72 [1669643,13391] 0 2024-05-13T22:48:12.378 INFO:tasks.workunit.client.0.smithi102.stdout:2/582: rename d7/c7e to d7/cbc 0 2024-05-13T22:48:12.396 INFO:tasks.workunit.client.0.smithi102.stdout:8/494: dwrite d2/d7/d27/f4c [0,4194304] 0 2024-05-13T22:48:12.398 INFO:tasks.workunit.client.0.smithi102.stdout:8/495: fdatasync d2/f6d 0 2024-05-13T22:48:12.408 INFO:tasks.workunit.client.0.smithi102.stdout:8/496: write d2/d7/d27/f3b [2859994,2962] 0 2024-05-13T22:48:12.411 INFO:tasks.workunit.client.0.smithi102.stdout:8/497: rmdir d2/d7/d12/d97 0 2024-05-13T22:48:12.414 INFO:tasks.workunit.client.0.smithi102.stdout:8/498: truncate d2/d7/d25/d41/d50/f34 1475638 0 2024-05-13T22:48:12.432 INFO:tasks.workunit.client.0.smithi102.stdout:8/499: dread d2/d7/d27/f3b [4194304,4194304] 0 2024-05-13T22:48:12.434 INFO:tasks.workunit.client.0.smithi102.stdout:8/500: symlink d2/d36/d49/d5d/l99 0 2024-05-13T22:48:12.437 INFO:tasks.workunit.client.0.smithi102.stdout:8/501: getdents d2/d7/d12/d76 0 2024-05-13T22:48:12.439 INFO:tasks.workunit.client.0.smithi102.stdout:8/502: symlink d2/d7/l9a 0 2024-05-13T22:48:12.497 INFO:tasks.workunit.client.0.smithi102.stdout:8/503: symlink d2/d7/d27/l9b 0 2024-05-13T22:48:12.515 INFO:tasks.workunit.client.0.smithi102.stdout:8/504: dread d2/d7/f69 [0,4194304] 0 2024-05-13T22:48:12.540 INFO:tasks.workunit.client.0.smithi102.stdout:8/505: creat d2/d7/d25/d41/d50/d40/f9c x:0 0 0 2024-05-13T22:48:12.544 INFO:tasks.workunit.client.0.smithi102.stdout:8/506: unlink d2/d7/d27/c66 0 2024-05-13T22:48:12.547 INFO:tasks.workunit.client.0.smithi102.stdout:8/507: mknod d2/d7/d27/d64/c9d 0 2024-05-13T22:48:12.548 INFO:tasks.workunit.client.0.smithi102.stdout:8/508: read - d2/f73 zero size 2024-05-13T22:48:12.549 INFO:tasks.workunit.client.0.smithi102.stdout:8/509: readlink d2/d36/d49/d5d/l2e 0 2024-05-13T22:48:12.556 INFO:tasks.workunit.client.0.smithi102.stdout:8/510: dwrite d2/d7/d27/f4c [4194304,4194304] 0 2024-05-13T22:48:12.563 INFO:tasks.workunit.client.0.smithi102.stdout:8/511: creat d2/d36/d49/d5d/f9e x:0 0 0 2024-05-13T22:48:12.569 INFO:tasks.workunit.client.0.smithi102.stdout:8/512: dwrite d2/d7/d25/d41/f75 [4194304,4194304] 0 2024-05-13T22:48:12.571 INFO:tasks.workunit.client.0.smithi102.stdout:8/513: write d2/d7/d25/f74 [2138611,14417] 0 2024-05-13T22:48:12.576 INFO:tasks.workunit.client.0.smithi102.stdout:8/514: rename d2/d36/d49/f6a to d2/d7/d12/d76/f9f 0 2024-05-13T22:48:12.579 INFO:tasks.workunit.client.0.smithi102.stdout:8/515: fsync d2/d7/f69 0 2024-05-13T22:48:12.581 INFO:tasks.workunit.client.0.smithi102.stdout:8/516: write d2/d7/f4e [4856007,61825] 0 2024-05-13T22:48:12.582 INFO:tasks.workunit.client.0.smithi102.stdout:8/517: chown d2/d36/c63 2567206 1 2024-05-13T22:48:12.582 INFO:tasks.workunit.client.0.smithi102.stdout:8/518: fsync d2/d7/d27/f77 0 2024-05-13T22:48:12.604 INFO:tasks.workunit.client.0.smithi102.stdout:8/519: dread d2/d36/f42 [0,4194304] 0 2024-05-13T22:48:12.610 INFO:tasks.workunit.client.0.smithi102.stdout:8/520: dwrite d2/d36/d49/d5d/f33 [0,4194304] 0 2024-05-13T22:48:12.614 INFO:tasks.workunit.client.0.smithi102.stdout:8/521: getdents d2/d7/d27/d64/d96 0 2024-05-13T22:48:12.616 INFO:tasks.workunit.client.0.smithi102.stdout:8/522: rmdir d2/d7/d25/d41 39 2024-05-13T22:48:12.622 INFO:tasks.workunit.client.0.smithi102.stdout:8/523: getdents d2/d36/d49/d5d 0 2024-05-13T22:48:12.629 INFO:tasks.workunit.client.0.smithi102.stdout:8/524: write d2/d7/d27/f46 [1796945,42737] 0 2024-05-13T22:48:12.632 INFO:tasks.workunit.client.0.smithi102.stdout:8/525: mkdir d2/d7/d25/d41/d6f/da0 0 2024-05-13T22:48:12.667 INFO:tasks.workunit.client.0.smithi102.stdout:8/526: fsync d2/d7/d27/f46 0 2024-05-13T22:48:12.684 INFO:tasks.workunit.client.0.smithi102.stdout:8/527: dwrite d2/d7/d25/d41/f65 [0,4194304] 0 2024-05-13T22:48:12.700 INFO:tasks.workunit.client.0.smithi102.stdout:8/528: dread d2/d7/d25/d41/d50/d40/f6c [4194304,4194304] 0 2024-05-13T22:48:12.706 INFO:tasks.workunit.client.0.smithi102.stdout:8/529: write d2/f56 [194185,114073] 0 2024-05-13T22:48:12.710 INFO:tasks.workunit.client.0.smithi102.stdout:8/530: rename d2/d7/d27/c8b to d2/d36/d49/d7e/ca1 0 2024-05-13T22:48:12.715 INFO:tasks.workunit.client.0.smithi102.stdout:8/531: dwrite d2/d7/d25/d41/d50/f35 [0,4194304] 0 2024-05-13T22:48:12.722 INFO:tasks.workunit.client.0.smithi102.stdout:8/532: truncate d2/d7/f4e 2620357 0 2024-05-13T22:48:12.729 INFO:tasks.workunit.client.0.smithi102.stdout:8/533: fsync d2/d7/d25/d41/d50/f35 0 2024-05-13T22:48:12.731 INFO:tasks.workunit.client.0.smithi102.stdout:8/534: unlink d2/f87 0 2024-05-13T22:48:12.735 INFO:tasks.workunit.client.0.smithi102.stdout:8/535: dread d2/d36/f42 [0,4194304] 0 2024-05-13T22:48:12.737 INFO:tasks.workunit.client.0.smithi102.stdout:8/536: mkdir d2/d7/da2 0 2024-05-13T22:48:12.738 INFO:tasks.workunit.client.0.smithi102.stdout:8/537: chown d2/d36/d49/d5d/f45 1386487858 1 2024-05-13T22:48:12.739 INFO:tasks.workunit.client.0.smithi102.stdout:8/538: write d2/d36/f42 [920795,56189] 0 2024-05-13T22:48:12.739 INFO:tasks.workunit.client.0.smithi102.stdout:8/539: readlink d2/d36/d49/l7c 0 2024-05-13T22:48:12.742 INFO:tasks.workunit.client.0.smithi102.stdout:8/540: rename d2/d36/d49/l5f to d2/d7/d25/d41/la3 0 2024-05-13T22:48:12.744 INFO:tasks.workunit.client.0.smithi102.stdout:8/541: creat d2/d7/d27/d64/fa4 x:0 0 0 2024-05-13T22:48:12.745 INFO:tasks.workunit.client.0.smithi102.stdout:8/542: fdatasync d2/d7/d25/d41/d50/d40/f5b 0 2024-05-13T22:48:12.747 INFO:tasks.workunit.client.0.smithi102.stdout:8/543: creat d2/d7/d25/d41/d50/d40/d85/fa5 x:0 0 0 2024-05-13T22:48:12.750 INFO:tasks.workunit.client.0.smithi102.stdout:8/544: dread d2/d7/d25/f6e [0,4194304] 0 2024-05-13T22:48:12.806 INFO:tasks.workunit.client.0.smithi102.stdout:6/644: sync 2024-05-13T22:48:12.809 INFO:tasks.workunit.client.0.smithi102.stdout:6/645: creat d9/db/d14/dcc/d5f/fdb x:0 0 0 2024-05-13T22:48:12.811 INFO:tasks.workunit.client.0.smithi102.stdout:6/646: creat d9/d45/da0/dd9/fdc x:0 0 0 2024-05-13T22:48:12.813 INFO:tasks.workunit.client.0.smithi102.stdout:6/647: mkdir d9/da3/ddd 0 2024-05-13T22:48:12.815 INFO:tasks.workunit.client.0.smithi102.stdout:6/648: link d9/d25/f88 d9/da3/fde 0 2024-05-13T22:48:12.818 INFO:tasks.workunit.client.0.smithi102.stdout:6/649: rmdir d9/da3/ddd 0 2024-05-13T22:48:13.031 INFO:tasks.workunit.client.0.smithi102.stdout:7/456: sync 2024-05-13T22:48:13.035 INFO:tasks.workunit.client.0.smithi102.stdout:7/457: rename d6/d13/d62/c64 to d6/d15/d4d/d55/d6f/d65/c81 0 2024-05-13T22:48:13.041 INFO:tasks.workunit.client.0.smithi102.stdout:7/458: dwrite d6/d13/d58/d20/d4b/f66 [0,4194304] 0 2024-05-13T22:48:13.042 INFO:tasks.workunit.client.0.smithi102.stdout:7/459: chown d6/d13/f7d 7498 1 2024-05-13T22:48:13.044 INFO:tasks.workunit.client.0.smithi102.stdout:7/460: unlink d6/f67 0 2024-05-13T22:48:13.046 INFO:tasks.workunit.client.0.smithi102.stdout:7/461: unlink d6/d13/de/fb 0 2024-05-13T22:48:13.047 INFO:tasks.workunit.client.0.smithi102.stdout:7/462: dread - d6/d15/d4d/d55/d6f/d49/f71 zero size 2024-05-13T22:48:13.094 INFO:tasks.workunit.client.0.smithi102.stdout:6/650: sync 2024-05-13T22:48:13.094 INFO:tasks.workunit.client.0.smithi102.stdout:0/606: sync 2024-05-13T22:48:13.094 INFO:tasks.workunit.client.0.smithi102.stdout:1/423: sync 2024-05-13T22:48:13.094 INFO:tasks.workunit.client.0.smithi102.stdout:9/589: sync 2024-05-13T22:48:13.094 INFO:tasks.workunit.client.0.smithi102.stdout:4/634: sync 2024-05-13T22:48:13.094 INFO:tasks.workunit.client.0.smithi102.stdout:8/545: sync 2024-05-13T22:48:13.094 INFO:tasks.workunit.client.0.smithi102.stdout:3/615: sync 2024-05-13T22:48:13.094 INFO:tasks.workunit.client.0.smithi102.stdout:2/583: sync 2024-05-13T22:48:13.094 INFO:tasks.workunit.client.0.smithi102.stdout:5/749: sync 2024-05-13T22:48:13.095 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:12 smithi102 ceph-mon[28599]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:13.095 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:12 smithi102 ceph-mon[28599]: pgmap v23: 65 pgs: 65 active+clean; 1.1 GiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 31 MiB/s rd, 101 MiB/s wr, 313 op/s 2024-05-13T22:48:13.095 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:12 smithi102 ceph-mon[28599]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:13.096 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:12 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/892473052' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:48:13.096 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:12 smithi102 ceph-mon[28599]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:13.101 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:12 smithi130 ceph-mon[36591]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:13.101 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:12 smithi130 ceph-mon[36591]: pgmap v23: 65 pgs: 65 active+clean; 1.1 GiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 31 MiB/s rd, 101 MiB/s wr, 313 op/s 2024-05-13T22:48:13.102 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:12 smithi130 ceph-mon[36591]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:13.102 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:12 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/892473052' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:48:13.102 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:12 smithi130 ceph-mon[36591]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:13.102 INFO:tasks.workunit.client.0.smithi102.stdout:0/607: mkdir d1/d4d/d79/d9f/dbd 0 2024-05-13T22:48:13.104 INFO:tasks.workunit.client.0.smithi102.stdout:5/750: dread de/d25/d59/d6f/f92 [0,4194304] 0 2024-05-13T22:48:13.107 INFO:tasks.workunit.client.0.smithi102.stdout:0/608: write d1/de/d32/f18 [1290346,113474] 0 2024-05-13T22:48:13.108 INFO:tasks.workunit.client.0.smithi102.stdout:8/546: mknod d2/d7/ca6 0 2024-05-13T22:48:13.109 INFO:tasks.workunit.client.0.smithi102.stdout:3/616: dwrite de/d10/d46/fcd [0,4194304] 0 2024-05-13T22:48:13.110 INFO:tasks.workunit.client.0.smithi102.stdout:9/590: dread - d1/d6/d45/d3e/d79/fa6 zero size 2024-05-13T22:48:13.112 INFO:tasks.workunit.client.0.smithi102.stdout:1/424: dwrite d2/d45/f48 [4194304,4194304] 0 2024-05-13T22:48:13.114 INFO:tasks.workunit.client.0.smithi102.stdout:1/425: chown d2/d4/d5/d19/d28/d6a/f59 1290 1 2024-05-13T22:48:13.116 INFO:tasks.workunit.client.0.smithi102.stdout:5/751: readlink de/d25/l38 0 2024-05-13T22:48:13.117 INFO:tasks.workunit.client.0.smithi102.stdout:2/584: dread d7/d9/d10/f37 [0,4194304] 0 2024-05-13T22:48:13.123 INFO:tasks.workunit.client.0.smithi102.stdout:3/617: readlink de/d10/d2a/d36/db2/ld8 0 2024-05-13T22:48:13.125 INFO:tasks.workunit.client.0.smithi102.stdout:3/618: chown de/d10/d50/lbf 4 1 2024-05-13T22:48:13.125 INFO:tasks.workunit.client.0.smithi102.stdout:5/752: dwrite de/d25/d43/d7a/fea [0,4194304] 0 2024-05-13T22:48:13.126 INFO:tasks.workunit.client.0.smithi102.stdout:9/591: mkdir d1/d6/db/d30/d8a/dce 0 2024-05-13T22:48:13.128 INFO:tasks.workunit.client.0.smithi102.stdout:5/753: truncate de/d25/d59/f8f 1264647 0 2024-05-13T22:48:13.128 INFO:tasks.workunit.client.0.smithi102.stdout:5/754: fdatasync de/f13 0 2024-05-13T22:48:13.132 INFO:tasks.workunit.client.0.smithi102.stdout:2/585: symlink d7/d9/d14/d61/lbd 0 2024-05-13T22:48:13.134 INFO:tasks.workunit.client.0.smithi102.stdout:9/592: dwrite d1/d6/db/d30/d47/f6d [0,4194304] 0 2024-05-13T22:48:13.136 INFO:tasks.workunit.client.0.smithi102.stdout:9/593: chown d1/d6/d45/d2d/dac/db0/fbb 3999 1 2024-05-13T22:48:13.137 INFO:tasks.workunit.client.0.smithi102.stdout:3/619: creat de/d10/d2a/d8c/da8/fd9 x:0 0 0 2024-05-13T22:48:13.148 INFO:tasks.workunit.client.0.smithi102.stdout:2/586: dread d7/d9/d10/d30/d53/f8e [0,4194304] 0 2024-05-13T22:48:13.149 INFO:tasks.workunit.client.0.smithi102.stdout:0/609: getdents d1/de/d32/d63 0 2024-05-13T22:48:13.150 INFO:tasks.workunit.client.0.smithi102.stdout:0/610: fsync d1/fb3 0 2024-05-13T22:48:13.151 INFO:tasks.workunit.client.0.smithi102.stdout:9/594: symlink d1/d6/d21/dab/lcf 0 2024-05-13T22:48:13.151 INFO:tasks.workunit.client.0.smithi102.stdout:0/611: readlink d1/d4d/l84 0 2024-05-13T22:48:13.157 INFO:tasks.workunit.client.0.smithi102.stdout:0/612: write d1/de/d32/d63/f7d [840863,40512] 0 2024-05-13T22:48:13.161 INFO:tasks.workunit.client.0.smithi102.stdout:9/595: mkdir d1/d6/d45/dd0 0 2024-05-13T22:48:13.161 INFO:tasks.workunit.client.0.smithi102.stdout:9/596: chown d1/d6/d3b/d5b/fa2 594623 1 2024-05-13T22:48:13.162 INFO:tasks.workunit.client.0.smithi102.stdout:0/613: truncate d1/de/d32/d24/f30 1021687 0 2024-05-13T22:48:13.163 INFO:tasks.workunit.client.0.smithi102.stdout:2/587: link d7/d9/d10/d1b/d54/d83/l9a d7/d9/d10/d30/d53/da4/lbe 0 2024-05-13T22:48:13.169 INFO:tasks.workunit.client.0.smithi102.stdout:0/614: dwrite d1/de/d32/d24/f74 [0,4194304] 0 2024-05-13T22:48:13.172 INFO:tasks.workunit.client.0.smithi102.stdout:7/463: write d6/f2e [2005638,93048] 0 2024-05-13T22:48:13.173 INFO:tasks.workunit.client.0.smithi102.stdout:2/588: mkdir d7/d15/d1e/dbf 0 2024-05-13T22:48:13.174 INFO:tasks.workunit.client.0.smithi102.stdout:0/615: creat d1/de/d32/d24/d2d/fbe x:0 0 0 2024-05-13T22:48:13.176 INFO:tasks.workunit.client.0.smithi102.stdout:7/464: mkdir d6/d15/d4d/d55/d82 0 2024-05-13T22:48:13.178 INFO:tasks.workunit.client.0.smithi102.stdout:2/589: symlink d7/d15/d67/d6a/lc0 0 2024-05-13T22:48:13.184 INFO:tasks.workunit.client.0.smithi102.stdout:0/616: mknod d1/d4d/cbf 0 2024-05-13T22:48:13.185 INFO:tasks.workunit.client.0.smithi102.stdout:2/590: truncate d7/d15/d1e/f5c 3897498 0 2024-05-13T22:48:13.189 INFO:tasks.workunit.client.0.smithi102.stdout:2/591: dread d7/d9/d10/d1b/f8a [0,4194304] 0 2024-05-13T22:48:13.191 INFO:tasks.workunit.client.0.smithi102.stdout:0/617: dread d1/de/d32/dab/fb0 [0,4194304] 0 2024-05-13T22:48:13.196 INFO:tasks.workunit.client.0.smithi102.stdout:2/592: creat d7/d9/d10/fc1 x:0 0 0 2024-05-13T22:48:13.197 INFO:tasks.workunit.client.0.smithi102.stdout:6/651: write d9/db/f3c [5223444,32335] 0 2024-05-13T22:48:13.197 INFO:tasks.workunit.client.0.smithi102.stdout:2/593: stat d7/d15/c7d 0 2024-05-13T22:48:13.199 INFO:tasks.workunit.client.0.smithi102.stdout:4/635: write d3/d4/d6/de/f3f [1076862,63989] 0 2024-05-13T22:48:13.200 INFO:tasks.workunit.client.0.smithi102.stdout:2/594: mknod d7/d15/d67/cc2 0 2024-05-13T22:48:13.203 INFO:tasks.workunit.client.0.smithi102.stdout:6/652: mkdir d9/d45/ddf 0 2024-05-13T22:48:13.205 INFO:tasks.workunit.client.0.smithi102.stdout:0/618: dread d1/de/d32/d63/f96 [0,4194304] 0 2024-05-13T22:48:13.208 INFO:tasks.workunit.client.0.smithi102.stdout:4/636: creat d3/d19/d23/d39/ddd/fe0 x:0 0 0 2024-05-13T22:48:13.210 INFO:tasks.workunit.client.0.smithi102.stdout:6/653: dread - d9/d25/d5a/f6d zero size 2024-05-13T22:48:13.210 INFO:tasks.workunit.client.0.smithi102.stdout:4/637: stat d3/d4/d6/de/faa 0 2024-05-13T22:48:13.213 INFO:tasks.workunit.client.0.smithi102.stdout:8/547: dwrite d2/d7/d25/d41/f48 [0,4194304] 0 2024-05-13T22:48:13.218 INFO:tasks.workunit.client.0.smithi102.stdout:6/654: symlink d9/d45/da0/dd9/le0 0 2024-05-13T22:48:13.220 INFO:tasks.workunit.client.0.smithi102.stdout:1/426: dwrite d2/d4/d38/f6e [0,4194304] 0 2024-05-13T22:48:13.221 INFO:tasks.workunit.client.0.smithi102.stdout:6/655: write d9/d45/d56/d5d/f87 [3469923,97514] 0 2024-05-13T22:48:13.223 INFO:tasks.workunit.client.0.smithi102.stdout:8/548: unlink d2/d7/d27/c52 0 2024-05-13T22:48:13.224 INFO:tasks.workunit.client.0.smithi102.stdout:1/427: chown d2/d45/f48 49 1 2024-05-13T22:48:13.228 INFO:tasks.workunit.client.0.smithi102.stdout:6/656: truncate d9/d25/d71/fb3 866705 0 2024-05-13T22:48:13.228 INFO:tasks.workunit.client.0.smithi102.stdout:2/595: dread d7/d9/d10/f33 [0,4194304] 0 2024-05-13T22:48:13.231 INFO:tasks.workunit.client.0.smithi102.stdout:5/755: dwrite de/d25/d2a/d4b/d4d/d50/d99/fc0 [0,4194304] 0 2024-05-13T22:48:13.233 INFO:tasks.workunit.client.0.smithi102.stdout:3/620: dwrite de/d10/d50/f89 [0,4194304] 0 2024-05-13T22:48:13.234 INFO:tasks.workunit.client.0.smithi102.stdout:4/638: rename d3/d4/d6/de/d8d/f91 to d3/d19/d23/d39/da7/d7d/fe1 0 2024-05-13T22:48:13.236 INFO:tasks.workunit.client.0.smithi102.stdout:4/639: chown d3/d4/d6/de 354785232 1 2024-05-13T22:48:13.237 INFO:tasks.workunit.client.0.smithi102.stdout:1/428: creat d2/d4/d38/f89 x:0 0 0 2024-05-13T22:48:13.237 INFO:tasks.workunit.client.0.smithi102.stdout:4/640: write d3/d19/dad/fbc [618880,88520] 0 2024-05-13T22:48:13.243 INFO:tasks.workunit.client.0.smithi102.stdout:2/596: creat d7/d9/d10/d1b/d54/fc3 x:0 0 0 2024-05-13T22:48:13.246 INFO:tasks.workunit.client.0.smithi102.stdout:9/597: truncate d1/d6/db/d30/d8a/f99 3300026 0 2024-05-13T22:48:13.251 INFO:tasks.workunit.client.0.smithi102.stdout:6/657: rename d9/d25/d60/d6a/f89 to d9/db/d14/d35/d47/dc1/fe1 0 2024-05-13T22:48:13.254 INFO:tasks.workunit.client.0.smithi102.stdout:1/429: mknod d2/d10/c8a 0 2024-05-13T22:48:13.254 INFO:tasks.workunit.client.0.smithi102.stdout:1/430: chown d2/d4/d26/f33 596271735 1 2024-05-13T22:48:13.255 INFO:tasks.workunit.client.0.smithi102.stdout:1/431: read d2/d4/d38/f6e [1953946,83167] 0 2024-05-13T22:48:13.256 INFO:tasks.workunit.client.0.smithi102.stdout:1/432: write d2/d4/f75 [1173542,63648] 0 2024-05-13T22:48:13.256 INFO:tasks.workunit.client.0.smithi102.stdout:7/465: write d6/d13/d58/d20/d4b/f6c [574478,2214] 0 2024-05-13T22:48:13.258 INFO:tasks.workunit.client.0.smithi102.stdout:8/549: link d2/d36/d49/d5d/f9e d2/d7/d12/d78/fa7 0 2024-05-13T22:48:13.261 INFO:tasks.workunit.client.0.smithi102.stdout:2/597: creat d7/d9/d14/d31/fc4 x:0 0 0 2024-05-13T22:48:13.267 INFO:tasks.workunit.client.0.smithi102.stdout:9/598: rename d1/d6/db/d30/c57 to d1/d6/d3b/d5b/cd1 0 2024-05-13T22:48:13.273 INFO:tasks.workunit.client.0.smithi102.stdout:0/619: write d1/de/d32/d24/d2d/d3e/f9e [617065,22865] 0 2024-05-13T22:48:13.276 INFO:tasks.workunit.client.0.smithi102.stdout:1/433: chown d2/d4/c43 0 1 2024-05-13T22:48:13.281 INFO:tasks.workunit.client.0.smithi102.stdout:8/550: symlink d2/d36/d49/la8 0 2024-05-13T22:48:13.284 INFO:tasks.workunit.client.0.smithi102.stdout:7/466: dread d6/d15/f3e [0,4194304] 0 2024-05-13T22:48:13.292 INFO:tasks.workunit.client.0.smithi102.stdout:6/658: truncate f0 1064317 0 2024-05-13T22:48:13.293 INFO:tasks.workunit.client.0.smithi102.stdout:5/756: write de/d25/d2a/d4b/d4d/d50/da3/df4/f55 [1949410,39575] 0 2024-05-13T22:48:13.297 INFO:tasks.workunit.client.0.smithi102.stdout:4/641: creat d3/d19/d23/d4e/d68/fe2 x:0 0 0 2024-05-13T22:48:13.300 INFO:tasks.workunit.client.0.smithi102.stdout:1/434: creat d2/d10/d62/d86/f8b x:0 0 0 2024-05-13T22:48:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:2/598: truncate d7/d15/fa5 1505174 0 2024-05-13T22:48:13.304 INFO:tasks.workunit.client.0.smithi102.stdout:2/599: read - d7/d15/d67/fa8 zero size 2024-05-13T22:48:13.309 INFO:tasks.workunit.client.0.smithi102.stdout:3/621: dwrite de/d1c/fcc [0,4194304] 0 2024-05-13T22:48:13.311 INFO:tasks.workunit.client.0.smithi102.stdout:6/659: symlink d9/d25/d60/d6a/dd6/le2 0 2024-05-13T22:48:13.311 INFO:tasks.workunit.client.0.smithi102.stdout:6/660: stat d9/db/d14/dcc/f57 0 2024-05-13T22:48:13.318 INFO:tasks.workunit.client.0.smithi102.stdout:6/661: dwrite d9/d45/d56/dd5/f80 [0,4194304] 0 2024-05-13T22:48:13.319 INFO:tasks.workunit.client.0.smithi102.stdout:6/662: dread - d9/db/d14/d35/d47/f8f zero size 2024-05-13T22:48:13.342 INFO:tasks.workunit.client.0.smithi102.stdout:6/663: rmdir d9/d45/d56/d5d 39 2024-05-13T22:48:13.344 INFO:tasks.workunit.client.0.smithi102.stdout:2/600: getdents d7/d9/d14/d31/dbb 0 2024-05-13T22:48:13.345 INFO:tasks.workunit.client.0.smithi102.stdout:9/599: getdents d1/d6/db/d4a/d81 0 2024-05-13T22:48:13.352 INFO:tasks.workunit.client.0.smithi102.stdout:4/642: dwrite d3/d4/d6/de/d1f/d41/f65 [0,4194304] 0 2024-05-13T22:48:13.353 INFO:tasks.workunit.client.0.smithi102.stdout:5/757: rename de/d25/d2a/d4b/d4d/d50/d99/dbb to de/d25/d2a/d4b/d4d/d114 0 2024-05-13T22:48:13.353 INFO:tasks.workunit.client.0.smithi102.stdout:4/643: stat d3/d4/d6/c44 0 2024-05-13T22:48:13.356 INFO:tasks.workunit.client.0.smithi102.stdout:1/435: creat d2/d4/d26/f8c x:0 0 0 2024-05-13T22:48:13.359 INFO:tasks.workunit.client.0.smithi102.stdout:8/551: getdents d2/d7/d25/d41/d50/d93 0 2024-05-13T22:48:13.359 INFO:tasks.workunit.client.0.smithi102.stdout:4/644: dread d3/d4/d6/de/d1f/f8c [0,4194304] 0 2024-05-13T22:48:13.360 INFO:tasks.workunit.client.0.smithi102.stdout:7/467: truncate d6/d13/d58/d20/d4b/f66 3398014 0 2024-05-13T22:48:13.360 INFO:tasks.workunit.client.0.smithi102.stdout:7/468: chown d6/d15/d4d/d55/d7f 64599 1 2024-05-13T22:48:13.361 INFO:tasks.workunit.client.0.smithi102.stdout:7/469: fdatasync d6/d15/f7e 0 2024-05-13T22:48:13.362 INFO:tasks.workunit.client.0.smithi102.stdout:6/664: mknod d9/d25/d60/d6a/dad/ce3 0 2024-05-13T22:48:13.364 INFO:tasks.workunit.client.0.smithi102.stdout:2/601: fsync d7/f75 0 2024-05-13T22:48:13.365 INFO:tasks.workunit.client.0.smithi102.stdout:4/645: dwrite d3/d4/fd [0,4194304] 0 2024-05-13T22:48:13.367 INFO:tasks.workunit.client.0.smithi102.stdout:9/600: creat d1/d6/d3b/fd2 x:0 0 0 2024-05-13T22:48:13.368 INFO:tasks.workunit.client.0.smithi102.stdout:3/622: write de/d68/faf [991844,125115] 0 2024-05-13T22:48:13.369 INFO:tasks.workunit.client.0.smithi102.stdout:3/623: readlink de/d10/d2a/d36/l8e 0 2024-05-13T22:48:13.370 INFO:tasks.workunit.client.0.smithi102.stdout:3/624: chown de/d10/d2a/d36/d88 99 1 2024-05-13T22:48:13.371 INFO:tasks.workunit.client.0.smithi102.stdout:3/625: stat de/d10/d50/d8f/c59 0 2024-05-13T22:48:13.372 INFO:tasks.workunit.client.0.smithi102.stdout:3/626: chown de/d10/d34/f38 1501608 1 2024-05-13T22:48:13.375 INFO:tasks.workunit.client.0.smithi102.stdout:5/758: unlink de/d25/d2a/d4b/l8d 0 2024-05-13T22:48:13.388 INFO:tasks.workunit.client.0.smithi102.stdout:3/627: read - de/d10/d2a/d36/db2/fbb zero size 2024-05-13T22:48:13.389 INFO:tasks.workunit.client.0.smithi102.stdout:3/628: write de/d10/d34/f9a [1592998,109953] 0 2024-05-13T22:48:13.392 INFO:tasks.workunit.client.0.smithi102.stdout:5/759: creat de/d25/d59/d7f/d86/da5/ddb/dee/f115 x:0 0 0 2024-05-13T22:48:13.394 INFO:tasks.workunit.client.0.smithi102.stdout:1/436: mknod d2/d4/c8d 0 2024-05-13T22:48:13.394 INFO:tasks.workunit.client.0.smithi102.stdout:9/601: rmdir d1/d6/d3b 39 2024-05-13T22:48:13.398 INFO:tasks.workunit.client.0.smithi102.stdout:6/665: mknod d9/d45/ce4 0 2024-05-13T22:48:13.400 INFO:tasks.workunit.client.0.smithi102.stdout:2/602: symlink d7/d9/d10/d25/d56/lc5 0 2024-05-13T22:48:13.401 INFO:tasks.workunit.client.0.smithi102.stdout:9/602: dwrite d1/d6/db/d30/f95 [0,4194304] 0 2024-05-13T22:48:13.403 INFO:tasks.workunit.client.0.smithi102.stdout:9/603: write d1/d6/d45/d3e/f56 [1241606,59588] 0 2024-05-13T22:48:13.408 INFO:tasks.workunit.client.0.smithi102.stdout:0/620: rename d1/de/d32/d24/d2d/fbe to d1/de/d32/d24/d2d/d38/fc0 0 2024-05-13T22:48:13.410 INFO:tasks.workunit.client.0.smithi102.stdout:3/629: symlink de/d10/d50/d63/lda 0 2024-05-13T22:48:13.415 INFO:tasks.workunit.client.0.smithi102.stdout:4/646: write d3/d19/d23/d39/da7/fd6 [1278482,79694] 0 2024-05-13T22:48:13.417 INFO:tasks.workunit.client.0.smithi102.stdout:1/437: read d2/d4/d5/d19/f4e [3520276,106088] 0 2024-05-13T22:48:13.417 INFO:tasks.workunit.client.0.smithi102.stdout:9/604: dread d1/d6/fa5 [0,4194304] 0 2024-05-13T22:48:13.418 INFO:tasks.workunit.client.0.smithi102.stdout:1/438: chown d2/d10/c1d 28546 1 2024-05-13T22:48:13.421 INFO:tasks.workunit.client.0.smithi102.stdout:2/603: symlink d7/d9/d10/d30/lc6 0 2024-05-13T22:48:13.430 INFO:tasks.workunit.client.0.smithi102.stdout:0/621: dread d1/de/d42/f93 [0,4194304] 0 2024-05-13T22:48:13.432 INFO:tasks.workunit.client.0.smithi102.stdout:6/666: dread d9/d45/d56/d5d/f9f [0,4194304] 0 2024-05-13T22:48:13.435 INFO:tasks.workunit.client.0.smithi102.stdout:9/605: mknod d1/d6/db/d30/cd3 0 2024-05-13T22:48:13.436 INFO:tasks.workunit.client.0.smithi102.stdout:5/760: write de/f61 [4844271,49109] 0 2024-05-13T22:48:13.440 INFO:tasks.workunit.client.0.smithi102.stdout:2/604: readlink d7/d9/l6c 0 2024-05-13T22:48:13.445 INFO:tasks.workunit.client.0.smithi102.stdout:0/622: fdatasync d1/de/d32/f3b 0 2024-05-13T22:48:13.445 INFO:tasks.workunit.client.0.smithi102.stdout:1/439: dread d2/d45/f5a [0,4194304] 0 2024-05-13T22:48:13.446 INFO:tasks.workunit.client.0.smithi102.stdout:4/647: mkdir d3/d19/d23/d4e/de3 0 2024-05-13T22:48:13.449 INFO:tasks.workunit.client.0.smithi102.stdout:9/606: stat d1/d6/d21/d71/lc9 0 2024-05-13T22:48:13.450 INFO:tasks.workunit.client.0.smithi102.stdout:9/607: write d1/d6/db/d30/d47/f6d [2775794,26921] 0 2024-05-13T22:48:13.459 INFO:tasks.workunit.client.0.smithi102.stdout:7/470: rename d6/d15/l6d to d6/l83 0 2024-05-13T22:48:13.461 INFO:tasks.workunit.client.0.smithi102.stdout:0/623: symlink d1/d4d/d79/d9f/lc1 0 2024-05-13T22:48:13.465 INFO:tasks.workunit.client.0.smithi102.stdout:4/648: mkdir d3/d19/d23/d3a/de4 0 2024-05-13T22:48:13.467 INFO:tasks.workunit.client.0.smithi102.stdout:6/667: symlink d9/d45/d56/le5 0 2024-05-13T22:48:13.468 INFO:tasks.workunit.client.0.smithi102.stdout:6/668: readlink d9/d25/d71/l73 0 2024-05-13T22:48:13.468 INFO:tasks.workunit.client.0.smithi102.stdout:1/440: dwrite d2/d4/d38/f6e [0,4194304] 0 2024-05-13T22:48:13.474 INFO:tasks.workunit.client.0.smithi102.stdout:2/605: getdents d7/d9/d10/da3 0 2024-05-13T22:48:13.482 INFO:tasks.workunit.client.0.smithi102.stdout:3/630: rename de/d10/f14 to de/d10/d46/fdb 0 2024-05-13T22:48:13.484 INFO:tasks.workunit.client.0.smithi102.stdout:5/761: dwrite de/d25/d2a/d4b/fc1 [0,4194304] 0 2024-05-13T22:48:13.491 INFO:tasks.workunit.client.0.smithi102.stdout:6/669: dread d9/db/d14/d35/f9d [0,4194304] 0 2024-05-13T22:48:13.493 INFO:tasks.workunit.client.0.smithi102.stdout:6/670: dread d9/db/d14/dcc/f65 [0,4194304] 0 2024-05-13T22:48:13.526 INFO:tasks.workunit.client.0.smithi102.stdout:7/471: mknod d6/d15/d4d/d55/d82/c84 0 2024-05-13T22:48:13.527 INFO:tasks.workunit.client.0.smithi102.stdout:0/624: getdents d1/de/d32/d8d 0 2024-05-13T22:48:13.530 INFO:tasks.workunit.client.0.smithi102.stdout:9/608: rename d1/d6b/f9e to d1/d6/d45/d2d/fd4 0 2024-05-13T22:48:13.531 INFO:tasks.workunit.client.0.smithi102.stdout:4/649: creat d3/d19/d23/d39/dac/fe5 x:0 0 0 2024-05-13T22:48:13.534 INFO:tasks.workunit.client.0.smithi102.stdout:5/762: mkdir de/d25/d59/d7f/d86/da5/ddb/dee/d116 0 2024-05-13T22:48:13.538 INFO:tasks.workunit.client.0.smithi102.stdout:7/472: write d6/d13/d2c/f5d [191036,105500] 0 2024-05-13T22:48:13.539 INFO:tasks.workunit.client.0.smithi102.stdout:0/625: fdatasync d1/de/d32/dab/fb0 0 2024-05-13T22:48:13.542 INFO:tasks.workunit.client.0.smithi102.stdout:2/606: rename d7/d9/d10/d25/l4e to d7/d9/d10/d30/d53/lc7 0 2024-05-13T22:48:13.545 INFO:tasks.workunit.client.0.smithi102.stdout:4/650: unlink d3/d19/d23/d39/d5d/f88 0 2024-05-13T22:48:13.548 INFO:tasks.workunit.client.0.smithi102.stdout:1/441: creat d2/d4/d26/f8e x:0 0 0 2024-05-13T22:48:13.557 INFO:tasks.workunit.client.0.smithi102.stdout:0/626: dwrite d1/de/d32/d24/d2d/f64 [0,4194304] 0 2024-05-13T22:48:13.561 INFO:tasks.workunit.client.0.smithi102.stdout:1/442: mknod d2/d4/c8f 0 2024-05-13T22:48:13.562 INFO:tasks.workunit.client.0.smithi102.stdout:6/671: getdents d9/d25/d60 0 2024-05-13T22:48:13.566 INFO:tasks.workunit.client.0.smithi102.stdout:7/473: rename d6/d15/f1b to d6/f85 0 2024-05-13T22:48:13.568 INFO:tasks.workunit.client.0.smithi102.stdout:0/627: truncate d1/de/d32/d24/d2d/d38/d3d/f9a 1243070 0 2024-05-13T22:48:13.568 INFO:tasks.workunit.client.0.smithi102.stdout:9/609: truncate d1/d6/d3b/d5b/f62 3931934 0 2024-05-13T22:48:13.573 INFO:tasks.workunit.client.0.smithi102.stdout:4/651: dread d3/d19/d23/d39/d5d/f98 [0,4194304] 0 2024-05-13T22:48:13.573 INFO:tasks.workunit.client.0.smithi102.stdout:6/672: rmdir d9/d25/d5a 39 2024-05-13T22:48:13.574 INFO:tasks.workunit.client.0.smithi102.stdout:1/443: dread d2/d45/f5a [0,4194304] 0 2024-05-13T22:48:13.575 INFO:tasks.workunit.client.0.smithi102.stdout:1/444: read d2/d10/f5d [13837,14143] 0 2024-05-13T22:48:13.575 INFO:tasks.workunit.client.0.smithi102.stdout:2/607: rename d7/d9/d10/d30/d53/l63 to d7/d9/d10/d1b/lc8 0 2024-05-13T22:48:13.575 INFO:tasks.workunit.client.0.smithi102.stdout:6/673: read f7 [7511530,79384] 0 2024-05-13T22:48:13.577 INFO:tasks.workunit.client.0.smithi102.stdout:7/474: creat d6/d15/d4d/d55/d6f/d65/f86 x:0 0 0 2024-05-13T22:48:13.578 INFO:tasks.workunit.client.0.smithi102.stdout:0/628: read d1/f60 [175337,80621] 0 2024-05-13T22:48:13.655 INFO:tasks.workunit.client.0.smithi102.stdout:4/652: write d3/d4/f7b [710310,76921] 0 2024-05-13T22:48:13.657 INFO:tasks.workunit.client.0.smithi102.stdout:4/653: write d3/d19/d23/d39/d5d/f96 [1298286,110849] 0 2024-05-13T22:48:13.677 INFO:tasks.workunit.client.0.smithi102.stdout:1/445: unlink d2/d4/d26/d40/d5e/l77 0 2024-05-13T22:48:13.682 INFO:tasks.workunit.client.0.smithi102.stdout:9/610: symlink d1/d6/d45/d3e/dbe/ld5 0 2024-05-13T22:48:13.689 INFO:tasks.workunit.client.0.smithi102.stdout:0/629: dwrite d1/de/d42/f93 [0,4194304] 0 2024-05-13T22:48:13.693 INFO:tasks.workunit.client.0.smithi102.stdout:7/475: symlink d6/d13/d58/l87 0 2024-05-13T22:48:13.695 INFO:tasks.workunit.client.0.smithi102.stdout:7/476: readlink d6/d15/d4d/d55/d6f/l73 0 2024-05-13T22:48:13.702 INFO:tasks.workunit.client.0.smithi102.stdout:7/477: mknod d6/d15/d4d/d55/d82/c88 0 2024-05-13T22:48:13.707 INFO:tasks.workunit.client.0.smithi102.stdout:4/654: link d3/d19/d23/d39/d5d/fde d3/d19/d23/d4e/d68/fe6 0 2024-05-13T22:48:13.708 INFO:tasks.workunit.client.0.smithi102.stdout:1/446: creat d2/d4/d26/f90 x:0 0 0 2024-05-13T22:48:13.711 INFO:tasks.workunit.client.0.smithi102.stdout:7/478: rmdir d6/d15/d4c 39 2024-05-13T22:48:13.714 INFO:tasks.workunit.client.0.smithi102.stdout:1/447: mkdir d2/d10/d62/d86/d91 0 2024-05-13T22:48:13.717 INFO:tasks.workunit.client.0.smithi102.stdout:0/630: write d1/de/d32/dab/fb0 [2255360,107891] 0 2024-05-13T22:48:13.718 INFO:tasks.workunit.client.0.smithi102.stdout:0/631: chown d1/fb3 13045004 1 2024-05-13T22:48:13.719 INFO:tasks.workunit.client.0.smithi102.stdout:7/479: symlink d6/d15/d4d/d55/d6f/l89 0 2024-05-13T22:48:13.720 INFO:tasks.workunit.client.0.smithi102.stdout:7/480: stat d6/d15/f3b 0 2024-05-13T22:48:13.721 INFO:tasks.workunit.client.0.smithi102.stdout:4/655: creat d3/d19/d23/d39/fe7 x:0 0 0 2024-05-13T22:48:13.726 INFO:tasks.workunit.client.0.smithi102.stdout:3/631: truncate de/d10/d2a/d36/f58 2611036 0 2024-05-13T22:48:13.728 INFO:tasks.workunit.client.0.smithi102.stdout:5/763: write de/d25/d2a/f9f [513202,36022] 0 2024-05-13T22:48:13.730 INFO:tasks.workunit.client.0.smithi102.stdout:7/481: creat d6/d13/d58/d20/d80/f8a x:0 0 0 2024-05-13T22:48:13.731 INFO:tasks.workunit.client.0.smithi102.stdout:1/448: dread d2/f46 [0,4194304] 0 2024-05-13T22:48:13.732 INFO:tasks.workunit.client.0.smithi102.stdout:1/449: stat d2/d4/d5 0 2024-05-13T22:48:13.737 INFO:tasks.workunit.client.0.smithi102.stdout:4/656: link d3/l9c d3/d19/dad/dc9/le8 0 2024-05-13T22:48:13.749 INFO:tasks.workunit.client.0.smithi102.stdout:2/608: write d7/d9/d10/d1b/f8a [2401303,89352] 0 2024-05-13T22:48:13.752 INFO:tasks.workunit.client.0.smithi102.stdout:5/764: creat de/d33/d34/f117 x:0 0 0 2024-05-13T22:48:13.756 INFO:tasks.workunit.client.0.smithi102.stdout:6/674: dwrite d9/d25/f40 [0,4194304] 0 2024-05-13T22:48:13.756 INFO:tasks.workunit.client.0.smithi102.stdout:5/765: creat de/d25/d59/d7f/d86/da5/f118 x:0 0 0 2024-05-13T22:48:13.761 INFO:tasks.workunit.client.0.smithi102.stdout:2/609: rmdir d7/d9/d10/da3 0 2024-05-13T22:48:13.765 INFO:tasks.workunit.client.0.smithi102.stdout:9/611: dwrite d1/d6/d45/d3e/f73 [0,4194304] 0 2024-05-13T22:48:13.768 INFO:tasks.workunit.client.0.smithi102.stdout:6/675: getdents d9/d25/d60/d6a/dd6 0 2024-05-13T22:48:13.770 INFO:tasks.workunit.client.0.smithi102.stdout:6/676: dread - d9/db/d14/d35/fc5 zero size 2024-05-13T22:48:13.771 INFO:tasks.workunit.client.0.smithi102.stdout:2/610: dwrite d7/d9/d14/f36 [0,4194304] 0 2024-05-13T22:48:13.771 INFO:tasks.workunit.client.0.smithi102.stdout:6/677: chown d9/db/d14/dcc/d5f 10739195 1 2024-05-13T22:48:13.773 INFO:tasks.workunit.client.0.smithi102.stdout:9/612: fdatasync d1/d6/db/d4a/d75/f82 0 2024-05-13T22:48:13.775 INFO:tasks.workunit.client.0.smithi102.stdout:6/678: mknod d9/da3/ce6 0 2024-05-13T22:48:13.778 INFO:tasks.workunit.client.0.smithi102.stdout:9/613: chown d1/d6/d45/da1/fb7 97 1 2024-05-13T22:48:13.781 INFO:tasks.workunit.client.0.smithi102.stdout:0/632: write d1/de/d32/f3b [3266864,58121] 0 2024-05-13T22:48:13.782 INFO:tasks.workunit.client.0.smithi102.stdout:6/679: dwrite d9/db/d2f/fcd [0,4194304] 0 2024-05-13T22:48:13.784 INFO:tasks.workunit.client.0.smithi102.stdout:0/633: truncate d1/de/d32/f89 658568 0 2024-05-13T22:48:13.786 INFO:tasks.workunit.client.0.smithi102.stdout:2/611: rename d7/d9/l9f to d7/d9/d11/lc9 0 2024-05-13T22:48:13.794 INFO:tasks.workunit.client.0.smithi102.stdout:0/634: readlink d1/l2a 0 2024-05-13T22:48:13.795 INFO:tasks.workunit.client.0.smithi102.stdout:3/632: write de/d10/d2a/d33/fb8 [889273,98243] 0 2024-05-13T22:48:13.801 INFO:tasks.workunit.client.0.smithi102.stdout:7/482: write d6/d13/d58/d20/d4b/f66 [1402630,27443] 0 2024-05-13T22:48:13.803 INFO:tasks.workunit.client.0.smithi102.stdout:4/657: write d3/d19/d23/d7e/f9f [1000998,97489] 0 2024-05-13T22:48:13.804 INFO:tasks.workunit.client.0.smithi102.stdout:1/450: dwrite d2/d4/d5/d19/d28/f35 [0,4194304] 0 2024-05-13T22:48:13.805 INFO:tasks.workunit.client.0.smithi102.stdout:4/658: stat d3/d19/d23/d3a 0 2024-05-13T22:48:13.806 INFO:tasks.workunit.client.0.smithi102.stdout:1/451: stat d2/d4/d26/d56 0 2024-05-13T22:48:13.807 INFO:tasks.workunit.client.0.smithi102.stdout:0/635: truncate d1/f9 2453014 0 2024-05-13T22:48:13.812 INFO:tasks.workunit.client.0.smithi102.stdout:7/483: dwrite d6/d15/f18 [4194304,4194304] 0 2024-05-13T22:48:13.815 INFO:tasks.workunit.client.0.smithi102.stdout:1/452: dwrite d2/d10/d62/d7b/f82 [0,4194304] 0 2024-05-13T22:48:13.816 INFO:tasks.workunit.client.0.smithi102.stdout:1/453: read - d2/d45/f84 zero size 2024-05-13T22:48:13.817 INFO:tasks.workunit.client.0.smithi102.stdout:6/680: dread d9/db/f36 [0,4194304] 0 2024-05-13T22:48:13.906 INFO:tasks.workunit.client.0.smithi102.stdout:4/659: symlink d3/d22/le9 0 2024-05-13T22:48:13.907 INFO:tasks.workunit.client.0.smithi102.stdout:0/636: creat d1/d77/fc2 x:0 0 0 2024-05-13T22:48:13.909 INFO:tasks.workunit.client.0.smithi102.stdout:0/637: read d1/fb3 [3452569,60561] 0 2024-05-13T22:48:13.914 INFO:tasks.workunit.client.0.smithi102.stdout:4/660: dwrite d3/d4/d6/de/d1f/d41/d6a/fa1 [0,4194304] 0 2024-05-13T22:48:13.916 INFO:tasks.workunit.client.0.smithi102.stdout:0/638: mknod d1/d4d/d79/d9f/cc3 0 2024-05-13T22:48:13.919 INFO:tasks.workunit.client.0.smithi102.stdout:1/454: rename d2/d4/d26/d56 to d2/d45/d79/d92 0 2024-05-13T22:48:13.923 INFO:tasks.workunit.client.0.smithi102.stdout:9/614: write d1/d6/d45/d3e/d79/fae [420932,113875] 0 2024-05-13T22:48:13.925 INFO:tasks.workunit.client.0.smithi102.stdout:3/633: write de/d10/d2a/d33/d61/f4c [211904,61077] 0 2024-05-13T22:48:13.930 INFO:tasks.workunit.client.0.smithi102.stdout:1/455: dwrite d2/d4/d5/d19/d28/d6a/f59 [0,4194304] 0 2024-05-13T22:48:13.931 INFO:tasks.workunit.client.0.smithi102.stdout:3/634: write de/d10/d50/fab [2737961,46995] 0 2024-05-13T22:48:13.937 INFO:tasks.workunit.client.0.smithi102.stdout:4/661: dwrite d3/d19/d23/d39/d77/f7c [0,4194304] 0 2024-05-13T22:48:13.939 INFO:tasks.workunit.client.0.smithi102.stdout:9/615: dwrite d1/d6/db/d77/f7b [0,4194304] 0 2024-05-13T22:48:13.940 INFO:tasks.workunit.client.0.smithi102.stdout:0/639: creat d1/de/d32/da6/fc4 x:0 0 0 2024-05-13T22:48:13.942 INFO:tasks.workunit.client.0.smithi102.stdout:2/612: creat d7/d9/d10/d30/d53/da4/fca x:0 0 0 2024-05-13T22:48:13.949 INFO:tasks.workunit.client.0.smithi102.stdout:3/635: truncate de/d10/d50/f66 1001026 0 2024-05-13T22:48:13.949 INFO:tasks.workunit.client.0.smithi102.stdout:7/484: dwrite d6/d15/f3b [0,4194304] 0 2024-05-13T22:48:13.951 INFO:tasks.workunit.client.0.smithi102.stdout:6/681: write d9/d25/d5a/f6d [930409,55763] 0 2024-05-13T22:48:13.955 INFO:tasks.workunit.client.0.smithi102.stdout:9/616: rename d1/d6/d21/dab to d1/d6/db/d4a/d81/dd6 0 2024-05-13T22:48:13.956 INFO:tasks.workunit.client.0.smithi102.stdout:9/617: dread - d1/d6/d21/f42 zero size 2024-05-13T22:48:13.961 INFO:tasks.workunit.client.0.smithi102.stdout:3/636: mkdir de/d10/d2a/d36/db2/ddc 0 2024-05-13T22:48:13.962 INFO:tasks.workunit.client.0.smithi102.stdout:7/485: symlink d6/d13/d62/l8b 0 2024-05-13T22:48:13.965 INFO:tasks.workunit.client.0.smithi102.stdout:6/682: unlink d9/db/d14/d35/c42 0 2024-05-13T22:48:13.967 INFO:tasks.workunit.client.0.smithi102.stdout:6/683: chown d9/db/f7d 1 1 2024-05-13T22:48:13.970 INFO:tasks.workunit.client.0.smithi102.stdout:4/662: dwrite d3/d19/d23/d39/da7/f75 [0,4194304] 0 2024-05-13T22:48:13.972 INFO:tasks.workunit.client.0.smithi102.stdout:1/456: creat d2/d4/d26/d40/f93 x:0 0 0 2024-05-13T22:48:13.973 INFO:tasks.workunit.client.0.smithi102.stdout:1/457: stat d2/d4/d26/f33 0 2024-05-13T22:48:13.977 INFO:tasks.workunit.client.0.smithi102.stdout:2/613: link d7/d15/fae d7/d9/d10/d1b/d54/fcb 0 2024-05-13T22:48:13.978 INFO:tasks.workunit.client.0.smithi102.stdout:3/637: truncate de/d10/d2a/f2e 930012 0 2024-05-13T22:48:13.985 INFO:tasks.workunit.client.0.smithi102.stdout:6/684: truncate d9/d25/fb1 1003822 0 2024-05-13T22:48:13.986 INFO:tasks.workunit.client.0.smithi102.stdout:6/685: stat d9/d25/d60/d6a/dad/lb2 0 2024-05-13T22:48:13.988 INFO:tasks.workunit.client.0.smithi102.stdout:9/618: write d1/d6/db/d30/d47/f90 [870664,81930] 0 2024-05-13T22:48:13.989 INFO:tasks.workunit.client.0.smithi102.stdout:4/663: symlink d3/d19/d23/d39/da7/da9/dd3/lea 0 2024-05-13T22:48:13.991 INFO:tasks.workunit.client.0.smithi102.stdout:1/458: unlink d2/d4/d5/d19/f78 0 2024-05-13T22:48:13.996 INFO:tasks.workunit.client.0.smithi102.stdout:7/486: creat d6/d15/d4c/f8c x:0 0 0 2024-05-13T22:48:13.997 INFO:tasks.workunit.client.0.smithi102.stdout:2/614: dread d7/d9/d10/d30/f80 [0,4194304] 0 2024-05-13T22:48:13.998 INFO:tasks.workunit.client.0.smithi102.stdout:0/640: getdents d1/de/d32/d24/d2d/d3e 0 2024-05-13T22:48:13.998 INFO:tasks.workunit.client.0.smithi102.stdout:0/641: chown d1/l99 26971 1 2024-05-13T22:48:13.998 INFO:tasks.workunit.client.0.smithi102.stdout:2/615: chown d7/d9/d10/d1b/d54/fcb 2020630 1 2024-05-13T22:48:14.003 INFO:tasks.workunit.client.0.smithi102.stdout:9/619: rename d1/d6/db/d4a/fc4 to d1/d6/db/d4a/fd7 0 2024-05-13T22:48:14.073 INFO:tasks.workunit.client.0.smithi102.stdout:8/552: sync 2024-05-13T22:48:14.075 INFO:tasks.workunit.client.0.smithi102.stdout:7/487: rmdir d6/d13/d58/d20/d4b 39 2024-05-13T22:48:14.081 INFO:tasks.workunit.client.0.smithi102.stdout:1/459: rename d2/d4/d5/d19/d28/d6a/c70 to d2/d4/d26/d40/c94 0 2024-05-13T22:48:14.083 INFO:tasks.workunit.client.0.smithi102.stdout:7/488: rename d6 to d6/d13/d62/d8d 22 2024-05-13T22:48:14.087 INFO:tasks.workunit.client.0.smithi102.stdout:6/686: dread d9/db/d14/dcc/f43 [0,4194304] 0 2024-05-13T22:48:14.089 INFO:tasks.workunit.client.0.smithi102.stdout:2/616: dwrite d7/f75 [0,4194304] 0 2024-05-13T22:48:14.090 INFO:tasks.workunit.client.0.smithi102.stdout:9/620: dwrite d1/d6/d45/d2d/f2e [4194304,4194304] 0 2024-05-13T22:48:14.090 INFO:tasks.workunit.client.0.smithi102.stdout:1/460: rmdir d2/d4/d5/d19/d28/d6a 39 2024-05-13T22:48:14.093 INFO:tasks.workunit.client.0.smithi102.stdout:4/664: dread d3/d19/d23/d39/fc1 [0,4194304] 0 2024-05-13T22:48:14.097 INFO:tasks.workunit.client.0.smithi102.stdout:8/553: dread d2/d7/d27/f31 [0,4194304] 0 2024-05-13T22:48:14.098 INFO:tasks.workunit.client.0.smithi102.stdout:9/621: creat d1/d6/db/d4a/d64/fd8 x:0 0 0 2024-05-13T22:48:14.104 INFO:tasks.workunit.client.0.smithi102.stdout:1/461: creat d2/d10/d62/d7b/f95 x:0 0 0 2024-05-13T22:48:14.104 INFO:tasks.workunit.client.0.smithi102.stdout:6/687: dwrite d9/db/d14/dcc/d5f/fa4 [0,4194304] 0 2024-05-13T22:48:14.109 INFO:tasks.workunit.client.0.smithi102.stdout:4/665: unlink d3/d19/d23/f43 0 2024-05-13T22:48:14.113 INFO:tasks.workunit.client.0.smithi102.stdout:6/688: dwrite d9/d45/d56/dd5/f80 [0,4194304] 0 2024-05-13T22:48:14.115 INFO:tasks.workunit.client.0.smithi102.stdout:6/689: chown d9/db/f77 4338 1 2024-05-13T22:48:14.122 INFO:tasks.workunit.client.0.smithi102.stdout:9/622: mkdir d1/d6/db/d4a/d75/d96/dc6/dd9 0 2024-05-13T22:48:14.123 INFO:tasks.workunit.client.0.smithi102.stdout:1/462: mkdir d2/d45/d96 0 2024-05-13T22:48:14.125 INFO:tasks.workunit.client.0.smithi102.stdout:6/690: dread - d9/db/d14/dcc/d5f/f9e zero size 2024-05-13T22:48:14.126 INFO:tasks.workunit.client.0.smithi102.stdout:6/691: stat f7 0 2024-05-13T22:48:14.130 INFO:tasks.workunit.client.0.smithi102.stdout:4/666: creat d3/d19/d23/d4e/feb x:0 0 0 2024-05-13T22:48:14.133 INFO:tasks.workunit.client.0.smithi102.stdout:6/692: creat d9/d25/d5a/fe7 x:0 0 0 2024-05-13T22:48:14.136 INFO:tasks.workunit.client.0.smithi102.stdout:1/463: symlink d2/d4/d26/d40/l97 0 2024-05-13T22:48:14.138 INFO:tasks.workunit.client.0.smithi102.stdout:4/667: mkdir d3/d4/d6/de/d8d/dec 0 2024-05-13T22:48:14.140 INFO:tasks.workunit.client.0.smithi102.stdout:7/489: truncate d6/d13/d58/d20/f4a 2928373 0 2024-05-13T22:48:14.141 INFO:tasks.workunit.client.0.smithi102.stdout:7/490: stat d6/l1f 0 2024-05-13T22:48:14.142 INFO:tasks.workunit.client.0.smithi102.stdout:8/554: dread d2/d7/d27/f46 [0,4194304] 0 2024-05-13T22:48:14.143 INFO:tasks.workunit.client.0.smithi102.stdout:8/555: stat d2/d7/l9a 0 2024-05-13T22:48:14.145 INFO:tasks.workunit.client.0.smithi102.stdout:4/668: dwrite d3/d4/d6/fb5 [0,4194304] 0 2024-05-13T22:48:14.146 INFO:tasks.workunit.client.0.smithi102.stdout:9/623: getdents d1/d6/d45/d2d/dac/db0/db8 0 2024-05-13T22:48:14.146 INFO:tasks.workunit.client.0.smithi102.stdout:2/617: dread d7/d9/d11/f38 [0,4194304] 0 2024-05-13T22:48:14.147 INFO:tasks.workunit.client.0.smithi102.stdout:1/464: read d2/ff [453213,83412] 0 2024-05-13T22:48:14.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:13 smithi130 ceph-mon[36591]: pgmap v24: 65 pgs: 65 active+clean; 1.4 GiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 51 MiB/s rd, 148 MiB/s wr, 473 op/s 2024-05-13T22:48:14.157 INFO:tasks.workunit.client.0.smithi102.stdout:4/669: rename d3/d19/dad/dc9/ccc to d3/d19/dad/dc9/dd5/ced 0 2024-05-13T22:48:14.163 INFO:tasks.workunit.client.0.smithi102.stdout:9/624: creat d1/d6/d21/d71/d91/fda x:0 0 0 2024-05-13T22:48:14.166 INFO:tasks.workunit.client.0.smithi102.stdout:6/693: getdents d9/db/d14/d35/d47/dc1 0 2024-05-13T22:48:14.167 INFO:tasks.workunit.client.0.smithi102.stdout:6/694: dread - d9/db/d14/dcc/d5f/f9e zero size 2024-05-13T22:48:14.170 INFO:tasks.workunit.client.0.smithi102.stdout:2/618: rename d7/d15/d67/d6a to d7/d9/d10/d25/d56/d59/d81/dcc 0 2024-05-13T22:48:14.171 INFO:tasks.workunit.client.0.smithi102.stdout:1/465: creat d2/d4/d5/d19/d28/d6a/f98 x:0 0 0 2024-05-13T22:48:14.174 INFO:tasks.workunit.client.0.smithi102.stdout:7/491: rmdir d6/d15/d4d/d55/d6f/d6e 0 2024-05-13T22:48:14.175 INFO:tasks.workunit.client.0.smithi102.stdout:6/695: mkdir d9/da3/de8 0 2024-05-13T22:48:14.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:13 smithi102 ceph-mon[28599]: pgmap v24: 65 pgs: 65 active+clean; 1.4 GiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 51 MiB/s rd, 148 MiB/s wr, 473 op/s 2024-05-13T22:48:14.177 INFO:tasks.workunit.client.0.smithi102.stdout:1/466: dwrite d2/d4/d5/d19/d28/d6a/f59 [0,4194304] 0 2024-05-13T22:48:14.179 INFO:tasks.workunit.client.0.smithi102.stdout:4/670: mkdir d3/dee 0 2024-05-13T22:48:14.184 INFO:tasks.workunit.client.0.smithi102.stdout:9/625: write d1/d6/d21/f42 [640052,83281] 0 2024-05-13T22:48:14.186 INFO:tasks.workunit.client.0.smithi102.stdout:7/492: dread - d6/d15/f47 zero size 2024-05-13T22:48:14.186 INFO:tasks.workunit.client.0.smithi102.stdout:6/696: fdatasync d9/db/d14/dcc/f65 0 2024-05-13T22:48:14.188 INFO:tasks.workunit.client.0.smithi102.stdout:1/467: dread d2/d10/d62/d7b/f7c [0,4194304] 0 2024-05-13T22:48:14.190 INFO:tasks.workunit.client.0.smithi102.stdout:2/619: creat d7/d15/d1e/d87/fcd x:0 0 0 2024-05-13T22:48:14.191 INFO:tasks.workunit.client.0.smithi102.stdout:2/620: fsync d7/d15/d67/fa8 0 2024-05-13T22:48:14.192 INFO:tasks.workunit.client.0.smithi102.stdout:2/621: read - d7/d15/fad zero size 2024-05-13T22:48:14.192 INFO:tasks.workunit.client.0.smithi102.stdout:8/556: getdents d2/d83 0 2024-05-13T22:48:14.201 INFO:tasks.workunit.client.0.smithi102.stdout:8/557: dread d2/d36/d49/d5d/f72 [0,4194304] 0 2024-05-13T22:48:14.204 INFO:tasks.workunit.client.0.smithi102.stdout:2/622: dread d7/d15/f52 [0,4194304] 0 2024-05-13T22:48:14.205 INFO:tasks.workunit.client.0.smithi102.stdout:1/468: dread d2/d4/d5/d19/f53 [4194304,4194304] 0 2024-05-13T22:48:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:4/671: truncate d3/d4/d6/f18 7446650 0 2024-05-13T22:48:14.209 INFO:tasks.workunit.client.0.smithi102.stdout:4/672: readlink d3/ldc 0 2024-05-13T22:48:14.210 INFO:tasks.workunit.client.0.smithi102.stdout:2/623: write d7/d9/d14/d61/f9d [925347,63232] 0 2024-05-13T22:48:14.211 INFO:tasks.workunit.client.0.smithi102.stdout:6/697: getdents d9/d25/d5a 0 2024-05-13T22:48:14.213 INFO:tasks.workunit.client.0.smithi102.stdout:8/558: symlink d2/d7/d27/d64/d96/la9 0 2024-05-13T22:48:14.214 INFO:tasks.workunit.client.0.smithi102.stdout:8/559: fdatasync d2/d7/d12/d76/f90 0 2024-05-13T22:48:14.214 INFO:tasks.workunit.client.0.smithi102.stdout:2/624: rmdir d7/d9/d10/d30/d53 39 2024-05-13T22:48:14.215 INFO:tasks.workunit.client.0.smithi102.stdout:8/560: chown d2/d36/d49/l7c 15553479 1 2024-05-13T22:48:14.215 INFO:tasks.workunit.client.0.smithi102.stdout:4/673: truncate d3/d19/d23/d4e/d68/f87 1041576 0 2024-05-13T22:48:14.216 INFO:tasks.workunit.client.0.smithi102.stdout:8/561: dread - d2/d7/d27/d64/fa4 zero size 2024-05-13T22:48:14.218 INFO:tasks.workunit.client.0.smithi102.stdout:6/698: mkdir d9/d25/d60/d6a/dd6/de9 0 2024-05-13T22:48:14.220 INFO:tasks.workunit.client.0.smithi102.stdout:2/625: rmdir d7/d9/d14/d31 39 2024-05-13T22:48:14.222 INFO:tasks.workunit.client.0.smithi102.stdout:6/699: dread d9/d25/d5a/f6d [0,4194304] 0 2024-05-13T22:48:14.227 INFO:tasks.workunit.client.0.smithi102.stdout:1/469: dread d2/d4/f51 [0,4194304] 0 2024-05-13T22:48:14.230 INFO:tasks.workunit.client.0.smithi102.stdout:5/766: sync 2024-05-13T22:48:14.234 INFO:tasks.workunit.client.0.smithi102.stdout:4/674: dwrite d3/d19/d23/d3a/f4a [0,4194304] 0 2024-05-13T22:48:14.235 INFO:tasks.workunit.client.0.smithi102.stdout:8/562: link d2/d36/d49/d5d/c1e d2/d36/d49/d5d/caa 0 2024-05-13T22:48:14.235 INFO:tasks.workunit.client.0.smithi102.stdout:5/767: chown de/d25/d59/d7f/d86/da5/ddb/dee/f115 5537108 1 2024-05-13T22:48:14.236 INFO:tasks.workunit.client.0.smithi102.stdout:6/700: dwrite d9/db/d14/d54/fa9 [4194304,4194304] 0 2024-05-13T22:48:14.237 INFO:tasks.workunit.client.0.smithi102.stdout:2/626: creat d7/d9/d10/d30/fce x:0 0 0 2024-05-13T22:48:14.238 INFO:tasks.workunit.client.0.smithi102.stdout:1/470: mknod d2/d4/c99 0 2024-05-13T22:48:14.243 INFO:tasks.workunit.client.0.smithi102.stdout:5/768: creat de/d25/d43/d7a/f119 x:0 0 0 2024-05-13T22:48:14.247 INFO:tasks.workunit.client.0.smithi102.stdout:8/563: mkdir d2/d7/d25/d41/d50/d93/dab 0 2024-05-13T22:48:14.248 INFO:tasks.workunit.client.0.smithi102.stdout:8/564: dread - d2/d7/d12/f7b zero size 2024-05-13T22:48:14.251 INFO:tasks.workunit.client.0.smithi102.stdout:4/675: creat d3/d4/d6/de/d8d/dec/fef x:0 0 0 2024-05-13T22:48:14.254 INFO:tasks.workunit.client.0.smithi102.stdout:6/701: rename d9/d45/d56/dd5/f80 to d9/d45/ddf/fea 0 2024-05-13T22:48:14.255 INFO:tasks.workunit.client.0.smithi102.stdout:6/702: readlink d9/d45/d56/l76 0 2024-05-13T22:48:14.259 INFO:tasks.workunit.client.0.smithi102.stdout:4/676: dwrite d3/d19/d23/d7e/f9f [0,4194304] 0 2024-05-13T22:48:14.261 INFO:tasks.workunit.client.0.smithi102.stdout:5/769: creat de/d33/d53/f11a x:0 0 0 2024-05-13T22:48:14.264 INFO:tasks.workunit.client.0.smithi102.stdout:8/565: fdatasync d2/d7/f8f 0 2024-05-13T22:48:14.268 INFO:tasks.workunit.client.0.smithi102.stdout:4/677: dwrite d3/d22/fcd [0,4194304] 0 2024-05-13T22:48:14.270 INFO:tasks.workunit.client.0.smithi102.stdout:4/678: write d3/d19/d23/d39/d77/f7c [1080560,1591] 0 2024-05-13T22:48:14.270 INFO:tasks.workunit.client.0.smithi102.stdout:8/566: dread d2/d7/d27/f31 [0,4194304] 0 2024-05-13T22:48:14.286 INFO:tasks.workunit.client.0.smithi102.stdout:5/770: mkdir de/d25/d43/df5/d112/d11b 0 2024-05-13T22:48:14.288 INFO:tasks.workunit.client.0.smithi102.stdout:2/627: getdents d7/d9/d10/d25/d56/d59/d81 0 2024-05-13T22:48:14.292 INFO:tasks.workunit.client.0.smithi102.stdout:6/703: write d9/f1a [1206612,25097] 0 2024-05-13T22:48:14.293 INFO:tasks.workunit.client.0.smithi102.stdout:6/704: readlink d9/d25/d60/d6a/dad/lb0 0 2024-05-13T22:48:14.294 INFO:tasks.workunit.client.0.smithi102.stdout:1/471: truncate d2/d4/d5/d19/d28/d6a/f59 1676230 0 2024-05-13T22:48:14.294 INFO:tasks.workunit.client.0.smithi102.stdout:6/705: write d9/db/d2f/fcd [3154158,113902] 0 2024-05-13T22:48:14.295 INFO:tasks.workunit.client.0.smithi102.stdout:5/771: creat de/d25/d58/f11c x:0 0 0 2024-05-13T22:48:14.296 INFO:tasks.workunit.client.0.smithi102.stdout:1/472: truncate d2/d10/d62/f88 618114 0 2024-05-13T22:48:14.297 INFO:tasks.workunit.client.0.smithi102.stdout:1/473: fdatasync d2/f54 0 2024-05-13T22:48:14.297 INFO:tasks.workunit.client.0.smithi102.stdout:6/706: chown d9/db/d14/dcc/f43 60 1 2024-05-13T22:48:14.303 INFO:tasks.workunit.client.0.smithi102.stdout:5/772: dread de/d33/f83 [0,4194304] 0 2024-05-13T22:48:14.304 INFO:tasks.workunit.client.0.smithi102.stdout:6/707: dwrite d9/db/d14/d35/dc9/fd2 [0,4194304] 0 2024-05-13T22:48:14.306 INFO:tasks.workunit.client.0.smithi102.stdout:1/474: truncate d2/d45/f5a 1627016 0 2024-05-13T22:48:14.309 INFO:tasks.workunit.client.0.smithi102.stdout:2/628: creat d7/d9/fcf x:0 0 0 2024-05-13T22:48:14.311 INFO:tasks.workunit.client.0.smithi102.stdout:5/773: unlink de/d25/d58/f11c 0 2024-05-13T22:48:14.312 INFO:tasks.workunit.client.0.smithi102.stdout:6/708: dread - d9/d25/d71/f7a zero size 2024-05-13T22:48:14.313 INFO:tasks.workunit.client.0.smithi102.stdout:4/679: truncate d3/d22/dab/fbb 1786058 0 2024-05-13T22:48:14.325 INFO:tasks.workunit.client.0.smithi102.stdout:4/680: dread - d3/d19/d23/d39/d5d/da5/fb7 zero size 2024-05-13T22:48:14.334 INFO:tasks.workunit.client.0.smithi102.stdout:5/774: dread de/f21 [0,4194304] 0 2024-05-13T22:48:14.334 INFO:tasks.workunit.client.0.smithi102.stdout:6/709: dread d9/db/d14/d35/f52 [0,4194304] 0 2024-05-13T22:48:14.336 INFO:tasks.workunit.client.0.smithi102.stdout:2/629: mkdir d7/d9/d14/d31/dd0 0 2024-05-13T22:48:14.339 INFO:tasks.workunit.client.0.smithi102.stdout:1/475: write d2/d4/d26/f33 [5209928,77947] 0 2024-05-13T22:48:14.340 INFO:tasks.workunit.client.0.smithi102.stdout:1/476: fsync d2/d4/f37 0 2024-05-13T22:48:14.342 INFO:tasks.workunit.client.0.smithi102.stdout:5/775: write de/d25/d2a/d4b/d4d/d50/f109 [1682216,2722] 0 2024-05-13T22:48:14.347 INFO:tasks.workunit.client.0.smithi102.stdout:6/710: truncate d9/db/d14/dcc/f48 3324103 0 2024-05-13T22:48:14.356 INFO:tasks.workunit.client.0.smithi102.stdout:1/477: dread d2/d4/d26/f33 [0,4194304] 0 2024-05-13T22:48:14.357 INFO:tasks.workunit.client.0.smithi102.stdout:5/776: rename de/d25/d2c/dd5 to de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d11d 0 2024-05-13T22:48:14.359 INFO:tasks.workunit.client.0.smithi102.stdout:4/681: dwrite d3/d19/d23/d4e/d68/fe6 [0,4194304] 0 2024-05-13T22:48:14.361 INFO:tasks.workunit.client.0.smithi102.stdout:6/711: dread d9/fa [0,4194304] 0 2024-05-13T22:48:14.365 INFO:tasks.workunit.client.0.smithi102.stdout:2/630: mkdir d7/d9/d10/d30/d53/dd1 0 2024-05-13T22:48:14.367 INFO:tasks.workunit.client.0.smithi102.stdout:1/478: fsync d2/f46 0 2024-05-13T22:48:14.372 INFO:tasks.workunit.client.0.smithi102.stdout:5/777: truncate de/d25/fd3 4916928 0 2024-05-13T22:48:14.374 INFO:tasks.workunit.client.0.smithi102.stdout:4/682: rmdir d3/d19/dad/dc9/dd5 39 2024-05-13T22:48:14.375 INFO:tasks.workunit.client.0.smithi102.stdout:1/479: dwrite d2/d4/d5/d19/d28/f35 [0,4194304] 0 2024-05-13T22:48:14.377 INFO:tasks.workunit.client.0.smithi102.stdout:6/712: rmdir d9/db/d14/d54 39 2024-05-13T22:48:14.380 INFO:tasks.workunit.client.0.smithi102.stdout:4/683: dwrite d3/d19/d23/d3a/f4a [0,4194304] 0 2024-05-13T22:48:14.396 INFO:tasks.workunit.client.0.smithi102.stdout:2/631: truncate d7/f75 4082065 0 2024-05-13T22:48:14.397 INFO:tasks.workunit.client.0.smithi102.stdout:5/778: creat de/d25/d43/df5/d112/d11b/f11e x:0 0 0 2024-05-13T22:48:14.403 INFO:tasks.workunit.client.0.smithi102.stdout:6/713: dread d9/f21 [0,4194304] 0 2024-05-13T22:48:14.405 INFO:tasks.workunit.client.0.smithi102.stdout:5/779: creat de/d25/d2a/d4b/d4d/d50/da3/df4/f11f x:0 0 0 2024-05-13T22:48:14.408 INFO:tasks.workunit.client.0.smithi102.stdout:4/684: creat d3/ff0 x:0 0 0 2024-05-13T22:48:14.409 INFO:tasks.workunit.client.0.smithi102.stdout:4/685: dread - d3/d19/d23/d39/fe7 zero size 2024-05-13T22:48:14.411 INFO:tasks.workunit.client.0.smithi102.stdout:4/686: write d3/d19/d23/d39/d5d/da5/fb7 [157277,127240] 0 2024-05-13T22:48:14.411 INFO:tasks.workunit.client.0.smithi102.stdout:2/632: creat d7/d9/d10/d30/d53/dd1/fd2 x:0 0 0 2024-05-13T22:48:14.416 INFO:tasks.workunit.client.0.smithi102.stdout:1/480: link d2/d4/d5/d19/f4e d2/d4/d26/f9a 0 2024-05-13T22:48:14.418 INFO:tasks.workunit.client.0.smithi102.stdout:2/633: rmdir d7/d9/d14/d31 39 2024-05-13T22:48:14.420 INFO:tasks.workunit.client.0.smithi102.stdout:2/634: read d7/d9/d10/d1b/f6b [3438955,94909] 0 2024-05-13T22:48:14.420 INFO:tasks.workunit.client.0.smithi102.stdout:3/638: sync 2024-05-13T22:48:14.424 INFO:tasks.workunit.client.0.smithi102.stdout:2/635: dread d7/d9/d10/d30/f80 [0,4194304] 0 2024-05-13T22:48:14.425 INFO:tasks.workunit.client.0.smithi102.stdout:2/636: write d7/d9/fcf [120357,121932] 0 2024-05-13T22:48:14.427 INFO:tasks.workunit.client.0.smithi102.stdout:6/714: creat d9/feb x:0 0 0 2024-05-13T22:48:14.428 INFO:tasks.workunit.client.0.smithi102.stdout:1/481: dread d2/d4/d5/f15 [0,4194304] 0 2024-05-13T22:48:14.429 INFO:tasks.workunit.client.0.smithi102.stdout:1/482: read d2/d4/d38/f68 [1067970,47909] 0 2024-05-13T22:48:14.475 INFO:tasks.workunit.client.0.smithi102.stdout:3/639: creat de/d10/d50/d63/da2/fdd x:0 0 0 2024-05-13T22:48:14.475 INFO:tasks.workunit.client.0.smithi102.stdout:2/637: symlink d7/d15/ld3 0 2024-05-13T22:48:14.477 INFO:tasks.workunit.client.0.smithi102.stdout:2/638: read - d7/d9/d10/fc1 zero size 2024-05-13T22:48:14.478 INFO:tasks.workunit.client.0.smithi102.stdout:5/780: rename lb to de/d33/l120 0 2024-05-13T22:48:14.479 INFO:tasks.workunit.client.0.smithi102.stdout:4/687: getdents d3/d19/d23/d39/da7 0 2024-05-13T22:48:14.484 INFO:tasks.workunit.client.0.smithi102.stdout:3/640: creat de/d10/d2a/d36/db2/fde x:0 0 0 2024-05-13T22:48:14.485 INFO:tasks.workunit.client.0.smithi102.stdout:5/781: unlink de/d25/fe8 0 2024-05-13T22:48:14.486 INFO:tasks.workunit.client.0.smithi102.stdout:1/483: dwrite d2/d10/d62/d7b/f95 [0,4194304] 0 2024-05-13T22:48:14.488 INFO:tasks.workunit.client.0.smithi102.stdout:6/715: dwrite d9/d45/d56/d5d/f87 [0,4194304] 0 2024-05-13T22:48:14.521 INFO:tasks.workunit.client.0.smithi102.stdout:3/641: creat de/d10/d46/dc8/fdf x:0 0 0 2024-05-13T22:48:14.527 INFO:tasks.workunit.client.0.smithi102.stdout:4/688: link d3/d4/d6/de/d1f/d41/f8a d3/d19/d23/d39/da7/d7d/ff1 0 2024-05-13T22:48:14.529 INFO:tasks.workunit.client.0.smithi102.stdout:1/484: symlink d2/d4/d26/d40/l9b 0 2024-05-13T22:48:14.531 INFO:tasks.workunit.client.0.smithi102.stdout:4/689: dread d3/d19/d23/d7e/f9f [0,4194304] 0 2024-05-13T22:48:14.532 INFO:tasks.workunit.client.0.smithi102.stdout:4/690: readlink d3/d19/d23/d7e/la8 0 2024-05-13T22:48:14.532 INFO:tasks.workunit.client.0.smithi102.stdout:3/642: stat de/d10/d50/d8f/f60 0 2024-05-13T22:48:14.535 INFO:tasks.workunit.client.0.smithi102.stdout:2/639: rename d7/d9/d14/d31/c66 to d7/d15/d1e/cd4 0 2024-05-13T22:48:14.539 INFO:tasks.workunit.client.0.smithi102.stdout:1/485: creat d2/d4/d5/d19/d28/d6a/f9c x:0 0 0 2024-05-13T22:48:14.542 INFO:tasks.workunit.client.0.smithi102.stdout:4/691: truncate d3/d19/d23/d4e/d68/d85/f9a 385698 0 2024-05-13T22:48:14.549 INFO:tasks.workunit.client.0.smithi102.stdout:4/692: mknod d3/d22/dab/cf2 0 2024-05-13T22:48:14.550 INFO:tasks.workunit.client.0.smithi102.stdout:4/693: chown d3/d19/d23/d4e 22610473 1 2024-05-13T22:48:14.553 INFO:tasks.workunit.client.0.smithi102.stdout:2/640: mkdir d7/d9/d14/d31/dbb/dd5 0 2024-05-13T22:48:14.559 INFO:tasks.workunit.client.0.smithi102.stdout:6/716: rename d9/d45/d56/d5d/f82 to d9/fec 0 2024-05-13T22:48:14.560 INFO:tasks.workunit.client.0.smithi102.stdout:6/717: readlink d9/db/d14/d35/l91 0 2024-05-13T22:48:14.562 INFO:tasks.workunit.client.0.smithi102.stdout:3/643: getdents de/d10/d2a/d33/daa 0 2024-05-13T22:48:14.566 INFO:tasks.workunit.client.0.smithi102.stdout:1/486: rename d2/d4/d38/f6e to d2/d45/d79/d92/f9d 0 2024-05-13T22:48:14.570 INFO:tasks.workunit.client.0.smithi102.stdout:2/641: truncate d7/d15/d1e/f26 825987 0 2024-05-13T22:48:14.571 INFO:tasks.workunit.client.0.smithi102.stdout:2/642: write d7/d9/d10/d30/fa9 [566631,1990] 0 2024-05-13T22:48:14.571 INFO:tasks.workunit.client.0.smithi102.stdout:2/643: chown d7/l4f 446331125 1 2024-05-13T22:48:14.572 INFO:tasks.workunit.client.0.smithi102.stdout:3/644: symlink de/d1c/d79/le0 0 2024-05-13T22:48:14.574 INFO:tasks.workunit.client.0.smithi102.stdout:4/694: creat d3/ff3 x:0 0 0 2024-05-13T22:48:14.576 INFO:tasks.workunit.client.0.smithi102.stdout:4/695: dread d3/d19/d23/d4e/f72 [0,4194304] 0 2024-05-13T22:48:14.578 INFO:tasks.workunit.client.0.smithi102.stdout:6/718: rename d9/db/d14/d35/d47/dc1/fe1 to d9/d25/d60/d6a/dd6/de9/fed 0 2024-05-13T22:48:14.582 INFO:tasks.workunit.client.0.smithi102.stdout:0/642: sync 2024-05-13T22:48:14.586 INFO:tasks.workunit.client.0.smithi102.stdout:2/644: creat d7/d9/d10/d25/d56/d59/fd6 x:0 0 0 2024-05-13T22:48:14.586 INFO:tasks.workunit.client.0.smithi102.stdout:2/645: chown d7/d9/d10/fb3 31 1 2024-05-13T22:48:14.594 INFO:tasks.workunit.client.0.smithi102.stdout:0/643: dread - d1/de/d32/d24/d2d/d38/d3d/f95 zero size 2024-05-13T22:48:14.601 INFO:tasks.workunit.client.0.smithi102.stdout:6/719: mkdir d9/d25/dee 0 2024-05-13T22:48:14.605 INFO:tasks.workunit.client.0.smithi102.stdout:0/644: creat d1/de/d32/d24/d2d/d38/d3d/d81/da5/fc5 x:0 0 0 2024-05-13T22:48:14.608 INFO:tasks.workunit.client.0.smithi102.stdout:2/646: rename d7/d9/d10/d25/d56/c7b to d7/cd7 0 2024-05-13T22:48:14.609 INFO:tasks.workunit.client.0.smithi102.stdout:5/782: write de/d25/d59/f5d [370481,5432] 0 2024-05-13T22:48:14.612 INFO:tasks.workunit.client.0.smithi102.stdout:0/645: dread d1/de/d32/d24/d2d/d3e/f73 [0,4194304] 0 2024-05-13T22:48:14.617 INFO:tasks.workunit.client.0.smithi102.stdout:4/696: rename d3/d19/d23/d39/d77/f7c to d3/d19/d23/d39/dac/ff4 0 2024-05-13T22:48:14.621 INFO:tasks.workunit.client.0.smithi102.stdout:0/646: symlink d1/de/d32/d63/lc6 0 2024-05-13T22:48:14.624 INFO:tasks.workunit.client.0.smithi102.stdout:4/697: dwrite d3/d4/d6/de/d8d/dec/fef [0,4194304] 0 2024-05-13T22:48:14.628 INFO:tasks.workunit.client.0.smithi102.stdout:5/783: creat de/d25/d58/dfb/f121 x:0 0 0 2024-05-13T22:48:14.631 INFO:tasks.workunit.client.0.smithi102.stdout:1/487: rmdir d2/d4/d5/d19/d28/d6a 39 2024-05-13T22:48:14.633 INFO:tasks.workunit.client.0.smithi102.stdout:1/488: dread d2/d10/d62/d7b/f7c [0,4194304] 0 2024-05-13T22:48:14.634 INFO:tasks.workunit.client.0.smithi102.stdout:0/647: read d1/de/d32/f37 [8822,119213] 0 2024-05-13T22:48:14.636 INFO:tasks.workunit.client.0.smithi102.stdout:2/647: unlink d7/d9/d7c/c8f 0 2024-05-13T22:48:14.640 INFO:tasks.workunit.client.0.smithi102.stdout:5/784: rename de/d25/d2a/d4b/cc7 to de/d25/d2a/d4b/d4d/d114/dda/c122 0 2024-05-13T22:48:14.641 INFO:tasks.workunit.client.0.smithi102.stdout:9/626: sync 2024-05-13T22:48:14.641 INFO:tasks.workunit.client.0.smithi102.stdout:8/567: sync 2024-05-13T22:48:14.641 INFO:tasks.workunit.client.0.smithi102.stdout:7/493: sync 2024-05-13T22:48:14.642 INFO:tasks.workunit.client.0.smithi102.stdout:8/568: write d2/d7/f69 [3449943,115489] 0 2024-05-13T22:48:14.643 INFO:tasks.workunit.client.0.smithi102.stdout:8/569: fdatasync d2/d7/d27/f77 0 2024-05-13T22:48:14.648 INFO:tasks.workunit.client.0.smithi102.stdout:7/494: dwrite d6/d13/d58/d20/d80/f8a [0,4194304] 0 2024-05-13T22:48:14.656 INFO:tasks.workunit.client.0.smithi102.stdout:9/627: truncate d1/f2b 3365716 0 2024-05-13T22:48:14.658 INFO:tasks.workunit.client.0.smithi102.stdout:9/628: rename d1/d6/db to d1/d6/db/d4a/d75/d96/dc6/ddb 22 2024-05-13T22:48:14.663 INFO:tasks.workunit.client.0.smithi102.stdout:1/489: mkdir d2/d4/d9e 0 2024-05-13T22:48:14.665 INFO:tasks.workunit.client.0.smithi102.stdout:0/648: symlink d1/lc7 0 2024-05-13T22:48:14.672 INFO:tasks.workunit.client.0.smithi102.stdout:8/570: mknod d2/d7/da2/cac 0 2024-05-13T22:48:14.672 INFO:tasks.workunit.client.0.smithi102.stdout:7/495: dread d6/d13/d58/d20/f50 [0,4194304] 0 2024-05-13T22:48:14.672 INFO:tasks.workunit.client.0.smithi102.stdout:3/645: dwrite f7 [0,4194304] 0 2024-05-13T22:48:14.673 INFO:tasks.workunit.client.0.smithi102.stdout:8/571: chown d2/d7/d12/d78 654 1 2024-05-13T22:48:14.674 INFO:tasks.workunit.client.0.smithi102.stdout:8/572: readlink d2/d7/d25/d41/d50/l59 0 2024-05-13T22:48:14.676 INFO:tasks.workunit.client.0.smithi102.stdout:9/629: rename d1/d6/db/d30/f95 to d1/d6/d21/fdc 0 2024-05-13T22:48:14.676 INFO:tasks.workunit.client.0.smithi102.stdout:8/573: fsync d2/d36/d49/f60 0 2024-05-13T22:48:14.683 INFO:tasks.workunit.client.0.smithi102.stdout:8/574: dwrite d2/d7/d25/d41/d50/f39 [0,4194304] 0 2024-05-13T22:48:14.690 INFO:tasks.workunit.client.0.smithi102.stdout:3/646: truncate de/d68/f9d 223706 0 2024-05-13T22:48:14.693 INFO:tasks.workunit.client.0.smithi102.stdout:7/496: dread d6/d13/d58/d20/d4b/f6c [0,4194304] 0 2024-05-13T22:48:14.695 INFO:tasks.workunit.client.0.smithi102.stdout:6/720: truncate d9/db/d14/dcc/d5f/fa4 1720734 0 2024-05-13T22:48:14.695 INFO:tasks.workunit.client.0.smithi102.stdout:6/721: readlink d9/d25/d60/d6a/lbc 0 2024-05-13T22:48:14.700 INFO:tasks.workunit.client.0.smithi102.stdout:9/630: dread d1/d6/db/f1b [4194304,4194304] 0 2024-05-13T22:48:14.702 INFO:tasks.workunit.client.0.smithi102.stdout:2/648: getdents d7/d15 0 2024-05-13T22:48:14.704 INFO:tasks.workunit.client.0.smithi102.stdout:8/575: write d2/d36/d49/d5d/f51 [1727639,112686] 0 2024-05-13T22:48:14.705 INFO:tasks.workunit.client.0.smithi102.stdout:2/649: dread d7/d9/d10/d1b/d54/fcb [0,4194304] 0 2024-05-13T22:48:14.706 INFO:tasks.workunit.client.0.smithi102.stdout:2/650: write d7/d9/d10/f33 [3807896,85493] 0 2024-05-13T22:48:14.710 INFO:tasks.workunit.client.0.smithi102.stdout:7/497: creat d6/d15/d4d/d55/f8e x:0 0 0 2024-05-13T22:48:14.714 INFO:tasks.workunit.client.0.smithi102.stdout:6/722: write d9/f8b [1083912,87616] 0 2024-05-13T22:48:14.717 INFO:tasks.workunit.client.0.smithi102.stdout:9/631: truncate d1/d5d/d6c/f86 763557 0 2024-05-13T22:48:14.717 INFO:tasks.workunit.client.0.smithi102.stdout:4/698: dwrite d3/d19/d23/d39/d5d/f98 [0,4194304] 0 2024-05-13T22:48:14.729 INFO:tasks.workunit.client.0.smithi102.stdout:5/785: dwrite f0 [0,4194304] 0 2024-05-13T22:48:14.731 INFO:tasks.workunit.client.0.smithi102.stdout:1/490: rename d2/d4/d26/d40/l50 to d2/d4/d5/d19/l9f 0 2024-05-13T22:48:14.733 INFO:tasks.workunit.client.0.smithi102.stdout:6/723: mknod d9/db/d14/d35/d47/cef 0 2024-05-13T22:48:14.737 INFO:tasks.workunit.client.0.smithi102.stdout:4/699: mkdir d3/d19/d23/d39/d77/df5 0 2024-05-13T22:48:14.740 INFO:tasks.workunit.client.0.smithi102.stdout:2/651: creat d7/d9/d14/d31/dbb/dd5/fd8 x:0 0 0 2024-05-13T22:48:14.749 INFO:tasks.workunit.client.0.smithi102.stdout:1/491: creat d2/d10/fa0 x:0 0 0 2024-05-13T22:48:14.749 INFO:tasks.workunit.client.0.smithi102.stdout:0/649: dwrite d1/d4d/d79/f80 [0,4194304] 0 2024-05-13T22:48:14.751 INFO:tasks.workunit.client.0.smithi102.stdout:6/724: fdatasync d9/db/d14/f19 0 2024-05-13T22:48:14.752 INFO:tasks.workunit.client.0.smithi102.stdout:9/632: truncate d1/d6/d45/f67 874209 0 2024-05-13T22:48:14.754 INFO:tasks.workunit.client.0.smithi102.stdout:1/492: dread d2/d4/d5/d19/d28/f35 [0,4194304] 0 2024-05-13T22:48:14.754 INFO:tasks.workunit.client.0.smithi102.stdout:4/700: creat d3/d19/d23/d3a/ff6 x:0 0 0 2024-05-13T22:48:14.766 INFO:tasks.workunit.client.0.smithi102.stdout:6/725: unlink d9/db/d14/f19 0 2024-05-13T22:48:14.770 INFO:tasks.workunit.client.0.smithi102.stdout:1/493: creat d2/d4/d5/fa1 x:0 0 0 2024-05-13T22:48:14.773 INFO:tasks.workunit.client.0.smithi102.stdout:2/652: link d7/d9/d10/f37 d7/d9/d10/d30/d53/dd1/fd9 0 2024-05-13T22:48:14.774 INFO:tasks.workunit.client.0.smithi102.stdout:4/701: mknod d3/d19/d23/d39/dac/cf7 0 2024-05-13T22:48:14.781 INFO:tasks.workunit.client.0.smithi102.stdout:2/653: rename d7/d15/c7d to d7/d15/cda 0 2024-05-13T22:48:14.783 INFO:tasks.workunit.client.0.smithi102.stdout:4/702: chown d3/d4/d5f/fa3 197644 1 2024-05-13T22:48:14.787 INFO:tasks.workunit.client.0.smithi102.stdout:9/633: creat d1/fdd x:0 0 0 2024-05-13T22:48:14.789 INFO:tasks.workunit.client.0.smithi102.stdout:1/494: mkdir d2/d4/d26/da2 0 2024-05-13T22:48:14.791 INFO:tasks.workunit.client.0.smithi102.stdout:3/647: write de/f37 [5674991,108092] 0 2024-05-13T22:48:14.791 INFO:tasks.workunit.client.0.smithi102.stdout:6/726: dread f7 [4194304,4194304] 0 2024-05-13T22:48:14.792 INFO:tasks.workunit.client.0.smithi102.stdout:8/576: dwrite d2/d36/d49/d5d/f67 [0,4194304] 0 2024-05-13T22:48:14.793 INFO:tasks.workunit.client.0.smithi102.stdout:2/654: symlink d7/d15/d67/ldb 0 2024-05-13T22:48:14.796 INFO:tasks.workunit.client.0.smithi102.stdout:4/703: truncate d3/d19/d23/d3a/f9d 353345 0 2024-05-13T22:48:14.797 INFO:tasks.workunit.client.0.smithi102.stdout:6/727: write d9/feb [564444,52203] 0 2024-05-13T22:48:14.799 INFO:tasks.workunit.client.0.smithi102.stdout:6/728: readlink d9/d45/d56/lca 0 2024-05-13T22:48:14.800 INFO:tasks.workunit.client.0.smithi102.stdout:1/495: symlink d2/d10/d62/d86/la3 0 2024-05-13T22:48:14.801 INFO:tasks.workunit.client.0.smithi102.stdout:1/496: write d2/d10/f85 [415806,83821] 0 2024-05-13T22:48:14.802 INFO:tasks.workunit.client.0.smithi102.stdout:9/634: mkdir d1/d6/db/d4a/d75/d96/dc6/dde 0 2024-05-13T22:48:14.805 INFO:tasks.workunit.client.0.smithi102.stdout:3/648: fdatasync de/d10/d2a/f7e 0 2024-05-13T22:48:14.806 INFO:tasks.workunit.client.0.smithi102.stdout:8/577: mkdir d2/d7/d27/dad 0 2024-05-13T22:48:14.811 INFO:tasks.workunit.client.0.smithi102.stdout:2/655: mknod d7/d9/d10/d25/d56/d59/d81/cdc 0 2024-05-13T22:48:14.813 INFO:tasks.workunit.client.0.smithi102.stdout:2/656: write d7/d9/d10/d25/d56/f8c [901546,5566] 0 2024-05-13T22:48:14.813 INFO:tasks.workunit.client.0.smithi102.stdout:7/498: truncate d6/d13/f16 2637911 0 2024-05-13T22:48:14.820 INFO:tasks.workunit.client.0.smithi102.stdout:9/635: rename d1/d6/d45/d3e/cc3 to d1/d6/d45/dd0/cdf 0 2024-05-13T22:48:14.824 INFO:tasks.workunit.client.0.smithi102.stdout:3/649: chown de/d10/d34/ld4 45 1 2024-05-13T22:48:14.825 INFO:tasks.workunit.client.0.smithi102.stdout:5/786: dwrite de/d25/d2c/ff7 [0,4194304] 0 2024-05-13T22:48:14.825 INFO:tasks.workunit.client.0.smithi102.stdout:7/499: dwrite d6/d13/d2c/f5d [0,4194304] 0 2024-05-13T22:48:14.827 INFO:tasks.workunit.client.0.smithi102.stdout:8/578: stat d2/f32 0 2024-05-13T22:48:14.831 INFO:tasks.workunit.client.0.smithi102.stdout:5/787: write de/d25/d2a/d4b/d4d/d50/da3/df4/f11f [558459,29992] 0 2024-05-13T22:48:14.835 INFO:tasks.workunit.client.0.smithi102.stdout:3/650: dwrite de/d10/d2a/d8c/d7f/fd6 [0,4194304] 0 2024-05-13T22:48:14.837 INFO:tasks.workunit.client.0.smithi102.stdout:0/650: write d1/de/d32/d24/d2d/d38/d3d/f9a [565932,74720] 0 2024-05-13T22:48:14.840 INFO:tasks.workunit.client.0.smithi102.stdout:1/497: mkdir d2/d45/da4 0 2024-05-13T22:48:14.846 INFO:tasks.workunit.client.0.smithi102.stdout:0/651: chown d1/de/d42/d61/fb5 1 1 2024-05-13T22:48:14.847 INFO:tasks.workunit.client.0.smithi102.stdout:0/652: fdatasync d1/de/d32/d24/d2d/d3e/f41 0 2024-05-13T22:48:14.849 INFO:tasks.workunit.client.0.smithi102.stdout:9/636: creat d1/d6/d21/d71/d91/fe0 x:0 0 0 2024-05-13T22:48:14.860 INFO:tasks.workunit.client.0.smithi102.stdout:3/651: unlink de/d1c/d90/cd1 0 2024-05-13T22:48:14.861 INFO:tasks.workunit.client.0.smithi102.stdout:3/652: fdatasync de/d10/d50/d63/da2/fdd 0 2024-05-13T22:48:14.863 INFO:tasks.workunit.client.0.smithi102.stdout:3/653: chown de/d10/d50/lbc 2574 1 2024-05-13T22:48:14.865 INFO:tasks.workunit.client.0.smithi102.stdout:0/653: creat d1/de/d32/d24/d2d/d3e/fc8 x:0 0 0 2024-05-13T22:48:14.868 INFO:tasks.workunit.client.0.smithi102.stdout:1/498: dread d2/d4/f20 [0,4194304] 0 2024-05-13T22:48:14.869 INFO:tasks.workunit.client.0.smithi102.stdout:1/499: read - d2/d45/d79/f81 zero size 2024-05-13T22:48:14.874 INFO:tasks.workunit.client.0.smithi102.stdout:1/500: dread d2/d10/d62/d7b/f95 [0,4194304] 0 2024-05-13T22:48:14.876 INFO:tasks.workunit.client.0.smithi102.stdout:2/657: mkdir d7/d9/d11/ddd 0 2024-05-13T22:48:14.878 INFO:tasks.workunit.client.0.smithi102.stdout:6/729: dwrite d9/db/d14/dcc/d5f/fa4 [0,4194304] 0 2024-05-13T22:48:14.878 INFO:tasks.workunit.client.0.smithi102.stdout:5/788: creat de/d25/d59/d7f/d86/da5/ddb/dee/d116/f123 x:0 0 0 2024-05-13T22:48:14.883 INFO:tasks.workunit.client.0.smithi102.stdout:0/654: rmdir d1/d4d/d79/d9f 39 2024-05-13T22:48:14.886 INFO:tasks.workunit.client.0.smithi102.stdout:4/704: truncate d3/d4/f31 1893530 0 2024-05-13T22:48:14.887 INFO:tasks.workunit.client.0.smithi102.stdout:4/705: chown d3/d19/d23/d39/d5d/da5/fb7 2 1 2024-05-13T22:48:14.895 INFO:tasks.workunit.client.0.smithi102.stdout:5/789: readlink de/d25/d2a/d4b/d4d/lfa 0 2024-05-13T22:48:14.899 INFO:tasks.workunit.client.0.smithi102.stdout:8/579: dwrite d2/d7/d12/f23 [0,4194304] 0 2024-05-13T22:48:14.901 INFO:tasks.workunit.client.0.smithi102.stdout:7/500: write d6/f7 [692556,112140] 0 2024-05-13T22:48:14.902 INFO:tasks.workunit.client.0.smithi102.stdout:8/580: read d2/d7/d25/d41/d50/d40/f5b [1413744,96703] 0 2024-05-13T22:48:14.905 INFO:tasks.workunit.client.0.smithi102.stdout:0/655: rename d1/de/d32/d24/d2d/d38/l75 to d1/de/d32/d24/d2d/d38/d3d/lc9 0 2024-05-13T22:48:14.906 INFO:tasks.workunit.client.0.smithi102.stdout:8/581: truncate d2/d7/d12/d76/f9f 529578 0 2024-05-13T22:48:14.912 INFO:tasks.workunit.client.0.smithi102.stdout:2/658: truncate d7/d15/d67/fa8 2404438 0 2024-05-13T22:48:14.914 INFO:tasks.workunit.client.0.smithi102.stdout:1/501: read d2/d4/d5/fa [1846899,127782] 0 2024-05-13T22:48:14.915 INFO:tasks.workunit.client.0.smithi102.stdout:5/790: creat de/d25/d2a/def/f124 x:0 0 0 2024-05-13T22:48:14.918 INFO:tasks.workunit.client.0.smithi102.stdout:5/791: read de/d25/d2c/ff7 [1043815,94728] 0 2024-05-13T22:48:14.920 INFO:tasks.workunit.client.0.smithi102.stdout:9/637: truncate d1/d6/d45/d3e/f56 242763 0 2024-05-13T22:48:14.922 INFO:tasks.workunit.client.0.smithi102.stdout:3/654: dwrite de/d10/d2a/d33/f40 [0,4194304] 0 2024-05-13T22:48:14.923 INFO:tasks.workunit.client.0.smithi102.stdout:5/792: write de/d25/d58/dfb/f121 [172931,117155] 0 2024-05-13T22:48:14.924 INFO:tasks.workunit.client.0.smithi102.stdout:9/638: read d1/d6/d45/d2d/fd4 [457009,104664] 0 2024-05-13T22:48:14.925 INFO:tasks.workunit.client.0.smithi102.stdout:8/582: rename d2/d7/d12 to d2/d7/d27/d64/d96/dae 0 2024-05-13T22:48:14.925 INFO:tasks.workunit.client.0.smithi102.stdout:1/502: rename d2/d10/d62/d86 to d2/d10/d62/d86/d91/da5 22 2024-05-13T22:48:14.930 INFO:tasks.workunit.client.0.smithi102.stdout:0/656: dread d1/de/d32/f36 [0,4194304] 0 2024-05-13T22:48:14.932 INFO:tasks.workunit.client.0.smithi102.stdout:4/706: write d3/d4/f5a [288669,73723] 0 2024-05-13T22:48:14.937 INFO:tasks.workunit.client.0.smithi102.stdout:2/659: truncate d7/f32 1714780 0 2024-05-13T22:48:14.938 INFO:tasks.workunit.client.0.smithi102.stdout:6/730: link d9/db/d14/dcc/cbf d9/db/d14/d35/d47/cf0 0 2024-05-13T22:48:14.941 INFO:tasks.workunit.client.0.smithi102.stdout:2/660: dread d7/d9/d10/d30/f80 [0,4194304] 0 2024-05-13T22:48:14.942 INFO:tasks.workunit.client.0.smithi102.stdout:7/501: write d6/d15/f37 [433303,7429] 0 2024-05-13T22:48:14.943 INFO:tasks.workunit.client.0.smithi102.stdout:1/503: rmdir d2/d4/d38 39 2024-05-13T22:48:14.947 INFO:tasks.workunit.client.0.smithi102.stdout:3/655: unlink de/d10/d2a/d36/l71 0 2024-05-13T22:48:14.948 INFO:tasks.workunit.client.0.smithi102.stdout:2/661: dwrite d7/d9/d10/d25/d27/f45 [0,4194304] 0 2024-05-13T22:48:14.950 INFO:tasks.workunit.client.0.smithi102.stdout:8/583: mknod d2/d7/d27/d64/caf 0 2024-05-13T22:48:14.953 INFO:tasks.workunit.client.0.smithi102.stdout:9/639: creat d1/d6/d45/da1/fe1 x:0 0 0 2024-05-13T22:48:14.954 INFO:tasks.workunit.client.0.smithi102.stdout:9/640: dread - d1/d6/db/d4a/fd7 zero size 2024-05-13T22:48:14.955 INFO:tasks.workunit.client.0.smithi102.stdout:0/657: unlink d1/de/d32/c46 0 2024-05-13T22:48:14.957 INFO:tasks.workunit.client.0.smithi102.stdout:4/707: creat d3/d4/d6/ff8 x:0 0 0 2024-05-13T22:48:14.963 INFO:tasks.workunit.client.0.smithi102.stdout:7/502: stat d6/d13/f16 0 2024-05-13T22:48:14.965 INFO:tasks.workunit.client.0.smithi102.stdout:1/504: unlink d2/d4/c8f 0 2024-05-13T22:48:14.967 INFO:tasks.workunit.client.0.smithi102.stdout:3/656: mkdir de/d67/de1 0 2024-05-13T22:48:14.968 INFO:tasks.workunit.client.0.smithi102.stdout:1/505: dread d2/d4/f20 [0,4194304] 0 2024-05-13T22:48:14.970 INFO:tasks.workunit.client.0.smithi102.stdout:2/662: rename d7/d9/d10/d25/l41 to d7/d9/d14/d31/dbb/dd5/lde 0 2024-05-13T22:48:14.971 INFO:tasks.workunit.client.0.smithi102.stdout:1/506: dread d2/d10/f85 [0,4194304] 0 2024-05-13T22:48:14.974 INFO:tasks.workunit.client.0.smithi102.stdout:9/641: creat d1/d6/db/d30/d47/fe2 x:0 0 0 2024-05-13T22:48:14.978 INFO:tasks.workunit.client.0.smithi102.stdout:4/708: mkdir d3/d22/dab/df9 0 2024-05-13T22:48:14.979 INFO:tasks.workunit.client.0.smithi102.stdout:6/731: getdents d9/d25/d5a 0 2024-05-13T22:48:14.980 INFO:tasks.workunit.client.0.smithi102.stdout:3/657: creat de/d10/d2a/d8c/da8/fe2 x:0 0 0 2024-05-13T22:48:14.981 INFO:tasks.workunit.client.0.smithi102.stdout:3/658: chown de/d10/d2a/d33/daa/lb6 49 1 2024-05-13T22:48:14.983 INFO:tasks.workunit.client.0.smithi102.stdout:7/503: rename d6/d13/l61 to d6/d13/d62/l8f 0 2024-05-13T22:48:14.985 INFO:tasks.workunit.client.0.smithi102.stdout:5/793: link de/d25/d59/d7f/d86/da5/lb0 de/d25/d2a/d4b/d4d/d114/l125 0 2024-05-13T22:48:14.986 INFO:tasks.workunit.client.0.smithi102.stdout:0/658: creat d1/de/d42/da7/fca x:0 0 0 2024-05-13T22:48:14.988 INFO:tasks.workunit.client.0.smithi102.stdout:9/642: creat d1/d6/db/d4a/d75/fe3 x:0 0 0 2024-05-13T22:48:14.992 INFO:tasks.workunit.client.0.smithi102.stdout:5/794: dwrite de/d25/d58/f101 [0,4194304] 0 2024-05-13T22:48:14.995 INFO:tasks.workunit.client.0.smithi102.stdout:6/732: rename d9/db/d2f to d9/db/d14/d35/dc9/df1 0 2024-05-13T22:48:14.999 INFO:tasks.workunit.client.0.smithi102.stdout:8/584: creat d2/d7/d25/fb0 x:0 0 0 2024-05-13T22:48:15.002 INFO:tasks.workunit.client.0.smithi102.stdout:4/709: mknod d3/d22/cfa 0 2024-05-13T22:48:15.007 INFO:tasks.workunit.client.0.smithi102.stdout:9/643: truncate d1/d6/d45/d3e/f93 4828443 0 2024-05-13T22:48:15.011 INFO:tasks.workunit.client.0.smithi102.stdout:3/659: rename de/d10/d50/d63/fd5 to de/d68/fe3 0 2024-05-13T22:48:15.012 INFO:tasks.workunit.client.0.smithi102.stdout:3/660: chown de/d10/d34/ld4 26418055 1 2024-05-13T22:48:15.015 INFO:tasks.workunit.client.0.smithi102.stdout:8/585: mknod d2/d7/d25/d41/d50/d93/cb1 0 2024-05-13T22:48:15.017 INFO:tasks.workunit.client.0.smithi102.stdout:4/710: creat d3/d19/d23/d39/dac/ffb x:0 0 0 2024-05-13T22:48:15.020 INFO:tasks.workunit.client.0.smithi102.stdout:9/644: rmdir d1/d6/db/d4a/d81 39 2024-05-13T22:48:15.022 INFO:tasks.workunit.client.0.smithi102.stdout:9/645: truncate d1/d6/f7c 121416 0 2024-05-13T22:48:15.024 INFO:tasks.workunit.client.0.smithi102.stdout:4/711: dwrite d3/d4/f1e [4194304,4194304] 0 2024-05-13T22:48:15.025 INFO:tasks.workunit.client.0.smithi102.stdout:7/504: rename d6/d13/d58/d20 to d6/d15/d4c/d90 0 2024-05-13T22:48:15.029 INFO:tasks.workunit.client.0.smithi102.stdout:3/661: mknod de/d10/d2a/d8c/d7f/ce4 0 2024-05-13T22:48:15.031 INFO:tasks.workunit.client.0.smithi102.stdout:2/663: write d7/d15/fa5 [2518865,65391] 0 2024-05-13T22:48:15.032 INFO:tasks.workunit.client.0.smithi102.stdout:2/664: chown d7/d15 11 1 2024-05-13T22:48:15.033 INFO:tasks.workunit.client.0.smithi102.stdout:2/665: dread - d7/d9/d10/d30/d53/da4/fca zero size 2024-05-13T22:48:15.034 INFO:tasks.workunit.client.0.smithi102.stdout:8/586: mkdir d2/d7/d25/d41/d50/d40/db2 0 2024-05-13T22:48:15.037 INFO:tasks.workunit.client.0.smithi102.stdout:7/505: readlink d6/l1c 0 2024-05-13T22:48:15.039 INFO:tasks.workunit.client.0.smithi102.stdout:7/506: read d6/d15/f7a [340900,63009] 0 2024-05-13T22:48:15.041 INFO:tasks.workunit.client.0.smithi102.stdout:1/507: getdents d2/d4/d5/d19/d28 0 2024-05-13T22:48:15.043 INFO:tasks.workunit.client.0.smithi102.stdout:1/508: stat d2/d4/d5/d19/d28/d6a 0 2024-05-13T22:48:15.044 INFO:tasks.workunit.client.0.smithi102.stdout:0/659: dwrite d1/de/f50 [0,4194304] 0 2024-05-13T22:48:15.049 INFO:tasks.workunit.client.0.smithi102.stdout:8/587: mkdir d2/d36/db3 0 2024-05-13T22:48:15.052 INFO:tasks.workunit.client.0.smithi102.stdout:2/666: rename d7/d9/d10/d25/d27/dab/caf to d7/d9/d10/d1b/cdf 0 2024-05-13T22:48:15.061 INFO:tasks.workunit.client.0.smithi102.stdout:6/733: write d9/db/d14/d35/dc9/df1/f62 [2559593,36239] 0 2024-05-13T22:48:15.064 INFO:tasks.workunit.client.0.smithi102.stdout:6/734: dread d9/d25/faa [0,4194304] 0 2024-05-13T22:48:15.064 INFO:tasks.workunit.client.0.smithi102.stdout:0/660: rmdir d1/de 39 2024-05-13T22:48:15.066 INFO:tasks.workunit.client.0.smithi102.stdout:8/588: mknod d2/d7/d25/d41/d50/d40/cb4 0 2024-05-13T22:48:15.067 INFO:tasks.workunit.client.0.smithi102.stdout:8/589: dread - d2/d7/d25/d41/d6f/f82 zero size 2024-05-13T22:48:15.070 INFO:tasks.workunit.client.0.smithi102.stdout:2/667: mknod d7/d9/d14/d31/dbb/ce0 0 2024-05-13T22:48:15.071 INFO:tasks.workunit.client.0.smithi102.stdout:3/662: dwrite de/d1c/f52 [0,4194304] 0 2024-05-13T22:48:15.074 INFO:tasks.workunit.client.0.smithi102.stdout:7/507: mknod d6/d13/d58/c91 0 2024-05-13T22:48:15.076 INFO:tasks.workunit.client.0.smithi102.stdout:4/712: write d3/d19/d23/d4e/d68/d85/fbf [1636389,26547] 0 2024-05-13T22:48:15.078 INFO:tasks.workunit.client.0.smithi102.stdout:1/509: getdents d2/d10/d62/d86/d91 0 2024-05-13T22:48:15.078 INFO:tasks.workunit.client.0.smithi102.stdout:1/510: fsync d2/f4b 0 2024-05-13T22:48:15.079 INFO:tasks.workunit.client.0.smithi102.stdout:1/511: stat d2/d45/d79 0 2024-05-13T22:48:15.079 INFO:tasks.workunit.client.0.smithi102.stdout:6/735: write d9/db/f36 [7620470,118610] 0 2024-05-13T22:48:15.084 INFO:tasks.workunit.client.0.smithi102.stdout:3/663: symlink de/d10/d2a/d36/da0/le5 0 2024-05-13T22:48:15.087 INFO:tasks.workunit.client.0.smithi102.stdout:4/713: fsync d3/d4/fb4 0 2024-05-13T22:48:15.090 INFO:tasks.workunit.client.0.smithi102.stdout:7/508: dwrite d6/d13/d2c/d39/f3f [0,4194304] 0 2024-05-13T22:48:15.091 INFO:tasks.workunit.client.0.smithi102.stdout:7/509: dread - d6/d15/f7e zero size 2024-05-13T22:48:15.092 INFO:tasks.workunit.client.0.smithi102.stdout:7/510: write d6/d22/f33 [1841255,11825] 0 2024-05-13T22:48:15.092 INFO:tasks.workunit.client.0.smithi102.stdout:7/511: stat d6/d22/f43 0 2024-05-13T22:48:15.096 INFO:tasks.workunit.client.0.smithi102.stdout:1/512: unlink d2/d10/c1d 0 2024-05-13T22:48:15.098 INFO:tasks.workunit.client.0.smithi102.stdout:6/736: fsync d9/db/f7d 0 2024-05-13T22:48:15.099 INFO:tasks.workunit.client.0.smithi102.stdout:0/661: fsync d1/de/d32/d24/f40 0 2024-05-13T22:48:15.100 INFO:tasks.workunit.client.0.smithi102.stdout:8/590: symlink d2/d7/d25/d41/d6f/da0/lb5 0 2024-05-13T22:48:15.103 INFO:tasks.workunit.client.0.smithi102.stdout:3/664: mknod de/d10/d2a/d36/ce6 0 2024-05-13T22:48:15.106 INFO:tasks.workunit.client.0.smithi102.stdout:2/668: dwrite d7/d9/d10/d30/d53/f5e [0,4194304] 0 2024-05-13T22:48:15.107 INFO:tasks.workunit.client.0.smithi102.stdout:2/669: stat d7/lb4 0 2024-05-13T22:48:15.112 INFO:tasks.workunit.client.0.smithi102.stdout:0/662: symlink d1/de/d42/da7/lcb 0 2024-05-13T22:48:15.114 INFO:tasks.workunit.client.0.smithi102.stdout:3/665: dread - de/d1c/d79/f81 zero size 2024-05-13T22:48:15.115 INFO:tasks.workunit.client.0.smithi102.stdout:4/714: symlink d3/d19/d23/d39/lfc 0 2024-05-13T22:48:15.118 INFO:tasks.workunit.client.0.smithi102.stdout:1/513: getdents d2/d10/d62/d86/d91 0 2024-05-13T22:48:15.119 INFO:tasks.workunit.client.0.smithi102.stdout:8/591: dwrite d2/d36/d49/d5d/f45 [0,4194304] 0 2024-05-13T22:48:15.120 INFO:tasks.workunit.client.0.smithi102.stdout:1/514: read - d2/d45/d79/f81 zero size 2024-05-13T22:48:15.120 INFO:tasks.workunit.client.0.smithi102.stdout:1/515: stat l1 0 2024-05-13T22:48:15.122 INFO:tasks.workunit.client.0.smithi102.stdout:6/737: unlink d9/db/f22 0 2024-05-13T22:48:15.126 INFO:tasks.workunit.client.0.smithi102.stdout:3/666: unlink de/d10/d2a/d36/l51 0 2024-05-13T22:48:15.131 INFO:tasks.workunit.client.0.smithi102.stdout:6/738: dwrite d9/db/f36 [4194304,4194304] 0 2024-05-13T22:48:15.132 INFO:tasks.workunit.client.0.smithi102.stdout:7/512: creat d6/d15/d4d/d55/d6f/f92 x:0 0 0 2024-05-13T22:48:15.133 INFO:tasks.workunit.client.0.smithi102.stdout:8/592: rmdir d2/d83 39 2024-05-13T22:48:15.138 INFO:tasks.workunit.client.0.smithi102.stdout:2/670: creat d7/d9/d10/d25/d56/fe1 x:0 0 0 2024-05-13T22:48:15.140 INFO:tasks.workunit.client.0.smithi102.stdout:3/667: fdatasync de/d10/d50/d8f/f60 0 2024-05-13T22:48:15.140 INFO:tasks.workunit.client.0.smithi102.stdout:2/671: read - d7/d9/d14/d31/dbb/dd5/fd8 zero size 2024-05-13T22:48:15.143 INFO:tasks.workunit.client.0.smithi102.stdout:7/513: dread - d6/d15/d4d/d55/d6f/d49/f4e zero size 2024-05-13T22:48:15.146 INFO:tasks.workunit.client.0.smithi102.stdout:1/516: dread d2/d4/fd [0,4194304] 0 2024-05-13T22:48:15.147 INFO:tasks.workunit.client.0.smithi102.stdout:2/672: dread d7/d9/d10/d30/f80 [0,4194304] 0 2024-05-13T22:48:15.148 INFO:tasks.workunit.client.0.smithi102.stdout:0/663: rename d1/de/d32/d24/d2d/d38/d3d/d81/da5/fc5 to d1/de/fcc 0 2024-05-13T22:48:15.151 INFO:tasks.workunit.client.0.smithi102.stdout:4/715: rmdir d3/d19/d23/dd2 0 2024-05-13T22:48:15.152 INFO:tasks.workunit.client.0.smithi102.stdout:6/739: mknod d9/cf2 0 2024-05-13T22:48:15.152 INFO:tasks.workunit.client.0.smithi102.stdout:7/514: dwrite d6/d15/f18 [0,4194304] 0 2024-05-13T22:48:15.158 INFO:tasks.workunit.client.0.smithi102.stdout:8/593: truncate d2/d7/d27/d64/d96/dae/f4f 257775 0 2024-05-13T22:48:15.163 INFO:tasks.workunit.client.0.smithi102.stdout:1/517: truncate d2/d4/f23 1142852 0 2024-05-13T22:48:15.165 INFO:tasks.workunit.client.0.smithi102.stdout:0/664: symlink d1/de/d32/d24/d2d/d3e/lcd 0 2024-05-13T22:48:15.165 INFO:tasks.workunit.client.0.smithi102.stdout:1/518: truncate d2/d4/f37 1606685 0 2024-05-13T22:48:15.167 INFO:tasks.workunit.client.0.smithi102.stdout:6/740: rename d9/db/d14/d35/dc9 to d9/db/d14/d35/dc9/df1/df3 22 2024-05-13T22:48:15.168 INFO:tasks.workunit.client.0.smithi102.stdout:2/673: dwrite d7/d9/d10/d30/f80 [0,4194304] 0 2024-05-13T22:48:15.170 INFO:tasks.workunit.client.0.smithi102.stdout:8/594: creat d2/d7/d27/fb6 x:0 0 0 2024-05-13T22:48:15.174 INFO:tasks.workunit.client.0.smithi102.stdout:6/741: dread d9/db/d14/dcc/d5f/fa4 [0,4194304] 0 2024-05-13T22:48:15.174 INFO:tasks.workunit.client.0.smithi102.stdout:3/668: getdents de/d10/d50/d8f 0 2024-05-13T22:48:15.175 INFO:tasks.workunit.client.0.smithi102.stdout:6/742: readlink d9/d25/d60/d6a/dad/lb2 0 2024-05-13T22:48:15.176 INFO:tasks.workunit.client.0.smithi102.stdout:3/669: dread - de/d10/d2a/d8c/da8/fd9 zero size 2024-05-13T22:48:15.180 INFO:tasks.workunit.client.0.smithi102.stdout:1/519: truncate d2/d4/d5/d19/d28/d6a/f69 666181 0 2024-05-13T22:48:15.181 INFO:tasks.workunit.client.0.smithi102.stdout:6/743: dread d9/feb [0,4194304] 0 2024-05-13T22:48:15.182 INFO:tasks.workunit.client.0.smithi102.stdout:6/744: chown d9/db/d14/f8d 1222938 1 2024-05-13T22:48:15.188 INFO:tasks.workunit.client.0.smithi102.stdout:0/665: rename d1/de/d42/da7 to d1/d4d/d79/d9f/dce 0 2024-05-13T22:48:15.191 INFO:tasks.workunit.client.0.smithi102.stdout:7/515: dread d6/d13/de/f12 [0,4194304] 0 2024-05-13T22:48:15.193 INFO:tasks.workunit.client.0.smithi102.stdout:3/670: mkdir de/d10/d2a/d36/db2/de7 0 2024-05-13T22:48:15.193 INFO:tasks.workunit.client.0.smithi102.stdout:7/516: chown d6/d15/d4c/d90/d4b/f66 1899683 1 2024-05-13T22:48:15.194 INFO:tasks.workunit.client.0.smithi102.stdout:3/671: chown de/f37 461716183 1 2024-05-13T22:48:15.197 INFO:tasks.workunit.client.0.smithi102.stdout:2/674: symlink d7/d15/d1e/le2 0 2024-05-13T22:48:15.197 INFO:tasks.workunit.client.0.smithi102.stdout:2/675: stat d7/l24 0 2024-05-13T22:48:15.199 INFO:tasks.workunit.client.0.smithi102.stdout:4/716: getdents d3/d19/d23/d39/d5d 0 2024-05-13T22:48:15.200 INFO:tasks.workunit.client.0.smithi102.stdout:3/672: dwrite de/d10/d2a/d33/fcb [0,4194304] 0 2024-05-13T22:48:15.201 INFO:tasks.workunit.client.0.smithi102.stdout:3/673: fsync de/d10/d50/f89 0 2024-05-13T22:48:15.205 INFO:tasks.workunit.client.0.smithi102.stdout:8/595: creat d2/d7/d25/d41/d50/d40/db2/fb7 x:0 0 0 2024-05-13T22:48:15.206 INFO:tasks.workunit.client.0.smithi102.stdout:0/666: creat d1/de/d32/d24/d2d/d38/d3d/fcf x:0 0 0 2024-05-13T22:48:15.208 INFO:tasks.workunit.client.0.smithi102.stdout:7/517: unlink d6/f70 0 2024-05-13T22:48:15.210 INFO:tasks.workunit.client.0.smithi102.stdout:2/676: write d7/d9/d14/d61/fa1 [221693,75832] 0 2024-05-13T22:48:15.213 INFO:tasks.workunit.client.0.smithi102.stdout:6/745: creat d9/da3/de8/ff4 x:0 0 0 2024-05-13T22:48:15.214 INFO:tasks.workunit.client.0.smithi102.stdout:6/746: chown d9/d45/d56/d5d/f87 0 1 2024-05-13T22:48:15.216 INFO:tasks.workunit.client.0.smithi102.stdout:8/596: stat d2/l1b 0 2024-05-13T22:48:15.217 INFO:tasks.workunit.client.0.smithi102.stdout:8/597: write d2/d7/d25/fb0 [667918,88727] 0 2024-05-13T22:48:15.224 INFO:tasks.workunit.client.0.smithi102.stdout:1/520: unlink d2/d4/d5/d19/d28/d6a/f69 0 2024-05-13T22:48:15.224 INFO:tasks.workunit.client.0.smithi102.stdout:1/521: chown d2/d4/fd 512880007 1 2024-05-13T22:48:15.227 INFO:tasks.workunit.client.0.smithi102.stdout:2/677: rename d7/d9/d10/d25/l43 to d7/d9/d14/d31/dbb/le3 0 2024-05-13T22:48:15.228 INFO:tasks.workunit.client.0.smithi102.stdout:6/747: stat d9/db/d14/d54/lc7 0 2024-05-13T22:48:15.230 INFO:tasks.workunit.client.0.smithi102.stdout:3/674: mkdir de/d10/d2a/d33/de8 0 2024-05-13T22:48:15.233 INFO:tasks.workunit.client.0.smithi102.stdout:0/667: write d1/d4d/f85 [250658,52327] 0 2024-05-13T22:48:15.236 INFO:tasks.workunit.client.0.smithi102.stdout:1/522: rename d2/f54 to d2/d45/d79/fa6 0 2024-05-13T22:48:15.237 INFO:tasks.workunit.client.0.smithi102.stdout:6/748: mknod d9/d45/d56/dd5/cf5 0 2024-05-13T22:48:15.238 INFO:tasks.workunit.client.0.smithi102.stdout:8/598: symlink d2/d36/db3/lb8 0 2024-05-13T22:48:15.239 INFO:tasks.workunit.client.0.smithi102.stdout:3/675: truncate de/d10/d2a/d33/f4d 5049 0 2024-05-13T22:48:15.243 INFO:tasks.workunit.client.0.smithi102.stdout:7/518: dwrite d6/d13/de/f59 [0,4194304] 0 2024-05-13T22:48:15.244 INFO:tasks.workunit.client.0.smithi102.stdout:1/523: rmdir d2/d10/d62 39 2024-05-13T22:48:15.246 INFO:tasks.workunit.client.0.smithi102.stdout:6/749: truncate d9/feb 1543690 0 2024-05-13T22:48:15.246 INFO:tasks.workunit.client.0.smithi102.stdout:1/524: truncate d2/d4/d26/f8c 866816 0 2024-05-13T22:48:15.249 INFO:tasks.workunit.client.0.smithi102.stdout:4/717: link d3/l21 d3/d22/lfd 0 2024-05-13T22:48:15.251 INFO:tasks.workunit.client.0.smithi102.stdout:3/676: mkdir de/d10/d2a/d36/db2/de9 0 2024-05-13T22:48:15.253 INFO:tasks.workunit.client.0.smithi102.stdout:8/599: write d2/d7/d27/d64/d96/dae/d76/f9f [462829,16515] 0 2024-05-13T22:48:15.254 INFO:tasks.workunit.client.0.smithi102.stdout:0/668: dread d1/f9 [0,4194304] 0 2024-05-13T22:48:15.258 INFO:tasks.workunit.client.0.smithi102.stdout:1/525: symlink d2/d4/d5/d19/d28/la7 0 2024-05-13T22:48:15.260 INFO:tasks.workunit.client.0.smithi102.stdout:6/750: truncate d9/db/d14/d35/d47/f70 1485053 0 2024-05-13T22:48:15.261 INFO:tasks.workunit.client.0.smithi102.stdout:2/678: write d7/d9/f29 [3618987,90575] 0 2024-05-13T22:48:15.262 INFO:tasks.workunit.client.0.smithi102.stdout:2/679: chown d7/d9/d10/d25/d56/d59/d81 20534747 1 2024-05-13T22:48:15.263 INFO:tasks.workunit.client.0.smithi102.stdout:2/680: dread - d7/d9/d14/d31/f97 zero size 2024-05-13T22:48:15.266 INFO:tasks.workunit.client.0.smithi102.stdout:8/600: creat d2/d7/d25/d41/d50/d40/d85/fb9 x:0 0 0 2024-05-13T22:48:15.269 INFO:tasks.workunit.client.0.smithi102.stdout:4/718: dread d3/d4/d6/f42 [4194304,4194304] 0 2024-05-13T22:48:15.275 INFO:tasks.workunit.client.0.smithi102.stdout:7/519: dwrite d6/d13/d2c/f3a [0,4194304] 0 2024-05-13T22:48:15.277 INFO:tasks.workunit.client.0.smithi102.stdout:6/751: mknod d9/d25/d60/d6a/dd6/de9/cf6 0 2024-05-13T22:48:15.279 INFO:tasks.workunit.client.0.smithi102.stdout:7/520: write d6/d13/d2c/f5c [4072645,48300] 0 2024-05-13T22:48:15.282 INFO:tasks.workunit.client.0.smithi102.stdout:3/677: dwrite fa [0,4194304] 0 2024-05-13T22:48:15.284 INFO:tasks.workunit.client.0.smithi102.stdout:8/601: dread d2/d7/d25/f74 [0,4194304] 0 2024-05-13T22:48:15.291 INFO:tasks.workunit.client.0.smithi102.stdout:1/526: creat d2/d10/d62/fa8 x:0 0 0 2024-05-13T22:48:15.305 INFO:tasks.workunit.client.0.smithi102.stdout:2/681: link d7/d15/l3e d7/d9/d10/d30/d53/da4/le4 0 2024-05-13T22:48:15.307 INFO:tasks.workunit.client.0.smithi102.stdout:0/669: getdents d1/de/d42 0 2024-05-13T22:48:15.308 INFO:tasks.workunit.client.0.smithi102.stdout:0/670: fsync d1/de/d32/d24/d2d/d3e/fc8 0 2024-05-13T22:48:15.311 INFO:tasks.workunit.client.0.smithi102.stdout:2/682: symlink d7/d9/d10/d25/d56/d59/d81/le5 0 2024-05-13T22:48:15.312 INFO:tasks.workunit.client.0.smithi102.stdout:8/602: getdents d2/d7/d25 0 2024-05-13T22:48:15.315 INFO:tasks.workunit.client.0.smithi102.stdout:2/683: stat d7/d9/l85 0 2024-05-13T22:48:15.316 INFO:tasks.workunit.client.0.smithi102.stdout:2/684: write d7/d9/d10/d25/d56/d59/fb0 [288859,51949] 0 2024-05-13T22:48:15.318 INFO:tasks.workunit.client.0.smithi102.stdout:1/527: dread d2/d45/d79/fa6 [0,4194304] 0 2024-05-13T22:48:15.320 INFO:tasks.workunit.client.0.smithi102.stdout:0/671: rename d1/de/d32/l48 to d1/de/d32/d8d/ld0 0 2024-05-13T22:48:15.324 INFO:tasks.workunit.client.0.smithi102.stdout:3/678: dwrite f2 [0,4194304] 0 2024-05-13T22:48:15.329 INFO:tasks.workunit.client.0.smithi102.stdout:1/528: rmdir d2/d10/d62 39 2024-05-13T22:48:15.331 INFO:tasks.workunit.client.0.smithi102.stdout:0/672: fsync d1/de/d32/d24/d2d/d38/fc0 0 2024-05-13T22:48:15.333 INFO:tasks.workunit.client.0.smithi102.stdout:2/685: fdatasync d7/d15/d1e/f26 0 2024-05-13T22:48:15.335 INFO:tasks.workunit.client.0.smithi102.stdout:3/679: creat de/d10/d50/d63/da2/fea x:0 0 0 2024-05-13T22:48:15.344 INFO:tasks.workunit.client.0.smithi102.stdout:2/686: symlink d7/d9/d10/d1b/d54/d83/le6 0 2024-05-13T22:48:15.346 INFO:tasks.workunit.client.0.smithi102.stdout:8/603: link d2/d7/f9 d2/d7/d25/d41/d50/d40/db2/fba 0 2024-05-13T22:48:15.348 INFO:tasks.workunit.client.0.smithi102.stdout:8/604: read d2/d7/d27/f3b [600844,120350] 0 2024-05-13T22:48:15.350 INFO:tasks.workunit.client.0.smithi102.stdout:1/529: write d2/d10/f5d [331393,36389] 0 2024-05-13T22:48:15.350 INFO:tasks.workunit.client.0.smithi102.stdout:1/530: dread - d2/d45/d79/f81 zero size 2024-05-13T22:48:15.353 INFO:tasks.workunit.client.0.smithi102.stdout:3/680: mknod de/d10/d2a/d36/db2/de7/ceb 0 2024-05-13T22:48:15.355 INFO:tasks.workunit.client.0.smithi102.stdout:8/605: mkdir d2/d7/d27/dbb 0 2024-05-13T22:48:15.357 INFO:tasks.workunit.client.0.smithi102.stdout:0/673: creat d1/de/d32/d24/fd1 x:0 0 0 2024-05-13T22:48:15.357 INFO:tasks.workunit.client.0.smithi102.stdout:2/687: rmdir d7/d9/d10/d1b 39 2024-05-13T22:48:15.357 INFO:tasks.workunit.client.0.smithi102.stdout:2/688: stat d7/d9/d7c 0 2024-05-13T22:48:15.358 INFO:tasks.workunit.client.0.smithi102.stdout:2/689: stat d7/d9/d10/d25/d56/f8c 0 2024-05-13T22:48:15.362 INFO:tasks.workunit.client.0.smithi102.stdout:8/606: write d2/d7/d27/f3b [1946440,55424] 0 2024-05-13T22:48:15.364 INFO:tasks.workunit.client.0.smithi102.stdout:1/531: mknod d2/d4/d26/da2/ca9 0 2024-05-13T22:48:15.372 INFO:tasks.workunit.client.0.smithi102.stdout:0/674: getdents d1/de/d42 0 2024-05-13T22:48:15.375 INFO:tasks.workunit.client.0.smithi102.stdout:2/690: dread d7/d15/d67/fa8 [0,4194304] 0 2024-05-13T22:48:15.376 INFO:tasks.workunit.client.0.smithi102.stdout:8/607: dread d2/d7/d27/d64/d96/dae/f13 [0,4194304] 0 2024-05-13T22:48:15.377 INFO:tasks.workunit.client.0.smithi102.stdout:3/681: getdents de/d10/d2a/d36/d88 0 2024-05-13T22:48:15.380 INFO:tasks.workunit.client.0.smithi102.stdout:1/532: rename d2/d4/d5/d19/d28/d6a/f98 to d2/d45/d79/faa 0 2024-05-13T22:48:15.382 INFO:tasks.workunit.client.0.smithi102.stdout:8/608: mkdir d2/d7/d27/d64/d96/dbc 0 2024-05-13T22:48:15.382 INFO:tasks.workunit.client.0.smithi102.stdout:8/609: stat d2/d36/d49/l7c 0 2024-05-13T22:48:15.384 INFO:tasks.workunit.client.0.smithi102.stdout:3/682: rename de/d10/d2a/fa4 to de/d10/d2a/fec 0 2024-05-13T22:48:15.385 INFO:tasks.workunit.client.0.smithi102.stdout:0/675: mknod d1/cd2 0 2024-05-13T22:48:15.393 INFO:tasks.workunit.client.0.smithi102.stdout:0/676: rmdir d1/d4d 39 2024-05-13T22:48:15.395 INFO:tasks.workunit.client.0.smithi102.stdout:1/533: mknod d2/d4/d5/d19/cab 0 2024-05-13T22:48:15.397 INFO:tasks.workunit.client.0.smithi102.stdout:3/683: dwrite de/d68/fe3 [0,4194304] 0 2024-05-13T22:48:15.402 INFO:tasks.workunit.client.0.smithi102.stdout:1/534: fdatasync d2/d4/d38/f89 0 2024-05-13T22:48:15.404 INFO:tasks.workunit.client.0.smithi102.stdout:8/610: getdents d2/d83 0 2024-05-13T22:48:15.404 INFO:tasks.workunit.client.0.smithi102.stdout:2/691: write d7/d15/d1e/f26 [578350,59890] 0 2024-05-13T22:48:15.413 INFO:tasks.workunit.client.0.smithi102.stdout:2/692: chown d7/d9/d10/d1b/d54/d83/le6 255 1 2024-05-13T22:48:15.420 INFO:tasks.workunit.client.0.smithi102.stdout:1/535: symlink d2/d4/d26/d40/lac 0 2024-05-13T22:48:15.424 INFO:tasks.workunit.client.0.smithi102.stdout:8/611: rename d2/d7/d25/d41/d50/f34 to d2/d36/d49/d5d/fbd 0 2024-05-13T22:48:15.430 INFO:tasks.workunit.client.0.smithi102.stdout:1/536: dread d2/d45/d79/d92/f64 [0,4194304] 0 2024-05-13T22:48:15.430 INFO:tasks.workunit.client.0.smithi102.stdout:0/677: write d1/d4d/f6a [173879,91101] 0 2024-05-13T22:48:15.431 INFO:tasks.workunit.client.0.smithi102.stdout:3/684: dwrite de/d10/d34/f93 [0,4194304] 0 2024-05-13T22:48:15.431 INFO:tasks.workunit.client.0.smithi102.stdout:1/537: chown l1 118861 1 2024-05-13T22:48:15.432 INFO:tasks.workunit.client.0.smithi102.stdout:8/612: truncate d2/f56 648956 0 2024-05-13T22:48:15.432 INFO:tasks.workunit.client.0.smithi102.stdout:2/693: creat d7/d9/d10/fe7 x:0 0 0 2024-05-13T22:48:15.432 INFO:tasks.workunit.client.0.smithi102.stdout:0/678: readlink d1/d4d/l84 0 2024-05-13T22:48:15.437 INFO:tasks.workunit.client.0.smithi102.stdout:3/685: readlink de/l13 0 2024-05-13T22:48:15.446 INFO:tasks.workunit.client.0.smithi102.stdout:1/538: dwrite d2/d4/f75 [0,4194304] 0 2024-05-13T22:48:15.448 INFO:tasks.workunit.client.0.smithi102.stdout:8/613: dwrite d2/d7/d27/d64/d96/dae/d76/f90 [0,4194304] 0 2024-05-13T22:48:15.448 INFO:tasks.workunit.client.0.smithi102.stdout:0/679: dwrite d1/de/d32/d24/d2d/d38/d3d/fcf [0,4194304] 0 2024-05-13T22:48:15.460 INFO:tasks.workunit.client.0.smithi102.stdout:8/614: fsync d2/d7/d27/d64/d96/dae/f13 0 2024-05-13T22:48:15.463 INFO:tasks.workunit.client.0.smithi102.stdout:1/539: symlink d2/d4/d5/d19/d28/lad 0 2024-05-13T22:48:15.464 INFO:tasks.workunit.client.0.smithi102.stdout:8/615: write d2/d7/d27/d64/d96/dae/f58 [5035484,86786] 0 2024-05-13T22:48:15.465 INFO:tasks.workunit.client.0.smithi102.stdout:0/680: symlink d1/de/d32/ld3 0 2024-05-13T22:48:15.466 INFO:tasks.workunit.client.0.smithi102.stdout:2/694: getdents d7/d9/d10/d25/d56/d59/d81/dcc 0 2024-05-13T22:48:15.467 INFO:tasks.workunit.client.0.smithi102.stdout:3/686: dwrite de/d10/d2a/f9c [0,4194304] 0 2024-05-13T22:48:15.471 INFO:tasks.workunit.client.0.smithi102.stdout:0/681: rmdir d1/de/d32/d24/d2d/d3e 39 2024-05-13T22:48:15.473 INFO:tasks.workunit.client.0.smithi102.stdout:8/616: rmdir d2/d7/d25/d41/d50/d93 39 2024-05-13T22:48:15.474 INFO:tasks.workunit.client.0.smithi102.stdout:0/682: write d1/de/d32/d24/f74 [1891970,113465] 0 2024-05-13T22:48:15.477 INFO:tasks.workunit.client.0.smithi102.stdout:3/687: dwrite f7 [4194304,4194304] 0 2024-05-13T22:48:15.477 INFO:tasks.workunit.client.0.smithi102.stdout:2/695: mkdir d7/d9/d14/d31/dd0/de8 0 2024-05-13T22:48:15.478 INFO:tasks.workunit.client.0.smithi102.stdout:2/696: chown d7/d15/fae 17641400 1 2024-05-13T22:48:15.479 INFO:tasks.workunit.client.0.smithi102.stdout:3/688: write fa [1331639,116495] 0 2024-05-13T22:48:15.480 INFO:tasks.workunit.client.0.smithi102.stdout:0/683: creat d1/d68/fd4 x:0 0 0 2024-05-13T22:48:15.490 INFO:tasks.workunit.client.0.smithi102.stdout:3/689: unlink de/d10/d2a/d33/d61/l57 0 2024-05-13T22:48:15.491 INFO:tasks.workunit.client.0.smithi102.stdout:2/697: creat d7/d15/d1e/dbf/fe9 x:0 0 0 2024-05-13T22:48:15.492 INFO:tasks.workunit.client.0.smithi102.stdout:2/698: stat d7/d9/d11 0 2024-05-13T22:48:15.495 INFO:tasks.workunit.client.0.smithi102.stdout:0/684: read d1/f5 [7315082,61741] 0 2024-05-13T22:48:15.499 INFO:tasks.workunit.client.0.smithi102.stdout:1/540: write d2/f46 [1809221,79112] 0 2024-05-13T22:48:15.579 INFO:tasks.workunit.client.0.smithi102.stdout:8/617: link d2/d36/c4b d2/d7/d27/d64/d96/dae/cbe 0 2024-05-13T22:48:15.580 INFO:tasks.workunit.client.0.smithi102.stdout:3/690: creat de/d10/d2a/d8c/d7f/fed x:0 0 0 2024-05-13T22:48:15.580 INFO:tasks.workunit.client.0.smithi102.stdout:2/699: fsync d7/d9/d14/d31/f97 0 2024-05-13T22:48:15.581 INFO:tasks.workunit.client.0.smithi102.stdout:0/685: read - d1/de/d32/d24/d2d/d38/f97 zero size 2024-05-13T22:48:15.582 INFO:tasks.workunit.client.0.smithi102.stdout:2/700: read d7/d9/d10/d25/d56/d59/fb0 [39279,89514] 0 2024-05-13T22:48:15.584 INFO:tasks.workunit.client.0.smithi102.stdout:8/618: creat d2/d83/fbf x:0 0 0 2024-05-13T22:48:15.584 INFO:tasks.workunit.client.0.smithi102.stdout:2/701: stat d7/d9/d10/d25/d27/dab 0 2024-05-13T22:48:15.585 INFO:tasks.workunit.client.0.smithi102.stdout:2/702: readlink d7/d9/d10/d30/l65 0 2024-05-13T22:48:15.588 INFO:tasks.workunit.client.0.smithi102.stdout:2/703: write d7/d9/d10/f33 [1690263,41649] 0 2024-05-13T22:48:15.588 INFO:tasks.workunit.client.0.smithi102.stdout:8/619: mkdir d2/d36/dc0 0 2024-05-13T22:48:15.592 INFO:tasks.workunit.client.0.smithi102.stdout:0/686: dwrite d1/de/d32/dab/fbc [0,4194304] 0 2024-05-13T22:48:15.594 INFO:tasks.workunit.client.0.smithi102.stdout:0/687: truncate d1/d77/fc2 105782 0 2024-05-13T22:48:15.595 INFO:tasks.workunit.client.0.smithi102.stdout:1/541: dwrite d2/d4/d5/d19/d28/f24 [0,4194304] 0 2024-05-13T22:48:15.600 INFO:tasks.workunit.client.0.smithi102.stdout:2/704: truncate d7/f75 566952 0 2024-05-13T22:48:15.601 INFO:tasks.workunit.client.0.smithi102.stdout:0/688: truncate d1/f9 3451967 0 2024-05-13T22:48:15.605 INFO:tasks.workunit.client.0.smithi102.stdout:2/705: symlink d7/d15/d1e/d87/lea 0 2024-05-13T22:48:15.607 INFO:tasks.workunit.client.0.smithi102.stdout:8/620: rmdir d2/d36/dc0 0 2024-05-13T22:48:15.611 INFO:tasks.workunit.client.0.smithi102.stdout:2/706: mkdir d7/d9/d14/d31/deb 0 2024-05-13T22:48:15.612 INFO:tasks.workunit.client.0.smithi102.stdout:2/707: dread - d7/d9/d10/d30/d53/da4/fca zero size 2024-05-13T22:48:15.615 INFO:tasks.workunit.client.0.smithi102.stdout:1/542: getdents d2/d4/d26/da2 0 2024-05-13T22:48:15.624 INFO:tasks.workunit.client.0.smithi102.stdout:1/543: link d2/d4/c99 d2/d10/d62/cae 0 2024-05-13T22:48:15.629 INFO:tasks.workunit.client.0.smithi102.stdout:1/544: dread d2/d4/d5/d19/d28/f35 [0,4194304] 0 2024-05-13T22:48:15.630 INFO:tasks.workunit.client.0.smithi102.stdout:1/545: readlink d2/d4/d26/l5f 0 2024-05-13T22:48:15.632 INFO:tasks.workunit.client.0.smithi102.stdout:1/546: mknod d2/d4/d9e/caf 0 2024-05-13T22:48:15.640 INFO:tasks.workunit.client.0.smithi102.stdout:3/691: write de/d10/d46/fdb [2411337,17599] 0 2024-05-13T22:48:15.643 INFO:tasks.workunit.client.0.smithi102.stdout:3/692: mkdir de/d10/d50/d63/d98/dee 0 2024-05-13T22:48:15.647 INFO:tasks.workunit.client.0.smithi102.stdout:3/693: rename de/d10/d34/f38 to de/d67/de1/fef 0 2024-05-13T22:48:15.647 INFO:tasks.workunit.client.0.smithi102.stdout:3/694: chown de/d10/d2a/d36/db2/de9 971 1 2024-05-13T22:48:15.653 INFO:tasks.workunit.client.0.smithi102.stdout:8/621: truncate d2/f56 217446 0 2024-05-13T22:48:15.653 INFO:tasks.workunit.client.0.smithi102.stdout:8/622: write d2/d7/d27/f46 [4473892,75819] 0 2024-05-13T22:48:15.656 INFO:tasks.workunit.client.0.smithi102.stdout:8/623: mknod d2/d36/db3/cc1 0 2024-05-13T22:48:15.657 INFO:tasks.workunit.client.0.smithi102.stdout:8/624: chown d2/d7/d25/d41/d50/l6b 719891 1 2024-05-13T22:48:15.658 INFO:tasks.workunit.client.0.smithi102.stdout:8/625: fdatasync d2/d7/d27/d64/d96/dae/d76/f9f 0 2024-05-13T22:48:15.663 INFO:tasks.workunit.client.0.smithi102.stdout:8/626: read d2/d7/f9 [4619520,60754] 0 2024-05-13T22:48:15.663 INFO:tasks.workunit.client.0.smithi102.stdout:1/547: dwrite d2/d4/d26/d40/f72 [0,4194304] 0 2024-05-13T22:48:15.665 INFO:tasks.workunit.client.0.smithi102.stdout:8/627: dread - d2/d7/d27/d64/d96/dae/d78/fa7 zero size 2024-05-13T22:48:15.667 INFO:tasks.workunit.client.0.smithi102.stdout:1/548: mknod d2/d4/d26/da2/cb0 0 2024-05-13T22:48:15.673 INFO:tasks.workunit.client.0.smithi102.stdout:3/695: write de/d10/d46/f6c [1446384,111552] 0 2024-05-13T22:48:15.674 INFO:tasks.workunit.client.0.smithi102.stdout:3/696: chown de/d10/d2a/d36/db2/fbb 36865842 1 2024-05-13T22:48:15.675 INFO:tasks.workunit.client.0.smithi102.stdout:1/549: write d2/d4/d26/f33 [2370207,63029] 0 2024-05-13T22:48:15.678 INFO:tasks.workunit.client.0.smithi102.stdout:2/708: dwrite d7/d15/d67/fa8 [0,4194304] 0 2024-05-13T22:48:15.678 INFO:tasks.workunit.client.0.smithi102.stdout:3/697: fdatasync de/d10/d50/d63/da2/fdd 0 2024-05-13T22:48:15.679 INFO:tasks.workunit.client.0.smithi102.stdout:8/628: dwrite d2/d7/d27/d64/d96/dae/f23 [4194304,4194304] 0 2024-05-13T22:48:15.681 INFO:tasks.workunit.client.0.smithi102.stdout:8/629: fsync d2/d36/f42 0 2024-05-13T22:48:15.686 INFO:tasks.workunit.client.0.smithi102.stdout:2/709: fdatasync d7/d9/d14/f62 0 2024-05-13T22:48:15.697 INFO:tasks.workunit.client.0.smithi102.stdout:8/630: fsync d2/d7/d25/d41/d50/d93/f98 0 2024-05-13T22:48:15.701 INFO:tasks.workunit.client.0.smithi102.stdout:2/710: dread d7/d9/f89 [0,4194304] 0 2024-05-13T22:48:15.703 INFO:tasks.workunit.client.0.smithi102.stdout:3/698: write de/d10/d2a/d33/f4d [705914,111197] 0 2024-05-13T22:48:15.704 INFO:tasks.workunit.client.0.smithi102.stdout:3/699: dread - de/d10/d50/d63/da2/fea zero size 2024-05-13T22:48:15.707 INFO:tasks.workunit.client.0.smithi102.stdout:3/700: mknod de/d10/d46/cf0 0 2024-05-13T22:48:15.717 INFO:tasks.workunit.client.0.smithi102.stdout:3/701: dwrite de/d10/d2a/d33/f4d [0,4194304] 0 2024-05-13T22:48:15.719 INFO:tasks.workunit.client.0.smithi102.stdout:8/631: dwrite d2/d36/d49/d5d/f26 [0,4194304] 0 2024-05-13T22:48:15.720 INFO:tasks.workunit.client.0.smithi102.stdout:2/711: dwrite d7/d9/f89 [0,4194304] 0 2024-05-13T22:48:15.721 INFO:tasks.workunit.client.0.smithi102.stdout:3/702: chown de/d10/d2a/d8c/d7f 74165550 1 2024-05-13T22:48:15.722 INFO:tasks.workunit.client.0.smithi102.stdout:3/703: dread - de/d1c/d79/f81 zero size 2024-05-13T22:48:15.723 INFO:tasks.workunit.client.0.smithi102.stdout:2/712: chown d7/d9/lc 123829069 1 2024-05-13T22:48:15.726 INFO:tasks.workunit.client.0.smithi102.stdout:3/704: symlink de/d10/d2a/d36/da0/lf1 0 2024-05-13T22:48:15.728 INFO:tasks.workunit.client.0.smithi102.stdout:8/632: symlink d2/d7/d27/dad/lc2 0 2024-05-13T22:48:15.730 INFO:tasks.workunit.client.0.smithi102.stdout:3/705: symlink de/d10/d50/lf2 0 2024-05-13T22:48:15.733 INFO:tasks.workunit.client.0.smithi102.stdout:2/713: link d7/d9/d10/d1b/l1d d7/d15/d1e/d87/lec 0 2024-05-13T22:48:15.735 INFO:tasks.workunit.client.0.smithi102.stdout:8/633: creat d2/d7/d25/d41/d50/d93/dab/fc3 x:0 0 0 2024-05-13T22:48:15.738 INFO:tasks.workunit.client.0.smithi102.stdout:2/714: dread d7/d9/d10/d25/d27/f5b [0,4194304] 0 2024-05-13T22:48:15.739 INFO:tasks.workunit.client.0.smithi102.stdout:8/634: creat d2/d36/d49/fc4 x:0 0 0 2024-05-13T22:48:15.740 INFO:tasks.workunit.client.0.smithi102.stdout:2/715: read d7/d9/d10/d1b/d54/fcb [876679,110098] 0 2024-05-13T22:48:15.878 INFO:tasks.workunit.client.0.smithi102.stdout:8/635: creat d2/d36/d49/d7e/fc5 x:0 0 0 2024-05-13T22:48:15.879 INFO:tasks.workunit.client.0.smithi102.stdout:3/706: write de/d68/f9d [162868,26272] 0 2024-05-13T22:48:15.880 INFO:tasks.workunit.client.0.smithi102.stdout:2/716: read d7/d9/d10/d25/f68 [2434654,5028] 0 2024-05-13T22:48:15.889 INFO:tasks.workunit.client.0.smithi102.stdout:2/717: dwrite d7/d9/fcf [0,4194304] 0 2024-05-13T22:48:15.892 INFO:tasks.workunit.client.0.smithi102.stdout:2/718: read - d7/d9/d10/d25/d56/d59/fd6 zero size 2024-05-13T22:48:15.900 INFO:tasks.workunit.client.0.smithi102.stdout:2/719: rename d7/d9/d10/d30/d53/dd1/fd9 to d7/d9/d10/d1b/d54/d83/fed 0 2024-05-13T22:48:15.901 INFO:tasks.workunit.client.0.smithi102.stdout:2/720: chown d7/d9/d14/d61/fa1 7 1 2024-05-13T22:48:15.904 INFO:tasks.workunit.client.0.smithi102.stdout:2/721: mkdir d7/d9/d10/d25/d27/dab/dee 0 2024-05-13T22:48:15.955 INFO:tasks.workunit.client.0.smithi102.stdout:5/795: sync 2024-05-13T22:48:15.973 INFO:tasks.workunit.client.0.smithi102.stdout:2/722: link d7/d9/d10/d25/d27/c9c d7/d15/d1e/da7/cef 0 2024-05-13T22:48:15.997 INFO:tasks.workunit.client.0.smithi102.stdout:5/796: rmdir de/d25/d59/d7f/d86/da5/ddb/dee/d116 39 2024-05-13T22:48:15.999 INFO:tasks.workunit.client.0.smithi102.stdout:2/723: write d7/d9/f29 [4993111,13850] 0 2024-05-13T22:48:16.001 INFO:tasks.workunit.client.0.smithi102.stdout:5/797: creat de/d25/d58/dfb/f126 x:0 0 0 2024-05-13T22:48:16.006 INFO:tasks.workunit.client.0.smithi102.stdout:5/798: dread de/d25/d43/f94 [0,4194304] 0 2024-05-13T22:48:16.007 INFO:tasks.workunit.client.0.smithi102.stdout:2/724: dwrite d7/d9/d10/d1b/f8a [4194304,4194304] 0 2024-05-13T22:48:16.009 INFO:tasks.workunit.client.0.smithi102.stdout:2/725: write d7/d9/d10/f33 [1913355,45806] 0 2024-05-13T22:48:16.012 INFO:tasks.workunit.client.0.smithi102.stdout:2/726: creat d7/d9/d14/d61/d82/ff0 x:0 0 0 2024-05-13T22:48:16.013 INFO:tasks.workunit.client.0.smithi102.stdout:2/727: chown d7/d9/d14/d61/d82/ff0 56243652 1 2024-05-13T22:48:16.016 INFO:tasks.workunit.client.0.smithi102.stdout:2/728: mknod d7/d9/d10/d30/d53/dd1/cf1 0 2024-05-13T22:48:16.017 INFO:tasks.workunit.client.0.smithi102.stdout:2/729: write d7/d9/d10/d1b/f8a [3420336,57882] 0 2024-05-13T22:48:16.024 INFO:tasks.workunit.client.0.smithi102.stdout:2/730: dwrite d7/d9/d10/d25/d56/fe1 [0,4194304] 0 2024-05-13T22:48:16.024 INFO:tasks.workunit.client.0.smithi102.stdout:2/731: stat d7/d9/d10/fc1 0 2024-05-13T22:48:16.027 INFO:tasks.workunit.client.0.smithi102.stdout:2/732: creat d7/d9/d10/d30/d7f/ff2 x:0 0 0 2024-05-13T22:48:16.072 INFO:tasks.workunit.client.0.smithi102.stdout:2/733: mkdir d7/d15/d67/df3 0 2024-05-13T22:48:16.072 INFO:tasks.workunit.client.0.smithi102.stdout:2/734: fdatasync d7/d15/fad 0 2024-05-13T22:48:16.073 INFO:tasks.workunit.client.0.smithi102.stdout:2/735: fsync d7/d15/d1e/d87/fcd 0 2024-05-13T22:48:16.074 INFO:tasks.workunit.client.0.smithi102.stdout:3/707: write de/d10/d2a/f7e [433389,50124] 0 2024-05-13T22:48:16.075 INFO:tasks.workunit.client.0.smithi102.stdout:2/736: chown d7/d15/d1e/d87/lec 58 1 2024-05-13T22:48:16.078 INFO:tasks.workunit.client.0.smithi102.stdout:8/636: truncate d2/d7/d27/d64/d96/dae/f23 6458476 0 2024-05-13T22:48:16.079 INFO:tasks.workunit.client.0.smithi102.stdout:3/708: read - de/d10/d46/fac zero size 2024-05-13T22:48:16.080 INFO:tasks.workunit.client.0.smithi102.stdout:8/637: fdatasync d2/d7/d25/d41/d50/d40/f5b 0 2024-05-13T22:48:16.084 INFO:tasks.workunit.client.0.smithi102.stdout:3/709: mkdir de/d68/df3 0 2024-05-13T22:48:16.086 INFO:tasks.workunit.client.0.smithi102.stdout:3/710: read de/d10/d50/f89 [2377694,98783] 0 2024-05-13T22:48:16.089 INFO:tasks.workunit.client.0.smithi102.stdout:3/711: dread de/d10/d2a/f7e [0,4194304] 0 2024-05-13T22:48:16.090 INFO:tasks.workunit.client.0.smithi102.stdout:8/638: link d2/d36/d49/l7c d2/d7/d27/d64/d96/dbc/lc6 0 2024-05-13T22:48:16.098 INFO:tasks.workunit.client.0.smithi102.stdout:8/639: mkdir d2/d7/d27/dbb/dc7 0 2024-05-13T22:48:16.099 INFO:tasks.workunit.client.0.smithi102.stdout:8/640: write d2/d7/d25/d41/d50/d40/d85/fb9 [744516,220] 0 2024-05-13T22:48:16.122 INFO:tasks.workunit.client.0.smithi102.stdout:8/641: dread d2/d7/d25/d41/d6f/f84 [0,4194304] 0 2024-05-13T22:48:16.126 INFO:tasks.workunit.client.0.smithi102.stdout:8/642: rename d2/d7/d25/d41/d50 to d2/d7/d27/d64/d80/dc8 0 2024-05-13T22:48:16.142 INFO:tasks.workunit.client.0.smithi102.stdout:2/737: truncate d7/d9/d10/f33 1433452 0 2024-05-13T22:48:16.150 INFO:tasks.workunit.client.0.smithi102.stdout:3/712: dwrite de/d10/d46/f6f [0,4194304] 0 2024-05-13T22:48:16.154 INFO:tasks.workunit.client.0.smithi102.stdout:5/799: mknod de/d25/d2a/d4b/d4d/d50/c127 0 2024-05-13T22:48:16.161 INFO:tasks.workunit.client.0.smithi102.stdout:8/643: write d2/f56 [830831,67015] 0 2024-05-13T22:48:16.162 INFO:tasks.workunit.client.0.smithi102.stdout:8/644: readlink d2/d36/d49/d5d/l99 0 2024-05-13T22:48:16.165 INFO:tasks.workunit.client.0.smithi102.stdout:8/645: unlink d2/d36/d49/d7e/fc5 0 2024-05-13T22:48:16.168 INFO:tasks.workunit.client.0.smithi102.stdout:8/646: creat d2/d7/d27/d64/d80/dc8/d40/d85/fc9 x:0 0 0 2024-05-13T22:48:16.169 INFO:tasks.workunit.client.0.smithi102.stdout:8/647: fsync d2/d7/d27/d64/d96/dae/f13 0 2024-05-13T22:48:16.173 INFO:tasks.workunit.client.0.smithi102.stdout:3/713: write de/d10/d50/d63/d98/fc4 [1065784,51063] 0 2024-05-13T22:48:16.176 INFO:tasks.workunit.client.0.smithi102.stdout:8/648: dwrite d2/d7/d25/d41/f75 [0,4194304] 0 2024-05-13T22:48:16.176 INFO:tasks.workunit.client.0.smithi102.stdout:5/800: write de/d25/d43/d7a/fea [5003232,77863] 0 2024-05-13T22:48:16.181 INFO:tasks.workunit.client.0.smithi102.stdout:3/714: truncate de/d10/d2a/f7e 1080030 0 2024-05-13T22:48:16.184 INFO:tasks.workunit.client.0.smithi102.stdout:8/649: chown d2/d36/d49/d5d/fbd 8 1 2024-05-13T22:48:16.186 INFO:tasks.workunit.client.0.smithi102.stdout:3/715: chown de/d10/d2a/fec 50 1 2024-05-13T22:48:16.189 INFO:tasks.workunit.client.0.smithi102.stdout:8/650: mknod d2/d7/d27/d64/d80/dc8/d93/dab/cca 0 2024-05-13T22:48:16.190 INFO:tasks.workunit.client.0.smithi102.stdout:3/716: dread de/d10/d2a/d33/f4d [0,4194304] 0 2024-05-13T22:48:16.192 INFO:tasks.workunit.client.0.smithi102.stdout:5/801: write de/f1e [1344084,127954] 0 2024-05-13T22:48:16.196 INFO:tasks.workunit.client.0.smithi102.stdout:5/802: truncate de/d25/d58/fa6 321190 0 2024-05-13T22:48:16.200 INFO:tasks.workunit.client.0.smithi102.stdout:3/717: getdents de/d10/d34 0 2024-05-13T22:48:16.201 INFO:tasks.workunit.client.0.smithi102.stdout:5/803: dread de/d25/d59/f8f [0,4194304] 0 2024-05-13T22:48:16.201 INFO:tasks.workunit.client.0.smithi102.stdout:3/718: chown de/d10/d2a/d8c 42 1 2024-05-13T22:48:16.202 INFO:tasks.workunit.client.0.smithi102.stdout:3/719: stat de/d10/d2a/d36/l8e 0 2024-05-13T22:48:16.202 INFO:tasks.workunit.client.0.smithi102.stdout:3/720: chown de/d67/l73 871097944 1 2024-05-13T22:48:16.203 INFO:tasks.workunit.client.0.smithi102.stdout:3/721: chown de/d10/d46/dc8/fdf 133 1 2024-05-13T22:48:16.205 INFO:tasks.workunit.client.0.smithi102.stdout:3/722: getdents de/d68/df3 0 2024-05-13T22:48:16.207 INFO:tasks.workunit.client.0.smithi102.stdout:3/723: creat de/d67/ff4 x:0 0 0 2024-05-13T22:48:16.208 INFO:tasks.workunit.client.0.smithi102.stdout:3/724: fsync de/d10/d2a/d8c/d7f/fed 0 2024-05-13T22:48:16.209 INFO:tasks.workunit.client.0.smithi102.stdout:3/725: stat de/d10/d46 0 2024-05-13T22:48:16.209 INFO:tasks.workunit.client.0.smithi102.stdout:5/804: dread de/f35 [0,4194304] 0 2024-05-13T22:48:16.215 INFO:tasks.workunit.client.0.smithi102.stdout:3/726: dwrite de/d10/d50/f47 [0,4194304] 0 2024-05-13T22:48:16.219 INFO:tasks.workunit.client.0.smithi102.stdout:3/727: creat de/d67/de1/ff5 x:0 0 0 2024-05-13T22:48:16.226 INFO:tasks.workunit.client.0.smithi102.stdout:3/728: getdents de/d10/d46 0 2024-05-13T22:48:16.233 INFO:tasks.workunit.client.0.smithi102.stdout:5/805: dwrite de/d75/fa4 [0,4194304] 0 2024-05-13T22:48:16.234 INFO:tasks.workunit.client.0.smithi102.stdout:3/729: dwrite de/d10/d2a/d33/daa/fb3 [0,4194304] 0 2024-05-13T22:48:16.236 INFO:tasks.workunit.client.0.smithi102.stdout:5/806: fdatasync de/d25/d59/fb4 0 2024-05-13T22:48:16.239 INFO:tasks.workunit.client.0.smithi102.stdout:3/730: mknod de/d10/d50/d63/cf6 0 2024-05-13T22:48:16.242 INFO:tasks.workunit.client.0.smithi102.stdout:5/807: dread de/d25/d58/f101 [0,4194304] 0 2024-05-13T22:48:16.243 INFO:tasks.workunit.client.0.smithi102.stdout:5/808: chown de/d25/d2a/d4b/f110 303992 1 2024-05-13T22:48:16.245 INFO:tasks.workunit.client.0.smithi102.stdout:3/731: truncate de/d1c/d79/f2b 4034422 0 2024-05-13T22:48:16.252 INFO:tasks.workunit.client.0.smithi102.stdout:3/732: rename de/d1c/d90 to de/d67/df7 0 2024-05-13T22:48:16.252 INFO:tasks.workunit.client.0.smithi102.stdout:5/809: link de/f11 de/d25/d2a/d4b/d4d/d50/da3/f128 0 2024-05-13T22:48:16.255 INFO:tasks.workunit.client.0.smithi102.stdout:3/733: truncate de/d10/f44 2015967 0 2024-05-13T22:48:16.314 INFO:tasks.workunit.client.0.smithi102.stdout:9/646: sync 2024-05-13T22:48:16.314 INFO:tasks.workunit.client.0.smithi102.stdout:4/719: sync 2024-05-13T22:48:16.318 INFO:tasks.workunit.client.0.smithi102.stdout:9/647: rmdir d1/d6/d45/d3e 39 2024-05-13T22:48:16.323 INFO:tasks.workunit.client.0.smithi102.stdout:4/720: symlink d3/d19/d23/d39/da7/d7d/lfe 0 2024-05-13T22:48:16.324 INFO:tasks.workunit.client.0.smithi102.stdout:4/721: chown d3/d4/f32 679691958 1 2024-05-13T22:48:16.329 INFO:tasks.workunit.client.0.smithi102.stdout:3/734: dwrite de/d10/d50/d63/f76 [0,4194304] 0 2024-05-13T22:48:16.331 INFO:tasks.workunit.client.0.smithi102.stdout:3/735: chown de/d10/d2a 1 1 2024-05-13T22:48:16.332 INFO:tasks.workunit.client.0.smithi102.stdout:3/736: readlink de/d1c/d79/l80 0 2024-05-13T22:48:16.333 INFO:tasks.workunit.client.0.smithi102.stdout:4/722: dwrite d3/d4/d6/fb5 [0,4194304] 0 2024-05-13T22:48:16.341 INFO:tasks.workunit.client.0.smithi102.stdout:4/723: truncate d3/d4/f47 2413048 0 2024-05-13T22:48:16.343 INFO:tasks.workunit.client.0.smithi102.stdout:3/737: dread de/d10/d50/d63/d98/fc3 [0,4194304] 0 2024-05-13T22:48:16.346 INFO:tasks.workunit.client.0.smithi102.stdout:3/738: dread de/d68/f9d [0,4194304] 0 2024-05-13T22:48:16.349 INFO:tasks.workunit.client.0.smithi102.stdout:9/648: dread d1/d6/d45/f4f [0,4194304] 0 2024-05-13T22:48:16.350 INFO:tasks.workunit.client.0.smithi102.stdout:9/649: chown d1/d6/db/d4a/fd7 1777 1 2024-05-13T22:48:16.352 INFO:tasks.workunit.client.0.smithi102.stdout:9/650: write d1/d6/d21/d71/fb3 [7113316,121486] 0 2024-05-13T22:48:16.352 INFO:tasks.workunit.client.0.smithi102.stdout:9/651: chown d1/d5d/d6c 8955248 1 2024-05-13T22:48:16.361 INFO:tasks.workunit.client.0.smithi102.stdout:9/652: unlink d1/d6/d45/d2d/f2e 0 2024-05-13T22:48:16.362 INFO:tasks.workunit.client.0.smithi102.stdout:9/653: fdatasync d1/d6/db/d30/d47/f6d 0 2024-05-13T22:48:16.366 INFO:tasks.workunit.client.0.smithi102.stdout:9/654: dread - d1/d6/d45/d3e/d79/fa6 zero size 2024-05-13T22:48:16.368 INFO:tasks.workunit.client.0.smithi102.stdout:4/724: truncate d3/d19/d23/f37 631402 0 2024-05-13T22:48:16.368 INFO:tasks.workunit.client.0.smithi102.stdout:4/725: dread - d3/d6b/fdf zero size 2024-05-13T22:48:16.369 INFO:tasks.workunit.client.0.smithi102.stdout:4/726: stat d3/d4/d6/f1c 0 2024-05-13T22:48:16.374 INFO:tasks.workunit.client.0.smithi102.stdout:3/739: truncate fa 4337543 0 2024-05-13T22:48:16.375 INFO:tasks.workunit.client.0.smithi102.stdout:4/727: mknod d3/d19/d23/d4e/d68/cff 0 2024-05-13T22:48:16.376 INFO:tasks.workunit.client.0.smithi102.stdout:3/740: truncate de/d10/d2a/d33/fb8 1870554 0 2024-05-13T22:48:16.377 INFO:tasks.workunit.client.0.smithi102.stdout:3/741: chown de/d1c/d79/c6b 11512 1 2024-05-13T22:48:16.378 INFO:tasks.workunit.client.0.smithi102.stdout:9/655: dread d1/f7 [0,4194304] 0 2024-05-13T22:48:16.378 INFO:tasks.workunit.client.0.smithi102.stdout:3/742: read de/d10/d2a/d33/fcb [288642,34635] 0 2024-05-13T22:48:16.379 INFO:tasks.workunit.client.0.smithi102.stdout:4/728: truncate d3/d6b/fc6 818179 0 2024-05-13T22:48:16.379 INFO:tasks.workunit.client.0.smithi102.stdout:9/656: write d1/d6/db/d4a/d64/fd8 [80011,9784] 0 2024-05-13T22:48:16.386 INFO:tasks.workunit.client.0.smithi102.stdout:3/743: dread de/d10/d2a/d33/d61/f64 [0,4194304] 0 2024-05-13T22:48:16.391 INFO:tasks.workunit.client.0.smithi102.stdout:3/744: unlink de/d1c/f91 0 2024-05-13T22:48:16.399 INFO:tasks.workunit.client.0.smithi102.stdout:4/729: dread d3/d4/d6/de/f15 [0,4194304] 0 2024-05-13T22:48:16.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:16 smithi130 ceph-mon[36591]: pgmap v25: 65 pgs: 65 active+clean; 1.4 GiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 98 MiB/s wr, 313 op/s 2024-05-13T22:48:16.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:16 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:48:16.403 INFO:tasks.workunit.client.0.smithi102.stdout:4/730: dread d3/d19/d23/d4e/f72 [0,4194304] 0 2024-05-13T22:48:16.411 INFO:tasks.workunit.client.0.smithi102.stdout:3/745: dread de/d10/d50/fab [0,4194304] 0 2024-05-13T22:48:16.413 INFO:tasks.workunit.client.0.smithi102.stdout:3/746: truncate de/d10/d50/d63/f76 4858655 0 2024-05-13T22:48:16.419 INFO:tasks.workunit.client.0.smithi102.stdout:9/657: dwrite d1/d6/d45/f4f [4194304,4194304] 0 2024-05-13T22:48:16.422 INFO:tasks.workunit.client.0.smithi102.stdout:4/731: dread d3/d4/d6/de/faa [0,4194304] 0 2024-05-13T22:48:16.424 INFO:tasks.workunit.client.0.smithi102.stdout:3/747: dwrite de/d10/d50/fab [4194304,4194304] 0 2024-05-13T22:48:16.425 INFO:tasks.workunit.client.0.smithi102.stdout:9/658: unlink d1/d6/db/d30/c4d 0 2024-05-13T22:48:16.427 INFO:tasks.workunit.client.0.smithi102.stdout:4/732: rmdir d3/d19/d23/d39/da7/d7d/d95 39 2024-05-13T22:48:16.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:16 smithi102 ceph-mon[28599]: pgmap v25: 65 pgs: 65 active+clean; 1.4 GiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 98 MiB/s wr, 313 op/s 2024-05-13T22:48:16.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:16 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:48:16.438 INFO:tasks.workunit.client.0.smithi102.stdout:9/659: dwrite d1/d6/db/d4a/fd7 [0,4194304] 0 2024-05-13T22:48:16.438 INFO:tasks.workunit.client.0.smithi102.stdout:3/748: unlink de/d10/d2a/f2e 0 2024-05-13T22:48:16.443 INFO:tasks.workunit.client.0.smithi102.stdout:3/749: read - de/d68/f94 zero size 2024-05-13T22:48:16.446 INFO:tasks.workunit.client.0.smithi102.stdout:9/660: creat d1/d6/db/d4a/d75/d96/fe4 x:0 0 0 2024-05-13T22:48:16.448 INFO:tasks.workunit.client.0.smithi102.stdout:3/750: fdatasync de/d68/f9d 0 2024-05-13T22:48:16.449 INFO:tasks.workunit.client.0.smithi102.stdout:3/751: dread - de/d10/d2a/d36/f49 zero size 2024-05-13T22:48:16.455 INFO:tasks.workunit.client.0.smithi102.stdout:3/752: dread de/d10/d2a/d33/d61/f4c [0,4194304] 0 2024-05-13T22:48:16.455 INFO:tasks.workunit.client.0.smithi102.stdout:9/661: creat d1/d6/d45/fe5 x:0 0 0 2024-05-13T22:48:16.457 INFO:tasks.workunit.client.0.smithi102.stdout:7/521: sync 2024-05-13T22:48:16.459 INFO:tasks.workunit.client.0.smithi102.stdout:3/753: creat de/d10/d2a/d36/da0/ff8 x:0 0 0 2024-05-13T22:48:16.461 INFO:tasks.workunit.client.0.smithi102.stdout:9/662: chown d1/d6/d21/d71/d91/lbf 11339808 1 2024-05-13T22:48:16.467 INFO:tasks.workunit.client.0.smithi102.stdout:3/754: dread de/d10/d2a/f7e [0,4194304] 0 2024-05-13T22:48:16.470 INFO:tasks.workunit.client.0.smithi102.stdout:3/755: dread f2 [0,4194304] 0 2024-05-13T22:48:16.478 INFO:tasks.workunit.client.0.smithi102.stdout:3/756: creat de/d68/ff9 x:0 0 0 2024-05-13T22:48:16.481 INFO:tasks.workunit.client.0.smithi102.stdout:9/663: link d1/d6/d21/d71/d91/l9c d1/d6/db/d4a/d75/d96/dc6/dd9/le6 0 2024-05-13T22:48:16.484 INFO:tasks.workunit.client.0.smithi102.stdout:3/757: read de/d10/f44 [1978119,54286] 0 2024-05-13T22:48:16.486 INFO:tasks.workunit.client.0.smithi102.stdout:7/522: truncate d6/d15/f18 5437117 0 2024-05-13T22:48:16.486 INFO:tasks.workunit.client.0.smithi102.stdout:7/523: fdatasync d6/d22/f7b 0 2024-05-13T22:48:16.490 INFO:tasks.workunit.client.0.smithi102.stdout:7/524: creat d6/d15/d4d/f93 x:0 0 0 2024-05-13T22:48:16.495 INFO:tasks.workunit.client.0.smithi102.stdout:9/664: rmdir d1/d6/db/d30/d8a/dce 0 2024-05-13T22:48:16.495 INFO:tasks.workunit.client.0.smithi102.stdout:3/758: creat de/d68/df3/ffa x:0 0 0 2024-05-13T22:48:16.496 INFO:tasks.workunit.client.0.smithi102.stdout:9/665: readlink d1/l2 0 2024-05-13T22:48:16.525 INFO:tasks.workunit.client.0.smithi102.stdout:9/666: creat d1/d6/db/d30/fe7 x:0 0 0 2024-05-13T22:48:16.526 INFO:tasks.workunit.client.0.smithi102.stdout:3/759: mknod de/d1c/cfb 0 2024-05-13T22:48:16.527 INFO:tasks.workunit.client.0.smithi102.stdout:9/667: dread - d1/d6/d45/d2d/dac/fb4 zero size 2024-05-13T22:48:16.529 INFO:tasks.workunit.client.0.smithi102.stdout:3/760: creat de/d10/d50/d8f/ffc x:0 0 0 2024-05-13T22:48:16.530 INFO:tasks.workunit.client.0.smithi102.stdout:3/761: chown de/d10/d2a/d36/db2/lb5 429 1 2024-05-13T22:48:16.533 INFO:tasks.workunit.client.0.smithi102.stdout:7/525: link d6/d15/d4c/d90/d4b/f6c d6/f94 0 2024-05-13T22:48:16.539 INFO:tasks.workunit.client.0.smithi102.stdout:7/526: mkdir d6/d15/d4d/d55/d6f/d49/d95 0 2024-05-13T22:48:16.540 INFO:tasks.workunit.client.0.smithi102.stdout:7/527: dread - d6/d22/f7b zero size 2024-05-13T22:48:16.546 INFO:tasks.workunit.client.0.smithi102.stdout:3/762: link de/d10/d2a/d36/db2/fbb de/d67/df7/ffd 0 2024-05-13T22:48:16.546 INFO:tasks.workunit.client.0.smithi102.stdout:3/763: write de/d10/d34/f93 [11625,35216] 0 2024-05-13T22:48:16.548 INFO:tasks.workunit.client.0.smithi102.stdout:3/764: read de/d10/d50/fc6 [2087049,67994] 0 2024-05-13T22:48:16.585 INFO:tasks.workunit.client.0.smithi102.stdout:9/668: write d1/d6/d3b/d5b/f62 [2122275,38603] 0 2024-05-13T22:48:16.593 INFO:tasks.workunit.client.0.smithi102.stdout:9/669: dread d1/d6/db/f7a [0,4194304] 0 2024-05-13T22:48:16.598 INFO:tasks.workunit.client.0.smithi102.stdout:3/765: write de/d1c/d79/f81 [402300,11772] 0 2024-05-13T22:48:16.599 INFO:tasks.workunit.client.0.smithi102.stdout:3/766: write de/d67/fd7 [27982,51692] 0 2024-05-13T22:48:16.600 INFO:tasks.workunit.client.0.smithi102.stdout:3/767: chown de/d10 1 1 2024-05-13T22:48:16.603 INFO:tasks.workunit.client.0.smithi102.stdout:3/768: symlink de/d10/d2a/d36/db2/lfe 0 2024-05-13T22:48:16.609 INFO:tasks.workunit.client.0.smithi102.stdout:3/769: getdents de/d10/d2a/d33/d61 0 2024-05-13T22:48:16.611 INFO:tasks.workunit.client.0.smithi102.stdout:3/770: mknod de/d10/d2a/cff 0 2024-05-13T22:48:16.614 INFO:tasks.workunit.client.0.smithi102.stdout:3/771: rename de/d10/d2a/d36/da0 to de/d10/d46/d100 0 2024-05-13T22:48:16.621 INFO:tasks.workunit.client.0.smithi102.stdout:3/772: dwrite de/d67/fd7 [0,4194304] 0 2024-05-13T22:48:16.623 INFO:tasks.workunit.client.0.smithi102.stdout:3/773: chown de/d68/df3 15 1 2024-05-13T22:48:16.627 INFO:tasks.workunit.client.0.smithi102.stdout:9/670: dwrite d1/f31 [4194304,4194304] 0 2024-05-13T22:48:16.628 INFO:tasks.workunit.client.0.smithi102.stdout:3/774: truncate de/d10/d2a/d33/f4d 4888718 0 2024-05-13T22:48:16.636 INFO:tasks.workunit.client.0.smithi102.stdout:7/528: dread d6/fc [4194304,4194304] 0 2024-05-13T22:48:16.636 INFO:tasks.workunit.client.0.smithi102.stdout:3/775: dwrite de/d67/de1/ff5 [0,4194304] 0 2024-05-13T22:48:16.638 INFO:tasks.workunit.client.0.smithi102.stdout:7/529: write d6/d15/f3e [1843689,102656] 0 2024-05-13T22:48:16.638 INFO:tasks.workunit.client.0.smithi102.stdout:3/776: dread - de/d67/ff4 zero size 2024-05-13T22:48:16.641 INFO:tasks.workunit.client.0.smithi102.stdout:3/777: mknod de/d10/d2a/d33/d61/c101 0 2024-05-13T22:48:16.642 INFO:tasks.workunit.client.0.smithi102.stdout:3/778: fsync de/d10/d50/d63/da2/fdd 0 2024-05-13T22:48:16.644 INFO:tasks.workunit.client.0.smithi102.stdout:7/530: rename d6/d15/f7a to d6/d15/d4c/d90/d80/f96 0 2024-05-13T22:48:16.646 INFO:tasks.workunit.client.0.smithi102.stdout:3/779: creat de/d10/d46/d100/f102 x:0 0 0 2024-05-13T22:48:16.647 INFO:tasks.workunit.client.0.smithi102.stdout:3/780: dread - de/d10/d2a/d36/f49 zero size 2024-05-13T22:48:16.651 INFO:tasks.workunit.client.0.smithi102.stdout:7/531: dread d6/d15/d4c/d90/d4b/f6c [0,4194304] 0 2024-05-13T22:48:16.652 INFO:tasks.workunit.client.0.smithi102.stdout:7/532: truncate d6/d15/d4d/d55/d6f/d65/f86 867482 0 2024-05-13T22:48:16.652 INFO:tasks.workunit.client.0.smithi102.stdout:7/533: stat d6/d13/d2c/d39 0 2024-05-13T22:48:16.683 INFO:tasks.workunit.client.0.smithi102.stdout:6/752: sync 2024-05-13T22:48:16.683 INFO:tasks.workunit.client.0.smithi102.stdout:0/689: sync 2024-05-13T22:48:16.684 INFO:tasks.workunit.client.0.smithi102.stdout:1/550: sync 2024-05-13T22:48:16.686 INFO:tasks.workunit.client.0.smithi102.stdout:7/534: mkdir d6/d15/d4d/d55/d6f/d49/d97 0 2024-05-13T22:48:16.687 INFO:tasks.workunit.client.0.smithi102.stdout:7/535: dread - d6/d15/d4d/d55/d6f/f92 zero size 2024-05-13T22:48:16.689 INFO:tasks.workunit.client.0.smithi102.stdout:0/690: symlink d1/d77/ld5 0 2024-05-13T22:48:16.690 INFO:tasks.workunit.client.0.smithi102.stdout:9/671: write d1/d5d/d6c/f86 [1561364,30571] 0 2024-05-13T22:48:16.693 INFO:tasks.workunit.client.0.smithi102.stdout:6/753: dread f0 [0,4194304] 0 2024-05-13T22:48:16.694 INFO:tasks.workunit.client.0.smithi102.stdout:6/754: write d9/d25/d71/fb3 [844882,55185] 0 2024-05-13T22:48:16.696 INFO:tasks.workunit.client.0.smithi102.stdout:7/536: rename d6/d22/f43 to d6/d15/d4c/f98 0 2024-05-13T22:48:16.697 INFO:tasks.workunit.client.0.smithi102.stdout:0/691: creat d1/de/d32/d8a/fd6 x:0 0 0 2024-05-13T22:48:16.698 INFO:tasks.workunit.client.0.smithi102.stdout:0/692: fdatasync d1/de/d32/dab/fb0 0 2024-05-13T22:48:16.704 INFO:tasks.workunit.client.0.smithi102.stdout:7/537: mknod d6/d15/d4d/d55/c99 0 2024-05-13T22:48:16.704 INFO:tasks.workunit.client.0.smithi102.stdout:0/693: dwrite d1/de/d32/d24/d2d/d38/d3d/fcf [0,4194304] 0 2024-05-13T22:48:16.707 INFO:tasks.workunit.client.0.smithi102.stdout:7/538: dread d6/d15/d4d/d55/d6f/d49/f71 [0,4194304] 0 2024-05-13T22:48:16.708 INFO:tasks.workunit.client.0.smithi102.stdout:1/551: write d2/d10/d62/d7b/f7c [1622896,78087] 0 2024-05-13T22:48:16.709 INFO:tasks.workunit.client.0.smithi102.stdout:9/672: creat d1/d6/db/d4a/d81/fe8 x:0 0 0 2024-05-13T22:48:16.711 INFO:tasks.workunit.client.0.smithi102.stdout:9/673: chown d1/d6/d21/l3c 615904 1 2024-05-13T22:48:16.712 INFO:tasks.workunit.client.0.smithi102.stdout:0/694: truncate d1/de/d42/d61/fb5 780057 0 2024-05-13T22:48:16.715 INFO:tasks.workunit.client.0.smithi102.stdout:1/552: write d2/d4/d5/d19/f53 [4266520,104121] 0 2024-05-13T22:48:16.717 INFO:tasks.workunit.client.0.smithi102.stdout:7/539: dwrite d6/d15/d4d/d55/d6f/d65/f86 [0,4194304] 0 2024-05-13T22:48:16.718 INFO:tasks.workunit.client.0.smithi102.stdout:9/674: creat d1/d6/d45/da1/fe9 x:0 0 0 2024-05-13T22:48:16.719 INFO:tasks.workunit.client.0.smithi102.stdout:7/540: fsync d6/d15/d4d/d55/d6f/f92 0 2024-05-13T22:48:16.726 INFO:tasks.workunit.client.0.smithi102.stdout:9/675: creat d1/d6/d45/d2d/dac/db0/fea x:0 0 0 2024-05-13T22:48:16.732 INFO:tasks.workunit.client.0.smithi102.stdout:9/676: creat d1/d6/db/d4a/d75/d96/dc6/feb x:0 0 0 2024-05-13T22:48:16.733 INFO:tasks.workunit.client.0.smithi102.stdout:7/541: dread d6/d13/d58/f26 [0,4194304] 0 2024-05-13T22:48:16.734 INFO:tasks.workunit.client.0.smithi102.stdout:1/553: write d2/d45/f5a [1283384,130553] 0 2024-05-13T22:48:16.735 INFO:tasks.workunit.client.0.smithi102.stdout:1/554: chown d2/d4/d26 332 1 2024-05-13T22:48:16.741 INFO:tasks.workunit.client.0.smithi102.stdout:0/695: write d1/de/d42/d61/fb5 [1246937,46178] 0 2024-05-13T22:48:16.742 INFO:tasks.workunit.client.0.smithi102.stdout:0/696: dread - d1/de/d32/d8a/fd6 zero size 2024-05-13T22:48:16.744 INFO:tasks.workunit.client.0.smithi102.stdout:1/555: mknod d2/d10/cb1 0 2024-05-13T22:48:16.744 INFO:tasks.workunit.client.0.smithi102.stdout:0/697: fdatasync d1/d4d/d79/d9f/dce/fca 0 2024-05-13T22:48:16.745 INFO:tasks.workunit.client.0.smithi102.stdout:7/542: creat d6/d15/d4d/d55/d6f/f9a x:0 0 0 2024-05-13T22:48:16.747 INFO:tasks.workunit.client.0.smithi102.stdout:9/677: dwrite d1/d6/f4e [0,4194304] 0 2024-05-13T22:48:16.749 INFO:tasks.workunit.client.0.smithi102.stdout:9/678: dread - d1/d6/d45/d3e/d79/fa6 zero size 2024-05-13T22:48:16.749 INFO:tasks.workunit.client.0.smithi102.stdout:1/556: truncate d2/d10/f44 2230712 0 2024-05-13T22:48:16.751 INFO:tasks.workunit.client.0.smithi102.stdout:0/698: creat d1/de/d32/d24/d2d/d91/fd7 x:0 0 0 2024-05-13T22:48:16.753 INFO:tasks.workunit.client.0.smithi102.stdout:1/557: mkdir d2/d4/d38/db2 0 2024-05-13T22:48:16.754 INFO:tasks.workunit.client.0.smithi102.stdout:0/699: chown d1/de/d32/d63/f98 1 1 2024-05-13T22:48:16.755 INFO:tasks.workunit.client.0.smithi102.stdout:7/543: dwrite d6/d15/d4d/f93 [0,4194304] 0 2024-05-13T22:48:16.777 INFO:tasks.workunit.client.0.smithi102.stdout:0/700: dread d1/f2c [0,4194304] 0 2024-05-13T22:48:16.799 INFO:tasks.workunit.client.0.smithi102.stdout:9/679: symlink d1/d6/db/d4a/d64/lec 0 2024-05-13T22:48:16.813 INFO:tasks.workunit.client.0.smithi102.stdout:8/651: sync 2024-05-13T22:48:16.814 INFO:tasks.workunit.client.0.smithi102.stdout:2/738: sync 2024-05-13T22:48:16.816 INFO:tasks.workunit.client.0.smithi102.stdout:0/701: truncate d1/de/d32/f18 2987458 0 2024-05-13T22:48:16.817 INFO:tasks.workunit.client.0.smithi102.stdout:0/702: chown d1/cd2 221988757 1 2024-05-13T22:48:16.817 INFO:tasks.workunit.client.0.smithi102.stdout:0/703: stat d1/lc7 0 2024-05-13T22:48:16.821 INFO:tasks.workunit.client.0.smithi102.stdout:8/652: unlink d2/d36/f5a 0 2024-05-13T22:48:16.822 INFO:tasks.workunit.client.0.smithi102.stdout:2/739: mkdir d7/d15/d1e/da7/df4 0 2024-05-13T22:48:16.825 INFO:tasks.workunit.client.0.smithi102.stdout:8/653: symlink d2/d7/d27/dad/lcb 0 2024-05-13T22:48:16.826 INFO:tasks.workunit.client.0.smithi102.stdout:2/740: rename d7/d9/d10/d30/d7f/c91 to d7/d15/d1e/d87/cf5 0 2024-05-13T22:48:16.922 INFO:tasks.workunit.client.0.smithi102.stdout:2/741: rmdir d7/d9/d10/d1b/d54 39 2024-05-13T22:48:16.925 INFO:tasks.workunit.client.0.smithi102.stdout:2/742: chown c2 64290411 1 2024-05-13T22:48:16.930 INFO:tasks.workunit.client.0.smithi102.stdout:1/558: dwrite d2/d4/d5/d19/d28/f35 [0,4194304] 0 2024-05-13T22:48:16.930 INFO:tasks.workunit.client.0.smithi102.stdout:9/680: dwrite d1/d6/d45/d2d/f3f [0,4194304] 0 2024-05-13T22:48:16.930 INFO:tasks.workunit.client.0.smithi102.stdout:8/654: link d2/d7/d27/d64/d96/dae/f4f d2/d7/d27/dbb/fcc 0 2024-05-13T22:48:16.932 INFO:tasks.workunit.client.0.smithi102.stdout:1/559: chown d2/d4/d26/l2d 5765 1 2024-05-13T22:48:16.933 INFO:tasks.workunit.client.0.smithi102.stdout:9/681: write d1/d6/d45/d3e/d79/fa6 [772917,119946] 0 2024-05-13T22:48:16.937 INFO:tasks.workunit.client.0.smithi102.stdout:1/560: rmdir d2/d10/d62/d86 39 2024-05-13T22:48:16.937 INFO:tasks.workunit.client.0.smithi102.stdout:2/743: dwrite d7/d9/d10/d30/f80 [0,4194304] 0 2024-05-13T22:48:16.939 INFO:tasks.workunit.client.0.smithi102.stdout:9/682: mknod d1/d6/d21/d71/ced 0 2024-05-13T22:48:16.942 INFO:tasks.workunit.client.0.smithi102.stdout:2/744: rename d7/d15/f52 to d7/d15/d1e/da7/db5/ff6 0 2024-05-13T22:48:16.943 INFO:tasks.workunit.client.0.smithi102.stdout:9/683: mknod d1/d6/db/d4a/d75/cee 0 2024-05-13T22:48:16.943 INFO:tasks.workunit.client.0.smithi102.stdout:9/684: read - d1/fdd zero size 2024-05-13T22:48:16.952 INFO:tasks.workunit.client.0.smithi102.stdout:5/810: sync 2024-05-13T22:48:16.953 INFO:tasks.workunit.client.0.smithi102.stdout:5/811: dread - de/d25/d2a/def/f124 zero size 2024-05-13T22:48:16.972 INFO:tasks.workunit.client.0.smithi102.stdout:9/685: dread d1/d6/db/f1b [0,4194304] 0 2024-05-13T22:48:16.978 INFO:tasks.workunit.client.0.smithi102.stdout:9/686: rename d1/d6b/cbc to d1/d6/db/d77/cef 0 2024-05-13T22:48:16.983 INFO:tasks.workunit.client.0.smithi102.stdout:9/687: rename d1/d6/db/f83 to d1/d6/db/d4a/d75/db9/ff0 0 2024-05-13T22:48:17.014 INFO:tasks.workunit.client.0.smithi102.stdout:8/655: dwrite d2/d7/d27/f3d [4194304,4194304] 0 2024-05-13T22:48:17.016 INFO:tasks.workunit.client.0.smithi102.stdout:1/561: write d2/d4/f23 [1947157,114241] 0 2024-05-13T22:48:17.017 INFO:tasks.workunit.client.0.smithi102.stdout:1/562: chown d2 0 1 2024-05-13T22:48:17.017 INFO:tasks.workunit.client.0.smithi102.stdout:8/656: rmdir d2/d7 39 2024-05-13T22:48:17.018 INFO:tasks.workunit.client.0.smithi102.stdout:1/563: chown d2/d10 0 1 2024-05-13T22:48:17.024 INFO:tasks.workunit.client.0.smithi102.stdout:1/564: symlink d2/d45/d96/lb3 0 2024-05-13T22:48:17.025 INFO:tasks.workunit.client.0.smithi102.stdout:5/812: dwrite de/d33/d53/f57 [0,4194304] 0 2024-05-13T22:48:17.027 INFO:tasks.workunit.client.0.smithi102.stdout:8/657: mknod d2/d7/d27/d64/d96/dae/d76/ccd 0 2024-05-13T22:48:17.032 INFO:tasks.workunit.client.0.smithi102.stdout:5/813: rename de/l37 to de/d25/d58/dfb/l129 0 2024-05-13T22:48:17.033 INFO:tasks.workunit.client.0.smithi102.stdout:9/688: write d1/d6/d45/d3e/f4c [3673896,76942] 0 2024-05-13T22:48:17.036 INFO:tasks.workunit.client.0.smithi102.stdout:1/565: creat d2/d4/d5/fb4 x:0 0 0 2024-05-13T22:48:17.037 INFO:tasks.workunit.client.0.smithi102.stdout:1/566: readlink d2/d4/d26/d40/l97 0 2024-05-13T22:48:17.039 INFO:tasks.workunit.client.0.smithi102.stdout:1/567: rmdir d2 39 2024-05-13T22:48:17.043 INFO:tasks.workunit.client.0.smithi102.stdout:5/814: rename de/d25/d59/d6f/f72 to de/d25/d2a/d4b/d4d/d50/f12a 0 2024-05-13T22:48:17.047 INFO:tasks.workunit.client.0.smithi102.stdout:9/689: rename d1/d6/d45/d3e/l41 to d1/d6/db/d4a/d81/dd6/lf1 0 2024-05-13T22:48:17.050 INFO:tasks.workunit.client.0.smithi102.stdout:5/815: dread de/d25/d43/d7a/fae [0,4194304] 0 2024-05-13T22:48:17.052 INFO:tasks.workunit.client.0.smithi102.stdout:1/568: creat d2/d4/fb5 x:0 0 0 2024-05-13T22:48:17.053 INFO:tasks.workunit.client.0.smithi102.stdout:1/569: stat d2/d45/d96/lb3 0 2024-05-13T22:48:17.055 INFO:tasks.workunit.client.0.smithi102.stdout:9/690: mknod d1/d6/d45/cf2 0 2024-05-13T22:48:17.055 INFO:tasks.workunit.client.0.smithi102.stdout:5/816: dread de/d25/d59/fb4 [0,4194304] 0 2024-05-13T22:48:17.061 INFO:tasks.workunit.client.0.smithi102.stdout:5/817: mkdir de/d25/d108/dc5/dca/d12b 0 2024-05-13T22:48:17.062 INFO:tasks.workunit.client.0.smithi102.stdout:5/818: write de/d25/d2a/d4b/fc1 [2706591,97169] 0 2024-05-13T22:48:17.062 INFO:tasks.workunit.client.0.smithi102.stdout:9/691: creat d1/d6/db/d4a/d64/dc0/ff3 x:0 0 0 2024-05-13T22:48:17.064 INFO:tasks.workunit.client.0.smithi102.stdout:8/658: truncate d2/d36/d49/d5d/f26 1426656 0 2024-05-13T22:48:17.066 INFO:tasks.workunit.client.0.smithi102.stdout:1/570: readlink d2/d4/d5/d19/d28/l3c 0 2024-05-13T22:48:17.068 INFO:tasks.workunit.client.0.smithi102.stdout:5/819: dread - de/d25/d43/d7a/f88 zero size 2024-05-13T22:48:17.072 INFO:tasks.workunit.client.0.smithi102.stdout:9/692: dwrite d1/d6/d3b/d5b/f62 [4194304,4194304] 0 2024-05-13T22:48:17.080 INFO:tasks.workunit.client.0.smithi102.stdout:4/733: sync 2024-05-13T22:48:17.085 INFO:tasks.workunit.client.0.smithi102.stdout:8/659: dread d2/d7/d25/d41/f48 [0,4194304] 0 2024-05-13T22:48:17.085 INFO:tasks.workunit.client.0.smithi102.stdout:4/734: mknod d3/c100 0 2024-05-13T22:48:17.089 INFO:tasks.workunit.client.0.smithi102.stdout:8/660: symlink d2/d7/d25/d41/lce 0 2024-05-13T22:48:17.090 INFO:tasks.workunit.client.0.smithi102.stdout:4/735: creat d3/d4/d6/de/d1f/f101 x:0 0 0 2024-05-13T22:48:17.094 INFO:tasks.workunit.client.0.smithi102.stdout:4/736: mkdir d3/d19/d23/d39/d5d/da5/d102 0 2024-05-13T22:48:17.095 INFO:tasks.workunit.client.0.smithi102.stdout:8/661: dwrite d2/d7/d27/f4c [0,4194304] 0 2024-05-13T22:48:17.097 INFO:tasks.workunit.client.0.smithi102.stdout:1/571: truncate d2/d4/d26/f33 428239 0 2024-05-13T22:48:17.099 INFO:tasks.workunit.client.0.smithi102.stdout:9/693: write d1/d6/d45/d2d/dac/db0/db8/fc7 [412202,4582] 0 2024-05-13T22:48:17.105 INFO:tasks.workunit.client.0.smithi102.stdout:5/820: dwrite de/d25/d59/d7f/d86/da5/ddb/dee/ff9 [0,4194304] 0 2024-05-13T22:48:17.106 INFO:tasks.workunit.client.0.smithi102.stdout:4/737: mkdir d3/d19/d23/d39/da7/d7d/d95/d103 0 2024-05-13T22:48:17.107 INFO:tasks.workunit.client.0.smithi102.stdout:4/738: chown d3/d4/d6/de/f15 1 1 2024-05-13T22:48:17.111 INFO:tasks.workunit.client.0.smithi102.stdout:9/694: read d1/d6/d45/d3e/f93 [3571377,101022] 0 2024-05-13T22:48:17.115 INFO:tasks.workunit.client.0.smithi102.stdout:1/572: unlink d2/d4/d5/d19/d28/l5b 0 2024-05-13T22:48:17.115 INFO:tasks.workunit.client.0.smithi102.stdout:5/821: dwrite de/d25/d59/d7f/d86/da5/ddb/dee/f115 [0,4194304] 0 2024-05-13T22:48:17.118 INFO:tasks.workunit.client.0.smithi102.stdout:8/662: dread d2/d36/d49/d5d/fbd [0,4194304] 0 2024-05-13T22:48:17.125 INFO:tasks.workunit.client.0.smithi102.stdout:5/822: mknod de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d11d/c12c 0 2024-05-13T22:48:17.126 INFO:tasks.workunit.client.0.smithi102.stdout:5/823: read - de/d33/d53/f11a zero size 2024-05-13T22:48:17.131 INFO:tasks.workunit.client.0.smithi102.stdout:5/824: readlink de/d25/d59/d7f/d86/da5/le7 0 2024-05-13T22:48:17.131 INFO:tasks.workunit.client.0.smithi102.stdout:8/663: dread d2/d7/d27/d64/d96/dae/f23 [4194304,4194304] 0 2024-05-13T22:48:17.132 INFO:tasks.workunit.client.0.smithi102.stdout:5/825: chown de/d25/d58/dfb/f126 769600804 1 2024-05-13T22:48:17.135 INFO:tasks.workunit.client.0.smithi102.stdout:8/664: mknod d2/d7/d27/dbb/ccf 0 2024-05-13T22:48:17.137 INFO:tasks.workunit.client.0.smithi102.stdout:5/826: read de/d25/d2a/d4b/f76 [1239157,94783] 0 2024-05-13T22:48:17.138 INFO:tasks.workunit.client.0.smithi102.stdout:9/695: link d1/d6/d45/dd0/cdf d1/d6/db/d4a/d64/cf4 0 2024-05-13T22:48:17.139 INFO:tasks.workunit.client.0.smithi102.stdout:4/739: dread d3/d19/dad/faf [0,4194304] 0 2024-05-13T22:48:17.139 INFO:tasks.workunit.client.0.smithi102.stdout:3/781: sync 2024-05-13T22:48:17.139 INFO:tasks.workunit.client.0.smithi102.stdout:6/755: sync 2024-05-13T22:48:17.141 INFO:tasks.workunit.client.0.smithi102.stdout:8/665: getdents d2/d7/d27/dbb/dc7 0 2024-05-13T22:48:17.141 INFO:tasks.workunit.client.0.smithi102.stdout:9/696: rmdir d1/d6/d21 39 2024-05-13T22:48:17.143 INFO:tasks.workunit.client.0.smithi102.stdout:3/782: creat de/d10/d46/dc8/f103 x:0 0 0 2024-05-13T22:48:17.144 INFO:tasks.workunit.client.0.smithi102.stdout:9/697: write d1/d6/db/d4a/d81/fe8 [933444,120158] 0 2024-05-13T22:48:17.149 INFO:tasks.workunit.client.0.smithi102.stdout:1/573: dread d2/d4/d26/f9a [0,4194304] 0 2024-05-13T22:48:17.149 INFO:tasks.workunit.client.0.smithi102.stdout:8/666: rename d2/d7/l8 to d2/d36/d49/d7e/ld0 0 2024-05-13T22:48:17.151 INFO:tasks.workunit.client.0.smithi102.stdout:3/783: rmdir de/d68 39 2024-05-13T22:48:17.152 INFO:tasks.workunit.client.0.smithi102.stdout:9/698: dwrite d1/d6/db/d77/f7b [0,4194304] 0 2024-05-13T22:48:17.157 INFO:tasks.workunit.client.0.smithi102.stdout:4/740: creat d3/d19/d23/f104 x:0 0 0 2024-05-13T22:48:17.160 INFO:tasks.workunit.client.0.smithi102.stdout:8/667: mknod d2/d7/d27/d64/d80/dc8/d93/cd1 0 2024-05-13T22:48:17.162 INFO:tasks.workunit.client.0.smithi102.stdout:5/827: write de/d25/d2a/d4b/d4d/d50/d6c/df8/fcd [438891,66106] 0 2024-05-13T22:48:17.164 INFO:tasks.workunit.client.0.smithi102.stdout:9/699: dread d1/d6/db/d30/d8a/faf [0,4194304] 0 2024-05-13T22:48:17.164 INFO:tasks.workunit.client.0.smithi102.stdout:5/828: stat de/fc8 0 2024-05-13T22:48:17.164 INFO:tasks.workunit.client.0.smithi102.stdout:4/741: write d3/d19/dad/faf [4722619,18066] 0 2024-05-13T22:48:17.173 INFO:tasks.workunit.client.0.smithi102.stdout:8/668: dwrite d2/d36/d49/d5d/f33 [0,4194304] 0 2024-05-13T22:48:17.175 INFO:tasks.workunit.client.0.smithi102.stdout:9/700: mknod d1/d6/d21/cf5 0 2024-05-13T22:48:17.176 INFO:tasks.workunit.client.0.smithi102.stdout:6/756: dwrite d9/db/d14/dcc/d5f/f9e [0,4194304] 0 2024-05-13T22:48:17.176 INFO:tasks.workunit.client.0.smithi102.stdout:5/829: rename de/d33 to de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d 0 2024-05-13T22:48:17.182 INFO:tasks.workunit.client.0.smithi102.stdout:8/669: read d2/d36/f42 [2145039,92515] 0 2024-05-13T22:48:17.184 INFO:tasks.workunit.client.0.smithi102.stdout:6/757: dread - d9/d25/d71/f7a zero size 2024-05-13T22:48:17.185 INFO:tasks.workunit.client.0.smithi102.stdout:4/742: dwrite d3/d4/d6/fb5 [0,4194304] 0 2024-05-13T22:48:17.185 INFO:tasks.workunit.client.0.smithi102.stdout:8/670: chown d2/d7/d27/d64/d96/dae 25629907 1 2024-05-13T22:48:17.186 INFO:tasks.workunit.client.0.smithi102.stdout:3/784: write de/d10/d2a/fec [25104,18177] 0 2024-05-13T22:48:17.188 INFO:tasks.workunit.client.0.smithi102.stdout:4/743: write d3/d19/dad/faf [649845,109125] 0 2024-05-13T22:48:17.190 INFO:tasks.workunit.client.0.smithi102.stdout:4/744: readlink d3/d4/lb0 0 2024-05-13T22:48:17.195 INFO:tasks.workunit.client.0.smithi102.stdout:8/671: mkdir d2/d7/d27/d64/d80/dc8/d40/d85/dd2 0 2024-05-13T22:48:17.196 INFO:tasks.workunit.client.0.smithi102.stdout:3/785: creat de/d68/f104 x:0 0 0 2024-05-13T22:48:17.197 INFO:tasks.workunit.client.0.smithi102.stdout:4/745: write d3/d19/d23/d39/da7/d7d/fcb [289777,129449] 0 2024-05-13T22:48:17.198 INFO:tasks.workunit.client.0.smithi102.stdout:4/746: write d3/d19/d23/d39/fe7 [54881,43648] 0 2024-05-13T22:48:17.200 INFO:tasks.workunit.client.0.smithi102.stdout:8/672: creat d2/d36/fd3 x:0 0 0 2024-05-13T22:48:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:4/747: mknod d3/d19/d23/d39/da7/da9/c105 0 2024-05-13T22:48:17.202 INFO:tasks.workunit.client.0.smithi102.stdout:4/748: readlink d3/ldc 0 2024-05-13T22:48:17.205 INFO:tasks.workunit.client.0.smithi102.stdout:3/786: rename de/d10/d2a/d36/db2/de7 to de/d67/d105 0 2024-05-13T22:48:17.210 INFO:tasks.workunit.client.0.smithi102.stdout:4/749: dwrite d3/d19/dad/fbc [0,4194304] 0 2024-05-13T22:48:17.211 INFO:tasks.workunit.client.0.smithi102.stdout:8/673: rename d2/d7/d27/d64/d80/dc8/l6b to d2/d7/d27/d64/d96/dae/d78/ld4 0 2024-05-13T22:48:17.215 INFO:tasks.workunit.client.0.smithi102.stdout:9/701: truncate d1/d6/d3b/d5b/f62 3059029 0 2024-05-13T22:48:17.215 INFO:tasks.workunit.client.0.smithi102.stdout:5/830: dread de/d25/d58/fdd [0,4194304] 0 2024-05-13T22:48:17.218 INFO:tasks.workunit.client.0.smithi102.stdout:8/674: mknod d2/d7/d27/d64/d96/dae/d78/cd5 0 2024-05-13T22:48:17.219 INFO:tasks.workunit.client.0.smithi102.stdout:6/758: dwrite d9/d45/d56/f9b [4194304,4194304] 0 2024-05-13T22:48:17.227 INFO:tasks.workunit.client.0.smithi102.stdout:5/831: truncate de/d25/d58/fd6 1374892 0 2024-05-13T22:48:17.227 INFO:tasks.workunit.client.0.smithi102.stdout:6/759: dwrite d9/db/d14/dcc/d5f/fdb [0,4194304] 0 2024-05-13T22:48:17.229 INFO:tasks.workunit.client.0.smithi102.stdout:6/760: write d9/f8b [1701227,59520] 0 2024-05-13T22:48:17.237 INFO:tasks.workunit.client.0.smithi102.stdout:3/787: write de/d68/f9d [409423,19044] 0 2024-05-13T22:48:17.238 INFO:tasks.workunit.client.0.smithi102.stdout:5/832: creat de/d25/d108/dc5/dca/f12e x:0 0 0 2024-05-13T22:48:17.245 INFO:tasks.workunit.client.0.smithi102.stdout:6/761: rename d9/d25/d60/d6a/dad to d9/d25/d60/d6a/dd6/de9/df7 0 2024-05-13T22:48:17.245 INFO:tasks.workunit.client.0.smithi102.stdout:3/788: dwrite de/d10/d50/d63/da2/fea [0,4194304] 0 2024-05-13T22:48:17.246 INFO:tasks.workunit.client.0.smithi102.stdout:3/789: dread - de/d68/df3/ffa zero size 2024-05-13T22:48:17.248 INFO:tasks.workunit.client.0.smithi102.stdout:9/702: getdents d1/d6/d45/da1 0 2024-05-13T22:48:17.248 INFO:tasks.workunit.client.0.smithi102.stdout:3/790: chown de/d67/d105 125 1 2024-05-13T22:48:17.250 INFO:tasks.workunit.client.0.smithi102.stdout:5/833: creat de/d25/d58/dfb/f12f x:0 0 0 2024-05-13T22:48:17.253 INFO:tasks.workunit.client.0.smithi102.stdout:6/762: truncate d9/d25/d60/f7e 407486 0 2024-05-13T22:48:17.253 INFO:tasks.workunit.client.0.smithi102.stdout:8/675: write d2/d36/d49/d5d/f9e [44873,10060] 0 2024-05-13T22:48:17.256 INFO:tasks.workunit.client.0.smithi102.stdout:3/791: creat de/d67/de1/f106 x:0 0 0 2024-05-13T22:48:17.261 INFO:tasks.workunit.client.0.smithi102.stdout:8/676: creat d2/d7/d27/d64/d80/fd6 x:0 0 0 2024-05-13T22:48:17.262 INFO:tasks.workunit.client.0.smithi102.stdout:8/677: stat d2/d7/d27/dad/lcb 0 2024-05-13T22:48:17.263 INFO:tasks.workunit.client.0.smithi102.stdout:5/834: creat de/d25/d108/d104/f130 x:0 0 0 2024-05-13T22:48:17.264 INFO:tasks.workunit.client.0.smithi102.stdout:8/678: write d2/d7/d27/d64/d80/dc8/f39 [2464809,7035] 0 2024-05-13T22:48:17.265 INFO:tasks.workunit.client.0.smithi102.stdout:3/792: rename de/d10/d46/d100/ff8 to de/d10/d2a/d36/db2/de9/f107 0 2024-05-13T22:48:17.268 INFO:tasks.workunit.client.0.smithi102.stdout:6/763: getdents d9/db/d14/d35/dd1 0 2024-05-13T22:48:17.272 INFO:tasks.workunit.client.0.smithi102.stdout:9/703: write d1/d6/db/d30/f36 [4697766,127594] 0 2024-05-13T22:48:17.273 INFO:tasks.workunit.client.0.smithi102.stdout:3/793: dwrite de/d10/d34/f9a [0,4194304] 0 2024-05-13T22:48:17.279 INFO:tasks.workunit.client.0.smithi102.stdout:8/679: truncate d2/d7/d27/d64/d80/dc8/d40/db2/fba 3867230 0 2024-05-13T22:48:17.281 INFO:tasks.workunit.client.0.smithi102.stdout:6/764: fsync d9/db/d14/dcc/fb4 0 2024-05-13T22:48:17.284 INFO:tasks.workunit.client.0.smithi102.stdout:9/704: creat d1/d6/d3b/ff6 x:0 0 0 2024-05-13T22:48:17.286 INFO:tasks.workunit.client.0.smithi102.stdout:5/835: fsync f2 0 2024-05-13T22:48:17.292 INFO:tasks.workunit.client.0.smithi102.stdout:6/765: creat d9/db/d14/d35/dc9/ff8 x:0 0 0 2024-05-13T22:48:17.301 INFO:tasks.workunit.client.0.smithi102.stdout:5/836: symlink de/d25/d59/d7f/d86/da5/l131 0 2024-05-13T22:48:17.305 INFO:tasks.workunit.client.0.smithi102.stdout:5/837: rmdir de/d25/d2a/d4b/d4d/d50/da3/df4/de4 39 2024-05-13T22:48:17.307 INFO:tasks.workunit.client.0.smithi102.stdout:6/766: mknod d9/cf9 0 2024-05-13T22:48:17.308 INFO:tasks.workunit.client.0.smithi102.stdout:6/767: fdatasync d9/db/d14/d35/d47/f8f 0 2024-05-13T22:48:17.310 INFO:tasks.workunit.client.0.smithi102.stdout:9/705: dwrite d1/d6/d45/d3e/f93 [0,4194304] 0 2024-05-13T22:48:17.311 INFO:tasks.workunit.client.0.smithi102.stdout:3/794: truncate de/d10/d34/f93 466311 0 2024-05-13T22:48:17.313 INFO:tasks.workunit.client.0.smithi102.stdout:5/838: mkdir de/d25/d2a/d4b/d132 0 2024-05-13T22:48:17.318 INFO:tasks.workunit.client.0.smithi102.stdout:9/706: symlink d1/d6/db/d30/d8a/lf7 0 2024-05-13T22:48:17.321 INFO:tasks.workunit.client.0.smithi102.stdout:5/839: dwrite de/f13 [0,4194304] 0 2024-05-13T22:48:17.324 INFO:tasks.workunit.client.0.smithi102.stdout:9/707: mknod d1/d6/db/d4a/d75/d96/cf8 0 2024-05-13T22:48:17.327 INFO:tasks.workunit.client.0.smithi102.stdout:6/768: dread d9/f1a [0,4194304] 0 2024-05-13T22:48:17.327 INFO:tasks.workunit.client.0.smithi102.stdout:6/769: fdatasync d9/da3/de8/ff4 0 2024-05-13T22:48:17.330 INFO:tasks.workunit.client.0.smithi102.stdout:3/795: link de/d1c/c28 de/d68/df3/c108 0 2024-05-13T22:48:17.334 INFO:tasks.workunit.client.0.smithi102.stdout:6/770: unlink d9/db/d14/d54/c86 0 2024-05-13T22:48:17.340 INFO:tasks.workunit.client.0.smithi102.stdout:3/796: read de/d10/d2a/f2f [530617,82156] 0 2024-05-13T22:48:17.340 INFO:tasks.workunit.client.0.smithi102.stdout:9/708: link d1/d6/d45/d2d/dac/db0/db8/fc7 d1/d5d/d6c/ff9 0 2024-05-13T22:48:17.341 INFO:tasks.workunit.client.0.smithi102.stdout:9/709: write d1/d6/d45/d3e/f93 [5216765,76181] 0 2024-05-13T22:48:17.343 INFO:tasks.workunit.client.0.smithi102.stdout:6/771: creat d9/d25/d60/d6a/dd6/de9/ffa x:0 0 0 2024-05-13T22:48:17.346 INFO:tasks.workunit.client.0.smithi102.stdout:3/797: unlink de/d68/ff9 0 2024-05-13T22:48:17.364 INFO:tasks.workunit.client.0.smithi102.stdout:5/840: write de/d25/d2a/d4b/d4d/d50/da3/df4/f67 [268217,102372] 0 2024-05-13T22:48:17.368 INFO:tasks.workunit.client.0.smithi102.stdout:7/544: sync 2024-05-13T22:48:17.368 INFO:tasks.workunit.client.0.smithi102.stdout:0/704: sync 2024-05-13T22:48:17.368 INFO:tasks.workunit.client.0.smithi102.stdout:2/745: sync 2024-05-13T22:48:17.371 INFO:tasks.workunit.client.0.smithi102.stdout:5/841: dwrite de/d25/d58/dfb/f121 [0,4194304] 0 2024-05-13T22:48:17.375 INFO:tasks.workunit.client.0.smithi102.stdout:0/705: dread - d1/de/d32/d8a/fb2 zero size 2024-05-13T22:48:17.428 INFO:tasks.workunit.client.0.smithi102.stdout:0/706: unlink d1/d4d/d79/d9f/cc3 0 2024-05-13T22:48:17.431 INFO:tasks.workunit.client.0.smithi102.stdout:5/842: chown de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d53/la8 20063274 1 2024-05-13T22:48:17.433 INFO:tasks.workunit.client.0.smithi102.stdout:6/772: dwrite d9/db/d14/dcc/f4b [0,4194304] 0 2024-05-13T22:48:17.437 INFO:tasks.workunit.client.0.smithi102.stdout:2/746: write d7/d15/d67/f70 [2124562,25943] 0 2024-05-13T22:48:17.438 INFO:tasks.workunit.client.0.smithi102.stdout:0/707: mkdir d1/de/d32/dd8 0 2024-05-13T22:48:17.475 INFO:tasks.workunit.client.0.smithi102.stdout:7/545: write d6/d15/f18 [4869754,48158] 0 2024-05-13T22:48:17.475 INFO:tasks.workunit.client.0.smithi102.stdout:7/546: chown d6/d13/d58/l48 49035977 1 2024-05-13T22:48:17.475 INFO:tasks.workunit.client.0.smithi102.stdout:5/843: symlink de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d113/l133 0 2024-05-13T22:48:17.475 INFO:tasks.workunit.client.0.smithi102.stdout:7/547: mkdir d6/d78/d9b 0 2024-05-13T22:48:17.476 INFO:tasks.workunit.client.0.smithi102.stdout:7/548: symlink d6/d13/d2c/d39/l9c 0 2024-05-13T22:48:17.476 INFO:tasks.workunit.client.0.smithi102.stdout:0/708: dread d1/de/d32/f3b [0,4194304] 0 2024-05-13T22:48:17.476 INFO:tasks.workunit.client.0.smithi102.stdout:5/844: dread de/d75/fa4 [0,4194304] 0 2024-05-13T22:48:17.476 INFO:tasks.workunit.client.0.smithi102.stdout:5/845: write de/d25/d58/dfb/f121 [37256,110625] 0 2024-05-13T22:48:17.476 INFO:tasks.workunit.client.0.smithi102.stdout:7/549: symlink d6/d13/d2c/l9d 0 2024-05-13T22:48:17.476 INFO:tasks.workunit.client.0.smithi102.stdout:5/846: creat de/d75/f134 x:0 0 0 2024-05-13T22:48:17.476 INFO:tasks.workunit.client.0.smithi102.stdout:5/847: chown de/d25/d43/d7a/f88 531886 1 2024-05-13T22:48:17.476 INFO:tasks.workunit.client.0.smithi102.stdout:7/550: rename d6/d13/d62/l8b to d6/d15/d4d/d55/d82/l9e 0 2024-05-13T22:48:17.477 INFO:tasks.workunit.client.0.smithi102.stdout:5/848: symlink de/d75/l135 0 2024-05-13T22:48:17.478 INFO:tasks.workunit.client.0.smithi102.stdout:7/551: creat d6/d15/d4d/f9f x:0 0 0 2024-05-13T22:48:17.481 INFO:tasks.workunit.client.0.smithi102.stdout:7/552: symlink d6/d15/d4d/d55/d6f/d49/la0 0 2024-05-13T22:48:17.483 INFO:tasks.workunit.client.0.smithi102.stdout:7/553: chown d6/d15/d4c/d90/f4a 120506 1 2024-05-13T22:48:17.485 INFO:tasks.workunit.client.0.smithi102.stdout:5/849: creat de/f136 x:0 0 0 2024-05-13T22:48:17.487 INFO:tasks.workunit.client.0.smithi102.stdout:5/850: chown de/d25/d2c/f8a 25548223 1 2024-05-13T22:48:17.488 INFO:tasks.workunit.client.0.smithi102.stdout:6/773: dwrite d9/db/d14/dcc/f43 [0,4194304] 0 2024-05-13T22:48:17.490 INFO:tasks.workunit.client.0.smithi102.stdout:6/774: chown d9/db/d14/d35/d47/dc1/fc6 6189765 1 2024-05-13T22:48:17.491 INFO:tasks.workunit.client.0.smithi102.stdout:7/554: mkdir d6/d13/d2c/da1 0 2024-05-13T22:48:17.492 INFO:tasks.workunit.client.0.smithi102.stdout:5/851: mknod de/d25/d2a/d4b/d4d/d50/d6c/c137 0 2024-05-13T22:48:17.500 INFO:tasks.workunit.client.0.smithi102.stdout:7/555: mknod d6/d15/d4d/d55/ca2 0 2024-05-13T22:48:17.500 INFO:tasks.workunit.client.0.smithi102.stdout:2/747: write d7/f32 [1519067,128000] 0 2024-05-13T22:48:17.511 INFO:tasks.workunit.client.0.smithi102.stdout:0/709: dwrite d1/d68/f86 [0,4194304] 0 2024-05-13T22:48:17.529 INFO:tasks.workunit.client.0.smithi102.stdout:6/775: write d9/db/f7d [1102405,69368] 0 2024-05-13T22:48:17.530 INFO:tasks.workunit.client.0.smithi102.stdout:6/776: chown d9/db/d14/d54 491808055 1 2024-05-13T22:48:17.532 INFO:tasks.workunit.client.0.smithi102.stdout:5/852: write de/d25/d59/d6f/f92 [556560,6026] 0 2024-05-13T22:48:17.542 INFO:tasks.workunit.client.0.smithi102.stdout:0/710: mknod d1/d4d/d79/d9f/cd9 0 2024-05-13T22:48:17.543 INFO:tasks.workunit.client.0.smithi102.stdout:6/777: mknod d9/d25/d60/d6a/dd6/cfb 0 2024-05-13T22:48:17.544 INFO:tasks.workunit.client.0.smithi102.stdout:5/853: mknod de/d25/d108/dc5/c138 0 2024-05-13T22:48:17.545 INFO:tasks.workunit.client.0.smithi102.stdout:5/854: chown de/d25/d43/df5/d112/d11b 782582 1 2024-05-13T22:48:17.548 INFO:tasks.workunit.client.0.smithi102.stdout:2/748: rename d7/d9/d14/d31/fc4 to d7/d9/d10/d1b/d54/ff7 0 2024-05-13T22:48:17.548 INFO:tasks.workunit.client.0.smithi102.stdout:2/749: chown d7/d9/d14/f73 28359 1 2024-05-13T22:48:17.557 INFO:tasks.workunit.client.0.smithi102.stdout:0/711: symlink d1/d4d/d79/lda 0 2024-05-13T22:48:17.561 INFO:tasks.workunit.client.0.smithi102.stdout:0/712: dread d1/de/d32/d24/d2d/d38/f69 [0,4194304] 0 2024-05-13T22:48:17.567 INFO:tasks.workunit.client.0.smithi102.stdout:2/750: mkdir d7/d9/d11/ddd/df8 0 2024-05-13T22:48:17.573 INFO:tasks.workunit.client.0.smithi102.stdout:2/751: mknod d7/d9/d10/d25/d27/cf9 0 2024-05-13T22:48:17.575 INFO:tasks.workunit.client.0.smithi102.stdout:5/855: getdents de 0 2024-05-13T22:48:17.576 INFO:tasks.workunit.client.0.smithi102.stdout:5/856: stat de/f4f 0 2024-05-13T22:48:17.576 INFO:tasks.workunit.client.0.smithi102.stdout:0/713: creat d1/de/d32/d24/d2d/d38/db7/fdb x:0 0 0 2024-05-13T22:48:17.578 INFO:tasks.workunit.client.0.smithi102.stdout:2/752: creat d7/d9/d14/d31/dbb/ffa x:0 0 0 2024-05-13T22:48:17.579 INFO:tasks.workunit.client.0.smithi102.stdout:7/556: write d6/d15/d4d/d55/f74 [587359,127897] 0 2024-05-13T22:48:17.580 INFO:tasks.workunit.client.0.smithi102.stdout:5/857: creat de/d25/d43/f139 x:0 0 0 2024-05-13T22:48:17.585 INFO:tasks.workunit.client.0.smithi102.stdout:6/778: rmdir d9 39 2024-05-13T22:48:17.586 INFO:tasks.workunit.client.0.smithi102.stdout:7/557: truncate d6/f94 566084 0 2024-05-13T22:48:17.588 INFO:tasks.workunit.client.0.smithi102.stdout:0/714: mknod d1/de/d32/d24/d2d/d38/cdc 0 2024-05-13T22:48:17.591 INFO:tasks.workunit.client.0.smithi102.stdout:5/858: mkdir de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d53/d13a 0 2024-05-13T22:48:17.598 INFO:tasks.workunit.client.0.smithi102.stdout:7/558: creat d6/d15/d4d/d55/fa3 x:0 0 0 2024-05-13T22:48:17.600 INFO:tasks.workunit.client.0.smithi102.stdout:0/715: creat d1/de/d32/d8d/fdd x:0 0 0 2024-05-13T22:48:17.601 INFO:tasks.workunit.client.0.smithi102.stdout:5/859: creat de/d25/d2a/d4b/d4d/d50/d6c/df8/f13b x:0 0 0 2024-05-13T22:48:17.603 INFO:tasks.workunit.client.0.smithi102.stdout:2/753: rename d7/d9/d10/d30/l65 to d7/d9/d14/lfb 0 2024-05-13T22:48:17.604 INFO:tasks.workunit.client.0.smithi102.stdout:2/754: chown d7/d9/d10/d25/d27/dab/dee 2 1 2024-05-13T22:48:17.605 INFO:tasks.workunit.client.0.smithi102.stdout:6/779: truncate d9/f10 1584471 0 2024-05-13T22:48:17.607 INFO:tasks.workunit.client.0.smithi102.stdout:6/780: read d9/db/d14/dcc/f7b [2394369,34722] 0 2024-05-13T22:48:17.607 INFO:tasks.workunit.client.0.smithi102.stdout:0/716: creat d1/de/d32/d8a/fde x:0 0 0 2024-05-13T22:48:17.608 INFO:tasks.workunit.client.0.smithi102.stdout:6/781: read d9/db/d14/d35/f9d [421961,108077] 0 2024-05-13T22:48:17.609 INFO:tasks.workunit.client.0.smithi102.stdout:5/860: fdatasync de/d25/d59/d7f/d86/da5/fd9 0 2024-05-13T22:48:17.610 INFO:tasks.workunit.client.0.smithi102.stdout:2/755: fsync d7/d9/d14/f62 0 2024-05-13T22:48:17.613 INFO:tasks.workunit.client.0.smithi102.stdout:0/717: dwrite d1/d4d/d79/f80 [0,4194304] 0 2024-05-13T22:48:17.628 INFO:tasks.workunit.client.0.smithi102.stdout:7/559: rmdir d6/d78/d9b 0 2024-05-13T22:48:17.630 INFO:tasks.workunit.client.0.smithi102.stdout:0/718: creat d1/d68/fdf x:0 0 0 2024-05-13T22:48:17.636 INFO:tasks.workunit.client.0.smithi102.stdout:0/719: dwrite d1/de/d32/d24/d2d/d91/fd7 [0,4194304] 0 2024-05-13T22:48:17.641 INFO:tasks.workunit.client.0.smithi102.stdout:0/720: dread - d1/de/d32/d24/d2d/f9d zero size 2024-05-13T22:48:17.647 INFO:tasks.workunit.client.0.smithi102.stdout:7/560: getdents d6/d15/d4d/d55/d6f/d65 0 2024-05-13T22:48:17.648 INFO:tasks.workunit.client.0.smithi102.stdout:0/721: symlink d1/de/d32/d24/d2d/d38/le0 0 2024-05-13T22:48:17.662 INFO:tasks.workunit.client.0.smithi102.stdout:5/861: dwrite de/f35 [0,4194304] 0 2024-05-13T22:48:17.663 INFO:tasks.workunit.client.0.smithi102.stdout:6/782: dwrite d9/db/f50 [0,4194304] 0 2024-05-13T22:48:17.665 INFO:tasks.workunit.client.0.smithi102.stdout:2/756: dwrite d7/d9/d10/fb3 [0,4194304] 0 2024-05-13T22:48:17.665 INFO:tasks.workunit.client.0.smithi102.stdout:1/574: sync 2024-05-13T22:48:17.669 INFO:tasks.workunit.client.0.smithi102.stdout:5/862: creat de/d25/d2a/d4b/dab/f13c x:0 0 0 2024-05-13T22:48:17.670 INFO:tasks.workunit.client.0.smithi102.stdout:6/783: dread - d9/d45/d56/dd5/fbd zero size 2024-05-13T22:48:17.675 INFO:tasks.workunit.client.0.smithi102.stdout:0/722: write d1/fad [78759,27020] 0 2024-05-13T22:48:17.677 INFO:tasks.workunit.client.0.smithi102.stdout:1/575: creat d2/d4/d26/d40/d5e/fb6 x:0 0 0 2024-05-13T22:48:17.677 INFO:tasks.workunit.client.0.smithi102.stdout:7/561: dread d6/d13/f16 [0,4194304] 0 2024-05-13T22:48:17.677 INFO:tasks.workunit.client.0.smithi102.stdout:5/863: readlink de/d25/d58/dfb/l129 0 2024-05-13T22:48:17.678 INFO:tasks.workunit.client.0.smithi102.stdout:6/784: creat d9/d45/d56/d5d/ffc x:0 0 0 2024-05-13T22:48:17.680 INFO:tasks.workunit.client.0.smithi102.stdout:7/562: chown d6/d15/d4d/d55/d6f/f51 16 1 2024-05-13T22:48:17.681 INFO:tasks.workunit.client.0.smithi102.stdout:5/864: fsync de/d25/d58/f101 0 2024-05-13T22:48:17.685 INFO:tasks.workunit.client.0.smithi102.stdout:1/576: creat d2/d45/d79/d92/fb7 x:0 0 0 2024-05-13T22:48:17.687 INFO:tasks.workunit.client.0.smithi102.stdout:0/723: dwrite d1/de/d32/d24/d2d/d3e/f41 [0,4194304] 0 2024-05-13T22:48:17.689 INFO:tasks.workunit.client.0.smithi102.stdout:1/577: rmdir d2 39 2024-05-13T22:48:17.691 INFO:tasks.workunit.client.0.smithi102.stdout:0/724: unlink d1/de/d32/d63/f98 0 2024-05-13T22:48:17.692 INFO:tasks.workunit.client.0.smithi102.stdout:1/578: readlink d2/d4/l18 0 2024-05-13T22:48:17.695 INFO:tasks.workunit.client.0.smithi102.stdout:1/579: read d2/d4/d26/d40/f72 [1582909,58680] 0 2024-05-13T22:48:17.705 INFO:tasks.workunit.client.0.smithi102.stdout:1/580: rename d2/d45/f5a to d2/d4/fb8 0 2024-05-13T22:48:17.706 INFO:tasks.workunit.client.0.smithi102.stdout:2/757: dwrite d7/d9/d11/f38 [0,4194304] 0 2024-05-13T22:48:17.706 INFO:tasks.workunit.client.0.smithi102.stdout:1/581: chown d2/d45/da4 12468677 1 2024-05-13T22:48:17.707 INFO:tasks.workunit.client.0.smithi102.stdout:1/582: chown d2/d10/c8a 4 1 2024-05-13T22:48:17.711 INFO:tasks.workunit.client.0.smithi102.stdout:2/758: symlink d7/d9/d10/d30/d53/dd1/lfc 0 2024-05-13T22:48:17.713 INFO:tasks.workunit.client.0.smithi102.stdout:5/865: dread de/d25/d2a/d4b/d4d/d50/da3/df4/f98 [0,4194304] 0 2024-05-13T22:48:17.715 INFO:tasks.workunit.client.0.smithi102.stdout:9/710: sync 2024-05-13T22:48:17.715 INFO:tasks.workunit.client.0.smithi102.stdout:4/750: sync 2024-05-13T22:48:17.715 INFO:tasks.workunit.client.0.smithi102.stdout:3/798: sync 2024-05-13T22:48:17.715 INFO:tasks.workunit.client.0.smithi102.stdout:8/680: sync 2024-05-13T22:48:17.716 INFO:tasks.workunit.client.0.smithi102.stdout:6/785: dwrite d9/d25/f30 [4194304,4194304] 0 2024-05-13T22:48:17.720 INFO:tasks.workunit.client.0.smithi102.stdout:5/866: symlink de/d25/d2c/l13d 0 2024-05-13T22:48:17.725 INFO:tasks.workunit.client.0.smithi102.stdout:7/563: dwrite d6/f85 [0,4194304] 0 2024-05-13T22:48:17.725 INFO:tasks.workunit.client.0.smithi102.stdout:0/725: write d1/de/d32/d24/d2d/d38/d3d/d81/fb8 [447952,10664] 0 2024-05-13T22:48:17.725 INFO:tasks.workunit.client.0.smithi102.stdout:9/711: stat d1/d6/d45/l2f 0 2024-05-13T22:48:17.726 INFO:tasks.workunit.client.0.smithi102.stdout:1/583: link d2/d4/d5/d19/d28/l3c d2/d4/d26/da2/lb9 0 2024-05-13T22:48:17.734 INFO:tasks.workunit.client.0.smithi102.stdout:3/799: symlink de/d10/d2a/d36/db2/ddc/l109 0 2024-05-13T22:48:17.734 INFO:tasks.workunit.client.0.smithi102.stdout:6/786: dread - d9/db/d14/d35/fc2 zero size 2024-05-13T22:48:17.736 INFO:tasks.workunit.client.0.smithi102.stdout:2/759: dread d7/d15/fa5 [0,4194304] 0 2024-05-13T22:48:17.737 INFO:tasks.workunit.client.0.smithi102.stdout:1/584: mkdir d2/d4/d38/db2/dba 0 2024-05-13T22:48:17.740 INFO:tasks.workunit.client.0.smithi102.stdout:4/751: dwrite d3/d6b/fdf [0,4194304] 0 2024-05-13T22:48:17.741 INFO:tasks.workunit.client.0.smithi102.stdout:0/726: truncate d1/de/d32/da6/fc4 688554 0 2024-05-13T22:48:17.750 INFO:tasks.workunit.client.0.smithi102.stdout:7/564: rename d6/d15/d4c/d90/f68 to d6/d15/d4d/fa4 0 2024-05-13T22:48:17.751 INFO:tasks.workunit.client.0.smithi102.stdout:3/800: dwrite de/d68/f104 [0,4194304] 0 2024-05-13T22:48:17.756 INFO:tasks.workunit.client.0.smithi102.stdout:4/752: symlink d3/d19/d23/d39/d5d/l106 0 2024-05-13T22:48:17.760 INFO:tasks.workunit.client.0.smithi102.stdout:5/867: write de/d25/d58/fa6 [99134,30712] 0 2024-05-13T22:48:17.763 INFO:tasks.workunit.client.0.smithi102.stdout:1/585: rename d2/c73 to d2/d4/d38/cbb 0 2024-05-13T22:48:17.764 INFO:tasks.workunit.client.0.smithi102.stdout:3/801: dwrite de/d10/d2a/d33/daa/fb3 [4194304,4194304] 0 2024-05-13T22:48:17.772 INFO:tasks.workunit.client.0.smithi102.stdout:4/753: fsync d3/d22/f74 0 2024-05-13T22:48:17.774 INFO:tasks.workunit.client.0.smithi102.stdout:3/802: dwrite de/d10/d46/f6f [0,4194304] 0 2024-05-13T22:48:17.775 INFO:tasks.workunit.client.0.smithi102.stdout:9/712: getdents d1/d6/d45/d2d 0 2024-05-13T22:48:17.778 INFO:tasks.workunit.client.0.smithi102.stdout:7/565: link d6/d15/f3e d6/d15/d4c/d90/d80/fa5 0 2024-05-13T22:48:17.779 INFO:tasks.workunit.client.0.smithi102.stdout:8/681: dread d2/d7/d27/d64/d96/dae/f58 [0,4194304] 0 2024-05-13T22:48:17.785 INFO:tasks.workunit.client.0.smithi102.stdout:2/760: dread d7/d9/d10/d25/f68 [0,4194304] 0 2024-05-13T22:48:17.785 INFO:tasks.workunit.client.0.smithi102.stdout:6/787: getdents d9/d25/d60/d6a/dd6/de9/df7 0 2024-05-13T22:48:17.791 INFO:tasks.workunit.client.0.smithi102.stdout:4/754: dread d3/d19/dad/fbc [0,4194304] 0 2024-05-13T22:48:17.792 INFO:tasks.workunit.client.0.smithi102.stdout:3/803: dwrite de/d1c/d79/f81 [0,4194304] 0 2024-05-13T22:48:17.793 INFO:tasks.workunit.client.0.smithi102.stdout:2/761: fsync d7/d9/d10/d30/fce 0 2024-05-13T22:48:17.793 INFO:tasks.workunit.client.0.smithi102.stdout:7/566: rename d6/d13/d2c/f5d to d6/d15/d4d/d55/d6f/d49/d97/fa6 0 2024-05-13T22:48:17.797 INFO:tasks.workunit.client.0.smithi102.stdout:2/762: fsync d7/d9/d10/fe7 0 2024-05-13T22:48:17.802 INFO:tasks.workunit.client.0.smithi102.stdout:8/682: dwrite d2/d7/d27/f3d [0,4194304] 0 2024-05-13T22:48:17.806 INFO:tasks.workunit.client.0.smithi102.stdout:2/763: chown d7/d9/d10/d25/d27/cf9 85 1 2024-05-13T22:48:17.806 INFO:tasks.workunit.client.0.smithi102.stdout:8/683: readlink d2/d36/d49/l79 0 2024-05-13T22:48:17.807 INFO:tasks.workunit.client.0.smithi102.stdout:1/586: truncate d2/d10/d62/d7b/f82 1900544 0 2024-05-13T22:48:17.810 INFO:tasks.workunit.client.0.smithi102.stdout:9/713: dwrite d1/d6/d3b/d5b/fa2 [0,4194304] 0 2024-05-13T22:48:17.813 INFO:tasks.workunit.client.0.smithi102.stdout:7/567: creat d6/d15/d4c/d90/d4b/fa7 x:0 0 0 2024-05-13T22:48:17.815 INFO:tasks.workunit.client.0.smithi102.stdout:4/755: mkdir d3/d19/d23/d4e/d68/d85/d107 0 2024-05-13T22:48:17.815 INFO:tasks.workunit.client.0.smithi102.stdout:2/764: chown d7/d9/d14/d31/f97 43047 1 2024-05-13T22:48:17.817 INFO:tasks.workunit.client.0.smithi102.stdout:3/804: dwrite de/d68/f104 [0,4194304] 0 2024-05-13T22:48:17.819 INFO:tasks.workunit.client.0.smithi102.stdout:8/684: fsync d2/d7/d27/d64/d96/dae/d76/f9f 0 2024-05-13T22:48:17.823 INFO:tasks.workunit.client.0.smithi102.stdout:1/587: creat d2/d45/d79/d92/fbc x:0 0 0 2024-05-13T22:48:17.823 INFO:tasks.workunit.client.0.smithi102.stdout:5/868: dread de/d25/d2a/d4b/d4d/d50/d6c/db9/fc6 [0,4194304] 0 2024-05-13T22:48:17.823 INFO:tasks.workunit.client.0.smithi102.stdout:8/685: write d2/d36/d49/d5d/f51 [2075618,81910] 0 2024-05-13T22:48:17.824 INFO:tasks.workunit.client.0.smithi102.stdout:6/788: write d9/db/d14/d35/f9d [1561108,66354] 0 2024-05-13T22:48:17.824 INFO:tasks.workunit.client.0.smithi102.stdout:9/714: symlink d1/d6/db/d30/d47/lfa 0 2024-05-13T22:48:17.827 INFO:tasks.workunit.client.0.smithi102.stdout:4/756: truncate d3/d4/fb4 1368947 0 2024-05-13T22:48:17.828 INFO:tasks.workunit.client.0.smithi102.stdout:1/588: read d2/d4/d26/f8c [655919,6876] 0 2024-05-13T22:48:17.830 INFO:tasks.workunit.client.0.smithi102.stdout:1/589: fdatasync d2/d10/fa0 0 2024-05-13T22:48:17.832 INFO:tasks.workunit.client.0.smithi102.stdout:3/805: mknod de/d10/d2a/d33/d61/c10a 0 2024-05-13T22:48:17.834 INFO:tasks.workunit.client.0.smithi102.stdout:5/869: creat de/d75/f13e x:0 0 0 2024-05-13T22:48:17.835 INFO:tasks.workunit.client.0.smithi102.stdout:9/715: mknod d1/d5d/d6c/cfb 0 2024-05-13T22:48:17.835 INFO:tasks.workunit.client.0.smithi102.stdout:3/806: truncate de/d10/d46/f6c 2298191 0 2024-05-13T22:48:17.835 INFO:tasks.workunit.client.0.smithi102.stdout:6/789: dwrite d9/db/d14/d35/f9d [0,4194304] 0 2024-05-13T22:48:17.837 INFO:tasks.workunit.client.0.smithi102.stdout:9/716: chown d1/d6/db/f1c 818163891 1 2024-05-13T22:48:17.837 INFO:tasks.workunit.client.0.smithi102.stdout:6/790: read - d9/da3/de8/ff4 zero size 2024-05-13T22:48:17.837 INFO:tasks.workunit.client.0.smithi102.stdout:7/568: write f3 [5012835,76239] 0 2024-05-13T22:48:17.840 INFO:tasks.workunit.client.0.smithi102.stdout:8/686: rmdir d2/d7/d25/d41/d6f 39 2024-05-13T22:48:17.846 INFO:tasks.workunit.client.0.smithi102.stdout:3/807: dwrite de/d67/fd7 [0,4194304] 0 2024-05-13T22:48:17.852 INFO:tasks.workunit.client.0.smithi102.stdout:1/590: unlink d2/d4/f20 0 2024-05-13T22:48:17.861 INFO:tasks.workunit.client.0.smithi102.stdout:7/569: rename d6/d15/d4d/d55 to d6/d15/d4d/d55/d6f/da8 22 2024-05-13T22:48:17.866 INFO:tasks.workunit.client.0.smithi102.stdout:1/591: mkdir d2/d4/d5/d19/d28/d6a/dbd 0 2024-05-13T22:48:17.868 INFO:tasks.workunit.client.0.smithi102.stdout:4/757: dwrite d3/d19/d23/d4e/f72 [0,4194304] 0 2024-05-13T22:48:17.870 INFO:tasks.workunit.client.0.smithi102.stdout:6/791: dread d9/db/d14/d35/dc9/df1/fcd [0,4194304] 0 2024-05-13T22:48:17.877 INFO:tasks.workunit.client.0.smithi102.stdout:5/870: write de/d25/d59/d7f/d86/da5/f10e [1020618,97331] 0 2024-05-13T22:48:17.882 INFO:tasks.workunit.client.0.smithi102.stdout:6/792: mknod d9/da3/cfd 0 2024-05-13T22:48:17.882 INFO:tasks.workunit.client.0.smithi102.stdout:3/808: dwrite de/d10/d2a/d36/db2/de9/f107 [0,4194304] 0 2024-05-13T22:48:17.884 INFO:tasks.workunit.client.0.smithi102.stdout:2/765: dwrite d7/d9/d10/d25/d27/f3b [0,4194304] 0 2024-05-13T22:48:17.887 INFO:tasks.workunit.client.0.smithi102.stdout:7/570: write f4 [1986818,104321] 0 2024-05-13T22:48:17.893 INFO:tasks.workunit.client.0.smithi102.stdout:4/758: dread d3/d19/d23/d4e/f72 [0,4194304] 0 2024-05-13T22:48:17.893 INFO:tasks.workunit.client.0.smithi102.stdout:8/687: dwrite d2/d7/d27/d64/d96/dae/f23 [0,4194304] 0 2024-05-13T22:48:17.895 INFO:tasks.workunit.client.0.smithi102.stdout:9/717: dwrite d1/d6/f60 [0,4194304] 0 2024-05-13T22:48:17.900 INFO:tasks.workunit.client.0.smithi102.stdout:1/592: write d2/d45/f47 [1628441,99711] 0 2024-05-13T22:48:17.901 INFO:tasks.workunit.client.0.smithi102.stdout:5/871: mknod de/d25/d2a/d4b/d4d/d93/c13f 0 2024-05-13T22:48:17.901 INFO:tasks.workunit.client.0.smithi102.stdout:1/593: chown d2/d10/f85 1788576878 1 2024-05-13T22:48:17.906 INFO:tasks.workunit.client.0.smithi102.stdout:2/766: chown d7/d9/d14/d31/dbb/le3 6195754 1 2024-05-13T22:48:17.910 INFO:tasks.workunit.client.0.smithi102.stdout:5/872: mknod de/d25/d43/dd1/c140 0 2024-05-13T22:48:17.914 INFO:tasks.workunit.client.0.smithi102.stdout:8/688: unlink d2/c5e 0 2024-05-13T22:48:17.916 INFO:tasks.workunit.client.0.smithi102.stdout:5/873: dread de/d25/d59/d7f/d86/da5/ddb/dee/f115 [0,4194304] 0 2024-05-13T22:48:17.918 INFO:tasks.workunit.client.0.smithi102.stdout:2/767: mknod d7/d15/d1e/dbf/cfd 0 2024-05-13T22:48:17.920 INFO:tasks.workunit.client.0.smithi102.stdout:4/759: link d3/d4/d6/de/d8d/dec/fef d3/d19/d23/d3a/da4/f108 0 2024-05-13T22:48:17.923 INFO:tasks.workunit.client.0.smithi102.stdout:7/571: rename d6/d78 to d6/da9 0 2024-05-13T22:48:17.925 INFO:tasks.workunit.client.0.smithi102.stdout:5/874: creat de/d25/d108/f141 x:0 0 0 2024-05-13T22:48:17.928 INFO:tasks.workunit.client.0.smithi102.stdout:4/760: fdatasync d3/d22/f74 0 2024-05-13T22:48:17.934 INFO:tasks.workunit.client.0.smithi102.stdout:3/809: write fa [4479560,92379] 0 2024-05-13T22:48:17.937 INFO:tasks.workunit.client.0.smithi102.stdout:4/761: rmdir d3/d19/d23/d39/da7/d7d 39 2024-05-13T22:48:17.941 INFO:tasks.workunit.client.0.smithi102.stdout:2/768: dread d7/d9/d14/f36 [4194304,4194304] 0 2024-05-13T22:48:17.941 INFO:tasks.workunit.client.0.smithi102.stdout:7/572: dread d6/d15/d4c/d90/d4b/f66 [0,4194304] 0 2024-05-13T22:48:17.942 INFO:tasks.workunit.client.0.smithi102.stdout:5/875: link de/d25/d2a/d4b/d4d/d50/f109 de/d25/d2a/d4b/d4d/d50/da3/df4/de4/f142 0 2024-05-13T22:48:17.944 INFO:tasks.workunit.client.0.smithi102.stdout:3/810: rmdir de/d10/d50/d63 39 2024-05-13T22:48:17.948 INFO:tasks.workunit.client.0.smithi102.stdout:2/769: chown d7/d9/d10/cba 35 1 2024-05-13T22:48:17.949 INFO:tasks.workunit.client.0.smithi102.stdout:2/770: chown d7/d9/d10/d25/f35 11 1 2024-05-13T22:48:17.952 INFO:tasks.workunit.client.0.smithi102.stdout:3/811: creat de/d10/d2a/d36/d88/f10b x:0 0 0 2024-05-13T22:48:17.954 INFO:tasks.workunit.client.0.smithi102.stdout:7/573: symlink d6/d13/d2c/da1/laa 0 2024-05-13T22:48:17.957 INFO:tasks.workunit.client.0.smithi102.stdout:2/771: unlink d7/d15/fae 0 2024-05-13T22:48:17.958 INFO:tasks.workunit.client.0.smithi102.stdout:8/689: dwrite d2/d36/d49/d5d/f26 [0,4194304] 0 2024-05-13T22:48:17.960 INFO:tasks.workunit.client.0.smithi102.stdout:8/690: fdatasync d2/d7/d27/d64/d80/dc8/f39 0 2024-05-13T22:48:17.960 INFO:tasks.workunit.client.0.smithi102.stdout:3/812: readlink de/d10/d50/d63/lc2 0 2024-05-13T22:48:17.970 INFO:tasks.workunit.client.0.smithi102.stdout:5/876: dwrite de/f21 [0,4194304] 0 2024-05-13T22:48:17.975 INFO:tasks.workunit.client.0.smithi102.stdout:3/813: creat de/d10/d46/dc8/f10c x:0 0 0 2024-05-13T22:48:17.978 INFO:tasks.workunit.client.0.smithi102.stdout:7/574: write d6/f2e [906200,111254] 0 2024-05-13T22:48:17.981 INFO:tasks.workunit.client.0.smithi102.stdout:2/772: unlink d7/d9/d14/d31/dbb/dd5/lde 0 2024-05-13T22:48:17.982 INFO:tasks.workunit.client.0.smithi102.stdout:7/575: write d6/f7 [186353,64178] 0 2024-05-13T22:48:17.986 INFO:tasks.workunit.client.0.smithi102.stdout:4/762: dwrite d3/d19/d23/d39/da7/d7d/f70 [0,4194304] 0 2024-05-13T22:48:17.987 INFO:tasks.workunit.client.0.smithi102.stdout:7/576: chown d6/fc 1 1 2024-05-13T22:48:17.990 INFO:tasks.workunit.client.0.smithi102.stdout:2/773: mkdir d7/d15/d1e/dbf/dfe 0 2024-05-13T22:48:17.990 INFO:tasks.workunit.client.0.smithi102.stdout:7/577: fdatasync d6/d13/d2c/d39/f3f 0 2024-05-13T22:48:17.991 INFO:tasks.workunit.client.0.smithi102.stdout:3/814: dwrite de/d1c/d79/f81 [0,4194304] 0 2024-05-13T22:48:17.993 INFO:tasks.workunit.client.0.smithi102.stdout:2/774: readlink d7/d9/d11/lc9 0 2024-05-13T22:48:17.995 INFO:tasks.workunit.client.0.smithi102.stdout:7/578: creat d6/d13/de/fab x:0 0 0 2024-05-13T22:48:17.996 INFO:tasks.workunit.client.0.smithi102.stdout:5/877: dread de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/f7b [0,4194304] 0 2024-05-13T22:48:17.999 INFO:tasks.workunit.client.0.smithi102.stdout:3/815: rmdir de/d67 39 2024-05-13T22:48:18.012 INFO:tasks.workunit.client.0.smithi102.stdout:2/775: mknod d7/d9/d10/d1b/d54/cff 0 2024-05-13T22:48:18.013 INFO:tasks.workunit.client.0.smithi102.stdout:3/816: mknod de/d67/de1/c10d 0 2024-05-13T22:48:18.016 INFO:tasks.workunit.client.0.smithi102.stdout:2/776: rename d7/d9/d14/d31/l46 to d7/d9/d7c/l100 0 2024-05-13T22:48:18.017 INFO:tasks.workunit.client.0.smithi102.stdout:2/777: chown d7/d15/d1e/da7/db5 52784 1 2024-05-13T22:48:18.018 INFO:tasks.workunit.client.0.smithi102.stdout:2/778: stat d7/d15/d1e/f26 0 2024-05-13T22:48:18.019 INFO:tasks.workunit.client.0.smithi102.stdout:7/579: write d6/d15/f37 [936992,106246] 0 2024-05-13T22:48:18.020 INFO:tasks.workunit.client.0.smithi102.stdout:5/878: write de/d25/d2a/d4b/d4d/d114/dda/ffc [881860,38765] 0 2024-05-13T22:48:18.021 INFO:tasks.workunit.client.0.smithi102.stdout:7/580: chown d6/d15/d4d/d55/d6f/l42 522419 1 2024-05-13T22:48:18.022 INFO:tasks.workunit.client.0.smithi102.stdout:4/763: dread d3/d22/f74 [0,4194304] 0 2024-05-13T22:48:18.029 INFO:tasks.workunit.client.0.smithi102.stdout:5/879: mkdir de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d113/d143 0 2024-05-13T22:48:18.031 INFO:tasks.workunit.client.0.smithi102.stdout:4/764: creat d3/d19/dad/dc9/dd5/f109 x:0 0 0 2024-05-13T22:48:18.033 INFO:tasks.workunit.client.0.smithi102.stdout:3/817: dwrite de/d10/d46/fac [0,4194304] 0 2024-05-13T22:48:18.033 INFO:tasks.workunit.client.0.smithi102.stdout:7/581: creat d6/d13/d58/fac x:0 0 0 2024-05-13T22:48:18.035 INFO:tasks.workunit.client.0.smithi102.stdout:7/582: stat d6/d13/d2c/d39 0 2024-05-13T22:48:18.036 INFO:tasks.workunit.client.0.smithi102.stdout:5/880: mkdir de/d25/d59/d7f/d86/da5/d144 0 2024-05-13T22:48:18.037 INFO:tasks.workunit.client.0.smithi102.stdout:4/765: symlink d3/d4/d6/de/d1f/d41/d6a/l10a 0 2024-05-13T22:48:18.038 INFO:tasks.workunit.client.0.smithi102.stdout:2/779: dread d7/d9/d14/f62 [0,4194304] 0 2024-05-13T22:48:18.040 INFO:tasks.workunit.client.0.smithi102.stdout:4/766: fdatasync d3/d19/d23/d3a/f4a 0 2024-05-13T22:48:18.041 INFO:tasks.workunit.client.0.smithi102.stdout:2/780: read d7/d9/d10/fb3 [3994980,123560] 0 2024-05-13T22:48:18.042 INFO:tasks.workunit.client.0.smithi102.stdout:7/583: mknod d6/d15/d4c/cad 0 2024-05-13T22:48:18.043 INFO:tasks.workunit.client.0.smithi102.stdout:3/818: dwrite de/d68/f104 [4194304,4194304] 0 2024-05-13T22:48:18.046 INFO:tasks.workunit.client.0.smithi102.stdout:7/584: readlink d6/d15/d4d/d55/d6f/l73 0 2024-05-13T22:48:18.046 INFO:tasks.workunit.client.0.smithi102.stdout:5/881: dread de/f21 [0,4194304] 0 2024-05-13T22:48:18.046 INFO:tasks.workunit.client.0.smithi102.stdout:3/819: fdatasync de/d10/d2a/d8c/d7f/fd6 0 2024-05-13T22:48:18.053 INFO:tasks.workunit.client.0.smithi102.stdout:2/781: fsync d7/d9/d10/d25/d27/f5b 0 2024-05-13T22:48:18.055 INFO:tasks.workunit.client.0.smithi102.stdout:7/585: creat d6/d15/d4c/d90/d80/fae x:0 0 0 2024-05-13T22:48:18.056 INFO:tasks.workunit.client.0.smithi102.stdout:3/820: mkdir de/d10/d2a/d8c/da8/d10e 0 2024-05-13T22:48:18.058 INFO:tasks.workunit.client.0.smithi102.stdout:5/882: symlink de/d25/d2a/d4b/d4d/d50/da3/df4/de4/l145 0 2024-05-13T22:48:18.058 INFO:tasks.workunit.client.0.smithi102.stdout:3/821: read de/d10/d2a/f3a [234889,33310] 0 2024-05-13T22:48:18.061 INFO:tasks.workunit.client.0.smithi102.stdout:4/767: creat d3/d19/d23/d3a/da4/f10b x:0 0 0 2024-05-13T22:48:18.066 INFO:tasks.workunit.client.0.smithi102.stdout:2/782: rename d7/d9/d10/d30/d53/da4/fca to d7/d9/d14/d31/dbb/dd5/f101 0 2024-05-13T22:48:18.071 INFO:tasks.workunit.client.0.smithi102.stdout:2/783: dread d7/d9/d10/d25/f68 [0,4194304] 0 2024-05-13T22:48:18.071 INFO:tasks.workunit.client.0.smithi102.stdout:3/822: creat de/d67/df7/f10f x:0 0 0 2024-05-13T22:48:18.072 INFO:tasks.workunit.client.0.smithi102.stdout:7/586: dread d6/d15/d4d/d55/d6f/d65/f86 [0,4194304] 0 2024-05-13T22:48:18.077 INFO:tasks.workunit.client.0.smithi102.stdout:5/883: unlink de/d25/d43/dd1/ldf 0 2024-05-13T22:48:18.085 INFO:tasks.workunit.client.0.smithi102.stdout:5/884: rename de/d25/d2a/d4b/d4d/d50/d6c/db9/fc6 to de/d25/d2a/d4b/d4d/d50/d6c/f146 0 2024-05-13T22:48:18.087 INFO:tasks.workunit.client.0.smithi102.stdout:4/768: rmdir d3/d19/d23/d4e/d68/d85/d107 0 2024-05-13T22:48:18.088 INFO:tasks.workunit.client.0.smithi102.stdout:7/587: dread d6/f7 [0,4194304] 0 2024-05-13T22:48:18.089 INFO:tasks.workunit.client.0.smithi102.stdout:7/588: write d6/d15/f3e [2429822,77225] 0 2024-05-13T22:48:18.094 INFO:tasks.workunit.client.0.smithi102.stdout:5/885: dread de/d25/d2a/def/f106 [0,4194304] 0 2024-05-13T22:48:18.095 INFO:tasks.workunit.client.0.smithi102.stdout:7/589: rmdir d6/d15/d4c 39 2024-05-13T22:48:18.096 INFO:tasks.workunit.client.0.smithi102.stdout:5/886: write de/d25/d2a/d4b/fc1 [1506699,10353] 0 2024-05-13T22:48:18.097 INFO:tasks.workunit.client.0.smithi102.stdout:2/784: dread d7/d9/d11/f39 [0,4194304] 0 2024-05-13T22:48:18.104 INFO:tasks.workunit.client.0.smithi102.stdout:2/785: mkdir d7/d9/d14/d61/d102 0 2024-05-13T22:48:18.104 INFO:tasks.workunit.client.0.smithi102.stdout:4/769: dread d3/d4/f47 [0,4194304] 0 2024-05-13T22:48:18.106 INFO:tasks.workunit.client.0.smithi102.stdout:2/786: truncate d7/d15/d1e/d87/fcd 655908 0 2024-05-13T22:48:18.106 INFO:tasks.workunit.client.0.smithi102.stdout:2/787: stat d7/d9/d11 0 2024-05-13T22:48:18.107 INFO:tasks.workunit.client.0.smithi102.stdout:2/788: write d7/d15/d1e/dbf/fe9 [499266,51326] 0 2024-05-13T22:48:18.111 INFO:tasks.workunit.client.0.smithi102.stdout:7/590: mknod d6/d13/d58/caf 0 2024-05-13T22:48:18.112 INFO:tasks.workunit.client.0.smithi102.stdout:4/770: dread - d3/d22/fce zero size 2024-05-13T22:48:18.113 INFO:tasks.workunit.client.0.smithi102.stdout:4/771: chown d3/d19/d23/d4e/d68/cff 0 1 2024-05-13T22:48:18.116 INFO:tasks.workunit.client.0.smithi102.stdout:5/887: write de/d25/d2a/d4b/d4d/d50/da3/df4/f98 [3393752,78084] 0 2024-05-13T22:48:18.118 INFO:tasks.workunit.client.0.smithi102.stdout:4/772: symlink d3/d19/d23/d4e/d68/d85/l10c 0 2024-05-13T22:48:18.120 INFO:tasks.workunit.client.0.smithi102.stdout:2/789: creat d7/d9/d11/f103 x:0 0 0 2024-05-13T22:48:18.122 INFO:tasks.workunit.client.0.smithi102.stdout:5/888: fdatasync de/d25/d2a/d4b/d4d/d50/da3/df4/f66 0 2024-05-13T22:48:18.130 INFO:tasks.workunit.client.0.smithi102.stdout:7/591: getdents d6/d13/de 0 2024-05-13T22:48:18.137 INFO:tasks.workunit.client.0.smithi102.stdout:2/790: link d7/d9/d14/d31/f97 d7/d9/d14/d61/d82/f104 0 2024-05-13T22:48:18.138 INFO:tasks.workunit.client.0.smithi102.stdout:7/592: stat d6/d13/c10 0 2024-05-13T22:48:18.139 INFO:tasks.workunit.client.0.smithi102.stdout:4/773: dwrite d3/d19/d23/d39/da7/d7d/fc5 [0,4194304] 0 2024-05-13T22:48:18.147 INFO:tasks.workunit.client.0.smithi102.stdout:5/889: dwrite de/d25/d2a/d4b/d4d/d50/da3/df4/de4/f142 [0,4194304] 0 2024-05-13T22:48:18.149 INFO:tasks.workunit.client.0.smithi102.stdout:7/593: symlink d6/d13/d2c/d39/lb0 0 2024-05-13T22:48:18.152 INFO:tasks.workunit.client.0.smithi102.stdout:7/594: write d6/d15/d4c/d90/d4b/fa7 [673613,1890] 0 2024-05-13T22:48:18.154 INFO:tasks.workunit.client.0.smithi102.stdout:5/890: mknod de/d25/d2a/d4b/d4d/d114/dda/c147 0 2024-05-13T22:48:18.155 INFO:tasks.workunit.client.0.smithi102.stdout:7/595: mknod d6/d13/de/cb1 0 2024-05-13T22:48:18.157 INFO:tasks.workunit.client.0.smithi102.stdout:7/596: dread - d6/d15/d4d/d55/d6f/d65/f6b zero size 2024-05-13T22:48:18.157 INFO:tasks.workunit.client.0.smithi102.stdout:5/891: creat de/d25/d58/dfb/f148 x:0 0 0 2024-05-13T22:48:18.162 INFO:tasks.workunit.client.0.smithi102.stdout:7/597: rename d6/d15/d4d/f93 to d6/d15/d4d/d55/d6f/d49/fb2 0 2024-05-13T22:48:18.163 INFO:tasks.workunit.client.0.smithi102.stdout:5/892: creat de/d25/d2a/d4b/d132/f149 x:0 0 0 2024-05-13T22:48:18.167 INFO:tasks.workunit.client.0.smithi102.stdout:7/598: mkdir d6/d13/d2c/d39/db3 0 2024-05-13T22:48:18.172 INFO:tasks.workunit.client.0.smithi102.stdout:2/791: truncate d7/d9/d10/d25/d56/f8c 794444 0 2024-05-13T22:48:18.173 INFO:tasks.workunit.client.0.smithi102.stdout:2/792: chown d7/d15/fa5 1278 1 2024-05-13T22:48:18.174 INFO:tasks.workunit.client.0.smithi102.stdout:4/774: truncate d3/d19/d23/d4e/d68/fe6 2821387 0 2024-05-13T22:48:18.174 INFO:tasks.workunit.client.0.smithi102.stdout:7/599: dwrite d6/d15/f18 [0,4194304] 0 2024-05-13T22:48:18.178 INFO:tasks.workunit.client.0.smithi102.stdout:7/600: fsync d6/d15/d4c/d90/d4b/f5f 0 2024-05-13T22:48:18.183 INFO:tasks.workunit.client.0.smithi102.stdout:2/793: dwrite d7/d9/d14/d61/d82/ff0 [0,4194304] 0 2024-05-13T22:48:18.187 INFO:tasks.workunit.client.0.smithi102.stdout:7/601: rename d6/d13/d2c/da1/laa to d6/d15/d4d/lb4 0 2024-05-13T22:48:18.191 INFO:tasks.workunit.client.0.smithi102.stdout:5/893: write de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d53/f107 [162638,70932] 0 2024-05-13T22:48:18.193 INFO:tasks.workunit.client.0.smithi102.stdout:2/794: creat d7/d9/d11/f105 x:0 0 0 2024-05-13T22:48:18.193 INFO:tasks.workunit.client.0.smithi102.stdout:2/795: readlink d7/lb4 0 2024-05-13T22:48:18.202 INFO:tasks.workunit.client.0.smithi102.stdout:4/775: dwrite d3/d19/d23/d4e/d68/f94 [0,4194304] 0 2024-05-13T22:48:18.235 INFO:tasks.workunit.client.0.smithi102.stdout:0/727: sync 2024-05-13T22:48:18.236 INFO:tasks.workunit.client.0.smithi102.stdout:0/728: chown d1/de/c26 2 1 2024-05-13T22:48:18.236 INFO:tasks.workunit.client.0.smithi102.stdout:2/796: unlink d7/l4f 0 2024-05-13T22:48:18.237 INFO:tasks.workunit.client.0.smithi102.stdout:5/894: dread de/d25/d59/f32 [4194304,4194304] 0 2024-05-13T22:48:18.365 INFO:tasks.workunit.client.0.smithi102.stdout:5/895: creat de/d25/d2a/d4b/d4d/d114/f14a x:0 0 0 2024-05-13T22:48:18.366 INFO:tasks.workunit.client.0.smithi102.stdout:7/602: truncate d6/f85 735380 0 2024-05-13T22:48:18.370 INFO:tasks.workunit.client.0.smithi102.stdout:0/729: dwrite d1/de/d32/d24/d2d/f64 [0,4194304] 0 2024-05-13T22:48:18.370 INFO:tasks.workunit.client.0.smithi102.stdout:2/797: dwrite d7/d15/d67/fa8 [4194304,4194304] 0 2024-05-13T22:48:18.370 INFO:tasks.workunit.client.0.smithi102.stdout:5/896: chown de/d25/d59/d7f/d86/da5/fd9 484615 1 2024-05-13T22:48:18.373 INFO:tasks.workunit.client.0.smithi102.stdout:5/897: chown de/d25/d59/d6f/f92 0 1 2024-05-13T22:48:18.374 INFO:tasks.workunit.client.0.smithi102.stdout:7/603: creat d6/da9/fb5 x:0 0 0 2024-05-13T22:48:18.377 INFO:tasks.workunit.client.0.smithi102.stdout:2/798: creat d7/d9/d10/d30/d53/f106 x:0 0 0 2024-05-13T22:48:18.384 INFO:tasks.workunit.client.0.smithi102.stdout:7/604: mknod d6/da9/cb6 0 2024-05-13T22:48:18.385 INFO:tasks.workunit.client.0.smithi102.stdout:7/605: chown d6/d13/de/l41 43455005 1 2024-05-13T22:48:18.386 INFO:tasks.workunit.client.0.smithi102.stdout:4/776: link d3/d19/d23/d39/d77/cb6 d3/d4/d6/de/d1f/c10d 0 2024-05-13T22:48:18.395 INFO:tasks.workunit.client.0.smithi102.stdout:2/799: symlink d7/d9/d10/d25/d27/dab/dee/l107 0 2024-05-13T22:48:18.395 INFO:tasks.workunit.client.0.smithi102.stdout:2/800: chown d7/d15/d1e/dbf 0 1 2024-05-13T22:48:18.397 INFO:tasks.workunit.client.0.smithi102.stdout:7/606: creat d6/d15/d4d/d55/d6f/d49/d95/fb7 x:0 0 0 2024-05-13T22:48:18.399 INFO:tasks.workunit.client.0.smithi102.stdout:2/801: fdatasync d7/ff 0 2024-05-13T22:48:18.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:18 smithi130 ceph-mon[36591]: pgmap v26: 65 pgs: 65 active+clean; 1.4 GiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 98 MiB/s wr, 313 op/s 2024-05-13T22:48:18.403 INFO:tasks.workunit.client.0.smithi102.stdout:7/607: mknod d6/d13/d2c/cb8 0 2024-05-13T22:48:18.406 INFO:tasks.workunit.client.0.smithi102.stdout:7/608: creat d6/d15/d4d/fb9 x:0 0 0 2024-05-13T22:48:18.412 INFO:tasks.workunit.client.0.smithi102.stdout:2/802: getdents d7/d9 0 2024-05-13T22:48:18.416 INFO:tasks.workunit.client.0.smithi102.stdout:6/793: sync 2024-05-13T22:48:18.418 INFO:tasks.workunit.client.0.smithi102.stdout:2/803: creat d7/d15/d1e/f108 x:0 0 0 2024-05-13T22:48:18.422 INFO:tasks.workunit.client.0.smithi102.stdout:2/804: symlink d7/d9/d10/d1b/l109 0 2024-05-13T22:48:18.424 INFO:tasks.workunit.client.0.smithi102.stdout:6/794: creat d9/db/d14/dcc/ffe x:0 0 0 2024-05-13T22:48:18.425 INFO:tasks.workunit.client.0.smithi102.stdout:6/795: read - d9/da3/fde zero size 2024-05-13T22:48:18.425 INFO:tasks.workunit.client.0.smithi102.stdout:6/796: stat d9/d45/d56/lca 0 2024-05-13T22:48:18.426 INFO:tasks.workunit.client.0.smithi102.stdout:2/805: chown d7/d15/d1e/da7/db5/ff6 181 1 2024-05-13T22:48:18.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:18 smithi102 ceph-mon[28599]: pgmap v26: 65 pgs: 65 active+clean; 1.4 GiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 98 MiB/s wr, 313 op/s 2024-05-13T22:48:18.431 INFO:tasks.workunit.client.0.smithi102.stdout:6/797: dwrite d9/da3/de8/ff4 [0,4194304] 0 2024-05-13T22:48:18.439 INFO:tasks.workunit.client.0.smithi102.stdout:6/798: stat d9/da3/lcb 0 2024-05-13T22:48:18.439 INFO:tasks.workunit.client.0.smithi102.stdout:6/799: dread - d9/db/d14/dcc/ffe zero size 2024-05-13T22:48:18.443 INFO:tasks.workunit.client.0.smithi102.stdout:6/800: mkdir d9/d25/d60/d6a/dd6/dff 0 2024-05-13T22:48:18.445 INFO:tasks.workunit.client.0.smithi102.stdout:6/801: fsync d9/d45/d56/f9b 0 2024-05-13T22:48:18.445 INFO:tasks.workunit.client.0.smithi102.stdout:2/806: dwrite d7/d9/d10/d1b/d54/d83/fed [4194304,4194304] 0 2024-05-13T22:48:18.456 INFO:tasks.workunit.client.0.smithi102.stdout:2/807: write d7/d9/d10/d1b/d54/ff7 [627627,89777] 0 2024-05-13T22:48:18.561 INFO:tasks.workunit.client.0.smithi102.stdout:2/808: getdents d7/d15/d1e/da7/db5 0 2024-05-13T22:48:18.565 INFO:tasks.workunit.client.0.smithi102.stdout:2/809: mknod d7/d15/d1e/c10a 0 2024-05-13T22:48:18.571 INFO:tasks.workunit.client.0.smithi102.stdout:2/810: dwrite d7/d15/d67/f70 [0,4194304] 0 2024-05-13T22:48:18.571 INFO:tasks.workunit.client.0.smithi102.stdout:2/811: readlink d7/d9/d10/d30/d53/l72 0 2024-05-13T22:48:18.574 INFO:tasks.workunit.client.0.smithi102.stdout:2/812: creat d7/d9/d10/d30/d7f/f10b x:0 0 0 2024-05-13T22:48:18.578 INFO:tasks.workunit.client.0.smithi102.stdout:2/813: symlink d7/d9/d10/d1b/d54/d83/l10c 0 2024-05-13T22:48:18.580 INFO:tasks.workunit.client.0.smithi102.stdout:2/814: write d7/d9/d14/d31/dbb/ffa [533042,87995] 0 2024-05-13T22:48:18.599 INFO:tasks.workunit.client.0.smithi102.stdout:5/898: dwrite de/d25/d43/d7a/fae [0,4194304] 0 2024-05-13T22:48:18.602 INFO:tasks.workunit.client.0.smithi102.stdout:5/899: read - de/d25/d2a/def/f124 zero size 2024-05-13T22:48:18.604 INFO:tasks.workunit.client.0.smithi102.stdout:4/777: dwrite d3/d4/f93 [0,4194304] 0 2024-05-13T22:48:18.606 INFO:tasks.workunit.client.0.smithi102.stdout:5/900: chown de/d25/d59/d7f/d86/da5/ddb/dee/d116/f123 24 1 2024-05-13T22:48:18.610 INFO:tasks.workunit.client.0.smithi102.stdout:5/901: mkdir de/d25/d2a/d4b/d14b 0 2024-05-13T22:48:18.614 INFO:tasks.workunit.client.0.smithi102.stdout:7/609: dwrite d6/d22/f24 [0,4194304] 0 2024-05-13T22:48:18.614 INFO:tasks.workunit.client.0.smithi102.stdout:5/902: mkdir de/d25/d2a/d4b/d4d/d50/da3/d14c 0 2024-05-13T22:48:18.615 INFO:tasks.workunit.client.0.smithi102.stdout:7/610: dread - d6/d15/f7e zero size 2024-05-13T22:48:18.617 INFO:tasks.workunit.client.0.smithi102.stdout:5/903: chown de/d25/d2a/d4b/d4d/d50/d6c/df8/ce5 249804 1 2024-05-13T22:48:18.620 INFO:tasks.workunit.client.0.smithi102.stdout:7/611: unlink d6/d15/d4d/d55/d6f/d65/f6b 0 2024-05-13T22:48:18.623 INFO:tasks.workunit.client.0.smithi102.stdout:7/612: mknod d6/d13/d2c/da1/cba 0 2024-05-13T22:48:18.623 INFO:tasks.workunit.client.0.smithi102.stdout:7/613: stat d6/d13/d2c 0 2024-05-13T22:48:18.626 INFO:tasks.workunit.client.0.smithi102.stdout:5/904: rename de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d53/ld2 to de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/l14d 0 2024-05-13T22:48:18.628 INFO:tasks.workunit.client.0.smithi102.stdout:7/614: creat d6/d15/d4d/d55/d6f/d49/fbb x:0 0 0 2024-05-13T22:48:18.629 INFO:tasks.workunit.client.0.smithi102.stdout:5/905: truncate de/d25/d59/f1c 4148799 0 2024-05-13T22:48:18.630 INFO:tasks.workunit.client.0.smithi102.stdout:7/615: symlink d6/d15/d4c/d90/d4b/lbc 0 2024-05-13T22:48:18.630 INFO:tasks.workunit.client.0.smithi102.stdout:5/906: fsync de/d25/d2a/d4b/d4d/d50/da3/df4/f98 0 2024-05-13T22:48:18.631 INFO:tasks.workunit.client.0.smithi102.stdout:7/616: write d6/d13/d2c/f3a [4743986,84344] 0 2024-05-13T22:48:18.633 INFO:tasks.workunit.client.0.smithi102.stdout:7/617: fsync d6/d13/de/f12 0 2024-05-13T22:48:18.641 INFO:tasks.workunit.client.0.smithi102.stdout:6/802: write d9/db/d14/d35/fc2 [691702,53433] 0 2024-05-13T22:48:18.643 INFO:tasks.workunit.client.0.smithi102.stdout:6/803: mkdir d9/db/d100 0 2024-05-13T22:48:18.645 INFO:tasks.workunit.client.0.smithi102.stdout:6/804: write d9/db/d14/d35/dc9/df1/fcd [1729789,75105] 0 2024-05-13T22:48:18.678 INFO:tasks.workunit.client.0.smithi102.stdout:2/815: truncate d7/d9/d14/f36 6818458 0 2024-05-13T22:48:18.679 INFO:tasks.workunit.client.0.smithi102.stdout:2/816: dread - d7/d9/d11/f103 zero size 2024-05-13T22:48:18.683 INFO:tasks.workunit.client.0.smithi102.stdout:4/778: write d3/d22/f78 [3340546,86973] 0 2024-05-13T22:48:18.687 INFO:tasks.workunit.client.0.smithi102.stdout:2/817: dwrite d7/d9/d10/d30/d53/f106 [0,4194304] 0 2024-05-13T22:48:18.803 INFO:tasks.workunit.client.1.smithi130.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-13T22:48:18.807 INFO:tasks.workunit.client.1.smithi130.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-13T22:48:18.807 INFO:tasks.workunit.client.1.smithi130.stderr:+ make 2024-05-13T22:48:18.839 INFO:tasks.workunit.client.1.smithi130.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-13T22:48:18.869 INFO:tasks.workunit.client.0.smithi102.stdout:7/618: truncate d6/d13/d2c/f3a 2173042 0 2024-05-13T22:48:18.869 INFO:tasks.workunit.client.0.smithi102.stdout:6/805: write d9/d25/d71/f7a [376525,124777] 0 2024-05-13T22:48:18.869 INFO:tasks.workunit.client.0.smithi102.stdout:7/619: chown d6/d13/d2c 105356560 1 2024-05-13T22:48:18.869 INFO:tasks.workunit.client.0.smithi102.stdout:2/818: write d7/d9/d10/fe7 [106558,13766] 0 2024-05-13T22:48:18.870 INFO:tasks.workunit.client.0.smithi102.stdout:4/779: creat d3/d19/d23/d39/da7/da9/dd3/f10e x:0 0 0 2024-05-13T22:48:18.875 INFO:tasks.workunit.client.0.smithi102.stdout:2/819: dwrite d7/d9/f89 [0,4194304] 0 2024-05-13T22:48:18.876 INFO:tasks.workunit.client.0.smithi102.stdout:7/620: dwrite d6/d15/d4c/f98 [0,4194304] 0 2024-05-13T22:48:18.922 INFO:tasks.workunit.client.0.smithi102.stdout:2/820: write d7/d15/fad [112405,33438] 0 2024-05-13T22:48:18.922 INFO:tasks.workunit.client.0.smithi102.stdout:6/806: creat d9/db/d14/d35/d47/dc1/f101 x:0 0 0 2024-05-13T22:48:18.927 INFO:tasks.workunit.client.0.smithi102.stdout:2/821: truncate d7/d9/d14/d61/d82/f104 828924 0 2024-05-13T22:48:18.931 INFO:tasks.workunit.client.0.smithi102.stdout:7/621: dread d6/d15/d4d/d55/d6f/d65/f86 [0,4194304] 0 2024-05-13T22:48:18.932 INFO:tasks.workunit.client.0.smithi102.stdout:2/822: creat d7/d15/d1e/da7/f10d x:0 0 0 2024-05-13T22:48:18.932 INFO:tasks.workunit.client.0.smithi102.stdout:6/807: dwrite d9/db/f50 [0,4194304] 0 2024-05-13T22:48:18.934 INFO:tasks.workunit.client.0.smithi102.stdout:6/808: stat d9/d45/d56/dd5/c99 0 2024-05-13T22:48:18.935 INFO:tasks.workunit.client.0.smithi102.stdout:2/823: creat d7/d9/d14/d31/dbb/f10e x:0 0 0 2024-05-13T22:48:18.937 INFO:tasks.workunit.client.0.smithi102.stdout:2/824: write d7/d15/d1e/f108 [560161,89262] 0 2024-05-13T22:48:18.938 INFO:tasks.workunit.client.0.smithi102.stdout:2/825: chown d7/d9/d14 1 1 2024-05-13T22:48:18.941 INFO:tasks.workunit.client.0.smithi102.stdout:2/826: truncate d7/d9/d11/f105 649906 0 2024-05-13T22:48:18.942 INFO:tasks.workunit.client.0.smithi102.stdout:7/622: getdents d6/d15/d4c/d90/d4b 0 2024-05-13T22:48:18.944 INFO:tasks.workunit.client.0.smithi102.stdout:6/809: creat d9/db/f102 x:0 0 0 2024-05-13T22:48:18.945 INFO:tasks.workunit.client.0.smithi102.stdout:4/780: dwrite d3/d19/d23/d4e/f76 [4194304,4194304] 0 2024-05-13T22:48:18.945 INFO:tasks.workunit.client.0.smithi102.stdout:2/827: mknod d7/d9/d14/d31/dbb/c10f 0 2024-05-13T22:48:18.954 INFO:tasks.workunit.client.0.smithi102.stdout:6/810: link d9/d25/d71/fb3 d9/db/d14/f103 0 2024-05-13T22:48:18.954 INFO:tasks.workunit.client.0.smithi102.stdout:2/828: dwrite d7/d9/f89 [4194304,4194304] 0 2024-05-13T22:48:18.956 INFO:tasks.workunit.client.0.smithi102.stdout:7/623: getdents d6/d15/d4d/d55/d6f/d65 0 2024-05-13T22:48:18.958 INFO:tasks.workunit.client.0.smithi102.stdout:4/781: write d3/d4/d6/de/d1f/f28 [1583906,77876] 0 2024-05-13T22:48:18.958 INFO:tasks.workunit.client.0.smithi102.stdout:2/829: readlink d7/d9/d10/d1b/d54/l57 0 2024-05-13T22:48:18.963 INFO:tasks.workunit.client.0.smithi102.stdout:4/782: unlink d3/d19/d23/d39/fe7 0 2024-05-13T22:48:18.963 INFO:tasks.workunit.client.0.smithi102.stdout:6/811: dwrite d9/d45/d56/f9b [4194304,4194304] 0 2024-05-13T22:48:18.965 INFO:tasks.workunit.client.0.smithi102.stdout:4/783: dread - d3/ff0 zero size 2024-05-13T22:48:18.972 INFO:tasks.workunit.client.0.smithi102.stdout:2/830: truncate d7/d9/fcf 746434 0 2024-05-13T22:48:18.973 INFO:tasks.workunit.client.0.smithi102.stdout:7/624: dread d6/d15/d4c/d90/d4b/f5f [0,4194304] 0 2024-05-13T22:48:18.975 INFO:tasks.workunit.client.0.smithi102.stdout:6/812: dwrite d9/f1a [0,4194304] 0 2024-05-13T22:48:18.975 INFO:tasks.workunit.client.0.smithi102.stdout:4/784: dwrite d3/d22/f78 [0,4194304] 0 2024-05-13T22:48:18.981 INFO:tasks.workunit.client.0.smithi102.stdout:2/831: truncate d7/d9/d14/f73 548359 0 2024-05-13T22:48:18.984 INFO:tasks.workunit.client.0.smithi102.stdout:4/785: unlink d3/d22/lfd 0 2024-05-13T22:48:18.985 INFO:tasks.workunit.client.0.smithi102.stdout:2/832: unlink d7/d9/d10/d25/d56/c69 0 2024-05-13T22:48:18.987 INFO:tasks.workunit.client.0.smithi102.stdout:7/625: link d6/d13/f16 d6/da9/fbd 0 2024-05-13T22:48:18.989 INFO:tasks.workunit.client.0.smithi102.stdout:4/786: unlink d3/d4/f5a 0 2024-05-13T22:48:18.999 INFO:tasks.workunit.client.0.smithi102.stdout:6/813: link d9/db/d14/dcc/c68 d9/db/d14/d35/dc9/c104 0 2024-05-13T22:48:19.001 INFO:tasks.workunit.client.0.smithi102.stdout:6/814: symlink d9/d25/d60/d6a/dd6/l105 0 2024-05-13T22:48:19.003 INFO:tasks.workunit.client.0.smithi102.stdout:4/787: link d3/d19/d23/d39/ddd/fe0 d3/d19/d23/d3a/da4/f10f 0 2024-05-13T22:48:19.004 INFO:tasks.workunit.client.0.smithi102.stdout:6/815: symlink d9/db/d14/d35/dd1/l106 0 2024-05-13T22:48:19.008 INFO:tasks.workunit.client.0.smithi102.stdout:6/816: creat d9/d25/d60/f107 x:0 0 0 2024-05-13T22:48:19.009 INFO:tasks.workunit.client.0.smithi102.stdout:6/817: write d9/db/d14/d35/fc5 [546398,122285] 0 2024-05-13T22:48:19.010 INFO:tasks.workunit.client.0.smithi102.stdout:2/833: rmdir d7 39 2024-05-13T22:48:19.010 INFO:tasks.workunit.client.0.smithi102.stdout:6/818: chown d9/db/d14/d35/fc2 21579 1 2024-05-13T22:48:19.014 INFO:tasks.workunit.client.0.smithi102.stdout:4/788: link d3/d4/d6/l13 d3/d19/d23/d4e/d68/l110 0 2024-05-13T22:48:19.017 INFO:tasks.workunit.client.0.smithi102.stdout:2/834: creat d7/d9/d10/d25/d56/d59/d81/f110 x:0 0 0 2024-05-13T22:48:19.019 INFO:tasks.workunit.client.0.smithi102.stdout:2/835: chown d7/d9/d10/d1b/f6b 69437 1 2024-05-13T22:48:19.021 INFO:tasks.workunit.client.0.smithi102.stdout:6/819: dwrite d9/f21 [0,4194304] 0 2024-05-13T22:48:19.022 INFO:tasks.workunit.client.0.smithi102.stdout:2/836: chown d7/ff 147700 1 2024-05-13T22:48:19.022 INFO:tasks.workunit.client.0.smithi102.stdout:7/626: dwrite d6/d15/f5e [0,4194304] 0 2024-05-13T22:48:19.024 INFO:tasks.workunit.client.0.smithi102.stdout:7/627: readlink d6/l1c 0 2024-05-13T22:48:19.028 INFO:tasks.workunit.client.0.smithi102.stdout:4/789: dwrite d3/d19/d23/d39/da7/da9/dd3/f10e [0,4194304] 0 2024-05-13T22:48:19.029 INFO:tasks.workunit.client.0.smithi102.stdout:7/628: symlink d6/d15/d4c/lbe 0 2024-05-13T22:48:19.033 INFO:tasks.workunit.client.0.smithi102.stdout:7/629: dread - d6/d22/f7b zero size 2024-05-13T22:48:19.034 INFO:tasks.workunit.client.0.smithi102.stdout:2/837: rename d7/d9/d14/d61/d82/f104 to d7/d9/d11/ddd/f111 0 2024-05-13T22:48:19.036 INFO:tasks.workunit.client.0.smithi102.stdout:6/820: dwrite d9/db/d14/d35/d47/dc1/f101 [0,4194304] 0 2024-05-13T22:48:19.037 INFO:tasks.workunit.client.0.smithi102.stdout:7/630: chown d6/d15/d4d/d55/d82/c84 1053 1 2024-05-13T22:48:19.040 INFO:tasks.workunit.client.0.smithi102.stdout:4/790: fsync d3/d19/d23/d39/da7/fb3 0 2024-05-13T22:48:19.041 INFO:tasks.workunit.client.0.smithi102.stdout:6/821: mknod d9/d45/d56/dd5/c108 0 2024-05-13T22:48:19.044 INFO:tasks.workunit.client.0.smithi102.stdout:7/631: dread d6/d15/f5e [0,4194304] 0 2024-05-13T22:48:19.045 INFO:tasks.workunit.client.0.smithi102.stdout:7/632: chown d6/d15/f1e 482515 1 2024-05-13T22:48:19.049 INFO:tasks.workunit.client.0.smithi102.stdout:7/633: creat d6/d15/d4c/d90/d4b/fbf x:0 0 0 2024-05-13T22:48:19.049 INFO:tasks.workunit.client.0.smithi102.stdout:7/634: chown d6/d15/d4d/d55/d6f/l73 8 1 2024-05-13T22:48:19.051 INFO:tasks.workunit.client.0.smithi102.stdout:7/635: write d6/d15/d4d/d55/f74 [330184,109104] 0 2024-05-13T22:48:19.052 INFO:tasks.workunit.client.0.smithi102.stdout:7/636: write d6/da9/fb5 [121418,56187] 0 2024-05-13T22:48:19.053 INFO:tasks.workunit.client.0.smithi102.stdout:7/637: fsync d6/d15/d4d/f9f 0 2024-05-13T22:48:19.056 INFO:tasks.workunit.client.0.smithi102.stdout:7/638: link d6/d15/d4c/cad d6/d15/d4d/d55/d6f/cc0 0 2024-05-13T22:48:19.057 INFO:tasks.workunit.client.0.smithi102.stdout:7/639: chown d6/d13/d2c/d39 185624620 1 2024-05-13T22:48:19.057 INFO:tasks.workunit.client.0.smithi102.stdout:7/640: fsync d6/d22/f7b 0 2024-05-13T22:48:19.063 INFO:tasks.workunit.client.0.smithi102.stdout:7/641: dwrite d6/d15/f18 [0,4194304] 0 2024-05-13T22:48:19.067 INFO:tasks.workunit.client.0.smithi102.stdout:2/838: write d7/d9/f29 [1599041,75765] 0 2024-05-13T22:48:19.067 INFO:tasks.workunit.client.0.smithi102.stdout:4/791: write d3/d4/f47 [1783863,97118] 0 2024-05-13T22:48:19.068 INFO:tasks.workunit.client.0.smithi102.stdout:6/822: write d9/db/d14/d35/d47/f70 [2480187,8003] 0 2024-05-13T22:48:19.071 INFO:tasks.workunit.client.0.smithi102.stdout:2/839: chown d7/d9/d10/d30 146542 1 2024-05-13T22:48:19.072 INFO:tasks.workunit.client.0.smithi102.stdout:6/823: symlink d9/d45/da0/l109 0 2024-05-13T22:48:19.073 INFO:tasks.workunit.client.0.smithi102.stdout:7/642: dwrite d6/d22/f33 [0,4194304] 0 2024-05-13T22:48:19.073 INFO:tasks.workunit.client.0.smithi102.stdout:4/792: creat d3/d4/f111 x:0 0 0 2024-05-13T22:48:19.077 INFO:tasks.workunit.client.0.smithi102.stdout:4/793: mknod d3/d4/d5f/c112 0 2024-05-13T22:48:19.078 INFO:tasks.workunit.client.0.smithi102.stdout:2/840: rename d7/d9/d10/d1b/l51 to d7/d9/d10/d25/d56/d59/d81/dcc/l112 0 2024-05-13T22:48:19.079 INFO:tasks.workunit.client.0.smithi102.stdout:6/824: mknod d9/db/d14/dcc/c10a 0 2024-05-13T22:48:19.080 INFO:tasks.workunit.client.0.smithi102.stdout:6/825: chown d9/d25/d71 4804540 1 2024-05-13T22:48:19.080 INFO:tasks.workunit.client.0.smithi102.stdout:2/841: chown d7/d9/d10/d25/d56/d59/fb0 1 1 2024-05-13T22:48:19.082 INFO:tasks.workunit.client.0.smithi102.stdout:7/643: rename d6/d15/d4c/d90/d4b to d6/d13/d2c/da1/dc1 0 2024-05-13T22:48:19.084 INFO:tasks.workunit.client.0.smithi102.stdout:6/826: mknod d9/c10b 0 2024-05-13T22:48:19.086 INFO:tasks.workunit.client.0.smithi102.stdout:7/644: creat d6/d15/d4d/d55/d6f/d49/fc2 x:0 0 0 2024-05-13T22:48:19.087 INFO:tasks.workunit.client.0.smithi102.stdout:2/842: rmdir d7/d15/d67/df3 0 2024-05-13T22:48:19.091 INFO:tasks.workunit.client.0.smithi102.stdout:2/843: creat d7/d15/d1e/dbf/dfe/f113 x:0 0 0 2024-05-13T22:48:19.093 INFO:tasks.workunit.client.0.smithi102.stdout:6/827: dwrite d9/db/d14/d35/d47/f70 [0,4194304] 0 2024-05-13T22:48:19.101 INFO:tasks.workunit.client.0.smithi102.stdout:6/828: rmdir d9/db 39 2024-05-13T22:48:19.104 INFO:tasks.workunit.client.0.smithi102.stdout:6/829: symlink d9/d25/d60/d6a/dd6/l10c 0 2024-05-13T22:48:19.106 INFO:tasks.workunit.client.0.smithi102.stdout:6/830: truncate d9/db/f102 930841 0 2024-05-13T22:48:19.106 INFO:tasks.workunit.client.0.smithi102.stdout:6/831: readlink d9/d45/da0/l109 0 2024-05-13T22:48:19.107 INFO:tasks.workunit.client.0.smithi102.stdout:7/645: write d6/d13/d2c/da1/dc1/f6c [146452,6781] 0 2024-05-13T22:48:19.108 INFO:tasks.workunit.client.0.smithi102.stdout:2/844: truncate d7/f75 3730749 0 2024-05-13T22:48:19.111 INFO:tasks.workunit.client.0.smithi102.stdout:2/845: creat d7/d9/d10/d30/d53/f114 x:0 0 0 2024-05-13T22:48:19.112 INFO:tasks.workunit.client.0.smithi102.stdout:6/832: dread d9/feb [0,4194304] 0 2024-05-13T22:48:19.112 INFO:tasks.workunit.client.0.smithi102.stdout:2/846: chown d7/d9/d10/d30/f80 85868 1 2024-05-13T22:48:19.115 INFO:tasks.workunit.client.0.smithi102.stdout:6/833: unlink d9/db/f7d 0 2024-05-13T22:48:19.117 INFO:tasks.workunit.client.0.smithi102.stdout:2/847: rename d7/d9/d10/fc1 to d7/d15/d1e/dbf/dfe/f115 0 2024-05-13T22:48:19.121 INFO:tasks.workunit.client.0.smithi102.stdout:6/834: dwrite d9/db/d14/dcc/f43 [4194304,4194304] 0 2024-05-13T22:48:19.123 INFO:tasks.workunit.client.0.smithi102.stdout:6/835: chown d9/db/d14/d35/dc9/df1/f62 112605088 1 2024-05-13T22:48:19.124 INFO:tasks.workunit.client.0.smithi102.stdout:6/836: write d9/d25/d5a/fe7 [615046,80159] 0 2024-05-13T22:48:19.124 INFO:tasks.workunit.client.0.smithi102.stdout:2/848: dread d7/d9/d10/d30/fa9 [0,4194304] 0 2024-05-13T22:48:19.127 INFO:tasks.workunit.client.0.smithi102.stdout:7/646: dread f4 [4194304,4194304] 0 2024-05-13T22:48:19.129 INFO:tasks.workunit.client.0.smithi102.stdout:2/849: rename d7/d15/d67/ldb to d7/d9/d14/d61/l116 0 2024-05-13T22:48:19.130 INFO:tasks.workunit.client.0.smithi102.stdout:7/647: mknod d6/d13/d2c/d39/cc3 0 2024-05-13T22:48:19.131 INFO:tasks.workunit.client.0.smithi102.stdout:7/648: fdatasync d6/d15/f3e 0 2024-05-13T22:48:19.131 INFO:tasks.workunit.client.0.smithi102.stdout:7/649: readlink d6/d13/d58/l48 0 2024-05-13T22:48:19.136 INFO:tasks.workunit.client.0.smithi102.stdout:2/850: dwrite d7/d9/d10/d1b/f8a [4194304,4194304] 0 2024-05-13T22:48:19.139 INFO:tasks.workunit.client.0.smithi102.stdout:2/851: readlink d7/d9/d7c/l100 0 2024-05-13T22:48:19.142 INFO:tasks.workunit.client.0.smithi102.stdout:2/852: symlink d7/d15/d1e/da7/l117 0 2024-05-13T22:48:19.146 INFO:tasks.workunit.client.0.smithi102.stdout:6/837: write d9/fbe [891093,57754] 0 2024-05-13T22:48:19.149 INFO:tasks.workunit.client.0.smithi102.stdout:7/650: dread d6/d13/d2c/f5c [0,4194304] 0 2024-05-13T22:48:19.149 INFO:tasks.workunit.client.0.smithi102.stdout:6/838: creat d9/da3/de8/f10d x:0 0 0 2024-05-13T22:48:19.150 INFO:tasks.workunit.client.0.smithi102.stdout:7/651: write d6/f94 [1196552,83129] 0 2024-05-13T22:48:19.158 INFO:tasks.workunit.client.0.smithi102.stdout:6/839: creat d9/db/d14/f10e x:0 0 0 2024-05-13T22:48:19.158 INFO:tasks.workunit.client.0.smithi102.stdout:6/840: creat d9/d25/d5a/f10f x:0 0 0 2024-05-13T22:48:19.158 INFO:tasks.workunit.client.0.smithi102.stdout:7/652: link d6/d15/d4d/d55/d6f/cc0 d6/d13/cc4 0 2024-05-13T22:48:19.158 INFO:tasks.workunit.client.0.smithi102.stdout:2/853: dwrite d7/d9/d14/d31/f64 [0,4194304] 0 2024-05-13T22:48:19.159 INFO:tasks.workunit.client.0.smithi102.stdout:7/653: read - d6/d15/f7e zero size 2024-05-13T22:48:19.160 INFO:tasks.workunit.client.0.smithi102.stdout:6/841: symlink d9/db/d14/d54/l110 0 2024-05-13T22:48:19.162 INFO:tasks.workunit.client.0.smithi102.stdout:2/854: creat d7/d9/d14/d61/d82/f118 x:0 0 0 2024-05-13T22:48:19.163 INFO:tasks.workunit.client.0.smithi102.stdout:7/654: creat d6/d15/d4c/fc5 x:0 0 0 2024-05-13T22:48:19.164 INFO:tasks.workunit.client.0.smithi102.stdout:6/842: mknod d9/d25/d60/d6a/dd6/c111 0 2024-05-13T22:48:19.164 INFO:tasks.workunit.client.0.smithi102.stdout:2/855: mknod d7/d9/d10/d30/d53/da4/c119 0 2024-05-13T22:48:19.167 INFO:tasks.workunit.client.0.smithi102.stdout:7/655: fsync d6/f85 0 2024-05-13T22:48:19.169 INFO:tasks.workunit.client.0.smithi102.stdout:2/856: unlink d7/d9/d10/d25/d27/cf9 0 2024-05-13T22:48:19.172 INFO:tasks.workunit.client.0.smithi102.stdout:2/857: mkdir d7/d15/d1e/da7/df4/d11a 0 2024-05-13T22:48:19.172 INFO:tasks.workunit.client.0.smithi102.stdout:7/656: link d6/d15/d4c/d90/l27 d6/d22/lc6 0 2024-05-13T22:48:19.174 INFO:tasks.workunit.client.0.smithi102.stdout:7/657: dread - d6/d15/d4c/f8c zero size 2024-05-13T22:48:19.175 INFO:tasks.workunit.client.0.smithi102.stdout:2/858: unlink d7/d15/d1e/d87/lea 0 2024-05-13T22:48:19.176 INFO:tasks.workunit.client.0.smithi102.stdout:7/658: write d6/d15/f5e [2078832,119407] 0 2024-05-13T22:48:19.178 INFO:tasks.workunit.client.0.smithi102.stdout:2/859: mknod d7/d9/d14/d61/d102/c11b 0 2024-05-13T22:48:19.180 INFO:tasks.workunit.client.0.smithi102.stdout:2/860: read - d7/d9/d10/d25/d56/d59/d81/f110 zero size 2024-05-13T22:48:19.187 INFO:tasks.workunit.client.0.smithi102.stdout:6/843: write d9/fec [26318,88522] 0 2024-05-13T22:48:19.190 INFO:tasks.workunit.client.0.smithi102.stdout:6/844: symlink d9/db/d14/d35/dc9/l112 0 2024-05-13T22:48:19.192 INFO:tasks.workunit.client.0.smithi102.stdout:6/845: write d9/db/d14/dcc/d5f/f9e [1455372,46373] 0 2024-05-13T22:48:19.201 INFO:tasks.workunit.client.0.smithi102.stdout:7/659: dwrite d6/d22/f23 [0,4194304] 0 2024-05-13T22:48:19.202 INFO:tasks.workunit.client.0.smithi102.stdout:2/861: dwrite d7/d9/d10/d25/d27/f5b [0,4194304] 0 2024-05-13T22:48:19.203 INFO:tasks.workunit.client.0.smithi102.stdout:2/862: dread - d7/d9/d10/d30/d53/f114 zero size 2024-05-13T22:48:19.204 INFO:tasks.workunit.client.0.smithi102.stdout:7/660: chown d6/d15/d4d/d55/d6f/d65/f86 2 1 2024-05-13T22:48:19.211 INFO:tasks.workunit.client.0.smithi102.stdout:2/863: unlink d7/d9/d14/d61/f93 0 2024-05-13T22:48:19.212 INFO:tasks.workunit.client.0.smithi102.stdout:7/661: dread d6/d22/f33 [0,4194304] 0 2024-05-13T22:48:19.212 INFO:tasks.workunit.client.0.smithi102.stdout:2/864: chown d7/d9/d11/ddd/df8 11821201 1 2024-05-13T22:48:19.214 INFO:tasks.workunit.client.0.smithi102.stdout:2/865: chown d7/d9/d14/d61/l86 125997 1 2024-05-13T22:48:19.215 INFO:tasks.workunit.client.1.smithi130.stderr:++ readlink -f fsstress 2024-05-13T22:48:19.215 INFO:tasks.workunit.client.0.smithi102.stdout:6/846: dread d9/db/fa1 [0,4194304] 0 2024-05-13T22:48:19.218 INFO:tasks.workunit.client.1.smithi130.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-13T22:48:19.218 INFO:tasks.workunit.client.1.smithi130.stderr:+ popd 2024-05-13T22:48:19.219 INFO:tasks.workunit.client.1.smithi130.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-13T22:48:19.219 INFO:tasks.workunit.client.1.smithi130.stderr:+ popd 2024-05-13T22:48:19.220 INFO:tasks.workunit.client.0.smithi102.stdout:6/847: truncate d9/d25/d60/f7e 1073566 0 2024-05-13T22:48:19.220 INFO:tasks.workunit.client.1.smithi130.stdout:~/cephtest/mnt.1/client.1/tmp 2024-05-13T22:48:19.220 INFO:tasks.workunit.client.1.smithi130.stderr:++ mktemp -d -p . 2024-05-13T22:48:19.221 INFO:tasks.workunit.client.0.smithi102.stdout:6/848: chown d9/d45/d56/lca 569 1 2024-05-13T22:48:19.221 INFO:tasks.workunit.client.0.smithi102.stdout:2/866: link d7/d9/d10/d30/d53/dd1/fd2 d7/d9/d14/d31/dbb/f11c 0 2024-05-13T22:48:19.223 INFO:tasks.workunit.client.1.smithi130.stderr:+ T=./tmp.zP6VXwHdJc 2024-05-13T22:48:19.223 INFO:tasks.workunit.client.1.smithi130.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.zP6VXwHdJc -l 1 -n 1000 -p 10 -v 2024-05-13T22:48:19.223 INFO:tasks.workunit.client.0.smithi102.stdout:6/849: read - d9/d25/d60/d6a/dd6/fd8 zero size 2024-05-13T22:48:19.265 INFO:tasks.workunit.client.0.smithi102.stdout:9/718: sync 2024-05-13T22:48:19.268 INFO:tasks.workunit.client.0.smithi102.stdout:2/867: dwrite d7/d15/fa5 [0,4194304] 0 2024-05-13T22:48:19.268 INFO:tasks.workunit.client.0.smithi102.stdout:9/719: dread - d1/d6/d3b/ff6 zero size 2024-05-13T22:48:19.269 INFO:tasks.workunit.client.1.smithi130.stdout:seed = 1715921155 2024-05-13T22:48:19.270 INFO:tasks.workunit.client.0.smithi102.stdout:7/662: dwrite d6/d15/d4c/d90/f75 [0,4194304] 0 2024-05-13T22:48:19.272 INFO:tasks.workunit.client.0.smithi102.stdout:7/663: write d6/d15/d4d/f9f [642114,1910] 0 2024-05-13T22:48:19.273 INFO:tasks.workunit.client.0.smithi102.stdout:9/720: read d1/d6/d3b/d5b/fa2 [824190,32004] 0 2024-05-13T22:48:19.279 INFO:tasks.workunit.client.0.smithi102.stdout:2/868: rename d7/d9/d10/d30/l99 to d7/d15/d1e/d87/l11d 0 2024-05-13T22:48:19.280 INFO:tasks.workunit.client.0.smithi102.stdout:2/869: stat d7/d9/d10/d1b/f6b 0 2024-05-13T22:48:19.280 INFO:tasks.workunit.client.1.smithi130.stdout:0/0: dread - no filename 2024-05-13T22:48:19.281 INFO:tasks.workunit.client.0.smithi102.stdout:6/850: write d9/d25/d5a/fb9 [767993,98942] 0 2024-05-13T22:48:19.283 INFO:tasks.workunit.client.0.smithi102.stdout:7/664: dwrite d6/f94 [0,4194304] 0 2024-05-13T22:48:19.287 INFO:tasks.workunit.client.0.smithi102.stdout:2/870: truncate d7/d9/f49 1195915 0 2024-05-13T22:48:19.288 INFO:tasks.workunit.client.0.smithi102.stdout:2/871: fdatasync d7/f32 0 2024-05-13T22:48:19.290 INFO:tasks.workunit.client.0.smithi102.stdout:6/851: fsync d9/d45/f75 0 2024-05-13T22:48:19.294 INFO:tasks.workunit.client.0.smithi102.stdout:9/721: write d1/d6/d3b/fd2 [765592,109849] 0 2024-05-13T22:48:19.296 INFO:tasks.workunit.client.0.smithi102.stdout:7/665: truncate d6/f77 752957 0 2024-05-13T22:48:19.300 INFO:tasks.workunit.client.0.smithi102.stdout:2/872: creat d7/d9/d10/d30/d53/f11e x:0 0 0 2024-05-13T22:48:19.300 INFO:tasks.workunit.client.1.smithi130.stdout:0/1: mkdir d0 0 2024-05-13T22:48:19.301 INFO:tasks.workunit.client.1.smithi130.stdout:0/2: dwrite - no filename 2024-05-13T22:48:19.301 INFO:tasks.workunit.client.1.smithi130.stdout:2/0: read - no filename 2024-05-13T22:48:19.301 INFO:tasks.workunit.client.1.smithi130.stdout:2/1: write - no filename 2024-05-13T22:48:19.301 INFO:tasks.workunit.client.1.smithi130.stdout:2/2: dread - no filename 2024-05-13T22:48:19.301 INFO:tasks.workunit.client.1.smithi130.stdout:2/3: write - no filename 2024-05-13T22:48:19.301 INFO:tasks.workunit.client.1.smithi130.stdout:2/4: truncate - no filename 2024-05-13T22:48:19.304 INFO:tasks.workunit.client.0.smithi102.stdout:2/873: dread d7/d9/d10/d25/f68 [0,4194304] 0 2024-05-13T22:48:19.304 INFO:tasks.workunit.client.0.smithi102.stdout:9/722: fsync d1/f16 0 2024-05-13T22:48:19.305 INFO:tasks.workunit.client.0.smithi102.stdout:2/874: fsync d7/d9/d14/d31/dbb/f10e 0 2024-05-13T22:48:19.305 INFO:tasks.workunit.client.0.smithi102.stdout:6/852: read d9/db/d14/dcc/d5f/fa4 [3917977,114066] 0 2024-05-13T22:48:19.306 INFO:tasks.workunit.client.0.smithi102.stdout:7/666: symlink d6/d15/d4d/d55/d82/lc7 0 2024-05-13T22:48:19.312 INFO:tasks.workunit.client.0.smithi102.stdout:2/875: dwrite d7/d15/d67/f70 [0,4194304] 0 2024-05-13T22:48:19.321 INFO:tasks.workunit.client.0.smithi102.stdout:2/876: dwrite d7/ff [0,4194304] 0 2024-05-13T22:48:19.330 INFO:tasks.workunit.client.0.smithi102.stdout:2/877: rmdir d7/d15/d1e/dbf/dfe 39 2024-05-13T22:48:19.335 INFO:tasks.workunit.client.0.smithi102.stdout:2/878: stat d7/d9/d10/d25/d27/c9c 0 2024-05-13T22:48:19.337 INFO:tasks.workunit.client.0.smithi102.stdout:6/853: write d9/d25/d60/d6a/dd6/de9/df7/faf [1149564,84453] 0 2024-05-13T22:48:19.339 INFO:tasks.workunit.client.0.smithi102.stdout:2/879: dread d7/d9/d14/f36 [4194304,4194304] 0 2024-05-13T22:48:19.342 INFO:tasks.workunit.client.0.smithi102.stdout:6/854: dread d9/fbe [0,4194304] 0 2024-05-13T22:48:19.346 INFO:tasks.workunit.client.0.smithi102.stdout:9/723: dwrite d1/f2b [0,4194304] 0 2024-05-13T22:48:19.347 INFO:tasks.workunit.client.0.smithi102.stdout:7/667: truncate d6/d13/d2c/da1/dc1/f66 1851374 0 2024-05-13T22:48:19.348 INFO:tasks.workunit.client.0.smithi102.stdout:7/668: chown d6/d15/d4d/d55 4 1 2024-05-13T22:48:19.352 INFO:tasks.workunit.client.0.smithi102.stdout:2/880: rename d7/d9/d10/d25/d56/l6f to d7/d9/d14/d61/l11f 0 2024-05-13T22:48:19.355 INFO:tasks.workunit.client.0.smithi102.stdout:7/669: rename d6/d15/d4d/d55/d6f/d49/f71 to d6/d15/d4d/d55/d6f/d65/fc8 0 2024-05-13T22:48:19.356 INFO:tasks.workunit.client.0.smithi102.stdout:2/881: mkdir d7/d15/d120 0 2024-05-13T22:48:19.360 INFO:tasks.workunit.client.0.smithi102.stdout:9/724: creat d1/d6/db/d4a/ffc x:0 0 0 2024-05-13T22:48:19.363 INFO:tasks.workunit.client.0.smithi102.stdout:2/882: symlink d7/d9/d10/d1b/d54/l121 0 2024-05-13T22:48:19.365 INFO:tasks.workunit.client.0.smithi102.stdout:7/670: creat d6/d13/d58/fc9 x:0 0 0 2024-05-13T22:48:19.366 INFO:tasks.workunit.client.0.smithi102.stdout:7/671: chown d6/d15/f3e 174565769 1 2024-05-13T22:48:19.370 INFO:tasks.workunit.client.0.smithi102.stdout:7/672: symlink d6/d22/lca 0 2024-05-13T22:48:19.376 INFO:tasks.workunit.client.0.smithi102.stdout:2/883: creat d7/d15/d1e/dbf/f122 x:0 0 0 2024-05-13T22:48:19.380 INFO:tasks.workunit.client.0.smithi102.stdout:2/884: dread d7/d15/d67/f70 [0,4194304] 0 2024-05-13T22:48:19.381 INFO:tasks.workunit.client.0.smithi102.stdout:2/885: chown d7/d9/d10/d25/c79 3 1 2024-05-13T22:48:19.385 INFO:tasks.workunit.client.0.smithi102.stdout:2/886: mknod d7/d9/d14/d61/d82/c123 0 2024-05-13T22:48:19.385 INFO:tasks.workunit.client.0.smithi102.stdout:2/887: readlink d7/d9/l71 0 2024-05-13T22:48:19.389 INFO:tasks.workunit.client.0.smithi102.stdout:7/673: dread d6/d13/d2c/da1/dc1/f66 [0,4194304] 0 2024-05-13T22:48:19.390 INFO:tasks.workunit.client.0.smithi102.stdout:7/674: readlink d6/d15/d4d/d55/d6f/d49/l57 0 2024-05-13T22:48:19.390 INFO:tasks.workunit.client.0.smithi102.stdout:7/675: chown d6/d13/d2c/d39/lb0 455559709 1 2024-05-13T22:48:19.391 INFO:tasks.workunit.client.0.smithi102.stdout:7/676: write d6/d15/d4c/f98 [5678721,41516] 0 2024-05-13T22:48:19.392 INFO:tasks.workunit.client.0.smithi102.stdout:2/888: fsync d7/f75 0 2024-05-13T22:48:19.392 INFO:tasks.workunit.client.0.smithi102.stdout:7/677: read d6/d22/f23 [2964034,130461] 0 2024-05-13T22:48:19.401 INFO:tasks.workunit.client.0.smithi102.stdout:2/889: dwrite d7/d9/d10/d30/d7f/f10b [0,4194304] 0 2024-05-13T22:48:19.402 INFO:tasks.workunit.client.0.smithi102.stdout:9/725: write d1/d6/db/d30/d47/f49 [3804666,48934] 0 2024-05-13T22:48:19.402 INFO:tasks.workunit.client.0.smithi102.stdout:7/678: dwrite d6/d15/d4d/d55/d6f/d49/fbb [0,4194304] 0 2024-05-13T22:48:19.404 INFO:tasks.workunit.client.0.smithi102.stdout:2/890: read - d7/d9/d10/d30/d53/f11e zero size 2024-05-13T22:48:19.412 INFO:tasks.workunit.client.0.smithi102.stdout:2/891: rename d7/c94 to d7/d15/d1e/d87/c124 0 2024-05-13T22:48:19.414 INFO:tasks.workunit.client.0.smithi102.stdout:7/679: dwrite d6/d15/d4c/d90/d80/f8a [0,4194304] 0 2024-05-13T22:48:19.418 INFO:tasks.workunit.client.0.smithi102.stdout:2/892: rmdir d7/d9/d10/d25/d27 39 2024-05-13T22:48:19.423 INFO:tasks.workunit.client.0.smithi102.stdout:7/680: dwrite d6/d15/d4c/d90/d80/fae [0,4194304] 0 2024-05-13T22:48:19.425 INFO:tasks.workunit.client.0.smithi102.stdout:7/681: chown d6/d15/d4d/d55/d6f/d49/fc2 515491064 1 2024-05-13T22:48:19.434 INFO:tasks.workunit.client.0.smithi102.stdout:7/682: unlink d6/d15/d4d/d55/d6f/d49/l56 0 2024-05-13T22:48:19.434 INFO:tasks.workunit.client.0.smithi102.stdout:7/683: chown d6/d13/d58/f34 59352 1 2024-05-13T22:48:19.454 INFO:tasks.workunit.client.0.smithi102.stdout:7/684: dread d6/d15/d4d/d55/d6f/d49/fb2 [0,4194304] 0 2024-05-13T22:48:19.463 INFO:tasks.workunit.client.0.smithi102.stdout:2/893: dwrite d7/d15/d1e/f5c [0,4194304] 0 2024-05-13T22:48:19.465 INFO:tasks.workunit.client.0.smithi102.stdout:2/894: chown d7/d9/d10/f33 3583 1 2024-05-13T22:48:19.473 INFO:tasks.workunit.client.0.smithi102.stdout:2/895: creat d7/d15/d1e/f125 x:0 0 0 2024-05-13T22:48:19.476 INFO:tasks.workunit.client.0.smithi102.stdout:2/896: creat d7/d9/d14/d31/dbb/f126 x:0 0 0 2024-05-13T22:48:19.479 INFO:tasks.workunit.client.0.smithi102.stdout:2/897: write d7/d9/d14/d31/dbb/dd5/f101 [741871,16504] 0 2024-05-13T22:48:19.483 INFO:tasks.workunit.client.0.smithi102.stdout:7/685: write d6/d15/d4c/d90/f50 [3136711,62685] 0 2024-05-13T22:48:19.488 INFO:tasks.workunit.client.0.smithi102.stdout:7/686: mkdir d6/d13/d2c/dcb 0 2024-05-13T22:48:19.489 INFO:tasks.workunit.client.0.smithi102.stdout:7/687: chown d6/d22/f24 26 1 2024-05-13T22:48:19.492 INFO:tasks.workunit.client.1.smithi130.stdout:1/0: creat f0 x:0 0 0 2024-05-13T22:48:19.494 INFO:tasks.workunit.client.1.smithi130.stdout:3/0: link - no file 2024-05-13T22:48:19.495 INFO:tasks.workunit.client.0.smithi102.stdout:7/688: creat d6/fcc x:0 0 0 2024-05-13T22:48:19.497 INFO:tasks.workunit.client.1.smithi130.stdout:2/5: creat f0 x:0 0 0 2024-05-13T22:48:19.498 INFO:tasks.workunit.client.1.smithi130.stdout:4/0: read - no filename 2024-05-13T22:48:19.498 INFO:tasks.workunit.client.1.smithi130.stdout:4/1: rename - no filename 2024-05-13T22:48:19.499 INFO:tasks.workunit.client.1.smithi130.stdout:0/3: mknod d0/c1 0 2024-05-13T22:48:19.501 INFO:tasks.workunit.client.1.smithi130.stdout:2/6: mknod c1 0 2024-05-13T22:48:19.501 INFO:tasks.workunit.client.1.smithi130.stdout:2/7: readlink - no filename 2024-05-13T22:48:19.501 INFO:tasks.workunit.client.1.smithi130.stdout:2/8: dread - f0 zero size 2024-05-13T22:48:19.502 INFO:tasks.workunit.client.0.smithi102.stdout:7/689: rename d6/d13/de/cb1 to d6/d13/d2c/ccd 0 2024-05-13T22:48:19.504 INFO:tasks.workunit.client.1.smithi130.stdout:5/0: dwrite - no filename 2024-05-13T22:48:19.504 INFO:tasks.workunit.client.1.smithi130.stdout:5/1: chown . 25 1 2024-05-13T22:48:19.504 INFO:tasks.workunit.client.1.smithi130.stdout:5/2: readlink - no filename 2024-05-13T22:48:19.505 INFO:tasks.workunit.client.1.smithi130.stdout:3/1: creat f0 x:0 0 0 2024-05-13T22:48:19.506 INFO:tasks.workunit.client.0.smithi102.stdout:2/898: truncate d7/d9/d14/d61/fa1 666549 0 2024-05-13T22:48:19.507 INFO:tasks.workunit.client.0.smithi102.stdout:7/690: stat d6/d15/d4d/d55/d6f/d49/d95 0 2024-05-13T22:48:19.507 INFO:tasks.workunit.client.1.smithi130.stdout:0/4: creat d0/f2 x:0 0 0 2024-05-13T22:48:19.508 INFO:tasks.workunit.client.1.smithi130.stdout:0/5: dread - d0/f2 zero size 2024-05-13T22:48:19.508 INFO:tasks.workunit.client.1.smithi130.stdout:0/6: readlink - no filename 2024-05-13T22:48:19.509 INFO:tasks.workunit.client.1.smithi130.stdout:0/7: write d0/f2 [224199,15198] 0 2024-05-13T22:48:19.512 INFO:tasks.workunit.client.1.smithi130.stdout:0/8: dread d0/f2 [0,4194304] 0 2024-05-13T22:48:19.513 INFO:tasks.workunit.client.1.smithi130.stdout:0/9: write d0/f2 [113353,103929] 0 2024-05-13T22:48:19.513 INFO:tasks.workunit.client.1.smithi130.stdout:4/2: mknod c0 0 2024-05-13T22:48:19.516 INFO:tasks.workunit.client.1.smithi130.stdout:3/2: symlink l1 0 2024-05-13T22:48:19.517 INFO:tasks.workunit.client.1.smithi130.stdout:3/3: chown f0 1592042 1 2024-05-13T22:48:19.518 INFO:tasks.workunit.client.1.smithi130.stdout:3/4: chown f0 18 1 2024-05-13T22:48:19.518 INFO:tasks.workunit.client.1.smithi130.stdout:1/1: getdents . 0 2024-05-13T22:48:19.519 INFO:tasks.workunit.client.1.smithi130.stdout:3/5: truncate f0 368923 0 2024-05-13T22:48:19.519 INFO:tasks.workunit.client.1.smithi130.stdout:1/2: chown f0 324 1 2024-05-13T22:48:19.520 INFO:tasks.workunit.client.1.smithi130.stdout:0/10: dwrite d0/f2 [0,4194304] 0 2024-05-13T22:48:19.520 INFO:tasks.workunit.client.1.smithi130.stdout:1/3: truncate f0 320773 0 2024-05-13T22:48:19.521 INFO:tasks.workunit.client.1.smithi130.stdout:2/9: link f0 f2 0 2024-05-13T22:48:19.523 INFO:tasks.workunit.client.1.smithi130.stdout:7/0: dwrite - no filename 2024-05-13T22:48:19.523 INFO:tasks.workunit.client.1.smithi130.stdout:7/1: stat - no entries 2024-05-13T22:48:19.523 INFO:tasks.workunit.client.1.smithi130.stdout:7/2: stat - no entries 2024-05-13T22:48:19.523 INFO:tasks.workunit.client.1.smithi130.stdout:7/3: rmdir - no directory 2024-05-13T22:48:19.523 INFO:tasks.workunit.client.1.smithi130.stdout:0/11: dread d0/f2 [0,4194304] 0 2024-05-13T22:48:19.526 INFO:tasks.workunit.client.1.smithi130.stdout:4/3: creat f1 x:0 0 0 2024-05-13T22:48:19.531 INFO:tasks.workunit.client.1.smithi130.stdout:5/3: mknod c0 0 2024-05-13T22:48:19.532 INFO:tasks.workunit.client.1.smithi130.stdout:5/4: readlink - no filename 2024-05-13T22:48:19.535 INFO:tasks.workunit.client.1.smithi130.stdout:3/6: creat f2 x:0 0 0 2024-05-13T22:48:19.536 INFO:tasks.workunit.client.1.smithi130.stdout:3/7: write f0 [995550,26808] 0 2024-05-13T22:48:19.538 INFO:tasks.workunit.client.1.smithi130.stdout:1/4: mknod c1 0 2024-05-13T22:48:19.543 INFO:tasks.workunit.client.1.smithi130.stdout:8/0: rmdir - no directory 2024-05-13T22:48:19.545 INFO:tasks.workunit.client.1.smithi130.stdout:6/0: symlink l0 0 2024-05-13T22:48:19.545 INFO:tasks.workunit.client.1.smithi130.stdout:6/1: stat l0 0 2024-05-13T22:48:19.546 INFO:tasks.workunit.client.1.smithi130.stdout:6/2: read - no filename 2024-05-13T22:48:19.546 INFO:tasks.workunit.client.1.smithi130.stdout:6/3: write - no filename 2024-05-13T22:48:19.550 INFO:tasks.workunit.client.1.smithi130.stdout:5/5: rename c0 to c1 0 2024-05-13T22:48:19.551 INFO:tasks.workunit.client.1.smithi130.stdout:0/12: write d0/f2 [4970369,104073] 0 2024-05-13T22:48:19.551 INFO:tasks.workunit.client.1.smithi130.stdout:0/13: chown d0/c1 501937 1 2024-05-13T22:48:19.552 INFO:tasks.workunit.client.1.smithi130.stdout:3/8: creat f3 x:0 0 0 2024-05-13T22:48:19.556 INFO:tasks.workunit.client.1.smithi130.stdout:2/10: link f2 f3 0 2024-05-13T22:48:19.558 INFO:tasks.workunit.client.1.smithi130.stdout:9/0: dwrite - no filename 2024-05-13T22:48:19.559 INFO:tasks.workunit.client.1.smithi130.stdout:9/1: write - no filename 2024-05-13T22:48:19.559 INFO:tasks.workunit.client.1.smithi130.stdout:9/2: rename - no filename 2024-05-13T22:48:19.559 INFO:tasks.workunit.client.1.smithi130.stdout:9/3: dwrite - no filename 2024-05-13T22:48:19.559 INFO:tasks.workunit.client.1.smithi130.stdout:9/4: rename - no filename 2024-05-13T22:48:19.559 INFO:tasks.workunit.client.1.smithi130.stdout:9/5: chown . 1794153518 1 2024-05-13T22:48:19.559 INFO:tasks.workunit.client.1.smithi130.stdout:9/6: unlink - no file 2024-05-13T22:48:19.559 INFO:tasks.workunit.client.1.smithi130.stdout:9/7: chown . 62015955 1 2024-05-13T22:48:19.559 INFO:tasks.workunit.client.1.smithi130.stdout:9/8: write - no filename 2024-05-13T22:48:19.560 INFO:tasks.workunit.client.1.smithi130.stdout:7/4: symlink l0 0 2024-05-13T22:48:19.560 INFO:tasks.workunit.client.0.smithi102.stdout:1/594: sync 2024-05-13T22:48:19.560 INFO:tasks.workunit.client.0.smithi102.stdout:0/730: sync 2024-05-13T22:48:19.560 INFO:tasks.workunit.client.0.smithi102.stdout:8/691: sync 2024-05-13T22:48:19.560 INFO:tasks.workunit.client.0.smithi102.stdout:3/823: sync 2024-05-13T22:48:19.560 INFO:tasks.workunit.client.0.smithi102.stdout:5/907: sync 2024-05-13T22:48:19.561 INFO:tasks.workunit.client.1.smithi130.stdout:6/4: creat f1 x:0 0 0 2024-05-13T22:48:19.561 INFO:tasks.workunit.client.1.smithi130.stdout:6/5: truncate f1 550215 0 2024-05-13T22:48:19.576 INFO:tasks.workunit.client.1.smithi130.stdout:5/6: creat f2 x:0 0 0 2024-05-13T22:48:19.576 INFO:tasks.workunit.client.1.smithi130.stdout:5/7: rmdir - no directory 2024-05-13T22:48:19.576 INFO:tasks.workunit.client.1.smithi130.stdout:5/8: readlink - no filename 2024-05-13T22:48:19.577 INFO:tasks.workunit.client.0.smithi102.stdout:8/692: mknod d2/d7/d27/cd7 0 2024-05-13T22:48:19.577 INFO:tasks.workunit.client.1.smithi130.stdout:0/14: creat d0/f3 x:0 0 0 2024-05-13T22:48:19.577 INFO:tasks.workunit.client.1.smithi130.stdout:0/15: read - d0/f3 zero size 2024-05-13T22:48:19.578 INFO:tasks.workunit.client.0.smithi102.stdout:5/908: creat de/d25/d43/df5/f14e x:0 0 0 2024-05-13T22:48:19.578 INFO:tasks.workunit.client.0.smithi102.stdout:5/909: fdatasync de/d25/d2a/d4b/d4d/d50/d6c/df8/f13b 0 2024-05-13T22:48:19.595 INFO:tasks.workunit.client.1.smithi130.stdout:3/9: mknod c4 0 2024-05-13T22:48:19.595 INFO:tasks.workunit.client.1.smithi130.stdout:3/10: truncate f3 829114 0 2024-05-13T22:48:19.597 INFO:tasks.workunit.client.0.smithi102.stdout:3/824: rename de/d10/d2a/d33/lae to de/d10/d2a/d8c/da8/l110 0 2024-05-13T22:48:19.599 INFO:tasks.workunit.client.1.smithi130.stdout:2/11: mkdir d4 0 2024-05-13T22:48:19.600 INFO:tasks.workunit.client.0.smithi102.stdout:0/731: symlink d1/de/d32/d24/le1 0 2024-05-13T22:48:19.601 INFO:tasks.workunit.client.1.smithi130.stdout:8/1: creat f0 x:0 0 0 2024-05-13T22:48:19.601 INFO:tasks.workunit.client.1.smithi130.stdout:8/2: dread - f0 zero size 2024-05-13T22:48:19.603 INFO:tasks.workunit.client.0.smithi102.stdout:8/693: creat d2/d7/d27/d64/d80/dc8/d40/db2/fd8 x:0 0 0 2024-05-13T22:48:19.606 INFO:tasks.workunit.client.1.smithi130.stdout:7/5: symlink l1 0 2024-05-13T22:48:19.608 INFO:tasks.workunit.client.1.smithi130.stdout:6/6: unlink f1 0 2024-05-13T22:48:19.609 INFO:tasks.workunit.client.0.smithi102.stdout:8/694: dwrite d2/d7/d27/f3b [0,4194304] 0 2024-05-13T22:48:19.609 INFO:tasks.workunit.client.0.smithi102.stdout:8/695: chown d2/d7/d25 20650 1 2024-05-13T22:48:19.611 INFO:tasks.workunit.client.0.smithi102.stdout:8/696: chown d2/d36/d49/d5d/fbd 102196 1 2024-05-13T22:48:19.615 INFO:tasks.workunit.client.1.smithi130.stdout:0/16: creat d0/f4 x:0 0 0 2024-05-13T22:48:19.616 INFO:tasks.workunit.client.1.smithi130.stdout:3/11: creat f5 x:0 0 0 2024-05-13T22:48:19.616 INFO:tasks.workunit.client.1.smithi130.stdout:3/12: rmdir - no directory 2024-05-13T22:48:19.617 INFO:tasks.workunit.client.1.smithi130.stdout:3/13: fdatasync f2 0 2024-05-13T22:48:19.617 INFO:tasks.workunit.client.1.smithi130.stdout:3/14: dread - f2 zero size 2024-05-13T22:48:19.618 INFO:tasks.workunit.client.0.smithi102.stdout:3/825: fdatasync de/d10/d50/f66 0 2024-05-13T22:48:19.620 INFO:tasks.workunit.client.0.smithi102.stdout:1/595: mkdir d2/d4/d5/d19/dbe 0 2024-05-13T22:48:19.624 INFO:tasks.workunit.client.1.smithi130.stdout:9/9: symlink l0 0 2024-05-13T22:48:19.625 INFO:tasks.workunit.client.1.smithi130.stdout:9/10: stat l0 0 2024-05-13T22:48:19.625 INFO:tasks.workunit.client.1.smithi130.stdout:9/11: chown l0 565 1 2024-05-13T22:48:19.627 INFO:tasks.workunit.client.1.smithi130.stdout:7/6: mknod c2 0 2024-05-13T22:48:19.627 INFO:tasks.workunit.client.1.smithi130.stdout:7/7: stat l1 0 2024-05-13T22:48:19.628 INFO:tasks.workunit.client.0.smithi102.stdout:3/826: symlink de/d10/d34/d5d/l111 0 2024-05-13T22:48:19.631 INFO:tasks.workunit.client.0.smithi102.stdout:1/596: symlink d2/d10/d62/d7b/lbf 0 2024-05-13T22:48:19.632 INFO:tasks.workunit.client.1.smithi130.stdout:6/7: creat f2 x:0 0 0 2024-05-13T22:48:19.632 INFO:tasks.workunit.client.1.smithi130.stdout:6/8: readlink l0 0 2024-05-13T22:48:19.633 INFO:tasks.workunit.client.1.smithi130.stdout:4/4: getdents . 0 2024-05-13T22:48:19.633 INFO:tasks.workunit.client.1.smithi130.stdout:4/5: dread - f1 zero size 2024-05-13T22:48:19.634 INFO:tasks.workunit.client.0.smithi102.stdout:8/697: creat d2/d7/d27/dbb/dc7/fd9 x:0 0 0 2024-05-13T22:48:19.635 INFO:tasks.workunit.client.0.smithi102.stdout:3/827: creat de/d10/d50/d8f/f112 x:0 0 0 2024-05-13T22:48:19.636 INFO:tasks.workunit.client.1.smithi130.stdout:0/17: creat d0/f5 x:0 0 0 2024-05-13T22:48:19.637 INFO:tasks.workunit.client.1.smithi130.stdout:0/18: truncate d0/f3 1047940 0 2024-05-13T22:48:19.638 INFO:tasks.workunit.client.1.smithi130.stdout:0/19: read - d0/f4 zero size 2024-05-13T22:48:19.639 INFO:tasks.workunit.client.1.smithi130.stdout:6/9: dwrite f2 [0,4194304] 0 2024-05-13T22:48:19.640 INFO:tasks.workunit.client.0.smithi102.stdout:8/698: rename d2/d7/d27/d64/d96/dae/d78/c8c to d2/d7/d27/cda 0 2024-05-13T22:48:19.641 INFO:tasks.workunit.client.1.smithi130.stdout:1/5: getdents . 0 2024-05-13T22:48:19.642 INFO:tasks.workunit.client.0.smithi102.stdout:3/828: creat de/d10/d2a/d36/db2/ddc/f113 x:0 0 0 2024-05-13T22:48:19.642 INFO:tasks.workunit.client.1.smithi130.stdout:1/6: write f0 [859730,115185] 0 2024-05-13T22:48:19.644 INFO:tasks.workunit.client.1.smithi130.stdout:8/3: fsync f0 0 2024-05-13T22:48:19.645 INFO:tasks.workunit.client.0.smithi102.stdout:5/910: write de/d25/d2a/def/f106 [497879,128947] 0 2024-05-13T22:48:19.646 INFO:tasks.workunit.client.1.smithi130.stdout:9/12: mkdir d1 0 2024-05-13T22:48:19.646 INFO:tasks.workunit.client.1.smithi130.stdout:0/20: dread d0/f3 [0,4194304] 0 2024-05-13T22:48:19.647 INFO:tasks.workunit.client.1.smithi130.stdout:9/13: dread - no filename 2024-05-13T22:48:19.648 INFO:tasks.workunit.client.1.smithi130.stdout:0/21: read d0/f3 [353124,52315] 0 2024-05-13T22:48:19.649 INFO:tasks.workunit.client.1.smithi130.stdout:7/8: mknod c3 0 2024-05-13T22:48:19.649 INFO:tasks.workunit.client.1.smithi130.stdout:6/10: dwrite f2 [0,4194304] 0 2024-05-13T22:48:19.650 INFO:tasks.workunit.client.0.smithi102.stdout:3/829: truncate de/d10/d46/fcd 2236011 0 2024-05-13T22:48:19.652 INFO:tasks.workunit.client.0.smithi102.stdout:1/597: symlink d2/d4/d26/d40/lc0 0 2024-05-13T22:48:19.652 INFO:tasks.workunit.client.1.smithi130.stdout:0/22: dread d0/f3 [0,4194304] 0 2024-05-13T22:48:19.654 INFO:tasks.workunit.client.0.smithi102.stdout:0/732: write d1/de/d32/d24/f30 [673299,59294] 0 2024-05-13T22:48:19.658 INFO:tasks.workunit.client.0.smithi102.stdout:8/699: rename d2/d7/f4e to d2/d7/d27/d64/fdb 0 2024-05-13T22:48:19.665 INFO:tasks.workunit.client.1.smithi130.stdout:6/11: fsync f2 0 2024-05-13T22:48:19.666 INFO:tasks.workunit.client.0.smithi102.stdout:3/830: rename de/d10/d50/d8f/ffc to de/d10/d2a/d36/d88/f114 0 2024-05-13T22:48:19.667 INFO:tasks.workunit.client.1.smithi130.stdout:2/12: link f2 d4/f5 0 2024-05-13T22:48:19.667 INFO:tasks.workunit.client.0.smithi102.stdout:0/733: dread d1/de/d32/f9c [0,4194304] 0 2024-05-13T22:48:19.669 INFO:tasks.workunit.client.1.smithi130.stdout:1/7: mknod c2 0 2024-05-13T22:48:19.671 INFO:tasks.workunit.client.1.smithi130.stdout:6/12: dwrite f2 [0,4194304] 0 2024-05-13T22:48:19.673 INFO:tasks.workunit.client.1.smithi130.stdout:8/4: mkdir d1 0 2024-05-13T22:48:19.675 INFO:tasks.workunit.client.1.smithi130.stdout:8/5: write f0 [786280,96737] 0 2024-05-13T22:48:19.676 INFO:tasks.workunit.client.1.smithi130.stdout:7/9: creat f4 x:0 0 0 2024-05-13T22:48:19.676 INFO:tasks.workunit.client.1.smithi130.stdout:8/6: write f0 [342690,63998] 0 2024-05-13T22:48:19.677 INFO:tasks.workunit.client.0.smithi102.stdout:3/831: mknod de/d10/d46/dc8/c115 0 2024-05-13T22:48:19.677 INFO:tasks.workunit.client.1.smithi130.stdout:7/10: truncate f4 223257 0 2024-05-13T22:48:19.678 INFO:tasks.workunit.client.1.smithi130.stdout:8/7: write f0 [517107,95809] 0 2024-05-13T22:48:19.680 INFO:tasks.workunit.client.0.smithi102.stdout:5/911: dwrite f2 [0,4194304] 0 2024-05-13T22:48:19.680 INFO:tasks.workunit.client.0.smithi102.stdout:0/734: mkdir d1/d68/de2 0 2024-05-13T22:48:19.682 INFO:tasks.workunit.client.0.smithi102.stdout:0/735: read - d1/de/d32/d24/d2d/d38/db7/fdb zero size 2024-05-13T22:48:19.682 INFO:tasks.workunit.client.0.smithi102.stdout:8/700: write d2/d7/d25/d41/d6f/f84 [4669298,103060] 0 2024-05-13T22:48:19.683 INFO:tasks.workunit.client.1.smithi130.stdout:4/6: getdents . 0 2024-05-13T22:48:19.686 INFO:tasks.workunit.client.1.smithi130.stdout:0/23: creat d0/f6 x:0 0 0 2024-05-13T22:48:19.686 INFO:tasks.workunit.client.1.smithi130.stdout:0/24: readlink - no filename 2024-05-13T22:48:19.688 INFO:tasks.workunit.client.1.smithi130.stdout:5/9: getdents . 0 2024-05-13T22:48:19.689 INFO:tasks.workunit.client.1.smithi130.stdout:8/8: dread f0 [0,4194304] 0 2024-05-13T22:48:19.689 INFO:tasks.workunit.client.1.smithi130.stdout:5/10: write f2 [509184,120374] 0 2024-05-13T22:48:19.689 INFO:tasks.workunit.client.0.smithi102.stdout:5/912: creat de/f14f x:0 0 0 2024-05-13T22:48:19.690 INFO:tasks.workunit.client.1.smithi130.stdout:8/9: write f0 [1626869,81818] 0 2024-05-13T22:48:19.691 INFO:tasks.workunit.client.0.smithi102.stdout:8/701: truncate d2/f32 696460 0 2024-05-13T22:48:19.693 INFO:tasks.workunit.client.0.smithi102.stdout:1/598: dwrite d2/d4/d26/f33 [0,4194304] 0 2024-05-13T22:48:19.694 INFO:tasks.workunit.client.1.smithi130.stdout:5/11: dread f2 [0,4194304] 0 2024-05-13T22:48:19.694 INFO:tasks.workunit.client.1.smithi130.stdout:5/12: rmdir - no directory 2024-05-13T22:48:19.695 INFO:tasks.workunit.client.1.smithi130.stdout:5/13: read f2 [99072,90415] 0 2024-05-13T22:48:19.695 INFO:tasks.workunit.client.1.smithi130.stdout:2/13: chown f0 79348173 1 2024-05-13T22:48:19.698 INFO:tasks.workunit.client.1.smithi130.stdout:6/13: mknod c3 0 2024-05-13T22:48:19.703 INFO:tasks.workunit.client.0.smithi102.stdout:0/736: dread d1/de/d32/d24/d2d/d3e/f9e [0,4194304] 0 2024-05-13T22:48:19.704 INFO:tasks.workunit.client.0.smithi102.stdout:8/702: mkdir d2/d7/d27/d64/d80/dc8/d40/d85/ddc 0 2024-05-13T22:48:19.706 INFO:tasks.workunit.client.1.smithi130.stdout:6/14: dwrite f2 [4194304,4194304] 0 2024-05-13T22:48:19.707 INFO:tasks.workunit.client.0.smithi102.stdout:3/832: truncate de/d68/f9d 262529 0 2024-05-13T22:48:19.711 INFO:tasks.workunit.client.1.smithi130.stdout:6/15: dread f2 [0,4194304] 0 2024-05-13T22:48:19.714 INFO:tasks.workunit.client.1.smithi130.stdout:4/7: symlink l2 0 2024-05-13T22:48:19.714 INFO:tasks.workunit.client.1.smithi130.stdout:4/8: read - f1 zero size 2024-05-13T22:48:19.715 INFO:tasks.workunit.client.0.smithi102.stdout:0/737: fsync d1/de/d32/d24/d2d/d38/d3d/f57 0 2024-05-13T22:48:19.717 INFO:tasks.workunit.client.1.smithi130.stdout:0/25: mkdir d0/d7 0 2024-05-13T22:48:19.717 INFO:tasks.workunit.client.0.smithi102.stdout:5/913: dwrite de/d25/d2a/d4b/d4d/d114/dda/ffc [0,4194304] 0 2024-05-13T22:48:19.718 INFO:tasks.workunit.client.1.smithi130.stdout:0/26: dread - d0/f5 zero size 2024-05-13T22:48:19.719 INFO:tasks.workunit.client.0.smithi102.stdout:8/703: symlink d2/d7/d27/d64/d96/dae/d78/ldd 0 2024-05-13T22:48:19.719 INFO:tasks.workunit.client.1.smithi130.stdout:0/27: fsync d0/f5 0 2024-05-13T22:48:19.721 INFO:tasks.workunit.client.1.smithi130.stdout:0/28: read - d0/f4 zero size 2024-05-13T22:48:19.722 INFO:tasks.workunit.client.1.smithi130.stdout:3/15: getdents . 0 2024-05-13T22:48:19.722 INFO:tasks.workunit.client.1.smithi130.stdout:4/9: dwrite f1 [0,4194304] 0 2024-05-13T22:48:19.723 INFO:tasks.workunit.client.1.smithi130.stdout:4/10: chown l2 2 1 2024-05-13T22:48:19.723 INFO:tasks.workunit.client.1.smithi130.stdout:3/16: truncate f2 862587 0 2024-05-13T22:48:19.725 INFO:tasks.workunit.client.1.smithi130.stdout:5/14: creat f3 x:0 0 0 2024-05-13T22:48:19.726 INFO:tasks.workunit.client.0.smithi102.stdout:1/599: dwrite d2/d10/d62/d7b/f95 [0,4194304] 0 2024-05-13T22:48:19.727 INFO:tasks.workunit.client.0.smithi102.stdout:4/794: sync 2024-05-13T22:48:19.733 INFO:tasks.workunit.client.0.smithi102.stdout:3/833: symlink de/d10/d2a/d8c/da8/d10e/l116 0 2024-05-13T22:48:19.733 INFO:tasks.workunit.client.1.smithi130.stdout:9/14: link l0 d1/l2 0 2024-05-13T22:48:19.733 INFO:tasks.workunit.client.1.smithi130.stdout:9/15: dwrite - no filename 2024-05-13T22:48:19.738 INFO:tasks.workunit.client.1.smithi130.stdout:7/11: rename f4 to f5 0 2024-05-13T22:48:19.738 INFO:tasks.workunit.client.0.smithi102.stdout:5/914: dread de/d25/d2a/d4b/d4d/fb5 [0,4194304] 0 2024-05-13T22:48:19.740 INFO:tasks.workunit.client.0.smithi102.stdout:4/795: rename d3/d19/d23/d39/da7/d7d/d95 to d3/dd8/d113 0 2024-05-13T22:48:19.743 INFO:tasks.workunit.client.0.smithi102.stdout:3/834: mkdir de/d67/d105/d117 0 2024-05-13T22:48:19.744 INFO:tasks.workunit.client.1.smithi130.stdout:6/16: mkdir d4 0 2024-05-13T22:48:19.746 INFO:tasks.workunit.client.1.smithi130.stdout:8/10: mkdir d1/d2 0 2024-05-13T22:48:19.747 INFO:tasks.workunit.client.0.smithi102.stdout:5/915: mkdir de/d25/d59/d7f/d86/da5/ddb/d150 0 2024-05-13T22:48:19.748 INFO:tasks.workunit.client.1.smithi130.stdout:0/29: dread d0/f3 [0,4194304] 0 2024-05-13T22:48:19.748 INFO:tasks.workunit.client.1.smithi130.stdout:4/11: creat f3 x:0 0 0 2024-05-13T22:48:19.750 INFO:tasks.workunit.client.1.smithi130.stdout:3/17: mknod c6 0 2024-05-13T22:48:19.753 INFO:tasks.workunit.client.1.smithi130.stdout:5/15: write f2 [962262,51173] 0 2024-05-13T22:48:19.753 INFO:tasks.workunit.client.1.smithi130.stdout:0/30: dread d0/f2 [0,4194304] 0 2024-05-13T22:48:19.754 INFO:tasks.workunit.client.0.smithi102.stdout:0/738: getdents d1/de/d32 0 2024-05-13T22:48:19.754 INFO:tasks.workunit.client.1.smithi130.stdout:5/16: read f2 [345988,29782] 0 2024-05-13T22:48:19.755 INFO:tasks.workunit.client.1.smithi130.stdout:2/14: creat d4/f6 x:0 0 0 2024-05-13T22:48:19.755 INFO:tasks.workunit.client.1.smithi130.stdout:1/8: truncate f0 98828 0 2024-05-13T22:48:19.756 INFO:tasks.workunit.client.0.smithi102.stdout:5/916: mknod de/d25/d2a/d4b/d4d/d50/da3/c151 0 2024-05-13T22:48:19.757 INFO:tasks.workunit.client.1.smithi130.stdout:0/31: chown d0/f3 541 1 2024-05-13T22:48:19.757 INFO:tasks.workunit.client.1.smithi130.stdout:2/15: dread - f2 zero size 2024-05-13T22:48:19.758 INFO:tasks.workunit.client.1.smithi130.stdout:2/16: read - f2 zero size 2024-05-13T22:48:19.758 INFO:tasks.workunit.client.0.smithi102.stdout:4/796: mknod d3/d19/c114 0 2024-05-13T22:48:19.759 INFO:tasks.workunit.client.0.smithi102.stdout:4/797: write d3/d19/dad/faf [5174331,35900] 0 2024-05-13T22:48:19.759 INFO:tasks.workunit.client.1.smithi130.stdout:2/17: dread - f0 zero size 2024-05-13T22:48:19.760 INFO:tasks.workunit.client.1.smithi130.stdout:3/18: dwrite f5 [0,4194304] 0 2024-05-13T22:48:19.760 INFO:tasks.workunit.client.1.smithi130.stdout:7/12: unlink c2 0 2024-05-13T22:48:19.762 INFO:tasks.workunit.client.0.smithi102.stdout:3/835: rename de/l1a to de/d10/d50/d63/l118 0 2024-05-13T22:48:19.762 INFO:tasks.workunit.client.0.smithi102.stdout:1/600: write d2/d4/d26/d40/f72 [3044980,2266] 0 2024-05-13T22:48:19.764 INFO:tasks.workunit.client.0.smithi102.stdout:0/739: creat d1/de/d32/dab/fe3 x:0 0 0 2024-05-13T22:48:19.765 INFO:tasks.workunit.client.0.smithi102.stdout:1/601: read d2/d45/d79/fa6 [1911829,98401] 0 2024-05-13T22:48:19.765 INFO:tasks.workunit.client.1.smithi130.stdout:5/17: dwrite f2 [0,4194304] 0 2024-05-13T22:48:19.765 INFO:tasks.workunit.client.1.smithi130.stdout:5/18: stat f3 0 2024-05-13T22:48:19.770 INFO:tasks.workunit.client.1.smithi130.stdout:6/17: write f2 [3349168,42998] 0 2024-05-13T22:48:19.773 INFO:tasks.workunit.client.0.smithi102.stdout:4/798: mkdir d3/d4/d6/de/d115 0 2024-05-13T22:48:19.775 INFO:tasks.workunit.client.1.smithi130.stdout:5/19: dwrite f3 [0,4194304] 0 2024-05-13T22:48:19.775 INFO:tasks.workunit.client.1.smithi130.stdout:4/12: creat f4 x:0 0 0 2024-05-13T22:48:19.776 INFO:tasks.workunit.client.0.smithi102.stdout:3/836: truncate de/d10/d2a/d36/f49 311407 0 2024-05-13T22:48:19.777 INFO:tasks.workunit.client.0.smithi102.stdout:3/837: write de/d10/d50/f89 [1156659,24726] 0 2024-05-13T22:48:19.780 INFO:tasks.workunit.client.0.smithi102.stdout:0/740: rename d1/d4d/l84 to d1/de/d32/da6/le4 0 2024-05-13T22:48:19.781 INFO:tasks.workunit.client.0.smithi102.stdout:0/741: fdatasync d1/de/d32/d24/fd1 0 2024-05-13T22:48:19.784 INFO:tasks.workunit.client.1.smithi130.stdout:9/16: creat d1/f3 x:0 0 0 2024-05-13T22:48:19.785 INFO:tasks.workunit.client.0.smithi102.stdout:5/917: mknod de/d25/d2a/d4b/d4d/d50/d6c/ded/c152 0 2024-05-13T22:48:19.785 INFO:tasks.workunit.client.1.smithi130.stdout:9/17: truncate d1/f3 590026 0 2024-05-13T22:48:19.786 INFO:tasks.workunit.client.1.smithi130.stdout:1/9: rename c1 to c3 0 2024-05-13T22:48:19.787 INFO:tasks.workunit.client.1.smithi130.stdout:9/18: write d1/f3 [1025963,32237] 0 2024-05-13T22:48:19.787 INFO:tasks.workunit.client.0.smithi102.stdout:4/799: truncate d3/d4/d6/de/d1f/d41/d6a/fa1 5602643 0 2024-05-13T22:48:19.787 INFO:tasks.workunit.client.1.smithi130.stdout:0/32: creat d0/f8 x:0 0 0 2024-05-13T22:48:19.788 INFO:tasks.workunit.client.0.smithi102.stdout:3/838: creat de/d10/d2a/f119 x:0 0 0 2024-05-13T22:48:19.790 INFO:tasks.workunit.client.1.smithi130.stdout:0/33: truncate d0/f5 514016 0 2024-05-13T22:48:19.790 INFO:tasks.workunit.client.1.smithi130.stdout:5/20: dwrite f2 [0,4194304] 0 2024-05-13T22:48:19.791 INFO:tasks.workunit.client.1.smithi130.stdout:0/34: chown d0/f2 51075 1 2024-05-13T22:48:19.792 INFO:tasks.workunit.client.1.smithi130.stdout:2/18: creat d4/f7 x:0 0 0 2024-05-13T22:48:19.792 INFO:tasks.workunit.client.0.smithi102.stdout:1/602: creat d2/d4/d5/d19/dbe/fc1 x:0 0 0 2024-05-13T22:48:19.792 INFO:tasks.workunit.client.1.smithi130.stdout:0/35: truncate d0/f6 771915 0 2024-05-13T22:48:19.794 INFO:tasks.workunit.client.1.smithi130.stdout:7/13: rename l1 to l6 0 2024-05-13T22:48:19.796 INFO:tasks.workunit.client.0.smithi102.stdout:3/839: dwrite de/d67/ff4 [0,4194304] 0 2024-05-13T22:48:19.796 INFO:tasks.workunit.client.1.smithi130.stdout:9/19: dread d1/f3 [0,4194304] 0 2024-05-13T22:48:19.797 INFO:tasks.workunit.client.1.smithi130.stdout:9/20: read d1/f3 [913024,57021] 0 2024-05-13T22:48:19.797 INFO:tasks.workunit.client.1.smithi130.stdout:8/11: creat d1/d2/f3 x:0 0 0 2024-05-13T22:48:19.798 INFO:tasks.workunit.client.0.smithi102.stdout:3/840: chown de/d10/d2a/d8c/d7f/fd6 502 1 2024-05-13T22:48:19.798 INFO:tasks.workunit.client.1.smithi130.stdout:9/21: chown d1 1918319 1 2024-05-13T22:48:19.804 INFO:tasks.workunit.client.1.smithi130.stdout:9/22: dwrite d1/f3 [0,4194304] 0 2024-05-13T22:48:19.805 INFO:tasks.workunit.client.1.smithi130.stdout:9/23: readlink l0 0 2024-05-13T22:48:19.806 INFO:tasks.workunit.client.0.smithi102.stdout:1/603: fsync d2/d10/f85 0 2024-05-13T22:48:19.808 INFO:tasks.workunit.client.0.smithi102.stdout:5/918: truncate de/f1d 599951 0 2024-05-13T22:48:19.810 INFO:tasks.workunit.client.0.smithi102.stdout:3/841: fdatasync de/d10/d2a/d33/d61/f4c 0 2024-05-13T22:48:19.822 INFO:tasks.workunit.client.0.smithi102.stdout:3/842: creat de/d10/d50/d63/d98/f11a x:0 0 0 2024-05-13T22:48:19.823 INFO:tasks.workunit.client.0.smithi102.stdout:5/919: dread de/d25/d2c/f78 [0,4194304] 0 2024-05-13T22:48:19.823 INFO:tasks.workunit.client.1.smithi130.stdout:5/21: creat f4 x:0 0 0 2024-05-13T22:48:19.824 INFO:tasks.workunit.client.0.smithi102.stdout:0/742: write d1/de/d32/f37 [411543,48207] 0 2024-05-13T22:48:19.824 INFO:tasks.workunit.client.0.smithi102.stdout:5/920: fsync de/d25/d2a/d4b/d4d/d50/f109 0 2024-05-13T22:48:19.825 INFO:tasks.workunit.client.1.smithi130.stdout:2/19: truncate f2 532931 0 2024-05-13T22:48:19.825 INFO:tasks.workunit.client.0.smithi102.stdout:0/743: chown d1/de/f1d 23309302 1 2024-05-13T22:48:19.826 INFO:tasks.workunit.client.0.smithi102.stdout:4/800: link d3/d19/d23/d39/da7/d7d/ff1 d3/d19/d23/d39/d5d/f116 0 2024-05-13T22:48:19.826 INFO:tasks.workunit.client.1.smithi130.stdout:5/22: dread f2 [0,4194304] 0 2024-05-13T22:48:19.829 INFO:tasks.workunit.client.1.smithi130.stdout:0/36: rename d0/f6 to d0/d7/f9 0 2024-05-13T22:48:19.829 INFO:tasks.workunit.client.1.smithi130.stdout:0/37: write d0/f8 [49811,10830] 0 2024-05-13T22:48:19.830 INFO:tasks.workunit.client.0.smithi102.stdout:3/843: dwrite de/d10/d2a/d36/d88/f10b [0,4194304] 0 2024-05-13T22:48:19.833 INFO:tasks.workunit.client.1.smithi130.stdout:8/12: mkdir d1/d2/d4 0 2024-05-13T22:48:19.834 INFO:tasks.workunit.client.1.smithi130.stdout:6/18: mknod d4/c5 0 2024-05-13T22:48:19.834 INFO:tasks.workunit.client.1.smithi130.stdout:6/19: write f2 [8207758,12261] 0 2024-05-13T22:48:19.836 INFO:tasks.workunit.client.1.smithi130.stdout:9/24: creat d1/f4 x:0 0 0 2024-05-13T22:48:19.837 INFO:tasks.workunit.client.0.smithi102.stdout:5/921: unlink de/d25/f47 0 2024-05-13T22:48:19.850 INFO:tasks.workunit.client.1.smithi130.stdout:5/23: symlink l5 0 2024-05-13T22:48:19.852 INFO:tasks.workunit.client.1.smithi130.stdout:7/14: dwrite f5 [0,4194304] 0 2024-05-13T22:48:19.854 INFO:tasks.workunit.client.0.smithi102.stdout:0/744: write d1/de/d32/d8a/fac [907424,41085] 0 2024-05-13T22:48:19.856 INFO:tasks.workunit.client.0.smithi102.stdout:3/844: symlink de/d67/d105/d117/l11b 0 2024-05-13T22:48:19.857 INFO:tasks.workunit.client.1.smithi130.stdout:0/38: creat d0/fa x:0 0 0 2024-05-13T22:48:19.857 INFO:tasks.workunit.client.0.smithi102.stdout:1/604: truncate d2/d4/d5/f15 1143046 0 2024-05-13T22:48:19.859 INFO:tasks.workunit.client.0.smithi102.stdout:5/922: write de/d25/d59/d7f/d86/da5/ffe [441136,66505] 0 2024-05-13T22:48:19.861 INFO:tasks.workunit.client.0.smithi102.stdout:0/745: symlink d1/de/d42/le5 0 2024-05-13T22:48:19.862 INFO:tasks.workunit.client.0.smithi102.stdout:0/746: chown d1/de/d32/d24/d2d/d38/cdc 29684 1 2024-05-13T22:48:19.862 INFO:tasks.workunit.client.1.smithi130.stdout:6/20: mknod d4/c6 0 2024-05-13T22:48:19.863 INFO:tasks.workunit.client.0.smithi102.stdout:4/801: link d3/d19/d23/d4e/d68/fe2 d3/d19/d23/d39/da7/da9/dd3/dd4/f117 0 2024-05-13T22:48:19.864 INFO:tasks.workunit.client.1.smithi130.stdout:9/25: creat d1/f5 x:0 0 0 2024-05-13T22:48:19.864 INFO:tasks.workunit.client.1.smithi130.stdout:9/26: readlink l0 0 2024-05-13T22:48:19.864 INFO:tasks.workunit.client.1.smithi130.stdout:9/27: chown d1/l2 124928742 1 2024-05-13T22:48:19.865 INFO:tasks.workunit.client.0.smithi102.stdout:3/845: write de/f37 [6760282,59973] 0 2024-05-13T22:48:19.866 INFO:tasks.workunit.client.0.smithi102.stdout:3/846: chown de/d1c/d79/f81 1 1 2024-05-13T22:48:19.869 INFO:tasks.workunit.client.1.smithi130.stdout:3/19: getdents . 0 2024-05-13T22:48:19.870 INFO:tasks.workunit.client.0.smithi102.stdout:5/923: mkdir de/d25/d59/d7f/d86/da5/ddb/d150/d153 0 2024-05-13T22:48:19.872 INFO:tasks.workunit.client.1.smithi130.stdout:2/20: link d4/f7 d4/f8 0 2024-05-13T22:48:19.873 INFO:tasks.workunit.client.0.smithi102.stdout:3/847: rmdir de/d10/d34 39 2024-05-13T22:48:19.878 INFO:tasks.workunit.client.0.smithi102.stdout:0/747: unlink d1/d4d/f7f 0 2024-05-13T22:48:19.878 INFO:tasks.workunit.client.0.smithi102.stdout:5/924: dwrite de/d25/d2a/d4b/d4d/d114/f14a [0,4194304] 0 2024-05-13T22:48:19.879 INFO:tasks.workunit.client.1.smithi130.stdout:7/15: mkdir d7 0 2024-05-13T22:48:19.881 INFO:tasks.workunit.client.1.smithi130.stdout:0/39: unlink d0/d7/f9 0 2024-05-13T22:48:19.882 INFO:tasks.workunit.client.0.smithi102.stdout:1/605: rename d2/d45/d79/fa6 to d2/fc2 0 2024-05-13T22:48:19.883 INFO:tasks.workunit.client.1.smithi130.stdout:8/13: creat d1/d2/d4/f5 x:0 0 0 2024-05-13T22:48:19.884 INFO:tasks.workunit.client.1.smithi130.stdout:8/14: chown d1 26 1 2024-05-13T22:48:19.884 INFO:tasks.workunit.client.0.smithi102.stdout:3/848: unlink de/d1c/d79/l26 0 2024-05-13T22:48:19.885 INFO:tasks.workunit.client.0.smithi102.stdout:4/802: write d3/d4/d6/de/d1f/f53 [321000,58613] 0 2024-05-13T22:48:19.886 INFO:tasks.workunit.client.1.smithi130.stdout:6/21: mknod d4/c7 0 2024-05-13T22:48:19.888 INFO:tasks.workunit.client.1.smithi130.stdout:4/13: getdents . 0 2024-05-13T22:48:19.890 INFO:tasks.workunit.client.1.smithi130.stdout:9/28: symlink d1/l6 0 2024-05-13T22:48:19.891 INFO:tasks.workunit.client.0.smithi102.stdout:5/925: mkdir de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d154 0 2024-05-13T22:48:19.894 INFO:tasks.workunit.client.1.smithi130.stdout:3/20: mknod c7 0 2024-05-13T22:48:19.894 INFO:tasks.workunit.client.1.smithi130.stdout:3/21: readlink l1 0 2024-05-13T22:48:19.895 INFO:tasks.workunit.client.1.smithi130.stdout:2/21: creat d4/f9 x:0 0 0 2024-05-13T22:48:19.899 INFO:tasks.workunit.client.1.smithi130.stdout:0/40: creat d0/d7/fb x:0 0 0 2024-05-13T22:48:19.901 INFO:tasks.workunit.client.0.smithi102.stdout:1/606: symlink d2/d4/d9e/lc3 0 2024-05-13T22:48:19.902 INFO:tasks.workunit.client.1.smithi130.stdout:6/22: creat d4/f8 x:0 0 0 2024-05-13T22:48:19.903 INFO:tasks.workunit.client.1.smithi130.stdout:4/14: creat f5 x:0 0 0 2024-05-13T22:48:19.904 INFO:tasks.workunit.client.0.smithi102.stdout:3/849: creat de/d10/d50/d63/d98/dee/f11c x:0 0 0 2024-05-13T22:48:19.905 INFO:tasks.workunit.client.1.smithi130.stdout:9/29: mkdir d1/d7 0 2024-05-13T22:48:19.906 INFO:tasks.workunit.client.1.smithi130.stdout:1/10: dread f0 [0,4194304] 0 2024-05-13T22:48:19.907 INFO:tasks.workunit.client.0.smithi102.stdout:1/607: readlink d2/d10/d62/d86/la3 0 2024-05-13T22:48:19.908 INFO:tasks.workunit.client.1.smithi130.stdout:7/16: mkdir d7/d8 0 2024-05-13T22:48:19.908 INFO:tasks.workunit.client.1.smithi130.stdout:7/17: fsync f5 0 2024-05-13T22:48:19.909 INFO:tasks.workunit.client.0.smithi102.stdout:4/803: creat d3/d19/d23/f118 x:0 0 0 2024-05-13T22:48:19.909 INFO:tasks.workunit.client.0.smithi102.stdout:4/804: stat d3/d19/d23/d4e/d68/d85/l10c 0 2024-05-13T22:48:19.910 INFO:tasks.workunit.client.1.smithi130.stdout:0/41: mkdir d0/d7/dc 0 2024-05-13T22:48:19.911 INFO:tasks.workunit.client.0.smithi102.stdout:0/748: link d1/c65 d1/de/d32/d24/d2d/d3e/ce6 0 2024-05-13T22:48:19.912 INFO:tasks.workunit.client.1.smithi130.stdout:6/23: creat d4/f9 x:0 0 0 2024-05-13T22:48:19.913 INFO:tasks.workunit.client.1.smithi130.stdout:4/15: creat f6 x:0 0 0 2024-05-13T22:48:19.914 INFO:tasks.workunit.client.0.smithi102.stdout:1/608: readlink d2/d4/d26/da2/lb9 0 2024-05-13T22:48:19.915 INFO:tasks.workunit.client.0.smithi102.stdout:1/609: write d2/ff [621448,2442] 0 2024-05-13T22:48:19.919 INFO:tasks.workunit.client.0.smithi102.stdout:5/926: write de/d75/fbd [826130,46863] 0 2024-05-13T22:48:19.929 INFO:tasks.workunit.client.0.smithi102.stdout:1/610: dread d2/d45/f47 [0,4194304] 0 2024-05-13T22:48:19.930 INFO:tasks.workunit.client.0.smithi102.stdout:0/749: creat d1/d4d/d79/d9f/dbd/fe7 x:0 0 0 2024-05-13T22:48:19.931 INFO:tasks.workunit.client.0.smithi102.stdout:3/850: getdents de/d10/d50/d8f 0 2024-05-13T22:48:19.932 INFO:tasks.workunit.client.0.smithi102.stdout:3/851: dread - de/d10/d46/d100/f102 zero size 2024-05-13T22:48:19.933 INFO:tasks.workunit.client.0.smithi102.stdout:3/852: stat de/d10/d50/lf2 0 2024-05-13T22:48:19.933 INFO:tasks.workunit.client.0.smithi102.stdout:4/805: rename d3/d19/d23/d39/lfc to d3/d22/l119 0 2024-05-13T22:48:19.933 INFO:tasks.workunit.client.0.smithi102.stdout:5/927: symlink de/l155 0 2024-05-13T22:48:19.934 INFO:tasks.workunit.client.0.smithi102.stdout:1/611: mkdir d2/d10/d62/d7b/dc4 0 2024-05-13T22:48:19.937 INFO:tasks.workunit.client.0.smithi102.stdout:0/750: symlink d1/de/d32/dd8/le8 0 2024-05-13T22:48:19.937 INFO:tasks.workunit.client.0.smithi102.stdout:5/928: chown de/d25/d2a/d4b/dab/lb2 62215684 1 2024-05-13T22:48:19.938 INFO:tasks.workunit.client.0.smithi102.stdout:4/806: dread d3/d6b/fc6 [0,4194304] 0 2024-05-13T22:48:19.939 INFO:tasks.workunit.client.0.smithi102.stdout:0/751: stat d1/de/d32/d24/d2d/d3e/fc8 0 2024-05-13T22:48:19.943 INFO:tasks.workunit.client.0.smithi102.stdout:0/752: symlink d1/de/d32/dd8/le9 0 2024-05-13T22:48:19.944 INFO:tasks.workunit.client.0.smithi102.stdout:0/753: mkdir d1/de/d42/dea 0 2024-05-13T22:48:19.953 INFO:tasks.workunit.client.0.smithi102.stdout:5/929: dread de/d25/fd3 [0,4194304] 0 2024-05-13T22:48:19.954 INFO:tasks.workunit.client.0.smithi102.stdout:4/807: dread d3/d19/d23/d39/da7/d7d/ff1 [0,4194304] 0 2024-05-13T22:48:19.954 INFO:tasks.workunit.client.0.smithi102.stdout:4/808: chown d3/dd8/d113/d103 3 1 2024-05-13T22:48:19.955 INFO:tasks.workunit.client.0.smithi102.stdout:4/809: dread - d3/d19/d23/d3a/da4/f10b zero size 2024-05-13T22:48:19.956 INFO:tasks.workunit.client.0.smithi102.stdout:5/930: getdents de/d25/d43/dd1 0 2024-05-13T22:48:19.959 INFO:tasks.workunit.client.0.smithi102.stdout:4/810: rename d3/d19/d23/d4e/d68/d85/c9e to d3/d4/d6/de/d8d/c11a 0 2024-05-13T22:48:19.960 INFO:tasks.workunit.client.0.smithi102.stdout:4/811: fdatasync d3/d4/d6/de/d1f/d41/fc2 0 2024-05-13T22:48:19.961 INFO:tasks.workunit.client.0.smithi102.stdout:4/812: chown d3/d22/dab/fbb 7634624 1 2024-05-13T22:48:19.962 INFO:tasks.workunit.client.0.smithi102.stdout:9/726: sync 2024-05-13T22:48:19.962 INFO:tasks.workunit.client.0.smithi102.stdout:2/899: sync 2024-05-13T22:48:19.962 INFO:tasks.workunit.client.0.smithi102.stdout:6/855: sync 2024-05-13T22:48:19.965 INFO:tasks.workunit.client.0.smithi102.stdout:6/856: mkdir d9/d25/d5a/d113 0 2024-05-13T22:48:19.966 INFO:tasks.workunit.client.0.smithi102.stdout:9/727: unlink d1/d6/db/d30/d47/lfa 0 2024-05-13T22:48:19.966 INFO:tasks.workunit.client.0.smithi102.stdout:4/813: dread d3/d4/f32 [0,4194304] 0 2024-05-13T22:48:19.969 INFO:tasks.workunit.client.1.smithi130.stdout:3/22: truncate f5 1787618 0 2024-05-13T22:48:19.969 INFO:tasks.workunit.client.0.smithi102.stdout:9/728: symlink d1/d5d/d6c/lfd 0 2024-05-13T22:48:19.970 INFO:tasks.workunit.client.0.smithi102.stdout:2/900: dwrite d7/d15/d1e/f125 [0,4194304] 0 2024-05-13T22:48:19.972 INFO:tasks.workunit.client.1.smithi130.stdout:0/42: rename d0/d7/dc to d0/d7/dd 0 2024-05-13T22:48:19.972 INFO:tasks.workunit.client.0.smithi102.stdout:4/814: rmdir d3/d19/d23/d39/da7/da9/dd3/dd4 39 2024-05-13T22:48:19.973 INFO:tasks.workunit.client.1.smithi130.stdout:6/24: mknod d4/ca 0 2024-05-13T22:48:19.976 INFO:tasks.workunit.client.1.smithi130.stdout:3/23: mkdir d8 0 2024-05-13T22:48:19.977 INFO:tasks.workunit.client.1.smithi130.stdout:6/25: chown d4/ca 29554 1 2024-05-13T22:48:19.977 INFO:tasks.workunit.client.1.smithi130.stdout:7/18: mkdir d7/d8/d9 0 2024-05-13T22:48:19.980 INFO:tasks.workunit.client.0.smithi102.stdout:4/815: fdatasync d3/d4/d6/de/f3f 0 2024-05-13T22:48:19.981 INFO:tasks.workunit.client.1.smithi130.stdout:4/16: dread f1 [0,4194304] 0 2024-05-13T22:48:19.982 INFO:tasks.workunit.client.1.smithi130.stdout:0/43: dread d0/f5 [0,4194304] 0 2024-05-13T22:48:19.982 INFO:tasks.workunit.client.1.smithi130.stdout:7/19: write f5 [4489106,77163] 0 2024-05-13T22:48:19.983 INFO:tasks.workunit.client.1.smithi130.stdout:8/15: getdents d1/d2 0 2024-05-13T22:48:19.984 INFO:tasks.workunit.client.1.smithi130.stdout:9/30: dwrite d1/f3 [0,4194304] 0 2024-05-13T22:48:19.985 INFO:tasks.workunit.client.1.smithi130.stdout:4/17: truncate f3 49084 0 2024-05-13T22:48:19.986 INFO:tasks.workunit.client.1.smithi130.stdout:1/11: dwrite f0 [0,4194304] 0 2024-05-13T22:48:19.986 INFO:tasks.workunit.client.0.smithi102.stdout:3/853: write de/d67/df7/ffd [1048061,70621] 0 2024-05-13T22:48:19.987 INFO:tasks.workunit.client.1.smithi130.stdout:4/18: chown l2 11417 1 2024-05-13T22:48:19.987 INFO:tasks.workunit.client.1.smithi130.stdout:4/19: stat f3 0 2024-05-13T22:48:19.987 INFO:tasks.workunit.client.1.smithi130.stdout:1/12: write f0 [1251733,59587] 0 2024-05-13T22:48:19.988 INFO:tasks.workunit.client.0.smithi102.stdout:1/612: truncate d2/d10/d62/d7b/f95 2971607 0 2024-05-13T22:48:19.990 INFO:tasks.workunit.client.1.smithi130.stdout:3/24: unlink f0 0 2024-05-13T22:48:19.992 INFO:tasks.workunit.client.0.smithi102.stdout:4/816: unlink d3/fba 0 2024-05-13T22:48:19.994 INFO:tasks.workunit.client.1.smithi130.stdout:6/26: rename c3 to d4/cb 0 2024-05-13T22:48:19.996 INFO:tasks.workunit.client.1.smithi130.stdout:8/16: dwrite d1/d2/d4/f5 [0,4194304] 0 2024-05-13T22:48:19.999 INFO:tasks.workunit.client.1.smithi130.stdout:6/27: dread f2 [4194304,4194304] 0 2024-05-13T22:48:20.001 INFO:tasks.workunit.client.1.smithi130.stdout:0/44: dread d0/f5 [0,4194304] 0 2024-05-13T22:48:20.003 INFO:tasks.workunit.client.1.smithi130.stdout:0/45: dread d0/f8 [0,4194304] 0 2024-05-13T22:48:20.003 INFO:tasks.workunit.client.1.smithi130.stdout:7/20: mkdir d7/da 0 2024-05-13T22:48:20.004 INFO:tasks.workunit.client.0.smithi102.stdout:0/754: dwrite d1/de/d42/f8e [0,4194304] 0 2024-05-13T22:48:20.004 INFO:tasks.workunit.client.1.smithi130.stdout:0/46: readlink - no filename 2024-05-13T22:48:20.005 INFO:tasks.workunit.client.1.smithi130.stdout:7/21: write f5 [2736713,78909] 0 2024-05-13T22:48:20.005 INFO:tasks.workunit.client.0.smithi102.stdout:5/931: dwrite de/d25/d2a/d4b/d4d/d50/da3/df4/f66 [0,4194304] 0 2024-05-13T22:48:20.006 INFO:tasks.workunit.client.0.smithi102.stdout:3/854: mknod de/d10/d34/d5d/c11d 0 2024-05-13T22:48:20.006 INFO:tasks.workunit.client.1.smithi130.stdout:6/28: dwrite f2 [0,4194304] 0 2024-05-13T22:48:20.009 INFO:tasks.workunit.client.1.smithi130.stdout:9/31: rmdir d1 39 2024-05-13T22:48:20.009 INFO:tasks.workunit.client.0.smithi102.stdout:3/855: write de/d10/d46/d100/f102 [1029468,8615] 0 2024-05-13T22:48:20.015 INFO:tasks.workunit.client.0.smithi102.stdout:6/857: dread d9/db/d14/d35/f9d [0,4194304] 0 2024-05-13T22:48:20.016 INFO:tasks.workunit.client.0.smithi102.stdout:4/817: dread d3/d22/dab/fbb [0,4194304] 0 2024-05-13T22:48:20.018 INFO:tasks.workunit.client.0.smithi102.stdout:0/755: rename d1/de/fcc to d1/de/d32/d24/d2d/d38/db7/feb 0 2024-05-13T22:48:20.018 INFO:tasks.workunit.client.0.smithi102.stdout:6/858: read d9/db/d14/d35/dc9/df1/f62 [4926131,72902] 0 2024-05-13T22:48:20.024 INFO:tasks.workunit.client.1.smithi130.stdout:4/20: dwrite f1 [0,4194304] 0 2024-05-13T22:48:20.028 INFO:tasks.workunit.client.0.smithi102.stdout:9/729: dwrite d1/d6/db/d4a/fa9 [0,4194304] 0 2024-05-13T22:48:20.031 INFO:tasks.workunit.client.1.smithi130.stdout:1/13: dwrite f0 [4194304,4194304] 0 2024-05-13T22:48:20.031 INFO:tasks.workunit.client.0.smithi102.stdout:9/730: chown d1/d6/d45/d2d/f3f 26096 1 2024-05-13T22:48:20.033 INFO:tasks.workunit.client.0.smithi102.stdout:2/901: dwrite d7/d9/fcf [0,4194304] 0 2024-05-13T22:48:20.036 INFO:tasks.workunit.client.0.smithi102.stdout:0/756: write d1/de/d32/f3b [4257684,53698] 0 2024-05-13T22:48:20.039 INFO:tasks.workunit.client.0.smithi102.stdout:2/902: write d7/d9/d10/d30/d53/f5e [94988,76617] 0 2024-05-13T22:48:20.041 INFO:tasks.workunit.client.1.smithi130.stdout:1/14: dwrite f0 [4194304,4194304] 0 2024-05-13T22:48:20.041 INFO:tasks.workunit.client.1.smithi130.stdout:8/17: creat d1/d2/f6 x:0 0 0 2024-05-13T22:48:20.043 INFO:tasks.workunit.client.0.smithi102.stdout:1/613: write d2/d10/f85 [799189,72856] 0 2024-05-13T22:48:20.047 INFO:tasks.workunit.client.0.smithi102.stdout:3/856: creat de/d1c/f11e x:0 0 0 2024-05-13T22:48:20.051 INFO:tasks.workunit.client.0.smithi102.stdout:0/757: mknod d1/d77/cec 0 2024-05-13T22:48:20.051 INFO:tasks.workunit.client.1.smithi130.stdout:6/29: chown d4/cb 0 1 2024-05-13T22:48:20.052 INFO:tasks.workunit.client.1.smithi130.stdout:1/15: dwrite f0 [0,4194304] 0 2024-05-13T22:48:20.052 INFO:tasks.workunit.client.1.smithi130.stdout:6/30: chown d4 566421 1 2024-05-13T22:48:20.055 INFO:tasks.workunit.client.1.smithi130.stdout:4/21: creat f7 x:0 0 0 2024-05-13T22:48:20.056 INFO:tasks.workunit.client.0.smithi102.stdout:3/857: rmdir de/d10/d34 39 2024-05-13T22:48:20.057 INFO:tasks.workunit.client.1.smithi130.stdout:3/25: creat d8/f9 x:0 0 0 2024-05-13T22:48:20.058 INFO:tasks.workunit.client.1.smithi130.stdout:4/22: write f5 [243421,80377] 0 2024-05-13T22:48:20.058 INFO:tasks.workunit.client.1.smithi130.stdout:4/23: rmdir - no directory 2024-05-13T22:48:20.062 INFO:tasks.workunit.client.0.smithi102.stdout:1/614: mknod d2/d4/d26/d40/cc5 0 2024-05-13T22:48:20.062 INFO:tasks.workunit.client.1.smithi130.stdout:6/31: dwrite d4/f9 [0,4194304] 0 2024-05-13T22:48:20.063 INFO:tasks.workunit.client.1.smithi130.stdout:8/18: mkdir d1/d7 0 2024-05-13T22:48:20.063 INFO:tasks.workunit.client.1.smithi130.stdout:8/19: readlink - no filename 2024-05-13T22:48:20.065 INFO:tasks.workunit.client.0.smithi102.stdout:3/858: unlink de/d10/d2a/d33/daa/fb3 0 2024-05-13T22:48:20.070 INFO:tasks.workunit.client.0.smithi102.stdout:0/758: rename d1/c65 to d1/de/d32/d24/d2d/d38/d3d/d81/ced 0 2024-05-13T22:48:20.071 INFO:tasks.workunit.client.0.smithi102.stdout:5/932: write de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d34/f117 [87784,118003] 0 2024-05-13T22:48:20.073 INFO:tasks.workunit.client.1.smithi130.stdout:9/32: creat d1/f8 x:0 0 0 2024-05-13T22:48:20.073 INFO:tasks.workunit.client.1.smithi130.stdout:9/33: write d1/f3 [1571503,49249] 0 2024-05-13T22:48:20.076 INFO:tasks.workunit.client.1.smithi130.stdout:1/16: mknod c4 0 2024-05-13T22:48:20.077 INFO:tasks.workunit.client.1.smithi130.stdout:3/26: creat d8/fa x:0 0 0 2024-05-13T22:48:20.079 INFO:tasks.workunit.client.0.smithi102.stdout:3/859: rmdir de/d10/d50/d63/d98 39 2024-05-13T22:48:20.080 INFO:tasks.workunit.client.0.smithi102.stdout:3/860: readlink de/d68/l84 0 2024-05-13T22:48:20.081 INFO:tasks.workunit.client.0.smithi102.stdout:4/818: dwrite d3/d4/d6/de/d8d/fb2 [0,4194304] 0 2024-05-13T22:48:20.081 INFO:tasks.workunit.client.0.smithi102.stdout:5/933: symlink de/d25/d2a/d4b/l156 0 2024-05-13T22:48:20.083 INFO:tasks.workunit.client.0.smithi102.stdout:5/934: chown de/d25/d2c/ff7 57 1 2024-05-13T22:48:20.086 INFO:tasks.workunit.client.0.smithi102.stdout:1/615: creat d2/d4/d38/db2/fc6 x:0 0 0 2024-05-13T22:48:20.088 INFO:tasks.workunit.client.0.smithi102.stdout:1/616: fdatasync d2/d10/fa0 0 2024-05-13T22:48:20.089 INFO:tasks.workunit.client.1.smithi130.stdout:6/32: dwrite f2 [0,4194304] 0 2024-05-13T22:48:20.092 INFO:tasks.workunit.client.1.smithi130.stdout:0/47: link d0/f2 d0/fe 0 2024-05-13T22:48:20.092 INFO:tasks.workunit.client.0.smithi102.stdout:9/731: dread d1/d6/fa5 [0,4194304] 0 2024-05-13T22:48:20.094 INFO:tasks.workunit.client.1.smithi130.stdout:4/24: dread f5 [0,4194304] 0 2024-05-13T22:48:20.095 INFO:tasks.workunit.client.0.smithi102.stdout:1/617: write d2/d4/f37 [2053359,17415] 0 2024-05-13T22:48:20.095 INFO:tasks.workunit.client.0.smithi102.stdout:5/935: dwrite de/d25/d2a/d4b/dab/f13c [0,4194304] 0 2024-05-13T22:48:20.102 INFO:tasks.workunit.client.1.smithi130.stdout:3/27: rename c4 to d8/cb 0 2024-05-13T22:48:20.103 INFO:tasks.workunit.client.0.smithi102.stdout:9/732: mkdir d1/d6/d45/d3e/dfe 0 2024-05-13T22:48:20.103 INFO:tasks.workunit.client.0.smithi102.stdout:2/903: write d7/d9/d10/d1b/d54/fc3 [296621,118792] 0 2024-05-13T22:48:20.104 INFO:tasks.workunit.client.0.smithi102.stdout:9/733: stat d1/d6/db/la4 0 2024-05-13T22:48:20.105 INFO:tasks.workunit.client.0.smithi102.stdout:9/734: write d1/d6/db/d4a/fd7 [2729678,66010] 0 2024-05-13T22:48:20.111 INFO:tasks.workunit.client.1.smithi130.stdout:8/20: unlink d1/d2/d4/f5 0 2024-05-13T22:48:20.112 INFO:tasks.workunit.client.0.smithi102.stdout:5/936: mknod de/d25/d43/df5/d112/c157 0 2024-05-13T22:48:20.114 INFO:tasks.workunit.client.1.smithi130.stdout:6/33: creat d4/fc x:0 0 0 2024-05-13T22:48:20.116 INFO:tasks.workunit.client.0.smithi102.stdout:4/819: dread d3/d19/d23/d3a/f4a [0,4194304] 0 2024-05-13T22:48:20.119 INFO:tasks.workunit.client.1.smithi130.stdout:9/34: mknod d1/d7/c9 0 2024-05-13T22:48:20.121 INFO:tasks.workunit.client.1.smithi130.stdout:4/25: mkdir d8 0 2024-05-13T22:48:20.121 INFO:tasks.workunit.client.1.smithi130.stdout:6/34: dwrite d4/fc [0,4194304] 0 2024-05-13T22:48:20.122 INFO:tasks.workunit.client.1.smithi130.stdout:6/35: chown d4 21724717 1 2024-05-13T22:48:20.122 INFO:tasks.workunit.client.0.smithi102.stdout:9/735: rename d1/d6/db/d30/d47/f6d to d1/d6/db/d30/d8a/fff 0 2024-05-13T22:48:20.126 INFO:tasks.workunit.client.0.smithi102.stdout:5/937: chown de/d25/d59/d7f/la9 2000807 1 2024-05-13T22:48:20.127 INFO:tasks.workunit.client.1.smithi130.stdout:3/28: mkdir d8/dc 0 2024-05-13T22:48:20.128 INFO:tasks.workunit.client.1.smithi130.stdout:6/36: dwrite f2 [4194304,4194304] 0 2024-05-13T22:48:20.129 INFO:tasks.workunit.client.1.smithi130.stdout:3/29: rename d8/dc to d8/dc/dd 22 2024-05-13T22:48:20.129 INFO:tasks.workunit.client.1.smithi130.stdout:6/37: stat l0 0 2024-05-13T22:48:20.129 INFO:tasks.workunit.client.1.smithi130.stdout:3/30: stat f2 0 2024-05-13T22:48:20.129 INFO:tasks.workunit.client.0.smithi102.stdout:3/861: getdents de/d67/df7 0 2024-05-13T22:48:20.130 INFO:tasks.workunit.client.1.smithi130.stdout:6/38: read d4/f9 [3774040,49436] 0 2024-05-13T22:48:20.130 INFO:tasks.workunit.client.1.smithi130.stdout:6/39: write f2 [7004523,66849] 0 2024-05-13T22:48:20.131 INFO:tasks.workunit.client.1.smithi130.stdout:6/40: readlink l0 0 2024-05-13T22:48:20.133 INFO:tasks.workunit.client.0.smithi102.stdout:4/820: rmdir d3/d4/d6 39 2024-05-13T22:48:20.136 INFO:tasks.workunit.client.0.smithi102.stdout:2/904: fsync d7/d9/d10/d25/d56/f8c 0 2024-05-13T22:48:20.138 INFO:tasks.workunit.client.0.smithi102.stdout:2/905: chown d7/d9/d14/d31/dbb/f126 6499 1 2024-05-13T22:48:20.141 INFO:tasks.workunit.client.0.smithi102.stdout:2/906: chown d7/d9/d10/d1b/d54/d83/lb7 13 1 2024-05-13T22:48:20.141 INFO:tasks.workunit.client.0.smithi102.stdout:1/618: dwrite d2/d10/f31 [0,4194304] 0 2024-05-13T22:48:20.141 INFO:tasks.workunit.client.0.smithi102.stdout:9/736: dread d1/f5c [0,4194304] 0 2024-05-13T22:48:20.148 INFO:tasks.workunit.client.1.smithi130.stdout:0/48: dwrite d0/f8 [0,4194304] 0 2024-05-13T22:48:20.148 INFO:tasks.workunit.client.1.smithi130.stdout:1/17: truncate f0 876857 0 2024-05-13T22:48:20.149 INFO:tasks.workunit.client.1.smithi130.stdout:4/26: dwrite f5 [0,4194304] 0 2024-05-13T22:48:20.150 INFO:tasks.workunit.client.1.smithi130.stdout:0/49: chown d0/f4 194 1 2024-05-13T22:48:20.151 INFO:tasks.workunit.client.1.smithi130.stdout:3/31: chown f5 1139903403 1 2024-05-13T22:48:20.152 INFO:tasks.workunit.client.1.smithi130.stdout:8/21: creat d1/d7/f8 x:0 0 0 2024-05-13T22:48:20.157 INFO:tasks.workunit.client.0.smithi102.stdout:9/737: unlink d1/d6/d45/d2d/ccd 0 2024-05-13T22:48:20.161 INFO:tasks.workunit.client.0.smithi102.stdout:5/938: truncate de/d25/d2a/d4b/d4d/d114/dda/fe3 1826209 0 2024-05-13T22:48:20.162 INFO:tasks.workunit.client.0.smithi102.stdout:3/862: rename de/d10/d34/d5d/c11d to de/d10/d2a/c11f 0 2024-05-13T22:48:20.164 INFO:tasks.workunit.client.0.smithi102.stdout:2/907: mknod d7/d15/d120/c127 0 2024-05-13T22:48:20.166 INFO:tasks.workunit.client.0.smithi102.stdout:1/619: fsync d2/d4/d5/d19/d28/d6a/f59 0 2024-05-13T22:48:20.168 INFO:tasks.workunit.client.0.smithi102.stdout:9/738: creat d1/d6/d45/d2d/dac/f100 x:0 0 0 2024-05-13T22:48:20.168 INFO:tasks.workunit.client.1.smithi130.stdout:0/50: symlink d0/d7/dd/lf 0 2024-05-13T22:48:20.169 INFO:tasks.workunit.client.0.smithi102.stdout:9/739: stat d1/d6/db/d30/c8b 0 2024-05-13T22:48:20.169 INFO:tasks.workunit.client.0.smithi102.stdout:5/939: read - de/d25/d2a/d4b/f10a zero size 2024-05-13T22:48:20.169 INFO:tasks.workunit.client.0.smithi102.stdout:4/821: mkdir d3/d4/d6/de/d115/d11b 0 2024-05-13T22:48:20.171 INFO:tasks.workunit.client.1.smithi130.stdout:6/41: link d4/fc d4/fd 0 2024-05-13T22:48:20.172 INFO:tasks.workunit.client.0.smithi102.stdout:2/908: write d7/d9/d10/d25/d27/f45 [3314625,65935] 0 2024-05-13T22:48:20.174 INFO:tasks.workunit.client.0.smithi102.stdout:1/620: dread - d2/d4/fb5 zero size 2024-05-13T22:48:20.174 INFO:tasks.workunit.client.0.smithi102.stdout:9/740: fdatasync d1/f5c 0 2024-05-13T22:48:20.176 INFO:tasks.workunit.client.1.smithi130.stdout:6/42: mknod d4/ce 0 2024-05-13T22:48:20.177 INFO:tasks.workunit.client.1.smithi130.stdout:9/35: getdents d1/d7 0 2024-05-13T22:48:20.177 INFO:tasks.workunit.client.1.smithi130.stdout:0/51: dread d0/fe [0,4194304] 0 2024-05-13T22:48:20.177 INFO:tasks.workunit.client.0.smithi102.stdout:9/741: read d1/d6/d45/d3e/f4c [6513805,61086] 0 2024-05-13T22:48:20.178 INFO:tasks.workunit.client.0.smithi102.stdout:4/822: creat d3/d4/d6/de/d8d/f11c x:0 0 0 2024-05-13T22:48:20.178 INFO:tasks.workunit.client.1.smithi130.stdout:9/36: chown d1/f4 1113 1 2024-05-13T22:48:20.178 INFO:tasks.workunit.client.1.smithi130.stdout:0/52: read - d0/fa zero size 2024-05-13T22:48:20.178 INFO:tasks.workunit.client.0.smithi102.stdout:9/742: dread - d1/d6/d45/d2d/dac/f100 zero size 2024-05-13T22:48:20.179 INFO:tasks.workunit.client.0.smithi102.stdout:9/743: stat d1/d6/db/d4a/f85 0 2024-05-13T22:48:20.180 INFO:tasks.workunit.client.0.smithi102.stdout:9/744: fsync d1/d6/d45/f4f 0 2024-05-13T22:48:20.180 INFO:tasks.workunit.client.1.smithi130.stdout:6/43: unlink f2 0 2024-05-13T22:48:20.181 INFO:tasks.workunit.client.0.smithi102.stdout:9/745: fdatasync d1/d6/db/d30/fe7 0 2024-05-13T22:48:20.181 INFO:tasks.workunit.client.1.smithi130.stdout:6/44: readlink l0 0 2024-05-13T22:48:20.182 INFO:tasks.workunit.client.1.smithi130.stdout:1/18: read f0 [217488,95372] 0 2024-05-13T22:48:20.183 INFO:tasks.workunit.client.1.smithi130.stdout:9/37: creat d1/d7/fa x:0 0 0 2024-05-13T22:48:20.184 INFO:tasks.workunit.client.1.smithi130.stdout:9/38: dread - d1/f4 zero size 2024-05-13T22:48:20.184 INFO:tasks.workunit.client.1.smithi130.stdout:8/22: getdents d1/d2/d4 0 2024-05-13T22:48:20.184 INFO:tasks.workunit.client.1.smithi130.stdout:9/39: dread - d1/f8 zero size 2024-05-13T22:48:20.186 INFO:tasks.workunit.client.0.smithi102.stdout:2/909: rename d7/l24 to d7/d15/l128 0 2024-05-13T22:48:20.187 INFO:tasks.workunit.client.0.smithi102.stdout:9/746: dwrite d1/d6/db/d30/d47/fe2 [0,4194304] 0 2024-05-13T22:48:20.187 INFO:tasks.workunit.client.0.smithi102.stdout:8/704: sync 2024-05-13T22:48:20.187 INFO:tasks.workunit.client.0.smithi102.stdout:7/691: sync 2024-05-13T22:48:20.190 INFO:tasks.workunit.client.0.smithi102.stdout:9/747: write d1/d6/db/d4a/d81/fe8 [1816156,87325] 0 2024-05-13T22:48:20.192 INFO:tasks.workunit.client.0.smithi102.stdout:9/748: readlink d1/d6/d21/d71/lc9 0 2024-05-13T22:48:20.193 INFO:tasks.workunit.client.1.smithi130.stdout:9/40: dwrite d1/f4 [0,4194304] 0 2024-05-13T22:48:20.193 INFO:tasks.workunit.client.0.smithi102.stdout:1/621: symlink d2/d4/d5/d19/d28/d6a/lc7 0 2024-05-13T22:48:20.194 INFO:tasks.workunit.client.1.smithi130.stdout:9/41: chown d1/l6 202 1 2024-05-13T22:48:20.195 INFO:tasks.workunit.client.0.smithi102.stdout:1/622: chown d2/d4/d26/d40/c6c 406129904 1 2024-05-13T22:48:20.195 INFO:tasks.workunit.client.0.smithi102.stdout:2/910: dwrite d7/f32 [0,4194304] 0 2024-05-13T22:48:20.195 INFO:tasks.workunit.client.1.smithi130.stdout:6/45: creat d4/ff x:0 0 0 2024-05-13T22:48:20.197 INFO:tasks.workunit.client.1.smithi130.stdout:3/32: write f5 [2774617,88914] 0 2024-05-13T22:48:20.198 INFO:tasks.workunit.client.1.smithi130.stdout:1/19: creat f5 x:0 0 0 2024-05-13T22:48:20.200 INFO:tasks.workunit.client.1.smithi130.stdout:0/53: dwrite d0/f5 [0,4194304] 0 2024-05-13T22:48:20.200 INFO:tasks.workunit.client.1.smithi130.stdout:6/46: write d4/ff [888769,76538] 0 2024-05-13T22:48:20.201 INFO:tasks.workunit.client.1.smithi130.stdout:8/23: rename f0 to d1/d2/d4/f9 0 2024-05-13T22:48:20.203 INFO:tasks.workunit.client.1.smithi130.stdout:8/24: chown d1/d7/f8 1 1 2024-05-13T22:48:20.206 INFO:tasks.workunit.client.1.smithi130.stdout:8/25: dread - d1/d2/f6 zero size 2024-05-13T22:48:20.207 INFO:tasks.workunit.client.0.smithi102.stdout:8/705: creat d2/d36/fde x:0 0 0 2024-05-13T22:48:20.208 INFO:tasks.workunit.client.1.smithi130.stdout:8/26: read - d1/d7/f8 zero size 2024-05-13T22:48:20.210 INFO:tasks.workunit.client.1.smithi130.stdout:9/42: dread d1/f4 [0,4194304] 0 2024-05-13T22:48:20.210 INFO:tasks.workunit.client.0.smithi102.stdout:3/863: truncate de/d10/d46/f6f 2907792 0 2024-05-13T22:48:20.210 INFO:tasks.workunit.client.1.smithi130.stdout:8/27: chown d1/d2/f6 383 1 2024-05-13T22:48:20.213 INFO:tasks.workunit.client.1.smithi130.stdout:3/33: creat d8/fe x:0 0 0 2024-05-13T22:48:20.213 INFO:tasks.workunit.client.1.smithi130.stdout:6/47: dread d4/fc [0,4194304] 0 2024-05-13T22:48:20.216 INFO:tasks.workunit.client.1.smithi130.stdout:0/54: mkdir d0/d7/d10 0 2024-05-13T22:48:20.216 INFO:tasks.workunit.client.0.smithi102.stdout:9/749: creat d1/d6/d21/d71/d91/f101 x:0 0 0 2024-05-13T22:48:20.219 INFO:tasks.workunit.client.1.smithi130.stdout:0/55: dread d0/f3 [0,4194304] 0 2024-05-13T22:48:20.220 INFO:tasks.workunit.client.1.smithi130.stdout:9/43: dwrite d1/f5 [0,4194304] 0 2024-05-13T22:48:20.221 INFO:tasks.workunit.client.1.smithi130.stdout:9/44: chown d1/f8 128 1 2024-05-13T22:48:20.224 INFO:tasks.workunit.client.0.smithi102.stdout:2/911: rename d7/d9/d10/d25 to d7/d15/d67/d129 0 2024-05-13T22:48:20.224 INFO:tasks.workunit.client.1.smithi130.stdout:1/20: fsync f5 0 2024-05-13T22:48:20.227 INFO:tasks.workunit.client.1.smithi130.stdout:6/48: symlink d4/l10 0 2024-05-13T22:48:20.227 INFO:tasks.workunit.client.0.smithi102.stdout:4/823: symlink d3/d22/dab/df9/l11d 0 2024-05-13T22:48:20.230 INFO:tasks.workunit.client.0.smithi102.stdout:8/706: symlink d2/d7/d25/d41/d6f/da0/ldf 0 2024-05-13T22:48:20.231 INFO:tasks.workunit.client.0.smithi102.stdout:3/864: mknod de/d10/d2a/d36/d88/c120 0 2024-05-13T22:48:20.232 INFO:tasks.workunit.client.1.smithi130.stdout:1/21: dwrite f5 [0,4194304] 0 2024-05-13T22:48:20.233 INFO:tasks.workunit.client.1.smithi130.stdout:9/45: mkdir d1/db 0 2024-05-13T22:48:20.235 INFO:tasks.workunit.client.1.smithi130.stdout:3/34: symlink d8/dc/lf 0 2024-05-13T22:48:20.236 INFO:tasks.workunit.client.0.smithi102.stdout:2/912: creat d7/d15/d67/d129/d27/dab/dee/f12a x:0 0 0 2024-05-13T22:48:20.237 INFO:tasks.workunit.client.0.smithi102.stdout:5/940: getdents de/d25/d2a 0 2024-05-13T22:48:20.238 INFO:tasks.workunit.client.1.smithi130.stdout:9/46: dread d1/f4 [0,4194304] 0 2024-05-13T22:48:20.240 INFO:tasks.workunit.client.1.smithi130.stdout:1/22: creat f6 x:0 0 0 2024-05-13T22:48:20.242 INFO:tasks.workunit.client.0.smithi102.stdout:8/707: creat d2/d7/d27/d64/d80/dc8/d40/fe0 x:0 0 0 2024-05-13T22:48:20.243 INFO:tasks.workunit.client.0.smithi102.stdout:8/708: write d2/d7/d27/d64/d80/dc8/d40/f5b [2953412,72987] 0 2024-05-13T22:48:20.244 INFO:tasks.workunit.client.0.smithi102.stdout:3/865: chown de/d10/d50/d63/d98/dee 8 1 2024-05-13T22:48:20.245 INFO:tasks.workunit.client.0.smithi102.stdout:7/692: write d6/d13/d2c/da1/dc1/f5f [254203,70049] 0 2024-05-13T22:48:20.247 INFO:tasks.workunit.client.1.smithi130.stdout:8/28: getdents d1/d7 0 2024-05-13T22:48:20.251 INFO:tasks.workunit.client.0.smithi102.stdout:1/623: write d2/d4/d26/f90 [133215,95626] 0 2024-05-13T22:48:20.253 INFO:tasks.workunit.client.1.smithi130.stdout:6/49: rmdir d4 39 2024-05-13T22:48:20.255 INFO:tasks.workunit.client.0.smithi102.stdout:8/709: creat d2/d7/d27/d64/d96/dae/fe1 x:0 0 0 2024-05-13T22:48:20.256 INFO:tasks.workunit.client.0.smithi102.stdout:8/710: chown d2/d36/db3 1228058760 1 2024-05-13T22:48:20.256 INFO:tasks.workunit.client.1.smithi130.stdout:0/56: getdents d0/d7/dd 0 2024-05-13T22:48:20.256 INFO:tasks.workunit.client.1.smithi130.stdout:0/57: read d0/fe [4398473,107630] 0 2024-05-13T22:48:20.257 INFO:tasks.workunit.client.1.smithi130.stdout:0/58: write d0/f4 [889100,126974] 0 2024-05-13T22:48:20.258 INFO:tasks.workunit.client.0.smithi102.stdout:3/866: mkdir de/d1c/d79/d121 0 2024-05-13T22:48:20.258 INFO:tasks.workunit.client.1.smithi130.stdout:0/59: write d0/f8 [4338020,19869] 0 2024-05-13T22:48:20.259 INFO:tasks.workunit.client.0.smithi102.stdout:3/867: write de/d10/d46/f6c [1131101,51619] 0 2024-05-13T22:48:20.261 INFO:tasks.workunit.client.1.smithi130.stdout:8/29: mknod d1/d2/d4/ca 0 2024-05-13T22:48:20.264 INFO:tasks.workunit.client.0.smithi102.stdout:7/693: dread d6/d15/d4c/f5a [0,4194304] 0 2024-05-13T22:48:20.265 INFO:tasks.workunit.client.0.smithi102.stdout:7/694: chown d6/d15/f3b 94509 1 2024-05-13T22:48:20.265 INFO:tasks.workunit.client.1.smithi130.stdout:9/47: link d1/d7/fa d1/db/fc 0 2024-05-13T22:48:20.266 INFO:tasks.workunit.client.0.smithi102.stdout:9/750: write d1/d6/db/d4a/f85 [319268,42540] 0 2024-05-13T22:48:20.266 INFO:tasks.workunit.client.1.smithi130.stdout:9/48: truncate d1/f8 239538 0 2024-05-13T22:48:20.268 INFO:tasks.workunit.client.1.smithi130.stdout:9/49: write d1/f8 [651073,75020] 0 2024-05-13T22:48:20.269 INFO:tasks.workunit.client.0.smithi102.stdout:1/624: dread - d2/d4/d26/d40/f93 zero size 2024-05-13T22:48:20.273 INFO:tasks.workunit.client.0.smithi102.stdout:4/824: write d3/d4/d6/de/d1f/d41/f65 [1782528,32355] 0 2024-05-13T22:48:20.275 INFO:tasks.workunit.client.0.smithi102.stdout:7/695: dwrite d6/d15/d4d/d55/f74 [0,4194304] 0 2024-05-13T22:48:20.278 INFO:tasks.workunit.client.0.smithi102.stdout:9/751: dwrite d1/d6/db/d30/f36 [0,4194304] 0 2024-05-13T22:48:20.279 INFO:tasks.workunit.client.0.smithi102.stdout:3/868: creat de/d10/d50/d63/d98/dee/f122 x:0 0 0 2024-05-13T22:48:20.287 INFO:tasks.workunit.client.0.smithi102.stdout:4/825: dwrite d3/d4/d6/de/d1f/f53 [0,4194304] 0 2024-05-13T22:48:20.292 INFO:tasks.workunit.client.0.smithi102.stdout:5/941: dwrite de/d25/d58/fdd [4194304,4194304] 0 2024-05-13T22:48:20.295 INFO:tasks.workunit.client.1.smithi130.stdout:3/35: getdents d8 0 2024-05-13T22:48:20.296 INFO:tasks.workunit.client.0.smithi102.stdout:2/913: dwrite d7/d9/d14/f73 [0,4194304] 0 2024-05-13T22:48:20.296 INFO:tasks.workunit.client.1.smithi130.stdout:3/36: chown d8/dc/lf 378402195 1 2024-05-13T22:48:20.296 INFO:tasks.workunit.client.1.smithi130.stdout:3/37: truncate d8/fe 564409 0 2024-05-13T22:48:20.298 INFO:tasks.workunit.client.1.smithi130.stdout:6/50: creat d4/f11 x:0 0 0 2024-05-13T22:48:20.299 INFO:tasks.workunit.client.1.smithi130.stdout:0/60: write d0/f3 [1804520,108963] 0 2024-05-13T22:48:20.307 INFO:tasks.workunit.client.0.smithi102.stdout:9/752: truncate d1/d6/fa5 2328852 0 2024-05-13T22:48:20.309 INFO:tasks.workunit.client.0.smithi102.stdout:9/753: write d1/d6/d21/d71/d91/fe0 [574559,88602] 0 2024-05-13T22:48:20.309 INFO:tasks.workunit.client.1.smithi130.stdout:1/23: getdents . 0 2024-05-13T22:48:20.310 INFO:tasks.workunit.client.1.smithi130.stdout:3/38: mknod d8/c10 0 2024-05-13T22:48:20.310 INFO:tasks.workunit.client.0.smithi102.stdout:9/754: chown d1/d6/d45/d3e/f93 5 1 2024-05-13T22:48:20.311 INFO:tasks.workunit.client.1.smithi130.stdout:6/51: creat d4/f12 x:0 0 0 2024-05-13T22:48:20.313 INFO:tasks.workunit.client.0.smithi102.stdout:5/942: mkdir de/d25/d43/df5/d158 0 2024-05-13T22:48:20.314 INFO:tasks.workunit.client.1.smithi130.stdout:0/61: dread d0/f3 [0,4194304] 0 2024-05-13T22:48:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:5/943: fdatasync de/d25/d43/f139 0 2024-05-13T22:48:20.314 INFO:tasks.workunit.client.0.smithi102.stdout:1/625: creat d2/d10/d62/d7b/dc4/fc8 x:0 0 0 2024-05-13T22:48:20.315 INFO:tasks.workunit.client.1.smithi130.stdout:0/62: fdatasync d0/fa 0 2024-05-13T22:48:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:2/914: rmdir d7/d15/d67/d129/d56/d59/d81 39 2024-05-13T22:48:20.315 INFO:tasks.workunit.client.0.smithi102.stdout:1/626: stat d2/d45/f47 0 2024-05-13T22:48:20.316 INFO:tasks.workunit.client.1.smithi130.stdout:6/52: dread d4/ff [0,4194304] 0 2024-05-13T22:48:20.316 INFO:tasks.workunit.client.0.smithi102.stdout:2/915: readlink d7/d9/d14/l8d 0 2024-05-13T22:48:20.317 INFO:tasks.workunit.client.0.smithi102.stdout:7/696: symlink d6/d15/d4d/d55/d6f/lce 0 2024-05-13T22:48:20.318 INFO:tasks.workunit.client.1.smithi130.stdout:3/39: dwrite f5 [0,4194304] 0 2024-05-13T22:48:20.321 INFO:tasks.workunit.client.0.smithi102.stdout:1/627: read d2/d4/d26/d40/f72 [3907777,111561] 0 2024-05-13T22:48:20.321 INFO:tasks.workunit.client.0.smithi102.stdout:8/711: getdents d2/d7/d27/d64/d96/dbc 0 2024-05-13T22:48:20.324 INFO:tasks.workunit.client.0.smithi102.stdout:8/712: chown d2/d7/d27/d64/d96/dae/d76/f90 262098 1 2024-05-13T22:48:20.398 INFO:tasks.workunit.client.0.smithi102.stdout:6/859: sync 2024-05-13T22:48:20.400 INFO:tasks.workunit.client.0.smithi102.stdout:9/755: rename d1/d5d to d1/d6/db/d4a/d75/d96/d102 0 2024-05-13T22:48:20.402 INFO:tasks.workunit.client.0.smithi102.stdout:5/944: creat de/d25/d59/d6f/f159 x:0 0 0 2024-05-13T22:48:20.405 INFO:tasks.workunit.client.0.smithi102.stdout:7/697: readlink d6/l25 0 2024-05-13T22:48:20.408 INFO:tasks.workunit.client.0.smithi102.stdout:2/916: mknod d7/d9/d14/d31/dbb/c12b 0 2024-05-13T22:48:20.413 INFO:tasks.workunit.client.0.smithi102.stdout:1/628: rmdir d2/d10/d62/d86 39 2024-05-13T22:48:20.414 INFO:tasks.workunit.client.0.smithi102.stdout:1/629: dread - d2/d4/d26/d40/d5e/fb6 zero size 2024-05-13T22:48:20.418 INFO:tasks.workunit.client.0.smithi102.stdout:6/860: mkdir d9/d25/d60/d6a/dd6/d114 0 2024-05-13T22:48:20.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:20 smithi102 ceph-mon[28599]: pgmap v27: 65 pgs: 65 active+clean; 1.6 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 59 MiB/s rd, 144 MiB/s wr, 487 op/s 2024-05-13T22:48:20.429 INFO:tasks.workunit.client.0.smithi102.stdout:3/869: dwrite de/d10/d2a/d36/f49 [0,4194304] 0 2024-05-13T22:48:20.429 INFO:tasks.workunit.client.0.smithi102.stdout:8/713: dread d2/f4 [0,4194304] 0 2024-05-13T22:48:20.431 INFO:tasks.workunit.client.0.smithi102.stdout:3/870: chown de/d10/d50/lf2 376962255 1 2024-05-13T22:48:20.432 INFO:tasks.workunit.client.0.smithi102.stdout:4/826: dwrite d3/d19/d23/d4e/d68/fe6 [0,4194304] 0 2024-05-13T22:48:20.441 INFO:tasks.workunit.client.0.smithi102.stdout:7/698: creat d6/d13/d2c/d39/db3/fcf x:0 0 0 2024-05-13T22:48:20.445 INFO:tasks.workunit.client.0.smithi102.stdout:8/714: rmdir d2/d36 39 2024-05-13T22:48:20.448 INFO:tasks.workunit.client.0.smithi102.stdout:1/630: mkdir d2/d4/d5/d19/dc9 0 2024-05-13T22:48:20.451 INFO:tasks.workunit.client.0.smithi102.stdout:1/631: write d2/d10/d62/d7b/dc4/fc8 [112272,90469] 0 2024-05-13T22:48:20.451 INFO:tasks.workunit.client.0.smithi102.stdout:3/871: chown de/d10/d50/d63/da2/lc7 52913 1 2024-05-13T22:48:20.455 INFO:tasks.workunit.client.0.smithi102.stdout:8/715: dwrite d2/d7/d27/d64/d96/dae/d76/f90 [4194304,4194304] 0 2024-05-13T22:48:20.462 INFO:tasks.workunit.client.0.smithi102.stdout:9/756: dwrite d1/d6/db/f1b [0,4194304] 0 2024-05-13T22:48:20.465 INFO:tasks.workunit.client.0.smithi102.stdout:5/945: dwrite de/d25/d43/f94 [0,4194304] 0 2024-05-13T22:48:20.466 INFO:tasks.workunit.client.0.smithi102.stdout:6/861: dread d9/db/d14/d54/fa9 [4194304,4194304] 0 2024-05-13T22:48:20.467 INFO:tasks.workunit.client.0.smithi102.stdout:3/872: symlink de/d10/d46/dc8/l123 0 2024-05-13T22:48:20.469 INFO:tasks.workunit.client.0.smithi102.stdout:6/862: readlink d9/db/d14/d54/lc7 0 2024-05-13T22:48:20.475 INFO:tasks.workunit.client.0.smithi102.stdout:2/917: dwrite d7/f75 [0,4194304] 0 2024-05-13T22:48:20.480 INFO:tasks.workunit.client.0.smithi102.stdout:9/757: dread d1/d6/db/f7a [0,4194304] 0 2024-05-13T22:48:20.486 INFO:tasks.workunit.client.0.smithi102.stdout:6/863: fsync d9/db/d14/dcc/f7b 0 2024-05-13T22:48:20.489 INFO:tasks.workunit.client.0.smithi102.stdout:1/632: symlink d2/d4/d5/d19/lca 0 2024-05-13T22:48:20.490 INFO:tasks.workunit.client.0.smithi102.stdout:3/873: dwrite de/d10/d2a/d33/d61/f96 [0,4194304] 0 2024-05-13T22:48:20.492 INFO:tasks.workunit.client.0.smithi102.stdout:7/699: rename d6/d15/d4c/d90 to d6/dd0 0 2024-05-13T22:48:20.495 INFO:tasks.workunit.client.0.smithi102.stdout:8/716: fsync d2/d7/d27/d64/d96/dae/f4f 0 2024-05-13T22:48:20.496 INFO:tasks.workunit.client.0.smithi102.stdout:0/759: sync 2024-05-13T22:48:20.497 INFO:tasks.workunit.client.0.smithi102.stdout:2/918: chown d7/d9/d10/d30/l47 1698819 1 2024-05-13T22:48:20.498 INFO:tasks.workunit.client.0.smithi102.stdout:4/827: link d3/c100 d3/d19/d23/d4e/d68/c11e 0 2024-05-13T22:48:20.511 INFO:tasks.workunit.client.0.smithi102.stdout:6/864: mknod d9/d45/d56/dd5/c115 0 2024-05-13T22:48:20.511 INFO:tasks.workunit.client.0.smithi102.stdout:9/758: dwrite d1/d6/d45/d2d/f8f [4194304,4194304] 0 2024-05-13T22:48:20.518 INFO:tasks.workunit.client.0.smithi102.stdout:6/865: dread d9/feb [0,4194304] 0 2024-05-13T22:48:20.519 INFO:tasks.workunit.client.0.smithi102.stdout:0/760: symlink d1/de/d32/lee 0 2024-05-13T22:48:20.523 INFO:tasks.workunit.client.0.smithi102.stdout:4/828: fdatasync d3/d22/fce 0 2024-05-13T22:48:20.524 INFO:tasks.workunit.client.0.smithi102.stdout:4/829: write d3/d19/d23/d3a/f4a [205376,66925] 0 2024-05-13T22:48:20.524 INFO:tasks.workunit.client.0.smithi102.stdout:5/946: link de/d25/d2c/l2d de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d11d/l15a 0 2024-05-13T22:48:20.528 INFO:tasks.workunit.client.0.smithi102.stdout:5/947: dread de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d34/f117 [0,4194304] 0 2024-05-13T22:48:20.548 INFO:tasks.workunit.client.1.smithi130.stdout:9/50: link d1/d7/c9 d1/db/cd 0 2024-05-13T22:48:20.549 INFO:tasks.workunit.client.1.smithi130.stdout:1/24: symlink l7 0 2024-05-13T22:48:20.550 INFO:tasks.workunit.client.1.smithi130.stdout:1/25: write f5 [3144323,98963] 0 2024-05-13T22:48:20.553 INFO:tasks.workunit.client.1.smithi130.stdout:0/63: mkdir d0/d11 0 2024-05-13T22:48:20.554 INFO:tasks.workunit.client.1.smithi130.stdout:6/53: mknod d4/c13 0 2024-05-13T22:48:20.555 INFO:tasks.workunit.client.1.smithi130.stdout:3/40: mknod d8/dc/c11 0 2024-05-13T22:48:20.557 INFO:tasks.workunit.client.1.smithi130.stdout:6/54: dread d4/fc [0,4194304] 0 2024-05-13T22:48:20.559 INFO:tasks.workunit.client.1.smithi130.stdout:1/26: creat f8 x:0 0 0 2024-05-13T22:48:20.562 INFO:tasks.workunit.client.1.smithi130.stdout:0/64: mknod d0/d7/c12 0 2024-05-13T22:48:20.563 INFO:tasks.workunit.client.1.smithi130.stdout:3/41: creat d8/dc/f12 x:0 0 0 2024-05-13T22:48:20.565 INFO:tasks.workunit.client.1.smithi130.stdout:6/55: symlink d4/l14 0 2024-05-13T22:48:20.568 INFO:tasks.workunit.client.1.smithi130.stdout:0/65: unlink d0/f5 0 2024-05-13T22:48:20.569 INFO:tasks.workunit.client.1.smithi130.stdout:3/42: dwrite d8/f9 [0,4194304] 0 2024-05-13T22:48:20.570 INFO:tasks.workunit.client.1.smithi130.stdout:6/56: mkdir d4/d15 0 2024-05-13T22:48:20.570 INFO:tasks.workunit.client.1.smithi130.stdout:3/43: truncate d8/dc/f12 842229 0 2024-05-13T22:48:20.571 INFO:tasks.workunit.client.1.smithi130.stdout:3/44: write d8/fa [478073,116748] 0 2024-05-13T22:48:20.572 INFO:tasks.workunit.client.1.smithi130.stdout:3/45: readlink d8/dc/lf 0 2024-05-13T22:48:20.572 INFO:tasks.workunit.client.1.smithi130.stdout:0/66: dread d0/f8 [0,4194304] 0 2024-05-13T22:48:20.573 INFO:tasks.workunit.client.1.smithi130.stdout:0/67: chown d0/fe 1012137 1 2024-05-13T22:48:20.575 INFO:tasks.workunit.client.1.smithi130.stdout:0/68: write d0/fa [736609,24370] 0 2024-05-13T22:48:20.580 INFO:tasks.workunit.client.1.smithi130.stdout:6/57: mkdir d4/d16 0 2024-05-13T22:48:20.583 INFO:tasks.workunit.client.1.smithi130.stdout:3/46: rename d8/dc/lf to d8/l13 0 2024-05-13T22:48:20.585 INFO:tasks.workunit.client.1.smithi130.stdout:0/69: mknod d0/c13 0 2024-05-13T22:48:20.585 INFO:tasks.workunit.client.1.smithi130.stdout:0/70: chown d0/d7/dd/lf 2887596 1 2024-05-13T22:48:20.587 INFO:tasks.workunit.client.1.smithi130.stdout:6/58: fdatasync d4/f9 0 2024-05-13T22:48:20.591 INFO:tasks.workunit.client.1.smithi130.stdout:3/47: rename d8/fe to d8/f14 0 2024-05-13T22:48:20.591 INFO:tasks.workunit.client.1.smithi130.stdout:3/48: stat d8/cb 0 2024-05-13T22:48:20.593 INFO:tasks.workunit.client.1.smithi130.stdout:3/49: dread f5 [0,4194304] 0 2024-05-13T22:48:20.594 INFO:tasks.workunit.client.1.smithi130.stdout:3/50: readlink l1 0 2024-05-13T22:48:20.594 INFO:tasks.workunit.client.1.smithi130.stdout:3/51: write f5 [3148975,93332] 0 2024-05-13T22:48:20.600 INFO:tasks.workunit.client.1.smithi130.stdout:6/59: mknod d4/d15/c17 0 2024-05-13T22:48:20.600 INFO:tasks.workunit.client.1.smithi130.stdout:6/60: dread - d4/f11 zero size 2024-05-13T22:48:20.602 INFO:tasks.workunit.client.1.smithi130.stdout:3/52: mkdir d8/dc/d15 0 2024-05-13T22:48:20.605 INFO:tasks.workunit.client.0.smithi102.stdout:3/874: mknod de/d10/c124 0 2024-05-13T22:48:20.606 INFO:tasks.workunit.client.1.smithi130.stdout:1/27: getdents . 0 2024-05-13T22:48:20.606 INFO:tasks.workunit.client.1.smithi130.stdout:0/71: link d0/c1 d0/d7/dd/c14 0 2024-05-13T22:48:20.607 INFO:tasks.workunit.client.1.smithi130.stdout:0/72: truncate d0/fa 1115795 0 2024-05-13T22:48:20.609 INFO:tasks.workunit.client.1.smithi130.stdout:1/28: dread f5 [0,4194304] 0 2024-05-13T22:48:20.611 INFO:tasks.workunit.client.1.smithi130.stdout:3/53: creat d8/f16 x:0 0 0 2024-05-13T22:48:20.612 INFO:tasks.workunit.client.1.smithi130.stdout:3/54: chown c7 54 1 2024-05-13T22:48:20.614 INFO:tasks.workunit.client.0.smithi102.stdout:8/717: mkdir d2/d36/d49/d7e/de2 0 2024-05-13T22:48:20.616 INFO:tasks.workunit.client.0.smithi102.stdout:6/866: rmdir d9/d45/d56/d5d 39 2024-05-13T22:48:20.617 INFO:tasks.workunit.client.0.smithi102.stdout:6/867: chown d9/d25/faa 293777 1 2024-05-13T22:48:20.618 INFO:tasks.workunit.client.1.smithi130.stdout:1/29: mknod c9 0 2024-05-13T22:48:20.618 INFO:tasks.workunit.client.1.smithi130.stdout:1/30: dread - f8 zero size 2024-05-13T22:48:20.620 INFO:tasks.workunit.client.1.smithi130.stdout:3/55: symlink d8/dc/l17 0 2024-05-13T22:48:20.623 INFO:tasks.workunit.client.1.smithi130.stdout:0/73: link d0/fa d0/d7/d10/f15 0 2024-05-13T22:48:20.625 INFO:tasks.workunit.client.0.smithi102.stdout:4/830: mknod d3/d22/dab/df9/c11f 0 2024-05-13T22:48:20.627 INFO:tasks.workunit.client.1.smithi130.stdout:1/31: rename c3 to ca 0 2024-05-13T22:48:20.630 INFO:tasks.workunit.client.0.smithi102.stdout:5/948: creat de/d25/d58/dfb/f15b x:0 0 0 2024-05-13T22:48:20.631 INFO:tasks.workunit.client.1.smithi130.stdout:0/74: creat d0/d7/f16 x:0 0 0 2024-05-13T22:48:20.634 INFO:tasks.workunit.client.0.smithi102.stdout:3/875: rename de/d10/f44 to de/d10/d50/d63/da2/f125 0 2024-05-13T22:48:20.634 INFO:tasks.workunit.client.1.smithi130.stdout:3/56: getdents d8/dc/d15 0 2024-05-13T22:48:20.635 INFO:tasks.workunit.client.1.smithi130.stdout:0/75: rmdir d0/d7/d10 39 2024-05-13T22:48:20.638 INFO:tasks.workunit.client.0.smithi102.stdout:6/868: unlink d9/d45/f75 0 2024-05-13T22:48:20.640 INFO:tasks.workunit.client.0.smithi102.stdout:0/761: readlink d1/de/d32/d8a/lb9 0 2024-05-13T22:48:20.641 INFO:tasks.workunit.client.1.smithi130.stdout:0/76: mkdir d0/d7/dd/d17 0 2024-05-13T22:48:20.647 INFO:tasks.workunit.client.0.smithi102.stdout:5/949: chown de/l74 422 1 2024-05-13T22:48:20.648 INFO:tasks.workunit.client.0.smithi102.stdout:1/633: link d2/d4/d26/da2/lb9 d2/d4/d26/lcb 0 2024-05-13T22:48:20.649 INFO:tasks.workunit.client.1.smithi130.stdout:0/77: link d0/d7/dd/lf d0/d7/d10/l18 0 2024-05-13T22:48:20.650 INFO:tasks.workunit.client.0.smithi102.stdout:9/759: write d1/d6/f1d [1509460,646] 0 2024-05-13T22:48:20.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:20 smithi130 ceph-mon[36591]: pgmap v27: 65 pgs: 65 active+clean; 1.6 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 59 MiB/s rd, 144 MiB/s wr, 487 op/s 2024-05-13T22:48:20.651 INFO:tasks.workunit.client.0.smithi102.stdout:9/760: chown d1/d6/db/d30/cd3 36 1 2024-05-13T22:48:20.651 INFO:tasks.workunit.client.0.smithi102.stdout:8/718: rename d2/d7/d25/d41/d6f/f8a to d2/d36/d49/d5d/fe3 0 2024-05-13T22:48:20.653 INFO:tasks.workunit.client.0.smithi102.stdout:3/876: mknod de/d10/d2a/d36/db2/ddc/c126 0 2024-05-13T22:48:20.653 INFO:tasks.workunit.client.1.smithi130.stdout:6/61: dwrite d4/fc [0,4194304] 0 2024-05-13T22:48:20.654 INFO:tasks.workunit.client.0.smithi102.stdout:7/700: getdents d6/d15 0 2024-05-13T22:48:20.658 INFO:tasks.workunit.client.1.smithi130.stdout:0/78: creat d0/d11/f19 x:0 0 0 2024-05-13T22:48:20.660 INFO:tasks.workunit.client.1.smithi130.stdout:6/62: dwrite d4/f12 [0,4194304] 0 2024-05-13T22:48:20.661 INFO:tasks.workunit.client.0.smithi102.stdout:0/762: unlink d1/de/d32/d8a/ca8 0 2024-05-13T22:48:20.662 INFO:tasks.workunit.client.0.smithi102.stdout:3/877: dwrite de/d10/d50/d63/d98/dee/f122 [0,4194304] 0 2024-05-13T22:48:20.665 INFO:tasks.workunit.client.0.smithi102.stdout:2/919: getdents d7/d9/d10/d30/d53 0 2024-05-13T22:48:20.666 INFO:tasks.workunit.client.1.smithi130.stdout:0/79: rename d0/d11/f19 to d0/d7/dd/d17/f1a 0 2024-05-13T22:48:20.667 INFO:tasks.workunit.client.1.smithi130.stdout:1/32: getdents . 0 2024-05-13T22:48:20.668 INFO:tasks.workunit.client.1.smithi130.stdout:1/33: chown l7 2 1 2024-05-13T22:48:20.668 INFO:tasks.workunit.client.1.smithi130.stdout:1/34: stat l7 0 2024-05-13T22:48:20.672 INFO:tasks.workunit.client.0.smithi102.stdout:9/761: dread d1/d6/f60 [0,4194304] 0 2024-05-13T22:48:20.672 INFO:tasks.workunit.client.0.smithi102.stdout:9/762: stat d1/d6b/l72 0 2024-05-13T22:48:20.676 INFO:tasks.workunit.client.0.smithi102.stdout:8/719: symlink d2/d36/d49/d7e/le4 0 2024-05-13T22:48:20.677 INFO:tasks.workunit.client.1.smithi130.stdout:0/80: creat d0/d7/dd/d17/f1b x:0 0 0 2024-05-13T22:48:20.680 INFO:tasks.workunit.client.1.smithi130.stdout:1/35: mkdir db 0 2024-05-13T22:48:20.680 INFO:tasks.workunit.client.0.smithi102.stdout:4/831: dwrite d3/d19/d23/d3a/f83 [0,4194304] 0 2024-05-13T22:48:20.682 INFO:tasks.workunit.client.0.smithi102.stdout:6/869: write d9/d45/d56/dd5/fbd [48807,62609] 0 2024-05-13T22:48:20.682 INFO:tasks.workunit.client.1.smithi130.stdout:0/81: dread d0/f3 [0,4194304] 0 2024-05-13T22:48:20.688 INFO:tasks.workunit.client.0.smithi102.stdout:2/920: creat d7/d15/d120/f12c x:0 0 0 2024-05-13T22:48:20.690 INFO:tasks.workunit.client.1.smithi130.stdout:0/82: creat d0/d7/dd/f1c x:0 0 0 2024-05-13T22:48:20.691 INFO:tasks.workunit.client.1.smithi130.stdout:6/63: getdents d4 0 2024-05-13T22:48:20.691 INFO:tasks.workunit.client.1.smithi130.stdout:6/64: chown d4/c5 2 1 2024-05-13T22:48:20.692 INFO:tasks.workunit.client.0.smithi102.stdout:1/634: link d2/d4/d5/fb4 d2/d10/d62/d7b/dc4/fcc 0 2024-05-13T22:48:20.692 INFO:tasks.workunit.client.1.smithi130.stdout:6/65: fsync d4/f11 0 2024-05-13T22:48:20.692 INFO:tasks.workunit.client.1.smithi130.stdout:0/83: readlink d0/d7/dd/lf 0 2024-05-13T22:48:20.695 INFO:tasks.workunit.client.1.smithi130.stdout:6/66: rename d4/cb to d4/d15/c18 0 2024-05-13T22:48:20.696 INFO:tasks.workunit.client.0.smithi102.stdout:0/763: unlink d1/de/d32/dd8/le8 0 2024-05-13T22:48:20.697 INFO:tasks.workunit.client.1.smithi130.stdout:0/84: unlink d0/d7/c12 0 2024-05-13T22:48:20.698 INFO:tasks.workunit.client.0.smithi102.stdout:6/870: creat d9/d45/d56/dd5/f116 x:0 0 0 2024-05-13T22:48:20.699 INFO:tasks.workunit.client.1.smithi130.stdout:6/67: unlink l0 0 2024-05-13T22:48:20.699 INFO:tasks.workunit.client.1.smithi130.stdout:6/68: chown d4/l10 1459 1 2024-05-13T22:48:20.702 INFO:tasks.workunit.client.0.smithi102.stdout:7/701: rename d6/d13/de to d6/d13/d58/dd1 0 2024-05-13T22:48:20.702 INFO:tasks.workunit.client.0.smithi102.stdout:6/871: dread d9/db/f102 [0,4194304] 0 2024-05-13T22:48:20.704 INFO:tasks.workunit.client.0.smithi102.stdout:5/950: write f0 [3402517,12015] 0 2024-05-13T22:48:20.704 INFO:tasks.workunit.client.0.smithi102.stdout:0/764: dwrite d1/d68/fdf [0,4194304] 0 2024-05-13T22:48:20.706 INFO:tasks.workunit.client.1.smithi130.stdout:6/69: unlink d4/fd 0 2024-05-13T22:48:20.707 INFO:tasks.workunit.client.0.smithi102.stdout:8/720: dread d2/d7/d27/d64/d96/dae/f13 [0,4194304] 0 2024-05-13T22:48:20.711 INFO:tasks.workunit.client.0.smithi102.stdout:9/763: creat d1/d6/db/d4a/f103 x:0 0 0 2024-05-13T22:48:20.712 INFO:tasks.workunit.client.1.smithi130.stdout:6/70: dwrite d4/f9 [0,4194304] 0 2024-05-13T22:48:20.713 INFO:tasks.workunit.client.0.smithi102.stdout:7/702: mkdir d6/d22/dd2 0 2024-05-13T22:48:20.714 INFO:tasks.workunit.client.0.smithi102.stdout:7/703: readlink d6/d13/d2c/l9d 0 2024-05-13T22:48:20.715 INFO:tasks.workunit.client.1.smithi130.stdout:6/71: creat d4/f19 x:0 0 0 2024-05-13T22:48:20.716 INFO:tasks.workunit.client.0.smithi102.stdout:0/765: truncate d1/fb3 1968543 0 2024-05-13T22:48:20.717 INFO:tasks.workunit.client.0.smithi102.stdout:2/921: creat d7/d9/d10/d1b/d54/f12d x:0 0 0 2024-05-13T22:48:20.722 INFO:tasks.workunit.client.0.smithi102.stdout:6/872: symlink d9/d25/l117 0 2024-05-13T22:48:20.723 INFO:tasks.workunit.client.0.smithi102.stdout:6/873: dread - d9/d25/d5a/f10f zero size 2024-05-13T22:48:20.724 INFO:tasks.workunit.client.0.smithi102.stdout:7/704: creat d6/d13/d2c/d39/db3/fd3 x:0 0 0 2024-05-13T22:48:20.727 INFO:tasks.workunit.client.0.smithi102.stdout:8/721: link d2/d7/d27/dbb/dc7/fd9 d2/d7/d27/dad/fe5 0 2024-05-13T22:48:20.729 INFO:tasks.workunit.client.0.smithi102.stdout:3/878: rename de/d10/d2a/d8c/d7f/fed to de/d10/f127 0 2024-05-13T22:48:20.732 INFO:tasks.workunit.client.0.smithi102.stdout:5/951: creat de/d25/f15c x:0 0 0 2024-05-13T22:48:20.739 INFO:tasks.workunit.client.0.smithi102.stdout:6/874: read d9/d45/d56/d5d/f87 [3186335,51312] 0 2024-05-13T22:48:20.741 INFO:tasks.workunit.client.0.smithi102.stdout:0/766: creat d1/d4d/fef x:0 0 0 2024-05-13T22:48:20.742 INFO:tasks.workunit.client.0.smithi102.stdout:8/722: unlink d2/d36/d49/l7c 0 2024-05-13T22:48:20.745 INFO:tasks.workunit.client.0.smithi102.stdout:0/767: fdatasync d1/de/d42/f93 0 2024-05-13T22:48:20.746 INFO:tasks.workunit.client.0.smithi102.stdout:6/875: truncate d9/d25/d60/d6a/dd6/de9/fed 707822 0 2024-05-13T22:48:20.747 INFO:tasks.workunit.client.0.smithi102.stdout:9/764: dwrite d1/d6/db/d4a/d75/d96/d102/d6c/ff9 [0,4194304] 0 2024-05-13T22:48:20.748 INFO:tasks.workunit.client.0.smithi102.stdout:8/723: read d2/d36/d49/d5d/f72 [1756428,121202] 0 2024-05-13T22:48:20.749 INFO:tasks.workunit.client.0.smithi102.stdout:5/952: dread de/d25/d59/f32 [4194304,4194304] 0 2024-05-13T22:48:20.750 INFO:tasks.workunit.client.0.smithi102.stdout:4/832: rename d3/d19/d23/d39/dac/fe5 to d3/d19/d23/d39/da7/f120 0 2024-05-13T22:48:20.754 INFO:tasks.workunit.client.1.smithi130.stdout:6/72: write d4/fc [5083257,20194] 0 2024-05-13T22:48:20.757 INFO:tasks.workunit.client.0.smithi102.stdout:8/724: truncate d2/f4 3401474 0 2024-05-13T22:48:20.757 INFO:tasks.workunit.client.0.smithi102.stdout:9/765: read d1/f3a [219326,73047] 0 2024-05-13T22:48:20.758 INFO:tasks.workunit.client.0.smithi102.stdout:7/705: dwrite d6/d15/d4d/d55/f8e [0,4194304] 0 2024-05-13T22:48:20.759 INFO:tasks.workunit.client.1.smithi130.stdout:6/73: dread d4/f9 [0,4194304] 0 2024-05-13T22:48:20.759 INFO:tasks.workunit.client.1.smithi130.stdout:6/74: dread - d4/f8 zero size 2024-05-13T22:48:20.765 INFO:tasks.workunit.client.0.smithi102.stdout:9/766: truncate d1/d6/db/d4a/d75/d96/d102/d6c/ff9 4215055 0 2024-05-13T22:48:20.765 INFO:tasks.workunit.client.1.smithi130.stdout:6/75: mknod d4/c1a 0 2024-05-13T22:48:20.766 INFO:tasks.workunit.client.0.smithi102.stdout:4/833: read d3/d19/d23/d39/f8b [3184522,79372] 0 2024-05-13T22:48:20.766 INFO:tasks.workunit.client.0.smithi102.stdout:6/876: write d9/d45/f6f [760723,47656] 0 2024-05-13T22:48:20.767 INFO:tasks.workunit.client.0.smithi102.stdout:7/706: dwrite d6/fc [0,4194304] 0 2024-05-13T22:48:20.772 INFO:tasks.workunit.client.0.smithi102.stdout:8/725: creat d2/d7/d27/d64/d80/dc8/d93/fe6 x:0 0 0 2024-05-13T22:48:20.775 INFO:tasks.workunit.client.1.smithi130.stdout:6/76: dwrite d4/f9 [0,4194304] 0 2024-05-13T22:48:20.777 INFO:tasks.workunit.client.0.smithi102.stdout:5/953: mknod de/d25/d59/d7f/d86/da5/d105/c15d 0 2024-05-13T22:48:20.781 INFO:tasks.workunit.client.0.smithi102.stdout:6/877: creat d9/d25/d60/d6a/dd6/f118 x:0 0 0 2024-05-13T22:48:20.782 INFO:tasks.workunit.client.0.smithi102.stdout:7/707: mkdir d6/d22/dd4 0 2024-05-13T22:48:20.783 INFO:tasks.workunit.client.0.smithi102.stdout:7/708: dread - d6/d15/f47 zero size 2024-05-13T22:48:20.784 INFO:tasks.workunit.client.1.smithi130.stdout:6/77: mknod d4/d16/c1b 0 2024-05-13T22:48:20.784 INFO:tasks.workunit.client.1.smithi130.stdout:6/78: dread - d4/f11 zero size 2024-05-13T22:48:20.785 INFO:tasks.workunit.client.0.smithi102.stdout:4/834: truncate d3/d19/d23/d3a/f56 343713 0 2024-05-13T22:48:20.785 INFO:tasks.workunit.client.0.smithi102.stdout:8/726: dread - d2/f73 zero size 2024-05-13T22:48:20.785 INFO:tasks.workunit.client.0.smithi102.stdout:4/835: chown d3/d19/d23/d39/ddd 106377 1 2024-05-13T22:48:20.787 INFO:tasks.workunit.client.0.smithi102.stdout:1/635: rename d2/d4/d26/d40/d5e/l7a to d2/d10/lcd 0 2024-05-13T22:48:20.788 INFO:tasks.workunit.client.0.smithi102.stdout:6/878: read d9/db/d14/d35/f52 [2693232,23279] 0 2024-05-13T22:48:20.788 INFO:tasks.workunit.client.0.smithi102.stdout:0/768: dread d1/de/d32/dab/fbc [0,4194304] 0 2024-05-13T22:48:20.789 INFO:tasks.workunit.client.0.smithi102.stdout:5/954: fsync de/d25/d59/f5d 0 2024-05-13T22:48:20.790 INFO:tasks.workunit.client.0.smithi102.stdout:6/879: fdatasync d9/d45/d56/dd5/f116 0 2024-05-13T22:48:20.791 INFO:tasks.workunit.client.1.smithi130.stdout:6/79: mknod d4/d15/c1c 0 2024-05-13T22:48:20.794 INFO:tasks.workunit.client.0.smithi102.stdout:7/709: mknod d6/d15/d4d/d55/d6f/d49/d97/cd5 0 2024-05-13T22:48:20.794 INFO:tasks.workunit.client.1.smithi130.stdout:6/80: mkdir d4/d15/d1d 0 2024-05-13T22:48:20.797 INFO:tasks.workunit.client.0.smithi102.stdout:9/767: write d1/d6/d45/d3e/d79/fae [1540583,14071] 0 2024-05-13T22:48:20.797 INFO:tasks.workunit.client.1.smithi130.stdout:6/81: readlink d4/l10 0 2024-05-13T22:48:20.798 INFO:tasks.workunit.client.0.smithi102.stdout:2/922: rename d7/d15/d1e/dbf/dfe/f113 to d7/d9/d14/d31/dd0/f12e 0 2024-05-13T22:48:20.801 INFO:tasks.workunit.client.1.smithi130.stdout:6/82: symlink d4/d15/d1d/l1e 0 2024-05-13T22:48:20.805 INFO:tasks.workunit.client.0.smithi102.stdout:6/880: mkdir d9/d45/da0/d119 0 2024-05-13T22:48:20.805 INFO:tasks.workunit.client.0.smithi102.stdout:6/881: fsync d9/fec 0 2024-05-13T22:48:20.806 INFO:tasks.workunit.client.0.smithi102.stdout:7/710: symlink d6/d13/d2c/d39/db3/ld6 0 2024-05-13T22:48:20.807 INFO:tasks.workunit.client.0.smithi102.stdout:4/836: write d3/d4/f2e [3404450,117733] 0 2024-05-13T22:48:20.808 INFO:tasks.workunit.client.0.smithi102.stdout:9/768: mknod d1/d6/db/d4a/d75/d96/dc6/c104 0 2024-05-13T22:48:20.812 INFO:tasks.workunit.client.0.smithi102.stdout:3/879: rename de/d10/d2a/f7e to de/d10/d46/d100/f128 0 2024-05-13T22:48:20.814 INFO:tasks.workunit.client.0.smithi102.stdout:5/955: write de/d25/d2c/f8a [927416,91441] 0 2024-05-13T22:48:20.817 INFO:tasks.workunit.client.0.smithi102.stdout:4/837: creat d3/d4/d6/f121 x:0 0 0 2024-05-13T22:48:20.817 INFO:tasks.workunit.client.0.smithi102.stdout:0/769: dread d1/de/d32/f17 [4194304,4194304] 0 2024-05-13T22:48:20.837 INFO:tasks.workunit.client.0.smithi102.stdout:8/727: link d2/d36/db3/lb8 d2/d7/d25/d41/d6f/da0/le7 0 2024-05-13T22:48:20.837 INFO:tasks.workunit.client.0.smithi102.stdout:2/923: mkdir d7/d15/d67/d129/d56/d59/d81/dcc/d12f 0 2024-05-13T22:48:20.838 INFO:tasks.workunit.client.0.smithi102.stdout:6/882: dread d9/db/d14/d35/f67 [0,4194304] 0 2024-05-13T22:48:20.838 INFO:tasks.workunit.client.0.smithi102.stdout:5/956: creat de/d25/d2a/d4b/d4d/d114/f15e x:0 0 0 2024-05-13T22:48:20.840 INFO:tasks.workunit.client.0.smithi102.stdout:9/769: write d1/d6/d21/d71/d91/fda [113761,58812] 0 2024-05-13T22:48:20.843 INFO:tasks.workunit.client.0.smithi102.stdout:1/636: rename d2/d4/f37 to d2/d4/d5/fce 0 2024-05-13T22:48:20.845 INFO:tasks.workunit.client.0.smithi102.stdout:9/770: mkdir d1/d6/d45/dd0/d105 0 2024-05-13T22:48:20.849 INFO:tasks.workunit.client.0.smithi102.stdout:4/838: rmdir d3/d19/d23/d4e/da6 0 2024-05-13T22:48:20.853 INFO:tasks.workunit.client.0.smithi102.stdout:6/883: link d9/db/d14/f10e d9/da3/de8/f11a 0 2024-05-13T22:48:20.857 INFO:tasks.workunit.client.0.smithi102.stdout:9/771: fsync d1/d6/d3b/f38 0 2024-05-13T22:48:20.859 INFO:tasks.workunit.client.0.smithi102.stdout:8/728: dwrite d2/d7/d27/d64/d80/fd6 [0,4194304] 0 2024-05-13T22:48:20.860 INFO:tasks.workunit.client.0.smithi102.stdout:5/957: dwrite de/d25/d58/dfb/f12f [0,4194304] 0 2024-05-13T22:48:20.863 INFO:tasks.workunit.client.0.smithi102.stdout:4/839: rename d3/d19/d23/d4e/f55 to d3/d19/d23/d7e/f122 0 2024-05-13T22:48:20.864 INFO:tasks.workunit.client.0.smithi102.stdout:2/924: dwrite d7/d15/d67/d129/d56/d59/d81/f110 [0,4194304] 0 2024-05-13T22:48:20.868 INFO:tasks.workunit.client.0.smithi102.stdout:0/770: dwrite d1/de/d32/d24/f3c [4194304,4194304] 0 2024-05-13T22:48:20.868 INFO:tasks.workunit.client.0.smithi102.stdout:2/925: fsync d7/ff 0 2024-05-13T22:48:20.869 INFO:tasks.workunit.client.0.smithi102.stdout:9/772: mkdir d1/d6/db/d4a/d75/d96/d102/d6c/d106 0 2024-05-13T22:48:20.871 INFO:tasks.workunit.client.0.smithi102.stdout:0/771: chown d1/de/f50 5 1 2024-05-13T22:48:20.872 INFO:tasks.workunit.client.0.smithi102.stdout:5/958: chown de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d53/f107 36785 1 2024-05-13T22:48:20.872 INFO:tasks.workunit.client.0.smithi102.stdout:0/772: stat d1/d4d/d79 0 2024-05-13T22:48:20.874 INFO:tasks.workunit.client.0.smithi102.stdout:2/926: write d7/d9/d10/d30/d7f/ff2 [105699,38084] 0 2024-05-13T22:48:20.874 INFO:tasks.workunit.client.0.smithi102.stdout:4/840: mkdir d3/d19/dad/dc9/d123 0 2024-05-13T22:48:20.876 INFO:tasks.workunit.client.0.smithi102.stdout:9/773: rmdir d1/d6/db 39 2024-05-13T22:48:20.876 INFO:tasks.workunit.client.0.smithi102.stdout:0/773: fdatasync d1/de/d32/d24/d2d/d38/db7/fdb 0 2024-05-13T22:48:20.877 INFO:tasks.workunit.client.0.smithi102.stdout:5/959: unlink de/l155 0 2024-05-13T22:48:20.879 INFO:tasks.workunit.client.0.smithi102.stdout:1/637: dread d2/d4/d5/d19/f4e [0,4194304] 0 2024-05-13T22:48:20.883 INFO:tasks.workunit.client.0.smithi102.stdout:2/927: rename d7/d9/d10/d1b/d54/cff to d7/d9/d14/d31/c130 0 2024-05-13T22:48:20.884 INFO:tasks.workunit.client.0.smithi102.stdout:0/774: dread d1/de/d32/f9c [0,4194304] 0 2024-05-13T22:48:20.885 INFO:tasks.workunit.client.0.smithi102.stdout:5/960: symlink de/d25/d2a/d4b/l15f 0 2024-05-13T22:48:20.887 INFO:tasks.workunit.client.0.smithi102.stdout:1/638: creat d2/d10/d62/d7b/dc4/fcf x:0 0 0 2024-05-13T22:48:20.887 INFO:tasks.workunit.client.0.smithi102.stdout:8/729: getdents d2/d7/d27/d64/d96/dbc 0 2024-05-13T22:48:20.888 INFO:tasks.workunit.client.0.smithi102.stdout:9/774: creat d1/d6/d45/dd0/d105/f107 x:0 0 0 2024-05-13T22:48:20.888 INFO:tasks.workunit.client.0.smithi102.stdout:4/841: rmdir d3/d19/d23/d3a/de4 0 2024-05-13T22:48:20.889 INFO:tasks.workunit.client.0.smithi102.stdout:0/775: fdatasync d1/de/d32/d24/d2d/d38/db7/feb 0 2024-05-13T22:48:20.891 INFO:tasks.workunit.client.0.smithi102.stdout:1/639: creat d2/d45/d96/fd0 x:0 0 0 2024-05-13T22:48:20.892 INFO:tasks.workunit.client.0.smithi102.stdout:5/961: symlink de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d11d/l160 0 2024-05-13T22:48:20.894 INFO:tasks.workunit.client.0.smithi102.stdout:8/730: mkdir d2/de8 0 2024-05-13T22:48:20.895 INFO:tasks.workunit.client.0.smithi102.stdout:5/962: write de/d25/d43/df5/f14e [442108,109608] 0 2024-05-13T22:48:20.898 INFO:tasks.workunit.client.0.smithi102.stdout:1/640: creat d2/d4/d5/d19/d28/fd1 x:0 0 0 2024-05-13T22:48:20.898 INFO:tasks.workunit.client.0.smithi102.stdout:0/776: dwrite d1/d68/f86 [0,4194304] 0 2024-05-13T22:48:20.900 INFO:tasks.workunit.client.0.smithi102.stdout:8/731: symlink d2/d7/d25/le9 0 2024-05-13T22:48:20.900 INFO:tasks.workunit.client.0.smithi102.stdout:9/775: rename d1/d6/db/c3d to d1/d6/db/d77/c108 0 2024-05-13T22:48:20.902 INFO:tasks.workunit.client.0.smithi102.stdout:1/641: mknod d2/d45/d79/cd2 0 2024-05-13T22:48:20.903 INFO:tasks.workunit.client.0.smithi102.stdout:0/777: creat d1/de/d42/d61/ff0 x:0 0 0 2024-05-13T22:48:20.904 INFO:tasks.workunit.client.0.smithi102.stdout:9/776: read d1/f17 [369393,99645] 0 2024-05-13T22:48:20.904 INFO:tasks.workunit.client.0.smithi102.stdout:5/963: mkdir de/d25/d108/dc5/dca/d12b/d161 0 2024-05-13T22:48:20.905 INFO:tasks.workunit.client.0.smithi102.stdout:1/642: chown d2/d4/d26/d40/l42 183292151 1 2024-05-13T22:48:20.910 INFO:tasks.workunit.client.0.smithi102.stdout:9/777: mknod d1/d6/d45/d3e/dbe/c109 0 2024-05-13T22:48:20.910 INFO:tasks.workunit.client.0.smithi102.stdout:8/732: dwrite d2/d7/d27/d64/d96/dae/f23 [0,4194304] 0 2024-05-13T22:48:20.913 INFO:tasks.workunit.client.0.smithi102.stdout:9/778: chown d1/d6/db/d4a/d75/d96/cf8 83172710 1 2024-05-13T22:48:20.913 INFO:tasks.workunit.client.0.smithi102.stdout:1/643: dread - d2/d4/d5/d19/d28/d6a/f9c zero size 2024-05-13T22:48:20.913 INFO:tasks.workunit.client.0.smithi102.stdout:8/733: write d2/d36/d49/d5d/f26 [1070963,33481] 0 2024-05-13T22:48:20.915 INFO:tasks.workunit.client.0.smithi102.stdout:0/778: creat d1/de/d32/d24/ff1 x:0 0 0 2024-05-13T22:48:20.919 INFO:tasks.workunit.client.0.smithi102.stdout:5/964: rename de/d25/d59/d7f/d86/da5/f10e to de/d25/d2a/d4b/d4d/d50/d6c/df8/f162 0 2024-05-13T22:48:20.922 INFO:tasks.workunit.client.0.smithi102.stdout:5/965: write de/d25/d58/fa6 [327362,67149] 0 2024-05-13T22:48:20.924 INFO:tasks.workunit.client.0.smithi102.stdout:0/779: write d1/de/d32/d8d/fdd [949139,6546] 0 2024-05-13T22:48:20.925 INFO:tasks.workunit.client.0.smithi102.stdout:9/779: mkdir d1/d6/d45/da1/d10a 0 2024-05-13T22:48:20.927 INFO:tasks.workunit.client.0.smithi102.stdout:1/644: dwrite d2/d4/d26/d40/d5e/fb6 [0,4194304] 0 2024-05-13T22:48:20.930 INFO:tasks.workunit.client.0.smithi102.stdout:5/966: creat de/d25/d59/d7f/d86/da5/d144/f163 x:0 0 0 2024-05-13T22:48:20.932 INFO:tasks.workunit.client.0.smithi102.stdout:9/780: dread d1/d6/db/d4a/f85 [0,4194304] 0 2024-05-13T22:48:20.932 INFO:tasks.workunit.client.0.smithi102.stdout:5/967: symlink de/d25/d59/d7f/d86/da5/d105/l164 0 2024-05-13T22:48:20.938 INFO:tasks.workunit.client.0.smithi102.stdout:9/781: rename d1/d6/d45/d3e/f93 to d1/d6b/f10b 0 2024-05-13T22:48:20.939 INFO:tasks.workunit.client.0.smithi102.stdout:9/782: chown d1/d6/d45/f4f 545 1 2024-05-13T22:48:20.947 INFO:tasks.workunit.client.0.smithi102.stdout:9/783: read d1/d6/f4e [217136,6332] 0 2024-05-13T22:48:20.949 INFO:tasks.workunit.client.0.smithi102.stdout:9/784: mknod d1/d6/db/d77/c10c 0 2024-05-13T22:48:20.954 INFO:tasks.workunit.client.0.smithi102.stdout:5/968: dread de/d25/d2a/d4b/d4d/d50/da3/df4/de4/f142 [0,4194304] 0 2024-05-13T22:48:20.957 INFO:tasks.workunit.client.0.smithi102.stdout:9/785: dwrite d1/d6/db/d4a/d75/d96/fe4 [0,4194304] 0 2024-05-13T22:48:20.958 INFO:tasks.workunit.client.0.smithi102.stdout:8/734: dread d2/d36/d49/d5d/f45 [0,4194304] 0 2024-05-13T22:48:20.961 INFO:tasks.workunit.client.0.smithi102.stdout:0/780: dread d1/de/d32/d24/d2d/d3e/f41 [0,4194304] 0 2024-05-13T22:48:20.968 INFO:tasks.workunit.client.0.smithi102.stdout:8/735: dwrite d2/d7/d27/d64/d80/dc8/d40/f5b [4194304,4194304] 0 2024-05-13T22:48:20.970 INFO:tasks.workunit.client.0.smithi102.stdout:8/736: write d2/d36/d49/f60 [5212459,79428] 0 2024-05-13T22:48:20.988 INFO:tasks.workunit.client.0.smithi102.stdout:5/969: dread de/d25/d43/d7a/fea [0,4194304] 0 2024-05-13T22:48:21.033 INFO:tasks.workunit.client.0.smithi102.stdout:9/786: dread - d1/d6/d45/d2d/dac/db0/fbb zero size 2024-05-13T22:48:21.038 INFO:tasks.workunit.client.0.smithi102.stdout:8/737: read d2/f22 [429193,101465] 0 2024-05-13T22:48:21.040 INFO:tasks.workunit.client.0.smithi102.stdout:5/970: chown de/d25/d2a/d4b/d4d/d50/da3/df4/c89 60 1 2024-05-13T22:48:21.042 INFO:tasks.workunit.client.0.smithi102.stdout:9/787: creat d1/d6/d45/d2d/dac/db0/f10d x:0 0 0 2024-05-13T22:48:21.042 INFO:tasks.workunit.client.0.smithi102.stdout:5/971: readlink de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/l9b 0 2024-05-13T22:48:21.045 INFO:tasks.workunit.client.0.smithi102.stdout:5/972: readlink de/d75/l91 0 2024-05-13T22:48:21.046 INFO:tasks.workunit.client.0.smithi102.stdout:5/973: fsync de/d25/d2a/d4b/d4d/d50/da3/df4/f67 0 2024-05-13T22:48:21.050 INFO:tasks.workunit.client.0.smithi102.stdout:9/788: symlink d1/d6/db/d4a/d75/d96/d102/d6c/d106/l10e 0 2024-05-13T22:48:21.054 INFO:tasks.workunit.client.0.smithi102.stdout:8/738: link d2/d7/d27/d64/d80/dc8/l59 d2/d7/d27/dbb/lea 0 2024-05-13T22:48:21.059 INFO:tasks.workunit.client.0.smithi102.stdout:8/739: creat d2/d7/d27/d64/d80/dc8/d40/d85/feb x:0 0 0 2024-05-13T22:48:21.061 INFO:tasks.workunit.client.0.smithi102.stdout:5/974: link de/d25/d2a/d4b/d132/f149 de/d25/d59/d7f/d86/da5/ddb/d103/f165 0 2024-05-13T22:48:21.068 INFO:tasks.workunit.client.0.smithi102.stdout:8/740: creat d2/d36/d49/d7e/de2/fec x:0 0 0 2024-05-13T22:48:21.073 INFO:tasks.workunit.client.0.smithi102.stdout:8/741: creat d2/d7/d27/d64/d80/dc8/d40/d85/ddc/fed x:0 0 0 2024-05-13T22:48:21.075 INFO:tasks.workunit.client.0.smithi102.stdout:5/975: dwrite de/f21 [4194304,4194304] 0 2024-05-13T22:48:21.077 INFO:tasks.workunit.client.0.smithi102.stdout:5/976: stat de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d34 0 2024-05-13T22:48:21.081 INFO:tasks.workunit.client.0.smithi102.stdout:5/977: symlink de/d25/d108/l166 0 2024-05-13T22:48:21.084 INFO:tasks.workunit.client.0.smithi102.stdout:5/978: mkdir de/d25/d43/d7a/d167 0 2024-05-13T22:48:21.090 INFO:tasks.workunit.client.0.smithi102.stdout:5/979: getdents de/d25/d43/df5 0 2024-05-13T22:48:21.093 INFO:tasks.workunit.client.0.smithi102.stdout:5/980: unlink de/d25/d2a/def/f124 0 2024-05-13T22:48:21.094 INFO:tasks.workunit.client.0.smithi102.stdout:5/981: dread - de/d25/d108/dc5/dca/f12e zero size 2024-05-13T22:48:21.100 INFO:tasks.workunit.client.0.smithi102.stdout:8/742: rename d2/d36/d49/d7e/ca1 to d2/d7/d27/d64/d96/cee 0 2024-05-13T22:48:21.102 INFO:tasks.workunit.client.0.smithi102.stdout:8/743: mknod d2/d7/d27/d64/d80/dc8/d93/dab/cef 0 2024-05-13T22:48:21.107 INFO:tasks.workunit.client.0.smithi102.stdout:8/744: link d2/d7/d27/d64/d80/dc8/d93/fe6 d2/d36/d49/d5d/ff0 0 2024-05-13T22:48:21.141 INFO:tasks.workunit.client.0.smithi102.stdout:9/789: write d1/d6/db/d30/d8a/f99 [1172488,13895] 0 2024-05-13T22:48:21.149 INFO:tasks.workunit.client.0.smithi102.stdout:5/982: mknod de/d25/d59/d7f/d86/da5/ddb/c168 0 2024-05-13T22:48:21.150 INFO:tasks.workunit.client.0.smithi102.stdout:5/983: chown de/d25/d2a/fa0 3311 1 2024-05-13T22:48:21.150 INFO:tasks.workunit.client.0.smithi102.stdout:5/984: stat de/d75/c8c 0 2024-05-13T22:48:21.151 INFO:tasks.workunit.client.0.smithi102.stdout:5/985: dread - de/d25/f15c zero size 2024-05-13T22:48:21.154 INFO:tasks.workunit.client.0.smithi102.stdout:0/781: rmdir d1/de/d32/d24/d2d/d38/d3d/d81 39 2024-05-13T22:48:21.156 INFO:tasks.workunit.client.0.smithi102.stdout:0/782: fsync d1/de/faa 0 2024-05-13T22:48:21.157 INFO:tasks.workunit.client.0.smithi102.stdout:8/745: creat d2/d7/d27/d64/ff1 x:0 0 0 2024-05-13T22:48:21.159 INFO:tasks.workunit.client.0.smithi102.stdout:9/790: dread d1/d6/db/d4a/d64/f9a [0,4194304] 0 2024-05-13T22:48:21.168 INFO:tasks.workunit.client.0.smithi102.stdout:0/783: link d1/de/c22 d1/d4d/cf2 0 2024-05-13T22:48:21.171 INFO:tasks.workunit.client.0.smithi102.stdout:0/784: creat d1/d68/ff3 x:0 0 0 2024-05-13T22:48:21.182 INFO:tasks.workunit.client.0.smithi102.stdout:9/791: dwrite d1/d6/d3b/d5b/f62 [0,4194304] 0 2024-05-13T22:48:21.183 INFO:tasks.workunit.client.0.smithi102.stdout:8/746: dwrite d2/d7/d25/f74 [0,4194304] 0 2024-05-13T22:48:21.184 INFO:tasks.workunit.client.0.smithi102.stdout:0/785: dwrite d1/fb3 [0,4194304] 0 2024-05-13T22:48:21.185 INFO:tasks.workunit.client.0.smithi102.stdout:8/747: chown d2/d36/d49/d7e/de2 123 1 2024-05-13T22:48:21.186 INFO:tasks.workunit.client.0.smithi102.stdout:9/792: read d1/f17 [352166,106096] 0 2024-05-13T22:48:21.189 INFO:tasks.workunit.client.0.smithi102.stdout:0/786: chown d1/de/d32/d24/d2d/d38/d3d/lc9 2367861 1 2024-05-13T22:48:21.191 INFO:tasks.workunit.client.0.smithi102.stdout:9/793: rmdir d1/d6/d45/d2d/dac 39 2024-05-13T22:48:21.195 INFO:tasks.workunit.client.0.smithi102.stdout:9/794: mkdir d1/d6/db/d4a/d75/d96/d102/d6c/d10f 0 2024-05-13T22:48:21.197 INFO:tasks.workunit.client.0.smithi102.stdout:9/795: write d1/d6/db/d4a/ffc [224354,44007] 0 2024-05-13T22:48:21.198 INFO:tasks.workunit.client.0.smithi102.stdout:8/748: rmdir d2/d7/d27/d64/d80/dc8/d40/d85/dd2 0 2024-05-13T22:48:21.199 INFO:tasks.workunit.client.0.smithi102.stdout:8/749: dread - d2/d7/d27/d64/d96/dae/fe1 zero size 2024-05-13T22:48:21.200 INFO:tasks.workunit.client.0.smithi102.stdout:0/787: getdents d1/d4d/d79/d9f 0 2024-05-13T22:48:21.203 INFO:tasks.workunit.client.0.smithi102.stdout:9/796: rmdir d1/d6/d45/d3e/dfe 0 2024-05-13T22:48:21.204 INFO:tasks.workunit.client.0.smithi102.stdout:8/750: mknod d2/d7/d27/d64/d96/dae/d76/cf2 0 2024-05-13T22:48:21.205 INFO:tasks.workunit.client.0.smithi102.stdout:8/751: write d2/d7/d27/d64/d96/dae/d76/f9f [815174,22769] 0 2024-05-13T22:48:21.208 INFO:tasks.workunit.client.0.smithi102.stdout:8/752: fdatasync d2/d7/d27/d64/d96/dae/f7b 0 2024-05-13T22:48:21.209 INFO:tasks.workunit.client.0.smithi102.stdout:8/753: fdatasync d2/d7/d27/d64/ff1 0 2024-05-13T22:48:21.210 INFO:tasks.workunit.client.0.smithi102.stdout:8/754: stat d2/d83/fbf 0 2024-05-13T22:48:21.212 INFO:tasks.workunit.client.0.smithi102.stdout:9/797: link d1/d6/c19 d1/d6/d45/d3e/dbe/c110 0 2024-05-13T22:48:21.213 INFO:tasks.workunit.client.0.smithi102.stdout:8/755: creat d2/d7/d27/d64/d96/ff3 x:0 0 0 2024-05-13T22:48:21.217 INFO:tasks.workunit.client.0.smithi102.stdout:8/756: chown d2/d7/d27/cda 195153462 1 2024-05-13T22:48:21.217 INFO:tasks.workunit.client.0.smithi102.stdout:9/798: dread d1/d6/d45/d3e/d79/fa6 [0,4194304] 0 2024-05-13T22:48:21.218 INFO:tasks.workunit.client.0.smithi102.stdout:9/799: chown d1/d6/db/d30/f36 1200844 1 2024-05-13T22:48:21.219 INFO:tasks.workunit.client.0.smithi102.stdout:9/800: chown d1/d6/d21/fdc 93058412 1 2024-05-13T22:48:21.220 INFO:tasks.workunit.client.0.smithi102.stdout:8/757: unlink d2/d7/d27/cd7 0 2024-05-13T22:48:21.221 INFO:tasks.workunit.client.0.smithi102.stdout:9/801: creat d1/d6/db/d4a/d75/d96/d102/f111 x:0 0 0 2024-05-13T22:48:21.224 INFO:tasks.workunit.client.0.smithi102.stdout:8/758: mkdir d2/d7/d25/df4 0 2024-05-13T22:48:21.225 INFO:tasks.workunit.client.0.smithi102.stdout:0/788: truncate d1/de/d32/d24/d2d/d38/d3d/d81/fb8 179733 0 2024-05-13T22:48:21.232 INFO:tasks.workunit.client.0.smithi102.stdout:8/759: dwrite d2/d7/d27/d64/d96/ff3 [0,4194304] 0 2024-05-13T22:48:21.232 INFO:tasks.workunit.client.0.smithi102.stdout:0/789: dread d1/de/d32/d24/f3c [4194304,4194304] 0 2024-05-13T22:48:21.233 INFO:tasks.workunit.client.0.smithi102.stdout:9/802: write d1/f17 [772367,8895] 0 2024-05-13T22:48:21.239 INFO:tasks.workunit.client.0.smithi102.stdout:8/760: dread d2/d7/d27/d64/d80/fd6 [0,4194304] 0 2024-05-13T22:48:21.241 INFO:tasks.workunit.client.0.smithi102.stdout:9/803: fsync d1/d6/db/d4a/d75/db9/ff0 0 2024-05-13T22:48:21.242 INFO:tasks.workunit.client.0.smithi102.stdout:0/790: mknod d1/de/d32/d24/d2d/d38/cf4 0 2024-05-13T22:48:21.243 INFO:tasks.workunit.client.0.smithi102.stdout:0/791: fdatasync d1/d4d/fef 0 2024-05-13T22:48:21.244 INFO:tasks.workunit.client.0.smithi102.stdout:8/761: readlink d2/d7/d25/d41/l61 0 2024-05-13T22:48:21.244 INFO:tasks.workunit.client.0.smithi102.stdout:8/762: chown d2/d36/d49/d5d/f33 7866761 1 2024-05-13T22:48:21.245 INFO:tasks.workunit.client.0.smithi102.stdout:9/804: creat d1/d6/db/d30/f112 x:0 0 0 2024-05-13T22:48:21.249 INFO:tasks.workunit.client.0.smithi102.stdout:8/763: mknod d2/d7/d27/d64/d96/cf5 0 2024-05-13T22:48:21.251 INFO:tasks.workunit.client.0.smithi102.stdout:8/764: truncate d2/d7/d27/d64/d96/dae/fe1 156128 0 2024-05-13T22:48:21.253 INFO:tasks.workunit.client.0.smithi102.stdout:9/805: dwrite d1/d6/db/d4a/fa9 [0,4194304] 0 2024-05-13T22:48:21.254 INFO:tasks.workunit.client.0.smithi102.stdout:8/765: write d2/f56 [1673049,76216] 0 2024-05-13T22:48:21.256 INFO:tasks.workunit.client.0.smithi102.stdout:0/792: dwrite d1/de/d32/d24/f83 [0,4194304] 0 2024-05-13T22:48:21.258 INFO:tasks.workunit.client.0.smithi102.stdout:8/766: mknod d2/d36/d49/d7e/de2/cf6 0 2024-05-13T22:48:21.266 INFO:tasks.workunit.client.0.smithi102.stdout:0/793: getdents d1/de/d32/d24/d2d/d91 0 2024-05-13T22:48:21.268 INFO:tasks.workunit.client.0.smithi102.stdout:8/767: getdents d2/d7/d27/d64/d96 0 2024-05-13T22:48:21.269 INFO:tasks.workunit.client.0.smithi102.stdout:8/768: write d2/d83/fbf [248470,32328] 0 2024-05-13T22:48:21.271 INFO:tasks.workunit.client.0.smithi102.stdout:0/794: rename d1/de/d32/d24/d2d/d38/d3d/fcf to d1/de/d32/d24/d2d/d38/d3d/ff5 0 2024-05-13T22:48:21.273 INFO:tasks.workunit.client.0.smithi102.stdout:0/795: symlink d1/d77/lf6 0 2024-05-13T22:48:21.274 INFO:tasks.workunit.client.0.smithi102.stdout:0/796: stat d1/de/c26 0 2024-05-13T22:48:21.281 INFO:tasks.workunit.client.0.smithi102.stdout:8/769: write d2/d7/d27/d64/d80/dc8/f35 [2711623,70810] 0 2024-05-13T22:48:21.283 INFO:tasks.workunit.client.0.smithi102.stdout:9/806: dread d1/d6/d3b/d5b/fa2 [0,4194304] 0 2024-05-13T22:48:21.286 INFO:tasks.workunit.client.0.smithi102.stdout:0/797: dwrite d1/de/d32/d24/d2d/d38/fa3 [0,4194304] 0 2024-05-13T22:48:21.287 INFO:tasks.workunit.client.0.smithi102.stdout:9/807: creat d1/d6/d3b/d5b/f113 x:0 0 0 2024-05-13T22:48:21.339 INFO:tasks.workunit.client.0.smithi102.stdout:0/798: chown d1/de/d42/dea 1362 1 2024-05-13T22:48:21.343 INFO:tasks.workunit.client.0.smithi102.stdout:8/770: dwrite d2/d7/d27/d64/d80/f92 [0,4194304] 0 2024-05-13T22:48:21.343 INFO:tasks.workunit.client.0.smithi102.stdout:9/808: mknod d1/d6/d45/d3e/dbe/c114 0 2024-05-13T22:48:21.350 INFO:tasks.workunit.client.0.smithi102.stdout:9/809: mknod d1/d6/d45/c115 0 2024-05-13T22:48:21.486 INFO:tasks.workunit.client.0.smithi102.stdout:8/771: rename d2/d7/d25/d41/la3 to d2/d7/d27/dbb/dc7/lf7 0 2024-05-13T22:48:21.487 INFO:tasks.workunit.client.0.smithi102.stdout:0/799: getdents d1/de/d32/dab 0 2024-05-13T22:48:21.488 INFO:tasks.workunit.client.0.smithi102.stdout:9/810: mkdir d1/d6/db/d4a/d81/d116 0 2024-05-13T22:48:21.489 INFO:tasks.workunit.client.0.smithi102.stdout:0/800: creat d1/de/d32/d8d/ff7 x:0 0 0 2024-05-13T22:48:21.491 INFO:tasks.workunit.client.0.smithi102.stdout:9/811: symlink d1/d6/db/l117 0 2024-05-13T22:48:21.492 INFO:tasks.workunit.client.0.smithi102.stdout:0/801: write d1/de/d32/d24/d2d/d3e/f41 [3938446,26031] 0 2024-05-13T22:48:21.492 INFO:tasks.workunit.client.0.smithi102.stdout:8/772: link d2/d7/d25/d41/d6f/f82 d2/d7/d25/ff8 0 2024-05-13T22:48:21.493 INFO:tasks.workunit.client.0.smithi102.stdout:9/812: unlink d1/d6/d45/d2d/fd4 0 2024-05-13T22:48:21.494 INFO:tasks.workunit.client.0.smithi102.stdout:0/802: truncate d1/de/d32/d24/f30 1428373 0 2024-05-13T22:48:21.495 INFO:tasks.workunit.client.0.smithi102.stdout:0/803: fsync d1/de/d32/d24/f31 0 2024-05-13T22:48:21.496 INFO:tasks.workunit.client.0.smithi102.stdout:8/773: getdents d2/d36/db3 0 2024-05-13T22:48:21.497 INFO:tasks.workunit.client.0.smithi102.stdout:9/813: rmdir d1/d6/db/d4a/d75/d96/d102 39 2024-05-13T22:48:21.499 INFO:tasks.workunit.client.0.smithi102.stdout:0/804: fdatasync d1/de/d32/d24/d2d/d38/fc0 0 2024-05-13T22:48:21.500 INFO:tasks.workunit.client.0.smithi102.stdout:9/814: mkdir d1/d6/db/d30/d47/d118 0 2024-05-13T22:48:21.503 INFO:tasks.workunit.client.0.smithi102.stdout:0/805: creat d1/d77/ff8 x:0 0 0 2024-05-13T22:48:21.507 INFO:tasks.workunit.client.0.smithi102.stdout:9/815: creat d1/d6/db/d4a/d75/d96/d102/d6c/d10f/f119 x:0 0 0 2024-05-13T22:48:21.509 INFO:tasks.workunit.client.0.smithi102.stdout:9/816: unlink d1/d6/db/d4a/d64/f9a 0 2024-05-13T22:48:21.510 INFO:tasks.workunit.client.0.smithi102.stdout:9/817: chown d1/d6/db/d4a/d75/cee 2 1 2024-05-13T22:48:21.683 INFO:tasks.workunit.client.0.smithi102.stdout:8/774: dwrite d2/d7/d27/f31 [0,4194304] 0 2024-05-13T22:48:21.684 INFO:tasks.workunit.client.0.smithi102.stdout:8/775: stat d2/d7/d27/d64/d80/dc8/d93/fe6 0 2024-05-13T22:48:21.686 INFO:tasks.workunit.client.0.smithi102.stdout:8/776: read d2/d7/d27/d64/d80/dc8/d40/f5b [1075984,97531] 0 2024-05-13T22:48:21.687 INFO:tasks.workunit.client.0.smithi102.stdout:8/777: write d2/d7/d27/d64/ff1 [679025,72195] 0 2024-05-13T22:48:21.746 INFO:tasks.workunit.client.0.smithi102.stdout:0/806: write d1/d4d/f5b [2418293,4322] 0 2024-05-13T22:48:21.748 INFO:tasks.workunit.client.0.smithi102.stdout:8/778: symlink d2/d7/d27/d64/d96/dae/d76/lf9 0 2024-05-13T22:48:21.754 INFO:tasks.workunit.client.0.smithi102.stdout:8/779: dwrite d2/d36/d49/d5d/f9e [0,4194304] 0 2024-05-13T22:48:21.755 INFO:tasks.workunit.client.0.smithi102.stdout:8/780: stat d2/d7/d27/d64/d96/dae/c1f 0 2024-05-13T22:48:21.763 INFO:tasks.workunit.client.0.smithi102.stdout:0/807: dread d1/de/d32/dab/fb0 [0,4194304] 0 2024-05-13T22:48:21.765 INFO:tasks.workunit.client.0.smithi102.stdout:0/808: write d1/de/d32/d24/fd1 [145005,2679] 0 2024-05-13T22:48:21.767 INFO:tasks.workunit.client.0.smithi102.stdout:0/809: chown d1/de/d32/d24/l8f 24980 1 2024-05-13T22:48:21.768 INFO:tasks.workunit.client.0.smithi102.stdout:8/781: dwrite d2/d7/d25/ff8 [0,4194304] 0 2024-05-13T22:48:21.769 INFO:tasks.workunit.client.0.smithi102.stdout:0/810: rmdir d1/de/d32/d24/d2d/d38/d3d/d81 39 2024-05-13T22:48:21.774 INFO:tasks.workunit.client.0.smithi102.stdout:8/782: link d2/d7/d27/d64/d80/dc8/d40/fe0 d2/d7/d27/d64/d80/dc8/d40/d85/ddc/ffa 0 2024-05-13T22:48:21.776 INFO:tasks.workunit.client.0.smithi102.stdout:0/811: link d1/de/d32/d24/d2d/d38/d3d/d81/fb8 d1/de/d42/ff9 0 2024-05-13T22:48:21.780 INFO:tasks.workunit.client.0.smithi102.stdout:0/812: link d1/l99 d1/d68/de2/lfa 0 2024-05-13T22:48:21.782 INFO:tasks.workunit.client.0.smithi102.stdout:0/813: unlink d1/de/f19 0 2024-05-13T22:48:21.784 INFO:tasks.workunit.client.0.smithi102.stdout:8/783: write d2/d7/d27/d64/d80/dc8/d40/db2/fb7 [201590,66275] 0 2024-05-13T22:48:21.787 INFO:tasks.workunit.client.0.smithi102.stdout:8/784: symlink d2/d7/d27/dbb/lfb 0 2024-05-13T22:48:21.791 INFO:tasks.workunit.client.0.smithi102.stdout:0/814: write d1/de/d32/d24/d2d/f5f [696471,11924] 0 2024-05-13T22:48:21.792 INFO:tasks.workunit.client.0.smithi102.stdout:8/785: getdents d2/d7/d27/d64/d80 0 2024-05-13T22:48:21.797 INFO:tasks.workunit.client.0.smithi102.stdout:8/786: truncate d2/d7/d27/d64/d80/dc8/d40/f6c 3049934 0 2024-05-13T22:48:21.798 INFO:tasks.workunit.client.0.smithi102.stdout:8/787: mknod d2/d7/d27/d64/d80/dc8/d93/dab/cfc 0 2024-05-13T22:48:21.801 INFO:tasks.workunit.client.0.smithi102.stdout:8/788: rename d2/d7/d27/f3b to d2/d7/d25/d41/d6f/da0/ffd 0 2024-05-13T22:48:21.875 INFO:tasks.workunit.client.0.smithi102.stdout:3/880: sync 2024-05-13T22:48:21.875 INFO:tasks.workunit.client.0.smithi102.stdout:7/711: sync 2024-05-13T22:48:21.875 INFO:tasks.workunit.client.0.smithi102.stdout:2/928: sync 2024-05-13T22:48:21.875 INFO:tasks.workunit.client.0.smithi102.stdout:6/884: sync 2024-05-13T22:48:21.877 INFO:tasks.workunit.client.0.smithi102.stdout:6/885: chown d9/db/d14/d35/d47/f70 175291663 1 2024-05-13T22:48:21.878 INFO:tasks.workunit.client.0.smithi102.stdout:7/712: creat d6/d15/d4c/fd7 x:0 0 0 2024-05-13T22:48:21.879 INFO:tasks.workunit.client.0.smithi102.stdout:6/886: read d9/d45/fda [734075,15626] 0 2024-05-13T22:48:21.879 INFO:tasks.workunit.client.0.smithi102.stdout:3/881: mkdir de/d10/d129 0 2024-05-13T22:48:21.879 INFO:tasks.workunit.client.0.smithi102.stdout:8/789: dwrite d2/d7/d25/d41/d6f/da0/ffd [4194304,4194304] 0 2024-05-13T22:48:21.973 INFO:tasks.workunit.client.0.smithi102.stdout:4/842: sync 2024-05-13T22:48:21.984 INFO:tasks.workunit.client.0.smithi102.stdout:6/887: link d9/d25/d5a/fe7 d9/d45/d56/f11b 0 2024-05-13T22:48:21.984 INFO:tasks.workunit.client.0.smithi102.stdout:7/713: dwrite d6/d13/d2c/d39/f3f [0,4194304] 0 2024-05-13T22:48:21.989 INFO:tasks.workunit.client.0.smithi102.stdout:6/888: fdatasync d9/db/d14/f103 0 2024-05-13T22:48:21.991 INFO:tasks.workunit.client.0.smithi102.stdout:6/889: write d9/f8b [1270732,11936] 0 2024-05-13T22:48:21.991 INFO:tasks.workunit.client.0.smithi102.stdout:3/882: link de/d10/d2a/d33/f3f de/d10/d2a/d36/f12a 0 2024-05-13T22:48:21.992 INFO:tasks.workunit.client.0.smithi102.stdout:3/883: stat de/d10/d2a/d8c/d7f/cd0 0 2024-05-13T22:48:21.993 INFO:tasks.workunit.client.0.smithi102.stdout:8/790: rename d2/d7/d27/d64/d80/dc8/d40/f6c to d2/d36/d49/d7e/ffe 0 2024-05-13T22:48:21.996 INFO:tasks.workunit.client.0.smithi102.stdout:7/714: write d6/d22/f33 [4852667,76114] 0 2024-05-13T22:48:22.000 INFO:tasks.workunit.client.0.smithi102.stdout:8/791: symlink d2/d7/d27/d64/d80/dc8/d40/lff 0 2024-05-13T22:48:22.002 INFO:tasks.workunit.client.0.smithi102.stdout:7/715: rename d6/d22/f23 to d6/dd0/d80/fd8 0 2024-05-13T22:48:22.002 INFO:tasks.workunit.client.0.smithi102.stdout:4/843: link d3/d19/d23/d4e/d68/d85/f9a d3/d4/d6/de/d1f/d41/d6a/f124 0 2024-05-13T22:48:22.004 INFO:tasks.workunit.client.0.smithi102.stdout:6/890: mkdir d9/d25/d11c 0 2024-05-13T22:48:22.007 INFO:tasks.workunit.client.0.smithi102.stdout:8/792: creat d2/d7/d27/d64/d80/f100 x:0 0 0 2024-05-13T22:48:22.008 INFO:tasks.workunit.client.0.smithi102.stdout:7/716: symlink d6/d15/ld9 0 2024-05-13T22:48:22.008 INFO:tasks.workunit.client.0.smithi102.stdout:7/717: stat d6/d13/d58/l87 0 2024-05-13T22:48:22.011 INFO:tasks.workunit.client.0.smithi102.stdout:6/891: rename d9/d45/d56/dd5/cf5 to d9/d25/d5a/c11d 0 2024-05-13T22:48:22.011 INFO:tasks.workunit.client.0.smithi102.stdout:4/844: dread d3/d19/d23/d39/d5d/da5/fb7 [0,4194304] 0 2024-05-13T22:48:22.012 INFO:tasks.workunit.client.0.smithi102.stdout:8/793: symlink d2/d36/d49/d5d/l101 0 2024-05-13T22:48:22.016 INFO:tasks.workunit.client.0.smithi102.stdout:6/892: symlink d9/d25/d60/d6a/l11e 0 2024-05-13T22:48:22.016 INFO:tasks.workunit.client.0.smithi102.stdout:6/893: readlink d9/d45/d56/l8a 0 2024-05-13T22:48:22.016 INFO:tasks.workunit.client.0.smithi102.stdout:3/884: write de/d10/d2a/d36/f12a [2665117,30274] 0 2024-05-13T22:48:22.018 INFO:tasks.workunit.client.0.smithi102.stdout:3/885: chown de/d10/d2a/d8c/da8/d10e/l116 706 1 2024-05-13T22:48:22.021 INFO:tasks.workunit.client.0.smithi102.stdout:6/894: rename d9/db/d14/dcc/d5f/fa4 to d9/d25/d71/f11f 0 2024-05-13T22:48:22.022 INFO:tasks.workunit.client.0.smithi102.stdout:7/718: write d6/d13/d58/f26 [105888,49127] 0 2024-05-13T22:48:22.022 INFO:tasks.workunit.client.0.smithi102.stdout:3/886: creat de/d10/d46/dc8/f12b x:0 0 0 2024-05-13T22:48:22.024 INFO:tasks.workunit.client.0.smithi102.stdout:7/719: write f4 [2113741,51579] 0 2024-05-13T22:48:22.025 INFO:tasks.workunit.client.0.smithi102.stdout:3/887: unlink de/d10/d50/fc6 0 2024-05-13T22:48:22.026 INFO:tasks.workunit.client.0.smithi102.stdout:6/895: mknod d9/d25/d60/d6a/dd6/dff/c120 0 2024-05-13T22:48:22.029 INFO:tasks.workunit.client.0.smithi102.stdout:8/794: dwrite d2/d36/d49/d5d/fbd [0,4194304] 0 2024-05-13T22:48:22.030 INFO:tasks.workunit.client.0.smithi102.stdout:6/896: symlink d9/l121 0 2024-05-13T22:48:22.034 INFO:tasks.workunit.client.0.smithi102.stdout:8/795: symlink d2/d36/d49/l102 0 2024-05-13T22:48:22.037 INFO:tasks.workunit.client.0.smithi102.stdout:6/897: write d9/db/d14/d35/f9d [3261437,117810] 0 2024-05-13T22:48:22.038 INFO:tasks.workunit.client.0.smithi102.stdout:7/720: dwrite d6/f7 [0,4194304] 0 2024-05-13T22:48:22.038 INFO:tasks.workunit.client.0.smithi102.stdout:8/796: write d2/d7/d27/d64/d80/dc8/d93/f98 [3707162,72671] 0 2024-05-13T22:48:22.040 INFO:tasks.workunit.client.0.smithi102.stdout:7/721: write d6/d13/d58/f26 [1233252,20169] 0 2024-05-13T22:48:22.040 INFO:tasks.workunit.client.0.smithi102.stdout:6/898: symlink d9/d45/da0/l122 0 2024-05-13T22:48:22.043 INFO:tasks.workunit.client.0.smithi102.stdout:7/722: mkdir d6/dd0/d80/dda 0 2024-05-13T22:48:22.046 INFO:tasks.workunit.client.0.smithi102.stdout:8/797: creat d2/d7/f103 x:0 0 0 2024-05-13T22:48:22.048 INFO:tasks.workunit.client.0.smithi102.stdout:8/798: chown d2/d7/d25/d41/d6f/da0/ffd 21 1 2024-05-13T22:48:22.050 INFO:tasks.workunit.client.0.smithi102.stdout:6/899: dwrite d9/db/d14/d35/dc9/df1/fcd [0,4194304] 0 2024-05-13T22:48:22.050 INFO:tasks.workunit.client.0.smithi102.stdout:3/888: dread de/d10/d2a/d8c/d7f/fd6 [0,4194304] 0 2024-05-13T22:48:22.055 INFO:tasks.workunit.client.0.smithi102.stdout:6/900: creat d9/d25/d71/f123 x:0 0 0 2024-05-13T22:48:22.057 INFO:tasks.workunit.client.0.smithi102.stdout:7/723: dread d6/d13/d58/f34 [0,4194304] 0 2024-05-13T22:48:22.060 INFO:tasks.workunit.client.0.smithi102.stdout:7/724: truncate d6/d13/d2c/d39/db3/fcf 323751 0 2024-05-13T22:48:22.060 INFO:tasks.workunit.client.0.smithi102.stdout:6/901: unlink d9/d25/d5a/c11d 0 2024-05-13T22:48:22.062 INFO:tasks.workunit.client.0.smithi102.stdout:6/902: chown d9/d25/d71/fb7 52611 1 2024-05-13T22:48:22.077 INFO:tasks.workunit.client.0.smithi102.stdout:3/889: write de/d10/d46/d100/f128 [2027033,120158] 0 2024-05-13T22:48:22.078 INFO:tasks.workunit.client.0.smithi102.stdout:6/903: write d9/d45/d56/d5d/f87 [581039,29050] 0 2024-05-13T22:48:22.079 INFO:tasks.workunit.client.0.smithi102.stdout:3/890: fsync de/d10/d2a/f54 0 2024-05-13T22:48:22.082 INFO:tasks.workunit.client.0.smithi102.stdout:6/904: creat d9/d25/d5a/f124 x:0 0 0 2024-05-13T22:48:22.084 INFO:tasks.workunit.client.0.smithi102.stdout:7/725: dwrite d6/d13/d2c/da1/dc1/f66 [0,4194304] 0 2024-05-13T22:48:22.084 INFO:tasks.workunit.client.0.smithi102.stdout:3/891: symlink de/d10/d129/l12c 0 2024-05-13T22:48:22.087 INFO:tasks.workunit.client.0.smithi102.stdout:6/905: mkdir d9/db/d100/d125 0 2024-05-13T22:48:22.089 INFO:tasks.workunit.client.0.smithi102.stdout:8/799: dread d2/d7/d27/d64/d80/dc8/d40/f5b [0,4194304] 0 2024-05-13T22:48:22.089 INFO:tasks.workunit.client.0.smithi102.stdout:3/892: symlink de/d67/de1/l12d 0 2024-05-13T22:48:22.092 INFO:tasks.workunit.client.0.smithi102.stdout:7/726: write d6/d15/d4c/fd7 [42358,108993] 0 2024-05-13T22:48:22.092 INFO:tasks.workunit.client.0.smithi102.stdout:6/906: fdatasync d9/f10 0 2024-05-13T22:48:22.096 INFO:tasks.workunit.client.0.smithi102.stdout:3/893: symlink de/d10/d2a/d33/de8/l12e 0 2024-05-13T22:48:22.099 INFO:tasks.workunit.client.0.smithi102.stdout:8/800: rename d2/d36/d49/d5d/f51 to d2/d7/d25/d41/f104 0 2024-05-13T22:48:22.100 INFO:tasks.workunit.client.0.smithi102.stdout:6/907: truncate d9/d25/fb1 2031524 0 2024-05-13T22:48:22.102 INFO:tasks.workunit.client.0.smithi102.stdout:8/801: dread - d2/d36/d49/fc4 zero size 2024-05-13T22:48:22.103 INFO:tasks.workunit.client.0.smithi102.stdout:8/802: write d2/d7/d25/d41/d6f/f84 [2922590,29808] 0 2024-05-13T22:48:22.118 INFO:tasks.workunit.client.0.smithi102.stdout:3/894: write de/d10/d50/f66 [321104,15726] 0 2024-05-13T22:48:22.120 INFO:tasks.workunit.client.0.smithi102.stdout:3/895: dread - de/d10/d2a/d36/d88/f114 zero size 2024-05-13T22:48:22.121 INFO:tasks.workunit.client.0.smithi102.stdout:7/727: rename d6/d13/d2c/f60 to d6/d15/d4d/d55/d6f/fdb 0 2024-05-13T22:48:22.122 INFO:tasks.workunit.client.0.smithi102.stdout:6/908: truncate d9/db/d14/dcc/f78 809838 0 2024-05-13T22:48:22.123 INFO:tasks.workunit.client.0.smithi102.stdout:7/728: symlink d6/d13/d2c/da1/dc1/ldc 0 2024-05-13T22:48:22.125 INFO:tasks.workunit.client.0.smithi102.stdout:8/803: dwrite d2/d7/d27/f3d [0,4194304] 0 2024-05-13T22:48:22.127 INFO:tasks.workunit.client.0.smithi102.stdout:8/804: stat d2/d7/c14 0 2024-05-13T22:48:22.127 INFO:tasks.workunit.client.0.smithi102.stdout:3/896: rename de/d10/d2a/d33/f4d to de/d10/d50/d63/d98/dee/f12f 0 2024-05-13T22:48:22.133 INFO:tasks.workunit.client.0.smithi102.stdout:7/729: dwrite d6/d13/d2c/da1/dc1/fbf [0,4194304] 0 2024-05-13T22:48:22.133 INFO:tasks.workunit.client.0.smithi102.stdout:8/805: mknod d2/d7/d27/dad/c105 0 2024-05-13T22:48:22.135 INFO:tasks.workunit.client.0.smithi102.stdout:6/909: write d9/db/f29 [3054430,13163] 0 2024-05-13T22:48:22.142 INFO:tasks.workunit.client.0.smithi102.stdout:8/806: dread d2/f22 [0,4194304] 0 2024-05-13T22:48:22.144 INFO:tasks.workunit.client.0.smithi102.stdout:3/897: write de/d10/d2a/d8c/da8/fe2 [793779,82390] 0 2024-05-13T22:48:22.145 INFO:tasks.workunit.client.0.smithi102.stdout:8/807: mkdir d2/d7/d27/d64/d96/dbc/d106 0 2024-05-13T22:48:22.148 INFO:tasks.workunit.client.0.smithi102.stdout:8/808: dread d2/d7/d27/d64/d96/dae/fe1 [0,4194304] 0 2024-05-13T22:48:22.150 INFO:tasks.workunit.client.0.smithi102.stdout:8/809: mknod d2/d7/d25/d41/d6f/da0/c107 0 2024-05-13T22:48:22.159 INFO:tasks.workunit.client.0.smithi102.stdout:7/730: dread d6/d15/d4d/d55/d6f/d49/d97/fa6 [0,4194304] 0 2024-05-13T22:48:22.163 INFO:tasks.workunit.client.0.smithi102.stdout:3/898: dread de/d10/d2a/d33/f40 [0,4194304] 0 2024-05-13T22:48:22.164 INFO:tasks.workunit.client.0.smithi102.stdout:7/731: write d6/d15/d4d/d55/d6f/f7c [205388,40514] 0 2024-05-13T22:48:22.165 INFO:tasks.workunit.client.0.smithi102.stdout:3/899: creat de/d10/d2a/d8c/da8/d10e/f130 x:0 0 0 2024-05-13T22:48:22.169 INFO:tasks.workunit.client.0.smithi102.stdout:8/810: dread d2/d7/d27/d64/d96/dae/d76/f90 [4194304,4194304] 0 2024-05-13T22:48:22.170 INFO:tasks.workunit.client.0.smithi102.stdout:7/732: write d6/d13/d2c/f3a [1465534,21487] 0 2024-05-13T22:48:22.173 INFO:tasks.workunit.client.0.smithi102.stdout:7/733: rmdir d6/d22 39 2024-05-13T22:48:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:5/986: sync 2024-05-13T22:48:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:9/818: sync 2024-05-13T22:48:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:1/645: sync 2024-05-13T22:48:22.176 INFO:tasks.workunit.client.0.smithi102.stdout:3/900: dwrite de/d10/d2a/f54 [0,4194304] 0 2024-05-13T22:48:22.178 INFO:tasks.workunit.client.0.smithi102.stdout:7/734: rename d6/d15/f7e to d6/d13/d2c/d39/fdd 0 2024-05-13T22:48:22.179 INFO:tasks.workunit.client.0.smithi102.stdout:1/646: write d2/d4/fb5 [135978,9140] 0 2024-05-13T22:48:22.181 INFO:tasks.workunit.client.0.smithi102.stdout:9/819: fdatasync d1/d6/d21/f48 0 2024-05-13T22:48:22.183 INFO:tasks.workunit.client.0.smithi102.stdout:7/735: chown d6/dd0/d80/fd8 8073004 1 2024-05-13T22:48:22.184 INFO:tasks.workunit.client.0.smithi102.stdout:5/987: stat de/d25/d2a/d4b/d4d/d50/d6c/f146 0 2024-05-13T22:48:22.185 INFO:tasks.workunit.client.0.smithi102.stdout:8/811: dwrite d2/d7/d27/d64/d96/dae/d76/f90 [0,4194304] 0 2024-05-13T22:48:22.186 INFO:tasks.workunit.client.0.smithi102.stdout:7/736: chown d6/d15/d4c/fd7 52012937 1 2024-05-13T22:48:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:5/988: fdatasync de/d25/d108/dc5/dca/f12e 0 2024-05-13T22:48:22.187 INFO:tasks.workunit.client.0.smithi102.stdout:5/989: dread - de/d25/f15c zero size 2024-05-13T22:48:22.188 INFO:tasks.workunit.client.0.smithi102.stdout:9/820: mkdir d1/d6/db/d4a/d75/d11a 0 2024-05-13T22:48:22.189 INFO:tasks.workunit.client.0.smithi102.stdout:3/901: link de/d1c/d79/c41 de/d10/d2a/d33/de8/c131 0 2024-05-13T22:48:22.193 INFO:tasks.workunit.client.0.smithi102.stdout:1/647: mkdir d2/d4/d5/d19/dd3 0 2024-05-13T22:48:22.193 INFO:tasks.workunit.client.0.smithi102.stdout:7/737: creat d6/d15/d4d/fde x:0 0 0 2024-05-13T22:48:22.193 INFO:tasks.workunit.client.0.smithi102.stdout:8/812: mkdir d2/d7/d27/d64/d96/dae/d108 0 2024-05-13T22:48:22.195 INFO:tasks.workunit.client.0.smithi102.stdout:3/902: creat de/d10/d34/d5d/f132 x:0 0 0 2024-05-13T22:48:22.197 INFO:tasks.workunit.client.0.smithi102.stdout:7/738: creat d6/d15/d4d/d55/fdf x:0 0 0 2024-05-13T22:48:22.197 INFO:tasks.workunit.client.0.smithi102.stdout:8/813: write d2/d36/d49/d7e/de2/fec [636646,100617] 0 2024-05-13T22:48:22.199 INFO:tasks.workunit.client.0.smithi102.stdout:1/648: fsync d2/d10/d62/d7b/dc4/fcc 0 2024-05-13T22:48:22.199 INFO:tasks.workunit.client.0.smithi102.stdout:9/821: dwrite d1/d6/d21/d71/d91/fda [0,4194304] 0 2024-05-13T22:48:22.201 INFO:tasks.workunit.client.0.smithi102.stdout:5/990: getdents de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d113 0 2024-05-13T22:48:22.203 INFO:tasks.workunit.client.0.smithi102.stdout:8/814: mkdir d2/d7/d27/d109 0 2024-05-13T22:48:22.204 INFO:tasks.workunit.client.0.smithi102.stdout:5/991: readlink de/d25/d59/d7f/d86/da5/le7 0 2024-05-13T22:48:22.204 INFO:tasks.workunit.client.0.smithi102.stdout:9/822: fdatasync d1/d6/db/d4a/d64/dc0/ff3 0 2024-05-13T22:48:22.205 INFO:tasks.workunit.client.0.smithi102.stdout:1/649: dread - d2/d45/d96/fd0 zero size 2024-05-13T22:48:22.208 INFO:tasks.workunit.client.0.smithi102.stdout:9/823: readlink d1/d6/d21/d71/lc9 0 2024-05-13T22:48:22.212 INFO:tasks.workunit.client.0.smithi102.stdout:9/824: write d1/d6/db/d30/f36 [5145713,127034] 0 2024-05-13T22:48:22.214 INFO:tasks.workunit.client.0.smithi102.stdout:5/992: dwrite de/d25/d43/df5/f14e [0,4194304] 0 2024-05-13T22:48:22.216 INFO:tasks.workunit.client.0.smithi102.stdout:9/825: rmdir d1/d6/d3b 39 2024-05-13T22:48:22.220 INFO:tasks.workunit.client.0.smithi102.stdout:9/826: fsync d1/d6b/f10b 0 2024-05-13T22:48:22.223 INFO:tasks.workunit.client.0.smithi102.stdout:1/650: rename d2/d4/f51 to d2/d4/d26/d40/fd4 0 2024-05-13T22:48:22.226 INFO:tasks.workunit.client.0.smithi102.stdout:7/739: dread d6/d15/d4d/d55/d6f/d49/fbb [0,4194304] 0 2024-05-13T22:48:22.226 INFO:tasks.workunit.client.0.smithi102.stdout:5/993: creat de/d25/d43/df5/d158/f169 x:0 0 0 2024-05-13T22:48:22.226 INFO:tasks.workunit.client.0.smithi102.stdout:3/903: dread de/d10/d2a/d36/db2/fd2 [0,4194304] 0 2024-05-13T22:48:22.227 INFO:tasks.workunit.client.0.smithi102.stdout:3/904: write de/d10/d50/d63/da2/fdd [491774,116269] 0 2024-05-13T22:48:22.228 INFO:tasks.workunit.client.0.smithi102.stdout:5/994: dread - de/d25/d59/d7f/d86/da5/ddb/dee/d116/f123 zero size 2024-05-13T22:48:22.228 INFO:tasks.workunit.client.0.smithi102.stdout:9/827: link d1/d6/db/d4a/d75/d96/d102/d6c/lfd d1/d6/d45/dd0/l11b 0 2024-05-13T22:48:22.229 INFO:tasks.workunit.client.0.smithi102.stdout:3/905: write fa [5611732,28159] 0 2024-05-13T22:48:22.229 INFO:tasks.workunit.client.0.smithi102.stdout:7/740: write d6/d22/f33 [4439806,5416] 0 2024-05-13T22:48:22.231 INFO:tasks.workunit.client.0.smithi102.stdout:1/651: mknod d2/d4/d26/d40/cd5 0 2024-05-13T22:48:22.231 INFO:tasks.workunit.client.0.smithi102.stdout:3/906: chown de/d10/d2a/d36/d88/f114 98652 1 2024-05-13T22:48:22.233 INFO:tasks.workunit.client.0.smithi102.stdout:7/741: write d6/f94 [3493594,105480] 0 2024-05-13T22:48:22.234 INFO:tasks.workunit.client.0.smithi102.stdout:9/828: mknod d1/d6/d45/d3e/dbe/c11c 0 2024-05-13T22:48:22.235 INFO:tasks.workunit.client.0.smithi102.stdout:1/652: fsync d2/d4/d26/d40/f72 0 2024-05-13T22:48:22.238 INFO:tasks.workunit.client.0.smithi102.stdout:7/742: rmdir d6/da9 39 2024-05-13T22:48:22.239 INFO:tasks.workunit.client.0.smithi102.stdout:9/829: rmdir d1/d6/d45/dd0 39 2024-05-13T22:48:22.241 INFO:tasks.workunit.client.0.smithi102.stdout:7/743: dread d6/d15/d4d/d55/d6f/d49/fbb [0,4194304] 0 2024-05-13T22:48:22.242 INFO:tasks.workunit.client.0.smithi102.stdout:3/907: mkdir de/d1c/d133 0 2024-05-13T22:48:22.244 INFO:tasks.workunit.client.0.smithi102.stdout:9/830: symlink d1/d6b/l11d 0 2024-05-13T22:48:22.244 INFO:tasks.workunit.client.0.smithi102.stdout:5/995: write de/d75/f85 [1490478,118622] 0 2024-05-13T22:48:22.245 INFO:tasks.workunit.client.0.smithi102.stdout:9/831: chown d1/d6/d21/l23 2 1 2024-05-13T22:48:22.248 INFO:tasks.workunit.client.0.smithi102.stdout:9/832: write d1/d6/d45/f4f [4063869,7783] 0 2024-05-13T22:48:22.250 INFO:tasks.workunit.client.0.smithi102.stdout:1/653: getdents d2/d4/d26 0 2024-05-13T22:48:22.254 INFO:tasks.workunit.client.0.smithi102.stdout:3/908: write de/d10/f97 [2580073,2664] 0 2024-05-13T22:48:22.258 INFO:tasks.workunit.client.0.smithi102.stdout:9/833: dwrite d1/d6/d3b/ff6 [0,4194304] 0 2024-05-13T22:48:22.266 INFO:tasks.workunit.client.0.smithi102.stdout:3/909: dwrite de/d1c/f11e [0,4194304] 0 2024-05-13T22:48:22.266 INFO:tasks.workunit.client.0.smithi102.stdout:1/654: link d2/d45/d79/d92/f61 d2/d45/fd6 0 2024-05-13T22:48:22.266 INFO:tasks.workunit.client.0.smithi102.stdout:7/744: dread d6/d15/f1e [0,4194304] 0 2024-05-13T22:48:22.266 INFO:tasks.workunit.client.0.smithi102.stdout:5/996: dwrite de/d25/d2a/d4b/d4d/d50/da3/df4/de4/d12d/d34/f4a [0,4194304] 0 2024-05-13T22:48:22.267 INFO:tasks.workunit.client.0.smithi102.stdout:9/834: creat d1/d6/db/d4a/d81/d116/f11e x:0 0 0 2024-05-13T22:48:22.269 INFO:tasks.workunit.client.0.smithi102.stdout:5/997: write de/d25/d2c/f8a [739600,12119] 0 2024-05-13T22:48:22.277 INFO:tasks.workunit.client.0.smithi102.stdout:3/910: rename de/d10/d34 to de/d68/df3/d134 0 2024-05-13T22:48:22.279 INFO:tasks.workunit.client.0.smithi102.stdout:1/655: truncate d2/d4/d5/d19/f53 231003 0 2024-05-13T22:48:22.280 INFO:tasks.workunit.client.0.smithi102.stdout:1/656: stat d2/d4/d26/da2/ca9 0 2024-05-13T22:48:22.284 INFO:tasks.workunit.client.0.smithi102.stdout:7/745: getdents d6/d15/d4d/d55/d6f/d49/d95 0 2024-05-13T22:48:22.284 INFO:tasks.workunit.client.0.smithi102.stdout:7/746: chown d6/d13/d2c/da1/dc1/f6c 635031526 1 2024-05-13T22:48:22.284 INFO:tasks.workunit.client.0.smithi102.stdout:5/998: creat de/d25/d59/d7f/d86/da5/ddb/dee/f16a x:0 0 0 2024-05-13T22:48:22.285 INFO:tasks.workunit.client.0.smithi102.stdout:3/911: dwrite fa [4194304,4194304] 0 2024-05-13T22:48:22.288 INFO:tasks.workunit.client.0.smithi102.stdout:7/747: creat d6/d15/d4d/d55/d6f/d65/fe0 x:0 0 0 2024-05-13T22:48:22.290 INFO:tasks.workunit.client.0.smithi102.stdout:3/912: stat de/d10/d50/d63/da2/f125 0 2024-05-13T22:48:22.291 INFO:tasks.workunit.client.0.smithi102.stdout:5/999: truncate de/f61 652054 0 2024-05-13T22:48:22.292 INFO:tasks.workunit.client.0.smithi102.stdout:7/748: mkdir d6/de1 0 2024-05-13T22:48:22.312 INFO:tasks.workunit.client.0.smithi102.stdout:3/913: dwrite de/d10/d2a/d36/db2/fd2 [0,4194304] 0 2024-05-13T22:48:22.313 INFO:tasks.workunit.client.0.smithi102.stdout:7/749: dwrite d6/f2e [0,4194304] 0 2024-05-13T22:48:22.317 INFO:tasks.workunit.client.0.smithi102.stdout:1/657: dread d2/d4/d5/fa [0,4194304] 0 2024-05-13T22:48:22.323 INFO:tasks.workunit.client.0.smithi102.stdout:1/658: rename d2/d10/d62/d86/f8b to d2/d4/d5/d19/dc9/fd7 0 2024-05-13T22:48:22.325 INFO:tasks.workunit.client.0.smithi102.stdout:3/914: dwrite de/d68/faf [0,4194304] 0 2024-05-13T22:48:22.329 INFO:tasks.workunit.client.0.smithi102.stdout:3/915: fsync de/d10/d2a/f9c 0 2024-05-13T22:48:22.331 INFO:tasks.workunit.client.0.smithi102.stdout:3/916: dread - de/d10/d2a/d8c/da8/d10e/f130 zero size 2024-05-13T22:48:22.332 INFO:tasks.workunit.client.0.smithi102.stdout:3/917: fdatasync de/d10/d2a/d36/db2/de9/f107 0 2024-05-13T22:48:22.333 INFO:tasks.workunit.client.0.smithi102.stdout:1/659: rename d2/d4/d5/d19/d28/lad to d2/d45/d79/d92/ld8 0 2024-05-13T22:48:22.337 INFO:tasks.workunit.client.0.smithi102.stdout:3/918: dwrite fa [0,4194304] 0 2024-05-13T22:48:22.391 INFO:tasks.workunit.client.0.smithi102.stdout:3/919: write de/d68/faf [315113,35205] 0 2024-05-13T22:48:22.391 INFO:tasks.workunit.client.0.smithi102.stdout:0/815: sync 2024-05-13T22:48:22.392 INFO:tasks.workunit.client.0.smithi102.stdout:1/660: unlink d2/d4/d26/lcb 0 2024-05-13T22:48:22.396 INFO:tasks.workunit.client.0.smithi102.stdout:3/920: rename de/d68/df3/d134/d5d to de/d10/d2a/d33/daa/d135 0 2024-05-13T22:48:22.398 INFO:tasks.workunit.client.0.smithi102.stdout:1/661: creat d2/d10/d62/d7b/fd9 x:0 0 0 2024-05-13T22:48:22.415 INFO:tasks.workunit.client.0.smithi102.stdout:1/662: mknod d2/d45/d79/d92/cda 0 2024-05-13T22:48:22.415 INFO:tasks.workunit.client.0.smithi102.stdout:3/921: fdatasync de/d67/de1/fef 0 2024-05-13T22:48:22.420 INFO:tasks.workunit.client.0.smithi102.stdout:3/922: readlink de/d10/d50/d63/l118 0 2024-05-13T22:48:22.421 INFO:tasks.workunit.client.0.smithi102.stdout:1/663: dwrite d2/d4/d26/d40/d5e/fb6 [4194304,4194304] 0 2024-05-13T22:48:22.421 INFO:tasks.workunit.client.0.smithi102.stdout:3/923: chown de/d68/fe3 26773647 1 2024-05-13T22:48:22.432 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:22 smithi102 ceph-mon[28599]: pgmap v28: 65 pgs: 65 active+clean; 1.6 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 41 MiB/s rd, 93 MiB/s wr, 334 op/s 2024-05-13T22:48:22.433 INFO:tasks.workunit.client.0.smithi102.stdout:3/924: unlink de/d10/d2a/d36/db2/lb5 0 2024-05-13T22:48:22.433 INFO:tasks.workunit.client.0.smithi102.stdout:3/925: symlink de/d10/d2a/d33/daa/l136 0 2024-05-13T22:48:22.456 INFO:tasks.workunit.client.1.smithi130.stdout:5/24: sync 2024-05-13T22:48:22.457 INFO:tasks.workunit.client.1.smithi130.stdout:5/25: chown f3 54937219 1 2024-05-13T22:48:22.462 INFO:tasks.workunit.client.1.smithi130.stdout:5/26: dwrite f4 [0,4194304] 0 2024-05-13T22:48:22.462 INFO:tasks.workunit.client.1.smithi130.stdout:5/27: stat c1 0 2024-05-13T22:48:22.465 INFO:tasks.workunit.client.1.smithi130.stdout:5/28: mkdir d6 0 2024-05-13T22:48:22.466 INFO:tasks.workunit.client.1.smithi130.stdout:5/29: write f3 [776032,17278] 0 2024-05-13T22:48:22.467 INFO:tasks.workunit.client.1.smithi130.stdout:5/30: write f4 [1704561,83553] 0 2024-05-13T22:48:22.476 INFO:tasks.workunit.client.0.smithi102.stdout:0/816: dwrite d1/de/d32/d63/f7d [0,4194304] 0 2024-05-13T22:48:22.477 INFO:tasks.workunit.client.0.smithi102.stdout:0/817: fdatasync d1/de/d42/d61/ff0 0 2024-05-13T22:48:22.477 INFO:tasks.workunit.client.1.smithi130.stdout:5/31: fdatasync f4 0 2024-05-13T22:48:22.478 INFO:tasks.workunit.client.0.smithi102.stdout:0/818: chown d1/d68/ff3 493083 1 2024-05-13T22:48:22.480 INFO:tasks.workunit.client.0.smithi102.stdout:0/819: mkdir d1/de/d32/dab/dfb 0 2024-05-13T22:48:22.482 INFO:tasks.workunit.client.1.smithi130.stdout:5/32: mkdir d6/d7 0 2024-05-13T22:48:22.483 INFO:tasks.workunit.client.0.smithi102.stdout:0/820: creat d1/de/d32/d63/ffc x:0 0 0 2024-05-13T22:48:22.484 INFO:tasks.workunit.client.1.smithi130.stdout:5/33: fsync f2 0 2024-05-13T22:48:22.486 INFO:tasks.workunit.client.0.smithi102.stdout:1/664: write d2/d10/d62/f88 [1516034,31840] 0 2024-05-13T22:48:22.487 INFO:tasks.workunit.client.0.smithi102.stdout:0/821: creat d1/de/d32/dab/dfb/ffd x:0 0 0 2024-05-13T22:48:22.490 INFO:tasks.workunit.client.1.smithi130.stdout:5/34: rename l5 to d6/d7/l8 0 2024-05-13T22:48:22.491 INFO:tasks.workunit.client.1.smithi130.stdout:5/35: fdatasync f4 0 2024-05-13T22:48:22.491 INFO:tasks.workunit.client.1.smithi130.stdout:5/36: chown d6/d7 61 1 2024-05-13T22:48:22.493 INFO:tasks.workunit.client.0.smithi102.stdout:1/665: dwrite d2/d10/d62/d7b/dc4/fcf [0,4194304] 0 2024-05-13T22:48:22.495 INFO:tasks.workunit.client.0.smithi102.stdout:1/666: stat d2/d45/d79/d92/cda 0 2024-05-13T22:48:22.496 INFO:tasks.workunit.client.0.smithi102.stdout:0/822: mknod d1/d4d/cfe 0 2024-05-13T22:48:22.498 INFO:tasks.workunit.client.0.smithi102.stdout:1/667: mkdir d2/d10/d62/d7b/ddb 0 2024-05-13T22:48:22.500 INFO:tasks.workunit.client.0.smithi102.stdout:1/668: mknod d2/d4/d26/da2/cdc 0 2024-05-13T22:48:22.501 INFO:tasks.workunit.client.0.smithi102.stdout:3/926: dwrite de/d10/d50/d63/f76 [0,4194304] 0 2024-05-13T22:48:22.502 INFO:tasks.workunit.client.0.smithi102.stdout:1/669: write d2/d4/d5/d19/d28/fd1 [598217,103682] 0 2024-05-13T22:48:22.504 INFO:tasks.workunit.client.0.smithi102.stdout:3/927: symlink de/d10/d2a/d8c/l137 0 2024-05-13T22:48:22.506 INFO:tasks.workunit.client.0.smithi102.stdout:3/928: mknod de/d68/df3/c138 0 2024-05-13T22:48:22.510 INFO:tasks.workunit.client.0.smithi102.stdout:3/929: symlink de/d10/d2a/l139 0 2024-05-13T22:48:22.527 INFO:tasks.workunit.client.1.smithi130.stdout:2/22: sync 2024-05-13T22:48:22.527 INFO:tasks.workunit.client.1.smithi130.stdout:6/83: sync 2024-05-13T22:48:22.527 INFO:tasks.workunit.client.1.smithi130.stdout:4/27: sync 2024-05-13T22:48:22.527 INFO:tasks.workunit.client.1.smithi130.stdout:1/36: sync 2024-05-13T22:48:22.528 INFO:tasks.workunit.client.1.smithi130.stdout:7/22: sync 2024-05-13T22:48:22.528 INFO:tasks.workunit.client.1.smithi130.stdout:9/51: sync 2024-05-13T22:48:22.528 INFO:tasks.workunit.client.1.smithi130.stdout:3/57: sync 2024-05-13T22:48:22.528 INFO:tasks.workunit.client.1.smithi130.stdout:8/30: sync 2024-05-13T22:48:22.528 INFO:tasks.workunit.client.1.smithi130.stdout:0/85: sync 2024-05-13T22:48:22.528 INFO:tasks.workunit.client.1.smithi130.stdout:6/84: fdatasync d4/f9 0 2024-05-13T22:48:22.529 INFO:tasks.workunit.client.1.smithi130.stdout:0/86: readlink d0/d7/dd/lf 0 2024-05-13T22:48:22.530 INFO:tasks.workunit.client.1.smithi130.stdout:8/31: dread - d1/d2/f6 zero size 2024-05-13T22:48:22.530 INFO:tasks.workunit.client.1.smithi130.stdout:3/58: write d8/f9 [2259825,37500] 0 2024-05-13T22:48:22.531 INFO:tasks.workunit.client.1.smithi130.stdout:6/85: write d4/f12 [4484379,67759] 0 2024-05-13T22:48:22.531 INFO:tasks.workunit.client.1.smithi130.stdout:2/23: chown d4/f5 3 1 2024-05-13T22:48:22.532 INFO:tasks.workunit.client.1.smithi130.stdout:8/32: write d1/d2/f6 [527413,100676] 0 2024-05-13T22:48:22.533 INFO:tasks.workunit.client.1.smithi130.stdout:5/37: dwrite f2 [0,4194304] 0 2024-05-13T22:48:22.535 INFO:tasks.workunit.client.1.smithi130.stdout:5/38: write f4 [1963508,7668] 0 2024-05-13T22:48:22.535 INFO:tasks.workunit.client.1.smithi130.stdout:0/87: dread d0/f2 [0,4194304] 0 2024-05-13T22:48:22.536 INFO:tasks.workunit.client.0.smithi102.stdout:3/930: getdents de/d10/d2a 0 2024-05-13T22:48:22.539 INFO:tasks.workunit.client.1.smithi130.stdout:7/23: unlink f5 0 2024-05-13T22:48:22.539 INFO:tasks.workunit.client.1.smithi130.stdout:7/24: write - no filename 2024-05-13T22:48:22.539 INFO:tasks.workunit.client.1.smithi130.stdout:7/25: dread - no filename 2024-05-13T22:48:22.539 INFO:tasks.workunit.client.1.smithi130.stdout:5/39: fsync f4 0 2024-05-13T22:48:22.540 INFO:tasks.workunit.client.0.smithi102.stdout:1/670: dwrite d2/d10/d62/d7b/f95 [0,4194304] 0 2024-05-13T22:48:22.541 INFO:tasks.workunit.client.1.smithi130.stdout:9/52: creat d1/fe x:0 0 0 2024-05-13T22:48:22.544 INFO:tasks.workunit.client.1.smithi130.stdout:5/40: dwrite f4 [0,4194304] 0 2024-05-13T22:48:22.546 INFO:tasks.workunit.client.1.smithi130.stdout:4/28: symlink d8/l9 0 2024-05-13T22:48:22.547 INFO:tasks.workunit.client.1.smithi130.stdout:4/29: chown d8 386 1 2024-05-13T22:48:22.549 INFO:tasks.workunit.client.1.smithi130.stdout:6/86: mkdir d4/d15/d1f 0 2024-05-13T22:48:22.550 INFO:tasks.workunit.client.1.smithi130.stdout:6/87: chown d4/f12 60656907 1 2024-05-13T22:48:22.552 INFO:tasks.workunit.client.1.smithi130.stdout:2/24: creat d4/fa x:0 0 0 2024-05-13T22:48:22.552 INFO:tasks.workunit.client.1.smithi130.stdout:2/25: dread - d4/f9 zero size 2024-05-13T22:48:22.555 INFO:tasks.workunit.client.1.smithi130.stdout:8/33: rename d1/d7 to d1/d2/db 0 2024-05-13T22:48:22.555 INFO:tasks.workunit.client.1.smithi130.stdout:5/41: fsync f2 0 2024-05-13T22:48:22.555 INFO:tasks.workunit.client.1.smithi130.stdout:8/34: chown d1/d2/db 3 1 2024-05-13T22:48:22.560 INFO:tasks.workunit.client.1.smithi130.stdout:0/88: rmdir d0/d7 39 2024-05-13T22:48:22.563 INFO:tasks.workunit.client.1.smithi130.stdout:0/89: dread d0/fe [0,4194304] 0 2024-05-13T22:48:22.569 INFO:tasks.workunit.client.1.smithi130.stdout:4/30: creat d8/fa x:0 0 0 2024-05-13T22:48:22.569 INFO:tasks.workunit.client.1.smithi130.stdout:5/42: fdatasync f4 0 2024-05-13T22:48:22.570 INFO:tasks.workunit.client.1.smithi130.stdout:3/59: creat d8/dc/d15/f18 x:0 0 0 2024-05-13T22:48:22.571 INFO:tasks.workunit.client.1.smithi130.stdout:4/31: write f4 [333955,65708] 0 2024-05-13T22:48:22.571 INFO:tasks.workunit.client.1.smithi130.stdout:9/53: dwrite d1/f4 [4194304,4194304] 0 2024-05-13T22:48:22.572 INFO:tasks.workunit.client.1.smithi130.stdout:9/54: readlink d1/l2 0 2024-05-13T22:48:22.572 INFO:tasks.workunit.client.1.smithi130.stdout:3/60: write d8/dc/f12 [648769,4731] 0 2024-05-13T22:48:22.572 INFO:tasks.workunit.client.1.smithi130.stdout:9/55: chown d1/d7/c9 15695 1 2024-05-13T22:48:22.573 INFO:tasks.workunit.client.1.smithi130.stdout:3/61: write d8/f9 [4999562,24530] 0 2024-05-13T22:48:22.573 INFO:tasks.workunit.client.1.smithi130.stdout:9/56: stat l0 0 2024-05-13T22:48:22.574 INFO:tasks.workunit.client.1.smithi130.stdout:9/57: dread - d1/fe zero size 2024-05-13T22:48:22.575 INFO:tasks.workunit.client.1.smithi130.stdout:9/58: read - d1/fe zero size 2024-05-13T22:48:22.578 INFO:tasks.workunit.client.1.smithi130.stdout:8/35: mkdir d1/dc 0 2024-05-13T22:48:22.578 INFO:tasks.workunit.client.1.smithi130.stdout:3/62: dread d8/dc/f12 [0,4194304] 0 2024-05-13T22:48:22.582 INFO:tasks.workunit.client.1.smithi130.stdout:7/26: mknod d7/da/cb 0 2024-05-13T22:48:22.582 INFO:tasks.workunit.client.1.smithi130.stdout:7/27: dwrite - no filename 2024-05-13T22:48:22.582 INFO:tasks.workunit.client.1.smithi130.stdout:7/28: stat d7/da/cb 0 2024-05-13T22:48:22.583 INFO:tasks.workunit.client.1.smithi130.stdout:7/29: stat c3 0 2024-05-13T22:48:22.584 INFO:tasks.workunit.client.1.smithi130.stdout:3/63: dwrite d8/dc/f12 [0,4194304] 0 2024-05-13T22:48:22.586 INFO:tasks.workunit.client.1.smithi130.stdout:5/43: mkdir d6/d7/d9 0 2024-05-13T22:48:22.588 INFO:tasks.workunit.client.1.smithi130.stdout:6/88: creat d4/d15/d1f/f20 x:0 0 0 2024-05-13T22:48:22.591 INFO:tasks.workunit.client.1.smithi130.stdout:4/32: mknod d8/cb 0 2024-05-13T22:48:22.592 INFO:tasks.workunit.client.1.smithi130.stdout:6/89: dread d4/fc [0,4194304] 0 2024-05-13T22:48:22.595 INFO:tasks.workunit.client.0.smithi102.stdout:3/931: rename de/d10/d2a/l139 to de/d10/d46/d100/l13a 0 2024-05-13T22:48:22.596 INFO:tasks.workunit.client.0.smithi102.stdout:3/932: chown de/d10/d2a/d33 0 1 2024-05-13T22:48:22.599 INFO:tasks.workunit.client.0.smithi102.stdout:3/933: symlink de/d10/d2a/d33/de8/l13b 0 2024-05-13T22:48:22.601 INFO:tasks.workunit.client.1.smithi130.stdout:2/26: dwrite f3 [0,4194304] 0 2024-05-13T22:48:22.601 INFO:tasks.workunit.client.1.smithi130.stdout:0/90: unlink d0/d7/dd/f1c 0 2024-05-13T22:48:22.603 INFO:tasks.workunit.client.0.smithi102.stdout:3/934: truncate de/d10/d50/fab 9180479 0 2024-05-13T22:48:22.604 INFO:tasks.workunit.client.1.smithi130.stdout:5/44: creat d6/d7/fa x:0 0 0 2024-05-13T22:48:22.608 INFO:tasks.workunit.client.1.smithi130.stdout:9/59: mknod d1/db/cf 0 2024-05-13T22:48:22.610 INFO:tasks.workunit.client.1.smithi130.stdout:8/36: rename d1/d2/d4/f9 to d1/d2/fd 0 2024-05-13T22:48:22.613 INFO:tasks.workunit.client.1.smithi130.stdout:8/37: write d1/d2/db/f8 [85355,14877] 0 2024-05-13T22:48:22.613 INFO:tasks.workunit.client.1.smithi130.stdout:2/27: mkdir d4/db 0 2024-05-13T22:48:22.614 INFO:tasks.workunit.client.1.smithi130.stdout:7/30: creat d7/d8/d9/fc x:0 0 0 2024-05-13T22:48:22.616 INFO:tasks.workunit.client.1.smithi130.stdout:7/31: chown d7/d8 1233510091 1 2024-05-13T22:48:22.617 INFO:tasks.workunit.client.1.smithi130.stdout:7/32: chown d7/d8/d9 1 1 2024-05-13T22:48:22.617 INFO:tasks.workunit.client.1.smithi130.stdout:9/60: dwrite d1/f8 [0,4194304] 0 2024-05-13T22:48:22.618 INFO:tasks.workunit.client.1.smithi130.stdout:7/33: dread - d7/d8/d9/fc zero size 2024-05-13T22:48:22.619 INFO:tasks.workunit.client.1.smithi130.stdout:5/45: creat d6/d7/d9/fb x:0 0 0 2024-05-13T22:48:22.620 INFO:tasks.workunit.client.1.smithi130.stdout:0/91: symlink d0/d7/l1d 0 2024-05-13T22:48:22.622 INFO:tasks.workunit.client.1.smithi130.stdout:8/38: symlink d1/d2/le 0 2024-05-13T22:48:22.623 INFO:tasks.workunit.client.0.smithi102.stdout:1/671: dwrite d2/d10/d62/d7b/dc4/fcc [0,4194304] 0 2024-05-13T22:48:22.627 INFO:tasks.workunit.client.0.smithi102.stdout:3/935: write de/d10/d46/fdb [1012010,86625] 0 2024-05-13T22:48:22.634 INFO:tasks.workunit.client.1.smithi130.stdout:8/39: dwrite d1/d2/f3 [0,4194304] 0 2024-05-13T22:48:22.634 INFO:tasks.workunit.client.1.smithi130.stdout:5/46: rename d6/d7/d9/fb to d6/d7/d9/fc 0 2024-05-13T22:48:22.634 INFO:tasks.workunit.client.1.smithi130.stdout:2/28: mkdir d4/db/dc 0 2024-05-13T22:48:22.637 INFO:tasks.workunit.client.1.smithi130.stdout:2/29: truncate d4/f9 573875 0 2024-05-13T22:48:22.641 INFO:tasks.workunit.client.1.smithi130.stdout:8/40: dwrite d1/d2/f6 [0,4194304] 0 2024-05-13T22:48:22.643 INFO:tasks.workunit.client.1.smithi130.stdout:8/41: stat d1/d2 0 2024-05-13T22:48:22.643 INFO:tasks.workunit.client.1.smithi130.stdout:8/42: readlink d1/d2/le 0 2024-05-13T22:48:22.646 INFO:tasks.workunit.client.1.smithi130.stdout:8/43: dread d1/d2/f3 [0,4194304] 0 2024-05-13T22:48:22.649 INFO:tasks.workunit.client.1.smithi130.stdout:8/44: dread d1/d2/f6 [0,4194304] 0 2024-05-13T22:48:22.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:22 smithi130 ceph-mon[36591]: pgmap v28: 65 pgs: 65 active+clean; 1.6 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 41 MiB/s rd, 93 MiB/s wr, 334 op/s 2024-05-13T22:48:22.670 INFO:tasks.workunit.client.0.smithi102.stdout:3/936: unlink de/d10/d2a/d8c/da8/cb9 0 2024-05-13T22:48:22.672 INFO:tasks.workunit.client.0.smithi102.stdout:1/672: symlink d2/d45/ldd 0 2024-05-13T22:48:22.672 INFO:tasks.workunit.client.1.smithi130.stdout:7/34: rmdir d7/d8/d9 39 2024-05-13T22:48:22.673 INFO:tasks.workunit.client.1.smithi130.stdout:4/33: getdents d8 0 2024-05-13T22:48:22.680 INFO:tasks.workunit.client.1.smithi130.stdout:4/34: dwrite f6 [0,4194304] 0 2024-05-13T22:48:22.681 INFO:tasks.workunit.client.0.smithi102.stdout:3/937: link de/d1c/d79/le0 de/d10/d50/d63/d98/l13c 0 2024-05-13T22:48:22.683 INFO:tasks.workunit.client.1.smithi130.stdout:2/30: creat d4/fd x:0 0 0 2024-05-13T22:48:22.685 INFO:tasks.workunit.client.0.smithi102.stdout:3/938: symlink de/d10/d2a/d8c/da8/l13d 0 2024-05-13T22:48:22.686 INFO:tasks.workunit.client.1.smithi130.stdout:1/37: sync 2024-05-13T22:48:22.689 INFO:tasks.workunit.client.0.smithi102.stdout:1/673: dwrite d2/f46 [0,4194304] 0 2024-05-13T22:48:22.690 INFO:tasks.workunit.client.0.smithi102.stdout:3/939: mknod de/d10/d50/d63/da2/c13e 0 2024-05-13T22:48:22.691 INFO:tasks.workunit.client.0.smithi102.stdout:3/940: fsync de/d68/f104 0 2024-05-13T22:48:22.691 INFO:tasks.workunit.client.1.smithi130.stdout:9/61: link d1/db/cd d1/db/c10 0 2024-05-13T22:48:22.692 INFO:tasks.workunit.client.1.smithi130.stdout:8/45: write d1/d2/f6 [4305907,845] 0 2024-05-13T22:48:22.692 INFO:tasks.workunit.client.1.smithi130.stdout:9/62: write d1/f4 [7333100,92801] 0 2024-05-13T22:48:22.692 INFO:tasks.workunit.client.1.smithi130.stdout:8/46: chown d1 15354 1 2024-05-13T22:48:22.693 INFO:tasks.workunit.client.1.smithi130.stdout:9/63: chown l0 37425126 1 2024-05-13T22:48:22.693 INFO:tasks.workunit.client.1.smithi130.stdout:8/47: readlink d1/d2/le 0 2024-05-13T22:48:22.694 INFO:tasks.workunit.client.1.smithi130.stdout:4/35: mkdir d8/dc 0 2024-05-13T22:48:22.700 INFO:tasks.workunit.client.1.smithi130.stdout:0/92: rename d0/d7/dd/d17/f1a to d0/f1e 0 2024-05-13T22:48:22.703 INFO:tasks.workunit.client.0.smithi102.stdout:1/674: write d2/d4/fd [1530680,23771] 0 2024-05-13T22:48:22.704 INFO:tasks.workunit.client.1.smithi130.stdout:2/31: mkdir d4/db/dc/de 0 2024-05-13T22:48:22.704 INFO:tasks.workunit.client.1.smithi130.stdout:4/36: dwrite f1 [0,4194304] 0 2024-05-13T22:48:22.705 INFO:tasks.workunit.client.1.smithi130.stdout:1/38: mkdir db/dc 0 2024-05-13T22:48:22.705 INFO:tasks.workunit.client.1.smithi130.stdout:4/37: fdatasync f5 0 2024-05-13T22:48:22.705 INFO:tasks.workunit.client.1.smithi130.stdout:7/35: creat d7/d8/fd x:0 0 0 2024-05-13T22:48:22.706 INFO:tasks.workunit.client.1.smithi130.stdout:7/36: stat l0 0 2024-05-13T22:48:22.706 INFO:tasks.workunit.client.1.smithi130.stdout:4/38: chown l2 8703 1 2024-05-13T22:48:22.706 INFO:tasks.workunit.client.1.smithi130.stdout:1/39: rename db/dc to db/dc/dd 22 2024-05-13T22:48:22.707 INFO:tasks.workunit.client.1.smithi130.stdout:1/40: truncate f8 612177 0 2024-05-13T22:48:22.707 INFO:tasks.workunit.client.1.smithi130.stdout:4/39: write f3 [896788,35108] 0 2024-05-13T22:48:22.708 INFO:tasks.workunit.client.0.smithi102.stdout:1/675: mknod d2/d4/d26/da2/cde 0 2024-05-13T22:48:22.708 INFO:tasks.workunit.client.1.smithi130.stdout:1/41: chown f0 1 1 2024-05-13T22:48:22.708 INFO:tasks.workunit.client.1.smithi130.stdout:1/42: truncate f8 1005569 0 2024-05-13T22:48:22.712 INFO:tasks.workunit.client.1.smithi130.stdout:0/93: mkdir d0/d7/dd/d17/d1f 0 2024-05-13T22:48:22.714 INFO:tasks.workunit.client.0.smithi102.stdout:1/676: fdatasync d2/f4b 0 2024-05-13T22:48:22.715 INFO:tasks.workunit.client.1.smithi130.stdout:2/32: creat d4/ff x:0 0 0 2024-05-13T22:48:22.715 INFO:tasks.workunit.client.0.smithi102.stdout:1/677: chown d2/d4/d5 2 1 2024-05-13T22:48:22.720 INFO:tasks.workunit.client.0.smithi102.stdout:1/678: fsync d2/fc2 0 2024-05-13T22:48:22.723 INFO:tasks.workunit.client.1.smithi130.stdout:9/64: truncate d1/f5 1695494 0 2024-05-13T22:48:22.724 INFO:tasks.workunit.client.0.smithi102.stdout:3/941: dread de/d1c/d79/f2b [0,4194304] 0 2024-05-13T22:48:22.724 INFO:tasks.workunit.client.0.smithi102.stdout:1/679: unlink d2/d45/f47 0 2024-05-13T22:48:22.726 INFO:tasks.workunit.client.0.smithi102.stdout:1/680: chown d2/d4/d5/d19/d28/d6a/f59 3093 1 2024-05-13T22:48:22.726 INFO:tasks.workunit.client.1.smithi130.stdout:1/43: symlink db/le 0 2024-05-13T22:48:22.727 INFO:tasks.workunit.client.0.smithi102.stdout:1/681: chown d2/d4/d5/d19/d28/f24 2694 1 2024-05-13T22:48:22.727 INFO:tasks.workunit.client.1.smithi130.stdout:1/44: chown f0 797 1 2024-05-13T22:48:22.728 INFO:tasks.workunit.client.0.smithi102.stdout:1/682: write d2/d4/d26/d40/f72 [2084502,41492] 0 2024-05-13T22:48:22.728 INFO:tasks.workunit.client.1.smithi130.stdout:2/33: fsync d4/ff 0 2024-05-13T22:48:22.730 INFO:tasks.workunit.client.1.smithi130.stdout:4/40: creat d8/dc/fd x:0 0 0 2024-05-13T22:48:22.731 INFO:tasks.workunit.client.1.smithi130.stdout:9/65: dwrite d1/f4 [4194304,4194304] 0 2024-05-13T22:48:22.732 INFO:tasks.workunit.client.1.smithi130.stdout:9/66: readlink d1/l6 0 2024-05-13T22:48:22.734 INFO:tasks.workunit.client.1.smithi130.stdout:0/94: mknod d0/c20 0 2024-05-13T22:48:22.734 INFO:tasks.workunit.client.1.smithi130.stdout:7/37: creat d7/d8/fe x:0 0 0 2024-05-13T22:48:22.735 INFO:tasks.workunit.client.1.smithi130.stdout:8/48: dwrite d1/d2/fd [0,4194304] 0 2024-05-13T22:48:22.736 INFO:tasks.workunit.client.1.smithi130.stdout:2/34: rename f0 to d4/db/dc/f10 0 2024-05-13T22:48:22.736 INFO:tasks.workunit.client.1.smithi130.stdout:8/49: read d1/d2/f6 [2851675,41393] 0 2024-05-13T22:48:22.737 INFO:tasks.workunit.client.1.smithi130.stdout:8/50: rename d1 to d1/d2/db/df 22 2024-05-13T22:48:22.737 INFO:tasks.workunit.client.1.smithi130.stdout:4/41: creat d8/fe x:0 0 0 2024-05-13T22:48:22.739 INFO:tasks.workunit.client.0.smithi102.stdout:1/683: dread d2/d4/fb8 [0,4194304] 0 2024-05-13T22:48:22.739 INFO:tasks.workunit.client.1.smithi130.stdout:4/42: fdatasync f1 0 2024-05-13T22:48:22.745 INFO:tasks.workunit.client.1.smithi130.stdout:4/43: dread f4 [0,4194304] 0 2024-05-13T22:48:22.745 INFO:tasks.workunit.client.1.smithi130.stdout:4/44: chown f4 185 1 2024-05-13T22:48:22.746 INFO:tasks.workunit.client.1.smithi130.stdout:4/45: dread - d8/dc/fd zero size 2024-05-13T22:48:22.749 INFO:tasks.workunit.client.1.smithi130.stdout:1/45: write f5 [2547856,58581] 0 2024-05-13T22:48:22.749 INFO:tasks.workunit.client.1.smithi130.stdout:4/46: dread f1 [0,4194304] 0 2024-05-13T22:48:22.750 INFO:tasks.workunit.client.1.smithi130.stdout:1/46: write f6 [924004,40228] 0 2024-05-13T22:48:22.750 INFO:tasks.workunit.client.1.smithi130.stdout:4/47: fsync f7 0 2024-05-13T22:48:22.753 INFO:tasks.workunit.client.0.smithi102.stdout:3/942: dread de/d1c/fcc [0,4194304] 0 2024-05-13T22:48:22.754 INFO:tasks.workunit.client.0.smithi102.stdout:3/943: write de/d1c/d79/f81 [1067945,92999] 0 2024-05-13T22:48:22.755 INFO:tasks.workunit.client.0.smithi102.stdout:1/684: rename d2/d4/c99 to d2/d4/d5/d19/cdf 0 2024-05-13T22:48:22.758 INFO:tasks.workunit.client.1.smithi130.stdout:8/51: fsync d1/d2/fd 0 2024-05-13T22:48:22.760 INFO:tasks.workunit.client.1.smithi130.stdout:4/48: mkdir d8/df 0 2024-05-13T22:48:22.761 INFO:tasks.workunit.client.1.smithi130.stdout:4/49: dread - d8/dc/fd zero size 2024-05-13T22:48:22.761 INFO:tasks.workunit.client.0.smithi102.stdout:1/685: symlink d2/d10/d62/d7b/ddb/le0 0 2024-05-13T22:48:22.763 INFO:tasks.workunit.client.0.smithi102.stdout:3/944: rename de/d10/d2a/cce to de/d1c/c13f 0 2024-05-13T22:48:22.765 INFO:tasks.workunit.client.0.smithi102.stdout:3/945: fdatasync de/d10/d2a/d33/fcb 0 2024-05-13T22:48:22.770 INFO:tasks.workunit.client.1.smithi130.stdout:2/35: rmdir d4/db/dc/de 0 2024-05-13T22:48:22.771 INFO:tasks.workunit.client.1.smithi130.stdout:8/52: write d1/d2/f3 [2763199,65642] 0 2024-05-13T22:48:22.772 INFO:tasks.workunit.client.0.smithi102.stdout:3/946: mkdir de/d10/d50/d63/da2/d140 0 2024-05-13T22:48:22.775 INFO:tasks.workunit.client.1.smithi130.stdout:8/53: dread d1/d2/f3 [0,4194304] 0 2024-05-13T22:48:22.775 INFO:tasks.workunit.client.1.smithi130.stdout:9/67: getdents d1/d7 0 2024-05-13T22:48:22.776 INFO:tasks.workunit.client.0.smithi102.stdout:1/686: truncate d2/d10/d62/d7b/f7c 833214 0 2024-05-13T22:48:22.776 INFO:tasks.workunit.client.1.smithi130.stdout:9/68: read d1/f8 [3037995,3520] 0 2024-05-13T22:48:22.777 INFO:tasks.workunit.client.1.smithi130.stdout:7/38: link l0 d7/d8/d9/lf 0 2024-05-13T22:48:22.777 INFO:tasks.workunit.client.1.smithi130.stdout:8/54: read d1/d2/db/f8 [21890,92340] 0 2024-05-13T22:48:22.778 INFO:tasks.workunit.client.1.smithi130.stdout:7/39: chown c3 951 1 2024-05-13T22:48:22.778 INFO:tasks.workunit.client.1.smithi130.stdout:2/36: symlink d4/l11 0 2024-05-13T22:48:22.779 INFO:tasks.workunit.client.1.smithi130.stdout:2/37: rename d4 to d4/db/dc/d12 22 2024-05-13T22:48:22.780 INFO:tasks.workunit.client.0.smithi102.stdout:1/687: creat d2/d45/da4/fe1 x:0 0 0 2024-05-13T22:48:22.782 INFO:tasks.workunit.client.1.smithi130.stdout:4/50: creat d8/df/f10 x:0 0 0 2024-05-13T22:48:22.784 INFO:tasks.workunit.client.0.smithi102.stdout:1/688: chown d2/d4/d5/d19/d28/d6a/c7f 62 1 2024-05-13T22:48:22.786 INFO:tasks.workunit.client.1.smithi130.stdout:1/47: rmdir db/dc 0 2024-05-13T22:48:22.786 INFO:tasks.workunit.client.1.smithi130.stdout:1/48: write f6 [1105674,127104] 0 2024-05-13T22:48:22.788 INFO:tasks.workunit.client.0.smithi102.stdout:1/689: rmdir d2/d4/d5/d19/dbe 39 2024-05-13T22:48:22.789 INFO:tasks.workunit.client.1.smithi130.stdout:3/64: sync 2024-05-13T22:48:22.789 INFO:tasks.workunit.client.0.smithi102.stdout:3/947: dwrite de/d10/d50/fab [8388608,4194304] 0 2024-05-13T22:48:22.791 INFO:tasks.workunit.client.1.smithi130.stdout:9/69: creat d1/f11 x:0 0 0 2024-05-13T22:48:22.791 INFO:tasks.workunit.client.1.smithi130.stdout:9/70: fdatasync d1/f11 0 2024-05-13T22:48:22.792 INFO:tasks.workunit.client.1.smithi130.stdout:7/40: rmdir d7/d8/d9 39 2024-05-13T22:48:22.795 INFO:tasks.workunit.client.1.smithi130.stdout:8/55: dread d1/d2/f6 [0,4194304] 0 2024-05-13T22:48:22.795 INFO:tasks.workunit.client.1.smithi130.stdout:2/38: unlink f2 0 2024-05-13T22:48:22.796 INFO:tasks.workunit.client.1.smithi130.stdout:2/39: dread - d4/f8 zero size 2024-05-13T22:48:22.796 INFO:tasks.workunit.client.0.smithi102.stdout:3/948: unlink de/d10/c124 0 2024-05-13T22:48:22.798 INFO:tasks.workunit.client.1.smithi130.stdout:4/51: mknod d8/df/c11 0 2024-05-13T22:48:22.799 INFO:tasks.workunit.client.1.smithi130.stdout:2/40: dread d4/f9 [0,4194304] 0 2024-05-13T22:48:22.799 INFO:tasks.workunit.client.0.smithi102.stdout:3/949: creat de/d10/d129/f141 x:0 0 0 2024-05-13T22:48:22.800 INFO:tasks.workunit.client.0.smithi102.stdout:1/690: creat d2/d4/d5/fe2 x:0 0 0 2024-05-13T22:48:22.801 INFO:tasks.workunit.client.1.smithi130.stdout:1/49: fsync f0 0 2024-05-13T22:48:22.802 INFO:tasks.workunit.client.1.smithi130.stdout:1/50: dread f0 [0,4194304] 0 2024-05-13T22:48:22.803 INFO:tasks.workunit.client.0.smithi102.stdout:3/950: creat de/d10/d46/f142 x:0 0 0 2024-05-13T22:48:22.804 INFO:tasks.workunit.client.1.smithi130.stdout:7/41: dread - d7/d8/d9/fc zero size 2024-05-13T22:48:22.805 INFO:tasks.workunit.client.0.smithi102.stdout:3/951: fsync de/f37 0 2024-05-13T22:48:22.805 INFO:tasks.workunit.client.0.smithi102.stdout:3/952: chown de/d1c/cfb 19386988 1 2024-05-13T22:48:22.805 INFO:tasks.workunit.client.1.smithi130.stdout:7/42: truncate d7/d8/fe 694241 0 2024-05-13T22:48:22.806 INFO:tasks.workunit.client.0.smithi102.stdout:1/691: getdents d2/d4/d38/db2/dba 0 2024-05-13T22:48:22.808 INFO:tasks.workunit.client.1.smithi130.stdout:8/56: truncate d1/d2/f6 2019354 0 2024-05-13T22:48:22.811 INFO:tasks.workunit.client.0.smithi102.stdout:3/953: dwrite de/d10/d2a/fec [0,4194304] 0 2024-05-13T22:48:22.811 INFO:tasks.workunit.client.1.smithi130.stdout:2/41: unlink d4/l11 0 2024-05-13T22:48:22.812 INFO:tasks.workunit.client.1.smithi130.stdout:2/42: fdatasync d4/f6 0 2024-05-13T22:48:22.816 INFO:tasks.workunit.client.1.smithi130.stdout:1/51: rename c9 to db/cf 0 2024-05-13T22:48:22.817 INFO:tasks.workunit.client.0.smithi102.stdout:3/954: symlink de/d67/d105/d117/l143 0 2024-05-13T22:48:22.818 INFO:tasks.workunit.client.0.smithi102.stdout:3/955: fdatasync de/d10/d46/fac 0 2024-05-13T22:48:22.821 INFO:tasks.workunit.client.1.smithi130.stdout:8/57: creat d1/f10 x:0 0 0 2024-05-13T22:48:22.825 INFO:tasks.workunit.client.0.smithi102.stdout:3/956: creat de/d67/d105/f144 x:0 0 0 2024-05-13T22:48:22.825 INFO:tasks.workunit.client.1.smithi130.stdout:1/52: rmdir db 39 2024-05-13T22:48:22.826 INFO:tasks.workunit.client.0.smithi102.stdout:1/692: dread d2/d4/f75 [0,4194304] 0 2024-05-13T22:48:22.828 INFO:tasks.workunit.client.1.smithi130.stdout:3/65: dwrite d8/dc/f12 [0,4194304] 0 2024-05-13T22:48:22.830 INFO:tasks.workunit.client.1.smithi130.stdout:1/53: stat db 0 2024-05-13T22:48:22.832 INFO:tasks.workunit.client.1.smithi130.stdout:4/52: getdents d8/df 0 2024-05-13T22:48:22.836 INFO:tasks.workunit.client.1.smithi130.stdout:3/66: dwrite f3 [0,4194304] 0 2024-05-13T22:48:22.837 INFO:tasks.workunit.client.0.smithi102.stdout:1/693: rename d2/d4/d5/d19/d28/c2e to d2/d4/d26/d40/d5e/ce3 0 2024-05-13T22:48:22.842 INFO:tasks.workunit.client.1.smithi130.stdout:1/54: symlink db/l10 0 2024-05-13T22:48:22.842 INFO:tasks.workunit.client.1.smithi130.stdout:4/53: creat d8/f12 x:0 0 0 2024-05-13T22:48:22.842 INFO:tasks.workunit.client.1.smithi130.stdout:8/58: write d1/d2/db/f8 [793671,99962] 0 2024-05-13T22:48:22.842 INFO:tasks.workunit.client.0.smithi102.stdout:1/694: symlink d2/d4/d26/d40/d5e/le4 0 2024-05-13T22:48:22.843 INFO:tasks.workunit.client.0.smithi102.stdout:1/695: chown d2/d4/d26/f90 502650 1 2024-05-13T22:48:22.843 INFO:tasks.workunit.client.1.smithi130.stdout:2/43: dwrite d4/f8 [0,4194304] 0 2024-05-13T22:48:22.844 INFO:tasks.workunit.client.1.smithi130.stdout:8/59: chown d1/d2/db/f8 55 1 2024-05-13T22:48:22.845 INFO:tasks.workunit.client.0.smithi102.stdout:3/957: write de/d10/d2a/d8c/da8/fd9 [356188,63198] 0 2024-05-13T22:48:22.846 INFO:tasks.workunit.client.0.smithi102.stdout:3/958: readlink de/d10/d2a/d36/db2/ld8 0 2024-05-13T22:48:22.846 INFO:tasks.workunit.client.1.smithi130.stdout:1/55: dread f5 [0,4194304] 0 2024-05-13T22:48:22.847 INFO:tasks.workunit.client.0.smithi102.stdout:3/959: readlink de/d10/d2a/l95 0 2024-05-13T22:48:22.847 INFO:tasks.workunit.client.0.smithi102.stdout:1/696: creat d2/d45/d79/fe5 x:0 0 0 2024-05-13T22:48:22.848 INFO:tasks.workunit.client.0.smithi102.stdout:1/697: stat d2/d4/d5/d19 0 2024-05-13T22:48:22.850 INFO:tasks.workunit.client.1.smithi130.stdout:4/54: mkdir d8/dc/d13 0 2024-05-13T22:48:22.850 INFO:tasks.workunit.client.1.smithi130.stdout:2/44: symlink d4/db/l13 0 2024-05-13T22:48:22.850 INFO:tasks.workunit.client.0.smithi102.stdout:3/960: truncate de/d67/de1/f106 630040 0 2024-05-13T22:48:22.851 INFO:tasks.workunit.client.0.smithi102.stdout:3/961: chown de/d67 292330 1 2024-05-13T22:48:22.851 INFO:tasks.workunit.client.0.smithi102.stdout:3/962: fdatasync de/d10/d46/f6c 0 2024-05-13T22:48:22.852 INFO:tasks.workunit.client.0.smithi102.stdout:1/698: creat d2/d4/d26/fe6 x:0 0 0 2024-05-13T22:48:22.852 INFO:tasks.workunit.client.1.smithi130.stdout:4/55: dread - d8/df/f10 zero size 2024-05-13T22:48:22.855 INFO:tasks.workunit.client.1.smithi130.stdout:2/45: creat d4/db/dc/f14 x:0 0 0 2024-05-13T22:48:22.855 INFO:tasks.workunit.client.1.smithi130.stdout:5/47: sync 2024-05-13T22:48:22.855 INFO:tasks.workunit.client.1.smithi130.stdout:6/90: sync 2024-05-13T22:48:22.856 INFO:tasks.workunit.client.1.smithi130.stdout:8/60: dwrite d1/d2/db/f8 [0,4194304] 0 2024-05-13T22:48:22.856 INFO:tasks.workunit.client.1.smithi130.stdout:4/56: unlink f1 0 2024-05-13T22:48:22.858 INFO:tasks.workunit.client.0.smithi102.stdout:3/963: symlink de/d10/d2a/d8c/da8/l145 0 2024-05-13T22:48:22.861 INFO:tasks.workunit.client.1.smithi130.stdout:3/67: fdatasync f3 0 2024-05-13T22:48:22.862 INFO:tasks.workunit.client.1.smithi130.stdout:5/48: mkdir d6/dd 0 2024-05-13T22:48:22.862 INFO:tasks.workunit.client.1.smithi130.stdout:5/49: fdatasync f4 0 2024-05-13T22:48:22.863 INFO:tasks.workunit.client.1.smithi130.stdout:5/50: rename d6 to d6/de 22 2024-05-13T22:48:22.865 INFO:tasks.workunit.client.1.smithi130.stdout:2/46: creat d4/db/dc/f15 x:0 0 0 2024-05-13T22:48:22.866 INFO:tasks.workunit.client.1.smithi130.stdout:6/91: mknod d4/d15/d1d/c21 0 2024-05-13T22:48:22.866 INFO:tasks.workunit.client.1.smithi130.stdout:6/92: stat d4/c13 0 2024-05-13T22:48:22.866 INFO:tasks.workunit.client.1.smithi130.stdout:6/93: dread - d4/f19 zero size 2024-05-13T22:48:22.868 INFO:tasks.workunit.client.1.smithi130.stdout:4/57: mkdir d8/d14 0 2024-05-13T22:48:22.868 INFO:tasks.workunit.client.1.smithi130.stdout:4/58: fsync f5 0 2024-05-13T22:48:22.869 INFO:tasks.workunit.client.0.smithi102.stdout:7/750: sync 2024-05-13T22:48:22.869 INFO:tasks.workunit.client.0.smithi102.stdout:4/845: sync 2024-05-13T22:48:22.869 INFO:tasks.workunit.client.0.smithi102.stdout:6/910: sync 2024-05-13T22:48:22.869 INFO:tasks.workunit.client.0.smithi102.stdout:8/815: sync 2024-05-13T22:48:22.869 INFO:tasks.workunit.client.0.smithi102.stdout:0/823: sync 2024-05-13T22:48:22.869 INFO:tasks.workunit.client.0.smithi102.stdout:9/835: sync 2024-05-13T22:48:22.869 INFO:tasks.workunit.client.0.smithi102.stdout:2/929: sync 2024-05-13T22:48:22.871 INFO:tasks.workunit.client.0.smithi102.stdout:3/964: rename de/d10/d2a/d8c/da8/fd9 to de/d10/d50/d63/f146 0 2024-05-13T22:48:22.872 INFO:tasks.workunit.client.1.smithi130.stdout:1/56: link c4 db/c11 0 2024-05-13T22:48:22.872 INFO:tasks.workunit.client.1.smithi130.stdout:1/57: truncate f8 1091682 0 2024-05-13T22:48:22.873 INFO:tasks.workunit.client.1.smithi130.stdout:1/58: chown f5 1 1 2024-05-13T22:48:22.873 INFO:tasks.workunit.client.1.smithi130.stdout:1/59: readlink db/l10 0 2024-05-13T22:48:22.873 INFO:tasks.workunit.client.0.smithi102.stdout:8/816: read d2/d7/d27/d64/d80/f92 [2977077,97041] 0 2024-05-13T22:48:22.875 INFO:tasks.workunit.client.0.smithi102.stdout:7/751: creat d6/dd0/d80/fe2 x:0 0 0 2024-05-13T22:48:22.877 INFO:tasks.workunit.client.0.smithi102.stdout:7/752: chown d6/d13/d2c 18929 1 2024-05-13T22:48:22.877 INFO:tasks.workunit.client.0.smithi102.stdout:6/911: mkdir d9/db/d14/d35/d47/dc1/d126 0 2024-05-13T22:48:22.878 INFO:tasks.workunit.client.0.smithi102.stdout:9/836: dread d1/d6/db/d4a/d75/db9/ff0 [0,4194304] 0 2024-05-13T22:48:22.880 INFO:tasks.workunit.client.0.smithi102.stdout:7/753: chown d6/d15/d4d/d55/d82/lc7 0 1 2024-05-13T22:48:22.880 INFO:tasks.workunit.client.0.smithi102.stdout:1/699: dread d2/d4/d38/f68 [0,4194304] 0 2024-05-13T22:48:22.881 INFO:tasks.workunit.client.0.smithi102.stdout:3/965: dwrite de/d10/d46/fac [0,4194304] 0 2024-05-13T22:48:22.881 INFO:tasks.workunit.client.1.smithi130.stdout:8/61: creat d1/dc/f11 x:0 0 0 2024-05-13T22:48:22.883 INFO:tasks.workunit.client.0.smithi102.stdout:4/846: dread - d3/d19/d23/d4e/d68/d85/fd0 zero size 2024-05-13T22:48:22.884 INFO:tasks.workunit.client.0.smithi102.stdout:9/837: dread d1/d6/db/f7a [0,4194304] 0 2024-05-13T22:48:22.885 INFO:tasks.workunit.client.1.smithi130.stdout:8/62: dread d1/d2/db/f8 [0,4194304] 0 2024-05-13T22:48:22.885 INFO:tasks.workunit.client.1.smithi130.stdout:6/94: rename d4/f19 to d4/d15/d1d/f22 0 2024-05-13T22:48:22.885 INFO:tasks.workunit.client.1.smithi130.stdout:6/95: chown d4/d15/d1f 906076515 1 2024-05-13T22:48:22.885 INFO:tasks.workunit.client.0.smithi102.stdout:9/838: fdatasync d1/f2b 0 2024-05-13T22:48:22.887 INFO:tasks.workunit.client.0.smithi102.stdout:0/824: rename d1/de/d32/d63/cae to d1/d4d/d79/d9f/cff 0 2024-05-13T22:48:22.888 INFO:tasks.workunit.client.0.smithi102.stdout:0/825: fdatasync d1/de/d32/d24/d2d/f64 0 2024-05-13T22:48:22.888 INFO:tasks.workunit.client.1.smithi130.stdout:4/59: symlink d8/l15 0 2024-05-13T22:48:22.889 INFO:tasks.workunit.client.1.smithi130.stdout:1/60: rmdir db 39 2024-05-13T22:48:22.889 INFO:tasks.workunit.client.0.smithi102.stdout:8/817: rmdir d2/d7/d27/d64/d80/dc8/d93/dab 39 2024-05-13T22:48:22.890 INFO:tasks.workunit.client.1.smithi130.stdout:4/60: write f7 [81365,107544] 0 2024-05-13T22:48:22.891 INFO:tasks.workunit.client.1.smithi130.stdout:0/95: sync 2024-05-13T22:48:22.892 INFO:tasks.workunit.client.1.smithi130.stdout:9/71: sync 2024-05-13T22:48:22.892 INFO:tasks.workunit.client.1.smithi130.stdout:4/61: read - d8/fa zero size 2024-05-13T22:48:22.892 INFO:tasks.workunit.client.1.smithi130.stdout:8/63: dwrite d1/d2/fd [0,4194304] 0 2024-05-13T22:48:22.892 INFO:tasks.workunit.client.1.smithi130.stdout:5/51: creat d6/dd/ff x:0 0 0 2024-05-13T22:48:22.893 INFO:tasks.workunit.client.1.smithi130.stdout:5/52: stat f4 0 2024-05-13T22:48:22.894 INFO:tasks.workunit.client.1.smithi130.stdout:2/47: write f3 [4887734,53603] 0 2024-05-13T22:48:22.896 INFO:tasks.workunit.client.0.smithi102.stdout:7/754: mknod d6/d15/d4d/d55/d82/ce3 0 2024-05-13T22:48:22.898 INFO:tasks.workunit.client.1.smithi130.stdout:6/96: creat d4/d16/f23 x:0 0 0 2024-05-13T22:48:22.898 INFO:tasks.workunit.client.0.smithi102.stdout:3/966: creat de/d10/d46/d100/f147 x:0 0 0 2024-05-13T22:48:22.899 INFO:tasks.workunit.client.1.smithi130.stdout:5/53: dread f3 [0,4194304] 0 2024-05-13T22:48:22.900 INFO:tasks.workunit.client.0.smithi102.stdout:4/847: chown d3/l11 10729 1 2024-05-13T22:48:22.902 INFO:tasks.workunit.client.1.smithi130.stdout:6/97: dread d4/f9 [0,4194304] 0 2024-05-13T22:48:22.903 INFO:tasks.workunit.client.1.smithi130.stdout:9/72: fdatasync d1/db/fc 0 2024-05-13T22:48:22.904 INFO:tasks.workunit.client.0.smithi102.stdout:3/967: dread de/d10/d50/d63/da2/fdd [0,4194304] 0 2024-05-13T22:48:22.905 INFO:tasks.workunit.client.1.smithi130.stdout:9/73: truncate d1/fe 422809 0 2024-05-13T22:48:22.906 INFO:tasks.workunit.client.1.smithi130.stdout:8/64: creat d1/dc/f12 x:0 0 0 2024-05-13T22:48:22.907 INFO:tasks.workunit.client.0.smithi102.stdout:9/839: readlink d1/d6/d45/d2d/dac/db0/lc1 0 2024-05-13T22:48:22.913 INFO:tasks.workunit.client.1.smithi130.stdout:2/48: mknod d4/c16 0 2024-05-13T22:48:22.913 INFO:tasks.workunit.client.1.smithi130.stdout:2/49: chown d4/fa 3 1 2024-05-13T22:48:22.914 INFO:tasks.workunit.client.1.smithi130.stdout:2/50: chown d4/c16 45 1 2024-05-13T22:48:22.915 INFO:tasks.workunit.client.1.smithi130.stdout:5/54: symlink d6/d7/d9/l10 0 2024-05-13T22:48:22.917 INFO:tasks.workunit.client.0.smithi102.stdout:6/912: fdatasync d9/d25/fb1 0 2024-05-13T22:48:22.919 INFO:tasks.workunit.client.1.smithi130.stdout:1/61: rename f0 to db/f12 0 2024-05-13T22:48:22.920 INFO:tasks.workunit.client.0.smithi102.stdout:7/755: symlink d6/d13/d2c/da1/dc1/le4 0 2024-05-13T22:48:22.921 INFO:tasks.workunit.client.1.smithi130.stdout:3/68: getdents d8 0 2024-05-13T22:48:22.923 INFO:tasks.workunit.client.0.smithi102.stdout:4/848: mkdir d3/d4/d6/de/d1f/d41/d125 0 2024-05-13T22:48:22.925 INFO:tasks.workunit.client.1.smithi130.stdout:8/65: symlink d1/d2/db/l13 0 2024-05-13T22:48:22.928 INFO:tasks.workunit.client.0.smithi102.stdout:0/826: mkdir d1/de/d42/dea/d100 0 2024-05-13T22:48:22.929 INFO:tasks.workunit.client.0.smithi102.stdout:8/818: write d2/d7/d27/d64/d80/dc8/d40/f5b [3237386,47156] 0 2024-05-13T22:48:22.930 INFO:tasks.workunit.client.1.smithi130.stdout:2/51: creat d4/db/f17 x:0 0 0 2024-05-13T22:48:22.932 INFO:tasks.workunit.client.0.smithi102.stdout:7/756: mkdir d6/d22/de5 0 2024-05-13T22:48:22.933 INFO:tasks.workunit.client.1.smithi130.stdout:6/98: rename d4/ca to d4/d15/c24 0 2024-05-13T22:48:22.935 INFO:tasks.workunit.client.1.smithi130.stdout:3/69: mkdir d8/dc/d19 0 2024-05-13T22:48:22.936 INFO:tasks.workunit.client.1.smithi130.stdout:3/70: stat d8/cb 0 2024-05-13T22:48:22.936 INFO:tasks.workunit.client.0.smithi102.stdout:4/849: creat d3/d4/d6/de/d8d/dec/f126 x:0 0 0 2024-05-13T22:48:22.936 INFO:tasks.workunit.client.1.smithi130.stdout:5/55: dwrite f3 [0,4194304] 0 2024-05-13T22:48:22.937 INFO:tasks.workunit.client.1.smithi130.stdout:9/74: unlink d1/f5 0 2024-05-13T22:48:22.941 INFO:tasks.workunit.client.0.smithi102.stdout:9/840: creat d1/d6/db/d4a/d75/d11a/f11f x:0 0 0 2024-05-13T22:48:22.942 INFO:tasks.workunit.client.0.smithi102.stdout:9/841: dread - d1/d6/db/d4a/d75/d11a/f11f zero size 2024-05-13T22:48:22.943 INFO:tasks.workunit.client.0.smithi102.stdout:9/842: chown d1/d6/db/d30/d47/f49 4 1 2024-05-13T22:48:22.945 INFO:tasks.workunit.client.0.smithi102.stdout:1/700: rename d2/d45/d96/lb3 to d2/d10/d62/le7 0 2024-05-13T22:48:22.945 INFO:tasks.workunit.client.0.smithi102.stdout:9/843: fsync d1/d6/d21/d71/d91/fe0 0 2024-05-13T22:48:22.948 INFO:tasks.workunit.client.1.smithi130.stdout:1/62: rename f5 to db/f13 0 2024-05-13T22:48:22.949 INFO:tasks.workunit.client.1.smithi130.stdout:3/71: mkdir d8/d1a 0 2024-05-13T22:48:22.951 INFO:tasks.workunit.client.1.smithi130.stdout:5/56: rmdir d6/dd 39 2024-05-13T22:48:22.952 INFO:tasks.workunit.client.1.smithi130.stdout:3/72: dread f5 [0,4194304] 0 2024-05-13T22:48:22.953 INFO:tasks.workunit.client.1.smithi130.stdout:9/75: write d1/f8 [335831,90373] 0 2024-05-13T22:48:22.954 INFO:tasks.workunit.client.0.smithi102.stdout:4/850: mknod d3/d4/d6/de/d115/c127 0 2024-05-13T22:48:22.957 INFO:tasks.workunit.client.1.smithi130.stdout:4/62: getdents d8/dc 0 2024-05-13T22:48:22.958 INFO:tasks.workunit.client.1.smithi130.stdout:4/63: chown d8/df/c11 4 1 2024-05-13T22:48:22.958 INFO:tasks.workunit.client.1.smithi130.stdout:4/64: read f6 [1523808,46440] 0 2024-05-13T22:48:22.959 INFO:tasks.workunit.client.1.smithi130.stdout:4/65: write f7 [361119,16080] 0 2024-05-13T22:48:22.961 INFO:tasks.workunit.client.0.smithi102.stdout:3/968: dwrite de/d10/d2a/d36/f58 [0,4194304] 0 2024-05-13T22:48:22.963 INFO:tasks.workunit.client.0.smithi102.stdout:6/913: rename d9/db/d14/d35/d47/fcf to d9/db/d14/dcc/d5f/f127 0 2024-05-13T22:48:22.966 INFO:tasks.workunit.client.1.smithi130.stdout:1/63: creat db/f14 x:0 0 0 2024-05-13T22:48:22.966 INFO:tasks.workunit.client.1.smithi130.stdout:1/64: dread - db/f14 zero size 2024-05-13T22:48:22.967 INFO:tasks.workunit.client.1.smithi130.stdout:1/65: chown db/l10 1509 1 2024-05-13T22:48:22.967 INFO:tasks.workunit.client.0.smithi102.stdout:8/819: truncate d2/d7/d25/fb0 39661 0 2024-05-13T22:48:22.969 INFO:tasks.workunit.client.1.smithi130.stdout:5/57: unlink c1 0 2024-05-13T22:48:22.970 INFO:tasks.workunit.client.0.smithi102.stdout:0/827: dwrite d1/de/d32/d8a/fd6 [0,4194304] 0 2024-05-13T22:48:22.973 INFO:tasks.workunit.client.1.smithi130.stdout:9/76: rmdir d1/d7 39 2024-05-13T22:48:22.974 INFO:tasks.workunit.client.1.smithi130.stdout:8/66: link d1/d2/db/l13 d1/l14 0 2024-05-13T22:48:22.976 INFO:tasks.workunit.client.0.smithi102.stdout:4/851: creat d3/dd8/f128 x:0 0 0 2024-05-13T22:48:22.977 INFO:tasks.workunit.client.1.smithi130.stdout:9/77: dread d1/f8 [0,4194304] 0 2024-05-13T22:48:22.977 INFO:tasks.workunit.client.1.smithi130.stdout:7/43: sync 2024-05-13T22:48:22.978 INFO:tasks.workunit.client.1.smithi130.stdout:7/44: fdatasync d7/d8/fd 0 2024-05-13T22:48:22.978 INFO:tasks.workunit.client.1.smithi130.stdout:9/78: readlink d1/l2 0 2024-05-13T22:48:22.979 INFO:tasks.workunit.client.0.smithi102.stdout:8/820: dread d2/d83/fbf [0,4194304] 0 2024-05-13T22:48:22.979 INFO:tasks.workunit.client.0.smithi102.stdout:9/844: dwrite d1/d6/d45/da1/fb7 [0,4194304] 0 2024-05-13T22:48:22.983 INFO:tasks.workunit.client.1.smithi130.stdout:6/99: rename d4/d15/c24 to d4/d16/c25 0 2024-05-13T22:48:22.984 INFO:tasks.workunit.client.0.smithi102.stdout:6/914: dread d9/db/f50 [0,4194304] 0 2024-05-13T22:48:22.985 INFO:tasks.workunit.client.1.smithi130.stdout:1/66: mknod db/c15 0 2024-05-13T22:48:22.987 INFO:tasks.workunit.client.0.smithi102.stdout:7/757: rmdir d6/d22/dd4 0 2024-05-13T22:48:22.989 INFO:tasks.workunit.client.1.smithi130.stdout:5/58: chown d6/dd 4416 1 2024-05-13T22:48:22.989 INFO:tasks.workunit.client.1.smithi130.stdout:5/59: fdatasync f4 0 2024-05-13T22:48:22.991 INFO:tasks.workunit.client.0.smithi102.stdout:0/828: rename d1/de/d32/d24/d2d/d38/d3d/lc9 to d1/de/d32/d24/d2d/d3e/l101 0 2024-05-13T22:48:22.992 INFO:tasks.workunit.client.0.smithi102.stdout:3/969: write de/d10/d50/d8f/fc0 [1495115,52283] 0 2024-05-13T22:48:22.996 INFO:tasks.workunit.client.1.smithi130.stdout:8/67: read d1/d2/f3 [3035148,110398] 0 2024-05-13T22:48:22.999 INFO:tasks.workunit.client.0.smithi102.stdout:3/970: dwrite de/d10/d2a/d36/d88/f10b [4194304,4194304] 0 2024-05-13T22:48:23.000 INFO:tasks.workunit.client.1.smithi130.stdout:0/96: sync 2024-05-13T22:48:23.002 INFO:tasks.workunit.client.1.smithi130.stdout:0/97: dread d0/f3 [0,4194304] 0 2024-05-13T22:48:23.002 INFO:tasks.workunit.client.1.smithi130.stdout:7/45: mknod d7/d8/d9/c10 0 2024-05-13T22:48:23.002 INFO:tasks.workunit.client.1.smithi130.stdout:7/46: fdatasync d7/d8/fe 0 2024-05-13T22:48:23.003 INFO:tasks.workunit.client.1.smithi130.stdout:7/47: chown d7/d8/d9/c10 243503 1 2024-05-13T22:48:23.004 INFO:tasks.workunit.client.1.smithi130.stdout:7/48: fdatasync d7/d8/fd 0 2024-05-13T22:48:23.004 INFO:tasks.workunit.client.1.smithi130.stdout:7/49: chown d7/d8/fe 50047032 1 2024-05-13T22:48:23.004 INFO:tasks.workunit.client.1.smithi130.stdout:3/73: truncate f5 2363873 0 2024-05-13T22:48:23.005 INFO:tasks.workunit.client.1.smithi130.stdout:3/74: stat d8/dc/d15 0 2024-05-13T22:48:23.005 INFO:tasks.workunit.client.1.smithi130.stdout:3/75: fsync f3 0 2024-05-13T22:48:23.007 INFO:tasks.workunit.client.0.smithi102.stdout:4/852: dread d3/d6b/fdf [0,4194304] 0 2024-05-13T22:48:23.008 INFO:tasks.workunit.client.1.smithi130.stdout:2/52: getdents d4/db 0 2024-05-13T22:48:23.011 INFO:tasks.workunit.client.1.smithi130.stdout:3/76: dwrite d8/dc/d15/f18 [0,4194304] 0 2024-05-13T22:48:23.012 INFO:tasks.workunit.client.1.smithi130.stdout:1/67: creat db/f16 x:0 0 0 2024-05-13T22:48:23.013 INFO:tasks.workunit.client.0.smithi102.stdout:0/829: symlink d1/de/d32/d24/d2d/d38/d3d/d81/l102 0 2024-05-13T22:48:23.015 INFO:tasks.workunit.client.0.smithi102.stdout:0/830: read - d1/de/d32/d24/ff1 zero size 2024-05-13T22:48:23.017 INFO:tasks.workunit.client.0.smithi102.stdout:0/831: fsync d1/de/d42/d61/ff0 0 2024-05-13T22:48:23.017 INFO:tasks.workunit.client.0.smithi102.stdout:8/821: dwrite d2/d7/d27/d64/d80/dc8/d40/d85/ddc/ffa [0,4194304] 0 2024-05-13T22:48:23.017 INFO:tasks.workunit.client.1.smithi130.stdout:8/68: rename d1/d2/d4 to d1/d2/db/d15 0 2024-05-13T22:48:23.020 INFO:tasks.workunit.client.0.smithi102.stdout:8/822: stat d2/d7/d27/d64/d96/dae/d76/lf9 0 2024-05-13T22:48:23.021 INFO:tasks.workunit.client.0.smithi102.stdout:3/971: mkdir de/d10/d2a/d8c/d7f/d148 0 2024-05-13T22:48:23.021 INFO:tasks.workunit.client.1.smithi130.stdout:7/50: chown d7/d8/d9/lf 1453 1 2024-05-13T22:48:23.024 INFO:tasks.workunit.client.1.smithi130.stdout:4/66: link d8/l15 d8/l16 0 2024-05-13T22:48:23.025 INFO:tasks.workunit.client.1.smithi130.stdout:4/67: read - d8/dc/fd zero size 2024-05-13T22:48:23.025 INFO:tasks.workunit.client.0.smithi102.stdout:4/853: creat d3/d19/d23/d7e/f129 x:0 0 0 2024-05-13T22:48:23.026 INFO:tasks.workunit.client.1.smithi130.stdout:4/68: read f6 [1670517,5514] 0 2024-05-13T22:48:23.029 INFO:tasks.workunit.client.1.smithi130.stdout:2/53: mkdir d4/db/dc/d18 0 2024-05-13T22:48:23.029 INFO:tasks.workunit.client.1.smithi130.stdout:2/54: dread - d4/db/dc/f14 zero size 2024-05-13T22:48:23.030 INFO:tasks.workunit.client.0.smithi102.stdout:7/758: mkdir d6/de1/de6 0 2024-05-13T22:48:23.030 INFO:tasks.workunit.client.1.smithi130.stdout:6/100: write d4/ff [1234750,130183] 0 2024-05-13T22:48:23.031 INFO:tasks.workunit.client.1.smithi130.stdout:6/101: chown d4/d16/c1b 761 1 2024-05-13T22:48:23.034 INFO:tasks.workunit.client.0.smithi102.stdout:6/915: dwrite d9/d25/f40 [0,4194304] 0 2024-05-13T22:48:23.036 INFO:tasks.workunit.client.1.smithi130.stdout:9/79: rename d1/l2 to d1/d7/l12 0 2024-05-13T22:48:23.038 INFO:tasks.workunit.client.1.smithi130.stdout:8/69: fdatasync d1/d2/f3 0 2024-05-13T22:48:23.040 INFO:tasks.workunit.client.0.smithi102.stdout:3/972: rename de/d10/d50/d63/da2/fdd to de/d10/d2a/d8c/da8/f149 0 2024-05-13T22:48:23.040 INFO:tasks.workunit.client.0.smithi102.stdout:3/973: chown f2 9581517 1 2024-05-13T22:48:23.043 INFO:tasks.workunit.client.1.smithi130.stdout:7/51: symlink d7/da/l11 0 2024-05-13T22:48:23.043 INFO:tasks.workunit.client.1.smithi130.stdout:7/52: stat d7/d8/fd 0 2024-05-13T22:48:23.046 INFO:tasks.workunit.client.1.smithi130.stdout:2/55: creat d4/f19 x:0 0 0 2024-05-13T22:48:23.047 INFO:tasks.workunit.client.0.smithi102.stdout:3/974: dwrite de/d10/d2a/d36/f58 [0,4194304] 0 2024-05-13T22:48:23.050 INFO:tasks.workunit.client.0.smithi102.stdout:4/854: rename d3/d19/d23/d3a/da4/f10b to d3/d19/d23/d39/ddd/f12a 0 2024-05-13T22:48:23.050 INFO:tasks.workunit.client.1.smithi130.stdout:2/56: dread f3 [0,4194304] 0 2024-05-13T22:48:23.050 INFO:tasks.workunit.client.1.smithi130.stdout:2/57: fdatasync d4/fa 0 2024-05-13T22:48:23.051 INFO:tasks.workunit.client.0.smithi102.stdout:9/845: dwrite d1/d6/fa5 [0,4194304] 0 2024-05-13T22:48:23.051 INFO:tasks.workunit.client.1.smithi130.stdout:3/77: mknod d8/d1a/c1b 0 2024-05-13T22:48:23.051 INFO:tasks.workunit.client.1.smithi130.stdout:2/58: truncate d4/fa 880068 0 2024-05-13T22:48:23.052 INFO:tasks.workunit.client.1.smithi130.stdout:2/59: dread - d4/db/dc/f15 zero size 2024-05-13T22:48:23.053 INFO:tasks.workunit.client.0.smithi102.stdout:8/823: write d2/d7/d27/d64/d80/dc8/d40/fe0 [4985002,119338] 0 2024-05-13T22:48:23.053 INFO:tasks.workunit.client.0.smithi102.stdout:4/855: read d3/d4/f2e [3780531,13801] 0 2024-05-13T22:48:23.055 INFO:tasks.workunit.client.0.smithi102.stdout:7/759: mknod d6/d15/d4d/d55/d6f/ce7 0 2024-05-13T22:48:23.058 INFO:tasks.workunit.client.1.smithi130.stdout:6/102: unlink d4/d16/f23 0 2024-05-13T22:48:23.059 INFO:tasks.workunit.client.1.smithi130.stdout:6/103: write d4/f12 [3002911,43737] 0 2024-05-13T22:48:23.062 INFO:tasks.workunit.client.1.smithi130.stdout:5/60: link f2 d6/f11 0 2024-05-13T22:48:23.063 INFO:tasks.workunit.client.0.smithi102.stdout:3/975: creat de/d10/d2a/d36/db2/de9/f14a x:0 0 0 2024-05-13T22:48:23.065 INFO:tasks.workunit.client.1.smithi130.stdout:9/80: symlink d1/db/l13 0 2024-05-13T22:48:23.066 INFO:tasks.workunit.client.0.smithi102.stdout:0/832: truncate d1/de/d32/d24/d2d/f5f 2731458 0 2024-05-13T22:48:23.068 INFO:tasks.workunit.client.1.smithi130.stdout:6/104: dwrite d4/d15/d1f/f20 [0,4194304] 0 2024-05-13T22:48:23.070 INFO:tasks.workunit.client.1.smithi130.stdout:0/98: creat d0/f21 x:0 0 0 2024-05-13T22:48:23.071 INFO:tasks.workunit.client.0.smithi102.stdout:9/846: rmdir d1/d6/d45/d3e/dbe 39 2024-05-13T22:48:23.071 INFO:tasks.workunit.client.0.smithi102.stdout:3/976: dwrite de/d10/d50/d8f/f112 [0,4194304] 0 2024-05-13T22:48:23.073 INFO:tasks.workunit.client.1.smithi130.stdout:7/53: mknod d7/d8/d9/c12 0 2024-05-13T22:48:23.075 INFO:tasks.workunit.client.0.smithi102.stdout:4/856: creat d3/d19/d23/d3a/f12b x:0 0 0 2024-05-13T22:48:23.081 INFO:tasks.workunit.client.1.smithi130.stdout:3/78: mkdir d8/d1a/d1c 0 2024-05-13T22:48:23.082 INFO:tasks.workunit.client.1.smithi130.stdout:2/60: creat d4/f1a x:0 0 0 2024-05-13T22:48:23.084 INFO:tasks.workunit.client.0.smithi102.stdout:3/977: symlink de/d10/d2a/l14b 0 2024-05-13T22:48:23.086 INFO:tasks.workunit.client.1.smithi130.stdout:5/61: symlink d6/dd/l12 0 2024-05-13T22:48:23.088 INFO:tasks.workunit.client.0.smithi102.stdout:7/760: dread d6/d13/d2c/f5c [0,4194304] 0 2024-05-13T22:48:23.089 INFO:tasks.workunit.client.0.smithi102.stdout:6/916: rename d9/d25/d60/d6a/c85 to d9/db/d14/c128 0 2024-05-13T22:48:23.089 INFO:tasks.workunit.client.1.smithi130.stdout:5/62: dread f4 [0,4194304] 0 2024-05-13T22:48:23.090 INFO:tasks.workunit.client.1.smithi130.stdout:5/63: write f4 [3098274,42062] 0 2024-05-13T22:48:23.091 INFO:tasks.workunit.client.0.smithi102.stdout:0/833: unlink d1/de/c22 0 2024-05-13T22:48:23.091 INFO:tasks.workunit.client.1.smithi130.stdout:5/64: write d6/d7/fa [373211,100632] 0 2024-05-13T22:48:23.092 INFO:tasks.workunit.client.0.smithi102.stdout:9/847: fsync d1/d6/d45/d3e/f56 0 2024-05-13T22:48:23.096 INFO:tasks.workunit.client.1.smithi130.stdout:6/105: creat d4/d15/d1f/f26 x:0 0 0 2024-05-13T22:48:23.099 INFO:tasks.workunit.client.0.smithi102.stdout:3/978: rename de/d10/d2a/d36/d88/f114 to de/d67/df7/f14c 0 2024-05-13T22:48:23.100 INFO:tasks.workunit.client.0.smithi102.stdout:3/979: chown de/d10/c15 58 1 2024-05-13T22:48:23.100 INFO:tasks.workunit.client.0.smithi102.stdout:8/824: dread d2/d7/d25/d41/f48 [0,4194304] 0 2024-05-13T22:48:23.101 INFO:tasks.workunit.client.0.smithi102.stdout:6/917: mkdir d9/d45/da0/d129 0 2024-05-13T22:48:23.108 INFO:tasks.workunit.client.0.smithi102.stdout:9/848: mkdir d1/d6/db/d30/d120 0 2024-05-13T22:48:23.108 INFO:tasks.workunit.client.0.smithi102.stdout:9/849: chown d1/d6/d3b/l58 88751368 1 2024-05-13T22:48:23.109 INFO:tasks.workunit.client.0.smithi102.stdout:4/857: creat d3/d19/d23/d4e/f12c x:0 0 0 2024-05-13T22:48:23.111 INFO:tasks.workunit.client.1.smithi130.stdout:1/68: getdents db 0 2024-05-13T22:48:23.112 INFO:tasks.workunit.client.0.smithi102.stdout:7/761: rename d6/d15/d4d/d55/fdf to d6/d15/d4d/d55/d6f/d49/d97/fe8 0 2024-05-13T22:48:23.113 INFO:tasks.workunit.client.1.smithi130.stdout:2/61: creat d4/f1b x:0 0 0 2024-05-13T22:48:23.113 INFO:tasks.workunit.client.0.smithi102.stdout:7/762: truncate d6/d15/d4d/d55/d6f/f7c 874013 0 2024-05-13T22:48:23.114 INFO:tasks.workunit.client.0.smithi102.stdout:7/763: fdatasync d6/dd0/d80/f8a 0 2024-05-13T22:48:23.114 INFO:tasks.workunit.client.1.smithi130.stdout:9/81: symlink d1/l14 0 2024-05-13T22:48:23.116 INFO:tasks.workunit.client.1.smithi130.stdout:5/65: fdatasync d6/f11 0 2024-05-13T22:48:23.117 INFO:tasks.workunit.client.1.smithi130.stdout:8/70: sync 2024-05-13T22:48:23.121 INFO:tasks.workunit.client.0.smithi102.stdout:9/850: symlink d1/d6/db/d4a/d75/d96/d102/d6c/d10f/l121 0 2024-05-13T22:48:23.123 INFO:tasks.workunit.client.1.smithi130.stdout:6/106: dwrite d4/d15/d1f/f20 [4194304,4194304] 0 2024-05-13T22:48:23.125 INFO:tasks.workunit.client.1.smithi130.stdout:6/107: truncate d4/d15/d1d/f22 788821 0 2024-05-13T22:48:23.125 INFO:tasks.workunit.client.1.smithi130.stdout:4/69: link d8/l16 d8/l17 0 2024-05-13T22:48:23.125 INFO:tasks.workunit.client.1.smithi130.stdout:6/108: chown d4/f9 30648676 1 2024-05-13T22:48:23.126 INFO:tasks.workunit.client.1.smithi130.stdout:4/70: chown d8/l15 0 1 2024-05-13T22:48:23.126 INFO:tasks.workunit.client.1.smithi130.stdout:4/71: chown c0 30760764 1 2024-05-13T22:48:23.127 INFO:tasks.workunit.client.1.smithi130.stdout:3/79: mkdir d8/dc/d19/d1d 0 2024-05-13T22:48:23.127 INFO:tasks.workunit.client.1.smithi130.stdout:3/80: stat d8/dc/d19/d1d 0 2024-05-13T22:48:23.128 INFO:tasks.workunit.client.1.smithi130.stdout:3/81: fsync d8/dc/d15/f18 0 2024-05-13T22:48:23.128 INFO:tasks.workunit.client.1.smithi130.stdout:1/69: creat db/f17 x:0 0 0 2024-05-13T22:48:23.132 INFO:tasks.workunit.client.1.smithi130.stdout:2/62: creat d4/db/f1c x:0 0 0 2024-05-13T22:48:23.133 INFO:tasks.workunit.client.1.smithi130.stdout:9/82: unlink d1/l6 0 2024-05-13T22:48:23.134 INFO:tasks.workunit.client.1.smithi130.stdout:3/82: dwrite d8/dc/f12 [0,4194304] 0 2024-05-13T22:48:23.135 INFO:tasks.workunit.client.0.smithi102.stdout:8/825: write d2/d7/d27/dad/fe5 [594695,26187] 0 2024-05-13T22:48:23.135 INFO:tasks.workunit.client.1.smithi130.stdout:9/83: dread - d1/db/fc zero size 2024-05-13T22:48:23.135 INFO:tasks.workunit.client.1.smithi130.stdout:3/83: truncate d8/f16 862539 0 2024-05-13T22:48:23.135 INFO:tasks.workunit.client.0.smithi102.stdout:0/834: write d1/de/d32/d24/d2d/d3e/f41 [5597099,115300] 0 2024-05-13T22:48:23.137 INFO:tasks.workunit.client.1.smithi130.stdout:5/66: creat d6/dd/f13 x:0 0 0 2024-05-13T22:48:23.138 INFO:tasks.workunit.client.1.smithi130.stdout:8/71: mkdir d1/d2/db/d15/d16 0 2024-05-13T22:48:23.138 INFO:tasks.workunit.client.1.smithi130.stdout:5/67: chown f2 523139 1 2024-05-13T22:48:23.140 INFO:tasks.workunit.client.1.smithi130.stdout:8/72: rename d1/d2 to d1/d2/db/d17 22 2024-05-13T22:48:23.141 INFO:tasks.workunit.client.1.smithi130.stdout:0/99: creat d0/d7/f22 x:0 0 0 2024-05-13T22:48:23.142 INFO:tasks.workunit.client.0.smithi102.stdout:8/826: dwrite d2/d7/d27/dad/fe5 [0,4194304] 0 2024-05-13T22:48:23.145 INFO:tasks.workunit.client.1.smithi130.stdout:7/54: rename d7/d8/fd to d7/d8/f13 0 2024-05-13T22:48:23.145 INFO:tasks.workunit.client.1.smithi130.stdout:7/55: dread - d7/d8/f13 zero size 2024-05-13T22:48:23.147 INFO:tasks.workunit.client.1.smithi130.stdout:6/109: mknod d4/d15/c27 0 2024-05-13T22:48:23.148 INFO:tasks.workunit.client.0.smithi102.stdout:4/858: truncate d3/d19/d23/d3a/da4/f108 2513120 0 2024-05-13T22:48:23.150 INFO:tasks.workunit.client.0.smithi102.stdout:6/918: truncate d9/d25/d5a/fb9 478375 0 2024-05-13T22:48:23.151 INFO:tasks.workunit.client.0.smithi102.stdout:6/919: readlink d9/db/d14/d35/dd1/l106 0 2024-05-13T22:48:23.151 INFO:tasks.workunit.client.0.smithi102.stdout:3/980: creat de/d10/d50/d63/f14d x:0 0 0 2024-05-13T22:48:23.152 INFO:tasks.workunit.client.0.smithi102.stdout:3/981: chown de/d10/d2a/d36/db2/ld8 65 1 2024-05-13T22:48:23.153 INFO:tasks.workunit.client.0.smithi102.stdout:3/982: write de/d10/d2a/f119 [39912,16474] 0 2024-05-13T22:48:23.154 INFO:tasks.workunit.client.0.smithi102.stdout:9/851: write d1/d6/d21/f42 [1561750,9653] 0 2024-05-13T22:48:23.156 INFO:tasks.workunit.client.0.smithi102.stdout:3/983: truncate de/d10/d2a/d8c/d7f/fd6 4981840 0 2024-05-13T22:48:23.161 INFO:tasks.workunit.client.0.smithi102.stdout:8/827: readlink d2/d7/d25/d41/d6f/da0/le7 0 2024-05-13T22:48:23.168 INFO:tasks.workunit.client.1.smithi130.stdout:5/68: rename d6/dd/l12 to d6/d7/d9/l14 0 2024-05-13T22:48:23.169 INFO:tasks.workunit.client.0.smithi102.stdout:7/764: creat d6/d15/d4d/d55/d6f/fe9 x:0 0 0 2024-05-13T22:48:23.172 INFO:tasks.workunit.client.1.smithi130.stdout:7/56: unlink l0 0 2024-05-13T22:48:23.173 INFO:tasks.workunit.client.0.smithi102.stdout:3/984: readlink de/d10/d50/d63/d98/l13c 0 2024-05-13T22:48:23.174 INFO:tasks.workunit.client.0.smithi102.stdout:3/985: chown de/d67/de1/c10d 2649251 1 2024-05-13T22:48:23.175 INFO:tasks.workunit.client.1.smithi130.stdout:6/110: mkdir d4/d15/d1d/d28 0 2024-05-13T22:48:23.177 INFO:tasks.workunit.client.1.smithi130.stdout:4/72: link f6 d8/f18 0 2024-05-13T22:48:23.180 INFO:tasks.workunit.client.0.smithi102.stdout:4/859: dread - d3/d19/d23/d39/da7/da9/dd3/dd4/f117 zero size 2024-05-13T22:48:23.180 INFO:tasks.workunit.client.0.smithi102.stdout:4/860: readlink d3/d22/le9 0 2024-05-13T22:48:23.181 INFO:tasks.workunit.client.1.smithi130.stdout:2/63: creat d4/db/dc/d18/f1d x:0 0 0 2024-05-13T22:48:23.182 INFO:tasks.workunit.client.1.smithi130.stdout:9/84: mkdir d1/d15 0 2024-05-13T22:48:23.183 INFO:tasks.workunit.client.0.smithi102.stdout:0/835: write d1/de/d32/d24/f40 [346127,119873] 0 2024-05-13T22:48:23.186 INFO:tasks.workunit.client.0.smithi102.stdout:7/765: symlink d6/d15/d4d/d55/d6f/d49/d97/lea 0 2024-05-13T22:48:23.187 INFO:tasks.workunit.client.0.smithi102.stdout:7/766: read - d6/d15/d4d/d55/d6f/d49/d97/fe8 zero size 2024-05-13T22:48:23.191 INFO:tasks.workunit.client.1.smithi130.stdout:7/57: write d7/d8/f13 [1047474,12764] 0 2024-05-13T22:48:23.193 INFO:tasks.workunit.client.0.smithi102.stdout:9/852: write d1/d6/f11 [2749837,124824] 0 2024-05-13T22:48:23.194 INFO:tasks.workunit.client.0.smithi102.stdout:9/853: chown d1/d6/db/d4a/d75/d96/dc6/feb 1879 1 2024-05-13T22:48:23.195 INFO:tasks.workunit.client.1.smithi130.stdout:1/70: link db/cf db/c18 0 2024-05-13T22:48:23.195 INFO:tasks.workunit.client.0.smithi102.stdout:9/854: readlink d1/d6/l92 0 2024-05-13T22:48:23.195 INFO:tasks.workunit.client.0.smithi102.stdout:8/828: creat d2/de8/f10a x:0 0 0 2024-05-13T22:48:23.196 INFO:tasks.workunit.client.0.smithi102.stdout:9/855: chown d1/d6/db/d77 2632 1 2024-05-13T22:48:23.196 INFO:tasks.workunit.client.1.smithi130.stdout:1/71: fdatasync db/f14 0 2024-05-13T22:48:23.196 INFO:tasks.workunit.client.1.smithi130.stdout:8/73: dwrite d1/d2/f6 [0,4194304] 0 2024-05-13T22:48:23.196 INFO:tasks.workunit.client.1.smithi130.stdout:1/72: chown f8 41559 1 2024-05-13T22:48:23.197 INFO:tasks.workunit.client.0.smithi102.stdout:4/861: stat d3/c100 0 2024-05-13T22:48:23.197 INFO:tasks.workunit.client.1.smithi130.stdout:1/73: fdatasync f6 0 2024-05-13T22:48:23.197 INFO:tasks.workunit.client.1.smithi130.stdout:2/64: creat d4/f1e x:0 0 0 2024-05-13T22:48:23.197 INFO:tasks.workunit.client.1.smithi130.stdout:1/74: dread - db/f17 zero size 2024-05-13T22:48:23.198 INFO:tasks.workunit.client.1.smithi130.stdout:9/85: readlink d1/d7/l12 0 2024-05-13T22:48:23.198 INFO:tasks.workunit.client.0.smithi102.stdout:6/920: rmdir d9/d25/dee 0 2024-05-13T22:48:23.200 INFO:tasks.workunit.client.0.smithi102.stdout:0/836: creat d1/d68/f103 x:0 0 0 2024-05-13T22:48:23.204 INFO:tasks.workunit.client.1.smithi130.stdout:2/65: dwrite d4/f1e [0,4194304] 0 2024-05-13T22:48:23.205 INFO:tasks.workunit.client.1.smithi130.stdout:4/73: creat d8/dc/d13/f19 x:0 0 0 2024-05-13T22:48:23.206 INFO:tasks.workunit.client.1.smithi130.stdout:4/74: dread - d8/f12 zero size 2024-05-13T22:48:23.206 INFO:tasks.workunit.client.0.smithi102.stdout:4/862: mkdir d3/d4/d5f/d12d 0 2024-05-13T22:48:23.213 INFO:tasks.workunit.client.1.smithi130.stdout:4/75: dwrite d8/dc/fd [0,4194304] 0 2024-05-13T22:48:23.215 INFO:tasks.workunit.client.0.smithi102.stdout:4/863: dwrite d3/d19/d23/d7e/f129 [0,4194304] 0 2024-05-13T22:48:23.215 INFO:tasks.workunit.client.0.smithi102.stdout:9/856: write d1/d6/d45/d3e/f63 [778505,33686] 0 2024-05-13T22:48:23.217 INFO:tasks.workunit.client.1.smithi130.stdout:8/74: mkdir d1/d2/db/d15/d18 0 2024-05-13T22:48:23.218 INFO:tasks.workunit.client.1.smithi130.stdout:8/75: dread - d1/dc/f12 zero size 2024-05-13T22:48:23.219 INFO:tasks.workunit.client.0.smithi102.stdout:3/986: rename de/d10/d50/fab to de/d10/d50/f14e 0 2024-05-13T22:48:23.224 INFO:tasks.workunit.client.1.smithi130.stdout:0/100: rename d0/f3 to d0/d7/f23 0 2024-05-13T22:48:23.224 INFO:tasks.workunit.client.1.smithi130.stdout:9/86: dread d1/f8 [0,4194304] 0 2024-05-13T22:48:23.225 INFO:tasks.workunit.client.1.smithi130.stdout:9/87: read - d1/d7/fa zero size 2024-05-13T22:48:23.226 INFO:tasks.workunit.client.0.smithi102.stdout:6/921: getdents d9/d25/d11c 0 2024-05-13T22:48:23.227 INFO:tasks.workunit.client.0.smithi102.stdout:3/987: dwrite de/d10/d2a/d8c/da8/d10e/f130 [0,4194304] 0 2024-05-13T22:48:23.227 INFO:tasks.workunit.client.1.smithi130.stdout:9/88: truncate d1/f11 573652 0 2024-05-13T22:48:23.228 INFO:tasks.workunit.client.0.smithi102.stdout:6/922: write d9/db/d14/d35/f9d [3751580,40721] 0 2024-05-13T22:48:23.233 INFO:tasks.workunit.client.1.smithi130.stdout:0/101: dwrite d0/d7/dd/d17/f1b [0,4194304] 0 2024-05-13T22:48:23.238 INFO:tasks.workunit.client.1.smithi130.stdout:5/69: getdents d6/dd 0 2024-05-13T22:48:23.239 INFO:tasks.workunit.client.1.smithi130.stdout:5/70: dread - d6/dd/f13 zero size 2024-05-13T22:48:23.240 INFO:tasks.workunit.client.1.smithi130.stdout:6/111: getdents d4/d15/d1f 0 2024-05-13T22:48:23.245 INFO:tasks.workunit.client.1.smithi130.stdout:1/75: dwrite db/f13 [0,4194304] 0 2024-05-13T22:48:23.276 INFO:tasks.workunit.client.1.smithi130.stdout:3/84: sync 2024-05-13T22:48:23.281 INFO:tasks.workunit.client.0.smithi102.stdout:9/857: mkdir d1/d6/db/d30/d122 0 2024-05-13T22:48:23.281 INFO:tasks.workunit.client.0.smithi102.stdout:4/864: creat d3/d19/d23/d7e/f12e x:0 0 0 2024-05-13T22:48:23.281 INFO:tasks.workunit.client.0.smithi102.stdout:7/767: truncate d6/dd0/f75 604007 0 2024-05-13T22:48:23.281 INFO:tasks.workunit.client.1.smithi130.stdout:4/76: fdatasync d8/dc/fd 0 2024-05-13T22:48:23.287 INFO:tasks.workunit.client.0.smithi102.stdout:9/858: symlink d1/d6/db/d77/l123 0 2024-05-13T22:48:23.289 INFO:tasks.workunit.client.1.smithi130.stdout:8/76: creat d1/d2/db/d15/d16/f19 x:0 0 0 2024-05-13T22:48:23.289 INFO:tasks.workunit.client.0.smithi102.stdout:3/988: dwrite de/d10/d2a/d36/db2/fbb [0,4194304] 0 2024-05-13T22:48:23.290 INFO:tasks.workunit.client.0.smithi102.stdout:3/989: write de/d68/faf [2978157,85556] 0 2024-05-13T22:48:23.291 INFO:tasks.workunit.client.1.smithi130.stdout:9/89: creat d1/d15/f16 x:0 0 0 2024-05-13T22:48:23.296 INFO:tasks.workunit.client.1.smithi130.stdout:5/71: dread d6/f11 [0,4194304] 0 2024-05-13T22:48:23.296 INFO:tasks.workunit.client.0.smithi102.stdout:0/837: truncate d1/de/d32/d8a/fac 3298694 0 2024-05-13T22:48:23.298 INFO:tasks.workunit.client.1.smithi130.stdout:6/112: rename d4/f11 to d4/f29 0 2024-05-13T22:48:23.299 INFO:tasks.workunit.client.0.smithi102.stdout:7/768: dwrite d6/d15/f1e [4194304,4194304] 0 2024-05-13T22:48:23.299 INFO:tasks.workunit.client.0.smithi102.stdout:6/923: creat d9/f12a x:0 0 0 2024-05-13T22:48:23.300 INFO:tasks.workunit.client.1.smithi130.stdout:1/76: chown db/c11 2 1 2024-05-13T22:48:23.303 INFO:tasks.workunit.client.1.smithi130.stdout:7/58: sync 2024-05-13T22:48:23.303 INFO:tasks.workunit.client.1.smithi130.stdout:7/59: stat d7/d8 0 2024-05-13T22:48:23.304 INFO:tasks.workunit.client.0.smithi102.stdout:9/859: rename d1/d6/d21/d71/d91/fda to d1/d6/d3b/d5b/f124 0 2024-05-13T22:48:23.305 INFO:tasks.workunit.client.1.smithi130.stdout:4/77: creat d8/f1a x:0 0 0 2024-05-13T22:48:23.306 INFO:tasks.workunit.client.0.smithi102.stdout:9/860: readlink d1/d6b/l11d 0 2024-05-13T22:48:23.308 INFO:tasks.workunit.client.0.smithi102.stdout:7/769: dwrite d6/dd0/d80/fe2 [0,4194304] 0 2024-05-13T22:48:23.309 INFO:tasks.workunit.client.1.smithi130.stdout:1/77: dwrite f6 [0,4194304] 0 2024-05-13T22:48:23.309 INFO:tasks.workunit.client.1.smithi130.stdout:0/102: mkdir d0/d7/d24 0 2024-05-13T22:48:23.309 INFO:tasks.workunit.client.1.smithi130.stdout:7/60: read d7/d8/f13 [965981,65759] 0 2024-05-13T22:48:23.310 INFO:tasks.workunit.client.0.smithi102.stdout:7/770: write d6/dd0/f50 [1091209,69783] 0 2024-05-13T22:48:23.310 INFO:tasks.workunit.client.1.smithi130.stdout:0/103: dread - d0/d7/fb zero size 2024-05-13T22:48:23.312 INFO:tasks.workunit.client.1.smithi130.stdout:0/104: chown d0/c13 11 1 2024-05-13T22:48:23.313 INFO:tasks.workunit.client.0.smithi102.stdout:6/924: rmdir d9/d45/ddf 39 2024-05-13T22:48:23.316 INFO:tasks.workunit.client.1.smithi130.stdout:9/90: rename l0 to d1/db/l17 0 2024-05-13T22:48:23.320 INFO:tasks.workunit.client.1.smithi130.stdout:1/78: dwrite db/f17 [0,4194304] 0 2024-05-13T22:48:23.320 INFO:tasks.workunit.client.1.smithi130.stdout:5/72: creat d6/d7/d9/f15 x:0 0 0 2024-05-13T22:48:23.322 INFO:tasks.workunit.client.1.smithi130.stdout:6/113: creat d4/d16/f2a x:0 0 0 2024-05-13T22:48:23.325 INFO:tasks.workunit.client.1.smithi130.stdout:3/85: link d8/f14 d8/dc/d19/f1e 0 2024-05-13T22:48:23.334 INFO:tasks.workunit.client.0.smithi102.stdout:4/865: dwrite d3/d4/d5f/fa3 [0,4194304] 0 2024-05-13T22:48:23.335 INFO:tasks.workunit.client.0.smithi102.stdout:3/990: dread de/d1c/f8a [0,4194304] 0 2024-05-13T22:48:23.338 INFO:tasks.workunit.client.1.smithi130.stdout:4/78: creat d8/f1b x:0 0 0 2024-05-13T22:48:23.339 INFO:tasks.workunit.client.1.smithi130.stdout:2/66: getdents d4/db/dc 0 2024-05-13T22:48:23.339 INFO:tasks.workunit.client.1.smithi130.stdout:4/79: write d8/f1b [604218,113924] 0 2024-05-13T22:48:23.340 INFO:tasks.workunit.client.0.smithi102.stdout:0/838: mkdir d1/d4d/d79/d9f/d104 0 2024-05-13T22:48:23.341 INFO:tasks.workunit.client.0.smithi102.stdout:7/771: dwrite d6/d15/f5e [0,4194304] 0 2024-05-13T22:48:23.343 INFO:tasks.workunit.client.1.smithi130.stdout:8/77: mkdir d1/d2/db/d15/d18/d1a 0 2024-05-13T22:48:23.343 INFO:tasks.workunit.client.0.smithi102.stdout:7/772: chown d6/d13/d2c/da1/dc1/le4 12 1 2024-05-13T22:48:23.347 INFO:tasks.workunit.client.1.smithi130.stdout:7/61: chown d7/d8/d9 35913 1 2024-05-13T22:48:23.349 INFO:tasks.workunit.client.1.smithi130.stdout:8/78: dwrite d1/d2/db/d15/d16/f19 [0,4194304] 0 2024-05-13T22:48:23.351 INFO:tasks.workunit.client.0.smithi102.stdout:9/861: write d1/d6/d21/d71/f9d [5113204,98076] 0 2024-05-13T22:48:23.352 INFO:tasks.workunit.client.0.smithi102.stdout:9/862: chown d1/d6/d45/d2d/dac/db0 226031670 1 2024-05-13T22:48:23.353 INFO:tasks.workunit.client.0.smithi102.stdout:6/925: mknod d9/c12b 0 2024-05-13T22:48:23.354 INFO:tasks.workunit.client.1.smithi130.stdout:8/79: dread d1/d2/fd [0,4194304] 0 2024-05-13T22:48:23.354 INFO:tasks.workunit.client.1.smithi130.stdout:9/91: symlink d1/d7/l18 0 2024-05-13T22:48:23.356 INFO:tasks.workunit.client.1.smithi130.stdout:9/92: write d1/fe [196307,46376] 0 2024-05-13T22:48:23.358 INFO:tasks.workunit.client.1.smithi130.stdout:1/79: rename f6 to db/f19 0 2024-05-13T22:48:23.361 INFO:tasks.workunit.client.0.smithi102.stdout:9/863: dwrite d1/f17 [0,4194304] 0 2024-05-13T22:48:23.364 INFO:tasks.workunit.client.1.smithi130.stdout:6/114: write d4/d15/d1d/f22 [844565,124468] 0 2024-05-13T22:48:23.365 INFO:tasks.workunit.client.1.smithi130.stdout:6/115: chown d4/d15 7830566 1 2024-05-13T22:48:23.365 INFO:tasks.workunit.client.1.smithi130.stdout:6/116: fsync d4/f12 0 2024-05-13T22:48:23.367 INFO:tasks.workunit.client.1.smithi130.stdout:6/117: truncate d4/d15/d1d/f22 1453014 0 2024-05-13T22:48:23.368 INFO:tasks.workunit.client.0.smithi102.stdout:4/866: creat d3/d4/d6/de/d115/d11b/f12f x:0 0 0 2024-05-13T22:48:23.374 INFO:tasks.workunit.client.0.smithi102.stdout:3/991: creat de/d10/d50/d63/da2/d140/f14f x:0 0 0 2024-05-13T22:48:23.380 INFO:tasks.workunit.client.0.smithi102.stdout:4/867: creat d3/dd8/f130 x:0 0 0 2024-05-13T22:48:23.382 INFO:tasks.workunit.client.0.smithi102.stdout:0/839: write d1/d4d/d79/f87 [1831854,12489] 0 2024-05-13T22:48:23.383 INFO:tasks.workunit.client.0.smithi102.stdout:7/773: write d6/d13/d58/dd1/f12 [1291659,43246] 0 2024-05-13T22:48:23.384 INFO:tasks.workunit.client.0.smithi102.stdout:3/992: symlink de/d10/d2a/d8c/d7f/l150 0 2024-05-13T22:48:23.386 INFO:tasks.workunit.client.0.smithi102.stdout:6/926: dread d9/d45/d56/f9b [4194304,4194304] 0 2024-05-13T22:48:23.386 INFO:tasks.workunit.client.0.smithi102.stdout:9/864: mkdir d1/d6/db/d4a/d75/d96/dc6/dd9/d125 0 2024-05-13T22:48:23.387 INFO:tasks.workunit.client.0.smithi102.stdout:4/868: truncate d3/d19/fae 601632 0 2024-05-13T22:48:23.388 INFO:tasks.workunit.client.0.smithi102.stdout:4/869: dread - d3/d19/d23/d4e/f12c zero size 2024-05-13T22:48:23.389 INFO:tasks.workunit.client.0.smithi102.stdout:7/774: creat d6/d15/d4d/d55/d82/feb x:0 0 0 2024-05-13T22:48:23.391 INFO:tasks.workunit.client.0.smithi102.stdout:7/775: readlink d6/d15/d4d/d55/d6f/d49/la0 0 2024-05-13T22:48:23.391 INFO:tasks.workunit.client.0.smithi102.stdout:3/993: rename de/d10/d2a/d8c/da8/l145 to de/d10/d50/d63/d98/l151 0 2024-05-13T22:48:23.392 INFO:tasks.workunit.client.0.smithi102.stdout:9/865: symlink d1/d6/d21/d71/l126 0 2024-05-13T22:48:23.395 INFO:tasks.workunit.client.0.smithi102.stdout:6/927: dwrite d9/d25/d60/f107 [0,4194304] 0 2024-05-13T22:48:23.399 INFO:tasks.workunit.client.0.smithi102.stdout:4/870: dread d3/d4/fb4 [0,4194304] 0 2024-05-13T22:48:23.401 INFO:tasks.workunit.client.0.smithi102.stdout:0/840: dread d1/de/d42/f93 [0,4194304] 0 2024-05-13T22:48:23.402 INFO:tasks.workunit.client.0.smithi102.stdout:9/866: mkdir d1/d6/d3b/d5b/d127 0 2024-05-13T22:48:23.406 INFO:tasks.workunit.client.0.smithi102.stdout:3/994: dwrite de/d10/d50/d63/f146 [0,4194304] 0 2024-05-13T22:48:23.406 INFO:tasks.workunit.client.0.smithi102.stdout:7/776: mknod d6/d13/d2c/dcb/cec 0 2024-05-13T22:48:23.418 INFO:tasks.workunit.client.0.smithi102.stdout:0/841: fdatasync d1/de/d32/d24/d2d/d38/fc0 0 2024-05-13T22:48:23.422 INFO:tasks.workunit.client.0.smithi102.stdout:7/777: mkdir d6/d15/d4d/d55/d6f/d65/ded 0 2024-05-13T22:48:23.431 INFO:tasks.workunit.client.0.smithi102.stdout:3/995: dread de/d10/d2a/d36/f12a [0,4194304] 0 2024-05-13T22:48:23.433 INFO:tasks.workunit.client.0.smithi102.stdout:7/778: rename d6/d13/d2c/d39/db3/fd3 to d6/d15/d4d/d55/d7f/fee 0 2024-05-13T22:48:23.437 INFO:tasks.workunit.client.0.smithi102.stdout:4/871: write d3/d19/d23/f104 [185690,83828] 0 2024-05-13T22:48:23.439 INFO:tasks.workunit.client.0.smithi102.stdout:7/779: mkdir d6/d15/d4d/d55/d6f/d49/def 0 2024-05-13T22:48:23.442 INFO:tasks.workunit.client.0.smithi102.stdout:6/928: write d9/db/d14/d35/d47/f5c [1059738,110822] 0 2024-05-13T22:48:23.444 INFO:tasks.workunit.client.0.smithi102.stdout:4/872: chown d3/d19/d23/d7e/f122 32816 1 2024-05-13T22:48:23.445 INFO:tasks.workunit.client.0.smithi102.stdout:9/867: dwrite d1/d6/d45/d3e/f73 [0,4194304] 0 2024-05-13T22:48:23.446 INFO:tasks.workunit.client.0.smithi102.stdout:9/868: chown d1/f17 20785 1 2024-05-13T22:48:23.448 INFO:tasks.workunit.client.0.smithi102.stdout:7/780: rename d6/d13/d2c/d39/f3f to d6/d13/d62/ff0 0 2024-05-13T22:48:23.449 INFO:tasks.workunit.client.0.smithi102.stdout:3/996: link de/d10/d46/cf0 de/d10/d2a/d36/db2/de9/c152 0 2024-05-13T22:48:23.450 INFO:tasks.workunit.client.0.smithi102.stdout:6/929: dwrite d9/d45/d56/dd5/fbd [0,4194304] 0 2024-05-13T22:48:23.454 INFO:tasks.workunit.client.0.smithi102.stdout:9/869: rename d1/fdd to d1/d6/db/d4a/d75/d96/dc6/dde/f128 0 2024-05-13T22:48:23.456 INFO:tasks.workunit.client.0.smithi102.stdout:7/781: symlink d6/d15/d4d/d55/lf1 0 2024-05-13T22:48:23.458 INFO:tasks.workunit.client.0.smithi102.stdout:7/782: write d6/d15/d4c/f8c [212991,11030] 0 2024-05-13T22:48:23.460 INFO:tasks.workunit.client.0.smithi102.stdout:7/783: fsync d6/dd0/d80/fa5 0 2024-05-13T22:48:23.461 INFO:tasks.workunit.client.0.smithi102.stdout:6/930: mkdir d9/d45/d56/dd5/d12c 0 2024-05-13T22:48:23.463 INFO:tasks.workunit.client.0.smithi102.stdout:4/873: symlink d3/d19/dad/dc9/l131 0 2024-05-13T22:48:23.463 INFO:tasks.workunit.client.0.smithi102.stdout:9/870: dwrite d1/d6/db/d30/f112 [0,4194304] 0 2024-05-13T22:48:23.463 INFO:tasks.workunit.client.0.smithi102.stdout:7/784: truncate d6/fc 6251238 0 2024-05-13T22:48:23.469 INFO:tasks.workunit.client.0.smithi102.stdout:9/871: symlink d1/d6/db/d30/d8a/l129 0 2024-05-13T22:48:23.472 INFO:tasks.workunit.client.0.smithi102.stdout:6/931: read d9/db/f77 [5439073,73552] 0 2024-05-13T22:48:23.472 INFO:tasks.workunit.client.0.smithi102.stdout:7/785: mkdir d6/dd0/df2 0 2024-05-13T22:48:23.475 INFO:tasks.workunit.client.0.smithi102.stdout:9/872: creat d1/d6/d45/da1/f12a x:0 0 0 2024-05-13T22:48:23.478 INFO:tasks.workunit.client.0.smithi102.stdout:7/786: dwrite d6/dd0/d80/f8a [0,4194304] 0 2024-05-13T22:48:23.479 INFO:tasks.workunit.client.0.smithi102.stdout:2/930: sync 2024-05-13T22:48:23.483 INFO:tasks.workunit.client.0.smithi102.stdout:7/787: readlink d6/d15/l38 0 2024-05-13T22:48:23.483 INFO:tasks.workunit.client.0.smithi102.stdout:7/788: readlink d6/l1c 0 2024-05-13T22:48:23.484 INFO:tasks.workunit.client.0.smithi102.stdout:7/789: readlink d6/d13/d2c/da1/dc1/lbc 0 2024-05-13T22:48:23.487 INFO:tasks.workunit.client.0.smithi102.stdout:4/874: rename d3/d19/d23/d3a/da4/f10f to d3/d19/d23/d4e/f132 0 2024-05-13T22:48:23.491 INFO:tasks.workunit.client.0.smithi102.stdout:7/790: symlink d6/d15/d4d/d55/d7f/lf3 0 2024-05-13T22:48:23.496 INFO:tasks.workunit.client.0.smithi102.stdout:2/931: rename d7/d15/d67/d129/d27 to d7/d9/d14/d61/d102/d131 0 2024-05-13T22:48:23.499 INFO:tasks.workunit.client.0.smithi102.stdout:4/875: readlink d3/d19/l40 0 2024-05-13T22:48:23.500 INFO:tasks.workunit.client.0.smithi102.stdout:9/873: dread d1/d6/f4e [0,4194304] 0 2024-05-13T22:48:23.504 INFO:tasks.workunit.client.0.smithi102.stdout:4/876: chown d3/d19/dad/dc9/le8 326774 1 2024-05-13T22:48:23.506 INFO:tasks.workunit.client.0.smithi102.stdout:7/791: creat d6/d22/dd2/ff4 x:0 0 0 2024-05-13T22:48:23.508 INFO:tasks.workunit.client.0.smithi102.stdout:9/874: truncate d1/d6/d3b/f38 739913 0 2024-05-13T22:48:23.510 INFO:tasks.workunit.client.0.smithi102.stdout:2/932: link d7/d9/d14/d61/d102/d131/dab/dee/f12a d7/d9/d14/d61/d102/d131/dab/dee/f132 0 2024-05-13T22:48:23.513 INFO:tasks.workunit.client.0.smithi102.stdout:4/877: rename d3/l11 to d3/d19/dad/dc9/d123/l133 0 2024-05-13T22:48:23.514 INFO:tasks.workunit.client.0.smithi102.stdout:3/997: dwrite de/d10/d2a/f2f [0,4194304] 0 2024-05-13T22:48:23.527 INFO:tasks.workunit.client.0.smithi102.stdout:3/998: truncate de/d1c/f52 1456219 0 2024-05-13T22:48:23.528 INFO:tasks.workunit.client.0.smithi102.stdout:3/999: write de/f37 [5716559,92142] 0 2024-05-13T22:48:23.529 INFO:tasks.workunit.client.0.smithi102.stdout:9/875: creat d1/d6/d45/d3e/dbe/f12b x:0 0 0 2024-05-13T22:48:23.537 INFO:tasks.workunit.client.0.smithi102.stdout:6/932: dwrite d9/db/d14/d35/f67 [0,4194304] 0 2024-05-13T22:48:23.538 INFO:tasks.workunit.client.0.smithi102.stdout:6/933: write d9/d25/d71/f7a [1488754,29944] 0 2024-05-13T22:48:23.560 INFO:tasks.workunit.client.0.smithi102.stdout:2/933: write d7/d9/d10/d30/d53/dd1/fd2 [723036,9943] 0 2024-05-13T22:48:23.599 INFO:tasks.workunit.client.1.smithi130.stdout:2/67: creat d4/f1f x:0 0 0 2024-05-13T22:48:23.600 INFO:tasks.workunit.client.1.smithi130.stdout:4/80: mknod d8/dc/c1c 0 2024-05-13T22:48:23.600 INFO:tasks.workunit.client.1.smithi130.stdout:2/68: truncate d4/fa 1487042 0 2024-05-13T22:48:23.600 INFO:tasks.workunit.client.1.smithi130.stdout:2/69: chown d4/db/f1c 4 1 2024-05-13T22:48:23.602 INFO:tasks.workunit.client.0.smithi102.stdout:9/876: mkdir d1/d6/db/d4a/d81/d116/d12c 0 2024-05-13T22:48:23.603 INFO:tasks.workunit.client.1.smithi130.stdout:0/105: creat d0/d7/dd/d17/d1f/f25 x:0 0 0 2024-05-13T22:48:23.607 INFO:tasks.workunit.client.1.smithi130.stdout:8/80: creat d1/d2/db/d15/d18/f1b x:0 0 0 2024-05-13T22:48:23.609 INFO:tasks.workunit.client.0.smithi102.stdout:2/934: unlink d7/d9/d10/d1b/d54/l121 0 2024-05-13T22:48:23.611 INFO:tasks.workunit.client.0.smithi102.stdout:8/829: sync 2024-05-13T22:48:23.611 INFO:tasks.workunit.client.0.smithi102.stdout:1/701: sync 2024-05-13T22:48:23.614 INFO:tasks.workunit.client.0.smithi102.stdout:4/878: rename d3/d4/d6/de/f15 to d3/d19/d23/d39/f134 0 2024-05-13T22:48:23.615 INFO:tasks.workunit.client.1.smithi130.stdout:5/73: getdents d6/dd 0 2024-05-13T22:48:23.617 INFO:tasks.workunit.client.0.smithi102.stdout:7/792: getdents d6 0 2024-05-13T22:48:23.618 INFO:tasks.workunit.client.0.smithi102.stdout:7/793: fsync d6/f94 0 2024-05-13T22:48:23.618 INFO:tasks.workunit.client.1.smithi130.stdout:1/80: dwrite db/f19 [0,4194304] 0 2024-05-13T22:48:23.619 INFO:tasks.workunit.client.1.smithi130.stdout:1/81: read f8 [11256,41941] 0 2024-05-13T22:48:23.619 INFO:tasks.workunit.client.0.smithi102.stdout:9/877: chown d1/d6/db/d4a/f52 8282463 1 2024-05-13T22:48:23.620 INFO:tasks.workunit.client.0.smithi102.stdout:6/934: dread d9/db/f3c [0,4194304] 0 2024-05-13T22:48:23.621 INFO:tasks.workunit.client.1.smithi130.stdout:3/86: unlink d8/f14 0 2024-05-13T22:48:23.622 INFO:tasks.workunit.client.1.smithi130.stdout:3/87: write f3 [3612891,20638] 0 2024-05-13T22:48:23.624 INFO:tasks.workunit.client.1.smithi130.stdout:6/118: symlink d4/d15/l2b 0 2024-05-13T22:48:23.629 INFO:tasks.workunit.client.1.smithi130.stdout:2/70: rename f3 to d4/db/dc/f20 0 2024-05-13T22:48:23.634 INFO:tasks.workunit.client.0.smithi102.stdout:9/878: mknod d1/d6/db/d4a/d81/dd6/c12d 0 2024-05-13T22:48:23.635 INFO:tasks.workunit.client.1.smithi130.stdout:8/81: truncate d1/d2/f3 5138459 0 2024-05-13T22:48:23.637 INFO:tasks.workunit.client.0.smithi102.stdout:6/935: dread d9/db/fa1 [0,4194304] 0 2024-05-13T22:48:23.638 INFO:tasks.workunit.client.0.smithi102.stdout:1/702: creat d2/d4/d5/d19/dd3/fe8 x:0 0 0 2024-05-13T22:48:23.638 INFO:tasks.workunit.client.0.smithi102.stdout:7/794: dread d6/d15/d4d/d55/d6f/d49/fb2 [0,4194304] 0 2024-05-13T22:48:23.639 INFO:tasks.workunit.client.1.smithi130.stdout:5/74: symlink d6/d7/d9/l16 0 2024-05-13T22:48:23.640 INFO:tasks.workunit.client.0.smithi102.stdout:7/795: chown d6/d13/d2c/d39/lb0 151297 1 2024-05-13T22:48:23.640 INFO:tasks.workunit.client.0.smithi102.stdout:7/796: fdatasync d6/d15/d4d/d55/d6f/fe9 0 2024-05-13T22:48:23.641 INFO:tasks.workunit.client.0.smithi102.stdout:7/797: chown d6/d13/d2c/cb8 1997 1 2024-05-13T22:48:23.644 INFO:tasks.workunit.client.1.smithi130.stdout:1/82: rename db/f19 to db/f1a 0 2024-05-13T22:48:23.644 INFO:tasks.workunit.client.1.smithi130.stdout:1/83: chown c2 62920 1 2024-05-13T22:48:23.646 INFO:tasks.workunit.client.0.smithi102.stdout:7/798: dread d6/f77 [0,4194304] 0 2024-05-13T22:48:23.649 INFO:tasks.workunit.client.0.smithi102.stdout:6/936: truncate d9/db/f36 9095570 0 2024-05-13T22:48:23.652 INFO:tasks.workunit.client.0.smithi102.stdout:8/830: rename d2/d7/d27/d64/d80/dc8/f35 to d2/f10b 0 2024-05-13T22:48:23.654 INFO:tasks.workunit.client.1.smithi130.stdout:2/71: creat d4/f21 x:0 0 0 2024-05-13T22:48:23.654 INFO:tasks.workunit.client.1.smithi130.stdout:2/72: chown d4/f21 10956236 1 2024-05-13T22:48:23.656 INFO:tasks.workunit.client.0.smithi102.stdout:2/935: getdents d7/d15/d67 0 2024-05-13T22:48:23.657 INFO:tasks.workunit.client.1.smithi130.stdout:8/82: creat d1/d2/db/d15/d16/f1c x:0 0 0 2024-05-13T22:48:23.659 INFO:tasks.workunit.client.0.smithi102.stdout:1/703: creat d2/d4/d5/d19/dbe/fe9 x:0 0 0 2024-05-13T22:48:23.659 INFO:tasks.workunit.client.0.smithi102.stdout:1/704: write d2/d45/d96/fd0 [715772,119110] 0 2024-05-13T22:48:23.663 INFO:tasks.workunit.client.1.smithi130.stdout:5/75: mkdir d6/dd/d17 0 2024-05-13T22:48:23.663 INFO:tasks.workunit.client.0.smithi102.stdout:0/842: sync 2024-05-13T22:48:23.663 INFO:tasks.workunit.client.1.smithi130.stdout:7/62: truncate d7/d8/f13 136842 0 2024-05-13T22:48:23.676 INFO:tasks.workunit.client.1.smithi130.stdout:2/73: creat d4/db/f22 x:0 0 0 2024-05-13T22:48:23.678 INFO:tasks.workunit.client.1.smithi130.stdout:0/106: rmdir d0/d11 0 2024-05-13T22:48:23.678 INFO:tasks.workunit.client.1.smithi130.stdout:1/84: dwrite f8 [0,4194304] 0 2024-05-13T22:48:23.679 INFO:tasks.workunit.client.1.smithi130.stdout:1/85: fdatasync db/f16 0 2024-05-13T22:48:23.680 INFO:tasks.workunit.client.1.smithi130.stdout:8/83: mknod d1/d2/c1d 0 2024-05-13T22:48:23.680 INFO:tasks.workunit.client.0.smithi102.stdout:0/843: creat d1/d4d/d79/d9f/dbd/f105 x:0 0 0 2024-05-13T22:48:23.681 INFO:tasks.workunit.client.0.smithi102.stdout:0/844: readlink d1/de/d32/d24/d2d/d3e/lcd 0 2024-05-13T22:48:23.682 INFO:tasks.workunit.client.1.smithi130.stdout:9/93: getdents d1/d7 0 2024-05-13T22:48:23.685 INFO:tasks.workunit.client.1.smithi130.stdout:3/88: link l1 d8/dc/d19/d1d/l1f 0 2024-05-13T22:48:23.686 INFO:tasks.workunit.client.1.smithi130.stdout:6/119: rmdir d4 39 2024-05-13T22:48:23.687 INFO:tasks.workunit.client.0.smithi102.stdout:1/705: unlink d2/d4/cc 0 2024-05-13T22:48:23.691 INFO:tasks.workunit.client.1.smithi130.stdout:2/74: creat d4/db/dc/d18/f23 x:0 0 0 2024-05-13T22:48:23.691 INFO:tasks.workunit.client.0.smithi102.stdout:9/879: rename d1/d6/db/d4a to d1/d6/d21/d12e 0 2024-05-13T22:48:23.692 INFO:tasks.workunit.client.1.smithi130.stdout:4/81: dwrite f6 [0,4194304] 0 2024-05-13T22:48:23.693 INFO:tasks.workunit.client.0.smithi102.stdout:9/880: write d1/d6/db/d30/f36 [5696612,2384] 0 2024-05-13T22:48:23.693 INFO:tasks.workunit.client.0.smithi102.stdout:8/831: creat d2/d7/d27/d64/d80/dc8/d93/f10c x:0 0 0 2024-05-13T22:48:23.695 INFO:tasks.workunit.client.1.smithi130.stdout:1/86: mkdir db/d1b 0 2024-05-13T22:48:23.696 INFO:tasks.workunit.client.1.smithi130.stdout:1/87: read - db/f16 zero size 2024-05-13T22:48:23.697 INFO:tasks.workunit.client.1.smithi130.stdout:0/107: symlink d0/d7/dd/d17/d1f/l26 0 2024-05-13T22:48:23.698 INFO:tasks.workunit.client.1.smithi130.stdout:2/75: dwrite d4/f1f [0,4194304] 0 2024-05-13T22:48:23.698 INFO:tasks.workunit.client.0.smithi102.stdout:1/706: dread d2/d45/f58 [0,4194304] 0 2024-05-13T22:48:23.699 INFO:tasks.workunit.client.1.smithi130.stdout:0/108: write d0/f21 [723861,88521] 0 2024-05-13T22:48:23.699 INFO:tasks.workunit.client.0.smithi102.stdout:0/845: truncate d1/de/d32/d24/d2d/f5f 2730951 0 2024-05-13T22:48:23.699 INFO:tasks.workunit.client.0.smithi102.stdout:4/879: write d3/d19/d23/d3a/da4/f108 [1724620,3399] 0 2024-05-13T22:48:23.701 INFO:tasks.workunit.client.0.smithi102.stdout:4/880: chown d3/d4/f7b 32 1 2024-05-13T22:48:23.705 INFO:tasks.workunit.client.0.smithi102.stdout:1/707: dwrite d2/d45/fd6 [0,4194304] 0 2024-05-13T22:48:23.707 INFO:tasks.workunit.client.0.smithi102.stdout:8/832: creat d2/d36/d49/d7e/f10d x:0 0 0 2024-05-13T22:48:23.708 INFO:tasks.workunit.client.1.smithi130.stdout:9/94: creat d1/d7/f19 x:0 0 0 2024-05-13T22:48:23.710 INFO:tasks.workunit.client.1.smithi130.stdout:0/109: dread d0/f21 [0,4194304] 0 2024-05-13T22:48:23.711 INFO:tasks.workunit.client.1.smithi130.stdout:7/63: symlink d7/d8/l14 0 2024-05-13T22:48:23.713 INFO:tasks.workunit.client.0.smithi102.stdout:4/881: mkdir d3/d19/d23/d39/da7/da9/d135 0 2024-05-13T22:48:23.718 INFO:tasks.workunit.client.0.smithi102.stdout:8/833: chown d2/d36/d49/d7e/ld0 27486622 1 2024-05-13T22:48:23.719 INFO:tasks.workunit.client.1.smithi130.stdout:6/120: write d4/d15/d1f/f26 [642119,114743] 0 2024-05-13T22:48:23.721 INFO:tasks.workunit.client.1.smithi130.stdout:6/121: chown d4/d15/c1c 436 1 2024-05-13T22:48:23.722 INFO:tasks.workunit.client.0.smithi102.stdout:6/937: dwrite d9/d45/da0/dd9/fdc [0,4194304] 0 2024-05-13T22:48:23.722 INFO:tasks.workunit.client.1.smithi130.stdout:3/89: creat d8/dc/d19/f20 x:0 0 0 2024-05-13T22:48:23.722 INFO:tasks.workunit.client.1.smithi130.stdout:0/110: dwrite d0/d7/dd/d17/d1f/f25 [0,4194304] 0 2024-05-13T22:48:23.726 INFO:tasks.workunit.client.1.smithi130.stdout:4/82: rename d8/df/f10 to d8/f1d 0 2024-05-13T22:48:23.726 INFO:tasks.workunit.client.1.smithi130.stdout:7/64: rename d7/da to d7/da/d15 22 2024-05-13T22:48:23.726 INFO:tasks.workunit.client.0.smithi102.stdout:2/936: dwrite d7/d9/d11/f38 [0,4194304] 0 2024-05-13T22:48:23.727 INFO:tasks.workunit.client.0.smithi102.stdout:7/799: truncate d6/d15/d4d/d55/d6f/d49/d97/fa6 355247 0 2024-05-13T22:48:23.728 INFO:tasks.workunit.client.1.smithi130.stdout:4/83: chown d8/f1a 118989 1 2024-05-13T22:48:23.728 INFO:tasks.workunit.client.1.smithi130.stdout:6/122: dread d4/d15/d1f/f20 [0,4194304] 0 2024-05-13T22:48:23.728 INFO:tasks.workunit.client.1.smithi130.stdout:4/84: chown f4 274196233 1 2024-05-13T22:48:23.740 INFO:tasks.workunit.client.0.smithi102.stdout:8/834: unlink d2/d7/d27/d64/d96/cf5 0 2024-05-13T22:48:23.742 INFO:tasks.workunit.client.0.smithi102.stdout:6/938: creat d9/d25/d60/f12d x:0 0 0 2024-05-13T22:48:23.744 INFO:tasks.workunit.client.0.smithi102.stdout:9/881: rename d1/d6/d21/d12e/d64/fd8 to d1/d6/d45/f12f 0 2024-05-13T22:48:23.752 INFO:tasks.workunit.client.1.smithi130.stdout:8/84: creat d1/d2/db/d15/d18/d1a/f1e x:0 0 0 2024-05-13T22:48:23.752 INFO:tasks.workunit.client.1.smithi130.stdout:8/85: readlink d1/d2/le 0 2024-05-13T22:48:23.752 INFO:tasks.workunit.client.0.smithi102.stdout:8/835: mknod d2/d36/d49/d7e/c10e 0 2024-05-13T22:48:23.753 INFO:tasks.workunit.client.1.smithi130.stdout:8/86: write d1/d2/db/d15/d18/d1a/f1e [827334,110927] 0 2024-05-13T22:48:23.754 INFO:tasks.workunit.client.1.smithi130.stdout:9/95: readlink d1/d7/l12 0 2024-05-13T22:48:23.754 INFO:tasks.workunit.client.1.smithi130.stdout:8/87: readlink d1/d2/db/l13 0 2024-05-13T22:48:23.756 INFO:tasks.workunit.client.0.smithi102.stdout:6/939: mknod d9/d25/d60/d6a/c12e 0 2024-05-13T22:48:23.757 INFO:tasks.workunit.client.1.smithi130.stdout:5/76: rmdir d6/dd/d17 0 2024-05-13T22:48:23.758 INFO:tasks.workunit.client.1.smithi130.stdout:5/77: write d6/dd/f13 [362768,55014] 0 2024-05-13T22:48:23.759 INFO:tasks.workunit.client.0.smithi102.stdout:8/836: dread d2/d7/d27/d64/d80/dc8/d40/d85/ddc/ffa [0,4194304] 0 2024-05-13T22:48:23.762 INFO:tasks.workunit.client.1.smithi130.stdout:5/78: dread f4 [0,4194304] 0 2024-05-13T22:48:23.762 INFO:tasks.workunit.client.1.smithi130.stdout:3/90: unlink d8/c10 0 2024-05-13T22:48:23.765 INFO:tasks.workunit.client.0.smithi102.stdout:7/800: mknod d6/d13/d2c/cf5 0 2024-05-13T22:48:23.770 INFO:tasks.workunit.client.0.smithi102.stdout:6/940: truncate d9/db/d14/dcc/f57 216502 0 2024-05-13T22:48:23.773 INFO:tasks.workunit.client.0.smithi102.stdout:0/846: rename d1/d68/de2/lfa to d1/de/d32/d24/d2d/d38/db7/l106 0 2024-05-13T22:48:23.780 INFO:tasks.workunit.client.1.smithi130.stdout:6/123: mkdir d4/d16/d2c 0 2024-05-13T22:48:23.780 INFO:tasks.workunit.client.1.smithi130.stdout:4/85: creat d8/f1e x:0 0 0 2024-05-13T22:48:23.781 INFO:tasks.workunit.client.1.smithi130.stdout:1/88: symlink db/d1b/l1c 0 2024-05-13T22:48:23.781 INFO:tasks.workunit.client.1.smithi130.stdout:4/86: write f5 [685843,94092] 0 2024-05-13T22:48:23.786 INFO:tasks.workunit.client.0.smithi102.stdout:8/837: dwrite d2/f4 [0,4194304] 0 2024-05-13T22:48:23.786 INFO:tasks.workunit.client.0.smithi102.stdout:7/801: read d6/d15/f37 [179739,30439] 0 2024-05-13T22:48:23.787 INFO:tasks.workunit.client.0.smithi102.stdout:2/937: write d7/d9/d10/d1b/d54/fcb [152020,31568] 0 2024-05-13T22:48:23.788 INFO:tasks.workunit.client.1.smithi130.stdout:4/87: dwrite d8/f1a [0,4194304] 0 2024-05-13T22:48:23.791 INFO:tasks.workunit.client.1.smithi130.stdout:9/96: creat d1/d7/f1a x:0 0 0 2024-05-13T22:48:23.793 INFO:tasks.workunit.client.0.smithi102.stdout:1/708: rename d2/d10/d62/c83 to d2/d4/d5/d19/dd3/cea 0 2024-05-13T22:48:23.795 INFO:tasks.workunit.client.0.smithi102.stdout:7/802: dread d6/d13/d58/f34 [0,4194304] 0 2024-05-13T22:48:23.795 INFO:tasks.workunit.client.1.smithi130.stdout:9/97: dread d1/f4 [0,4194304] 0 2024-05-13T22:48:23.795 INFO:tasks.workunit.client.1.smithi130.stdout:8/88: creat d1/dc/f1f x:0 0 0 2024-05-13T22:48:23.796 INFO:tasks.workunit.client.0.smithi102.stdout:0/847: mkdir d1/de/d42/d61/d107 0 2024-05-13T22:48:23.801 INFO:tasks.workunit.client.0.smithi102.stdout:8/838: rmdir d2/d7/d27/d64/d96/dae/d76 39 2024-05-13T22:48:23.805 INFO:tasks.workunit.client.0.smithi102.stdout:6/941: write d9/db/d14/d35/dc9/ff8 [146547,54399] 0 2024-05-13T22:48:23.811 INFO:tasks.workunit.client.0.smithi102.stdout:7/803: chown d6/dd0/l27 52 1 2024-05-13T22:48:23.811 INFO:tasks.workunit.client.0.smithi102.stdout:6/942: dwrite d9/db/d14/d35/d47/f70 [0,4194304] 0 2024-05-13T22:48:23.812 INFO:tasks.workunit.client.1.smithi130.stdout:6/124: read d4/d15/d1f/f20 [7515898,105847] 0 2024-05-13T22:48:23.812 INFO:tasks.workunit.client.0.smithi102.stdout:7/804: readlink d6/d15/d4d/d55/d6f/l73 0 2024-05-13T22:48:23.813 INFO:tasks.workunit.client.1.smithi130.stdout:6/125: chown d4/d15/d1d 13 1 2024-05-13T22:48:23.813 INFO:tasks.workunit.client.1.smithi130.stdout:3/91: dwrite d8/dc/d19/f20 [0,4194304] 0 2024-05-13T22:48:23.815 INFO:tasks.workunit.client.0.smithi102.stdout:0/848: rmdir d1/de/d32/dd8 39 2024-05-13T22:48:23.821 INFO:tasks.workunit.client.1.smithi130.stdout:4/88: creat d8/f1f x:0 0 0 2024-05-13T22:48:23.821 INFO:tasks.workunit.client.0.smithi102.stdout:4/882: rename d3/d19/d23/d39/da7/f75 to d3/d19/d23/d3a/f136 0 2024-05-13T22:48:23.823 INFO:tasks.workunit.client.0.smithi102.stdout:4/883: write d3/d19/d23/d4e/f12c [300284,64236] 0 2024-05-13T22:48:23.823 INFO:tasks.workunit.client.1.smithi130.stdout:4/89: write d8/dc/fd [3326561,23367] 0 2024-05-13T22:48:23.825 INFO:tasks.workunit.client.0.smithi102.stdout:8/839: dread d2/d7/d27/d64/d80/dc8/d40/f5b [4194304,4194304] 0 2024-05-13T22:48:23.825 INFO:tasks.workunit.client.1.smithi130.stdout:9/98: rename d1/db/c10 to d1/db/c1b 0 2024-05-13T22:48:23.827 INFO:tasks.workunit.client.1.smithi130.stdout:8/89: symlink d1/d2/db/d15/d16/l20 0 2024-05-13T22:48:23.827 INFO:tasks.workunit.client.1.smithi130.stdout:5/79: dwrite d6/f11 [0,4194304] 0 2024-05-13T22:48:23.829 INFO:tasks.workunit.client.1.smithi130.stdout:5/80: write f2 [2782935,91012] 0 2024-05-13T22:48:23.829 INFO:tasks.workunit.client.0.smithi102.stdout:7/805: creat d6/dd0/d80/ff6 x:0 0 0 2024-05-13T22:48:23.830 INFO:tasks.workunit.client.0.smithi102.stdout:7/806: truncate d6/fcc 498011 0 2024-05-13T22:48:23.830 INFO:tasks.workunit.client.0.smithi102.stdout:7/807: stat d6/d13/d2c/d39/l9c 0 2024-05-13T22:48:23.831 INFO:tasks.workunit.client.0.smithi102.stdout:7/808: readlink d6/dd0/l27 0 2024-05-13T22:48:23.831 INFO:tasks.workunit.client.1.smithi130.stdout:0/111: link d0/f1e d0/f27 0 2024-05-13T22:48:23.832 INFO:tasks.workunit.client.1.smithi130.stdout:0/112: stat d0/f2 0 2024-05-13T22:48:23.833 INFO:tasks.workunit.client.1.smithi130.stdout:4/90: dwrite d8/dc/fd [0,4194304] 0 2024-05-13T22:48:23.833 INFO:tasks.workunit.client.0.smithi102.stdout:0/849: truncate d1/d4d/d79/d9f/dce/fca 207797 0 2024-05-13T22:48:23.837 INFO:tasks.workunit.client.0.smithi102.stdout:2/938: creat d7/d9/d14/f133 x:0 0 0 2024-05-13T22:48:23.837 INFO:tasks.workunit.client.0.smithi102.stdout:2/939: chown d7/d9/lc 2196 1 2024-05-13T22:48:23.839 INFO:tasks.workunit.client.1.smithi130.stdout:6/126: creat d4/d15/d1f/f2d x:0 0 0 2024-05-13T22:48:23.840 INFO:tasks.workunit.client.1.smithi130.stdout:6/127: read d4/d15/d1f/f20 [2246712,19322] 0 2024-05-13T22:48:23.840 INFO:tasks.workunit.client.1.smithi130.stdout:6/128: chown d4/d16/d2c 6080 1 2024-05-13T22:48:23.842 INFO:tasks.workunit.client.0.smithi102.stdout:6/943: mkdir d9/db/d14/d35/d47/dc1/d126/d12f 0 2024-05-13T22:48:23.844 INFO:tasks.workunit.client.0.smithi102.stdout:7/809: symlink d6/d13/d2c/da1/dc1/lf7 0 2024-05-13T22:48:23.845 INFO:tasks.workunit.client.1.smithi130.stdout:2/76: getdents d4/db/dc/d18 0 2024-05-13T22:48:23.847 INFO:tasks.workunit.client.0.smithi102.stdout:0/850: stat d1/de/d32/f89 0 2024-05-13T22:48:23.848 INFO:tasks.workunit.client.1.smithi130.stdout:2/77: dread d4/fa [0,4194304] 0 2024-05-13T22:48:23.848 INFO:tasks.workunit.client.0.smithi102.stdout:1/709: truncate d2/d10/d62/d7b/f95 293723 0 2024-05-13T22:48:23.848 INFO:tasks.workunit.client.0.smithi102.stdout:1/710: fdatasync d2/d4/d26/fe6 0 2024-05-13T22:48:23.850 INFO:tasks.workunit.client.0.smithi102.stdout:4/884: creat d3/d19/d23/d4e/d68/dcf/f137 x:0 0 0 2024-05-13T22:48:23.852 INFO:tasks.workunit.client.0.smithi102.stdout:2/940: symlink d7/d9/d14/d61/d102/d131/dab/dee/l134 0 2024-05-13T22:48:23.853 INFO:tasks.workunit.client.1.smithi130.stdout:9/99: mknod d1/d7/c1c 0 2024-05-13T22:48:23.857 INFO:tasks.workunit.client.0.smithi102.stdout:6/944: rename d9/da3/fde to d9/d45/da0/f130 0 2024-05-13T22:48:23.859 INFO:tasks.workunit.client.1.smithi130.stdout:8/90: rename d1/d2/db/d15/d16/f19 to d1/dc/f21 0 2024-05-13T22:48:23.859 INFO:tasks.workunit.client.1.smithi130.stdout:1/89: truncate f8 155593 0 2024-05-13T22:48:23.859 INFO:tasks.workunit.client.1.smithi130.stdout:8/91: fdatasync d1/dc/f11 0 2024-05-13T22:48:23.860 INFO:tasks.workunit.client.1.smithi130.stdout:1/90: chown db/c11 7 1 2024-05-13T22:48:23.860 INFO:tasks.workunit.client.1.smithi130.stdout:8/92: dread - d1/dc/f1f zero size 2024-05-13T22:48:23.861 INFO:tasks.workunit.client.0.smithi102.stdout:0/851: rmdir d1/de/d32/d8d 39 2024-05-13T22:48:23.864 INFO:tasks.workunit.client.1.smithi130.stdout:7/65: dread d7/d8/f13 [0,4194304] 0 2024-05-13T22:48:23.864 INFO:tasks.workunit.client.0.smithi102.stdout:1/711: mknod d2/d10/d62/d86/ceb 0 2024-05-13T22:48:23.865 INFO:tasks.workunit.client.0.smithi102.stdout:7/810: dwrite d6/d15/d4d/d55/d7f/fee [0,4194304] 0 2024-05-13T22:48:23.867 INFO:tasks.workunit.client.1.smithi130.stdout:0/113: unlink d0/f2 0 2024-05-13T22:48:23.867 INFO:tasks.workunit.client.1.smithi130.stdout:0/114: read d0/d7/f23 [1111824,97841] 0 2024-05-13T22:48:23.870 INFO:tasks.workunit.client.1.smithi130.stdout:4/91: creat d8/df/f20 x:0 0 0 2024-05-13T22:48:23.873 INFO:tasks.workunit.client.0.smithi102.stdout:8/840: write d2/d7/f9 [996358,80979] 0 2024-05-13T22:48:23.875 INFO:tasks.workunit.client.1.smithi130.stdout:6/129: rmdir d4/d15 39 2024-05-13T22:48:23.875 INFO:tasks.workunit.client.1.smithi130.stdout:0/115: dwrite d0/d7/fb [0,4194304] 0 2024-05-13T22:48:23.876 INFO:tasks.workunit.client.0.smithi102.stdout:6/945: truncate d9/d25/d60/d6a/dd6/fd8 1036122 0 2024-05-13T22:48:23.881 INFO:tasks.workunit.client.1.smithi130.stdout:2/78: unlink d4/f6 0 2024-05-13T22:48:23.883 INFO:tasks.workunit.client.0.smithi102.stdout:0/852: creat d1/de/d32/d8a/f108 x:0 0 0 2024-05-13T22:48:23.885 INFO:tasks.workunit.client.1.smithi130.stdout:9/100: unlink d1/fe 0 2024-05-13T22:48:23.885 INFO:tasks.workunit.client.1.smithi130.stdout:9/101: stat d1/d7/fa 0 2024-05-13T22:48:23.887 INFO:tasks.workunit.client.0.smithi102.stdout:2/941: write d7/d9/d14/d61/d102/d131/dab/dee/f132 [1006042,87056] 0 2024-05-13T22:48:23.887 INFO:tasks.workunit.client.0.smithi102.stdout:1/712: write d2/d4/d38/f68 [2056980,66709] 0 2024-05-13T22:48:23.890 INFO:tasks.workunit.client.1.smithi130.stdout:3/92: truncate f3 3850856 0 2024-05-13T22:48:23.891 INFO:tasks.workunit.client.0.smithi102.stdout:7/811: symlink d6/dd0/d80/lf8 0 2024-05-13T22:48:23.891 INFO:tasks.workunit.client.1.smithi130.stdout:3/93: read d8/dc/d19/f20 [2624432,9500] 0 2024-05-13T22:48:23.892 INFO:tasks.workunit.client.1.smithi130.stdout:9/102: dwrite d1/f11 [0,4194304] 0 2024-05-13T22:48:23.893 INFO:tasks.workunit.client.1.smithi130.stdout:9/103: stat d1/d7/c1c 0 2024-05-13T22:48:23.893 INFO:tasks.workunit.client.0.smithi102.stdout:4/885: symlink d3/d19/l138 0 2024-05-13T22:48:23.895 INFO:tasks.workunit.client.1.smithi130.stdout:1/91: rename db/f12 to db/f1d 0 2024-05-13T22:48:23.898 INFO:tasks.workunit.client.1.smithi130.stdout:8/93: creat d1/d2/db/d15/f22 x:0 0 0 2024-05-13T22:48:23.899 INFO:tasks.workunit.client.0.smithi102.stdout:8/841: creat d2/d83/f10f x:0 0 0 2024-05-13T22:48:23.900 INFO:tasks.workunit.client.1.smithi130.stdout:9/104: dwrite d1/f11 [0,4194304] 0 2024-05-13T22:48:23.901 INFO:tasks.workunit.client.1.smithi130.stdout:9/105: stat d1/f8 0 2024-05-13T22:48:23.902 INFO:tasks.workunit.client.0.smithi102.stdout:6/946: mknod d9/db/d14/dcc/d5f/c131 0 2024-05-13T22:48:23.905 INFO:tasks.workunit.client.1.smithi130.stdout:4/92: rmdir d8/df 39 2024-05-13T22:48:23.905 INFO:tasks.workunit.client.0.smithi102.stdout:6/947: read d9/db/f77 [4357150,61214] 0 2024-05-13T22:48:23.906 INFO:tasks.workunit.client.0.smithi102.stdout:0/853: symlink d1/d68/l109 0 2024-05-13T22:48:23.908 INFO:tasks.workunit.client.0.smithi102.stdout:8/842: dwrite d2/f10b [0,4194304] 0 2024-05-13T22:48:23.912 INFO:tasks.workunit.client.0.smithi102.stdout:7/812: rename d6/d15/d4d/d55/d6f/l42 to d6/lf9 0 2024-05-13T22:48:23.915 INFO:tasks.workunit.client.0.smithi102.stdout:0/854: dread d1/de/faa [0,4194304] 0 2024-05-13T22:48:23.917 INFO:tasks.workunit.client.0.smithi102.stdout:2/942: dread d7/d9/d14/f62 [0,4194304] 0 2024-05-13T22:48:23.920 INFO:tasks.workunit.client.1.smithi130.stdout:2/79: rename d4/ff to d4/db/dc/f24 0 2024-05-13T22:48:23.921 INFO:tasks.workunit.client.0.smithi102.stdout:6/948: fsync d9/d25/d71/fb3 0 2024-05-13T22:48:23.922 INFO:tasks.workunit.client.0.smithi102.stdout:6/949: chown d9/d25/d5a/d113 0 1 2024-05-13T22:48:23.923 INFO:tasks.workunit.client.0.smithi102.stdout:8/843: truncate d2/d7/d27/d64/d96/dae/fe1 335598 0 2024-05-13T22:48:23.923 INFO:tasks.workunit.client.0.smithi102.stdout:6/950: write d9/d45/d56/dd5/fbd [3536691,127043] 0 2024-05-13T22:48:23.929 INFO:tasks.workunit.client.1.smithi130.stdout:2/80: dwrite d4/db/f17 [0,4194304] 0 2024-05-13T22:48:23.929 INFO:tasks.workunit.client.0.smithi102.stdout:0/855: chown d1/de/d32/f72 911050055 1 2024-05-13T22:48:23.930 INFO:tasks.workunit.client.0.smithi102.stdout:8/844: dwrite d2/d36/fde [0,4194304] 0 2024-05-13T22:48:23.938 INFO:tasks.workunit.client.0.smithi102.stdout:4/886: dwrite d3/d19/d23/d4e/d68/d85/fd0 [0,4194304] 0 2024-05-13T22:48:23.939 INFO:tasks.workunit.client.1.smithi130.stdout:3/94: read d8/dc/d19/f1e [248028,97098] 0 2024-05-13T22:48:23.944 INFO:tasks.workunit.client.0.smithi102.stdout:6/951: dread - d9/d45/d56/d5d/ffc zero size 2024-05-13T22:48:23.946 INFO:tasks.workunit.client.1.smithi130.stdout:1/92: mknod db/c1e 0 2024-05-13T22:48:23.948 INFO:tasks.workunit.client.1.smithi130.stdout:8/94: rmdir d1/d2/db/d15/d18/d1a 39 2024-05-13T22:48:23.949 INFO:tasks.workunit.client.0.smithi102.stdout:2/943: write d7/d9/d14/d31/f77 [2657289,114444] 0 2024-05-13T22:48:23.949 INFO:tasks.workunit.client.1.smithi130.stdout:8/95: chown d1/l14 18104 1 2024-05-13T22:48:23.950 INFO:tasks.workunit.client.1.smithi130.stdout:8/96: stat d1/d2/le 0 2024-05-13T22:48:23.953 INFO:tasks.workunit.client.1.smithi130.stdout:7/66: symlink d7/d8/d9/l16 0 2024-05-13T22:48:23.955 INFO:tasks.workunit.client.1.smithi130.stdout:6/130: fsync d4/d15/d1f/f20 0 2024-05-13T22:48:23.959 INFO:tasks.workunit.client.0.smithi102.stdout:6/952: chown d9/db/d14/d35/d47/cf0 1981902 1 2024-05-13T22:48:23.961 INFO:tasks.workunit.client.1.smithi130.stdout:2/81: creat d4/db/dc/f25 x:0 0 0 2024-05-13T22:48:23.961 INFO:tasks.workunit.client.0.smithi102.stdout:2/944: rmdir d7/d9/d10/d30 39 2024-05-13T22:48:23.961 INFO:tasks.workunit.client.1.smithi130.stdout:2/82: read - d4/db/dc/d18/f23 zero size 2024-05-13T22:48:23.962 INFO:tasks.workunit.client.0.smithi102.stdout:8/845: dread d2/f32 [0,4194304] 0 2024-05-13T22:48:23.963 INFO:tasks.workunit.client.1.smithi130.stdout:5/81: getdents d6/d7 0 2024-05-13T22:48:23.963 INFO:tasks.workunit.client.0.smithi102.stdout:2/945: truncate d7/d9/d10/d1b/d54/f12d 996002 0 2024-05-13T22:48:23.963 INFO:tasks.workunit.client.1.smithi130.stdout:5/82: write d6/f11 [1432923,33274] 0 2024-05-13T22:48:23.964 INFO:tasks.workunit.client.0.smithi102.stdout:4/887: truncate d3/d19/d23/d3a/f9d 344880 0 2024-05-13T22:48:23.964 INFO:tasks.workunit.client.0.smithi102.stdout:8/846: chown d2/d7/d27/dad/lc2 939680 1 2024-05-13T22:48:23.966 INFO:tasks.workunit.client.1.smithi130.stdout:1/93: mkdir db/d1b/d1f 0 2024-05-13T22:48:23.967 INFO:tasks.workunit.client.1.smithi130.stdout:1/94: read db/f1a [1298462,119911] 0 2024-05-13T22:48:23.967 INFO:tasks.workunit.client.1.smithi130.stdout:1/95: stat c2 0 2024-05-13T22:48:23.967 INFO:tasks.workunit.client.1.smithi130.stdout:8/97: symlink d1/d2/db/l23 0 2024-05-13T22:48:23.968 INFO:tasks.workunit.client.0.smithi102.stdout:2/946: rmdir d7/d15 39 2024-05-13T22:48:23.968 INFO:tasks.workunit.client.1.smithi130.stdout:1/96: stat db/f14 0 2024-05-13T22:48:23.970 INFO:tasks.workunit.client.0.smithi102.stdout:0/856: write d1/de/d32/dab/fb4 [4259590,59227] 0 2024-05-13T22:48:23.970 INFO:tasks.workunit.client.1.smithi130.stdout:1/97: write db/f16 [499130,17615] 0 2024-05-13T22:48:23.970 INFO:tasks.workunit.client.1.smithi130.stdout:7/67: mknod d7/da/c17 0 2024-05-13T22:48:23.972 INFO:tasks.workunit.client.0.smithi102.stdout:8/847: truncate d2/d7/f69 3721147 0 2024-05-13T22:48:23.973 INFO:tasks.workunit.client.0.smithi102.stdout:8/848: chown d2/d36/d49/d7e/de2/fec 99066 1 2024-05-13T22:48:23.973 INFO:tasks.workunit.client.0.smithi102.stdout:8/849: stat d2/d7/d27/d64/d96/dae/c1f 0 2024-05-13T22:48:23.974 INFO:tasks.workunit.client.1.smithi130.stdout:4/93: rename d8/f18 to d8/df/f21 0 2024-05-13T22:48:23.976 INFO:tasks.workunit.client.0.smithi102.stdout:7/813: link d6/d15/d4d/f9f d6/dd0/df2/ffa 0 2024-05-13T22:48:23.977 INFO:tasks.workunit.client.1.smithi130.stdout:0/116: creat d0/d7/f28 x:0 0 0 2024-05-13T22:48:23.978 INFO:tasks.workunit.client.0.smithi102.stdout:2/947: readlink d7/d9/d14/d61/l116 0 2024-05-13T22:48:23.979 INFO:tasks.workunit.client.0.smithi102.stdout:6/953: write d9/db/d14/dcc/d5f/fdb [2470893,4829] 0 2024-05-13T22:48:23.980 INFO:tasks.workunit.client.1.smithi130.stdout:2/83: mkdir d4/db/d26 0 2024-05-13T22:48:23.986 INFO:tasks.workunit.client.1.smithi130.stdout:9/106: link d1/f4 d1/d15/f1d 0 2024-05-13T22:48:23.988 INFO:tasks.workunit.client.0.smithi102.stdout:7/814: unlink d6/d15/d4c/fd7 0 2024-05-13T22:48:23.990 INFO:tasks.workunit.client.1.smithi130.stdout:8/98: rename d1/dc/f12 to d1/dc/f24 0 2024-05-13T22:48:23.992 INFO:tasks.workunit.client.1.smithi130.stdout:1/98: creat db/d1b/f20 x:0 0 0 2024-05-13T22:48:23.992 INFO:tasks.workunit.client.1.smithi130.stdout:1/99: dread - db/f14 zero size 2024-05-13T22:48:23.993 INFO:tasks.workunit.client.0.smithi102.stdout:6/954: symlink d9/db/d100/l132 0 2024-05-13T22:48:23.994 INFO:tasks.workunit.client.1.smithi130.stdout:7/68: rename d7/da/l11 to d7/da/l18 0 2024-05-13T22:48:23.995 INFO:tasks.workunit.client.0.smithi102.stdout:4/888: write d3/d19/d23/d39/ddd/fe0 [705903,41333] 0 2024-05-13T22:48:23.996 INFO:tasks.workunit.client.0.smithi102.stdout:0/857: write d1/de/d32/d24/d2d/d38/f97 [1044722,105087] 0 2024-05-13T22:48:23.999 INFO:tasks.workunit.client.1.smithi130.stdout:5/83: rename d6/d7/d9 to d6/dd/d18 0 2024-05-13T22:48:24.000 INFO:tasks.workunit.client.1.smithi130.stdout:5/84: fsync f2 0 2024-05-13T22:48:24.000 INFO:tasks.workunit.client.1.smithi130.stdout:5/85: write f2 [4921704,28699] 0 2024-05-13T22:48:24.002 INFO:tasks.workunit.client.0.smithi102.stdout:8/850: dwrite d2/d7/d25/d41/f65 [4194304,4194304] 0 2024-05-13T22:48:24.004 INFO:tasks.workunit.client.0.smithi102.stdout:7/815: mknod d6/d15/d4c/cfb 0 2024-05-13T22:48:24.006 INFO:tasks.workunit.client.0.smithi102.stdout:2/948: chown d7/d15/d1e/d87/cf5 13 1 2024-05-13T22:48:24.007 INFO:tasks.workunit.client.1.smithi130.stdout:2/84: rmdir d4/db/dc/d18 39 2024-05-13T22:48:24.012 INFO:tasks.workunit.client.0.smithi102.stdout:2/949: dwrite d7/d15/d1e/f5c [0,4194304] 0 2024-05-13T22:48:24.012 INFO:tasks.workunit.client.0.smithi102.stdout:4/889: readlink d3/d19/dad/dc9/dd5/ldb 0 2024-05-13T22:48:24.017 INFO:tasks.workunit.client.1.smithi130.stdout:8/99: dread d1/d2/db/f8 [0,4194304] 0 2024-05-13T22:48:24.021 INFO:tasks.workunit.client.1.smithi130.stdout:4/94: mkdir d8/d22 0 2024-05-13T22:48:24.022 INFO:tasks.workunit.client.1.smithi130.stdout:8/100: dread d1/d2/f6 [0,4194304] 0 2024-05-13T22:48:24.023 INFO:tasks.workunit.client.0.smithi102.stdout:7/816: chown d6/d13/d2c/da1/dc1/fa7 3880 1 2024-05-13T22:48:24.024 INFO:tasks.workunit.client.1.smithi130.stdout:1/100: rename c2 to db/d1b/c21 0 2024-05-13T22:48:24.026 INFO:tasks.workunit.client.1.smithi130.stdout:5/86: read f4 [2993681,87908] 0 2024-05-13T22:48:24.029 INFO:tasks.workunit.client.1.smithi130.stdout:6/131: creat d4/f2e x:0 0 0 2024-05-13T22:48:24.030 INFO:tasks.workunit.client.0.smithi102.stdout:0/858: mkdir d1/de/d32/d24/d2d/d38/d3d/d10a 0 2024-05-13T22:48:24.031 INFO:tasks.workunit.client.1.smithi130.stdout:0/117: dread - d0/f27 zero size 2024-05-13T22:48:24.032 INFO:tasks.workunit.client.0.smithi102.stdout:0/859: truncate d1/de/d32/d24/d2d/d38/db7/feb 926386 0 2024-05-13T22:48:24.032 INFO:tasks.workunit.client.0.smithi102.stdout:2/950: fsync d7/d9/d14/d61/d82/ff0 0 2024-05-13T22:48:24.034 INFO:tasks.workunit.client.1.smithi130.stdout:9/107: dwrite d1/f8 [0,4194304] 0 2024-05-13T22:48:24.035 INFO:tasks.workunit.client.0.smithi102.stdout:4/890: creat d3/d19/d23/d4e/d68/d85/f139 x:0 0 0 2024-05-13T22:48:24.038 INFO:tasks.workunit.client.0.smithi102.stdout:6/955: write d9/d25/d71/f11f [3733280,103802] 0 2024-05-13T22:48:24.040 INFO:tasks.workunit.client.0.smithi102.stdout:8/851: dwrite d2/d7/d27/dbb/fcc [0,4194304] 0 2024-05-13T22:48:24.043 INFO:tasks.workunit.client.1.smithi130.stdout:7/69: read - d7/d8/d9/fc zero size 2024-05-13T22:48:24.045 INFO:tasks.workunit.client.1.smithi130.stdout:4/95: unlink d8/df/f21 0 2024-05-13T22:48:24.048 INFO:tasks.workunit.client.1.smithi130.stdout:8/101: creat d1/d2/f25 x:0 0 0 2024-05-13T22:48:24.048 INFO:tasks.workunit.client.0.smithi102.stdout:4/891: chown d3/d4/d6/c33 0 1 2024-05-13T22:48:24.048 INFO:tasks.workunit.client.0.smithi102.stdout:4/892: readlink d3/d19/l138 0 2024-05-13T22:48:24.049 INFO:tasks.workunit.client.0.smithi102.stdout:0/860: dread d1/fad [0,4194304] 0 2024-05-13T22:48:24.050 INFO:tasks.workunit.client.1.smithi130.stdout:1/101: rename db/f13 to db/f22 0 2024-05-13T22:48:24.050 INFO:tasks.workunit.client.1.smithi130.stdout:4/96: dread f5 [0,4194304] 0 2024-05-13T22:48:24.051 INFO:tasks.workunit.client.1.smithi130.stdout:5/87: write d6/dd/d18/fc [285614,3248] 0 2024-05-13T22:48:24.052 INFO:tasks.workunit.client.0.smithi102.stdout:2/951: dread d7/d9/d10/d30/d53/f8e [0,4194304] 0 2024-05-13T22:48:24.052 INFO:tasks.workunit.client.1.smithi130.stdout:6/132: creat d4/d15/d1f/f2f x:0 0 0 2024-05-13T22:48:24.052 INFO:tasks.workunit.client.1.smithi130.stdout:5/88: dread - d6/dd/ff zero size 2024-05-13T22:48:24.053 INFO:tasks.workunit.client.1.smithi130.stdout:6/133: stat d4/f12 0 2024-05-13T22:48:24.057 INFO:tasks.workunit.client.0.smithi102.stdout:4/893: fsync d3/d19/d23/d39/da7/d7d/fc5 0 2024-05-13T22:48:24.059 INFO:tasks.workunit.client.0.smithi102.stdout:0/861: creat d1/de/d32/dab/f10b x:0 0 0 2024-05-13T22:48:24.061 INFO:tasks.workunit.client.1.smithi130.stdout:3/95: getdents d8/dc/d19/d1d 0 2024-05-13T22:48:24.061 INFO:tasks.workunit.client.1.smithi130.stdout:1/102: dwrite db/f14 [0,4194304] 0 2024-05-13T22:48:24.061 INFO:tasks.workunit.client.1.smithi130.stdout:3/96: readlink d8/dc/l17 0 2024-05-13T22:48:24.062 INFO:tasks.workunit.client.0.smithi102.stdout:7/817: rename d6/d13/d62/l8f to d6/de1/lfc 0 2024-05-13T22:48:24.063 INFO:tasks.workunit.client.0.smithi102.stdout:7/818: chown d6/d13/d58/dd1/f12 386514230 1 2024-05-13T22:48:24.064 INFO:tasks.workunit.client.0.smithi102.stdout:6/956: mkdir d9/d45/ddf/d133 0 2024-05-13T22:48:24.068 INFO:tasks.workunit.client.1.smithi130.stdout:4/97: mknod d8/dc/d13/c23 0 2024-05-13T22:48:24.068 INFO:tasks.workunit.client.1.smithi130.stdout:4/98: stat d8/dc/fd 0 2024-05-13T22:48:24.070 INFO:tasks.workunit.client.1.smithi130.stdout:5/89: mkdir d6/d7/d19 0 2024-05-13T22:48:24.071 INFO:tasks.workunit.client.0.smithi102.stdout:8/852: dwrite d2/d7/d25/d41/f48 [0,4194304] 0 2024-05-13T22:48:24.072 INFO:tasks.workunit.client.1.smithi130.stdout:2/85: write d4/db/dc/f10 [1411971,45863] 0 2024-05-13T22:48:24.076 INFO:tasks.workunit.client.0.smithi102.stdout:0/862: mknod d1/de/d42/dea/d100/c10c 0 2024-05-13T22:48:24.076 INFO:tasks.workunit.client.0.smithi102.stdout:0/863: readlink d1/l2a 0 2024-05-13T22:48:24.076 INFO:tasks.workunit.client.0.smithi102.stdout:0/864: chown d1/d4d 283231 1 2024-05-13T22:48:24.077 INFO:tasks.workunit.client.0.smithi102.stdout:8/853: mkdir d2/d7/d25/d41/d6f/da0/d110 0 2024-05-13T22:48:24.078 INFO:tasks.workunit.client.1.smithi130.stdout:1/103: mknod db/d1b/c23 0 2024-05-13T22:48:24.080 INFO:tasks.workunit.client.0.smithi102.stdout:2/952: rename d7/d9/d10/d30/d53/da4/le4 to d7/d9/d10/d30/d53/l135 0 2024-05-13T22:48:24.080 INFO:tasks.workunit.client.0.smithi102.stdout:0/865: read d1/de/d32/d24/d2d/f64 [937617,115804] 0 2024-05-13T22:48:24.081 INFO:tasks.workunit.client.1.smithi130.stdout:5/90: dwrite f2 [0,4194304] 0 2024-05-13T22:48:24.085 INFO:tasks.workunit.client.0.smithi102.stdout:8/854: dwrite d2/d36/d49/d5d/f26 [0,4194304] 0 2024-05-13T22:48:24.085 INFO:tasks.workunit.client.0.smithi102.stdout:0/866: truncate d1/de/d32/d24/d2d/d38/d3d/ff5 5205794 0 2024-05-13T22:48:24.087 INFO:tasks.workunit.client.0.smithi102.stdout:2/953: mknod d7/d15/d1e/da7/db5/c136 0 2024-05-13T22:48:24.088 INFO:tasks.workunit.client.1.smithi130.stdout:3/97: creat d8/dc/f21 x:0 0 0 2024-05-13T22:48:24.089 INFO:tasks.workunit.client.0.smithi102.stdout:0/867: write d1/de/d32/d24/d2d/d38/db7/feb [1428556,49625] 0 2024-05-13T22:48:24.089 INFO:tasks.workunit.client.1.smithi130.stdout:2/86: dwrite d4/db/dc/f15 [0,4194304] 0 2024-05-13T22:48:24.094 INFO:tasks.workunit.client.1.smithi130.stdout:9/108: dwrite d1/d15/f1d [4194304,4194304] 0 2024-05-13T22:48:24.095 INFO:tasks.workunit.client.1.smithi130.stdout:6/134: creat d4/d15/d1d/d28/f30 x:0 0 0 2024-05-13T22:48:24.096 INFO:tasks.workunit.client.0.smithi102.stdout:8/855: dread d2/d7/d27/d64/d96/dae/f4f [0,4194304] 0 2024-05-13T22:48:24.096 INFO:tasks.workunit.client.1.smithi130.stdout:0/118: mkdir d0/d7/d29 0 2024-05-13T22:48:24.098 INFO:tasks.workunit.client.0.smithi102.stdout:4/894: dwrite d3/d4/d6/fb5 [0,4194304] 0 2024-05-13T22:48:24.099 INFO:tasks.workunit.client.0.smithi102.stdout:2/954: dwrite d7/d9/d11/f105 [0,4194304] 0 2024-05-13T22:48:24.099 INFO:tasks.workunit.client.0.smithi102.stdout:8/856: chown d2/d7/d27/d64/d96/dbc 9 1 2024-05-13T22:48:24.099 INFO:tasks.workunit.client.1.smithi130.stdout:2/87: dread d4/db/dc/f15 [0,4194304] 0 2024-05-13T22:48:24.100 INFO:tasks.workunit.client.1.smithi130.stdout:9/109: dread d1/f11 [0,4194304] 0 2024-05-13T22:48:24.123 INFO:tasks.workunit.client.0.smithi102.stdout:6/957: dread d9/db/f36 [4194304,4194304] 0 2024-05-13T22:48:24.130 INFO:tasks.workunit.client.0.smithi102.stdout:4/895: dread d3/d4/f93 [0,4194304] 0 2024-05-13T22:48:24.140 INFO:tasks.workunit.client.0.smithi102.stdout:7/819: rename d6/d15/d4d/d55/d82/c84 to d6/cfd 0 2024-05-13T22:48:24.143 INFO:tasks.workunit.client.0.smithi102.stdout:0/868: mkdir d1/d4d/d79/d9f/dbd/d10d 0 2024-05-13T22:48:24.145 INFO:tasks.workunit.client.1.smithi130.stdout:7/70: creat d7/d8/f19 x:0 0 0 2024-05-13T22:48:24.151 INFO:tasks.workunit.client.1.smithi130.stdout:4/99: link f3 d8/df/f24 0 2024-05-13T22:48:24.152 INFO:tasks.workunit.client.0.smithi102.stdout:2/955: truncate d7/d9/d14/f62 2083360 0 2024-05-13T22:48:24.153 INFO:tasks.workunit.client.0.smithi102.stdout:2/956: readlink d7/d9/d14/d31/dbb/le3 0 2024-05-13T22:48:24.154 INFO:tasks.workunit.client.0.smithi102.stdout:2/957: truncate d7/d9/d10/d30/d53/f114 395260 0 2024-05-13T22:48:24.155 INFO:tasks.workunit.client.0.smithi102.stdout:2/958: stat d7/d15/d1e/d87/lec 0 2024-05-13T22:48:24.156 INFO:tasks.workunit.client.1.smithi130.stdout:6/135: creat d4/d15/f31 x:0 0 0 2024-05-13T22:48:24.161 INFO:tasks.workunit.client.1.smithi130.stdout:2/88: rename d4/db/f17 to d4/db/f27 0 2024-05-13T22:48:24.162 INFO:tasks.workunit.client.1.smithi130.stdout:2/89: stat d4/db/dc 0 2024-05-13T22:48:24.166 INFO:tasks.workunit.client.0.smithi102.stdout:2/959: truncate d7/d15/d67/d129/f35 3358605 0 2024-05-13T22:48:24.167 INFO:tasks.workunit.client.1.smithi130.stdout:5/91: write f4 [2065987,94146] 0 2024-05-13T22:48:24.168 INFO:tasks.workunit.client.0.smithi102.stdout:6/958: mknod d9/d25/d60/d6a/dd6/d114/c134 0 2024-05-13T22:48:24.168 INFO:tasks.workunit.client.1.smithi130.stdout:2/90: dread d4/db/dc/f20 [0,4194304] 0 2024-05-13T22:48:24.169 INFO:tasks.workunit.client.0.smithi102.stdout:6/959: chown d9/db/d14/f10e 8166473 1 2024-05-13T22:48:24.170 INFO:tasks.workunit.client.0.smithi102.stdout:7/820: mkdir d6/d15/d4c/dfe 0 2024-05-13T22:48:24.174 INFO:tasks.workunit.client.0.smithi102.stdout:8/857: creat d2/d7/d27/d64/d96/dae/f111 x:0 0 0 2024-05-13T22:48:24.175 INFO:tasks.workunit.client.0.smithi102.stdout:6/960: dread d9/d25/d71/f7a [0,4194304] 0 2024-05-13T22:48:24.178 INFO:tasks.workunit.client.0.smithi102.stdout:2/960: rename d7/d9/d14/d31/dd0/f12e to d7/d9/d14/d31/dbb/f137 0 2024-05-13T22:48:24.179 INFO:tasks.workunit.client.1.smithi130.stdout:6/136: mkdir d4/d15/d1d/d32 0 2024-05-13T22:48:24.179 INFO:tasks.workunit.client.1.smithi130.stdout:6/137: dread - d4/f2e zero size 2024-05-13T22:48:24.180 INFO:tasks.workunit.client.1.smithi130.stdout:6/138: dread - d4/f2e zero size 2024-05-13T22:48:24.180 INFO:tasks.workunit.client.1.smithi130.stdout:0/119: getdents d0/d7/d24 0 2024-05-13T22:48:24.181 INFO:tasks.workunit.client.0.smithi102.stdout:6/961: dwrite d9/db/f29 [4194304,4194304] 0 2024-05-13T22:48:24.184 INFO:tasks.workunit.client.1.smithi130.stdout:1/104: truncate db/f14 619454 0 2024-05-13T22:48:24.185 INFO:tasks.workunit.client.0.smithi102.stdout:0/869: creat d1/de/d32/d24/d2d/d38/d3d/f10e x:0 0 0 2024-05-13T22:48:24.189 INFO:tasks.workunit.client.1.smithi130.stdout:1/105: dread db/f17 [0,4194304] 0 2024-05-13T22:48:24.191 INFO:tasks.workunit.client.0.smithi102.stdout:2/961: rmdir d7/d15/d67/d129/d56 39 2024-05-13T22:48:24.192 INFO:tasks.workunit.client.1.smithi130.stdout:1/106: dread db/f16 [0,4194304] 0 2024-05-13T22:48:24.192 INFO:tasks.workunit.client.1.smithi130.stdout:1/107: stat db/d1b/l1c 0 2024-05-13T22:48:24.193 INFO:tasks.workunit.client.0.smithi102.stdout:2/962: fsync d7/d9/d10/d1b/d54/fcb 0 2024-05-13T22:48:24.193 INFO:tasks.workunit.client.1.smithi130.stdout:7/71: mknod d7/d8/d9/c1a 0 2024-05-13T22:48:24.193 INFO:tasks.workunit.client.1.smithi130.stdout:1/108: truncate db/f16 939824 0 2024-05-13T22:48:24.196 INFO:tasks.workunit.client.0.smithi102.stdout:6/962: truncate d9/f10 2273588 0 2024-05-13T22:48:24.200 INFO:tasks.workunit.client.0.smithi102.stdout:0/870: chown d1/c14 70 1 2024-05-13T22:48:24.202 INFO:tasks.workunit.client.1.smithi130.stdout:9/110: rename d1/d7/f19 to d1/f1e 0 2024-05-13T22:48:24.209 INFO:tasks.workunit.client.0.smithi102.stdout:6/963: dread d9/d25/d71/f11f [0,4194304] 0 2024-05-13T22:48:24.210 INFO:tasks.workunit.client.1.smithi130.stdout:5/92: mkdir d6/d1a 0 2024-05-13T22:48:24.215 INFO:tasks.workunit.client.1.smithi130.stdout:2/91: link d4/db/dc/f25 d4/db/f28 0 2024-05-13T22:48:24.215 INFO:tasks.workunit.client.0.smithi102.stdout:8/858: dwrite d2/d36/d49/d5d/fe3 [0,4194304] 0 2024-05-13T22:48:24.216 INFO:tasks.workunit.client.0.smithi102.stdout:8/859: readlink d2/d7/d25/d41/d6f/da0/ldf 0 2024-05-13T22:48:24.217 INFO:tasks.workunit.client.1.smithi130.stdout:4/100: rmdir d8/d14 0 2024-05-13T22:48:24.219 INFO:tasks.workunit.client.0.smithi102.stdout:2/963: symlink d7/l138 0 2024-05-13T22:48:24.221 INFO:tasks.workunit.client.0.smithi102.stdout:6/964: mkdir d9/d25/d60/d6a/dd6/de9/df7/d135 0 2024-05-13T22:48:24.225 INFO:tasks.workunit.client.0.smithi102.stdout:7/821: rename d6/d13/d58/dd1/fab to d6/fff 0 2024-05-13T22:48:24.227 INFO:tasks.workunit.client.1.smithi130.stdout:5/93: creat d6/dd/f1b x:0 0 0 2024-05-13T22:48:24.228 INFO:tasks.workunit.client.0.smithi102.stdout:0/871: getdents d1/de/d32/d24/d2d/d91 0 2024-05-13T22:48:24.229 INFO:tasks.workunit.client.1.smithi130.stdout:5/94: read d6/dd/f13 [295766,44399] 0 2024-05-13T22:48:24.229 INFO:tasks.workunit.client.1.smithi130.stdout:7/72: symlink d7/d8/l1b 0 2024-05-13T22:48:24.230 INFO:tasks.workunit.client.1.smithi130.stdout:5/95: write d6/dd/f1b [656928,53797] 0 2024-05-13T22:48:24.230 INFO:tasks.workunit.client.1.smithi130.stdout:7/73: write d7/d8/f19 [212180,75001] 0 2024-05-13T22:48:24.230 INFO:tasks.workunit.client.1.smithi130.stdout:1/109: symlink db/d1b/d1f/l24 0 2024-05-13T22:48:24.232 INFO:tasks.workunit.client.1.smithi130.stdout:7/74: write d7/d8/d9/fc [466311,32324] 0 2024-05-13T22:48:24.233 INFO:tasks.workunit.client.1.smithi130.stdout:4/101: truncate f5 3161155 0 2024-05-13T22:48:24.235 INFO:tasks.workunit.client.0.smithi102.stdout:0/872: creat d1/de/d32/d63/f10f x:0 0 0 2024-05-13T22:48:24.237 INFO:tasks.workunit.client.1.smithi130.stdout:9/111: mknod d1/c1f 0 2024-05-13T22:48:24.240 INFO:tasks.workunit.client.1.smithi130.stdout:1/110: fdatasync db/f17 0 2024-05-13T22:48:24.241 INFO:tasks.workunit.client.0.smithi102.stdout:6/965: creat d9/db/d14/dcc/f136 x:0 0 0 2024-05-13T22:48:24.242 INFO:tasks.workunit.client.0.smithi102.stdout:6/966: chown d9/db/d14/d35/dc9 21 1 2024-05-13T22:48:24.242 INFO:tasks.workunit.client.1.smithi130.stdout:5/96: readlink d6/dd/d18/l14 0 2024-05-13T22:48:24.245 INFO:tasks.workunit.client.0.smithi102.stdout:0/873: rename d1/f5 to d1/de/d32/d63/f110 0 2024-05-13T22:48:24.246 INFO:tasks.workunit.client.1.smithi130.stdout:2/92: creat d4/db/dc/d18/f29 x:0 0 0 2024-05-13T22:48:24.247 INFO:tasks.workunit.client.1.smithi130.stdout:2/93: dread - d4/fd zero size 2024-05-13T22:48:24.248 INFO:tasks.workunit.client.0.smithi102.stdout:6/967: symlink d9/d25/d60/l137 0 2024-05-13T22:48:24.249 INFO:tasks.workunit.client.1.smithi130.stdout:4/102: symlink d8/dc/d13/l25 0 2024-05-13T22:48:24.250 INFO:tasks.workunit.client.0.smithi102.stdout:0/874: symlink d1/de/d42/dea/l111 0 2024-05-13T22:48:24.252 INFO:tasks.workunit.client.0.smithi102.stdout:6/968: mkdir d9/d25/d60/d6a/dd6/d138 0 2024-05-13T22:48:24.254 INFO:tasks.workunit.client.1.smithi130.stdout:9/112: mknod d1/d7/c20 0 2024-05-13T22:48:24.254 INFO:tasks.workunit.client.1.smithi130.stdout:9/113: dread - d1/f1e zero size 2024-05-13T22:48:24.255 INFO:tasks.workunit.client.0.smithi102.stdout:8/860: dwrite d2/d36/d49/d5d/f45 [0,4194304] 0 2024-05-13T22:48:24.258 INFO:tasks.workunit.client.0.smithi102.stdout:6/969: creat d9/d25/d60/d6a/dd6/de9/f139 x:0 0 0 2024-05-13T22:48:24.260 INFO:tasks.workunit.client.1.smithi130.stdout:9/114: dwrite d1/d7/f1a [0,4194304] 0 2024-05-13T22:48:24.265 INFO:tasks.workunit.client.0.smithi102.stdout:7/822: dwrite d6/d15/d4d/fa4 [0,4194304] 0 2024-05-13T22:48:24.265 INFO:tasks.workunit.client.0.smithi102.stdout:8/861: dread d2/d7/d27/d64/d96/dae/fe1 [0,4194304] 0 2024-05-13T22:48:24.265 INFO:tasks.workunit.client.1.smithi130.stdout:2/94: mkdir d4/db/dc/d18/d2a 0 2024-05-13T22:48:24.266 INFO:tasks.workunit.client.1.smithi130.stdout:1/111: dwrite db/f1a [0,4194304] 0 2024-05-13T22:48:24.271 INFO:tasks.workunit.client.1.smithi130.stdout:6/139: rename d4/d15/c18 to d4/c33 0 2024-05-13T22:48:24.273 INFO:tasks.workunit.client.1.smithi130.stdout:6/140: write d4/d15/d1d/d28/f30 [844068,89638] 0 2024-05-13T22:48:24.275 INFO:tasks.workunit.client.1.smithi130.stdout:6/141: write d4/d15/d1f/f2f [951745,51895] 0 2024-05-13T22:48:24.276 INFO:tasks.workunit.client.0.smithi102.stdout:6/970: link d9/d45/ddf/fea d9/d25/d60/f13a 0 2024-05-13T22:48:24.277 INFO:tasks.workunit.client.1.smithi130.stdout:5/97: creat d6/d7/d19/f1c x:0 0 0 2024-05-13T22:48:24.278 INFO:tasks.workunit.client.0.smithi102.stdout:7/823: creat d6/d15/d4d/d55/d6f/d49/f100 x:0 0 0 2024-05-13T22:48:24.281 INFO:tasks.workunit.client.0.smithi102.stdout:0/875: getdents d1/de/d32/d24 0 2024-05-13T22:48:24.282 INFO:tasks.workunit.client.1.smithi130.stdout:7/75: creat d7/d8/f1c x:0 0 0 2024-05-13T22:48:24.283 INFO:tasks.workunit.client.0.smithi102.stdout:6/971: rename d9/db/d14/d35/dc9/fd2 to d9/da3/f13b 0 2024-05-13T22:48:24.284 INFO:tasks.workunit.client.1.smithi130.stdout:6/142: dwrite d4/d15/d1f/f2f [0,4194304] 0 2024-05-13T22:48:24.285 INFO:tasks.workunit.client.0.smithi102.stdout:7/824: write d6/d15/f3e [1564963,80783] 0 2024-05-13T22:48:24.285 INFO:tasks.workunit.client.1.smithi130.stdout:1/112: write db/f1d [1140182,112773] 0 2024-05-13T22:48:24.285 INFO:tasks.workunit.client.1.smithi130.stdout:9/115: dread d1/f11 [0,4194304] 0 2024-05-13T22:48:24.286 INFO:tasks.workunit.client.0.smithi102.stdout:9/882: sync 2024-05-13T22:48:24.286 INFO:tasks.workunit.client.1.smithi130.stdout:1/113: chown db/f16 7 1 2024-05-13T22:48:24.287 INFO:tasks.workunit.client.0.smithi102.stdout:9/883: write d1/d6b/f10b [1659734,94268] 0 2024-05-13T22:48:24.288 INFO:tasks.workunit.client.1.smithi130.stdout:1/114: chown db/cf 2313 1 2024-05-13T22:48:24.288 INFO:tasks.workunit.client.0.smithi102.stdout:6/972: dread d9/d45/da0/dd9/fdc [0,4194304] 0 2024-05-13T22:48:24.289 INFO:tasks.workunit.client.0.smithi102.stdout:9/884: fdatasync d1/d6/db/d30/f112 0 2024-05-13T22:48:24.289 INFO:tasks.workunit.client.1.smithi130.stdout:5/98: mknod d6/dd/c1d 0 2024-05-13T22:48:24.291 INFO:tasks.workunit.client.1.smithi130.stdout:2/95: symlink d4/db/dc/d18/d2a/l2b 0 2024-05-13T22:48:24.292 INFO:tasks.workunit.client.1.smithi130.stdout:7/76: dread d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:24.292 INFO:tasks.workunit.client.1.smithi130.stdout:6/143: creat d4/d15/d1d/d28/f34 x:0 0 0 2024-05-13T22:48:24.294 INFO:tasks.workunit.client.1.smithi130.stdout:0/120: rename d0/c13 to d0/d7/dd/d17/c2a 0 2024-05-13T22:48:24.294 INFO:tasks.workunit.client.1.smithi130.stdout:1/115: truncate db/f16 772680 0 2024-05-13T22:48:24.295 INFO:tasks.workunit.client.1.smithi130.stdout:2/96: creat d4/db/f2c x:0 0 0 2024-05-13T22:48:24.297 INFO:tasks.workunit.client.1.smithi130.stdout:7/77: mkdir d7/da/d1d 0 2024-05-13T22:48:24.298 INFO:tasks.workunit.client.1.smithi130.stdout:0/121: symlink d0/d7/dd/l2b 0 2024-05-13T22:48:24.299 INFO:tasks.workunit.client.1.smithi130.stdout:9/116: dwrite d1/f8 [0,4194304] 0 2024-05-13T22:48:24.301 INFO:tasks.workunit.client.1.smithi130.stdout:0/122: dread - d0/d7/f16 zero size 2024-05-13T22:48:24.301 INFO:tasks.workunit.client.1.smithi130.stdout:2/97: chown d4/db/dc/d18/f29 2253 1 2024-05-13T22:48:24.304 INFO:tasks.workunit.client.1.smithi130.stdout:9/117: mknod d1/c21 0 2024-05-13T22:48:24.304 INFO:tasks.workunit.client.1.smithi130.stdout:5/99: dwrite f3 [0,4194304] 0 2024-05-13T22:48:24.304 INFO:tasks.workunit.client.1.smithi130.stdout:0/123: truncate d0/d7/f28 952971 0 2024-05-13T22:48:24.304 INFO:tasks.workunit.client.1.smithi130.stdout:9/118: chown d1 0 1 2024-05-13T22:48:24.306 INFO:tasks.workunit.client.1.smithi130.stdout:5/100: write f3 [1819143,7217] 0 2024-05-13T22:48:24.308 INFO:tasks.workunit.client.1.smithi130.stdout:2/98: link d4/f8 d4/db/d26/f2d 0 2024-05-13T22:48:24.309 INFO:tasks.workunit.client.1.smithi130.stdout:9/119: rmdir d1/d15 39 2024-05-13T22:48:24.311 INFO:tasks.workunit.client.1.smithi130.stdout:0/124: rename d0/d7/d24 to d0/d7/dd/d17/d1f/d2c 0 2024-05-13T22:48:24.311 INFO:tasks.workunit.client.1.smithi130.stdout:5/101: mknod d6/c1e 0 2024-05-13T22:48:24.312 INFO:tasks.workunit.client.1.smithi130.stdout:7/78: dwrite d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:24.313 INFO:tasks.workunit.client.1.smithi130.stdout:2/99: creat d4/db/f2e x:0 0 0 2024-05-13T22:48:24.314 INFO:tasks.workunit.client.1.smithi130.stdout:0/125: chown d0/d7/dd/d17/f1b 454 1 2024-05-13T22:48:24.316 INFO:tasks.workunit.client.1.smithi130.stdout:5/102: write d6/dd/f13 [115555,61508] 0 2024-05-13T22:48:24.318 INFO:tasks.workunit.client.1.smithi130.stdout:2/100: truncate d4/fa 2485028 0 2024-05-13T22:48:24.319 INFO:tasks.workunit.client.1.smithi130.stdout:7/79: rename c3 to d7/c1e 0 2024-05-13T22:48:24.319 INFO:tasks.workunit.client.1.smithi130.stdout:5/103: write f4 [3495139,122112] 0 2024-05-13T22:48:24.321 INFO:tasks.workunit.client.1.smithi130.stdout:0/126: rename d0/d7/dd/d17/d1f/l26 to d0/d7/dd/d17/d1f/d2c/l2d 0 2024-05-13T22:48:24.322 INFO:tasks.workunit.client.1.smithi130.stdout:2/101: write d4/db/dc/d18/f29 [775290,84484] 0 2024-05-13T22:48:24.323 INFO:tasks.workunit.client.1.smithi130.stdout:2/102: fdatasync d4/db/f22 0 2024-05-13T22:48:24.325 INFO:tasks.workunit.client.1.smithi130.stdout:7/80: mkdir d7/da/d1f 0 2024-05-13T22:48:24.325 INFO:tasks.workunit.client.0.smithi102.stdout:7/825: creat d6/d13/d2c/da1/f101 x:0 0 0 2024-05-13T22:48:24.329 INFO:tasks.workunit.client.1.smithi130.stdout:2/103: creat d4/db/dc/d18/d2a/f2f x:0 0 0 2024-05-13T22:48:24.331 INFO:tasks.workunit.client.0.smithi102.stdout:6/973: creat d9/d25/d60/d6a/dd6/dff/f13c x:0 0 0 2024-05-13T22:48:24.332 INFO:tasks.workunit.client.1.smithi130.stdout:5/104: dread d6/dd/f1b [0,4194304] 0 2024-05-13T22:48:24.334 INFO:tasks.workunit.client.0.smithi102.stdout:7/826: unlink d6/d15/d4d/d55/f74 0 2024-05-13T22:48:24.341 INFO:tasks.workunit.client.0.smithi102.stdout:7/827: chown d6/dd0/d80/ff6 68117741 1 2024-05-13T22:48:24.342 INFO:tasks.workunit.client.1.smithi130.stdout:3/98: sync 2024-05-13T22:48:24.343 INFO:tasks.workunit.client.1.smithi130.stdout:8/102: sync 2024-05-13T22:48:24.343 INFO:tasks.workunit.client.0.smithi102.stdout:6/974: creat d9/db/d14/d35/d47/dc1/d126/d12f/f13d x:0 0 0 2024-05-13T22:48:24.343 INFO:tasks.workunit.client.1.smithi130.stdout:7/81: creat d7/da/d1f/f20 x:0 0 0 2024-05-13T22:48:24.344 INFO:tasks.workunit.client.0.smithi102.stdout:8/862: write d2/d7/d27/d64/d80/dc8/f39 [3982837,101034] 0 2024-05-13T22:48:24.346 INFO:tasks.workunit.client.0.smithi102.stdout:7/828: chown d6/f2e 8 1 2024-05-13T22:48:24.349 INFO:tasks.workunit.client.1.smithi130.stdout:3/99: dread d8/f9 [0,4194304] 0 2024-05-13T22:48:24.350 INFO:tasks.workunit.client.1.smithi130.stdout:3/100: write d8/fa [413715,124204] 0 2024-05-13T22:48:24.351 INFO:tasks.workunit.client.1.smithi130.stdout:3/101: dread - d8/dc/f21 zero size 2024-05-13T22:48:24.351 INFO:tasks.workunit.client.0.smithi102.stdout:6/975: truncate d9/db/d14/d35/d47/dc1/fc6 1452641 0 2024-05-13T22:48:24.351 INFO:tasks.workunit.client.1.smithi130.stdout:3/102: chown d8/d1a 39 1 2024-05-13T22:48:24.354 INFO:tasks.workunit.client.0.smithi102.stdout:8/863: fsync d2/d7/d25/fb0 0 2024-05-13T22:48:24.356 INFO:tasks.workunit.client.1.smithi130.stdout:6/144: rmdir d4 39 2024-05-13T22:48:24.357 INFO:tasks.workunit.client.0.smithi102.stdout:6/976: symlink d9/da3/de8/l13e 0 2024-05-13T22:48:24.359 INFO:tasks.workunit.client.0.smithi102.stdout:7/829: mkdir d6/d15/d4d/d55/d6f/d102 0 2024-05-13T22:48:24.364 INFO:tasks.workunit.client.0.smithi102.stdout:6/977: dread - d9/d25/f88 zero size 2024-05-13T22:48:24.365 INFO:tasks.workunit.client.0.smithi102.stdout:6/978: write d9/db/f29 [562865,8751] 0 2024-05-13T22:48:24.367 INFO:tasks.workunit.client.0.smithi102.stdout:7/830: unlink d6/d15/d4d/d55/d6f/d65/fc8 0 2024-05-13T22:48:24.368 INFO:tasks.workunit.client.1.smithi130.stdout:1/116: dwrite db/f14 [0,4194304] 0 2024-05-13T22:48:24.369 INFO:tasks.workunit.client.1.smithi130.stdout:1/117: chown db/f14 2 1 2024-05-13T22:48:24.376 INFO:tasks.workunit.client.1.smithi130.stdout:9/120: truncate d1/d7/f1a 3012190 0 2024-05-13T22:48:24.376 INFO:tasks.workunit.client.0.smithi102.stdout:6/979: fdatasync d9/db/d14/d35/dc9/df1/f62 0 2024-05-13T22:48:24.377 INFO:tasks.workunit.client.1.smithi130.stdout:1/118: dwrite db/d1b/f20 [0,4194304] 0 2024-05-13T22:48:24.379 INFO:tasks.workunit.client.0.smithi102.stdout:7/831: mkdir d6/d13/d2c/da1/d103 0 2024-05-13T22:48:24.381 INFO:tasks.workunit.client.0.smithi102.stdout:8/864: creat d2/d7/d27/d64/d96/dae/f112 x:0 0 0 2024-05-13T22:48:24.382 INFO:tasks.workunit.client.0.smithi102.stdout:8/865: chown d2/d36/d49/d7e/c10e 836870 1 2024-05-13T22:48:24.382 INFO:tasks.workunit.client.1.smithi130.stdout:1/119: dread db/f1a [0,4194304] 0 2024-05-13T22:48:24.383 INFO:tasks.workunit.client.1.smithi130.stdout:1/120: read db/f14 [1327119,6886] 0 2024-05-13T22:48:24.385 INFO:tasks.workunit.client.1.smithi130.stdout:0/127: write d0/f21 [1332086,121933] 0 2024-05-13T22:48:24.387 INFO:tasks.workunit.client.0.smithi102.stdout:7/832: symlink d6/d13/d2c/da1/l104 0 2024-05-13T22:48:24.389 INFO:tasks.workunit.client.0.smithi102.stdout:8/866: creat d2/d7/d27/d64/d96/dae/d78/f113 x:0 0 0 2024-05-13T22:48:24.390 INFO:tasks.workunit.client.0.smithi102.stdout:6/980: dread d9/d25/d71/fb7 [0,4194304] 0 2024-05-13T22:48:24.391 INFO:tasks.workunit.client.0.smithi102.stdout:7/833: fdatasync d6/d15/d4d/d55/d6f/d49/f4e 0 2024-05-13T22:48:24.391 INFO:tasks.workunit.client.0.smithi102.stdout:6/981: chown d9/d45/d56/dd5/d12c 341651862 1 2024-05-13T22:48:24.395 INFO:tasks.workunit.client.0.smithi102.stdout:6/982: dread d9/db/fa1 [0,4194304] 0 2024-05-13T22:48:24.399 INFO:tasks.workunit.client.0.smithi102.stdout:6/983: symlink d9/db/d14/d35/d47/dc1/l13f 0 2024-05-13T22:48:24.402 INFO:tasks.workunit.client.1.smithi130.stdout:4/103: sync 2024-05-13T22:48:24.402 INFO:tasks.workunit.client.0.smithi102.stdout:6/984: unlink d9/d25/d5a/cbb 0 2024-05-13T22:48:24.409 INFO:tasks.workunit.client.0.smithi102.stdout:7/834: dread d6/dd0/d80/fa5 [0,4194304] 0 2024-05-13T22:48:24.410 INFO:tasks.workunit.client.0.smithi102.stdout:6/985: creat d9/db/d14/d35/f140 x:0 0 0 2024-05-13T22:48:24.416 INFO:tasks.workunit.client.0.smithi102.stdout:6/986: mknod d9/d45/d56/c141 0 2024-05-13T22:48:24.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:24 smithi102 ceph-mon[28599]: pgmap v29: 65 pgs: 65 active+clean; 2.0 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 60 MiB/s rd, 159 MiB/s wr, 487 op/s 2024-05-13T22:48:24.433 INFO:tasks.workunit.client.0.smithi102.stdout:6/987: dread d9/db/d14/dcc/f43 [0,4194304] 0 2024-05-13T22:48:24.444 INFO:tasks.workunit.client.0.smithi102.stdout:6/988: getdents d9/d45/d56/d5d 0 2024-05-13T22:48:24.449 INFO:tasks.workunit.client.0.smithi102.stdout:7/835: dwrite d6/d13/d2c/da1/dc1/fa7 [0,4194304] 0 2024-05-13T22:48:24.454 INFO:tasks.workunit.client.0.smithi102.stdout:7/836: mkdir d6/dd0/df2/d105 0 2024-05-13T22:48:24.457 INFO:tasks.workunit.client.0.smithi102.stdout:7/837: write d6/d15/f3e [2085051,56653] 0 2024-05-13T22:48:24.461 INFO:tasks.workunit.client.0.smithi102.stdout:7/838: mknod d6/dd0/d80/dda/c106 0 2024-05-13T22:48:24.466 INFO:tasks.workunit.client.0.smithi102.stdout:7/839: creat d6/dd0/df2/d105/f107 x:0 0 0 2024-05-13T22:48:24.468 INFO:tasks.workunit.client.0.smithi102.stdout:7/840: mknod d6/d15/d4d/c108 0 2024-05-13T22:48:24.474 INFO:tasks.workunit.client.0.smithi102.stdout:7/841: mknod d6/de1/de6/c109 0 2024-05-13T22:48:24.477 INFO:tasks.workunit.client.1.smithi130.stdout:8/103: rmdir d1/d2 39 2024-05-13T22:48:24.477 INFO:tasks.workunit.client.1.smithi130.stdout:8/104: chown d1 0 1 2024-05-13T22:48:24.478 INFO:tasks.workunit.client.0.smithi102.stdout:6/989: write d9/db/d14/d35/d47/fa8 [458196,1794] 0 2024-05-13T22:48:24.479 INFO:tasks.workunit.client.1.smithi130.stdout:7/82: readlink d7/da/l18 0 2024-05-13T22:48:24.481 INFO:tasks.workunit.client.0.smithi102.stdout:7/842: unlink d6/da9/cb6 0 2024-05-13T22:48:24.481 INFO:tasks.workunit.client.1.smithi130.stdout:3/103: readlink d8/dc/d19/d1d/l1f 0 2024-05-13T22:48:24.482 INFO:tasks.workunit.client.0.smithi102.stdout:6/990: dread - d9/db/d14/d35/d47/f8f zero size 2024-05-13T22:48:24.483 INFO:tasks.workunit.client.0.smithi102.stdout:6/991: stat d9/d45/d56/c141 0 2024-05-13T22:48:24.483 INFO:tasks.workunit.client.1.smithi130.stdout:6/145: fdatasync d4/f12 0 2024-05-13T22:48:24.487 INFO:tasks.workunit.client.1.smithi130.stdout:9/121: mknod d1/db/c22 0 2024-05-13T22:48:24.491 INFO:tasks.workunit.client.1.smithi130.stdout:1/121: mknod db/c25 0 2024-05-13T22:48:24.494 INFO:tasks.workunit.client.1.smithi130.stdout:4/104: unlink f3 0 2024-05-13T22:48:24.498 INFO:tasks.workunit.client.1.smithi130.stdout:0/128: dread d0/fa [0,4194304] 0 2024-05-13T22:48:24.499 INFO:tasks.workunit.client.1.smithi130.stdout:5/105: creat d6/f1f x:0 0 0 2024-05-13T22:48:24.500 INFO:tasks.workunit.client.1.smithi130.stdout:5/106: read d6/f11 [461754,67417] 0 2024-05-13T22:48:24.500 INFO:tasks.workunit.client.1.smithi130.stdout:7/83: mkdir d7/da/d1f/d21 0 2024-05-13T22:48:24.501 INFO:tasks.workunit.client.1.smithi130.stdout:7/84: chown d7 3860221 1 2024-05-13T22:48:24.502 INFO:tasks.workunit.client.0.smithi102.stdout:6/992: dread d9/fec [0,4194304] 0 2024-05-13T22:48:24.502 INFO:tasks.workunit.client.1.smithi130.stdout:6/146: creat d4/d16/f35 x:0 0 0 2024-05-13T22:48:24.506 INFO:tasks.workunit.client.0.smithi102.stdout:6/993: unlink d9/db/d14/c39 0 2024-05-13T22:48:24.506 INFO:tasks.workunit.client.1.smithi130.stdout:5/107: dwrite d6/dd/d18/f15 [0,4194304] 0 2024-05-13T22:48:24.508 INFO:tasks.workunit.client.0.smithi102.stdout:6/994: mknod d9/d45/ddf/c142 0 2024-05-13T22:48:24.510 INFO:tasks.workunit.client.0.smithi102.stdout:6/995: chown d9/db/f50 7 1 2024-05-13T22:48:24.510 INFO:tasks.workunit.client.1.smithi130.stdout:2/104: getdents d4 0 2024-05-13T22:48:24.510 INFO:tasks.workunit.client.0.smithi102.stdout:7/843: dwrite d6/d15/d4d/d55/d6f/d49/f4e [0,4194304] 0 2024-05-13T22:48:24.511 INFO:tasks.workunit.client.0.smithi102.stdout:6/996: dread - d9/d25/d60/f12d zero size 2024-05-13T22:48:24.512 INFO:tasks.workunit.client.0.smithi102.stdout:1/713: sync 2024-05-13T22:48:24.514 INFO:tasks.workunit.client.0.smithi102.stdout:1/714: read d2/d45/d96/fd0 [537820,38998] 0 2024-05-13T22:48:24.515 INFO:tasks.workunit.client.0.smithi102.stdout:7/844: read - d6/d22/dd2/ff4 zero size 2024-05-13T22:48:24.516 INFO:tasks.workunit.client.1.smithi130.stdout:0/129: read d0/fe [1142090,55610] 0 2024-05-13T22:48:24.516 INFO:tasks.workunit.client.1.smithi130.stdout:4/105: dread d8/df/f24 [0,4194304] 0 2024-05-13T22:48:24.516 INFO:tasks.workunit.client.0.smithi102.stdout:7/845: write d6/dd0/f50 [508408,21647] 0 2024-05-13T22:48:24.517 INFO:tasks.workunit.client.1.smithi130.stdout:0/130: chown d0/f21 680702 1 2024-05-13T22:48:24.517 INFO:tasks.workunit.client.0.smithi102.stdout:7/846: readlink d6/d15/d4d/d55/d6f/l63 0 2024-05-13T22:48:24.521 INFO:tasks.workunit.client.1.smithi130.stdout:8/105: rename d1/d2/db/d15/d18/f1b to d1/dc/f26 0 2024-05-13T22:48:24.522 INFO:tasks.workunit.client.0.smithi102.stdout:7/847: read d6/d13/d2c/da1/dc1/f5f [207259,103122] 0 2024-05-13T22:48:24.529 INFO:tasks.workunit.client.1.smithi130.stdout:7/85: rename d7/da/l18 to d7/da/l22 0 2024-05-13T22:48:24.531 INFO:tasks.workunit.client.1.smithi130.stdout:7/86: dread d7/d8/fe [0,4194304] 0 2024-05-13T22:48:24.532 INFO:tasks.workunit.client.0.smithi102.stdout:7/848: dwrite d6/d15/d4d/fde [0,4194304] 0 2024-05-13T22:48:24.536 INFO:tasks.workunit.client.1.smithi130.stdout:5/108: symlink d6/d7/l20 0 2024-05-13T22:48:24.536 INFO:tasks.workunit.client.1.smithi130.stdout:5/109: chown d6/dd/ff 4069437 1 2024-05-13T22:48:24.541 INFO:tasks.workunit.client.0.smithi102.stdout:6/997: write d9/d25/d60/f7e [705421,114533] 0 2024-05-13T22:48:24.543 INFO:tasks.workunit.client.0.smithi102.stdout:1/715: dwrite d2/d45/f84 [0,4194304] 0 2024-05-13T22:48:24.551 INFO:tasks.workunit.client.0.smithi102.stdout:6/998: dwrite d9/db/d14/d35/f67 [0,4194304] 0 2024-05-13T22:48:24.551 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:24 smithi130 ceph-mon[36591]: pgmap v29: 65 pgs: 65 active+clean; 2.0 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 60 MiB/s rd, 159 MiB/s wr, 487 op/s 2024-05-13T22:48:24.551 INFO:tasks.workunit.client.1.smithi130.stdout:0/131: mkdir d0/d7/dd/d2e 0 2024-05-13T22:48:24.552 INFO:tasks.workunit.client.1.smithi130.stdout:1/122: truncate db/d1b/f20 691488 0 2024-05-13T22:48:24.557 INFO:tasks.workunit.client.1.smithi130.stdout:8/106: creat d1/d2/f27 x:0 0 0 2024-05-13T22:48:24.559 INFO:tasks.workunit.client.1.smithi130.stdout:8/107: write d1/dc/f1f [560731,43054] 0 2024-05-13T22:48:24.560 INFO:tasks.workunit.client.1.smithi130.stdout:2/105: dwrite d4/db/f28 [0,4194304] 0 2024-05-13T22:48:24.562 INFO:tasks.workunit.client.1.smithi130.stdout:6/147: unlink d4/d16/f35 0 2024-05-13T22:48:24.572 INFO:tasks.workunit.client.1.smithi130.stdout:5/110: dwrite f2 [0,4194304] 0 2024-05-13T22:48:24.572 INFO:tasks.workunit.client.1.smithi130.stdout:1/123: creat db/f26 x:0 0 0 2024-05-13T22:48:24.573 INFO:tasks.workunit.client.1.smithi130.stdout:5/111: stat d6/d7/fa 0 2024-05-13T22:48:24.576 INFO:tasks.workunit.client.1.smithi130.stdout:3/104: getdents d8 0 2024-05-13T22:48:24.580 INFO:tasks.workunit.client.1.smithi130.stdout:2/106: mkdir d4/db/dc/d18/d2a/d30 0 2024-05-13T22:48:24.581 INFO:tasks.workunit.client.1.smithi130.stdout:5/112: dwrite d6/dd/d18/fc [0,4194304] 0 2024-05-13T22:48:24.582 INFO:tasks.workunit.client.1.smithi130.stdout:6/148: rename d4/l10 to d4/d16/l36 0 2024-05-13T22:48:24.582 INFO:tasks.workunit.client.1.smithi130.stdout:6/149: chown d4/ce 378 1 2024-05-13T22:48:24.583 INFO:tasks.workunit.client.1.smithi130.stdout:9/122: truncate d1/d15/f1d 2148507 0 2024-05-13T22:48:24.583 INFO:tasks.workunit.client.1.smithi130.stdout:6/150: fdatasync d4/d15/d1f/f2d 0 2024-05-13T22:48:24.584 INFO:tasks.workunit.client.1.smithi130.stdout:6/151: readlink d4/d15/l2b 0 2024-05-13T22:48:24.584 INFO:tasks.workunit.client.1.smithi130.stdout:0/132: creat d0/d7/dd/d2e/f2f x:0 0 0 2024-05-13T22:48:24.585 INFO:tasks.workunit.client.1.smithi130.stdout:0/133: chown d0/d7/dd/d17/d1f 30 1 2024-05-13T22:48:24.585 INFO:tasks.workunit.client.1.smithi130.stdout:0/134: chown d0/d7/dd/d17/f1b 4067872 1 2024-05-13T22:48:24.586 INFO:tasks.workunit.client.1.smithi130.stdout:1/124: mknod db/d1b/c27 0 2024-05-13T22:48:24.597 INFO:tasks.workunit.client.1.smithi130.stdout:2/107: unlink d4/db/dc/d18/f29 0 2024-05-13T22:48:24.599 INFO:tasks.workunit.client.1.smithi130.stdout:9/123: mkdir d1/d7/d23 0 2024-05-13T22:48:24.602 INFO:tasks.workunit.client.1.smithi130.stdout:0/135: chown d0/f1e 214 1 2024-05-13T22:48:24.605 INFO:tasks.workunit.client.1.smithi130.stdout:3/105: unlink f5 0 2024-05-13T22:48:24.605 INFO:tasks.workunit.client.1.smithi130.stdout:3/106: fsync d8/dc/f12 0 2024-05-13T22:48:24.611 INFO:tasks.workunit.client.1.smithi130.stdout:4/106: sync 2024-05-13T22:48:24.612 INFO:tasks.workunit.client.1.smithi130.stdout:4/107: write d8/f1e [225563,123755] 0 2024-05-13T22:48:24.619 INFO:tasks.workunit.client.1.smithi130.stdout:5/113: mkdir d6/d1a/d21 0 2024-05-13T22:48:24.630 INFO:tasks.workunit.client.1.smithi130.stdout:8/108: link d1/d2/c1d d1/d2/db/d15/d18/d1a/c28 0 2024-05-13T22:48:24.631 INFO:tasks.workunit.client.1.smithi130.stdout:8/109: write d1/d2/f25 [52817,20888] 0 2024-05-13T22:48:24.633 INFO:tasks.workunit.client.1.smithi130.stdout:6/152: dwrite d4/fc [0,4194304] 0 2024-05-13T22:48:24.635 INFO:tasks.workunit.client.1.smithi130.stdout:5/114: creat d6/d7/f22 x:0 0 0 2024-05-13T22:48:24.636 INFO:tasks.workunit.client.1.smithi130.stdout:8/110: dread d1/dc/f21 [0,4194304] 0 2024-05-13T22:48:24.646 INFO:tasks.workunit.client.1.smithi130.stdout:2/108: dwrite d4/f5 [4194304,4194304] 0 2024-05-13T22:48:24.648 INFO:tasks.workunit.client.1.smithi130.stdout:2/109: chown d4/db/dc/f20 116107086 1 2024-05-13T22:48:24.650 INFO:tasks.workunit.client.1.smithi130.stdout:5/115: rmdir d6/dd 39 2024-05-13T22:48:24.652 INFO:tasks.workunit.client.1.smithi130.stdout:5/116: write f4 [2927365,72290] 0 2024-05-13T22:48:24.653 INFO:tasks.workunit.client.0.smithi102.stdout:7/849: creat d6/d13/d58/f10a x:0 0 0 2024-05-13T22:48:24.654 INFO:tasks.workunit.client.1.smithi130.stdout:0/136: dwrite d0/f27 [0,4194304] 0 2024-05-13T22:48:24.656 INFO:tasks.workunit.client.0.smithi102.stdout:1/716: dread d2/d4/f23 [0,4194304] 0 2024-05-13T22:48:24.657 INFO:tasks.workunit.client.1.smithi130.stdout:2/110: dread d4/f8 [0,4194304] 0 2024-05-13T22:48:24.657 INFO:tasks.workunit.client.0.smithi102.stdout:6/999: creat d9/d45/d56/f143 x:0 0 0 2024-05-13T22:48:24.660 INFO:tasks.workunit.client.1.smithi130.stdout:2/111: truncate d4/db/dc/f24 769500 0 2024-05-13T22:48:24.661 INFO:tasks.workunit.client.1.smithi130.stdout:1/125: getdents db/d1b 0 2024-05-13T22:48:24.664 INFO:tasks.workunit.client.1.smithi130.stdout:3/107: truncate d8/f16 494049 0 2024-05-13T22:48:24.664 INFO:tasks.workunit.client.0.smithi102.stdout:1/717: dread d2/f4b [0,4194304] 0 2024-05-13T22:48:24.664 INFO:tasks.workunit.client.1.smithi130.stdout:0/137: dwrite d0/f1e [0,4194304] 0 2024-05-13T22:48:24.667 INFO:tasks.workunit.client.1.smithi130.stdout:7/87: sync 2024-05-13T22:48:24.672 INFO:tasks.workunit.client.0.smithi102.stdout:1/718: mkdir d2/d4/d5/d19/d28/d6a/dbd/dec 0 2024-05-13T22:48:24.673 INFO:tasks.workunit.client.1.smithi130.stdout:0/138: dwrite d0/d7/f28 [0,4194304] 0 2024-05-13T22:48:24.676 INFO:tasks.workunit.client.0.smithi102.stdout:1/719: truncate d2/d45/d96/fd0 256022 0 2024-05-13T22:48:24.678 INFO:tasks.workunit.client.0.smithi102.stdout:1/720: dread d2/d45/f58 [0,4194304] 0 2024-05-13T22:48:24.681 INFO:tasks.workunit.client.1.smithi130.stdout:6/153: creat d4/f37 x:0 0 0 2024-05-13T22:48:24.683 INFO:tasks.workunit.client.0.smithi102.stdout:1/721: unlink d2/d4/d26/d40/l42 0 2024-05-13T22:48:24.684 INFO:tasks.workunit.client.1.smithi130.stdout:3/108: creat d8/dc/f22 x:0 0 0 2024-05-13T22:48:24.687 INFO:tasks.workunit.client.1.smithi130.stdout:7/88: creat d7/da/d1f/f23 x:0 0 0 2024-05-13T22:48:24.688 INFO:tasks.workunit.client.1.smithi130.stdout:2/112: dwrite d4/db/d26/f2d [4194304,4194304] 0 2024-05-13T22:48:24.689 INFO:tasks.workunit.client.0.smithi102.stdout:1/722: truncate d2/d4/d5/d19/d28/d6a/f59 300197 0 2024-05-13T22:48:24.689 INFO:tasks.workunit.client.1.smithi130.stdout:7/89: write d7/d8/fe [1624261,34828] 0 2024-05-13T22:48:24.689 INFO:tasks.workunit.client.1.smithi130.stdout:2/113: fdatasync d4/db/dc/f14 0 2024-05-13T22:48:24.689 INFO:tasks.workunit.client.1.smithi130.stdout:8/111: link d1/d2/db/f8 d1/d2/db/d15/f29 0 2024-05-13T22:48:24.692 INFO:tasks.workunit.client.1.smithi130.stdout:5/117: mknod d6/d1a/d21/c23 0 2024-05-13T22:48:24.693 INFO:tasks.workunit.client.1.smithi130.stdout:5/118: truncate f2 5800226 0 2024-05-13T22:48:24.694 INFO:tasks.workunit.client.1.smithi130.stdout:4/108: getdents d8/df 0 2024-05-13T22:48:24.695 INFO:tasks.workunit.client.0.smithi102.stdout:1/723: dwrite d2/d4/d26/d40/f72 [0,4194304] 0 2024-05-13T22:48:24.696 INFO:tasks.workunit.client.1.smithi130.stdout:4/109: write d8/f1f [601547,95485] 0 2024-05-13T22:48:24.699 INFO:tasks.workunit.client.1.smithi130.stdout:6/154: creat d4/d15/d1d/f38 x:0 0 0 2024-05-13T22:48:24.699 INFO:tasks.workunit.client.1.smithi130.stdout:9/124: dread d1/d15/f1d [0,4194304] 0 2024-05-13T22:48:24.699 INFO:tasks.workunit.client.1.smithi130.stdout:2/114: dwrite d4/db/f1c [0,4194304] 0 2024-05-13T22:48:24.704 INFO:tasks.workunit.client.0.smithi102.stdout:1/724: dwrite d2/d45/d79/d92/f61 [0,4194304] 0 2024-05-13T22:48:24.704 INFO:tasks.workunit.client.1.smithi130.stdout:8/112: creat d1/d2/db/d15/d16/f2a x:0 0 0 2024-05-13T22:48:24.706 INFO:tasks.workunit.client.1.smithi130.stdout:0/139: link d0/d7/dd/d17/d1f/f25 d0/d7/dd/d17/d1f/d2c/f30 0 2024-05-13T22:48:24.707 INFO:tasks.workunit.client.1.smithi130.stdout:5/119: mkdir d6/d7/d24 0 2024-05-13T22:48:24.708 INFO:tasks.workunit.client.1.smithi130.stdout:6/155: dwrite d4/d16/f2a [0,4194304] 0 2024-05-13T22:48:24.709 INFO:tasks.workunit.client.1.smithi130.stdout:1/126: link db/c11 db/d1b/c28 0 2024-05-13T22:48:24.709 INFO:tasks.workunit.client.1.smithi130.stdout:4/110: mkdir d8/dc/d26 0 2024-05-13T22:48:24.711 INFO:tasks.workunit.client.1.smithi130.stdout:6/156: chown d4/d15/c27 297942 1 2024-05-13T22:48:24.718 INFO:tasks.workunit.client.1.smithi130.stdout:6/157: dread d4/d15/d1f/f2f [0,4194304] 0 2024-05-13T22:48:24.722 INFO:tasks.workunit.client.1.smithi130.stdout:8/113: rmdir d1/d2/db 39 2024-05-13T22:48:24.723 INFO:tasks.workunit.client.1.smithi130.stdout:5/120: creat d6/d7/f25 x:0 0 0 2024-05-13T22:48:24.724 INFO:tasks.workunit.client.1.smithi130.stdout:0/140: creat d0/d7/dd/d17/d1f/d2c/f31 x:0 0 0 2024-05-13T22:48:24.725 INFO:tasks.workunit.client.1.smithi130.stdout:2/115: dwrite d4/db/f27 [0,4194304] 0 2024-05-13T22:48:24.727 INFO:tasks.workunit.client.1.smithi130.stdout:2/116: fsync d4/f7 0 2024-05-13T22:48:24.728 INFO:tasks.workunit.client.1.smithi130.stdout:0/141: stat d0/d7/dd/d17/d1f/f25 0 2024-05-13T22:48:24.728 INFO:tasks.workunit.client.0.smithi102.stdout:1/725: rename d2/d4/d5/d19/d28 to d2/d4/d5/d19/ded 0 2024-05-13T22:48:24.729 INFO:tasks.workunit.client.1.smithi130.stdout:6/158: dwrite d4/f12 [0,4194304] 0 2024-05-13T22:48:24.729 INFO:tasks.workunit.client.1.smithi130.stdout:2/117: chown d4/db/f22 8088796 1 2024-05-13T22:48:24.730 INFO:tasks.workunit.client.1.smithi130.stdout:2/118: chown d4/db/dc/f25 2871 1 2024-05-13T22:48:24.731 INFO:tasks.workunit.client.1.smithi130.stdout:2/119: chown d4/f1f 12 1 2024-05-13T22:48:24.731 INFO:tasks.workunit.client.1.smithi130.stdout:2/120: read d4/f5 [5763212,47352] 0 2024-05-13T22:48:24.731 INFO:tasks.workunit.client.1.smithi130.stdout:7/90: rmdir d7/da/d1f/d21 0 2024-05-13T22:48:24.737 INFO:tasks.workunit.client.1.smithi130.stdout:8/114: fdatasync d1/d2/fd 0 2024-05-13T22:48:24.738 INFO:tasks.workunit.client.1.smithi130.stdout:3/109: write d8/f9 [2632768,27393] 0 2024-05-13T22:48:24.743 INFO:tasks.workunit.client.1.smithi130.stdout:5/121: dwrite d6/dd/f13 [0,4194304] 0 2024-05-13T22:48:24.744 INFO:tasks.workunit.client.1.smithi130.stdout:5/122: fsync d6/dd/d18/fc 0 2024-05-13T22:48:24.794 INFO:tasks.workunit.client.1.smithi130.stdout:4/111: sync 2024-05-13T22:48:24.795 INFO:tasks.workunit.client.1.smithi130.stdout:1/127: truncate db/f16 1800902 0 2024-05-13T22:48:24.801 INFO:tasks.workunit.client.1.smithi130.stdout:2/121: mknod d4/db/dc/d18/c31 0 2024-05-13T22:48:24.801 INFO:tasks.workunit.client.1.smithi130.stdout:9/125: link d1/d7/l18 d1/d15/l24 0 2024-05-13T22:48:24.808 INFO:tasks.workunit.client.1.smithi130.stdout:5/123: rmdir d6/d7 39 2024-05-13T22:48:24.811 INFO:tasks.workunit.client.1.smithi130.stdout:4/112: mknod d8/dc/c27 0 2024-05-13T22:48:24.813 INFO:tasks.workunit.client.0.smithi102.stdout:1/726: write d2/d4/d5/f15 [1279310,127163] 0 2024-05-13T22:48:24.814 INFO:tasks.workunit.client.0.smithi102.stdout:1/727: stat d2/d45/ldd 0 2024-05-13T22:48:24.815 INFO:tasks.workunit.client.1.smithi130.stdout:1/128: dread db/f1a [0,4194304] 0 2024-05-13T22:48:24.817 INFO:tasks.workunit.client.1.smithi130.stdout:1/129: dread db/d1b/f20 [0,4194304] 0 2024-05-13T22:48:24.817 INFO:tasks.workunit.client.1.smithi130.stdout:2/122: mknod d4/db/c32 0 2024-05-13T22:48:24.819 INFO:tasks.workunit.client.1.smithi130.stdout:1/130: truncate db/f26 641577 0 2024-05-13T22:48:24.819 INFO:tasks.workunit.client.1.smithi130.stdout:8/115: mknod d1/d2/db/c2b 0 2024-05-13T22:48:24.820 INFO:tasks.workunit.client.1.smithi130.stdout:3/110: symlink d8/d1a/d1c/l23 0 2024-05-13T22:48:24.821 INFO:tasks.workunit.client.1.smithi130.stdout:0/142: dwrite d0/d7/dd/d17/d1f/f25 [4194304,4194304] 0 2024-05-13T22:48:24.821 INFO:tasks.workunit.client.1.smithi130.stdout:5/124: unlink f2 0 2024-05-13T22:48:24.823 INFO:tasks.workunit.client.1.smithi130.stdout:4/113: creat d8/dc/f28 x:0 0 0 2024-05-13T22:48:24.823 INFO:tasks.workunit.client.1.smithi130.stdout:4/114: chown f4 901 1 2024-05-13T22:48:24.825 INFO:tasks.workunit.client.1.smithi130.stdout:6/159: rmdir d4/d15/d1d/d32 0 2024-05-13T22:48:24.825 INFO:tasks.workunit.client.1.smithi130.stdout:4/115: write d8/dc/d13/f19 [356238,76216] 0 2024-05-13T22:48:24.827 INFO:tasks.workunit.client.1.smithi130.stdout:2/123: dwrite d4/db/f2c [0,4194304] 0 2024-05-13T22:48:24.827 INFO:tasks.workunit.client.1.smithi130.stdout:4/116: fdatasync d8/dc/fd 0 2024-05-13T22:48:24.828 INFO:tasks.workunit.client.1.smithi130.stdout:9/126: link d1/f1e d1/d15/f25 0 2024-05-13T22:48:24.829 INFO:tasks.workunit.client.1.smithi130.stdout:9/127: chown d1/db/cf 1701 1 2024-05-13T22:48:24.832 INFO:tasks.workunit.client.0.smithi102.stdout:1/728: dwrite d2/d4/d26/f8c [0,4194304] 0 2024-05-13T22:48:24.834 INFO:tasks.workunit.client.1.smithi130.stdout:8/116: creat d1/d2/db/d15/d16/f2c x:0 0 0 2024-05-13T22:48:24.838 INFO:tasks.workunit.client.1.smithi130.stdout:1/131: dwrite db/f22 [0,4194304] 0 2024-05-13T22:48:24.840 INFO:tasks.workunit.client.1.smithi130.stdout:0/143: creat d0/d7/dd/f32 x:0 0 0 2024-05-13T22:48:24.841 INFO:tasks.workunit.client.1.smithi130.stdout:7/91: rename d7/d8/d9/c10 to d7/d8/d9/c24 0 2024-05-13T22:48:24.842 INFO:tasks.workunit.client.1.smithi130.stdout:0/144: read d0/f1e [1650444,98594] 0 2024-05-13T22:48:24.846 INFO:tasks.workunit.client.1.smithi130.stdout:6/160: creat d4/d16/f39 x:0 0 0 2024-05-13T22:48:24.847 INFO:tasks.workunit.client.0.smithi102.stdout:1/729: creat d2/d10/fee x:0 0 0 2024-05-13T22:48:24.847 INFO:tasks.workunit.client.1.smithi130.stdout:5/125: dread d6/d7/fa [0,4194304] 0 2024-05-13T22:48:24.848 INFO:tasks.workunit.client.1.smithi130.stdout:1/132: chown db/f1d 3296 1 2024-05-13T22:48:24.848 INFO:tasks.workunit.client.1.smithi130.stdout:2/124: mkdir d4/d33 0 2024-05-13T22:48:24.851 INFO:tasks.workunit.client.1.smithi130.stdout:9/128: rmdir d1/d7 39 2024-05-13T22:48:24.851 INFO:tasks.workunit.client.0.smithi102.stdout:1/730: mkdir d2/d45/da4/def 0 2024-05-13T22:48:24.853 INFO:tasks.workunit.client.1.smithi130.stdout:8/117: dwrite d1/f10 [0,4194304] 0 2024-05-13T22:48:24.854 INFO:tasks.workunit.client.1.smithi130.stdout:2/125: write d4/db/f2c [1361837,18935] 0 2024-05-13T22:48:24.861 INFO:tasks.workunit.client.1.smithi130.stdout:1/133: rmdir db/d1b 39 2024-05-13T22:48:24.863 INFO:tasks.workunit.client.1.smithi130.stdout:6/161: dwrite d4/ff [0,4194304] 0 2024-05-13T22:48:24.863 INFO:tasks.workunit.client.1.smithi130.stdout:7/92: dwrite d7/da/d1f/f20 [0,4194304] 0 2024-05-13T22:48:24.864 INFO:tasks.workunit.client.1.smithi130.stdout:9/129: symlink d1/db/l26 0 2024-05-13T22:48:24.869 INFO:tasks.workunit.client.1.smithi130.stdout:6/162: creat d4/d15/f3a x:0 0 0 2024-05-13T22:48:24.869 INFO:tasks.workunit.client.1.smithi130.stdout:2/126: creat d4/db/dc/d18/d2a/d30/f34 x:0 0 0 2024-05-13T22:48:24.872 INFO:tasks.workunit.client.1.smithi130.stdout:6/163: dread - d4/f37 zero size 2024-05-13T22:48:24.873 INFO:tasks.workunit.client.1.smithi130.stdout:3/111: dread f3 [0,4194304] 0 2024-05-13T22:48:24.874 INFO:tasks.workunit.client.1.smithi130.stdout:1/134: rmdir db/d1b 39 2024-05-13T22:48:24.875 INFO:tasks.workunit.client.1.smithi130.stdout:0/145: rmdir d0/d7 39 2024-05-13T22:48:24.875 INFO:tasks.workunit.client.1.smithi130.stdout:7/93: dwrite d7/da/d1f/f20 [0,4194304] 0 2024-05-13T22:48:24.875 INFO:tasks.workunit.client.1.smithi130.stdout:3/112: dread - d8/dc/f22 zero size 2024-05-13T22:48:24.876 INFO:tasks.workunit.client.1.smithi130.stdout:9/130: creat d1/d7/f27 x:0 0 0 2024-05-13T22:48:24.878 INFO:tasks.workunit.client.1.smithi130.stdout:2/127: unlink d4/db/dc/f24 0 2024-05-13T22:48:24.879 INFO:tasks.workunit.client.1.smithi130.stdout:2/128: readlink d4/db/l13 0 2024-05-13T22:48:24.881 INFO:tasks.workunit.client.1.smithi130.stdout:8/118: link d1/d2/db/d15/f29 d1/f2d 0 2024-05-13T22:48:24.882 INFO:tasks.workunit.client.1.smithi130.stdout:6/164: symlink d4/d15/d1f/l3b 0 2024-05-13T22:48:24.886 INFO:tasks.workunit.client.1.smithi130.stdout:3/113: mknod d8/dc/d15/c24 0 2024-05-13T22:48:24.890 INFO:tasks.workunit.client.1.smithi130.stdout:1/135: truncate f8 898867 0 2024-05-13T22:48:24.893 INFO:tasks.workunit.client.1.smithi130.stdout:9/131: mkdir d1/d7/d23/d28 0 2024-05-13T22:48:24.894 INFO:tasks.workunit.client.1.smithi130.stdout:4/117: sync 2024-05-13T22:48:24.894 INFO:tasks.workunit.client.1.smithi130.stdout:3/114: rmdir d8/d1a/d1c 39 2024-05-13T22:48:24.894 INFO:tasks.workunit.client.0.smithi102.stdout:1/731: dwrite d2/d10/d62/d7b/f82 [0,4194304] 0 2024-05-13T22:48:24.896 INFO:tasks.workunit.client.1.smithi130.stdout:7/94: fdatasync d7/da/d1f/f20 0 2024-05-13T22:48:24.896 INFO:tasks.workunit.client.1.smithi130.stdout:2/129: creat d4/d33/f35 x:0 0 0 2024-05-13T22:48:24.896 INFO:tasks.workunit.client.1.smithi130.stdout:3/115: dread f2 [0,4194304] 0 2024-05-13T22:48:24.902 INFO:tasks.workunit.client.1.smithi130.stdout:1/136: dread db/d1b/f20 [0,4194304] 0 2024-05-13T22:48:24.902 INFO:tasks.workunit.client.0.smithi102.stdout:1/732: mkdir d2/d4/d5/df0 0 2024-05-13T22:48:24.903 INFO:tasks.workunit.client.1.smithi130.stdout:3/116: dwrite d8/f9 [4194304,4194304] 0 2024-05-13T22:48:24.903 INFO:tasks.workunit.client.1.smithi130.stdout:9/132: fsync d1/d7/f27 0 2024-05-13T22:48:24.904 INFO:tasks.workunit.client.1.smithi130.stdout:9/133: read - d1/d15/f16 zero size 2024-05-13T22:48:24.905 INFO:tasks.workunit.client.0.smithi102.stdout:1/733: creat d2/d4/d26/ff1 x:0 0 0 2024-05-13T22:48:24.905 INFO:tasks.workunit.client.0.smithi102.stdout:1/734: readlink d2/d4/d26/d40/l9b 0 2024-05-13T22:48:24.909 INFO:tasks.workunit.client.1.smithi130.stdout:4/118: mkdir d8/dc/d29 0 2024-05-13T22:48:24.911 INFO:tasks.workunit.client.1.smithi130.stdout:7/95: mknod d7/da/d1f/c25 0 2024-05-13T22:48:24.912 INFO:tasks.workunit.client.1.smithi130.stdout:2/130: creat d4/db/dc/d18/f36 x:0 0 0 2024-05-13T22:48:24.912 INFO:tasks.workunit.client.0.smithi102.stdout:1/735: dwrite d2/d4/d26/fe6 [0,4194304] 0 2024-05-13T22:48:24.915 INFO:tasks.workunit.client.1.smithi130.stdout:5/126: symlink d6/d1a/d21/l26 0 2024-05-13T22:48:24.918 INFO:tasks.workunit.client.0.smithi102.stdout:1/736: symlink d2/d4/d38/db2/dba/lf2 0 2024-05-13T22:48:24.919 INFO:tasks.workunit.client.1.smithi130.stdout:6/165: rmdir d4/d16/d2c 0 2024-05-13T22:48:24.922 INFO:tasks.workunit.client.1.smithi130.stdout:4/119: rename d8/d22 to d8/dc/d26/d2a 0 2024-05-13T22:48:24.923 INFO:tasks.workunit.client.0.smithi102.stdout:1/737: creat d2/d10/d62/ff3 x:0 0 0 2024-05-13T22:48:24.926 INFO:tasks.workunit.client.1.smithi130.stdout:8/119: getdents d1/d2/db/d15/d18/d1a 0 2024-05-13T22:48:24.930 INFO:tasks.workunit.client.1.smithi130.stdout:9/134: creat d1/d7/d23/d28/f29 x:0 0 0 2024-05-13T22:48:24.930 INFO:tasks.workunit.client.1.smithi130.stdout:0/146: getdents d0/d7/d10 0 2024-05-13T22:48:24.932 INFO:tasks.workunit.client.1.smithi130.stdout:6/166: mkdir d4/d15/d1f/d3c 0 2024-05-13T22:48:24.933 INFO:tasks.workunit.client.1.smithi130.stdout:0/147: dread d0/f21 [0,4194304] 0 2024-05-13T22:48:24.933 INFO:tasks.workunit.client.0.smithi102.stdout:1/738: getdents d2/d4/d26/d40 0 2024-05-13T22:48:24.935 INFO:tasks.workunit.client.1.smithi130.stdout:7/96: symlink d7/da/d1d/l26 0 2024-05-13T22:48:24.937 INFO:tasks.workunit.client.1.smithi130.stdout:4/120: rename d8/fe to d8/dc/d26/f2b 0 2024-05-13T22:48:24.938 INFO:tasks.workunit.client.0.smithi102.stdout:1/739: mkdir d2/d45/df4 0 2024-05-13T22:48:24.938 INFO:tasks.workunit.client.0.smithi102.stdout:1/740: readlink d2/d4/d26/d40/lc0 0 2024-05-13T22:48:24.940 INFO:tasks.workunit.client.0.smithi102.stdout:1/741: truncate d2/d4/d38/f68 4851756 0 2024-05-13T22:48:24.940 INFO:tasks.workunit.client.1.smithi130.stdout:9/135: creat d1/d7/f2a x:0 0 0 2024-05-13T22:48:24.942 INFO:tasks.workunit.client.1.smithi130.stdout:1/137: dwrite db/f16 [0,4194304] 0 2024-05-13T22:48:24.943 INFO:tasks.workunit.client.1.smithi130.stdout:3/117: getdents d8 0 2024-05-13T22:48:24.946 INFO:tasks.workunit.client.1.smithi130.stdout:9/136: dread - d1/d7/fa zero size 2024-05-13T22:48:24.947 INFO:tasks.workunit.client.1.smithi130.stdout:2/131: getdents d4/db 0 2024-05-13T22:48:24.948 INFO:tasks.workunit.client.0.smithi102.stdout:2/964: sync 2024-05-13T22:48:24.948 INFO:tasks.workunit.client.0.smithi102.stdout:0/876: sync 2024-05-13T22:48:24.948 INFO:tasks.workunit.client.0.smithi102.stdout:4/896: sync 2024-05-13T22:48:24.950 INFO:tasks.workunit.client.0.smithi102.stdout:1/742: write d2/d45/d79/d92/f64 [1290569,92329] 0 2024-05-13T22:48:24.952 INFO:tasks.workunit.client.1.smithi130.stdout:7/97: dwrite d7/da/d1f/f20 [0,4194304] 0 2024-05-13T22:48:24.952 INFO:tasks.workunit.client.1.smithi130.stdout:4/121: creat d8/dc/d29/f2c x:0 0 0 2024-05-13T22:48:24.954 INFO:tasks.workunit.client.1.smithi130.stdout:5/127: dwrite d6/dd/f13 [0,4194304] 0 2024-05-13T22:48:24.956 INFO:tasks.workunit.client.1.smithi130.stdout:0/148: dwrite d0/f4 [0,4194304] 0 2024-05-13T22:48:24.956 INFO:tasks.workunit.client.1.smithi130.stdout:8/120: rename d1/d2/db/l13 to d1/d2/db/d15/d18/l2e 0 2024-05-13T22:48:24.956 INFO:tasks.workunit.client.1.smithi130.stdout:5/128: write d6/dd/f13 [430748,1047] 0 2024-05-13T22:48:24.958 INFO:tasks.workunit.client.1.smithi130.stdout:9/137: rmdir d1/d7 39 2024-05-13T22:48:24.959 INFO:tasks.workunit.client.1.smithi130.stdout:9/138: truncate d1/f1e 412743 0 2024-05-13T22:48:24.959 INFO:tasks.workunit.client.0.smithi102.stdout:1/743: dwrite d2/d10/d62/d7b/dc4/fc8 [0,4194304] 0 2024-05-13T22:48:24.961 INFO:tasks.workunit.client.1.smithi130.stdout:1/138: symlink db/l29 0 2024-05-13T22:48:24.962 INFO:tasks.workunit.client.0.smithi102.stdout:4/897: write d3/d4/d6/f80 [1259218,57008] 0 2024-05-13T22:48:24.966 INFO:tasks.workunit.client.0.smithi102.stdout:0/877: getdents d1/d4d/d79/d9f/dbd/d10d 0 2024-05-13T22:48:24.967 INFO:tasks.workunit.client.1.smithi130.stdout:3/118: fsync d8/f16 0 2024-05-13T22:48:24.968 INFO:tasks.workunit.client.0.smithi102.stdout:2/965: creat d7/d9/d10/d30/f139 x:0 0 0 2024-05-13T22:48:24.972 INFO:tasks.workunit.client.0.smithi102.stdout:4/898: rename d3/c100 to d3/d19/d23/d39/ddd/c13a 0 2024-05-13T22:48:24.973 INFO:tasks.workunit.client.0.smithi102.stdout:4/899: chown d3/d4/d5f 8 1 2024-05-13T22:48:24.973 INFO:tasks.workunit.client.1.smithi130.stdout:3/119: dwrite d8/fa [0,4194304] 0 2024-05-13T22:48:24.974 INFO:tasks.workunit.client.0.smithi102.stdout:0/878: symlink d1/de/d42/d61/l112 0 2024-05-13T22:48:24.976 INFO:tasks.workunit.client.1.smithi130.stdout:6/167: rename d4/f8 to d4/d15/d1d/f3d 0 2024-05-13T22:48:24.979 INFO:tasks.workunit.client.0.smithi102.stdout:2/966: truncate d7/d15/d67/d129/d56/d59/d81/f110 4752193 0 2024-05-13T22:48:24.979 INFO:tasks.workunit.client.1.smithi130.stdout:8/121: unlink d1/d2/f27 0 2024-05-13T22:48:24.981 INFO:tasks.workunit.client.1.smithi130.stdout:9/139: mknod d1/d15/c2b 0 2024-05-13T22:48:24.982 INFO:tasks.workunit.client.1.smithi130.stdout:1/139: mkdir db/d1b/d1f/d2a 0 2024-05-13T22:48:24.983 INFO:tasks.workunit.client.0.smithi102.stdout:2/967: unlink d7/d15/c9b 0 2024-05-13T22:48:24.985 INFO:tasks.workunit.client.0.smithi102.stdout:4/900: unlink d3/d4/d6/de/d8d/c11a 0 2024-05-13T22:48:24.987 INFO:tasks.workunit.client.0.smithi102.stdout:2/968: creat d7/d15/d1e/d87/f13a x:0 0 0 2024-05-13T22:48:24.989 INFO:tasks.workunit.client.1.smithi130.stdout:7/98: mknod d7/d8/c27 0 2024-05-13T22:48:24.989 INFO:tasks.workunit.client.1.smithi130.stdout:1/140: dwrite db/f26 [0,4194304] 0 2024-05-13T22:48:24.989 INFO:tasks.workunit.client.0.smithi102.stdout:4/901: fsync d3/d19/d23/d7e/f9f 0 2024-05-13T22:48:24.990 INFO:tasks.workunit.client.0.smithi102.stdout:4/902: write d3/d19/d23/d4e/f12c [1277098,26063] 0 2024-05-13T22:48:24.990 INFO:tasks.workunit.client.1.smithi130.stdout:4/122: symlink d8/l2d 0 2024-05-13T22:48:24.991 INFO:tasks.workunit.client.0.smithi102.stdout:2/969: mknod d7/d9/d10/d30/d7f/c13b 0 2024-05-13T22:48:24.995 INFO:tasks.workunit.client.1.smithi130.stdout:5/129: dread d6/d7/fa [0,4194304] 0 2024-05-13T22:48:24.995 INFO:tasks.workunit.client.0.smithi102.stdout:0/879: getdents d1/de/d42/dea 0 2024-05-13T22:48:24.995 INFO:tasks.workunit.client.0.smithi102.stdout:0/880: dread - d1/de/d32/d24/d2d/f9d zero size 2024-05-13T22:48:24.996 INFO:tasks.workunit.client.1.smithi130.stdout:6/168: write d4/d15/d1d/f3d [289473,119050] 0 2024-05-13T22:48:24.997 INFO:tasks.workunit.client.0.smithi102.stdout:4/903: symlink d3/d4/d6/de/d8d/l13b 0 2024-05-13T22:48:24.997 INFO:tasks.workunit.client.0.smithi102.stdout:0/881: dread - d1/de/d32/d24/ff1 zero size 2024-05-13T22:48:25.000 INFO:tasks.workunit.client.0.smithi102.stdout:9/885: sync 2024-05-13T22:48:25.000 INFO:tasks.workunit.client.1.smithi130.stdout:8/122: symlink d1/d2/db/l2f 0 2024-05-13T22:48:25.001 INFO:tasks.workunit.client.0.smithi102.stdout:9/886: chown d1/d6/db/d30/cba 400 1 2024-05-13T22:48:25.002 INFO:tasks.workunit.client.1.smithi130.stdout:9/140: chown d1/d7/l12 56 1 2024-05-13T22:48:25.002 INFO:tasks.workunit.client.0.smithi102.stdout:4/904: symlink d3/d19/d23/d39/da7/da9/dd3/dd4/l13c 0 2024-05-13T22:48:25.002 INFO:tasks.workunit.client.1.smithi130.stdout:8/123: dread - d1/d2/db/d15/f22 zero size 2024-05-13T22:48:25.003 INFO:tasks.workunit.client.1.smithi130.stdout:5/130: dwrite d6/d7/f22 [0,4194304] 0 2024-05-13T22:48:25.004 INFO:tasks.workunit.client.1.smithi130.stdout:8/124: chown d1/d2/db/d15/d18 0 1 2024-05-13T22:48:25.005 INFO:tasks.workunit.client.1.smithi130.stdout:2/132: creat d4/db/dc/f37 x:0 0 0 2024-05-13T22:48:25.005 INFO:tasks.workunit.client.0.smithi102.stdout:2/970: mknod d7/d9/d14/c13c 0 2024-05-13T22:48:25.006 INFO:tasks.workunit.client.1.smithi130.stdout:8/125: write d1/d2/f25 [1112330,43604] 0 2024-05-13T22:48:25.009 INFO:tasks.workunit.client.1.smithi130.stdout:5/131: dread d6/dd/d18/f15 [0,4194304] 0 2024-05-13T22:48:25.013 INFO:tasks.workunit.client.0.smithi102.stdout:1/744: dwrite d2/d4/d26/f9a [0,4194304] 0 2024-05-13T22:48:25.013 INFO:tasks.workunit.client.0.smithi102.stdout:4/905: dwrite d3/d19/d23/d4e/f12c [0,4194304] 0 2024-05-13T22:48:25.014 INFO:tasks.workunit.client.0.smithi102.stdout:4/906: stat d3/d4/d6/de/d1f/f28 0 2024-05-13T22:48:25.016 INFO:tasks.workunit.client.1.smithi130.stdout:1/141: unlink db/d1b/c27 0 2024-05-13T22:48:25.018 INFO:tasks.workunit.client.0.smithi102.stdout:9/887: mkdir d1/d6/d21/d12e/d75/d96/dc6/d130 0 2024-05-13T22:48:25.028 INFO:tasks.workunit.client.0.smithi102.stdout:1/745: rename d2/d45/d79/d92/cda to d2/d4/d38/db2/dba/cf5 0 2024-05-13T22:48:25.030 INFO:tasks.workunit.client.1.smithi130.stdout:3/120: truncate d8/dc/d15/f18 787144 0 2024-05-13T22:48:25.036 INFO:tasks.workunit.client.0.smithi102.stdout:0/882: link d1/de/d42/dea/d100/c10c d1/de/d32/d24/d2d/d3e/c113 0 2024-05-13T22:48:25.036 INFO:tasks.workunit.client.0.smithi102.stdout:0/883: chown d1/d68/de2 38977 1 2024-05-13T22:48:25.037 INFO:tasks.workunit.client.1.smithi130.stdout:0/149: write d0/f8 [5189130,12914] 0 2024-05-13T22:48:25.039 INFO:tasks.workunit.client.1.smithi130.stdout:9/141: fsync d1/db/fc 0 2024-05-13T22:48:25.044 INFO:tasks.workunit.client.1.smithi130.stdout:8/126: readlink d1/l14 0 2024-05-13T22:48:25.046 INFO:tasks.workunit.client.0.smithi102.stdout:0/884: creat d1/de/d42/d61/f114 x:0 0 0 2024-05-13T22:48:25.047 INFO:tasks.workunit.client.0.smithi102.stdout:0/885: write d1/de/d32/d24/f40 [1919577,85266] 0 2024-05-13T22:48:25.047 INFO:tasks.workunit.client.1.smithi130.stdout:2/133: symlink d4/db/dc/d18/d2a/d30/l38 0 2024-05-13T22:48:25.051 INFO:tasks.workunit.client.1.smithi130.stdout:8/127: dwrite d1/d2/f25 [0,4194304] 0 2024-05-13T22:48:25.054 INFO:tasks.workunit.client.0.smithi102.stdout:9/888: write d1/d6/d45/da1/fe9 [143814,21075] 0 2024-05-13T22:48:25.055 INFO:tasks.workunit.client.1.smithi130.stdout:1/142: creat db/d1b/f2b x:0 0 0 2024-05-13T22:48:25.056 INFO:tasks.workunit.client.1.smithi130.stdout:2/134: dwrite d4/fd [0,4194304] 0 2024-05-13T22:48:25.057 INFO:tasks.workunit.client.0.smithi102.stdout:4/907: creat d3/d19/d23/d4e/f13d x:0 0 0 2024-05-13T22:48:25.059 INFO:tasks.workunit.client.1.smithi130.stdout:4/123: symlink d8/l2e 0 2024-05-13T22:48:25.063 INFO:tasks.workunit.client.0.smithi102.stdout:0/886: creat d1/de/d32/d24/d2d/d38/d3d/d81/f115 x:0 0 0 2024-05-13T22:48:25.064 INFO:tasks.workunit.client.1.smithi130.stdout:3/121: write d8/dc/d19/f20 [3394484,79014] 0 2024-05-13T22:48:25.064 INFO:tasks.workunit.client.0.smithi102.stdout:2/971: getdents d7/d15/d1e/da7 0 2024-05-13T22:48:25.067 INFO:tasks.workunit.client.0.smithi102.stdout:1/746: creat d2/d4/d5/d19/ded/ff6 x:0 0 0 2024-05-13T22:48:25.070 INFO:tasks.workunit.client.0.smithi102.stdout:4/908: mknod d3/d19/d23/d4e/d68/d85/c13e 0 2024-05-13T22:48:25.071 INFO:tasks.workunit.client.0.smithi102.stdout:4/909: read d3/d4/d6/fb5 [1520291,63814] 0 2024-05-13T22:48:25.075 INFO:tasks.workunit.client.1.smithi130.stdout:9/142: unlink d1/d7/d23/d28/f29 0 2024-05-13T22:48:25.076 INFO:tasks.workunit.client.0.smithi102.stdout:2/972: stat d7/d15/d1e/d87/l11d 0 2024-05-13T22:48:25.077 INFO:tasks.workunit.client.0.smithi102.stdout:0/887: rmdir d1/de/d42/d61 39 2024-05-13T22:48:25.077 INFO:tasks.workunit.client.0.smithi102.stdout:2/973: write d7/d15/d1e/f5c [287351,68832] 0 2024-05-13T22:48:25.083 INFO:tasks.workunit.client.1.smithi130.stdout:5/132: mkdir d6/d27 0 2024-05-13T22:48:25.086 INFO:tasks.workunit.client.1.smithi130.stdout:6/169: truncate d4/d15/d1d/d28/f30 595762 0 2024-05-13T22:48:25.090 INFO:tasks.workunit.client.1.smithi130.stdout:6/170: write d4/d15/d1f/f20 [4028635,111658] 0 2024-05-13T22:48:25.090 INFO:tasks.workunit.client.1.smithi130.stdout:6/171: truncate d4/f2e 498336 0 2024-05-13T22:48:25.090 INFO:tasks.workunit.client.1.smithi130.stdout:6/172: readlink d4/d15/d1f/l3b 0 2024-05-13T22:48:25.090 INFO:tasks.workunit.client.0.smithi102.stdout:0/888: read - d1/de/d42/d61/f114 zero size 2024-05-13T22:48:25.091 INFO:tasks.workunit.client.0.smithi102.stdout:9/889: write d1/d6/d45/f67 [708710,242] 0 2024-05-13T22:48:25.094 INFO:tasks.workunit.client.1.smithi130.stdout:2/135: mknod d4/d33/c39 0 2024-05-13T22:48:25.096 INFO:tasks.workunit.client.1.smithi130.stdout:4/124: unlink d8/f1d 0 2024-05-13T22:48:25.096 INFO:tasks.workunit.client.1.smithi130.stdout:4/125: dread - d8/fa zero size 2024-05-13T22:48:25.099 INFO:tasks.workunit.client.0.smithi102.stdout:1/747: write d2/d10/f44 [1909767,129779] 0 2024-05-13T22:48:25.100 INFO:tasks.workunit.client.0.smithi102.stdout:7/850: sync 2024-05-13T22:48:25.100 INFO:tasks.workunit.client.0.smithi102.stdout:8/867: sync 2024-05-13T22:48:25.101 INFO:tasks.workunit.client.0.smithi102.stdout:7/851: stat d6/d13/d58/caf 0 2024-05-13T22:48:25.101 INFO:tasks.workunit.client.1.smithi130.stdout:7/99: sync 2024-05-13T22:48:25.102 INFO:tasks.workunit.client.1.smithi130.stdout:7/100: chown d7/d8 881163545 1 2024-05-13T22:48:25.102 INFO:tasks.workunit.client.1.smithi130.stdout:0/150: creat d0/d7/d29/f33 x:0 0 0 2024-05-13T22:48:25.106 INFO:tasks.workunit.client.0.smithi102.stdout:8/868: write d2/d7/d27/d64/d80/dc8/d40/db2/fb7 [306306,12902] 0 2024-05-13T22:48:25.106 INFO:tasks.workunit.client.0.smithi102.stdout:4/910: symlink d3/d19/d23/d4e/l13f 0 2024-05-13T22:48:25.108 INFO:tasks.workunit.client.0.smithi102.stdout:8/869: fsync d2/d7/d25/d41/f65 0 2024-05-13T22:48:25.109 INFO:tasks.workunit.client.0.smithi102.stdout:0/889: truncate d1/de/d32/d24/d2d/d38/d3d/f57 2965451 0 2024-05-13T22:48:25.110 INFO:tasks.workunit.client.0.smithi102.stdout:0/890: dread - d1/d4d/d79/d9f/dbd/f105 zero size 2024-05-13T22:48:25.112 INFO:tasks.workunit.client.1.smithi130.stdout:5/133: creat d6/d7/f28 x:0 0 0 2024-05-13T22:48:25.113 INFO:tasks.workunit.client.0.smithi102.stdout:2/974: creat d7/d15/d1e/da7/df4/d11a/f13d x:0 0 0 2024-05-13T22:48:25.114 INFO:tasks.workunit.client.0.smithi102.stdout:1/748: rename d2/d4/d38/db2/fc6 to d2/d4/d5/d19/dc9/ff7 0 2024-05-13T22:48:25.115 INFO:tasks.workunit.client.0.smithi102.stdout:7/852: rename d6/d15/d4d/d55/d6f to d6/d15/d4d/d55/d6f/d102/d10b 22 2024-05-13T22:48:25.115 INFO:tasks.workunit.client.1.smithi130.stdout:8/128: link d1/dc/f24 d1/d2/db/d15/f30 0 2024-05-13T22:48:25.116 INFO:tasks.workunit.client.1.smithi130.stdout:8/129: write d1/d2/db/d15/d16/f1c [709582,2319] 0 2024-05-13T22:48:25.119 INFO:tasks.workunit.client.1.smithi130.stdout:4/126: symlink d8/dc/d29/l2f 0 2024-05-13T22:48:25.120 INFO:tasks.workunit.client.0.smithi102.stdout:9/890: creat d1/d6/db/d30/d122/f131 x:0 0 0 2024-05-13T22:48:25.121 INFO:tasks.workunit.client.1.smithi130.stdout:3/122: creat d8/d1a/d1c/f25 x:0 0 0 2024-05-13T22:48:25.122 INFO:tasks.workunit.client.1.smithi130.stdout:8/130: dwrite d1/d2/db/d15/d18/d1a/f1e [0,4194304] 0 2024-05-13T22:48:25.124 INFO:tasks.workunit.client.1.smithi130.stdout:0/151: mknod d0/d7/d29/c34 0 2024-05-13T22:48:25.125 INFO:tasks.workunit.client.0.smithi102.stdout:4/911: dread d3/d19/d23/d3a/f4a [0,4194304] 0 2024-05-13T22:48:25.126 INFO:tasks.workunit.client.0.smithi102.stdout:1/749: dread - d2/d45/d79/f81 zero size 2024-05-13T22:48:25.127 INFO:tasks.workunit.client.0.smithi102.stdout:2/975: mknod d7/d9/d10/d30/d53/da4/c13e 0 2024-05-13T22:48:25.129 INFO:tasks.workunit.client.0.smithi102.stdout:9/891: dwrite d1/d6/d45/d3e/f73 [0,4194304] 0 2024-05-13T22:48:25.129 INFO:tasks.workunit.client.0.smithi102.stdout:7/853: symlink d6/d15/d4c/l10c 0 2024-05-13T22:48:25.131 INFO:tasks.workunit.client.1.smithi130.stdout:1/143: link db/cf db/d1b/d1f/d2a/c2c 0 2024-05-13T22:48:25.133 INFO:tasks.workunit.client.1.smithi130.stdout:2/136: link d4/f1e d4/db/f3a 0 2024-05-13T22:48:25.134 INFO:tasks.workunit.client.0.smithi102.stdout:1/750: dwrite d2/d10/f44 [0,4194304] 0 2024-05-13T22:48:25.134 INFO:tasks.workunit.client.0.smithi102.stdout:1/751: chown d2/d45 12 1 2024-05-13T22:48:25.143 INFO:tasks.workunit.client.1.smithi130.stdout:0/152: mknod d0/d7/dd/d17/c35 0 2024-05-13T22:48:25.145 INFO:tasks.workunit.client.0.smithi102.stdout:4/912: dread - d3/d19/d23/d39/da7/f120 zero size 2024-05-13T22:48:25.145 INFO:tasks.workunit.client.1.smithi130.stdout:0/153: stat d0/d7/dd/d2e 0 2024-05-13T22:48:25.146 INFO:tasks.workunit.client.1.smithi130.stdout:9/143: creat d1/f2c x:0 0 0 2024-05-13T22:48:25.148 INFO:tasks.workunit.client.1.smithi130.stdout:5/134: creat d6/d27/f29 x:0 0 0 2024-05-13T22:48:25.148 INFO:tasks.workunit.client.1.smithi130.stdout:4/127: rmdir d8/dc 39 2024-05-13T22:48:25.150 INFO:tasks.workunit.client.0.smithi102.stdout:9/892: creat d1/d6b/f132 x:0 0 0 2024-05-13T22:48:25.150 INFO:tasks.workunit.client.1.smithi130.stdout:1/144: chown db/d1b/d1f/d2a/c2c 10800261 1 2024-05-13T22:48:25.155 INFO:tasks.workunit.client.0.smithi102.stdout:4/913: creat d3/dd8/f140 x:0 0 0 2024-05-13T22:48:25.157 INFO:tasks.workunit.client.0.smithi102.stdout:8/870: dwrite d2/d7/d27/d64/d80/dc8/d93/f98 [4194304,4194304] 0 2024-05-13T22:48:25.162 INFO:tasks.workunit.client.1.smithi130.stdout:9/144: dwrite d1/d7/f1a [0,4194304] 0 2024-05-13T22:48:25.163 INFO:tasks.workunit.client.1.smithi130.stdout:5/135: creat d6/d1a/f2a x:0 0 0 2024-05-13T22:48:25.164 INFO:tasks.workunit.client.0.smithi102.stdout:8/871: dwrite d2/d7/d27/d64/ff1 [0,4194304] 0 2024-05-13T22:48:25.165 INFO:tasks.workunit.client.0.smithi102.stdout:0/891: getdents d1/d4d 0 2024-05-13T22:48:25.166 INFO:tasks.workunit.client.0.smithi102.stdout:0/892: chown d1/lc7 668 1 2024-05-13T22:48:25.167 INFO:tasks.workunit.client.1.smithi130.stdout:4/128: dread - d8/dc/d29/f2c zero size 2024-05-13T22:48:25.173 INFO:tasks.workunit.client.1.smithi130.stdout:7/101: getdents d7/da/d1d 0 2024-05-13T22:48:25.174 INFO:tasks.workunit.client.1.smithi130.stdout:7/102: fdatasync d7/d8/f19 0 2024-05-13T22:48:25.175 INFO:tasks.workunit.client.1.smithi130.stdout:7/103: truncate d7/d8/d9/fc 1146941 0 2024-05-13T22:48:25.176 INFO:tasks.workunit.client.1.smithi130.stdout:7/104: chown d7/da/d1d/l26 15641 1 2024-05-13T22:48:25.176 INFO:tasks.workunit.client.1.smithi130.stdout:9/145: rmdir d1/d15 39 2024-05-13T22:48:25.177 INFO:tasks.workunit.client.1.smithi130.stdout:9/146: truncate d1/d7/f27 571050 0 2024-05-13T22:48:25.178 INFO:tasks.workunit.client.0.smithi102.stdout:7/854: dwrite d6/d15/f47 [0,4194304] 0 2024-05-13T22:48:25.178 INFO:tasks.workunit.client.0.smithi102.stdout:8/872: unlink d2/d7/d27/d64/d96/ff3 0 2024-05-13T22:48:25.179 INFO:tasks.workunit.client.1.smithi130.stdout:5/136: creat d6/dd/d18/f2b x:0 0 0 2024-05-13T22:48:25.180 INFO:tasks.workunit.client.0.smithi102.stdout:0/893: rmdir d1/d4d 39 2024-05-13T22:48:25.181 INFO:tasks.workunit.client.1.smithi130.stdout:4/129: creat d8/dc/d26/f30 x:0 0 0 2024-05-13T22:48:25.183 INFO:tasks.workunit.client.0.smithi102.stdout:4/914: creat d3/dee/f141 x:0 0 0 2024-05-13T22:48:25.184 INFO:tasks.workunit.client.1.smithi130.stdout:1/145: mknod db/d1b/d1f/d2a/c2d 0 2024-05-13T22:48:25.185 INFO:tasks.workunit.client.0.smithi102.stdout:1/752: link d2/d4/d5/d19/l41 d2/d4/d5/d19/ded/d6a/lf8 0 2024-05-13T22:48:25.185 INFO:tasks.workunit.client.1.smithi130.stdout:2/137: creat d4/db/dc/f3b x:0 0 0 2024-05-13T22:48:25.186 INFO:tasks.workunit.client.1.smithi130.stdout:5/137: dwrite d6/d7/f25 [0,4194304] 0 2024-05-13T22:48:25.186 INFO:tasks.workunit.client.0.smithi102.stdout:1/753: readlink d2/d4/d5/d19/ded/la7 0 2024-05-13T22:48:25.188 INFO:tasks.workunit.client.0.smithi102.stdout:7/855: creat d6/d15/d4c/f10d x:0 0 0 2024-05-13T22:48:25.190 INFO:tasks.workunit.client.0.smithi102.stdout:7/856: truncate d6/dd0/df2/ffa 955913 0 2024-05-13T22:48:25.195 INFO:tasks.workunit.client.0.smithi102.stdout:4/915: unlink d3/d22/dab/cf2 0 2024-05-13T22:48:25.196 INFO:tasks.workunit.client.0.smithi102.stdout:1/754: rmdir d2/d4/d5/d19/dc9 39 2024-05-13T22:48:25.199 INFO:tasks.workunit.client.0.smithi102.stdout:4/916: rename d3/d19/d23/d4e/f72 to d3/d4/d6/de/d1f/f142 0 2024-05-13T22:48:25.200 INFO:tasks.workunit.client.0.smithi102.stdout:4/917: chown d3/d19/d23/d4e/d68/d85/fd0 114 1 2024-05-13T22:48:25.203 INFO:tasks.workunit.client.0.smithi102.stdout:1/755: creat d2/d4/d5/df0/ff9 x:0 0 0 2024-05-13T22:48:25.204 INFO:tasks.workunit.client.0.smithi102.stdout:1/756: chown d2/fc2 55242 1 2024-05-13T22:48:25.206 INFO:tasks.workunit.client.0.smithi102.stdout:4/918: rename d3/d4/d6/de/d1f/d41/f8a to d3/d19/d23/d39/d5d/da5/d102/f143 0 2024-05-13T22:48:25.214 INFO:tasks.workunit.client.0.smithi102.stdout:1/757: link l1 d2/d4/d5/d19/dc9/lfa 0 2024-05-13T22:48:25.216 INFO:tasks.workunit.client.0.smithi102.stdout:1/758: fsync d2/d4/d26/d40/d5e/f80 0 2024-05-13T22:48:25.217 INFO:tasks.workunit.client.1.smithi130.stdout:6/173: sync 2024-05-13T22:48:25.217 INFO:tasks.workunit.client.0.smithi102.stdout:8/873: write d2/d7/d27/d64/d80/dc8/d40/d85/fb9 [768777,15783] 0 2024-05-13T22:48:25.218 INFO:tasks.workunit.client.0.smithi102.stdout:1/759: write d2/d4/d5/d19/dbe/fc1 [57082,41799] 0 2024-05-13T22:48:25.219 INFO:tasks.workunit.client.1.smithi130.stdout:7/105: readlink l6 0 2024-05-13T22:48:25.219 INFO:tasks.workunit.client.1.smithi130.stdout:9/147: mkdir d1/d7/d23/d28/d2d 0 2024-05-13T22:48:25.220 INFO:tasks.workunit.client.0.smithi102.stdout:1/760: fsync d2/d4/d5/d19/ded/ff6 0 2024-05-13T22:48:25.221 INFO:tasks.workunit.client.1.smithi130.stdout:0/154: truncate d0/d7/dd/d17/d1f/f25 7513103 0 2024-05-13T22:48:25.221 INFO:tasks.workunit.client.1.smithi130.stdout:6/174: chown d4/d15/d1d/f3d 9342792 1 2024-05-13T22:48:25.222 INFO:tasks.workunit.client.1.smithi130.stdout:2/138: write d4/db/f3a [893652,123677] 0 2024-05-13T22:48:25.222 INFO:tasks.workunit.client.0.smithi102.stdout:0/894: dread d1/de/d32/d24/d2d/f53 [0,4194304] 0 2024-05-13T22:48:25.225 INFO:tasks.workunit.client.1.smithi130.stdout:1/146: mkdir db/d2e 0 2024-05-13T22:48:25.227 INFO:tasks.workunit.client.0.smithi102.stdout:7/857: dwrite d6/f94 [0,4194304] 0 2024-05-13T22:48:25.230 INFO:tasks.workunit.client.0.smithi102.stdout:4/919: write d3/d19/d23/d39/da7/da9/dd3/f10e [708809,16861] 0 2024-05-13T22:48:25.231 INFO:tasks.workunit.client.0.smithi102.stdout:1/761: truncate d2/d4/d5/fa 3959486 0 2024-05-13T22:48:25.231 INFO:tasks.workunit.client.0.smithi102.stdout:4/920: readlink d3/d19/d23/d7e/la8 0 2024-05-13T22:48:25.232 INFO:tasks.workunit.client.0.smithi102.stdout:1/762: read - d2/d4/d5/fe2 zero size 2024-05-13T22:48:25.233 INFO:tasks.workunit.client.1.smithi130.stdout:7/106: symlink d7/da/d1d/l28 0 2024-05-13T22:48:25.235 INFO:tasks.workunit.client.0.smithi102.stdout:0/895: mkdir d1/de/d32/d8a/d116 0 2024-05-13T22:48:25.237 INFO:tasks.workunit.client.0.smithi102.stdout:0/896: dread d1/fad [0,4194304] 0 2024-05-13T22:48:25.240 INFO:tasks.workunit.client.0.smithi102.stdout:4/921: unlink d3/d4/d6/c48 0 2024-05-13T22:48:25.241 INFO:tasks.workunit.client.1.smithi130.stdout:4/130: dwrite f5 [0,4194304] 0 2024-05-13T22:48:25.243 INFO:tasks.workunit.client.1.smithi130.stdout:0/155: symlink d0/d7/dd/d2e/l36 0 2024-05-13T22:48:25.245 INFO:tasks.workunit.client.0.smithi102.stdout:7/858: dread d6/d13/d62/ff0 [0,4194304] 0 2024-05-13T22:48:25.247 INFO:tasks.workunit.client.1.smithi130.stdout:4/131: dwrite f5 [0,4194304] 0 2024-05-13T22:48:25.248 INFO:tasks.workunit.client.1.smithi130.stdout:4/132: stat d8/dc/d26/f30 0 2024-05-13T22:48:25.249 INFO:tasks.workunit.client.1.smithi130.stdout:4/133: write d8/df/f20 [268967,36339] 0 2024-05-13T22:48:25.250 INFO:tasks.workunit.client.1.smithi130.stdout:4/134: chown d8/f1f 18464 1 2024-05-13T22:48:25.250 INFO:tasks.workunit.client.1.smithi130.stdout:2/139: dread d4/f5 [4194304,4194304] 0 2024-05-13T22:48:25.251 INFO:tasks.workunit.client.1.smithi130.stdout:2/140: write d4/d33/f35 [370830,62306] 0 2024-05-13T22:48:25.253 INFO:tasks.workunit.client.0.smithi102.stdout:4/922: fsync d3/d19/d23/d39/da7/da9/dd3/dd4/f117 0 2024-05-13T22:48:25.253 INFO:tasks.workunit.client.0.smithi102.stdout:8/874: dwrite d2/d7/d27/d64/d80/dc8/d93/dab/fc3 [0,4194304] 0 2024-05-13T22:48:25.255 INFO:tasks.workunit.client.0.smithi102.stdout:8/875: write d2/d7/d25/d41/d6f/f84 [4741982,24153] 0 2024-05-13T22:48:25.259 INFO:tasks.workunit.client.0.smithi102.stdout:0/897: dread d1/de/d32/d24/f40 [0,4194304] 0 2024-05-13T22:48:25.281 INFO:tasks.workunit.client.1.smithi130.stdout:3/123: sync 2024-05-13T22:48:25.337 INFO:tasks.workunit.client.1.smithi130.stdout:8/131: sync 2024-05-13T22:48:25.352 INFO:tasks.workunit.client.1.smithi130.stdout:9/148: dwrite d1/db/fc [0,4194304] 0 2024-05-13T22:48:25.355 INFO:tasks.workunit.client.1.smithi130.stdout:1/147: rename db/f26 to db/d1b/d1f/d2a/f2f 0 2024-05-13T22:48:25.359 INFO:tasks.workunit.client.1.smithi130.stdout:5/138: link d6/d1a/f2a d6/d7/d19/f2c 0 2024-05-13T22:48:25.361 INFO:tasks.workunit.client.1.smithi130.stdout:7/107: rmdir d7/da 39 2024-05-13T22:48:25.362 INFO:tasks.workunit.client.1.smithi130.stdout:7/108: chown d7/d8/f13 945923285 1 2024-05-13T22:48:25.362 INFO:tasks.workunit.client.1.smithi130.stdout:1/148: dwrite db/d1b/f2b [0,4194304] 0 2024-05-13T22:48:25.364 INFO:tasks.workunit.client.0.smithi102.stdout:4/923: creat d3/d4/d6/de/d8d/dec/f144 x:0 0 0 2024-05-13T22:48:25.367 INFO:tasks.workunit.client.1.smithi130.stdout:0/156: creat d0/d7/d29/f37 x:0 0 0 2024-05-13T22:48:25.367 INFO:tasks.workunit.client.1.smithi130.stdout:0/157: chown d0/f27 12465 1 2024-05-13T22:48:25.368 INFO:tasks.workunit.client.0.smithi102.stdout:8/876: creat d2/d36/d49/f114 x:0 0 0 2024-05-13T22:48:25.370 INFO:tasks.workunit.client.1.smithi130.stdout:4/135: creat d8/dc/d26/d2a/f31 x:0 0 0 2024-05-13T22:48:25.370 INFO:tasks.workunit.client.1.smithi130.stdout:4/136: fdatasync d8/dc/d29/f2c 0 2024-05-13T22:48:25.371 INFO:tasks.workunit.client.0.smithi102.stdout:0/898: readlink d1/de/d32/d24/d2d/d3e/l101 0 2024-05-13T22:48:25.371 INFO:tasks.workunit.client.1.smithi130.stdout:4/137: write f5 [222269,22644] 0 2024-05-13T22:48:25.372 INFO:tasks.workunit.client.1.smithi130.stdout:2/141: symlink d4/d33/l3c 0 2024-05-13T22:48:25.372 INFO:tasks.workunit.client.1.smithi130.stdout:2/142: write d4/db/f2c [1578463,117277] 0 2024-05-13T22:48:25.373 INFO:tasks.workunit.client.1.smithi130.stdout:2/143: stat d4/db/dc/d18/c31 0 2024-05-13T22:48:25.374 INFO:tasks.workunit.client.1.smithi130.stdout:6/175: link d4/d16/f39 d4/d15/d1f/f3e 0 2024-05-13T22:48:25.376 INFO:tasks.workunit.client.1.smithi130.stdout:3/124: creat d8/f26 x:0 0 0 2024-05-13T22:48:25.377 INFO:tasks.workunit.client.1.smithi130.stdout:3/125: chown d8/cb 2277 1 2024-05-13T22:48:25.379 INFO:tasks.workunit.client.1.smithi130.stdout:3/126: write d8/dc/f21 [991213,13679] 0 2024-05-13T22:48:25.380 INFO:tasks.workunit.client.1.smithi130.stdout:2/144: dwrite d4/db/f3a [0,4194304] 0 2024-05-13T22:48:25.382 INFO:tasks.workunit.client.1.smithi130.stdout:2/145: dread - d4/db/f2e zero size 2024-05-13T22:48:25.382 INFO:tasks.workunit.client.0.smithi102.stdout:1/763: rename d2/d4/d5/d19/ded/d6a/lf8 to d2/d4/d5/d19/lfb 0 2024-05-13T22:48:25.382 INFO:tasks.workunit.client.1.smithi130.stdout:9/149: symlink d1/d7/d23/d28/l2e 0 2024-05-13T22:48:25.383 INFO:tasks.workunit.client.0.smithi102.stdout:8/877: mkdir d2/d7/d27/d64/d80/dc8/d115 0 2024-05-13T22:48:25.384 INFO:tasks.workunit.client.1.smithi130.stdout:2/146: write d4/db/f22 [919749,98094] 0 2024-05-13T22:48:25.387 INFO:tasks.workunit.client.1.smithi130.stdout:3/127: dwrite d8/f26 [0,4194304] 0 2024-05-13T22:48:25.388 INFO:tasks.workunit.client.1.smithi130.stdout:5/139: truncate d6/d7/fa 195830 0 2024-05-13T22:48:25.391 INFO:tasks.workunit.client.1.smithi130.stdout:3/128: dread d8/fa [0,4194304] 0 2024-05-13T22:48:25.392 INFO:tasks.workunit.client.0.smithi102.stdout:1/764: dwrite d2/d45/d79/d92/f9d [4194304,4194304] 0 2024-05-13T22:48:25.401 INFO:tasks.workunit.client.0.smithi102.stdout:8/878: dwrite d2/d7/d27/d64/d96/dae/d76/f9f [0,4194304] 0 2024-05-13T22:48:25.404 INFO:tasks.workunit.client.1.smithi130.stdout:6/176: unlink d4/c6 0 2024-05-13T22:48:25.406 INFO:tasks.workunit.client.0.smithi102.stdout:1/765: unlink d2/d4/d5/d19/dd3/cea 0 2024-05-13T22:48:25.411 INFO:tasks.workunit.client.0.smithi102.stdout:1/766: dread d2/d45/d79/d92/f61 [0,4194304] 0 2024-05-13T22:48:25.412 INFO:tasks.workunit.client.0.smithi102.stdout:1/767: dread - d2/d10/d62/d7b/fd9 zero size 2024-05-13T22:48:25.412 INFO:tasks.workunit.client.1.smithi130.stdout:6/177: dwrite d4/d15/d1d/d28/f34 [0,4194304] 0 2024-05-13T22:48:25.413 INFO:tasks.workunit.client.0.smithi102.stdout:1/768: write d2/d4/d5/d19/f4e [2789996,129934] 0 2024-05-13T22:48:25.414 INFO:tasks.workunit.client.0.smithi102.stdout:8/879: symlink d2/d7/d27/d64/d80/dc8/d93/l116 0 2024-05-13T22:48:25.416 INFO:tasks.workunit.client.0.smithi102.stdout:8/880: dread - d2/d36/d49/d5d/ff0 zero size 2024-05-13T22:48:25.418 INFO:tasks.workunit.client.1.smithi130.stdout:5/140: creat d6/d27/f2d x:0 0 0 2024-05-13T22:48:25.419 INFO:tasks.workunit.client.1.smithi130.stdout:5/141: truncate d6/d7/f28 703445 0 2024-05-13T22:48:25.420 INFO:tasks.workunit.client.1.smithi130.stdout:1/149: symlink db/l30 0 2024-05-13T22:48:25.422 INFO:tasks.workunit.client.1.smithi130.stdout:5/142: dread d6/dd/d18/f15 [0,4194304] 0 2024-05-13T22:48:25.425 INFO:tasks.workunit.client.0.smithi102.stdout:1/769: fsync d2/d4/d5/d19/ded/f35 0 2024-05-13T22:48:25.427 INFO:tasks.workunit.client.1.smithi130.stdout:0/158: mknod d0/d7/c38 0 2024-05-13T22:48:25.431 INFO:tasks.workunit.client.0.smithi102.stdout:7/859: truncate d6/dd0/df2/ffa 545809 0 2024-05-13T22:48:25.432 INFO:tasks.workunit.client.1.smithi130.stdout:8/132: write d1/d2/fd [2057686,16509] 0 2024-05-13T22:48:25.433 INFO:tasks.workunit.client.1.smithi130.stdout:8/133: dread - d1/d2/db/d15/d16/f2c zero size 2024-05-13T22:48:25.437 INFO:tasks.workunit.client.1.smithi130.stdout:9/150: mkdir d1/d7/d23/d28/d2d/d2f 0 2024-05-13T22:48:25.441 INFO:tasks.workunit.client.1.smithi130.stdout:9/151: dread d1/db/fc [0,4194304] 0 2024-05-13T22:48:25.442 INFO:tasks.workunit.client.0.smithi102.stdout:1/770: creat d2/d4/d26/da2/ffc x:0 0 0 2024-05-13T22:48:25.443 INFO:tasks.workunit.client.0.smithi102.stdout:0/899: dwrite d1/de/d32/f72 [0,4194304] 0 2024-05-13T22:48:25.445 INFO:tasks.workunit.client.0.smithi102.stdout:1/771: dread d2/d45/f58 [0,4194304] 0 2024-05-13T22:48:25.446 INFO:tasks.workunit.client.0.smithi102.stdout:1/772: truncate d2/d4/fb5 260303 0 2024-05-13T22:48:25.447 INFO:tasks.workunit.client.0.smithi102.stdout:1/773: fsync d2/d4/d5/d19/f53 0 2024-05-13T22:48:25.450 INFO:tasks.workunit.client.0.smithi102.stdout:8/881: link d2/d7/d27/d64/d80/dc8/d93/f10c d2/d7/d27/d64/d80/dc8/d40/db2/f117 0 2024-05-13T22:48:25.454 INFO:tasks.workunit.client.1.smithi130.stdout:7/109: dwrite d7/d8/f13 [0,4194304] 0 2024-05-13T22:48:25.454 INFO:tasks.workunit.client.1.smithi130.stdout:3/129: mkdir d8/dc/d19/d27 0 2024-05-13T22:48:25.456 INFO:tasks.workunit.client.0.smithi102.stdout:7/860: creat d6/d15/d4d/d55/d6f/d65/ded/f10e x:0 0 0 2024-05-13T22:48:25.457 INFO:tasks.workunit.client.0.smithi102.stdout:7/861: write d6/da9/fbd [747882,47679] 0 2024-05-13T22:48:25.458 INFO:tasks.workunit.client.0.smithi102.stdout:7/862: fsync d6/d15/d4c/f8c 0 2024-05-13T22:48:25.460 INFO:tasks.workunit.client.1.smithi130.stdout:4/138: link c0 d8/dc/d13/c32 0 2024-05-13T22:48:25.460 INFO:tasks.workunit.client.1.smithi130.stdout:7/110: dwrite d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:25.466 INFO:tasks.workunit.client.0.smithi102.stdout:8/882: creat d2/d7/d27/d64/d80/dc8/d40/d85/f118 x:0 0 0 2024-05-13T22:48:25.470 INFO:tasks.workunit.client.1.smithi130.stdout:1/150: creat db/d2e/f31 x:0 0 0 2024-05-13T22:48:25.471 INFO:tasks.workunit.client.0.smithi102.stdout:0/900: symlink d1/de/d32/d24/d2d/d38/d3d/l117 0 2024-05-13T22:48:25.473 INFO:tasks.workunit.client.0.smithi102.stdout:7/863: mknod d6/de1/de6/c10f 0 2024-05-13T22:48:25.475 INFO:tasks.workunit.client.0.smithi102.stdout:1/774: rename d2/d45/d79/d92/ld8 to d2/d10/d62/d7b/dc4/lfd 0 2024-05-13T22:48:25.477 INFO:tasks.workunit.client.0.smithi102.stdout:1/775: write d2/d4/d5/d19/dbe/fc1 [1119614,66162] 0 2024-05-13T22:48:25.478 INFO:tasks.workunit.client.1.smithi130.stdout:2/147: sync 2024-05-13T22:48:25.479 INFO:tasks.workunit.client.1.smithi130.stdout:2/148: dread - d4/db/dc/d18/f36 zero size 2024-05-13T22:48:25.481 INFO:tasks.workunit.client.0.smithi102.stdout:7/864: rename d6/dd0/l40 to d6/d13/d2c/da1/l110 0 2024-05-13T22:48:25.482 INFO:tasks.workunit.client.1.smithi130.stdout:2/149: dread d4/fa [0,4194304] 0 2024-05-13T22:48:25.482 INFO:tasks.workunit.client.0.smithi102.stdout:7/865: fdatasync d6/d15/d4c/f98 0 2024-05-13T22:48:25.482 INFO:tasks.workunit.client.1.smithi130.stdout:2/150: read - d4/db/f2e zero size 2024-05-13T22:48:25.483 INFO:tasks.workunit.client.1.smithi130.stdout:2/151: write d4/d33/f35 [1245575,120473] 0 2024-05-13T22:48:25.486 INFO:tasks.workunit.client.0.smithi102.stdout:8/883: link d2/d36/d49/f114 d2/d7/d27/d64/d96/f119 0 2024-05-13T22:48:25.489 INFO:tasks.workunit.client.0.smithi102.stdout:0/901: stat d1/d4d/cf2 0 2024-05-13T22:48:25.490 INFO:tasks.workunit.client.1.smithi130.stdout:2/152: dwrite d4/db/f22 [0,4194304] 0 2024-05-13T22:48:25.490 INFO:tasks.workunit.client.1.smithi130.stdout:2/153: fsync d4/fd 0 2024-05-13T22:48:25.491 INFO:tasks.workunit.client.1.smithi130.stdout:2/154: fdatasync d4/f7 0 2024-05-13T22:48:25.492 INFO:tasks.workunit.client.1.smithi130.stdout:7/111: symlink d7/da/l29 0 2024-05-13T22:48:25.493 INFO:tasks.workunit.client.0.smithi102.stdout:1/776: dread d2/d10/d62/d7b/f7c [0,4194304] 0 2024-05-13T22:48:25.493 INFO:tasks.workunit.client.0.smithi102.stdout:7/866: readlink d6/l25 0 2024-05-13T22:48:25.495 INFO:tasks.workunit.client.1.smithi130.stdout:8/134: creat d1/d2/db/d15/d18/f31 x:0 0 0 2024-05-13T22:48:25.502 INFO:tasks.workunit.client.0.smithi102.stdout:0/902: read d1/de/d32/d8a/fac [2029386,102648] 0 2024-05-13T22:48:25.502 INFO:tasks.workunit.client.1.smithi130.stdout:0/159: write d0/f27 [2460186,595] 0 2024-05-13T22:48:25.510 INFO:tasks.workunit.client.0.smithi102.stdout:8/884: mkdir d2/d7/d27/d64/d96/dae/d11a 0 2024-05-13T22:48:25.511 INFO:tasks.workunit.client.0.smithi102.stdout:1/777: dread d2/d10/d62/d7b/dc4/fcc [0,4194304] 0 2024-05-13T22:48:25.512 INFO:tasks.workunit.client.0.smithi102.stdout:1/778: fdatasync d2/d10/fa0 0 2024-05-13T22:48:25.513 INFO:tasks.workunit.client.1.smithi130.stdout:5/143: write d6/f11 [5533069,48517] 0 2024-05-13T22:48:25.513 INFO:tasks.workunit.client.0.smithi102.stdout:1/779: dread - d2/d4/d5/df0/ff9 zero size 2024-05-13T22:48:25.514 INFO:tasks.workunit.client.0.smithi102.stdout:1/780: stat d2/d10/d62/d7b/dc4 0 2024-05-13T22:48:25.515 INFO:tasks.workunit.client.0.smithi102.stdout:1/781: chown d2/d4/d38/db2 141547059 1 2024-05-13T22:48:25.516 INFO:tasks.workunit.client.0.smithi102.stdout:1/782: fdatasync d2/d10/d62/d7b/f82 0 2024-05-13T22:48:25.517 INFO:tasks.workunit.client.1.smithi130.stdout:8/135: unlink d1/d2/db/l2f 0 2024-05-13T22:48:25.522 INFO:tasks.workunit.client.1.smithi130.stdout:1/151: rename db/d1b/f2b to db/f32 0 2024-05-13T22:48:25.522 INFO:tasks.workunit.client.1.smithi130.stdout:9/152: write d1/d15/f25 [447690,112265] 0 2024-05-13T22:48:25.524 INFO:tasks.workunit.client.1.smithi130.stdout:3/130: getdents d8/dc/d19 0 2024-05-13T22:48:25.525 INFO:tasks.workunit.client.0.smithi102.stdout:1/783: creat d2/d4/d5/d19/ded/d6a/dbd/ffe x:0 0 0 2024-05-13T22:48:25.525 INFO:tasks.workunit.client.1.smithi130.stdout:3/131: write d8/dc/d19/f20 [4751067,84512] 0 2024-05-13T22:48:25.526 INFO:tasks.workunit.client.1.smithi130.stdout:3/132: fsync d8/f9 0 2024-05-13T22:48:25.526 INFO:tasks.workunit.client.0.smithi102.stdout:0/903: dread d1/de/d32/d24/d2d/d91/fd7 [0,4194304] 0 2024-05-13T22:48:25.527 INFO:tasks.workunit.client.1.smithi130.stdout:1/152: dread db/d1b/d1f/d2a/f2f [0,4194304] 0 2024-05-13T22:48:25.528 INFO:tasks.workunit.client.1.smithi130.stdout:5/144: symlink d6/dd/d18/l2e 0 2024-05-13T22:48:25.530 INFO:tasks.workunit.client.0.smithi102.stdout:8/885: rename d2/f6d to d2/d7/d27/d64/d96/dbc/d106/f11b 0 2024-05-13T22:48:25.536 INFO:tasks.workunit.client.1.smithi130.stdout:7/112: rename d7/d8/d9 to d7/d8/d9/d2a 22 2024-05-13T22:48:25.537 INFO:tasks.workunit.client.0.smithi102.stdout:7/867: write d6/da9/fb5 [366223,109934] 0 2024-05-13T22:48:25.537 INFO:tasks.workunit.client.0.smithi102.stdout:0/904: chown d1/de/d32/f36 1604 1 2024-05-13T22:48:25.538 INFO:tasks.workunit.client.0.smithi102.stdout:0/905: readlink d1/de/l7b 0 2024-05-13T22:48:25.540 INFO:tasks.workunit.client.0.smithi102.stdout:0/906: write d1/de/d32/dab/dfb/ffd [752412,51246] 0 2024-05-13T22:48:25.546 INFO:tasks.workunit.client.1.smithi130.stdout:6/178: sync 2024-05-13T22:48:25.547 INFO:tasks.workunit.client.1.smithi130.stdout:2/155: link d4/db/c32 d4/db/c3d 0 2024-05-13T22:48:25.554 INFO:tasks.workunit.client.1.smithi130.stdout:5/145: unlink d6/d7/l8 0 2024-05-13T22:48:25.556 INFO:tasks.workunit.client.1.smithi130.stdout:5/146: write d6/dd/f13 [1350819,24030] 0 2024-05-13T22:48:25.557 INFO:tasks.workunit.client.1.smithi130.stdout:8/136: mknod d1/c32 0 2024-05-13T22:48:25.557 INFO:tasks.workunit.client.1.smithi130.stdout:9/153: dwrite d1/d15/f1d [0,4194304] 0 2024-05-13T22:48:25.560 INFO:tasks.workunit.client.1.smithi130.stdout:9/154: dread d1/d15/f25 [0,4194304] 0 2024-05-13T22:48:25.562 INFO:tasks.workunit.client.1.smithi130.stdout:7/113: rmdir d7/d8 39 2024-05-13T22:48:25.563 INFO:tasks.workunit.client.1.smithi130.stdout:1/153: write db/d1b/f20 [1492989,68452] 0 2024-05-13T22:48:25.564 INFO:tasks.workunit.client.1.smithi130.stdout:6/179: mknod d4/d15/d1f/c3f 0 2024-05-13T22:48:25.567 INFO:tasks.workunit.client.1.smithi130.stdout:7/114: dread d7/da/d1f/f20 [0,4194304] 0 2024-05-13T22:48:25.568 INFO:tasks.workunit.client.1.smithi130.stdout:2/156: symlink d4/db/dc/d18/d2a/l3e 0 2024-05-13T22:48:25.570 INFO:tasks.workunit.client.1.smithi130.stdout:3/133: creat d8/dc/d19/d27/f28 x:0 0 0 2024-05-13T22:48:25.571 INFO:tasks.workunit.client.1.smithi130.stdout:2/157: dread d4/db/dc/f25 [0,4194304] 0 2024-05-13T22:48:25.571 INFO:tasks.workunit.client.1.smithi130.stdout:2/158: write d4/f21 [209070,14467] 0 2024-05-13T22:48:25.572 INFO:tasks.workunit.client.1.smithi130.stdout:2/159: write d4/f1a [410909,127875] 0 2024-05-13T22:48:25.581 INFO:tasks.workunit.client.1.smithi130.stdout:0/160: link d0/d7/dd/d17/d1f/d2c/l2d d0/d7/d29/l39 0 2024-05-13T22:48:25.584 INFO:tasks.workunit.client.1.smithi130.stdout:9/155: dread d1/db/fc [0,4194304] 0 2024-05-13T22:48:25.586 INFO:tasks.workunit.client.1.smithi130.stdout:1/154: rename db/l10 to db/d1b/d1f/d2a/l33 0 2024-05-13T22:48:25.594 INFO:tasks.workunit.client.1.smithi130.stdout:7/115: dwrite d7/d8/d9/fc [0,4194304] 0 2024-05-13T22:48:25.595 INFO:tasks.workunit.client.1.smithi130.stdout:2/160: creat d4/f3f x:0 0 0 2024-05-13T22:48:25.596 INFO:tasks.workunit.client.1.smithi130.stdout:7/116: truncate d7/d8/f13 4516580 0 2024-05-13T22:48:25.597 INFO:tasks.workunit.client.1.smithi130.stdout:5/147: mkdir d6/d2f 0 2024-05-13T22:48:25.601 INFO:tasks.workunit.client.1.smithi130.stdout:5/148: dwrite d6/dd/f13 [0,4194304] 0 2024-05-13T22:48:25.601 INFO:tasks.workunit.client.1.smithi130.stdout:5/149: chown d6/dd/d18/l14 3 1 2024-05-13T22:48:25.610 INFO:tasks.workunit.client.1.smithi130.stdout:1/155: creat db/d1b/d1f/d2a/f34 x:0 0 0 2024-05-13T22:48:25.611 INFO:tasks.workunit.client.0.smithi102.stdout:2/976: sync 2024-05-13T22:48:25.618 INFO:tasks.workunit.client.1.smithi130.stdout:7/117: mkdir d7/da/d1f/d2b 0 2024-05-13T22:48:25.619 INFO:tasks.workunit.client.1.smithi130.stdout:7/118: truncate d7/da/d1f/f23 562142 0 2024-05-13T22:48:25.622 INFO:tasks.workunit.client.1.smithi130.stdout:8/137: creat d1/f33 x:0 0 0 2024-05-13T22:48:25.623 INFO:tasks.workunit.client.1.smithi130.stdout:5/150: mknod d6/d7/d19/c30 0 2024-05-13T22:48:25.628 INFO:tasks.workunit.client.1.smithi130.stdout:2/161: symlink d4/db/dc/l40 0 2024-05-13T22:48:25.629 INFO:tasks.workunit.client.1.smithi130.stdout:9/156: dread d1/f3 [0,4194304] 0 2024-05-13T22:48:25.629 INFO:tasks.workunit.client.1.smithi130.stdout:7/119: unlink d7/da/d1f/f23 0 2024-05-13T22:48:25.630 INFO:tasks.workunit.client.1.smithi130.stdout:0/161: dwrite d0/fe [0,4194304] 0 2024-05-13T22:48:25.630 INFO:tasks.workunit.client.1.smithi130.stdout:4/139: sync 2024-05-13T22:48:25.631 INFO:tasks.workunit.client.1.smithi130.stdout:0/162: stat d0/d7/dd/d17 0 2024-05-13T22:48:25.632 INFO:tasks.workunit.client.1.smithi130.stdout:8/138: write d1/dc/f21 [5215489,48946] 0 2024-05-13T22:48:25.633 INFO:tasks.workunit.client.1.smithi130.stdout:5/151: rename d6/d7/d19/c30 to d6/d7/c31 0 2024-05-13T22:48:25.633 INFO:tasks.workunit.client.1.smithi130.stdout:8/139: fsync d1/d2/db/d15/f22 0 2024-05-13T22:48:25.638 INFO:tasks.workunit.client.1.smithi130.stdout:6/180: getdents d4/d16 0 2024-05-13T22:48:25.638 INFO:tasks.workunit.client.1.smithi130.stdout:6/181: chown d4/d15/d1f/f2f 44221326 1 2024-05-13T22:48:25.642 INFO:tasks.workunit.client.1.smithi130.stdout:6/182: dread d4/d15/d1d/d28/f34 [0,4194304] 0 2024-05-13T22:48:25.643 INFO:tasks.workunit.client.1.smithi130.stdout:2/162: creat d4/db/f41 x:0 0 0 2024-05-13T22:48:25.644 INFO:tasks.workunit.client.1.smithi130.stdout:7/120: rename d7/da/d1f/d2b to d7/da/d1f/d2c 0 2024-05-13T22:48:25.646 INFO:tasks.workunit.client.1.smithi130.stdout:4/140: mkdir d8/dc/d26/d2a/d33 0 2024-05-13T22:48:25.648 INFO:tasks.workunit.client.1.smithi130.stdout:0/163: mknod d0/d7/dd/d17/d1f/d2c/c3a 0 2024-05-13T22:48:25.650 INFO:tasks.workunit.client.1.smithi130.stdout:0/164: dread - d0/d7/d29/f37 zero size 2024-05-13T22:48:25.650 INFO:tasks.workunit.client.1.smithi130.stdout:7/121: dwrite d7/d8/f1c [0,4194304] 0 2024-05-13T22:48:25.651 INFO:tasks.workunit.client.1.smithi130.stdout:9/157: rename d1/d7/f2a to d1/d7/d23/d28/d2d/f30 0 2024-05-13T22:48:25.651 INFO:tasks.workunit.client.1.smithi130.stdout:1/156: write db/f14 [2210944,61213] 0 2024-05-13T22:48:25.657 INFO:tasks.workunit.client.1.smithi130.stdout:2/163: mkdir d4/d33/d42 0 2024-05-13T22:48:25.661 INFO:tasks.workunit.client.1.smithi130.stdout:6/183: symlink d4/d15/d1d/d28/l40 0 2024-05-13T22:48:25.662 INFO:tasks.workunit.client.0.smithi102.stdout:2/977: fsync d7/d15/f19 0 2024-05-13T22:48:25.666 INFO:tasks.workunit.client.0.smithi102.stdout:1/784: rename d2/d10/d62/d86/d91 to d2/d4/d5/d19/ded/dff 0 2024-05-13T22:48:25.667 INFO:tasks.workunit.client.0.smithi102.stdout:1/785: fdatasync d2/d10/d62/ff3 0 2024-05-13T22:48:25.670 INFO:tasks.workunit.client.0.smithi102.stdout:0/907: mknod d1/de/d42/d61/d107/c118 0 2024-05-13T22:48:25.673 INFO:tasks.workunit.client.1.smithi130.stdout:7/122: unlink d7/d8/d9/fc 0 2024-05-13T22:48:25.674 INFO:tasks.workunit.client.1.smithi130.stdout:7/123: chown d7/da/d1d/l28 346472 1 2024-05-13T22:48:25.674 INFO:tasks.workunit.client.1.smithi130.stdout:8/140: dread d1/d2/db/d15/f29 [0,4194304] 0 2024-05-13T22:48:25.675 INFO:tasks.workunit.client.1.smithi130.stdout:0/165: unlink d0/d7/dd/d17/f1b 0 2024-05-13T22:48:25.678 INFO:tasks.workunit.client.1.smithi130.stdout:5/152: rename d6/d1a/d21 to d6/d7/d32 0 2024-05-13T22:48:25.680 INFO:tasks.workunit.client.0.smithi102.stdout:7/868: rename d6/d15/d4d/d55/d6f/d49/c69 to d6/d13/d2c/d39/c111 0 2024-05-13T22:48:25.680 INFO:tasks.workunit.client.1.smithi130.stdout:0/166: dwrite d0/fe [0,4194304] 0 2024-05-13T22:48:25.681 INFO:tasks.workunit.client.1.smithi130.stdout:3/134: sync 2024-05-13T22:48:25.682 INFO:tasks.workunit.client.0.smithi102.stdout:1/786: symlink d2/d45/d96/l100 0 2024-05-13T22:48:25.682 INFO:tasks.workunit.client.1.smithi130.stdout:9/158: truncate d1/f1e 444507 0 2024-05-13T22:48:25.682 INFO:tasks.workunit.client.1.smithi130.stdout:3/135: read f3 [2468453,106247] 0 2024-05-13T22:48:25.683 INFO:tasks.workunit.client.1.smithi130.stdout:2/164: dread d4/db/dc/f15 [0,4194304] 0 2024-05-13T22:48:25.683 INFO:tasks.workunit.client.0.smithi102.stdout:8/886: write d2/d7/d25/fb0 [1010831,34814] 0 2024-05-13T22:48:25.683 INFO:tasks.workunit.client.0.smithi102.stdout:1/787: write d2/d10/fa0 [492331,45192] 0 2024-05-13T22:48:25.684 INFO:tasks.workunit.client.0.smithi102.stdout:0/908: creat d1/de/d32/dab/f119 x:0 0 0 2024-05-13T22:48:25.684 INFO:tasks.workunit.client.1.smithi130.stdout:2/165: stat d4/db 0 2024-05-13T22:48:25.685 INFO:tasks.workunit.client.0.smithi102.stdout:8/887: readlink d2/d7/d27/d64/d80/dc8/d93/l116 0 2024-05-13T22:48:25.686 INFO:tasks.workunit.client.0.smithi102.stdout:2/978: symlink d7/d9/d14/l13f 0 2024-05-13T22:48:25.686 INFO:tasks.workunit.client.1.smithi130.stdout:3/136: dread d8/dc/f21 [0,4194304] 0 2024-05-13T22:48:25.691 INFO:tasks.workunit.client.0.smithi102.stdout:1/788: fsync d2/d10/f5d 0 2024-05-13T22:48:25.694 INFO:tasks.workunit.client.1.smithi130.stdout:3/137: dwrite d8/d1a/d1c/f25 [0,4194304] 0 2024-05-13T22:48:25.694 INFO:tasks.workunit.client.0.smithi102.stdout:8/888: dread - d2/d7/d27/d64/d80/dc8/d40/f9c zero size 2024-05-13T22:48:25.694 INFO:tasks.workunit.client.0.smithi102.stdout:4/924: sync 2024-05-13T22:48:25.694 INFO:tasks.workunit.client.0.smithi102.stdout:9/893: sync 2024-05-13T22:48:25.698 INFO:tasks.workunit.client.1.smithi130.stdout:4/141: dwrite f6 [0,4194304] 0 2024-05-13T22:48:25.703 INFO:tasks.workunit.client.0.smithi102.stdout:8/889: dwrite d2/d7/d25/d41/d6f/f82 [0,4194304] 0 2024-05-13T22:48:25.708 INFO:tasks.workunit.client.0.smithi102.stdout:9/894: dwrite d1/d6/d21/d12e/d81/d116/f11e [0,4194304] 0 2024-05-13T22:48:25.714 INFO:tasks.workunit.client.1.smithi130.stdout:1/157: rename db/d2e to db/d1b/d1f/d2a/d35 0 2024-05-13T22:48:25.715 INFO:tasks.workunit.client.0.smithi102.stdout:9/895: dwrite d1/d6b/f10b [4194304,4194304] 0 2024-05-13T22:48:25.717 INFO:tasks.workunit.client.0.smithi102.stdout:1/789: creat d2/d45/d79/f101 x:0 0 0 2024-05-13T22:48:25.718 INFO:tasks.workunit.client.0.smithi102.stdout:0/909: creat d1/de/d32/d8a/d116/f11a x:0 0 0 2024-05-13T22:48:25.719 INFO:tasks.workunit.client.0.smithi102.stdout:1/790: chown d2/d4/d5/d19/dd3 65288 1 2024-05-13T22:48:25.721 INFO:tasks.workunit.client.0.smithi102.stdout:0/910: dread - d1/d68/ff3 zero size 2024-05-13T22:48:25.728 INFO:tasks.workunit.client.0.smithi102.stdout:4/925: mkdir d3/d6b/d145 0 2024-05-13T22:48:25.732 INFO:tasks.workunit.client.1.smithi130.stdout:6/184: symlink d4/l41 0 2024-05-13T22:48:25.733 INFO:tasks.workunit.client.1.smithi130.stdout:6/185: write d4/d16/f2a [1037480,19128] 0 2024-05-13T22:48:25.734 INFO:tasks.workunit.client.1.smithi130.stdout:6/186: chown d4/d15/c27 61 1 2024-05-13T22:48:25.736 INFO:tasks.workunit.client.1.smithi130.stdout:9/159: write d1/f3 [1224669,43366] 0 2024-05-13T22:48:25.738 INFO:tasks.workunit.client.1.smithi130.stdout:3/138: mknod d8/dc/d19/d1d/c29 0 2024-05-13T22:48:25.739 INFO:tasks.workunit.client.1.smithi130.stdout:3/139: fsync d8/dc/f22 0 2024-05-13T22:48:25.741 INFO:tasks.workunit.client.1.smithi130.stdout:4/142: symlink d8/dc/d29/l34 0 2024-05-13T22:48:25.742 INFO:tasks.workunit.client.1.smithi130.stdout:4/143: readlink d8/l9 0 2024-05-13T22:48:25.746 INFO:tasks.workunit.client.0.smithi102.stdout:1/791: creat d2/d4/d38/db2/f102 x:0 0 0 2024-05-13T22:48:25.750 INFO:tasks.workunit.client.0.smithi102.stdout:8/890: dread d2/f10b [0,4194304] 0 2024-05-13T22:48:25.751 INFO:tasks.workunit.client.0.smithi102.stdout:8/891: fdatasync d2/d7/d27/d64/d96/dae/d78/fa7 0 2024-05-13T22:48:25.752 INFO:tasks.workunit.client.0.smithi102.stdout:2/979: link d7/l138 d7/d9/d14/d61/l140 0 2024-05-13T22:48:25.755 INFO:tasks.workunit.client.0.smithi102.stdout:7/869: link d6/d13/d58/fc9 d6/d15/f112 0 2024-05-13T22:48:25.756 INFO:tasks.workunit.client.1.smithi130.stdout:6/187: truncate d4/d15/d1f/f2f 4860499 0 2024-05-13T22:48:25.756 INFO:tasks.workunit.client.0.smithi102.stdout:7/870: chown d6/d22/f33 458 1 2024-05-13T22:48:25.758 INFO:tasks.workunit.client.0.smithi102.stdout:9/896: fsync d1/d6/d21/d12e/d75/d96/dc6/dde/f128 0 2024-05-13T22:48:25.760 INFO:tasks.workunit.client.1.smithi130.stdout:3/140: mknod d8/dc/d19/d27/c2a 0 2024-05-13T22:48:25.762 INFO:tasks.workunit.client.1.smithi130.stdout:6/188: dwrite d4/d15/f31 [0,4194304] 0 2024-05-13T22:48:25.763 INFO:tasks.workunit.client.1.smithi130.stdout:7/124: creat d7/d8/f2d x:0 0 0 2024-05-13T22:48:25.763 INFO:tasks.workunit.client.0.smithi102.stdout:1/792: creat d2/d4/d5/d19/dbe/f103 x:0 0 0 2024-05-13T22:48:25.764 INFO:tasks.workunit.client.0.smithi102.stdout:1/793: stat d2/d10/f85 0 2024-05-13T22:48:25.767 INFO:tasks.workunit.client.1.smithi130.stdout:4/144: rename d8/dc/d26/d2a to d8/dc/d35 0 2024-05-13T22:48:25.768 INFO:tasks.workunit.client.0.smithi102.stdout:4/926: dwrite d3/d4/d6/de/d1f/d41/fc2 [0,4194304] 0 2024-05-13T22:48:25.768 INFO:tasks.workunit.client.1.smithi130.stdout:4/145: fdatasync d8/dc/d26/f30 0 2024-05-13T22:48:25.769 INFO:tasks.workunit.client.1.smithi130.stdout:7/125: dread d7/d8/f1c [0,4194304] 0 2024-05-13T22:48:25.771 INFO:tasks.workunit.client.1.smithi130.stdout:9/160: write d1/d7/fa [2287390,61175] 0 2024-05-13T22:48:25.772 INFO:tasks.workunit.client.1.smithi130.stdout:6/189: dread d4/d15/d1d/f22 [0,4194304] 0 2024-05-13T22:48:25.773 INFO:tasks.workunit.client.0.smithi102.stdout:8/892: fsync d2/d7/d27/d64/d96/dae/f4f 0 2024-05-13T22:48:25.778 INFO:tasks.workunit.client.0.smithi102.stdout:7/871: rmdir d6/dd0 39 2024-05-13T22:48:25.779 INFO:tasks.workunit.client.1.smithi130.stdout:8/141: sync 2024-05-13T22:48:25.780 INFO:tasks.workunit.client.0.smithi102.stdout:0/911: dwrite d1/de/d32/f37 [0,4194304] 0 2024-05-13T22:48:25.781 INFO:tasks.workunit.client.0.smithi102.stdout:0/912: fdatasync d1/de/d32/d24/d2d/d38/db7/feb 0 2024-05-13T22:48:25.787 INFO:tasks.workunit.client.1.smithi130.stdout:4/146: rename d8/dc/fd to d8/dc/d26/f36 0 2024-05-13T22:48:25.788 INFO:tasks.workunit.client.0.smithi102.stdout:2/980: dwrite d7/d15/d1e/dbf/fe9 [0,4194304] 0 2024-05-13T22:48:25.794 INFO:tasks.workunit.client.0.smithi102.stdout:4/927: rmdir d3/d19/d23/d39/da7/da9/dd3/dd4 39 2024-05-13T22:48:25.795 INFO:tasks.workunit.client.1.smithi130.stdout:3/141: dread d8/dc/d15/f18 [0,4194304] 0 2024-05-13T22:48:25.796 INFO:tasks.workunit.client.1.smithi130.stdout:2/166: getdents d4/db/dc 0 2024-05-13T22:48:25.796 INFO:tasks.workunit.client.0.smithi102.stdout:8/893: mknod d2/d7/d27/d64/d80/dc8/d40/c11c 0 2024-05-13T22:48:25.796 INFO:tasks.workunit.client.1.smithi130.stdout:2/167: dread - d4/db/f2e zero size 2024-05-13T22:48:25.798 INFO:tasks.workunit.client.1.smithi130.stdout:6/190: creat d4/d15/d1f/f42 x:0 0 0 2024-05-13T22:48:25.800 INFO:tasks.workunit.client.0.smithi102.stdout:7/872: creat d6/de1/de6/f113 x:0 0 0 2024-05-13T22:48:25.802 INFO:tasks.workunit.client.0.smithi102.stdout:1/794: mknod d2/c104 0 2024-05-13T22:48:25.803 INFO:tasks.workunit.client.1.smithi130.stdout:4/147: symlink d8/dc/d35/l37 0 2024-05-13T22:48:25.804 INFO:tasks.workunit.client.1.smithi130.stdout:4/148: truncate d8/f1e 641093 0 2024-05-13T22:48:25.806 INFO:tasks.workunit.client.1.smithi130.stdout:0/167: sync 2024-05-13T22:48:25.806 INFO:tasks.workunit.client.1.smithi130.stdout:5/153: sync 2024-05-13T22:48:25.806 INFO:tasks.workunit.client.0.smithi102.stdout:0/913: rename d1/d4d/d79/d9f/dbd to d1/de/d32/d8a/d116/d11b 0 2024-05-13T22:48:25.806 INFO:tasks.workunit.client.0.smithi102.stdout:7/873: dwrite f4 [0,4194304] 0 2024-05-13T22:48:25.807 INFO:tasks.workunit.client.1.smithi130.stdout:1/158: getdents db/d1b 0 2024-05-13T22:48:25.809 INFO:tasks.workunit.client.1.smithi130.stdout:0/168: write d0/d7/f22 [295765,121538] 0 2024-05-13T22:48:25.810 INFO:tasks.workunit.client.1.smithi130.stdout:5/154: chown d6/d7/d19/f1c 23963108 1 2024-05-13T22:48:25.810 INFO:tasks.workunit.client.0.smithi102.stdout:4/928: unlink d3/d4/d6/de/d1f/d41/c6c 0 2024-05-13T22:48:25.812 INFO:tasks.workunit.client.1.smithi130.stdout:9/161: mknod d1/d7/d23/d28/d2d/d2f/c31 0 2024-05-13T22:48:25.812 INFO:tasks.workunit.client.1.smithi130.stdout:0/169: chown d0/d7/l1d 4578579 1 2024-05-13T22:48:25.813 INFO:tasks.workunit.client.0.smithi102.stdout:8/894: creat d2/d7/d27/d64/f11d x:0 0 0 2024-05-13T22:48:25.815 INFO:tasks.workunit.client.1.smithi130.stdout:4/149: dwrite d8/dc/d29/f2c [0,4194304] 0 2024-05-13T22:48:25.817 INFO:tasks.workunit.client.0.smithi102.stdout:9/897: creat d1/d6/d45/d2d/f133 x:0 0 0 2024-05-13T22:48:25.818 INFO:tasks.workunit.client.1.smithi130.stdout:2/168: creat d4/db/dc/f43 x:0 0 0 2024-05-13T22:48:25.821 INFO:tasks.workunit.client.1.smithi130.stdout:2/169: stat d4/fd 0 2024-05-13T22:48:25.821 INFO:tasks.workunit.client.1.smithi130.stdout:1/159: dread db/d1b/f20 [0,4194304] 0 2024-05-13T22:48:25.821 INFO:tasks.workunit.client.1.smithi130.stdout:3/142: link d8/d1a/d1c/f25 d8/f2b 0 2024-05-13T22:48:25.822 INFO:tasks.workunit.client.1.smithi130.stdout:0/170: truncate d0/d7/dd/f32 901311 0 2024-05-13T22:48:25.826 INFO:tasks.workunit.client.0.smithi102.stdout:2/981: mknod d7/d9/d14/d31/deb/c141 0 2024-05-13T22:48:25.826 INFO:tasks.workunit.client.1.smithi130.stdout:0/171: readlink d0/d7/dd/lf 0 2024-05-13T22:48:25.828 INFO:tasks.workunit.client.1.smithi130.stdout:9/162: mkdir d1/d7/d23/d28/d32 0 2024-05-13T22:48:25.829 INFO:tasks.workunit.client.1.smithi130.stdout:5/155: dwrite d6/d7/d19/f1c [0,4194304] 0 2024-05-13T22:48:25.830 INFO:tasks.workunit.client.1.smithi130.stdout:6/191: dread d4/f9 [0,4194304] 0 2024-05-13T22:48:25.832 INFO:tasks.workunit.client.1.smithi130.stdout:4/150: dwrite f5 [0,4194304] 0 2024-05-13T22:48:25.834 INFO:tasks.workunit.client.0.smithi102.stdout:1/795: mkdir d2/d4/d105 0 2024-05-13T22:48:25.834 INFO:tasks.workunit.client.0.smithi102.stdout:2/982: dwrite d7/d9/d14/f133 [0,4194304] 0 2024-05-13T22:48:25.835 INFO:tasks.workunit.client.1.smithi130.stdout:5/156: read f3 [3830940,55921] 0 2024-05-13T22:48:25.835 INFO:tasks.workunit.client.1.smithi130.stdout:6/192: write d4/d15/d1f/f26 [168790,48499] 0 2024-05-13T22:48:25.838 INFO:tasks.workunit.client.1.smithi130.stdout:9/163: dwrite d1/d15/f16 [0,4194304] 0 2024-05-13T22:48:25.838 INFO:tasks.workunit.client.1.smithi130.stdout:8/142: truncate d1/d2/f6 3785830 0 2024-05-13T22:48:25.848 INFO:tasks.workunit.client.0.smithi102.stdout:1/796: dwrite d2/d4/d26/f9a [0,4194304] 0 2024-05-13T22:48:25.853 INFO:tasks.workunit.client.0.smithi102.stdout:0/914: write d1/de/f50 [3896027,36764] 0 2024-05-13T22:48:25.853 INFO:tasks.workunit.client.0.smithi102.stdout:4/929: creat d3/d19/d23/d39/da7/da9/dd3/dd4/f146 x:0 0 0 2024-05-13T22:48:25.857 INFO:tasks.workunit.client.1.smithi130.stdout:0/172: mknod d0/d7/d29/c3b 0 2024-05-13T22:48:25.859 INFO:tasks.workunit.client.1.smithi130.stdout:0/173: fdatasync d0/d7/dd/d17/d1f/d2c/f31 0 2024-05-13T22:48:25.860 INFO:tasks.workunit.client.0.smithi102.stdout:2/983: chown d7/d15/l3e 138228 1 2024-05-13T22:48:25.863 INFO:tasks.workunit.client.1.smithi130.stdout:7/126: rename d7/c1e to d7/d8/d9/c2e 0 2024-05-13T22:48:25.865 INFO:tasks.workunit.client.0.smithi102.stdout:8/895: dread d2/d7/d27/d64/d80/dc8/d40/fe0 [0,4194304] 0 2024-05-13T22:48:25.867 INFO:tasks.workunit.client.0.smithi102.stdout:8/896: chown d2/d7/d27/d64/d96/dae/f111 153 1 2024-05-13T22:48:25.868 INFO:tasks.workunit.client.1.smithi130.stdout:4/151: dread d8/df/f24 [0,4194304] 0 2024-05-13T22:48:25.868 INFO:tasks.workunit.client.1.smithi130.stdout:4/152: dread - d8/f12 zero size 2024-05-13T22:48:25.869 INFO:tasks.workunit.client.1.smithi130.stdout:4/153: readlink d8/dc/d35/l37 0 2024-05-13T22:48:25.870 INFO:tasks.workunit.client.0.smithi102.stdout:9/898: write d1/d6/d21/d12e/d81/fe8 [1491167,130601] 0 2024-05-13T22:48:25.870 INFO:tasks.workunit.client.1.smithi130.stdout:4/154: dread - d8/fa zero size 2024-05-13T22:48:25.873 INFO:tasks.workunit.client.1.smithi130.stdout:6/193: symlink d4/d15/d1f/l43 0 2024-05-13T22:48:25.874 INFO:tasks.workunit.client.1.smithi130.stdout:6/194: dread - d4/d15/d1f/f42 zero size 2024-05-13T22:48:25.876 INFO:tasks.workunit.client.1.smithi130.stdout:8/143: fsync d1/f2d 0 2024-05-13T22:48:25.878 INFO:tasks.workunit.client.0.smithi102.stdout:9/899: chown d1/d6/d45/dd0/d105 853 1 2024-05-13T22:48:25.879 INFO:tasks.workunit.client.0.smithi102.stdout:4/930: mkdir d3/d19/dad/dc9/d147 0 2024-05-13T22:48:25.880 INFO:tasks.workunit.client.1.smithi130.stdout:3/143: write d8/dc/d15/f18 [1641757,45433] 0 2024-05-13T22:48:25.881 INFO:tasks.workunit.client.1.smithi130.stdout:6/195: dwrite d4/d15/d1f/f2d [0,4194304] 0 2024-05-13T22:48:25.884 INFO:tasks.workunit.client.1.smithi130.stdout:7/127: symlink d7/da/d1f/d2c/l2f 0 2024-05-13T22:48:25.885 INFO:tasks.workunit.client.0.smithi102.stdout:9/900: mknod d1/d6/db/d30/d8a/c134 0 2024-05-13T22:48:25.887 INFO:tasks.workunit.client.1.smithi130.stdout:4/155: creat d8/dc/f38 x:0 0 0 2024-05-13T22:48:25.892 INFO:tasks.workunit.client.1.smithi130.stdout:4/156: dwrite f7 [0,4194304] 0 2024-05-13T22:48:25.899 INFO:tasks.workunit.client.0.smithi102.stdout:2/984: write d7/d15/d1e/da7/f10d [226528,74428] 0 2024-05-13T22:48:25.900 INFO:tasks.workunit.client.0.smithi102.stdout:7/874: dwrite d6/dd0/d80/f96 [0,4194304] 0 2024-05-13T22:48:25.901 INFO:tasks.workunit.client.0.smithi102.stdout:8/897: dread d2/d7/d25/f6e [0,4194304] 0 2024-05-13T22:48:25.906 INFO:tasks.workunit.client.0.smithi102.stdout:4/931: link d3/d4/d6/de/d1f/f101 d3/d19/dad/dc9/d147/f148 0 2024-05-13T22:48:25.906 INFO:tasks.workunit.client.0.smithi102.stdout:1/797: dwrite d2/ff [0,4194304] 0 2024-05-13T22:48:25.907 INFO:tasks.workunit.client.0.smithi102.stdout:7/875: chown d6/dd0/df2/d105/f107 26595546 1 2024-05-13T22:48:25.907 INFO:tasks.workunit.client.0.smithi102.stdout:8/898: write d2/d36/d49/d5d/fbd [3433121,49559] 0 2024-05-13T22:48:25.911 INFO:tasks.workunit.client.0.smithi102.stdout:1/798: fdatasync d2/d4/d26/ff1 0 2024-05-13T22:48:25.911 INFO:tasks.workunit.client.0.smithi102.stdout:8/899: stat d2/d7/d27/d64/d80/dc8/c3a 0 2024-05-13T22:48:25.912 INFO:tasks.workunit.client.1.smithi130.stdout:3/144: fdatasync d8/fa 0 2024-05-13T22:48:25.914 INFO:tasks.workunit.client.0.smithi102.stdout:0/915: dwrite d1/de/d32/d24/f62 [0,4194304] 0 2024-05-13T22:48:25.914 INFO:tasks.workunit.client.0.smithi102.stdout:1/799: write d2/d4/d5/d19/f4e [1416123,54958] 0 2024-05-13T22:48:25.921 INFO:tasks.workunit.client.0.smithi102.stdout:4/932: dwrite d3/dee/f141 [0,4194304] 0 2024-05-13T22:48:25.923 INFO:tasks.workunit.client.1.smithi130.stdout:6/196: unlink d4/d15/d1f/f42 0 2024-05-13T22:48:25.926 INFO:tasks.workunit.client.1.smithi130.stdout:7/128: creat d7/da/d1f/f30 x:0 0 0 2024-05-13T22:48:25.927 INFO:tasks.workunit.client.1.smithi130.stdout:7/129: truncate d7/d8/f2d 874783 0 2024-05-13T22:48:25.929 INFO:tasks.workunit.client.0.smithi102.stdout:9/901: dwrite d1/d6/db/d30/fe7 [0,4194304] 0 2024-05-13T22:48:25.931 INFO:tasks.workunit.client.0.smithi102.stdout:7/876: mkdir d6/d13/d2c/d39/d114 0 2024-05-13T22:48:25.934 INFO:tasks.workunit.client.0.smithi102.stdout:9/902: chown d1/d6/d21/d12e/d75/fe3 0 1 2024-05-13T22:48:25.935 INFO:tasks.workunit.client.0.smithi102.stdout:1/800: dwrite d2/d10/d62/d7b/dc4/fc8 [0,4194304] 0 2024-05-13T22:48:25.937 INFO:tasks.workunit.client.1.smithi130.stdout:0/174: write d0/d7/dd/d17/d1f/f25 [98245,54167] 0 2024-05-13T22:48:25.938 INFO:tasks.workunit.client.1.smithi130.stdout:0/175: write d0/d7/dd/d2e/f2f [747434,21637] 0 2024-05-13T22:48:25.940 INFO:tasks.workunit.client.1.smithi130.stdout:4/157: mknod d8/dc/d26/c39 0 2024-05-13T22:48:25.940 INFO:tasks.workunit.client.1.smithi130.stdout:9/164: creat d1/f33 x:0 0 0 2024-05-13T22:48:25.943 INFO:tasks.workunit.client.1.smithi130.stdout:7/130: read d7/d8/f2d [382707,4840] 0 2024-05-13T22:48:25.945 INFO:tasks.workunit.client.1.smithi130.stdout:8/144: mknod d1/c34 0 2024-05-13T22:48:25.945 INFO:tasks.workunit.client.0.smithi102.stdout:2/985: dwrite d7/d15/fad [0,4194304] 0 2024-05-13T22:48:25.950 INFO:tasks.workunit.client.1.smithi130.stdout:1/160: link db/d1b/d1f/l24 db/l36 0 2024-05-13T22:48:25.953 INFO:tasks.workunit.client.0.smithi102.stdout:7/877: fdatasync d6/d15/d4d/d55/d6f/f9a 0 2024-05-13T22:48:25.955 INFO:tasks.workunit.client.0.smithi102.stdout:7/878: chown d6/d15/d4d/d55/d6f/d49/d95/fb7 116403335 1 2024-05-13T22:48:25.957 INFO:tasks.workunit.client.0.smithi102.stdout:1/801: symlink d2/d4/d38/db2/dba/l106 0 2024-05-13T22:48:25.960 INFO:tasks.workunit.client.0.smithi102.stdout:8/900: dwrite d2/d36/f42 [0,4194304] 0 2024-05-13T22:48:26.039 INFO:tasks.workunit.client.1.smithi130.stdout:5/157: getdents d6/d1a 0 2024-05-13T22:48:26.040 INFO:tasks.workunit.client.1.smithi130.stdout:6/197: dwrite d4/d15/d1d/d28/f34 [0,4194304] 0 2024-05-13T22:48:26.041 INFO:tasks.workunit.client.1.smithi130.stdout:6/198: chown d4/d15/d1d/f38 5 1 2024-05-13T22:48:26.046 INFO:tasks.workunit.client.0.smithi102.stdout:4/933: dwrite d3/d19/d23/d3a/f51 [0,4194304] 0 2024-05-13T22:48:26.050 INFO:tasks.workunit.client.1.smithi130.stdout:3/145: dwrite f3 [0,4194304] 0 2024-05-13T22:48:26.057 INFO:tasks.workunit.client.0.smithi102.stdout:4/934: dwrite d3/d19/d23/d4e/f132 [0,4194304] 0 2024-05-13T22:48:26.059 INFO:tasks.workunit.client.1.smithi130.stdout:3/146: dread d8/dc/d15/f18 [0,4194304] 0 2024-05-13T22:48:26.060 INFO:tasks.workunit.client.1.smithi130.stdout:3/147: read d8/dc/d15/f18 [892197,24369] 0 2024-05-13T22:48:26.060 INFO:tasks.workunit.client.0.smithi102.stdout:4/935: read d3/d4/f32 [233835,56854] 0 2024-05-13T22:48:26.061 INFO:tasks.workunit.client.1.smithi130.stdout:9/165: creat d1/d7/d23/d28/d2d/f34 x:0 0 0 2024-05-13T22:48:26.064 INFO:tasks.workunit.client.0.smithi102.stdout:9/903: creat d1/d6/d3b/d5b/d127/f135 x:0 0 0 2024-05-13T22:48:26.067 INFO:tasks.workunit.client.1.smithi130.stdout:2/170: link d4/db/dc/f14 d4/db/dc/d18/d2a/f44 0 2024-05-13T22:48:26.069 INFO:tasks.workunit.client.0.smithi102.stdout:2/986: dread d7/d9/d14/d61/d102/d131/f3b [0,4194304] 0 2024-05-13T22:48:26.075 INFO:tasks.workunit.client.0.smithi102.stdout:9/904: dwrite d1/d6/db/d30/d8a/f99 [0,4194304] 0 2024-05-13T22:48:26.078 INFO:tasks.workunit.client.1.smithi130.stdout:1/161: mknod db/d1b/d1f/d2a/c37 0 2024-05-13T22:48:26.078 INFO:tasks.workunit.client.1.smithi130.stdout:1/162: chown db/f14 1513 1 2024-05-13T22:48:26.087 INFO:tasks.workunit.client.1.smithi130.stdout:5/158: rmdir d6/d1a 39 2024-05-13T22:48:26.088 INFO:tasks.workunit.client.1.smithi130.stdout:6/199: stat d4/d15/d1f/f2f 0 2024-05-13T22:48:26.089 INFO:tasks.workunit.client.1.smithi130.stdout:6/200: chown d4/c7 838 1 2024-05-13T22:48:26.089 INFO:tasks.workunit.client.1.smithi130.stdout:6/201: stat d4/d15/c17 0 2024-05-13T22:48:26.093 INFO:tasks.workunit.client.1.smithi130.stdout:3/148: rmdir d8/dc/d19 39 2024-05-13T22:48:26.094 INFO:tasks.workunit.client.0.smithi102.stdout:4/936: dread d3/d4/d6/fb5 [0,4194304] 0 2024-05-13T22:48:26.097 INFO:tasks.workunit.client.1.smithi130.stdout:9/166: creat d1/db/f35 x:0 0 0 2024-05-13T22:48:26.097 INFO:tasks.workunit.client.1.smithi130.stdout:9/167: chown d1/f8 23946206 1 2024-05-13T22:48:26.098 INFO:tasks.workunit.client.0.smithi102.stdout:2/987: truncate d7/d9/d14/d31/dbb/dd5/fd8 139831 0 2024-05-13T22:48:26.099 INFO:tasks.workunit.client.0.smithi102.stdout:2/988: chown d7/d9/d10/fe7 29977 1 2024-05-13T22:48:26.100 INFO:tasks.workunit.client.0.smithi102.stdout:2/989: chown d7/d9/d14/l22 1356915 1 2024-05-13T22:48:26.100 INFO:tasks.workunit.client.1.smithi130.stdout:2/171: chown d4/db/dc/f37 17 1 2024-05-13T22:48:26.103 INFO:tasks.workunit.client.0.smithi102.stdout:2/990: read d7/d9/d10/d30/d53/f114 [588,130644] 0 2024-05-13T22:48:26.103 INFO:tasks.workunit.client.0.smithi102.stdout:9/905: dread d1/d6/db/f66 [0,4194304] 0 2024-05-13T22:48:26.104 INFO:tasks.workunit.client.1.smithi130.stdout:9/168: dwrite d1/f4 [0,4194304] 0 2024-05-13T22:48:26.104 INFO:tasks.workunit.client.1.smithi130.stdout:1/163: mkdir db/d1b/d1f/d2a/d38 0 2024-05-13T22:48:26.105 INFO:tasks.workunit.client.0.smithi102.stdout:0/916: getdents d1/d4d/d79/d9f 0 2024-05-13T22:48:26.105 INFO:tasks.workunit.client.0.smithi102.stdout:1/802: dread d2/d4/d5/d19/ded/f4d [0,4194304] 0 2024-05-13T22:48:26.109 INFO:tasks.workunit.client.1.smithi130.stdout:5/159: creat d6/dd/f33 x:0 0 0 2024-05-13T22:48:26.110 INFO:tasks.workunit.client.1.smithi130.stdout:2/172: dwrite d4/db/dc/d18/f1d [0,4194304] 0 2024-05-13T22:48:26.110 INFO:tasks.workunit.client.0.smithi102.stdout:4/937: unlink d3/d4/d6/de/c6d 0 2024-05-13T22:48:26.112 INFO:tasks.workunit.client.1.smithi130.stdout:5/160: write d6/dd/d18/f2b [944449,97185] 0 2024-05-13T22:48:26.120 INFO:tasks.workunit.client.1.smithi130.stdout:5/161: dwrite d6/dd/f33 [0,4194304] 0 2024-05-13T22:48:26.124 INFO:tasks.workunit.client.0.smithi102.stdout:1/803: mkdir d2/d4/d5/d19/ded/d6a/dbd/d107 0 2024-05-13T22:48:26.133 INFO:tasks.workunit.client.0.smithi102.stdout:0/917: truncate d1/f9 3349305 0 2024-05-13T22:48:26.134 INFO:tasks.workunit.client.1.smithi130.stdout:4/158: getdents d8/dc/d26 0 2024-05-13T22:48:26.137 INFO:tasks.workunit.client.1.smithi130.stdout:3/149: mkdir d8/dc/d19/d2c 0 2024-05-13T22:48:26.140 INFO:tasks.workunit.client.0.smithi102.stdout:4/938: link d3/d19/d23/d39/da7/d7d/lbd d3/d19/d23/d39/da7/da9/d135/l149 0 2024-05-13T22:48:26.141 INFO:tasks.workunit.client.1.smithi130.stdout:7/131: getdents d7/da/d1f 0 2024-05-13T22:48:26.145 INFO:tasks.workunit.client.1.smithi130.stdout:2/173: symlink d4/d33/d42/l45 0 2024-05-13T22:48:26.149 INFO:tasks.workunit.client.1.smithi130.stdout:3/150: dread d8/dc/f21 [0,4194304] 0 2024-05-13T22:48:26.149 INFO:tasks.workunit.client.1.smithi130.stdout:5/162: link d6/d27/f29 d6/d27/f34 0 2024-05-13T22:48:26.151 INFO:tasks.workunit.client.1.smithi130.stdout:7/132: creat d7/da/d1f/d2c/f31 x:0 0 0 2024-05-13T22:48:26.153 INFO:tasks.workunit.client.1.smithi130.stdout:2/174: symlink d4/d33/l46 0 2024-05-13T22:48:26.154 INFO:tasks.workunit.client.1.smithi130.stdout:4/159: getdents d8/dc/d35/d33 0 2024-05-13T22:48:26.155 INFO:tasks.workunit.client.0.smithi102.stdout:0/918: dread d1/d68/f86 [0,4194304] 0 2024-05-13T22:48:26.156 INFO:tasks.workunit.client.1.smithi130.stdout:4/160: read d8/f1b [606797,28709] 0 2024-05-13T22:48:26.156 INFO:tasks.workunit.client.1.smithi130.stdout:3/151: dread d8/d1a/d1c/f25 [0,4194304] 0 2024-05-13T22:48:26.159 INFO:tasks.workunit.client.1.smithi130.stdout:2/175: creat d4/db/dc/d18/d2a/d30/f47 x:0 0 0 2024-05-13T22:48:26.160 INFO:tasks.workunit.client.1.smithi130.stdout:5/163: creat d6/d7/d24/f35 x:0 0 0 2024-05-13T22:48:26.161 INFO:tasks.workunit.client.1.smithi130.stdout:7/133: dwrite d7/da/d1f/d2c/f31 [0,4194304] 0 2024-05-13T22:48:26.168 INFO:tasks.workunit.client.1.smithi130.stdout:4/161: creat d8/dc/d29/f3a x:0 0 0 2024-05-13T22:48:26.171 INFO:tasks.workunit.client.1.smithi130.stdout:2/176: mkdir d4/d33/d48 0 2024-05-13T22:48:26.171 INFO:tasks.workunit.client.1.smithi130.stdout:2/177: chown d4/f9 7516951 1 2024-05-13T22:48:26.172 INFO:tasks.workunit.client.1.smithi130.stdout:5/164: creat d6/dd/d18/f36 x:0 0 0 2024-05-13T22:48:26.173 INFO:tasks.workunit.client.1.smithi130.stdout:4/162: unlink d8/l16 0 2024-05-13T22:48:26.173 INFO:tasks.workunit.client.1.smithi130.stdout:5/165: truncate d6/dd/ff 508382 0 2024-05-13T22:48:26.176 INFO:tasks.workunit.client.1.smithi130.stdout:4/163: mknod d8/df/c3b 0 2024-05-13T22:48:26.179 INFO:tasks.workunit.client.1.smithi130.stdout:5/166: rename d6/d7/d24/f35 to d6/dd/f37 0 2024-05-13T22:48:26.181 INFO:tasks.workunit.client.1.smithi130.stdout:4/164: dwrite f7 [0,4194304] 0 2024-05-13T22:48:26.182 INFO:tasks.workunit.client.1.smithi130.stdout:4/165: chown f6 22724 1 2024-05-13T22:48:26.184 INFO:tasks.workunit.client.1.smithi130.stdout:5/167: creat d6/d7/d32/f38 x:0 0 0 2024-05-13T22:48:26.185 INFO:tasks.workunit.client.1.smithi130.stdout:5/168: chown d6/dd/d18/fc 13433714 1 2024-05-13T22:48:26.186 INFO:tasks.workunit.client.1.smithi130.stdout:4/166: creat d8/df/f3c x:0 0 0 2024-05-13T22:48:26.187 INFO:tasks.workunit.client.1.smithi130.stdout:5/169: write d6/d7/f22 [4319584,117718] 0 2024-05-13T22:48:26.188 INFO:tasks.workunit.client.1.smithi130.stdout:5/170: chown d6/d7/d32/c23 28818660 1 2024-05-13T22:48:26.192 INFO:tasks.workunit.client.1.smithi130.stdout:6/202: rmdir d4/d15/d1f 39 2024-05-13T22:48:26.194 INFO:tasks.workunit.client.1.smithi130.stdout:6/203: truncate d4/fc 5745939 0 2024-05-13T22:48:26.195 INFO:tasks.workunit.client.1.smithi130.stdout:4/167: dwrite d8/dc/d29/f2c [4194304,4194304] 0 2024-05-13T22:48:26.199 INFO:tasks.workunit.client.1.smithi130.stdout:4/168: symlink d8/df/l3d 0 2024-05-13T22:48:26.200 INFO:tasks.workunit.client.1.smithi130.stdout:2/178: dread d4/db/dc/f20 [0,4194304] 0 2024-05-13T22:48:26.208 INFO:tasks.workunit.client.1.smithi130.stdout:2/179: dread d4/db/f28 [0,4194304] 0 2024-05-13T22:48:26.209 INFO:tasks.workunit.client.1.smithi130.stdout:6/204: getdents d4/d15/d1f 0 2024-05-13T22:48:26.210 INFO:tasks.workunit.client.1.smithi130.stdout:2/180: write d4/db/dc/d18/f36 [846517,92380] 0 2024-05-13T22:48:26.210 INFO:tasks.workunit.client.1.smithi130.stdout:4/169: link d8/dc/d13/l25 d8/dc/l3e 0 2024-05-13T22:48:26.213 INFO:tasks.workunit.client.1.smithi130.stdout:6/205: truncate d4/d15/d1d/f22 1878160 0 2024-05-13T22:48:26.219 INFO:tasks.workunit.client.1.smithi130.stdout:4/170: mkdir d8/dc/d35/d33/d3f 0 2024-05-13T22:48:26.223 INFO:tasks.workunit.client.0.smithi102.stdout:0/919: mknod d1/de/d32/d24/c11c 0 2024-05-13T22:48:26.224 INFO:tasks.workunit.client.0.smithi102.stdout:8/901: write d2/d7/d27/d64/d80/dc8/d40/fe0 [961627,58740] 0 2024-05-13T22:48:26.224 INFO:tasks.workunit.client.1.smithi130.stdout:4/171: dwrite d8/dc/d35/f31 [0,4194304] 0 2024-05-13T22:48:26.226 INFO:tasks.workunit.client.0.smithi102.stdout:8/902: write d2/d7/d27/d64/d96/dae/f23 [4745447,117043] 0 2024-05-13T22:48:26.226 INFO:tasks.workunit.client.0.smithi102.stdout:0/920: creat d1/d4d/d79/d9f/dce/f11d x:0 0 0 2024-05-13T22:48:26.227 INFO:tasks.workunit.client.0.smithi102.stdout:8/903: chown d2/d7/f9 961996 1 2024-05-13T22:48:26.229 INFO:tasks.workunit.client.1.smithi130.stdout:4/172: creat d8/df/f40 x:0 0 0 2024-05-13T22:48:26.229 INFO:tasks.workunit.client.1.smithi130.stdout:4/173: chown d8/f1a 431636 1 2024-05-13T22:48:26.234 INFO:tasks.workunit.client.1.smithi130.stdout:4/174: mkdir d8/dc/d26/d41 0 2024-05-13T22:48:26.234 INFO:tasks.workunit.client.1.smithi130.stdout:4/175: read - d8/dc/d29/f3a zero size 2024-05-13T22:48:26.237 INFO:tasks.workunit.client.0.smithi102.stdout:7/879: mkdir d6/d15/d115 0 2024-05-13T22:48:26.241 INFO:tasks.workunit.client.1.smithi130.stdout:4/176: dwrite d8/dc/d13/f19 [0,4194304] 0 2024-05-13T22:48:26.241 INFO:tasks.workunit.client.1.smithi130.stdout:4/177: chown d8/dc/f28 13 1 2024-05-13T22:48:26.246 INFO:tasks.workunit.client.1.smithi130.stdout:4/178: rmdir d8/dc/d26/d41 0 2024-05-13T22:48:26.246 INFO:tasks.workunit.client.1.smithi130.stdout:4/179: chown d8/cb 307982 1 2024-05-13T22:48:26.251 INFO:tasks.workunit.client.1.smithi130.stdout:0/176: rmdir d0 39 2024-05-13T22:48:26.256 INFO:tasks.workunit.client.1.smithi130.stdout:0/177: dread d0/fe [0,4194304] 0 2024-05-13T22:48:26.256 INFO:tasks.workunit.client.0.smithi102.stdout:9/906: dwrite d1/d6/f60 [4194304,4194304] 0 2024-05-13T22:48:26.256 INFO:tasks.workunit.client.0.smithi102.stdout:2/991: dwrite d7/d15/d1e/da7/db5/ff6 [0,4194304] 0 2024-05-13T22:48:26.266 INFO:tasks.workunit.client.0.smithi102.stdout:2/992: fsync d7/d9/d11/f39 0 2024-05-13T22:48:26.268 INFO:tasks.workunit.client.1.smithi130.stdout:1/164: write f8 [1301000,49508] 0 2024-05-13T22:48:26.272 INFO:tasks.workunit.client.0.smithi102.stdout:9/907: creat d1/d6/d45/da1/d10a/f136 x:0 0 0 2024-05-13T22:48:26.272 INFO:tasks.workunit.client.0.smithi102.stdout:2/993: dwrite d7/d9/d11/f105 [0,4194304] 0 2024-05-13T22:48:26.275 INFO:tasks.workunit.client.1.smithi130.stdout:3/152: rmdir d8/dc/d19/d1d 39 2024-05-13T22:48:26.278 INFO:tasks.workunit.client.0.smithi102.stdout:2/994: creat d7/d15/d1e/da7/df4/f142 x:0 0 0 2024-05-13T22:48:26.281 INFO:tasks.workunit.client.0.smithi102.stdout:9/908: rename d1/d6/d45/da1/cb5 to d1/d6/d21/d71/c137 0 2024-05-13T22:48:26.286 INFO:tasks.workunit.client.1.smithi130.stdout:0/178: dread d0/f1e [0,4194304] 0 2024-05-13T22:48:26.286 INFO:tasks.workunit.client.0.smithi102.stdout:2/995: dread d7/d9/f49 [0,4194304] 0 2024-05-13T22:48:26.287 INFO:tasks.workunit.client.1.smithi130.stdout:3/153: read d8/dc/d19/f20 [277321,70398] 0 2024-05-13T22:48:26.287 INFO:tasks.workunit.client.1.smithi130.stdout:0/179: dread - d0/d7/dd/d17/d1f/d2c/f31 zero size 2024-05-13T22:48:26.288 INFO:tasks.workunit.client.0.smithi102.stdout:9/909: rename d1/d6/d45/dd0/cdf to d1/d6/d21/d12e/d75/c138 0 2024-05-13T22:48:26.289 INFO:tasks.workunit.client.0.smithi102.stdout:2/996: mknod d7/d9/c143 0 2024-05-13T22:48:26.291 INFO:tasks.workunit.client.1.smithi130.stdout:3/154: dread d8/dc/d19/f1e [0,4194304] 0 2024-05-13T22:48:26.293 INFO:tasks.workunit.client.0.smithi102.stdout:2/997: creat d7/d9/d10/d30/d7f/f144 x:0 0 0 2024-05-13T22:48:26.295 INFO:tasks.workunit.client.0.smithi102.stdout:9/910: dwrite d1/d6/db/d30/d122/f131 [0,4194304] 0 2024-05-13T22:48:26.295 INFO:tasks.workunit.client.0.smithi102.stdout:2/998: chown d7/d15/d1e/da7/db5/ff6 891 1 2024-05-13T22:48:26.298 INFO:tasks.workunit.client.1.smithi130.stdout:3/155: mknod d8/dc/d19/d2c/c2d 0 2024-05-13T22:48:26.300 INFO:tasks.workunit.client.1.smithi130.stdout:0/180: getdents d0/d7/dd/d17 0 2024-05-13T22:48:26.306 INFO:tasks.workunit.client.1.smithi130.stdout:3/156: creat d8/dc/d19/d1d/f2e x:0 0 0 2024-05-13T22:48:26.306 INFO:tasks.workunit.client.1.smithi130.stdout:0/181: dwrite d0/f8 [4194304,4194304] 0 2024-05-13T22:48:26.307 INFO:tasks.workunit.client.0.smithi102.stdout:9/911: link d1/d6/d3b/d5b/d127/f135 d1/d6/d45/d2d/dac/f139 0 2024-05-13T22:48:26.309 INFO:tasks.workunit.client.1.smithi130.stdout:2/181: rename d4/d33 to d4/d49 0 2024-05-13T22:48:26.311 INFO:tasks.workunit.client.0.smithi102.stdout:2/999: creat d7/d15/d1e/f145 x:0 0 0 2024-05-13T22:48:26.311 INFO:tasks.workunit.client.1.smithi130.stdout:2/182: write d4/db/dc/d18/f36 [69989,115136] 0 2024-05-13T22:48:26.312 INFO:tasks.workunit.client.1.smithi130.stdout:3/157: chown d8/dc/d19/d27/f28 4214 1 2024-05-13T22:48:26.317 INFO:tasks.workunit.client.0.smithi102.stdout:9/912: mkdir d1/d6/d45/d3e/d13a 0 2024-05-13T22:48:26.318 INFO:tasks.workunit.client.1.smithi130.stdout:2/183: mkdir d4/d49/d48/d4a 0 2024-05-13T22:48:26.320 INFO:tasks.workunit.client.1.smithi130.stdout:2/184: truncate d4/db/dc/d18/d2a/d30/f34 396813 0 2024-05-13T22:48:26.320 INFO:tasks.workunit.client.0.smithi102.stdout:9/913: unlink d1/d6/d45/d3e/f68 0 2024-05-13T22:48:26.322 INFO:tasks.workunit.client.1.smithi130.stdout:2/185: mkdir d4/d49/d42/d4b 0 2024-05-13T22:48:26.323 INFO:tasks.workunit.client.0.smithi102.stdout:9/914: mknod d1/d6/d21/d12e/d75/d96/dc6/dd9/c13b 0 2024-05-13T22:48:26.326 INFO:tasks.workunit.client.1.smithi130.stdout:2/186: link d4/db/f2e d4/d49/d48/f4c 0 2024-05-13T22:48:26.327 INFO:tasks.workunit.client.0.smithi102.stdout:9/915: truncate d1/d6/db/f1c 7392317 0 2024-05-13T22:48:26.329 INFO:tasks.workunit.client.1.smithi130.stdout:2/187: creat d4/db/dc/f4d x:0 0 0 2024-05-13T22:48:26.330 INFO:tasks.workunit.client.1.smithi130.stdout:5/171: truncate d6/dd/f33 2897220 0 2024-05-13T22:48:26.332 INFO:tasks.workunit.client.1.smithi130.stdout:8/145: unlink d1/f2d 0 2024-05-13T22:48:26.333 INFO:tasks.workunit.client.0.smithi102.stdout:9/916: link d1/d6/d45/f4f d1/d6/d21/d12e/d64/dc0/f13c 0 2024-05-13T22:48:26.334 INFO:tasks.workunit.client.0.smithi102.stdout:9/917: readlink d1/d6/db/d30/l7d 0 2024-05-13T22:48:26.335 INFO:tasks.workunit.client.0.smithi102.stdout:4/939: mkdir d3/d19/d23/d14a 0 2024-05-13T22:48:26.336 INFO:tasks.workunit.client.1.smithi130.stdout:5/172: creat d6/d2f/f39 x:0 0 0 2024-05-13T22:48:26.338 INFO:tasks.workunit.client.1.smithi130.stdout:8/146: rename d1/d2/db/l23 to d1/d2/db/d15/d16/l35 0 2024-05-13T22:48:26.339 INFO:tasks.workunit.client.1.smithi130.stdout:8/147: dread - d1/d2/db/d15/f22 zero size 2024-05-13T22:48:26.340 INFO:tasks.workunit.client.1.smithi130.stdout:2/188: creat d4/db/dc/f4e x:0 0 0 2024-05-13T22:48:26.341 INFO:tasks.workunit.client.1.smithi130.stdout:2/189: write d4/db/dc/f4d [991245,120147] 0 2024-05-13T22:48:26.342 INFO:tasks.workunit.client.0.smithi102.stdout:7/880: mknod d6/d15/d4d/d55/d6f/c116 0 2024-05-13T22:48:26.344 INFO:tasks.workunit.client.0.smithi102.stdout:9/918: rename d1/d6/db/d30/d8a/l129 to d1/d6/d45/d2d/dac/l13d 0 2024-05-13T22:48:26.345 INFO:tasks.workunit.client.1.smithi130.stdout:5/173: creat d6/dd/d18/f3a x:0 0 0 2024-05-13T22:48:26.348 INFO:tasks.workunit.client.1.smithi130.stdout:8/148: rename d1/d2/db/d15/d18/d1a/f1e to d1/dc/f36 0 2024-05-13T22:48:26.349 INFO:tasks.workunit.client.1.smithi130.stdout:8/149: dread - d1/dc/f24 zero size 2024-05-13T22:48:26.350 INFO:tasks.workunit.client.1.smithi130.stdout:5/174: dread d6/d7/f22 [0,4194304] 0 2024-05-13T22:48:26.351 INFO:tasks.workunit.client.1.smithi130.stdout:6/206: write d4/d15/d1d/d28/f30 [330755,52103] 0 2024-05-13T22:48:26.353 INFO:tasks.workunit.client.1.smithi130.stdout:5/175: truncate d6/dd/f1b 1696909 0 2024-05-13T22:48:26.355 INFO:tasks.workunit.client.1.smithi130.stdout:8/150: dwrite d1/f33 [0,4194304] 0 2024-05-13T22:48:26.356 INFO:tasks.workunit.client.1.smithi130.stdout:8/151: dread - d1/d2/db/d15/f22 zero size 2024-05-13T22:48:26.358 INFO:tasks.workunit.client.1.smithi130.stdout:8/152: write d1/d2/f25 [691216,35649] 0 2024-05-13T22:48:26.363 INFO:tasks.workunit.client.1.smithi130.stdout:8/153: dwrite d1/d2/db/d15/d18/f31 [0,4194304] 0 2024-05-13T22:48:26.366 INFO:tasks.workunit.client.0.smithi102.stdout:8/904: dwrite d2/d7/d27/f4c [0,4194304] 0 2024-05-13T22:48:26.369 INFO:tasks.workunit.client.0.smithi102.stdout:7/881: creat d6/f117 x:0 0 0 2024-05-13T22:48:26.370 INFO:tasks.workunit.client.0.smithi102.stdout:8/905: truncate d2/d7/d27/d64/d80/dc8/d93/f10c 630991 0 2024-05-13T22:48:26.377 INFO:tasks.workunit.client.1.smithi130.stdout:2/190: creat d4/d49/d42/d4b/f4f x:0 0 0 2024-05-13T22:48:26.380 INFO:tasks.workunit.client.1.smithi130.stdout:9/169: sync 2024-05-13T22:48:26.382 INFO:tasks.workunit.client.0.smithi102.stdout:8/906: rename d2/d7/f103 to d2/d7/d25/df4/f11e 0 2024-05-13T22:48:26.383 INFO:tasks.workunit.client.1.smithi130.stdout:4/180: dwrite f4 [0,4194304] 0 2024-05-13T22:48:26.384 INFO:tasks.workunit.client.1.smithi130.stdout:5/176: rename d6/f11 to d6/d7/f3b 0 2024-05-13T22:48:26.385 INFO:tasks.workunit.client.1.smithi130.stdout:4/181: read d8/df/f20 [160650,28434] 0 2024-05-13T22:48:26.390 INFO:tasks.workunit.client.1.smithi130.stdout:5/177: dwrite d6/d2f/f39 [0,4194304] 0 2024-05-13T22:48:26.392 INFO:tasks.workunit.client.0.smithi102.stdout:7/882: truncate d6/f85 915838 0 2024-05-13T22:48:26.399 INFO:tasks.workunit.client.0.smithi102.stdout:8/907: symlink d2/d7/d25/df4/l11f 0 2024-05-13T22:48:26.402 INFO:tasks.workunit.client.0.smithi102.stdout:0/921: link d1/de/d32/d24/d2d/d3e/c9b d1/de/d32/dab/c11e 0 2024-05-13T22:48:26.406 INFO:tasks.workunit.client.1.smithi130.stdout:9/170: mknod d1/d7/d23/d28/d2d/d2f/c36 0 2024-05-13T22:48:26.408 INFO:tasks.workunit.client.0.smithi102.stdout:8/908: creat d2/d7/d27/d64/d80/dc8/d40/d85/ddc/f120 x:0 0 0 2024-05-13T22:48:26.409 INFO:tasks.workunit.client.0.smithi102.stdout:8/909: chown d2/d7/d25/d41/l61 869 1 2024-05-13T22:48:26.410 INFO:tasks.workunit.client.1.smithi130.stdout:5/178: dread f4 [0,4194304] 0 2024-05-13T22:48:26.415 INFO:tasks.workunit.client.1.smithi130.stdout:4/182: dread d8/f1a [0,4194304] 0 2024-05-13T22:48:26.415 INFO:tasks.workunit.client.1.smithi130.stdout:4/183: dread - d8/dc/f38 zero size 2024-05-13T22:48:26.416 INFO:tasks.workunit.client.1.smithi130.stdout:4/184: dread - d8/df/f40 zero size 2024-05-13T22:48:26.417 INFO:tasks.workunit.client.1.smithi130.stdout:4/185: chown d8/f1f 104575329 1 2024-05-13T22:48:26.417 INFO:tasks.workunit.client.1.smithi130.stdout:4/186: fsync d8/fa 0 2024-05-13T22:48:26.418 INFO:tasks.workunit.client.0.smithi102.stdout:8/910: dwrite d2/d7/d27/d64/d80/dc8/d40/d85/ddc/fed [0,4194304] 0 2024-05-13T22:48:26.422 INFO:tasks.workunit.client.1.smithi130.stdout:3/158: dwrite d8/f2b [0,4194304] 0 2024-05-13T22:48:26.423 INFO:tasks.workunit.client.0.smithi102.stdout:7/883: dread d6/dd0/d80/fd8 [0,4194304] 0 2024-05-13T22:48:26.423 INFO:tasks.workunit.client.0.smithi102.stdout:0/922: creat d1/de/f11f x:0 0 0 2024-05-13T22:48:26.425 INFO:tasks.workunit.client.1.smithi130.stdout:2/191: creat d4/db/dc/d18/d2a/f50 x:0 0 0 2024-05-13T22:48:26.426 INFO:tasks.workunit.client.0.smithi102.stdout:8/911: mknod d2/de8/c121 0 2024-05-13T22:48:26.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:26 smithi102 ceph-mon[28599]: pgmap v30: 65 pgs: 65 active+clean; 2.0 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 111 MiB/s wr, 326 op/s 2024-05-13T22:48:26.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:26 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:26.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:26 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:26.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:26 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:48:26.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:26 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:48:26.428 INFO:tasks.workunit.client.1.smithi130.stdout:3/159: dread d8/d1a/d1c/f25 [0,4194304] 0 2024-05-13T22:48:26.429 INFO:tasks.workunit.client.1.smithi130.stdout:3/160: stat d8/dc/l17 0 2024-05-13T22:48:26.429 INFO:tasks.workunit.client.0.smithi102.stdout:0/923: symlink d1/de/d32/dab/l120 0 2024-05-13T22:48:26.431 INFO:tasks.workunit.client.0.smithi102.stdout:0/924: write d1/de/d32/d63/f10f [578196,107045] 0 2024-05-13T22:48:26.432 INFO:tasks.workunit.client.1.smithi130.stdout:9/171: creat d1/d15/f37 x:0 0 0 2024-05-13T22:48:26.432 INFO:tasks.workunit.client.1.smithi130.stdout:2/192: dread d4/f21 [0,4194304] 0 2024-05-13T22:48:26.433 INFO:tasks.workunit.client.1.smithi130.stdout:2/193: read - d4/db/dc/d18/d2a/f50 zero size 2024-05-13T22:48:26.434 INFO:tasks.workunit.client.1.smithi130.stdout:2/194: write d4/f1a [533300,83738] 0 2024-05-13T22:48:26.434 INFO:tasks.workunit.client.1.smithi130.stdout:2/195: readlink d4/d49/l3c 0 2024-05-13T22:48:26.436 INFO:tasks.workunit.client.0.smithi102.stdout:7/884: rename d6/d13/c10 to d6/d15/d4d/d55/d6f/d49/d95/c118 0 2024-05-13T22:48:26.443 INFO:tasks.workunit.client.0.smithi102.stdout:0/925: mkdir d1/de/d32/d8a/d116/d121 0 2024-05-13T22:48:26.444 INFO:tasks.workunit.client.1.smithi130.stdout:7/134: sync 2024-05-13T22:48:26.444 INFO:tasks.workunit.client.1.smithi130.stdout:1/165: sync 2024-05-13T22:48:26.445 INFO:tasks.workunit.client.1.smithi130.stdout:3/161: fdatasync d8/f2b 0 2024-05-13T22:48:26.446 INFO:tasks.workunit.client.1.smithi130.stdout:1/166: write db/f1d [810949,87118] 0 2024-05-13T22:48:26.446 INFO:tasks.workunit.client.1.smithi130.stdout:8/154: chown d1/d2/db/d15/d16/l35 3876666 1 2024-05-13T22:48:26.447 INFO:tasks.workunit.client.0.smithi102.stdout:0/926: read d1/de/d32/d24/d2d/d38/fa3 [3551661,77861] 0 2024-05-13T22:48:26.447 INFO:tasks.workunit.client.1.smithi130.stdout:3/162: chown d8/dc/d15 591 1 2024-05-13T22:48:26.447 INFO:tasks.workunit.client.1.smithi130.stdout:3/163: chown d8/d1a/d1c 28104 1 2024-05-13T22:48:26.448 INFO:tasks.workunit.client.1.smithi130.stdout:4/187: rename f5 to d8/df/f42 0 2024-05-13T22:48:26.452 INFO:tasks.workunit.client.0.smithi102.stdout:8/912: mkdir d2/d122 0 2024-05-13T22:48:26.453 INFO:tasks.workunit.client.0.smithi102.stdout:4/940: write d3/d4/d6/de/d1f/f28 [3660201,39528] 0 2024-05-13T22:48:26.456 INFO:tasks.workunit.client.0.smithi102.stdout:4/941: write d3/d19/d23/d7e/f12e [31090,62373] 0 2024-05-13T22:48:26.457 INFO:tasks.workunit.client.0.smithi102.stdout:4/942: chown d3/d22/f3e 244 1 2024-05-13T22:48:26.457 INFO:tasks.workunit.client.0.smithi102.stdout:0/927: fsync d1/f60 0 2024-05-13T22:48:26.460 INFO:tasks.workunit.client.0.smithi102.stdout:8/913: dread d2/d7/d27/d64/ff1 [0,4194304] 0 2024-05-13T22:48:26.463 INFO:tasks.workunit.client.0.smithi102.stdout:9/919: dwrite d1/d6/d45/d2d/dac/db0/fea [0,4194304] 0 2024-05-13T22:48:26.471 INFO:tasks.workunit.client.0.smithi102.stdout:4/943: dread d3/d19/d23/f82 [0,4194304] 0 2024-05-13T22:48:26.476 INFO:tasks.workunit.client.1.smithi130.stdout:1/167: symlink db/d1b/l39 0 2024-05-13T22:48:26.479 INFO:tasks.workunit.client.1.smithi130.stdout:8/155: mkdir d1/d2/db/d15/d18/d1a/d37 0 2024-05-13T22:48:26.479 INFO:tasks.workunit.client.1.smithi130.stdout:8/156: chown d1/d2/db/d15/ca 642132173 1 2024-05-13T22:48:26.484 INFO:tasks.workunit.client.0.smithi102.stdout:9/920: mknod d1/d6/d21/d12e/d75/d96/d102/d6c/d106/c13e 0 2024-05-13T22:48:26.484 INFO:tasks.workunit.client.0.smithi102.stdout:9/921: chown d1/c8e 13 1 2024-05-13T22:48:26.486 INFO:tasks.workunit.client.0.smithi102.stdout:9/922: write d1/d6/d3b/d5b/f113 [436191,114202] 0 2024-05-13T22:48:26.487 INFO:tasks.workunit.client.1.smithi130.stdout:9/172: rename d1/d7/l12 to d1/d7/d23/d28/l38 0 2024-05-13T22:48:26.488 INFO:tasks.workunit.client.1.smithi130.stdout:9/173: write d1/d15/f37 [882055,48048] 0 2024-05-13T22:48:26.492 INFO:tasks.workunit.client.1.smithi130.stdout:6/207: link d4/d15/d1d/f22 d4/d15/f44 0 2024-05-13T22:48:26.492 INFO:tasks.workunit.client.1.smithi130.stdout:6/208: chown d4/d15/l2b 728 1 2024-05-13T22:48:26.493 INFO:tasks.workunit.client.1.smithi130.stdout:6/209: readlink d4/d15/d1d/l1e 0 2024-05-13T22:48:26.495 INFO:tasks.workunit.client.1.smithi130.stdout:0/182: sync 2024-05-13T22:48:26.495 INFO:tasks.workunit.client.0.smithi102.stdout:9/923: mkdir d1/d6/db/d13f 0 2024-05-13T22:48:26.497 INFO:tasks.workunit.client.1.smithi130.stdout:8/157: unlink d1/dc/f36 0 2024-05-13T22:48:26.501 INFO:tasks.workunit.client.0.smithi102.stdout:4/944: getdents d3/d4/d6/de 0 2024-05-13T22:48:26.505 INFO:tasks.workunit.client.1.smithi130.stdout:9/174: readlink d1/d15/l24 0 2024-05-13T22:48:26.508 INFO:tasks.workunit.client.1.smithi130.stdout:5/179: getdents d6/d7/d24 0 2024-05-13T22:48:26.509 INFO:tasks.workunit.client.1.smithi130.stdout:5/180: write d6/d2f/f39 [3845063,55001] 0 2024-05-13T22:48:26.509 INFO:tasks.workunit.client.1.smithi130.stdout:5/181: fdatasync d6/d7/f25 0 2024-05-13T22:48:26.511 INFO:tasks.workunit.client.1.smithi130.stdout:0/183: mkdir d0/d7/d29/d3c 0 2024-05-13T22:48:26.512 INFO:tasks.workunit.client.0.smithi102.stdout:4/945: mknod d3/d19/dad/dc9/c14b 0 2024-05-13T22:48:26.513 INFO:tasks.workunit.client.1.smithi130.stdout:4/188: creat d8/f43 x:0 0 0 2024-05-13T22:48:26.520 INFO:tasks.workunit.client.0.smithi102.stdout:4/946: fdatasync d3/d19/d23/d4e/d68/d85/f9a 0 2024-05-13T22:48:26.521 INFO:tasks.workunit.client.1.smithi130.stdout:6/210: truncate d4/f9 4016767 0 2024-05-13T22:48:26.522 INFO:tasks.workunit.client.1.smithi130.stdout:6/211: write d4/d15/d1d/f38 [533945,63437] 0 2024-05-13T22:48:26.527 INFO:tasks.workunit.client.0.smithi102.stdout:4/947: rename d3/d22/le9 to d3/d4/d6/de/l14c 0 2024-05-13T22:48:26.528 INFO:tasks.workunit.client.0.smithi102.stdout:0/928: rmdir d1/de 39 2024-05-13T22:48:26.529 INFO:tasks.workunit.client.1.smithi130.stdout:4/189: mkdir d8/dc/d35/d33/d44 0 2024-05-13T22:48:26.541 INFO:tasks.workunit.client.1.smithi130.stdout:4/190: symlink d8/dc/d13/l45 0 2024-05-13T22:48:26.543 INFO:tasks.workunit.client.0.smithi102.stdout:7/885: dwrite d6/d13/d58/fc9 [0,4194304] 0 2024-05-13T22:48:26.545 INFO:tasks.workunit.client.1.smithi130.stdout:7/135: rename d7/da/l22 to d7/d8/d9/l32 0 2024-05-13T22:48:26.545 INFO:tasks.workunit.client.1.smithi130.stdout:1/168: rename db/d1b to db/d1b/d3a 22 2024-05-13T22:48:26.546 INFO:tasks.workunit.client.1.smithi130.stdout:7/136: chown d7/da/d1f 12271 1 2024-05-13T22:48:26.547 INFO:tasks.workunit.client.1.smithi130.stdout:2/196: dwrite d4/db/f22 [4194304,4194304] 0 2024-05-13T22:48:26.547 INFO:tasks.workunit.client.1.smithi130.stdout:1/169: write db/f1d [1878452,71019] 0 2024-05-13T22:48:26.547 INFO:tasks.workunit.client.1.smithi130.stdout:7/137: chown d7/da/d1f 2755 1 2024-05-13T22:48:26.549 INFO:tasks.workunit.client.1.smithi130.stdout:6/212: mknod d4/d15/c45 0 2024-05-13T22:48:26.550 INFO:tasks.workunit.client.1.smithi130.stdout:1/170: truncate db/d1b/d1f/d2a/f34 567272 0 2024-05-13T22:48:26.555 INFO:tasks.workunit.client.1.smithi130.stdout:9/175: getdents d1/db 0 2024-05-13T22:48:26.555 INFO:tasks.workunit.client.0.smithi102.stdout:8/914: write d2/d7/d27/d64/d96/dae/fe1 [271467,48326] 0 2024-05-13T22:48:26.556 INFO:tasks.workunit.client.0.smithi102.stdout:8/915: fsync d2/d7/d27/d64/d80/f100 0 2024-05-13T22:48:26.558 INFO:tasks.workunit.client.0.smithi102.stdout:8/916: chown d2/d7/d25/d41/f75 30 1 2024-05-13T22:48:26.559 INFO:tasks.workunit.client.0.smithi102.stdout:8/917: chown d2/d36/d49/l86 35826 1 2024-05-13T22:48:26.562 INFO:tasks.workunit.client.1.smithi130.stdout:3/164: dwrite d8/dc/d19/f1e [0,4194304] 0 2024-05-13T22:48:26.565 INFO:tasks.workunit.client.0.smithi102.stdout:8/918: dwrite d2/d7/d27/d64/d80/dc8/d40/d85/f118 [0,4194304] 0 2024-05-13T22:48:26.567 INFO:tasks.workunit.client.1.smithi130.stdout:2/197: symlink d4/db/dc/d18/l51 0 2024-05-13T22:48:26.575 INFO:tasks.workunit.client.1.smithi130.stdout:6/213: mkdir d4/d15/d1f/d46 0 2024-05-13T22:48:26.576 INFO:tasks.workunit.client.0.smithi102.stdout:9/924: dwrite d1/d6/db/f1c [0,4194304] 0 2024-05-13T22:48:26.581 INFO:tasks.workunit.client.1.smithi130.stdout:1/171: creat db/d1b/d1f/d2a/d35/f3b x:0 0 0 2024-05-13T22:48:26.585 INFO:tasks.workunit.client.1.smithi130.stdout:5/182: write d6/d7/d19/f2c [464664,92706] 0 2024-05-13T22:48:26.588 INFO:tasks.workunit.client.0.smithi102.stdout:4/948: write d3/d19/d23/f37 [660698,115694] 0 2024-05-13T22:48:26.589 INFO:tasks.workunit.client.1.smithi130.stdout:1/172: dwrite db/f14 [4194304,4194304] 0 2024-05-13T22:48:26.589 INFO:tasks.workunit.client.0.smithi102.stdout:4/949: chown d3/d19/d23/d39/d5d/da5/fb7 1642225236 1 2024-05-13T22:48:26.590 INFO:tasks.workunit.client.1.smithi130.stdout:1/173: chown db/f1d 233 1 2024-05-13T22:48:26.591 INFO:tasks.workunit.client.1.smithi130.stdout:3/165: creat d8/dc/d15/f2f x:0 0 0 2024-05-13T22:48:26.591 INFO:tasks.workunit.client.1.smithi130.stdout:1/174: chown db/d1b/d1f/d2a/f34 10 1 2024-05-13T22:48:26.594 INFO:tasks.workunit.client.1.smithi130.stdout:7/138: unlink d7/d8/d9/c12 0 2024-05-13T22:48:26.595 INFO:tasks.workunit.client.1.smithi130.stdout:3/166: read d8/f9 [5354789,64049] 0 2024-05-13T22:48:26.597 INFO:tasks.workunit.client.1.smithi130.stdout:0/184: sync 2024-05-13T22:48:26.598 INFO:tasks.workunit.client.0.smithi102.stdout:0/929: dwrite d1/de/d32/d8a/fac [0,4194304] 0 2024-05-13T22:48:26.598 INFO:tasks.workunit.client.1.smithi130.stdout:6/214: creat d4/d15/d1d/d28/f47 x:0 0 0 2024-05-13T22:48:26.601 INFO:tasks.workunit.client.1.smithi130.stdout:4/191: link d8/dc/d35/l37 d8/dc/d35/d33/d3f/l46 0 2024-05-13T22:48:26.602 INFO:tasks.workunit.client.1.smithi130.stdout:4/192: truncate f7 4486789 0 2024-05-13T22:48:26.606 INFO:tasks.workunit.client.1.smithi130.stdout:0/185: dwrite d0/d7/dd/d17/d1f/f25 [0,4194304] 0 2024-05-13T22:48:26.606 INFO:tasks.workunit.client.1.smithi130.stdout:6/215: dwrite d4/d15/f3a [0,4194304] 0 2024-05-13T22:48:26.610 INFO:tasks.workunit.client.1.smithi130.stdout:8/158: rename d1/d2/db/d15/d18/d1a/d37 to d1/d2/d38 0 2024-05-13T22:48:26.612 INFO:tasks.workunit.client.1.smithi130.stdout:6/216: dread d4/d15/f3a [0,4194304] 0 2024-05-13T22:48:26.613 INFO:tasks.workunit.client.1.smithi130.stdout:7/139: rmdir d7/da/d1f 39 2024-05-13T22:48:26.614 INFO:tasks.workunit.client.1.smithi130.stdout:6/217: chown d4/d15/d1f/f2f 0 1 2024-05-13T22:48:26.619 INFO:tasks.workunit.client.1.smithi130.stdout:6/218: dwrite d4/d15/f3a [0,4194304] 0 2024-05-13T22:48:26.638 INFO:tasks.workunit.client.1.smithi130.stdout:9/176: rename d1/f2c to d1/d7/d23/d28/f39 0 2024-05-13T22:48:26.640 INFO:tasks.workunit.client.1.smithi130.stdout:8/159: unlink d1/d2/db/d15/d16/f2a 0 2024-05-13T22:48:26.641 INFO:tasks.workunit.client.1.smithi130.stdout:5/183: symlink d6/l3c 0 2024-05-13T22:48:26.642 INFO:tasks.workunit.client.1.smithi130.stdout:5/184: dread - d6/d27/f34 zero size 2024-05-13T22:48:26.643 INFO:tasks.workunit.client.1.smithi130.stdout:5/185: fdatasync d6/dd/d18/fc 0 2024-05-13T22:48:26.644 INFO:tasks.workunit.client.1.smithi130.stdout:5/186: write d6/d7/f25 [5058966,45745] 0 2024-05-13T22:48:26.648 INFO:tasks.workunit.client.1.smithi130.stdout:4/193: write d8/df/f20 [532489,24046] 0 2024-05-13T22:48:26.650 INFO:tasks.workunit.client.1.smithi130.stdout:0/186: write d0/fa [1712659,6381] 0 2024-05-13T22:48:26.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:26 smithi130 ceph-mon[36591]: pgmap v30: 65 pgs: 65 active+clean; 2.0 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 111 MiB/s wr, 326 op/s 2024-05-13T22:48:26.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:26 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:26.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:26 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:26.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:26 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:48:26.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:26 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:48:26.655 INFO:tasks.workunit.client.1.smithi130.stdout:8/160: fsync d1/d2/db/d15/f30 0 2024-05-13T22:48:26.658 INFO:tasks.workunit.client.1.smithi130.stdout:5/187: read f4 [1215518,78784] 0 2024-05-13T22:48:26.659 INFO:tasks.workunit.client.1.smithi130.stdout:8/161: dread d1/dc/f1f [0,4194304] 0 2024-05-13T22:48:26.659 INFO:tasks.workunit.client.1.smithi130.stdout:3/167: creat d8/dc/d19/f30 x:0 0 0 2024-05-13T22:48:26.661 INFO:tasks.workunit.client.1.smithi130.stdout:6/219: mknod d4/d15/d1f/d46/c48 0 2024-05-13T22:48:26.663 INFO:tasks.workunit.client.1.smithi130.stdout:4/194: creat d8/dc/d13/f47 x:0 0 0 2024-05-13T22:48:26.666 INFO:tasks.workunit.client.1.smithi130.stdout:2/198: rename d4/db/dc/f25 to d4/d49/d48/d4a/f52 0 2024-05-13T22:48:26.667 INFO:tasks.workunit.client.1.smithi130.stdout:2/199: dread - d4/db/dc/d18/d2a/d30/f47 zero size 2024-05-13T22:48:26.667 INFO:tasks.workunit.client.1.smithi130.stdout:9/177: link d1/f33 d1/d7/d23/d28/d2d/f3a 0 2024-05-13T22:48:26.669 INFO:tasks.workunit.client.1.smithi130.stdout:9/178: truncate d1/d7/d23/d28/d2d/f34 507709 0 2024-05-13T22:48:26.670 INFO:tasks.workunit.client.1.smithi130.stdout:9/179: write d1/d7/fa [3913755,93354] 0 2024-05-13T22:48:26.671 INFO:tasks.workunit.client.0.smithi102.stdout:9/925: unlink d1/d6/d21/d12e/d64/dc0/cca 0 2024-05-13T22:48:26.672 INFO:tasks.workunit.client.1.smithi130.stdout:8/162: dread d1/f10 [0,4194304] 0 2024-05-13T22:48:26.672 INFO:tasks.workunit.client.1.smithi130.stdout:7/140: creat d7/da/f33 x:0 0 0 2024-05-13T22:48:26.674 INFO:tasks.workunit.client.1.smithi130.stdout:5/188: creat d6/d7/d32/f3d x:0 0 0 2024-05-13T22:48:26.676 INFO:tasks.workunit.client.1.smithi130.stdout:3/168: symlink d8/dc/d19/d27/l31 0 2024-05-13T22:48:26.677 INFO:tasks.workunit.client.1.smithi130.stdout:9/180: dwrite d1/f3 [0,4194304] 0 2024-05-13T22:48:26.679 INFO:tasks.workunit.client.0.smithi102.stdout:9/926: unlink d1/d6/db/d30/f36 0 2024-05-13T22:48:26.679 INFO:tasks.workunit.client.0.smithi102.stdout:9/927: chown d1/d6/db/f1b 3 1 2024-05-13T22:48:26.680 INFO:tasks.workunit.client.1.smithi130.stdout:4/195: symlink d8/dc/d35/d33/l48 0 2024-05-13T22:48:26.682 INFO:tasks.workunit.client.1.smithi130.stdout:0/187: mknod d0/d7/c3d 0 2024-05-13T22:48:26.682 INFO:tasks.workunit.client.0.smithi102.stdout:8/919: rmdir d2/d7/d27/d64/d96/dae/d108 0 2024-05-13T22:48:26.687 INFO:tasks.workunit.client.1.smithi130.stdout:1/175: rename db/d1b/c21 to db/d1b/d1f/c3c 0 2024-05-13T22:48:26.688 INFO:tasks.workunit.client.0.smithi102.stdout:9/928: mknod d1/d6/d21/d12e/d64/c140 0 2024-05-13T22:48:26.691 INFO:tasks.workunit.client.0.smithi102.stdout:8/920: creat d2/d7/d27/d64/d96/dae/d78/f123 x:0 0 0 2024-05-13T22:48:26.692 INFO:tasks.workunit.client.0.smithi102.stdout:8/921: fdatasync d2/d7/d27/d64/d96/dae/f112 0 2024-05-13T22:48:26.695 INFO:tasks.workunit.client.0.smithi102.stdout:9/929: mkdir d1/d6/d21/d71/d141 0 2024-05-13T22:48:26.697 INFO:tasks.workunit.client.0.smithi102.stdout:8/922: mkdir d2/d36/db3/d124 0 2024-05-13T22:48:26.700 INFO:tasks.workunit.client.1.smithi130.stdout:5/189: creat d6/dd/f3e x:0 0 0 2024-05-13T22:48:26.703 INFO:tasks.workunit.client.0.smithi102.stdout:4/950: dwrite d3/d19/d23/d4e/d68/d85/fbf [0,4194304] 0 2024-05-13T22:48:26.709 INFO:tasks.workunit.client.1.smithi130.stdout:3/169: symlink d8/dc/d19/d2c/l32 0 2024-05-13T22:48:26.711 INFO:tasks.workunit.client.1.smithi130.stdout:9/181: write d1/d7/d23/d28/f39 [823688,96419] 0 2024-05-13T22:48:26.714 INFO:tasks.workunit.client.1.smithi130.stdout:0/188: fdatasync d0/f1e 0 2024-05-13T22:48:26.716 INFO:tasks.workunit.client.1.smithi130.stdout:9/182: dwrite d1/d15/f16 [0,4194304] 0 2024-05-13T22:48:26.718 INFO:tasks.workunit.client.0.smithi102.stdout:0/930: dwrite d1/de/d32/d24/f30 [0,4194304] 0 2024-05-13T22:48:26.718 INFO:tasks.workunit.client.1.smithi130.stdout:2/200: link d4/db/f3a d4/d49/d42/f53 0 2024-05-13T22:48:26.719 INFO:tasks.workunit.client.1.smithi130.stdout:2/201: write d4/db/dc/d18/f1d [1458171,30397] 0 2024-05-13T22:48:26.721 INFO:tasks.workunit.client.0.smithi102.stdout:9/930: write d1/d6/d21/d12e/d75/d96/d102/d6c/f86 [2508061,57419] 0 2024-05-13T22:48:26.723 INFO:tasks.workunit.client.0.smithi102.stdout:8/923: write d2/d7/d27/d64/d80/dc8/d93/fe6 [564350,4865] 0 2024-05-13T22:48:26.727 INFO:tasks.workunit.client.0.smithi102.stdout:1/804: sync 2024-05-13T22:48:26.730 INFO:tasks.workunit.client.0.smithi102.stdout:9/931: dwrite d1/d6/d45/f67 [0,4194304] 0 2024-05-13T22:48:26.731 INFO:tasks.workunit.client.1.smithi130.stdout:6/220: creat d4/f49 x:0 0 0 2024-05-13T22:48:26.732 INFO:tasks.workunit.client.1.smithi130.stdout:6/221: write d4/d15/d1f/f20 [1736655,47427] 0 2024-05-13T22:48:26.732 INFO:tasks.workunit.client.1.smithi130.stdout:6/222: readlink d4/d15/d1f/l43 0 2024-05-13T22:48:26.734 INFO:tasks.workunit.client.0.smithi102.stdout:4/951: getdents d3/d19/d23/d7e 0 2024-05-13T22:48:26.736 INFO:tasks.workunit.client.0.smithi102.stdout:9/932: dread - d1/d6/d45/d2d/dac/fb4 zero size 2024-05-13T22:48:26.737 INFO:tasks.workunit.client.1.smithi130.stdout:5/190: write d6/d27/f34 [683019,31717] 0 2024-05-13T22:48:26.739 INFO:tasks.workunit.client.0.smithi102.stdout:9/933: dread - d1/d6b/f132 zero size 2024-05-13T22:48:26.740 INFO:tasks.workunit.client.0.smithi102.stdout:1/805: dwrite d2/d10/d62/d7b/dc4/fc8 [0,4194304] 0 2024-05-13T22:48:26.740 INFO:tasks.workunit.client.0.smithi102.stdout:0/931: symlink d1/de/d32/dd8/l122 0 2024-05-13T22:48:26.743 INFO:tasks.workunit.client.0.smithi102.stdout:8/924: truncate d2/d7/d27/d64/d80/dc8/d40/f9c 364527 0 2024-05-13T22:48:26.746 INFO:tasks.workunit.client.0.smithi102.stdout:4/952: fdatasync d3/d19/d23/d3a/ff6 0 2024-05-13T22:48:26.750 INFO:tasks.workunit.client.0.smithi102.stdout:9/934: unlink d1/f16 0 2024-05-13T22:48:26.750 INFO:tasks.workunit.client.1.smithi130.stdout:7/141: sync 2024-05-13T22:48:26.752 INFO:tasks.workunit.client.1.smithi130.stdout:2/202: symlink d4/db/l54 0 2024-05-13T22:48:26.757 INFO:tasks.workunit.client.0.smithi102.stdout:0/932: unlink d1/l6d 0 2024-05-13T22:48:26.758 INFO:tasks.workunit.client.0.smithi102.stdout:9/935: creat d1/d6/d21/d12e/d75/d96/dc6/dde/f142 x:0 0 0 2024-05-13T22:48:26.762 INFO:tasks.workunit.client.1.smithi130.stdout:1/176: link db/d1b/f20 db/d1b/d1f/d2a/d35/f3d 0 2024-05-13T22:48:26.763 INFO:tasks.workunit.client.0.smithi102.stdout:0/933: fdatasync d1/d68/fd4 0 2024-05-13T22:48:26.767 INFO:tasks.workunit.client.0.smithi102.stdout:9/936: mkdir d1/d6/d21/d12e/d75/db9/d143 0 2024-05-13T22:48:26.768 INFO:tasks.workunit.client.1.smithi130.stdout:8/163: getdents d1/d2/db/d15/d16 0 2024-05-13T22:48:26.769 INFO:tasks.workunit.client.0.smithi102.stdout:8/925: write d2/d7/d27/d64/ff1 [3181124,37611] 0 2024-05-13T22:48:26.771 INFO:tasks.workunit.client.0.smithi102.stdout:4/953: write d3/d4/d6/de/d1f/d41/fc7 [451004,5518] 0 2024-05-13T22:48:26.772 INFO:tasks.workunit.client.0.smithi102.stdout:0/934: mknod d1/d68/c123 0 2024-05-13T22:48:26.774 INFO:tasks.workunit.client.0.smithi102.stdout:1/806: getdents d2/d4/d5/d19/ded/d6a/dbd 0 2024-05-13T22:48:26.775 INFO:tasks.workunit.client.0.smithi102.stdout:1/807: dread - d2/d4/d5/d19/ded/ff6 zero size 2024-05-13T22:48:26.776 INFO:tasks.workunit.client.1.smithi130.stdout:5/191: mknod d6/c3f 0 2024-05-13T22:48:26.777 INFO:tasks.workunit.client.0.smithi102.stdout:9/937: symlink d1/d6/d21/d12e/d75/db9/l144 0 2024-05-13T22:48:26.777 INFO:tasks.workunit.client.1.smithi130.stdout:9/183: write d1/f1e [56724,120508] 0 2024-05-13T22:48:26.777 INFO:tasks.workunit.client.1.smithi130.stdout:4/196: link d8/l15 d8/dc/d13/l49 0 2024-05-13T22:48:26.778 INFO:tasks.workunit.client.1.smithi130.stdout:4/197: stat d8/df/f3c 0 2024-05-13T22:48:26.779 INFO:tasks.workunit.client.0.smithi102.stdout:4/954: fdatasync d3/d6b/fdf 0 2024-05-13T22:48:26.781 INFO:tasks.workunit.client.0.smithi102.stdout:0/935: unlink d1/de/f1d 0 2024-05-13T22:48:26.781 INFO:tasks.workunit.client.1.smithi130.stdout:7/142: symlink d7/d8/d9/l34 0 2024-05-13T22:48:26.782 INFO:tasks.workunit.client.0.smithi102.stdout:4/955: read d3/d4/f32 [609077,86827] 0 2024-05-13T22:48:26.782 INFO:tasks.workunit.client.0.smithi102.stdout:1/808: readlink d2/d4/d5/d19/l41 0 2024-05-13T22:48:26.785 INFO:tasks.workunit.client.1.smithi130.stdout:2/203: link d4/f1a d4/db/d26/f55 0 2024-05-13T22:48:26.786 INFO:tasks.workunit.client.0.smithi102.stdout:8/926: unlink d2/l1b 0 2024-05-13T22:48:26.788 INFO:tasks.workunit.client.0.smithi102.stdout:8/927: truncate d2/d7/d27/d64/d96/dae/d78/f123 82649 0 2024-05-13T22:48:26.788 INFO:tasks.workunit.client.1.smithi130.stdout:6/223: rename d4/c1a to d4/c4a 0 2024-05-13T22:48:26.789 INFO:tasks.workunit.client.1.smithi130.stdout:3/170: getdents d8/dc/d19 0 2024-05-13T22:48:26.792 INFO:tasks.workunit.client.1.smithi130.stdout:9/184: creat d1/d15/f3b x:0 0 0 2024-05-13T22:48:26.793 INFO:tasks.workunit.client.1.smithi130.stdout:0/189: getdents d0 0 2024-05-13T22:48:26.794 INFO:tasks.workunit.client.0.smithi102.stdout:1/809: unlink d2/d4/d26/f90 0 2024-05-13T22:48:26.795 INFO:tasks.workunit.client.0.smithi102.stdout:1/810: stat d2/d10/d62/d7b/f82 0 2024-05-13T22:48:26.795 INFO:tasks.workunit.client.1.smithi130.stdout:8/164: dwrite d1/d2/db/d15/f30 [0,4194304] 0 2024-05-13T22:48:26.798 INFO:tasks.workunit.client.0.smithi102.stdout:9/938: mkdir d1/d6/db/d145 0 2024-05-13T22:48:26.799 INFO:tasks.workunit.client.1.smithi130.stdout:7/143: read d7/d8/f2d [165765,126963] 0 2024-05-13T22:48:26.804 INFO:tasks.workunit.client.1.smithi130.stdout:2/204: mkdir d4/d56 0 2024-05-13T22:48:26.805 INFO:tasks.workunit.client.0.smithi102.stdout:0/936: fsync d1/de/d32/da6/fc4 0 2024-05-13T22:48:26.806 INFO:tasks.workunit.client.1.smithi130.stdout:6/224: mknod d4/d16/c4b 0 2024-05-13T22:48:26.809 INFO:tasks.workunit.client.1.smithi130.stdout:3/171: creat d8/f33 x:0 0 0 2024-05-13T22:48:26.810 INFO:tasks.workunit.client.0.smithi102.stdout:1/811: symlink d2/d4/d38/db2/dba/l108 0 2024-05-13T22:48:26.813 INFO:tasks.workunit.client.1.smithi130.stdout:5/192: creat d6/d1a/f40 x:0 0 0 2024-05-13T22:48:26.820 INFO:tasks.workunit.client.0.smithi102.stdout:9/939: mkdir d1/d6/d21/d12e/d75/db9/d146 0 2024-05-13T22:48:26.822 INFO:tasks.workunit.client.1.smithi130.stdout:9/185: creat d1/d7/d23/d28/d2d/d2f/f3c x:0 0 0 2024-05-13T22:48:26.823 INFO:tasks.workunit.client.0.smithi102.stdout:8/928: creat d2/d7/d27/d109/f125 x:0 0 0 2024-05-13T22:48:26.827 INFO:tasks.workunit.client.1.smithi130.stdout:4/198: mknod d8/dc/d35/d33/d44/c4a 0 2024-05-13T22:48:26.827 INFO:tasks.workunit.client.1.smithi130.stdout:9/186: dwrite d1/d15/f37 [0,4194304] 0 2024-05-13T22:48:26.829 INFO:tasks.workunit.client.1.smithi130.stdout:9/187: dread - d1/d7/d23/d28/d2d/f3a zero size 2024-05-13T22:48:26.830 INFO:tasks.workunit.client.1.smithi130.stdout:0/190: rename d0/d7/dd/d17/d1f/d2c/c3a to d0/d7/dd/c3e 0 2024-05-13T22:48:26.832 INFO:tasks.workunit.client.0.smithi102.stdout:4/956: creat d3/d19/f14d x:0 0 0 2024-05-13T22:48:26.837 INFO:tasks.workunit.client.1.smithi130.stdout:3/172: rename d8/f26 to d8/f34 0 2024-05-13T22:48:26.838 INFO:tasks.workunit.client.0.smithi102.stdout:7/886: sync 2024-05-13T22:48:26.839 INFO:tasks.workunit.client.1.smithi130.stdout:7/144: dwrite d7/da/d1f/f30 [0,4194304] 0 2024-05-13T22:48:26.839 INFO:tasks.workunit.client.0.smithi102.stdout:8/929: creat d2/d7/d27/d64/d96/dbc/f126 x:0 0 0 2024-05-13T22:48:26.840 INFO:tasks.workunit.client.0.smithi102.stdout:7/887: dread - d6/d13/d2c/da1/f101 zero size 2024-05-13T22:48:26.841 INFO:tasks.workunit.client.1.smithi130.stdout:5/193: creat d6/d7/d24/f41 x:0 0 0 2024-05-13T22:48:26.841 INFO:tasks.workunit.client.0.smithi102.stdout:7/888: chown d6/d13/d2c/d39 0 1 2024-05-13T22:48:26.842 INFO:tasks.workunit.client.1.smithi130.stdout:5/194: chown d6/d7/d32/f38 94803344 1 2024-05-13T22:48:26.844 INFO:tasks.workunit.client.0.smithi102.stdout:1/812: write d2/d45/f48 [3267144,74824] 0 2024-05-13T22:48:26.845 INFO:tasks.workunit.client.0.smithi102.stdout:9/940: write d1/d6/d21/d71/fb3 [355768,116986] 0 2024-05-13T22:48:26.850 INFO:tasks.workunit.client.1.smithi130.stdout:5/195: dwrite d6/d1a/f40 [0,4194304] 0 2024-05-13T22:48:26.851 INFO:tasks.workunit.client.0.smithi102.stdout:0/937: mkdir d1/d124 0 2024-05-13T22:48:26.854 INFO:tasks.workunit.client.0.smithi102.stdout:1/813: creat d2/d10/d62/d86/f109 x:0 0 0 2024-05-13T22:48:26.855 INFO:tasks.workunit.client.0.smithi102.stdout:1/814: fsync d2/fc2 0 2024-05-13T22:48:26.856 INFO:tasks.workunit.client.0.smithi102.stdout:1/815: write d2/d4/d38/f68 [4186198,86859] 0 2024-05-13T22:48:26.857 INFO:tasks.workunit.client.1.smithi130.stdout:2/205: truncate d4/f7 5542592 0 2024-05-13T22:48:26.860 INFO:tasks.workunit.client.0.smithi102.stdout:4/957: fsync d3/d19/d23/d3a/f9d 0 2024-05-13T22:48:26.869 INFO:tasks.workunit.client.1.smithi130.stdout:0/191: dread d0/d7/dd/d17/d1f/f25 [4194304,4194304] 0 2024-05-13T22:48:26.869 INFO:tasks.workunit.client.1.smithi130.stdout:5/196: symlink d6/d27/l42 0 2024-05-13T22:48:26.870 INFO:tasks.workunit.client.1.smithi130.stdout:9/188: truncate d1/d7/d23/d28/d2d/f3a 212415 0 2024-05-13T22:48:26.873 INFO:tasks.workunit.client.1.smithi130.stdout:8/165: truncate d1/d2/db/d15/f30 1143761 0 2024-05-13T22:48:26.877 INFO:tasks.workunit.client.1.smithi130.stdout:2/206: symlink d4/l57 0 2024-05-13T22:48:26.878 INFO:tasks.workunit.client.0.smithi102.stdout:4/958: mknod d3/d19/d23/d4e/d68/d85/c14e 0 2024-05-13T22:48:26.880 INFO:tasks.workunit.client.0.smithi102.stdout:4/959: dread - d3/d19/d23/d39/ddd/f12a zero size 2024-05-13T22:48:26.880 INFO:tasks.workunit.client.1.smithi130.stdout:6/225: rename d4/d15/c17 to d4/c4c 0 2024-05-13T22:48:26.881 INFO:tasks.workunit.client.1.smithi130.stdout:6/226: stat d4/d15/c45 0 2024-05-13T22:48:26.883 INFO:tasks.workunit.client.1.smithi130.stdout:0/192: creat d0/d7/dd/d2e/f3f x:0 0 0 2024-05-13T22:48:26.884 INFO:tasks.workunit.client.0.smithi102.stdout:9/941: dwrite d1/d6/d45/fe5 [0,4194304] 0 2024-05-13T22:48:26.884 INFO:tasks.workunit.client.1.smithi130.stdout:0/193: truncate d0/fa 2243518 0 2024-05-13T22:48:26.884 INFO:tasks.workunit.client.0.smithi102.stdout:8/930: link d2/d7/d27/d64/d80/dc8/d40/cb4 d2/d36/d49/d7e/c127 0 2024-05-13T22:48:26.887 INFO:tasks.workunit.client.1.smithi130.stdout:5/197: creat d6/d7/f43 x:0 0 0 2024-05-13T22:48:26.888 INFO:tasks.workunit.client.0.smithi102.stdout:8/931: read d2/d7/d27/d64/d80/dc8/d40/f5b [1929988,43080] 0 2024-05-13T22:48:26.890 INFO:tasks.workunit.client.0.smithi102.stdout:1/816: unlink d2/d10/lcd 0 2024-05-13T22:48:26.892 INFO:tasks.workunit.client.1.smithi130.stdout:1/177: sync 2024-05-13T22:48:26.894 INFO:tasks.workunit.client.1.smithi130.stdout:2/207: mkdir d4/db/dc/d18/d58 0 2024-05-13T22:48:26.897 INFO:tasks.workunit.client.1.smithi130.stdout:4/199: rename d8/dc/d13/f19 to d8/dc/d35/d33/d44/f4b 0 2024-05-13T22:48:26.903 INFO:tasks.workunit.client.0.smithi102.stdout:8/932: unlink d2/d7/d27/d64/d80/dc8/d40/d85/feb 0 2024-05-13T22:48:26.905 INFO:tasks.workunit.client.1.smithi130.stdout:5/198: rename d6/d7/d32/f38 to d6/d7/d32/f44 0 2024-05-13T22:48:26.907 INFO:tasks.workunit.client.0.smithi102.stdout:0/938: readlink d1/de/d32/ld3 0 2024-05-13T22:48:26.910 INFO:tasks.workunit.client.0.smithi102.stdout:7/889: link d6/d15/d4c/cad d6/d13/d58/c119 0 2024-05-13T22:48:26.911 INFO:tasks.workunit.client.1.smithi130.stdout:5/199: dwrite d6/d7/d19/f1c [0,4194304] 0 2024-05-13T22:48:26.913 INFO:tasks.workunit.client.0.smithi102.stdout:9/942: mkdir d1/d6/d45/d147 0 2024-05-13T22:48:26.916 INFO:tasks.workunit.client.1.smithi130.stdout:3/173: rename d8/dc/d19/d1d to d8/d35 0 2024-05-13T22:48:26.918 INFO:tasks.workunit.client.1.smithi130.stdout:4/200: mknod d8/dc/d29/c4c 0 2024-05-13T22:48:26.919 INFO:tasks.workunit.client.1.smithi130.stdout:4/201: readlink d8/l9 0 2024-05-13T22:48:26.920 INFO:tasks.workunit.client.1.smithi130.stdout:4/202: truncate d8/dc/d29/f3a 956636 0 2024-05-13T22:48:26.920 INFO:tasks.workunit.client.0.smithi102.stdout:1/817: write d2/d10/f31 [8004374,64747] 0 2024-05-13T22:48:26.921 INFO:tasks.workunit.client.0.smithi102.stdout:1/818: readlink d2/d45/d96/l100 0 2024-05-13T22:48:26.923 INFO:tasks.workunit.client.0.smithi102.stdout:1/819: write d2/d45/d79/d92/f9d [6980610,115847] 0 2024-05-13T22:48:26.923 INFO:tasks.workunit.client.0.smithi102.stdout:7/890: chown d6/d13/d58/f34 3195518 1 2024-05-13T22:48:26.924 INFO:tasks.workunit.client.0.smithi102.stdout:8/933: write d2/f22 [993472,19741] 0 2024-05-13T22:48:26.943 INFO:tasks.workunit.client.0.smithi102.stdout:4/960: dwrite d3/d19/dad/dc9/d147/f148 [0,4194304] 0 2024-05-13T22:48:26.944 INFO:tasks.workunit.client.1.smithi130.stdout:8/166: creat d1/d2/f39 x:0 0 0 2024-05-13T22:48:26.945 INFO:tasks.workunit.client.1.smithi130.stdout:1/178: fsync db/d1b/f20 0 2024-05-13T22:48:26.945 INFO:tasks.workunit.client.0.smithi102.stdout:7/891: readlink d6/l1c 0 2024-05-13T22:48:26.947 INFO:tasks.workunit.client.1.smithi130.stdout:2/208: creat d4/db/dc/d18/d58/f59 x:0 0 0 2024-05-13T22:48:26.948 INFO:tasks.workunit.client.0.smithi102.stdout:0/939: creat d1/d4d/d79/d9f/d104/f125 x:0 0 0 2024-05-13T22:48:26.951 INFO:tasks.workunit.client.0.smithi102.stdout:1/820: truncate d2/d45/fd6 4884836 0 2024-05-13T22:48:26.952 INFO:tasks.workunit.client.0.smithi102.stdout:1/821: dread - d2/d10/d62/d7b/fd9 zero size 2024-05-13T22:48:26.952 INFO:tasks.workunit.client.0.smithi102.stdout:4/961: truncate d3/d22/f3e 2164119 0 2024-05-13T22:48:26.954 INFO:tasks.workunit.client.0.smithi102.stdout:7/892: rmdir d6/d15/d4d/d55/d6f/d65/ded 39 2024-05-13T22:48:26.954 INFO:tasks.workunit.client.0.smithi102.stdout:4/962: truncate d3/d19/d23/d3a/da4/f108 2783325 0 2024-05-13T22:48:26.955 INFO:tasks.workunit.client.1.smithi130.stdout:5/200: dwrite d6/dd/f37 [0,4194304] 0 2024-05-13T22:48:26.955 INFO:tasks.workunit.client.1.smithi130.stdout:5/201: chown d6/d2f/f39 28 1 2024-05-13T22:48:26.957 INFO:tasks.workunit.client.1.smithi130.stdout:8/167: creat d1/dc/f3a x:0 0 0 2024-05-13T22:48:26.958 INFO:tasks.workunit.client.1.smithi130.stdout:8/168: stat d1/d2/le 0 2024-05-13T22:48:26.958 INFO:tasks.workunit.client.1.smithi130.stdout:7/145: sync 2024-05-13T22:48:26.958 INFO:tasks.workunit.client.0.smithi102.stdout:4/963: dread - d3/d4/d6/f121 zero size 2024-05-13T22:48:26.958 INFO:tasks.workunit.client.0.smithi102.stdout:7/893: dread d6/f85 [0,4194304] 0 2024-05-13T22:48:26.959 INFO:tasks.workunit.client.1.smithi130.stdout:7/146: fsync d7/da/d1f/f30 0 2024-05-13T22:48:26.960 INFO:tasks.workunit.client.1.smithi130.stdout:1/179: symlink db/d1b/d1f/d2a/l3e 0 2024-05-13T22:48:26.962 INFO:tasks.workunit.client.0.smithi102.stdout:0/940: readlink d1/de/d32/d8d/ld0 0 2024-05-13T22:48:26.964 INFO:tasks.workunit.client.0.smithi102.stdout:0/941: stat d1/de/d32/d8a/fd6 0 2024-05-13T22:48:26.964 INFO:tasks.workunit.client.1.smithi130.stdout:7/147: dwrite d7/da/d1f/f30 [0,4194304] 0 2024-05-13T22:48:26.965 INFO:tasks.workunit.client.0.smithi102.stdout:8/934: creat d2/d7/d25/f128 x:0 0 0 2024-05-13T22:48:26.967 INFO:tasks.workunit.client.0.smithi102.stdout:1/822: dwrite d2/d45/d79/faa [0,4194304] 0 2024-05-13T22:48:26.968 INFO:tasks.workunit.client.0.smithi102.stdout:8/935: write d2/d36/d49/d5d/f45 [1377161,3266] 0 2024-05-13T22:48:26.968 INFO:tasks.workunit.client.0.smithi102.stdout:4/964: rmdir d3/d4/d6/de/d115/d11b 39 2024-05-13T22:48:26.971 INFO:tasks.workunit.client.0.smithi102.stdout:4/965: stat d3/d19/d23/d39/da7/da9/dd3/lea 0 2024-05-13T22:48:26.974 INFO:tasks.workunit.client.1.smithi130.stdout:6/227: getdents d4/d15/d1f 0 2024-05-13T22:48:26.975 INFO:tasks.workunit.client.0.smithi102.stdout:0/942: readlink d1/l99 0 2024-05-13T22:48:26.975 INFO:tasks.workunit.client.1.smithi130.stdout:4/203: dwrite d8/f1b [0,4194304] 0 2024-05-13T22:48:26.976 INFO:tasks.workunit.client.1.smithi130.stdout:5/202: creat d6/d7/f45 x:0 0 0 2024-05-13T22:48:26.978 INFO:tasks.workunit.client.1.smithi130.stdout:0/194: getdents d0/d7/dd/d17/d1f 0 2024-05-13T22:48:26.980 INFO:tasks.workunit.client.1.smithi130.stdout:0/195: chown d0/d7/dd/d2e/f2f 102 1 2024-05-13T22:48:26.982 INFO:tasks.workunit.client.1.smithi130.stdout:8/169: creat d1/d2/db/f3b x:0 0 0 2024-05-13T22:48:26.983 INFO:tasks.workunit.client.0.smithi102.stdout:4/966: creat d3/d19/d23/d3a/f14f x:0 0 0 2024-05-13T22:48:26.984 INFO:tasks.workunit.client.0.smithi102.stdout:4/967: chown d3/d19/d23/d39/ddd 2809 1 2024-05-13T22:48:26.985 INFO:tasks.workunit.client.1.smithi130.stdout:0/196: dread d0/d7/dd/d17/d1f/f25 [0,4194304] 0 2024-05-13T22:48:26.985 INFO:tasks.workunit.client.0.smithi102.stdout:7/894: dread d6/d15/d4c/f98 [0,4194304] 0 2024-05-13T22:48:26.987 INFO:tasks.workunit.client.1.smithi130.stdout:3/174: rename d8/f9 to d8/dc/f36 0 2024-05-13T22:48:26.987 INFO:tasks.workunit.client.1.smithi130.stdout:3/175: stat d8/f16 0 2024-05-13T22:48:26.988 INFO:tasks.workunit.client.0.smithi102.stdout:8/936: symlink d2/d7/d27/d64/d80/dc8/d93/l129 0 2024-05-13T22:48:26.991 INFO:tasks.workunit.client.1.smithi130.stdout:2/209: dwrite d4/f9 [0,4194304] 0 2024-05-13T22:48:26.992 INFO:tasks.workunit.client.0.smithi102.stdout:7/895: dwrite d6/d13/d2c/da1/dc1/f66 [0,4194304] 0 2024-05-13T22:48:26.992 INFO:tasks.workunit.client.0.smithi102.stdout:7/896: stat d6/d15/f5e 0 2024-05-13T22:48:26.993 INFO:tasks.workunit.client.1.smithi130.stdout:9/189: sync 2024-05-13T22:48:26.993 INFO:tasks.workunit.client.1.smithi130.stdout:6/228: read d4/d15/f44 [1332714,27731] 0 2024-05-13T22:48:26.994 INFO:tasks.workunit.client.1.smithi130.stdout:4/204: write d8/dc/d26/f36 [610887,120709] 0 2024-05-13T22:48:26.996 INFO:tasks.workunit.client.0.smithi102.stdout:8/937: dread - d2/d36/fd3 zero size 2024-05-13T22:48:26.997 INFO:tasks.workunit.client.1.smithi130.stdout:0/197: dwrite d0/d7/d10/f15 [0,4194304] 0 2024-05-13T22:48:26.998 INFO:tasks.workunit.client.0.smithi102.stdout:0/943: rename d1/de/d32/d24/ff1 to d1/d68/de2/f126 0 2024-05-13T22:48:26.999 INFO:tasks.workunit.client.1.smithi130.stdout:6/229: chown d4/d15/c45 6 1 2024-05-13T22:48:27.002 INFO:tasks.workunit.client.1.smithi130.stdout:6/230: chown d4 4308 1 2024-05-13T22:48:27.002 INFO:tasks.workunit.client.1.smithi130.stdout:1/180: dread db/f17 [0,4194304] 0 2024-05-13T22:48:27.002 INFO:tasks.workunit.client.1.smithi130.stdout:3/176: truncate d8/f2b 4080460 0 2024-05-13T22:48:27.007 INFO:tasks.workunit.client.1.smithi130.stdout:5/203: link d6/d27/f2d d6/d7/f46 0 2024-05-13T22:48:27.007 INFO:tasks.workunit.client.1.smithi130.stdout:4/205: symlink d8/dc/l4d 0 2024-05-13T22:48:27.008 INFO:tasks.workunit.client.1.smithi130.stdout:2/210: symlink d4/db/dc/d18/d2a/d30/l5a 0 2024-05-13T22:48:27.008 INFO:tasks.workunit.client.1.smithi130.stdout:8/170: dread d1/d2/db/f8 [0,4194304] 0 2024-05-13T22:48:27.010 INFO:tasks.workunit.client.1.smithi130.stdout:8/171: chown d1/c32 1591 1 2024-05-13T22:48:27.010 INFO:tasks.workunit.client.0.smithi102.stdout:8/938: dread - d2/d7/d27/d64/d96/dae/d78/f113 zero size 2024-05-13T22:48:27.011 INFO:tasks.workunit.client.1.smithi130.stdout:8/172: chown d1/d2/fd 2599570 1 2024-05-13T22:48:27.012 INFO:tasks.workunit.client.1.smithi130.stdout:0/198: creat d0/d7/d29/f40 x:0 0 0 2024-05-13T22:48:27.014 INFO:tasks.workunit.client.0.smithi102.stdout:8/939: creat d2/d36/db3/f12a x:0 0 0 2024-05-13T22:48:27.017 INFO:tasks.workunit.client.1.smithi130.stdout:8/173: dwrite d1/d2/db/d15/f22 [0,4194304] 0 2024-05-13T22:48:27.022 INFO:tasks.workunit.client.0.smithi102.stdout:8/940: dwrite d2/d7/d25/ff8 [0,4194304] 0 2024-05-13T22:48:27.029 INFO:tasks.workunit.client.1.smithi130.stdout:7/148: rename d7/d8/fe to d7/da/f35 0 2024-05-13T22:48:27.031 INFO:tasks.workunit.client.0.smithi102.stdout:1/823: truncate d2/d10/d62/d7b/dc4/fc8 1152314 0 2024-05-13T22:48:27.036 INFO:tasks.workunit.client.0.smithi102.stdout:8/941: getdents d2/d7/d27/d64/d96/dbc/d106 0 2024-05-13T22:48:27.039 INFO:tasks.workunit.client.1.smithi130.stdout:4/206: dread d8/dc/d26/f36 [0,4194304] 0 2024-05-13T22:48:27.039 INFO:tasks.workunit.client.0.smithi102.stdout:8/942: truncate d2/d36/d49/f114 358888 0 2024-05-13T22:48:27.041 INFO:tasks.workunit.client.0.smithi102.stdout:4/968: dwrite d3/d4/d6/de/d1f/f142 [0,4194304] 0 2024-05-13T22:48:27.045 INFO:tasks.workunit.client.0.smithi102.stdout:7/897: dwrite d6/d15/d4c/fc5 [0,4194304] 0 2024-05-13T22:48:27.048 INFO:tasks.workunit.client.1.smithi130.stdout:2/211: rmdir d4/db 39 2024-05-13T22:48:27.048 INFO:tasks.workunit.client.0.smithi102.stdout:7/898: write d6/d15/f5e [2473919,108305] 0 2024-05-13T22:48:27.050 INFO:tasks.workunit.client.1.smithi130.stdout:1/181: write db/f1a [1162165,17405] 0 2024-05-13T22:48:27.052 INFO:tasks.workunit.client.0.smithi102.stdout:8/943: mkdir d2/d7/d27/d64/d80/dc8/d40/d12b 0 2024-05-13T22:48:27.053 INFO:tasks.workunit.client.1.smithi130.stdout:2/212: dread d4/f5 [0,4194304] 0 2024-05-13T22:48:27.055 INFO:tasks.workunit.client.1.smithi130.stdout:3/177: creat d8/dc/f37 x:0 0 0 2024-05-13T22:48:27.055 INFO:tasks.workunit.client.1.smithi130.stdout:1/182: dread db/f17 [0,4194304] 0 2024-05-13T22:48:27.056 INFO:tasks.workunit.client.1.smithi130.stdout:9/190: truncate d1/d7/fa 85488 0 2024-05-13T22:48:27.056 INFO:tasks.workunit.client.1.smithi130.stdout:1/183: chown db/l36 213 1 2024-05-13T22:48:27.056 INFO:tasks.workunit.client.1.smithi130.stdout:1/184: fdatasync db/f16 0 2024-05-13T22:48:27.057 INFO:tasks.workunit.client.0.smithi102.stdout:0/944: mkdir d1/de/d32/d24/d2d/d91/d127 0 2024-05-13T22:48:27.058 INFO:tasks.workunit.client.0.smithi102.stdout:0/945: write d1/de/d32/f37 [1332901,130592] 0 2024-05-13T22:48:27.060 INFO:tasks.workunit.client.0.smithi102.stdout:4/969: read d3/d19/d23/d39/d5d/f96 [1216542,100878] 0 2024-05-13T22:48:27.061 INFO:tasks.workunit.client.1.smithi130.stdout:4/207: symlink d8/dc/d35/d33/l4e 0 2024-05-13T22:48:27.062 INFO:tasks.workunit.client.0.smithi102.stdout:1/824: link d2/d10/d62/d86/ceb d2/d45/da4/def/c10a 0 2024-05-13T22:48:27.062 INFO:tasks.workunit.client.1.smithi130.stdout:8/174: write d1/dc/f1f [1214764,110917] 0 2024-05-13T22:48:27.063 INFO:tasks.workunit.client.1.smithi130.stdout:6/231: getdents d4/d15/d1d 0 2024-05-13T22:48:27.067 INFO:tasks.workunit.client.1.smithi130.stdout:4/208: dwrite d8/f1f [0,4194304] 0 2024-05-13T22:48:27.068 INFO:tasks.workunit.client.1.smithi130.stdout:4/209: chown d8/dc 483810 1 2024-05-13T22:48:27.070 INFO:tasks.workunit.client.1.smithi130.stdout:0/199: rename d0/fa to d0/d7/dd/f41 0 2024-05-13T22:48:27.071 INFO:tasks.workunit.client.0.smithi102.stdout:8/944: creat d2/d36/d49/d7e/de2/f12c x:0 0 0 2024-05-13T22:48:27.075 INFO:tasks.workunit.client.1.smithi130.stdout:7/149: dwrite d7/d8/f2d [0,4194304] 0 2024-05-13T22:48:27.076 INFO:tasks.workunit.client.1.smithi130.stdout:7/150: chown d7/d8/l14 99338035 1 2024-05-13T22:48:27.079 INFO:tasks.workunit.client.0.smithi102.stdout:8/945: dwrite d2/d7/d27/d64/f11d [0,4194304] 0 2024-05-13T22:48:27.081 INFO:tasks.workunit.client.0.smithi102.stdout:4/970: mknod d3/dee/c150 0 2024-05-13T22:48:27.081 INFO:tasks.workunit.client.0.smithi102.stdout:8/946: write d2/d36/d49/d5d/f26 [4231099,44499] 0 2024-05-13T22:48:27.083 INFO:tasks.workunit.client.0.smithi102.stdout:4/971: readlink d3/d19/d23/d4e/l9b 0 2024-05-13T22:48:27.083 INFO:tasks.workunit.client.1.smithi130.stdout:9/191: mkdir d1/d7/d23/d3d 0 2024-05-13T22:48:27.090 INFO:tasks.workunit.client.0.smithi102.stdout:7/899: link d6/d15/d4d/d55/d6f/d49/f100 d6/d13/d2c/da1/d103/f11a 0 2024-05-13T22:48:27.093 INFO:tasks.workunit.client.1.smithi130.stdout:5/204: getdents d6/dd 0 2024-05-13T22:48:27.097 INFO:tasks.workunit.client.0.smithi102.stdout:8/947: write d2/d7/d27/d64/d96/dae/d78/f123 [512018,34384] 0 2024-05-13T22:48:27.098 INFO:tasks.workunit.client.0.smithi102.stdout:8/948: write d2/d7/d25/ff8 [840135,56204] 0 2024-05-13T22:48:27.104 INFO:tasks.workunit.client.0.smithi102.stdout:1/825: dwrite d2/d4/d26/d40/fd4 [4194304,4194304] 0 2024-05-13T22:48:27.105 INFO:tasks.workunit.client.0.smithi102.stdout:7/900: chown d6/f77 28 1 2024-05-13T22:48:27.106 INFO:tasks.workunit.client.0.smithi102.stdout:7/901: chown d6/d15/d4c/fc5 1003 1 2024-05-13T22:48:27.107 INFO:tasks.workunit.client.0.smithi102.stdout:7/902: fsync d6/d15/d4d/d55/d7f/fee 0 2024-05-13T22:48:27.108 INFO:tasks.workunit.client.1.smithi130.stdout:2/213: mknod d4/c5b 0 2024-05-13T22:48:27.113 INFO:tasks.workunit.client.0.smithi102.stdout:8/949: creat d2/d36/d49/f12d x:0 0 0 2024-05-13T22:48:27.113 INFO:tasks.workunit.client.1.smithi130.stdout:3/178: dread d8/d1a/d1c/f25 [0,4194304] 0 2024-05-13T22:48:27.115 INFO:tasks.workunit.client.1.smithi130.stdout:1/185: truncate db/f17 1737493 0 2024-05-13T22:48:27.116 INFO:tasks.workunit.client.0.smithi102.stdout:1/826: mkdir d2/d4/d38/db2/dba/d10b 0 2024-05-13T22:48:27.117 INFO:tasks.workunit.client.1.smithi130.stdout:5/205: chown d6/f1f 44785381 1 2024-05-13T22:48:27.118 INFO:tasks.workunit.client.0.smithi102.stdout:0/946: link d1/de/d32/l88 d1/de/d32/dab/dfb/l128 0 2024-05-13T22:48:27.118 INFO:tasks.workunit.client.0.smithi102.stdout:0/947: chown d1/de/d32/d8a/d116/d11b/d10d 808405 1 2024-05-13T22:48:27.120 INFO:tasks.workunit.client.0.smithi102.stdout:7/903: write d6/f7 [3733823,12333] 0 2024-05-13T22:48:27.121 INFO:tasks.workunit.client.1.smithi130.stdout:6/232: mknod d4/d15/c4d 0 2024-05-13T22:48:27.124 INFO:tasks.workunit.client.1.smithi130.stdout:5/206: dwrite d6/d7/f45 [0,4194304] 0 2024-05-13T22:48:27.125 INFO:tasks.workunit.client.1.smithi130.stdout:5/207: fsync d6/dd/d18/f36 0 2024-05-13T22:48:27.125 INFO:tasks.workunit.client.1.smithi130.stdout:0/200: dread - d0/d7/d29/f37 zero size 2024-05-13T22:48:27.126 INFO:tasks.workunit.client.1.smithi130.stdout:5/208: chown d6/dd/ff 4512 1 2024-05-13T22:48:27.127 INFO:tasks.workunit.client.1.smithi130.stdout:5/209: truncate d6/dd/d18/f36 296287 0 2024-05-13T22:48:27.128 INFO:tasks.workunit.client.1.smithi130.stdout:2/214: fsync d4/db/dc/d18/f36 0 2024-05-13T22:48:27.129 INFO:tasks.workunit.client.1.smithi130.stdout:7/151: mknod d7/c36 0 2024-05-13T22:48:27.129 INFO:tasks.workunit.client.1.smithi130.stdout:2/215: fsync d4/db/f41 0 2024-05-13T22:48:27.131 INFO:tasks.workunit.client.1.smithi130.stdout:2/216: dread - d4/db/dc/d18/d2a/d30/f47 zero size 2024-05-13T22:48:27.132 INFO:tasks.workunit.client.1.smithi130.stdout:9/192: symlink d1/d7/d23/d28/d32/l3e 0 2024-05-13T22:48:27.133 INFO:tasks.workunit.client.1.smithi130.stdout:2/217: read d4/f1a [135650,106704] 0 2024-05-13T22:48:27.133 INFO:tasks.workunit.client.1.smithi130.stdout:9/193: fdatasync d1/d15/f3b 0 2024-05-13T22:48:27.134 INFO:tasks.workunit.client.1.smithi130.stdout:9/194: read d1/f3 [2531208,71645] 0 2024-05-13T22:48:27.134 INFO:tasks.workunit.client.0.smithi102.stdout:7/904: chown d6/d13/d2c/d39/c111 6136691 1 2024-05-13T22:48:27.135 INFO:tasks.workunit.client.1.smithi130.stdout:9/195: dread - d1/d15/f3b zero size 2024-05-13T22:48:27.136 INFO:tasks.workunit.client.0.smithi102.stdout:4/972: getdents d3/d19/dad/dc9/d123 0 2024-05-13T22:48:27.138 INFO:tasks.workunit.client.1.smithi130.stdout:1/186: rmdir db 39 2024-05-13T22:48:27.139 INFO:tasks.workunit.client.0.smithi102.stdout:8/950: fsync d2/d36/d49/d7e/ffe 0 2024-05-13T22:48:27.140 INFO:tasks.workunit.client.0.smithi102.stdout:7/905: dwrite d6/da9/fbd [0,4194304] 0 2024-05-13T22:48:27.141 INFO:tasks.workunit.client.1.smithi130.stdout:8/175: dwrite d1/dc/f24 [0,4194304] 0 2024-05-13T22:48:27.142 INFO:tasks.workunit.client.0.smithi102.stdout:7/906: write d6/d13/d2c/da1/f101 [496186,73941] 0 2024-05-13T22:48:27.143 INFO:tasks.workunit.client.1.smithi130.stdout:6/233: fsync d4/d15/d1f/f3e 0 2024-05-13T22:48:27.143 INFO:tasks.workunit.client.0.smithi102.stdout:8/951: write d2/d7/d27/d64/d80/dc8/d40/db2/fba [1643178,35147] 0 2024-05-13T22:48:27.143 INFO:tasks.workunit.client.0.smithi102.stdout:8/952: chown d2/d7/d25/f6e 39847977 1 2024-05-13T22:48:27.146 INFO:tasks.workunit.client.0.smithi102.stdout:1/827: link d2/d4/d5/d19/l9f d2/d4/d9e/l10c 0 2024-05-13T22:48:27.149 INFO:tasks.workunit.client.1.smithi130.stdout:7/152: symlink d7/da/d1d/l37 0 2024-05-13T22:48:27.150 INFO:tasks.workunit.client.0.smithi102.stdout:1/828: truncate d2/d10/d62/fa8 837988 0 2024-05-13T22:48:27.151 INFO:tasks.workunit.client.1.smithi130.stdout:7/153: truncate d7/da/f33 230118 0 2024-05-13T22:48:27.151 INFO:tasks.workunit.client.1.smithi130.stdout:5/210: dread d6/dd/d18/f15 [0,4194304] 0 2024-05-13T22:48:27.152 INFO:tasks.workunit.client.1.smithi130.stdout:7/154: chown d7 10051092 1 2024-05-13T22:48:27.152 INFO:tasks.workunit.client.1.smithi130.stdout:2/218: symlink d4/db/dc/d18/d2a/l5c 0 2024-05-13T22:48:27.153 INFO:tasks.workunit.client.1.smithi130.stdout:7/155: fdatasync d7/da/d1f/f30 0 2024-05-13T22:48:27.157 INFO:tasks.workunit.client.1.smithi130.stdout:8/176: rmdir d1/dc 39 2024-05-13T22:48:27.157 INFO:tasks.workunit.client.0.smithi102.stdout:1/829: dread d2/d4/d5/d19/f53 [0,4194304] 0 2024-05-13T22:48:27.158 INFO:tasks.workunit.client.0.smithi102.stdout:1/830: dread - d2/d4/d5/d19/ded/ff6 zero size 2024-05-13T22:48:27.159 INFO:tasks.workunit.client.1.smithi130.stdout:3/179: dwrite d8/dc/d19/f1e [0,4194304] 0 2024-05-13T22:48:27.160 INFO:tasks.workunit.client.1.smithi130.stdout:4/210: getdents d8/dc/d35 0 2024-05-13T22:48:27.162 INFO:tasks.workunit.client.1.smithi130.stdout:6/234: mkdir d4/d15/d1d/d28/d4e 0 2024-05-13T22:48:27.163 INFO:tasks.workunit.client.1.smithi130.stdout:4/211: chown d8/dc/d29/c4c 1 1 2024-05-13T22:48:27.165 INFO:tasks.workunit.client.0.smithi102.stdout:1/831: mkdir d2/d4/d5/d19/ded/d10d 0 2024-05-13T22:48:27.166 INFO:tasks.workunit.client.0.smithi102.stdout:1/832: write d2/ff [856402,26239] 0 2024-05-13T22:48:27.168 INFO:tasks.workunit.client.1.smithi130.stdout:5/211: rmdir d6/d7 39 2024-05-13T22:48:27.168 INFO:tasks.workunit.client.0.smithi102.stdout:0/948: dread d1/de/d42/f8e [0,4194304] 0 2024-05-13T22:48:27.169 INFO:tasks.workunit.client.1.smithi130.stdout:6/235: dwrite d4/d16/f2a [0,4194304] 0 2024-05-13T22:48:27.171 INFO:tasks.workunit.client.1.smithi130.stdout:4/212: dwrite d8/df/f20 [0,4194304] 0 2024-05-13T22:48:27.175 INFO:tasks.workunit.client.0.smithi102.stdout:0/949: rename d1/d68/f86 to d1/de/d32/d8a/d116/d11b/d10d/f129 0 2024-05-13T22:48:27.177 INFO:tasks.workunit.client.0.smithi102.stdout:0/950: chown d1/de/d32/dab/dfb/ffd 1422 1 2024-05-13T22:48:27.181 INFO:tasks.workunit.client.0.smithi102.stdout:8/953: dread d2/d7/d27/dad/fe5 [0,4194304] 0 2024-05-13T22:48:27.181 INFO:tasks.workunit.client.1.smithi130.stdout:6/236: dwrite d4/d15/d1d/d28/f47 [0,4194304] 0 2024-05-13T22:48:27.181 INFO:tasks.workunit.client.1.smithi130.stdout:4/213: dwrite d8/dc/f38 [0,4194304] 0 2024-05-13T22:48:27.186 INFO:tasks.workunit.client.1.smithi130.stdout:2/219: creat d4/d49/d48/f5d x:0 0 0 2024-05-13T22:48:27.187 INFO:tasks.workunit.client.1.smithi130.stdout:2/220: readlink d4/d49/l46 0 2024-05-13T22:48:27.188 INFO:tasks.workunit.client.1.smithi130.stdout:2/221: chown d4/d49/d42 1 1 2024-05-13T22:48:27.188 INFO:tasks.workunit.client.1.smithi130.stdout:8/177: write d1/dc/f24 [4082640,53414] 0 2024-05-13T22:48:27.194 INFO:tasks.workunit.client.0.smithi102.stdout:1/833: dread d2/d4/f23 [0,4194304] 0 2024-05-13T22:48:27.240 INFO:tasks.workunit.client.0.smithi102.stdout:8/954: chown d2/d7/d27/d64/d80/dc8/d40/cb4 10334 1 2024-05-13T22:48:27.242 INFO:tasks.workunit.client.1.smithi130.stdout:3/180: write d8/dc/d19/f20 [1661406,28568] 0 2024-05-13T22:48:27.244 INFO:tasks.workunit.client.0.smithi102.stdout:1/834: mkdir d2/d4/d26/d40/d10e 0 2024-05-13T22:48:27.246 INFO:tasks.workunit.client.1.smithi130.stdout:9/196: truncate d1/d7/f27 302701 0 2024-05-13T22:48:27.247 INFO:tasks.workunit.client.0.smithi102.stdout:8/955: creat d2/d7/d27/dbb/dc7/f12e x:0 0 0 2024-05-13T22:48:27.249 INFO:tasks.workunit.client.0.smithi102.stdout:0/951: rename d1/de/d32/d8a/d116/d11b/d10d/f129 to d1/de/d32/d24/d2d/d38/db7/f12a 0 2024-05-13T22:48:27.251 INFO:tasks.workunit.client.0.smithi102.stdout:1/835: rmdir d2/d4/d5/d19/dd3 39 2024-05-13T22:48:27.251 INFO:tasks.workunit.client.0.smithi102.stdout:1/836: read d2/d4/d5/f15 [526960,94163] 0 2024-05-13T22:48:27.256 INFO:tasks.workunit.client.1.smithi130.stdout:7/156: dwrite d7/da/d1f/f20 [0,4194304] 0 2024-05-13T22:48:27.259 INFO:tasks.workunit.client.0.smithi102.stdout:0/952: read d1/de/d32/d24/f31 [3432106,110451] 0 2024-05-13T22:48:27.260 INFO:tasks.workunit.client.1.smithi130.stdout:6/237: mknod d4/d15/d1f/d46/c4f 0 2024-05-13T22:48:27.261 INFO:tasks.workunit.client.1.smithi130.stdout:6/238: read d4/d15/d1d/d28/f47 [669330,52924] 0 2024-05-13T22:48:27.261 INFO:tasks.workunit.client.1.smithi130.stdout:4/214: fdatasync d8/df/f20 0 2024-05-13T22:48:27.264 INFO:tasks.workunit.client.0.smithi102.stdout:1/837: creat d2/f10f x:0 0 0 2024-05-13T22:48:27.265 INFO:tasks.workunit.client.0.smithi102.stdout:1/838: fsync d2/d10/f44 0 2024-05-13T22:48:27.266 INFO:tasks.workunit.client.0.smithi102.stdout:1/839: readlink d2/d4/d5/d19/lca 0 2024-05-13T22:48:27.266 INFO:tasks.workunit.client.0.smithi102.stdout:0/953: creat d1/de/d32/d8d/f12b x:0 0 0 2024-05-13T22:48:27.267 INFO:tasks.workunit.client.0.smithi102.stdout:8/956: link d2/d36/c4d d2/d7/d27/d64/d80/dc8/d93/dab/c12f 0 2024-05-13T22:48:27.268 INFO:tasks.workunit.client.0.smithi102.stdout:8/957: chown d2/d83/fbf 1737855 1 2024-05-13T22:48:27.268 INFO:tasks.workunit.client.1.smithi130.stdout:2/222: mknod d4/d49/d42/d4b/c5e 0 2024-05-13T22:48:27.269 INFO:tasks.workunit.client.0.smithi102.stdout:8/958: fsync d2/d7/f9 0 2024-05-13T22:48:27.269 INFO:tasks.workunit.client.0.smithi102.stdout:8/959: fdatasync d2/d7/d25/fb0 0 2024-05-13T22:48:27.271 INFO:tasks.workunit.client.0.smithi102.stdout:8/960: read d2/d7/d27/d64/d96/f119 [71760,79570] 0 2024-05-13T22:48:27.272 INFO:tasks.workunit.client.0.smithi102.stdout:8/961: readlink d2/d7/d27/d64/d96/dae/d76/lf9 0 2024-05-13T22:48:27.272 INFO:tasks.workunit.client.1.smithi130.stdout:3/181: symlink d8/dc/d15/l38 0 2024-05-13T22:48:27.276 INFO:tasks.workunit.client.1.smithi130.stdout:9/197: creat d1/d7/d23/d28/d2d/d2f/f3f x:0 0 0 2024-05-13T22:48:27.276 INFO:tasks.workunit.client.1.smithi130.stdout:9/198: fdatasync d1/d15/f3b 0 2024-05-13T22:48:27.278 INFO:tasks.workunit.client.1.smithi130.stdout:5/212: mknod d6/c47 0 2024-05-13T22:48:27.279 INFO:tasks.workunit.client.1.smithi130.stdout:5/213: write d6/dd/f3e [888678,85247] 0 2024-05-13T22:48:27.281 INFO:tasks.workunit.client.0.smithi102.stdout:1/840: mknod d2/d45/c110 0 2024-05-13T22:48:27.286 INFO:tasks.workunit.client.1.smithi130.stdout:4/215: unlink d8/dc/d29/l2f 0 2024-05-13T22:48:27.287 INFO:tasks.workunit.client.0.smithi102.stdout:8/962: truncate d2/d7/d27/d64/d96/dbc/d106/f11b 866804 0 2024-05-13T22:48:27.290 INFO:tasks.workunit.client.1.smithi130.stdout:4/216: dwrite d8/f1e [0,4194304] 0 2024-05-13T22:48:27.291 INFO:tasks.workunit.client.1.smithi130.stdout:4/217: stat d8/df/c3b 0 2024-05-13T22:48:27.292 INFO:tasks.workunit.client.1.smithi130.stdout:4/218: write d8/fa [807913,34598] 0 2024-05-13T22:48:27.292 INFO:tasks.workunit.client.0.smithi102.stdout:1/841: unlink d2/d4/l18 0 2024-05-13T22:48:27.298 INFO:tasks.workunit.client.0.smithi102.stdout:8/963: rename d2/d7/d27/d64/d96/f119 to d2/d7/d27/d64/d80/dc8/d40/d85/f130 0 2024-05-13T22:48:27.298 INFO:tasks.workunit.client.0.smithi102.stdout:0/954: dwrite d1/de/d32/d24/d2d/d38/d3d/f57 [0,4194304] 0 2024-05-13T22:48:27.299 INFO:tasks.workunit.client.1.smithi130.stdout:0/201: link d0/f21 d0/f42 0 2024-05-13T22:48:27.299 INFO:tasks.workunit.client.1.smithi130.stdout:9/199: rmdir d1/d7/d23 39 2024-05-13T22:48:27.299 INFO:tasks.workunit.client.1.smithi130.stdout:7/157: link d7/da/d1f/f30 d7/da/d1f/d2c/f38 0 2024-05-13T22:48:27.299 INFO:tasks.workunit.client.0.smithi102.stdout:0/955: readlink d1/d4d/d79/la2 0 2024-05-13T22:48:27.300 INFO:tasks.workunit.client.0.smithi102.stdout:8/964: fdatasync d2/d7/d25/f6e 0 2024-05-13T22:48:27.303 INFO:tasks.workunit.client.1.smithi130.stdout:7/158: dread d7/da/d1f/d2c/f38 [0,4194304] 0 2024-05-13T22:48:27.305 INFO:tasks.workunit.client.0.smithi102.stdout:0/956: mknod d1/de/d32/d8a/d116/d11b/d10d/c12c 0 2024-05-13T22:48:27.305 INFO:tasks.workunit.client.0.smithi102.stdout:0/957: stat d1/de/d32/dab/fe3 0 2024-05-13T22:48:27.306 INFO:tasks.workunit.client.1.smithi130.stdout:1/187: getdents db/d1b/d1f/d2a/d35 0 2024-05-13T22:48:27.311 INFO:tasks.workunit.client.0.smithi102.stdout:8/965: rename d2/d7/d27/d64/d96/dae/f112 to d2/d122/f131 0 2024-05-13T22:48:27.312 INFO:tasks.workunit.client.0.smithi102.stdout:8/966: chown d2/d36/d49/f12d 102019849 1 2024-05-13T22:48:27.313 INFO:tasks.workunit.client.1.smithi130.stdout:8/178: rename d1/d2/d38 to d1/d2/d3c 0 2024-05-13T22:48:27.314 INFO:tasks.workunit.client.0.smithi102.stdout:8/967: dread d2/d36/d49/d5d/ff0 [0,4194304] 0 2024-05-13T22:48:27.315 INFO:tasks.workunit.client.1.smithi130.stdout:4/219: creat d8/dc/d35/d33/f4f x:0 0 0 2024-05-13T22:48:27.315 INFO:tasks.workunit.client.0.smithi102.stdout:8/968: readlink d2/d36/d49/l86 0 2024-05-13T22:48:27.316 INFO:tasks.workunit.client.0.smithi102.stdout:1/842: link d2/d10/d62/d86/ceb d2/d45/da4/def/c111 0 2024-05-13T22:48:27.318 INFO:tasks.workunit.client.1.smithi130.stdout:5/214: dwrite d6/d27/f2d [0,4194304] 0 2024-05-13T22:48:27.320 INFO:tasks.workunit.client.0.smithi102.stdout:0/958: unlink d1/d77/ff8 0 2024-05-13T22:48:27.322 INFO:tasks.workunit.client.1.smithi130.stdout:7/159: unlink d7/da/d1d/l37 0 2024-05-13T22:48:27.323 INFO:tasks.workunit.client.1.smithi130.stdout:5/215: dread d6/d1a/f2a [0,4194304] 0 2024-05-13T22:48:27.323 INFO:tasks.workunit.client.1.smithi130.stdout:5/216: read d6/d2f/f39 [3927654,49768] 0 2024-05-13T22:48:27.323 INFO:tasks.workunit.client.0.smithi102.stdout:1/843: mkdir d2/d4/d38/db2/dba/d112 0 2024-05-13T22:48:27.325 INFO:tasks.workunit.client.1.smithi130.stdout:2/223: creat d4/f5f x:0 0 0 2024-05-13T22:48:27.325 INFO:tasks.workunit.client.1.smithi130.stdout:2/224: write d4/f19 [55822,123996] 0 2024-05-13T22:48:27.327 INFO:tasks.workunit.client.1.smithi130.stdout:8/179: creat d1/d2/db/f3d x:0 0 0 2024-05-13T22:48:27.330 INFO:tasks.workunit.client.1.smithi130.stdout:3/182: rename l1 to d8/d1a/l39 0 2024-05-13T22:48:27.332 INFO:tasks.workunit.client.1.smithi130.stdout:8/180: dwrite d1/dc/f21 [0,4194304] 0 2024-05-13T22:48:27.336 INFO:tasks.workunit.client.0.smithi102.stdout:8/969: creat d2/d7/d27/d64/d96/dae/f132 x:0 0 0 2024-05-13T22:48:27.344 INFO:tasks.workunit.client.0.smithi102.stdout:8/970: creat d2/d7/d25/d41/d6f/f133 x:0 0 0 2024-05-13T22:48:27.349 INFO:tasks.workunit.client.0.smithi102.stdout:1/844: getdents d2/d4/d26/d40 0 2024-05-13T22:48:27.351 INFO:tasks.workunit.client.0.smithi102.stdout:8/971: dwrite d2/d7/d27/d64/d80/dc8/d40/d85/ddc/ffa [0,4194304] 0 2024-05-13T22:48:27.351 INFO:tasks.workunit.client.0.smithi102.stdout:0/959: dread d1/de/d32/d63/f96 [0,4194304] 0 2024-05-13T22:48:27.354 INFO:tasks.workunit.client.0.smithi102.stdout:1/845: mkdir d2/d45/d96/d113 0 2024-05-13T22:48:27.358 INFO:tasks.workunit.client.0.smithi102.stdout:0/960: chown d1/de/d32/c58 6463 1 2024-05-13T22:48:27.360 INFO:tasks.workunit.client.0.smithi102.stdout:8/972: rename d2/d36/d49/d7e/le4 to d2/d36/d49/d5d/l134 0 2024-05-13T22:48:27.365 INFO:tasks.workunit.client.0.smithi102.stdout:1/846: symlink d2/d4/d5/l114 0 2024-05-13T22:48:27.366 INFO:tasks.workunit.client.0.smithi102.stdout:0/961: symlink d1/de/d32/d24/d2d/d38/d3d/d10a/l12d 0 2024-05-13T22:48:27.367 INFO:tasks.workunit.client.0.smithi102.stdout:8/973: dwrite d2/d7/d27/f4c [8388608,4194304] 0 2024-05-13T22:48:27.367 INFO:tasks.workunit.client.0.smithi102.stdout:1/847: chown d2/d4/d26/da2/cb0 426 1 2024-05-13T22:48:27.369 INFO:tasks.workunit.client.0.smithi102.stdout:0/962: chown d1/de/d32/d24/d2d/d38/le0 214 1 2024-05-13T22:48:27.371 INFO:tasks.workunit.client.0.smithi102.stdout:8/974: creat d2/d7/d27/f135 x:0 0 0 2024-05-13T22:48:27.372 INFO:tasks.workunit.client.0.smithi102.stdout:1/848: dread - d2/d4/d5/d19/dc9/fd7 zero size 2024-05-13T22:48:27.374 INFO:tasks.workunit.client.0.smithi102.stdout:8/975: symlink d2/d36/l136 0 2024-05-13T22:48:27.374 INFO:tasks.workunit.client.0.smithi102.stdout:0/963: read d1/de/d32/d24/d2d/d38/d3d/d81/fb8 [164688,110514] 0 2024-05-13T22:48:27.377 INFO:tasks.workunit.client.0.smithi102.stdout:8/976: creat d2/d7/d27/d64/d80/dc8/d40/d85/f137 x:0 0 0 2024-05-13T22:48:27.381 INFO:tasks.workunit.client.0.smithi102.stdout:8/977: rename d2/d7/d27/d64/d80/dc8/d93/dab/fc3 to d2/d7/d27/d64/d96/dae/d11a/f138 0 2024-05-13T22:48:27.385 INFO:tasks.workunit.client.0.smithi102.stdout:8/978: rename d2/d7/d27/d64/d80/dc8/d40/d85/fc9 to d2/de8/f139 0 2024-05-13T22:48:27.386 INFO:tasks.workunit.client.0.smithi102.stdout:8/979: chown d2/d7/d25/f6e 219232950 1 2024-05-13T22:48:27.388 INFO:tasks.workunit.client.0.smithi102.stdout:8/980: creat d2/d7/d27/d64/f13a x:0 0 0 2024-05-13T22:48:27.388 INFO:tasks.workunit.client.0.smithi102.stdout:8/981: chown d2/d36/d49/d5d/ff0 1784 1 2024-05-13T22:48:27.394 INFO:tasks.workunit.client.0.smithi102.stdout:1/849: dwrite d2/d4/f23 [0,4194304] 0 2024-05-13T22:48:27.396 INFO:tasks.workunit.client.0.smithi102.stdout:8/982: truncate d2/d7/d25/d41/f48 3089436 0 2024-05-13T22:48:27.397 INFO:tasks.workunit.client.0.smithi102.stdout:1/850: mknod d2/d4/d38/db2/dba/d10b/c115 0 2024-05-13T22:48:27.398 INFO:tasks.workunit.client.0.smithi102.stdout:8/983: read d2/d36/d49/d5d/fbd [3335521,35819] 0 2024-05-13T22:48:27.399 INFO:tasks.workunit.client.0.smithi102.stdout:1/851: truncate d2/d4/d5/d19/dbe/fe9 231085 0 2024-05-13T22:48:27.400 INFO:tasks.workunit.client.0.smithi102.stdout:8/984: rmdir d2/d7/d27/d64/d80/dc8/d93 39 2024-05-13T22:48:27.402 INFO:tasks.workunit.client.0.smithi102.stdout:8/985: truncate d2/d7/d27/d64/d96/dae/d78/f113 225631 0 2024-05-13T22:48:27.404 INFO:tasks.workunit.client.1.smithi130.stdout:2/225: symlink d4/d49/d48/l60 0 2024-05-13T22:48:27.404 INFO:tasks.workunit.client.1.smithi130.stdout:9/200: link d1/d15/f16 d1/d7/d23/d3d/f40 0 2024-05-13T22:48:27.405 INFO:tasks.workunit.client.0.smithi102.stdout:8/986: mkdir d2/d7/d27/d64/d96/dae/d76/d13b 0 2024-05-13T22:48:27.405 INFO:tasks.workunit.client.1.smithi130.stdout:7/160: mkdir d7/d8/d39 0 2024-05-13T22:48:27.405 INFO:tasks.workunit.client.1.smithi130.stdout:4/220: truncate d8/df/f24 366954 0 2024-05-13T22:48:27.406 INFO:tasks.workunit.client.1.smithi130.stdout:3/183: write d8/f2b [1855129,6424] 0 2024-05-13T22:48:27.406 INFO:tasks.workunit.client.1.smithi130.stdout:9/201: chown d1/d7/c1c 1911690 1 2024-05-13T22:48:27.408 INFO:tasks.workunit.client.1.smithi130.stdout:1/188: link db/l29 db/l3f 0 2024-05-13T22:48:27.408 INFO:tasks.workunit.client.1.smithi130.stdout:2/226: rmdir d4/d49/d42/d4b 39 2024-05-13T22:48:27.409 INFO:tasks.workunit.client.1.smithi130.stdout:7/161: rmdir d7/da 39 2024-05-13T22:48:27.411 INFO:tasks.workunit.client.1.smithi130.stdout:9/202: readlink d1/d7/d23/d28/l38 0 2024-05-13T22:48:27.412 INFO:tasks.workunit.client.1.smithi130.stdout:9/203: write d1/d15/f3b [935787,8426] 0 2024-05-13T22:48:27.415 INFO:tasks.workunit.client.0.smithi102.stdout:0/964: dread d1/de/d32/d24/f74 [0,4194304] 0 2024-05-13T22:48:27.417 INFO:tasks.workunit.client.0.smithi102.stdout:8/987: write d2/d36/d49/f60 [2229609,90979] 0 2024-05-13T22:48:27.418 INFO:tasks.workunit.client.1.smithi130.stdout:7/162: truncate d7/da/f33 411039 0 2024-05-13T22:48:27.419 INFO:tasks.workunit.client.0.smithi102.stdout:0/965: creat d1/d4d/d79/d9f/dce/f12e x:0 0 0 2024-05-13T22:48:27.420 INFO:tasks.workunit.client.0.smithi102.stdout:8/988: write d2/d7/d25/df4/f11e [648536,2983] 0 2024-05-13T22:48:27.421 INFO:tasks.workunit.client.1.smithi130.stdout:8/181: dwrite d1/d2/f3 [0,4194304] 0 2024-05-13T22:48:27.421 INFO:tasks.workunit.client.1.smithi130.stdout:9/204: dread d1/d15/f3b [0,4194304] 0 2024-05-13T22:48:27.424 INFO:tasks.workunit.client.1.smithi130.stdout:6/239: sync 2024-05-13T22:48:27.425 INFO:tasks.workunit.client.0.smithi102.stdout:8/989: symlink d2/d7/d25/d41/d6f/da0/d110/l13c 0 2024-05-13T22:48:27.425 INFO:tasks.workunit.client.1.smithi130.stdout:6/240: chown d4/d16/l36 116432650 1 2024-05-13T22:48:27.426 INFO:tasks.workunit.client.1.smithi130.stdout:7/163: mknod d7/da/d1d/c3a 0 2024-05-13T22:48:27.427 INFO:tasks.workunit.client.1.smithi130.stdout:1/189: getdents db/d1b/d1f/d2a/d35 0 2024-05-13T22:48:27.427 INFO:tasks.workunit.client.1.smithi130.stdout:9/205: mknod d1/d7/d23/d28/c41 0 2024-05-13T22:48:27.429 INFO:tasks.workunit.client.1.smithi130.stdout:4/221: truncate d8/dc/d35/f31 1912408 0 2024-05-13T22:48:27.430 INFO:tasks.workunit.client.1.smithi130.stdout:2/227: dwrite d4/f1e [0,4194304] 0 2024-05-13T22:48:27.430 INFO:tasks.workunit.client.1.smithi130.stdout:3/184: truncate d8/dc/d19/f20 3219821 0 2024-05-13T22:48:27.435 INFO:tasks.workunit.client.1.smithi130.stdout:4/222: unlink d8/df/c3b 0 2024-05-13T22:48:27.435 INFO:tasks.workunit.client.1.smithi130.stdout:8/182: dwrite d1/dc/f11 [0,4194304] 0 2024-05-13T22:48:27.436 INFO:tasks.workunit.client.1.smithi130.stdout:1/190: creat db/d1b/d1f/d2a/d35/f40 x:0 0 0 2024-05-13T22:48:27.436 INFO:tasks.workunit.client.1.smithi130.stdout:6/241: dread d4/d15/f44 [0,4194304] 0 2024-05-13T22:48:27.437 INFO:tasks.workunit.client.1.smithi130.stdout:4/223: chown f4 12415 1 2024-05-13T22:48:27.438 INFO:tasks.workunit.client.1.smithi130.stdout:2/228: creat d4/d56/f61 x:0 0 0 2024-05-13T22:48:27.438 INFO:tasks.workunit.client.1.smithi130.stdout:9/206: dread d1/d7/fa [0,4194304] 0 2024-05-13T22:48:27.439 INFO:tasks.workunit.client.1.smithi130.stdout:9/207: stat d1 0 2024-05-13T22:48:27.439 INFO:tasks.workunit.client.1.smithi130.stdout:2/229: chown d4/f1f 377552 1 2024-05-13T22:48:27.439 INFO:tasks.workunit.client.1.smithi130.stdout:7/164: dread d7/da/d1f/d2c/f38 [0,4194304] 0 2024-05-13T22:48:27.440 INFO:tasks.workunit.client.1.smithi130.stdout:7/165: stat d7 0 2024-05-13T22:48:27.441 INFO:tasks.workunit.client.1.smithi130.stdout:7/166: readlink d7/d8/d9/l34 0 2024-05-13T22:48:27.441 INFO:tasks.workunit.client.1.smithi130.stdout:7/167: read d7/d8/f2d [3218414,20451] 0 2024-05-13T22:48:27.442 INFO:tasks.workunit.client.0.smithi102.stdout:0/966: dwrite d1/f9 [4194304,4194304] 0 2024-05-13T22:48:27.446 INFO:tasks.workunit.client.1.smithi130.stdout:6/242: mkdir d4/d15/d1f/d46/d50 0 2024-05-13T22:48:27.449 INFO:tasks.workunit.client.1.smithi130.stdout:7/168: dwrite d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:27.450 INFO:tasks.workunit.client.1.smithi130.stdout:4/224: creat d8/dc/d35/f50 x:0 0 0 2024-05-13T22:48:27.454 INFO:tasks.workunit.client.1.smithi130.stdout:9/208: unlink d1/d7/d23/d28/d2d/f30 0 2024-05-13T22:48:27.454 INFO:tasks.workunit.client.1.smithi130.stdout:1/191: rmdir db/d1b/d1f/d2a/d35 39 2024-05-13T22:48:27.456 INFO:tasks.workunit.client.0.smithi102.stdout:0/967: symlink d1/de/d32/d24/d2d/d38/l12f 0 2024-05-13T22:48:27.458 INFO:tasks.workunit.client.1.smithi130.stdout:4/225: dwrite d8/f1b [0,4194304] 0 2024-05-13T22:48:27.460 INFO:tasks.workunit.client.0.smithi102.stdout:0/968: mkdir d1/d68/d130 0 2024-05-13T22:48:27.462 INFO:tasks.workunit.client.1.smithi130.stdout:8/183: creat d1/d2/db/d15/d18/f3e x:0 0 0 2024-05-13T22:48:27.463 INFO:tasks.workunit.client.0.smithi102.stdout:0/969: unlink d1/d4d/d79/d9f/dce/f11d 0 2024-05-13T22:48:27.464 INFO:tasks.workunit.client.1.smithi130.stdout:4/226: dread d8/df/f20 [0,4194304] 0 2024-05-13T22:48:27.466 INFO:tasks.workunit.client.1.smithi130.stdout:4/227: stat d8/dc/d35/d33/d44 0 2024-05-13T22:48:27.469 INFO:tasks.workunit.client.1.smithi130.stdout:6/243: creat d4/d15/d1f/d46/d50/f51 x:0 0 0 2024-05-13T22:48:27.470 INFO:tasks.workunit.client.1.smithi130.stdout:4/228: dread d8/dc/d26/f36 [0,4194304] 0 2024-05-13T22:48:27.471 INFO:tasks.workunit.client.1.smithi130.stdout:6/244: chown d4/c13 1 1 2024-05-13T22:48:27.475 INFO:tasks.workunit.client.1.smithi130.stdout:6/245: write d4/d16/f2a [834895,56549] 0 2024-05-13T22:48:27.475 INFO:tasks.workunit.client.1.smithi130.stdout:1/192: creat db/d1b/d1f/d2a/f41 x:0 0 0 2024-05-13T22:48:27.475 INFO:tasks.workunit.client.1.smithi130.stdout:4/229: symlink d8/df/l51 0 2024-05-13T22:48:27.476 INFO:tasks.workunit.client.1.smithi130.stdout:2/230: dwrite d4/db/dc/d18/d2a/f44 [0,4194304] 0 2024-05-13T22:48:27.476 INFO:tasks.workunit.client.1.smithi130.stdout:8/184: rmdir d1/d2/d3c 0 2024-05-13T22:48:27.478 INFO:tasks.workunit.client.1.smithi130.stdout:4/230: read d8/dc/f38 [3637427,102096] 0 2024-05-13T22:48:27.478 INFO:tasks.workunit.client.1.smithi130.stdout:2/231: write d4/f5f [294104,97509] 0 2024-05-13T22:48:27.479 INFO:tasks.workunit.client.1.smithi130.stdout:8/185: write d1/d2/db/d15/f22 [3178417,66072] 0 2024-05-13T22:48:27.479 INFO:tasks.workunit.client.1.smithi130.stdout:1/193: unlink db/l30 0 2024-05-13T22:48:27.479 INFO:tasks.workunit.client.1.smithi130.stdout:2/232: dread - d4/db/f41 zero size 2024-05-13T22:48:27.480 INFO:tasks.workunit.client.1.smithi130.stdout:1/194: chown db/d1b/l1c 962061 1 2024-05-13T22:48:27.480 INFO:tasks.workunit.client.1.smithi130.stdout:8/186: stat d1/d2/db/d15/d18 0 2024-05-13T22:48:27.480 INFO:tasks.workunit.client.1.smithi130.stdout:1/195: fsync db/f16 0 2024-05-13T22:48:27.481 INFO:tasks.workunit.client.1.smithi130.stdout:8/187: write d1/dc/f21 [4073120,86684] 0 2024-05-13T22:48:27.484 INFO:tasks.workunit.client.1.smithi130.stdout:6/246: truncate d4/d16/f39 200502 0 2024-05-13T22:48:27.486 INFO:tasks.workunit.client.1.smithi130.stdout:4/231: creat d8/dc/d26/f52 x:0 0 0 2024-05-13T22:48:27.487 INFO:tasks.workunit.client.1.smithi130.stdout:2/233: dread - d4/db/dc/f3b zero size 2024-05-13T22:48:27.488 INFO:tasks.workunit.client.1.smithi130.stdout:2/234: fdatasync d4/db/dc/d18/f23 0 2024-05-13T22:48:27.489 INFO:tasks.workunit.client.1.smithi130.stdout:2/235: fdatasync d4/f19 0 2024-05-13T22:48:27.489 INFO:tasks.workunit.client.1.smithi130.stdout:7/169: truncate d7/da/d1f/f30 4073450 0 2024-05-13T22:48:27.491 INFO:tasks.workunit.client.1.smithi130.stdout:8/188: creat d1/dc/f3f x:0 0 0 2024-05-13T22:48:27.493 INFO:tasks.workunit.client.1.smithi130.stdout:6/247: mkdir d4/d15/d1f/d46/d52 0 2024-05-13T22:48:27.494 INFO:tasks.workunit.client.1.smithi130.stdout:8/189: dread - d1/d2/db/d15/d16/f2c zero size 2024-05-13T22:48:27.497 INFO:tasks.workunit.client.1.smithi130.stdout:6/248: chown d4/d15/d1f/d46/d50 0 1 2024-05-13T22:48:27.497 INFO:tasks.workunit.client.1.smithi130.stdout:4/232: dwrite d8/df/f3c [0,4194304] 0 2024-05-13T22:48:27.497 INFO:tasks.workunit.client.1.smithi130.stdout:2/236: chown d4/f7 31280470 1 2024-05-13T22:48:27.498 INFO:tasks.workunit.client.1.smithi130.stdout:8/190: chown d1/dc/f3f 0 1 2024-05-13T22:48:27.502 INFO:tasks.workunit.client.1.smithi130.stdout:8/191: write d1/f33 [3000838,120599] 0 2024-05-13T22:48:27.503 INFO:tasks.workunit.client.1.smithi130.stdout:9/209: dwrite d1/f33 [0,4194304] 0 2024-05-13T22:48:27.503 INFO:tasks.workunit.client.1.smithi130.stdout:6/249: creat d4/d15/d1f/d46/d50/f53 x:0 0 0 2024-05-13T22:48:27.507 INFO:tasks.workunit.client.1.smithi130.stdout:7/170: read d7/da/d1f/d2c/f38 [3290291,23032] 0 2024-05-13T22:48:27.512 INFO:tasks.workunit.client.1.smithi130.stdout:9/210: creat d1/d7/d23/d28/d2d/f42 x:0 0 0 2024-05-13T22:48:27.513 INFO:tasks.workunit.client.1.smithi130.stdout:7/171: symlink d7/da/d1f/d2c/l3b 0 2024-05-13T22:48:27.514 INFO:tasks.workunit.client.1.smithi130.stdout:6/250: dread d4/d15/d1d/d28/f30 [0,4194304] 0 2024-05-13T22:48:27.515 INFO:tasks.workunit.client.1.smithi130.stdout:9/211: chown d1/f8 2103409 1 2024-05-13T22:48:27.515 INFO:tasks.workunit.client.1.smithi130.stdout:4/233: dwrite f4 [0,4194304] 0 2024-05-13T22:48:27.517 INFO:tasks.workunit.client.1.smithi130.stdout:7/172: write d7/d8/f19 [2322010,57136] 0 2024-05-13T22:48:27.521 INFO:tasks.workunit.client.1.smithi130.stdout:1/196: dwrite db/d1b/d1f/d2a/d35/f3d [0,4194304] 0 2024-05-13T22:48:27.523 INFO:tasks.workunit.client.1.smithi130.stdout:4/234: rename d8/df/f42 to d8/dc/d26/f53 0 2024-05-13T22:48:27.525 INFO:tasks.workunit.client.1.smithi130.stdout:4/235: stat d8/cb 0 2024-05-13T22:48:27.528 INFO:tasks.workunit.client.1.smithi130.stdout:1/197: mkdir db/d1b/d1f/d2a/d42 0 2024-05-13T22:48:27.529 INFO:tasks.workunit.client.1.smithi130.stdout:4/236: dread - d8/dc/d26/f30 zero size 2024-05-13T22:48:27.530 INFO:tasks.workunit.client.1.smithi130.stdout:9/212: dwrite d1/d15/f1d [0,4194304] 0 2024-05-13T22:48:27.534 INFO:tasks.workunit.client.1.smithi130.stdout:1/198: symlink db/d1b/d1f/l43 0 2024-05-13T22:48:27.534 INFO:tasks.workunit.client.1.smithi130.stdout:9/213: stat d1/f33 0 2024-05-13T22:48:27.535 INFO:tasks.workunit.client.1.smithi130.stdout:1/199: write db/f1d [925043,112651] 0 2024-05-13T22:48:27.536 INFO:tasks.workunit.client.1.smithi130.stdout:0/202: sync 2024-05-13T22:48:27.537 INFO:tasks.workunit.client.1.smithi130.stdout:4/237: stat d8/dc/d13/c32 0 2024-05-13T22:48:27.537 INFO:tasks.workunit.client.1.smithi130.stdout:1/200: write f8 [2258582,34169] 0 2024-05-13T22:48:27.537 INFO:tasks.workunit.client.1.smithi130.stdout:9/214: rmdir d1/d7/d23 39 2024-05-13T22:48:27.539 INFO:tasks.workunit.client.1.smithi130.stdout:6/251: truncate d4/d15/d1f/f20 1103591 0 2024-05-13T22:48:27.539 INFO:tasks.workunit.client.1.smithi130.stdout:1/201: chown db/cf 4861669 1 2024-05-13T22:48:27.540 INFO:tasks.workunit.client.1.smithi130.stdout:7/173: dwrite d7/da/d1f/f30 [0,4194304] 0 2024-05-13T22:48:27.544 INFO:tasks.workunit.client.1.smithi130.stdout:2/237: dread d4/db/d26/f2d [0,4194304] 0 2024-05-13T22:48:27.545 INFO:tasks.workunit.client.1.smithi130.stdout:0/203: write d0/d7/dd/d17/d1f/f25 [1557332,15809] 0 2024-05-13T22:48:27.548 INFO:tasks.workunit.client.1.smithi130.stdout:2/238: dread d4/f5f [0,4194304] 0 2024-05-13T22:48:27.550 INFO:tasks.workunit.client.1.smithi130.stdout:1/202: symlink db/d1b/d1f/d2a/d35/l44 0 2024-05-13T22:48:27.551 INFO:tasks.workunit.client.1.smithi130.stdout:1/203: write f8 [2056851,16427] 0 2024-05-13T22:48:27.553 INFO:tasks.workunit.client.1.smithi130.stdout:1/204: write db/d1b/d1f/d2a/d35/f40 [610984,58732] 0 2024-05-13T22:48:27.553 INFO:tasks.workunit.client.1.smithi130.stdout:1/205: stat db/d1b/d1f/d2a 0 2024-05-13T22:48:27.555 INFO:tasks.workunit.client.1.smithi130.stdout:4/238: link d8/dc/d26/f52 d8/dc/d35/d33/d44/f54 0 2024-05-13T22:48:27.558 INFO:tasks.workunit.client.1.smithi130.stdout:7/174: symlink d7/d8/d39/l3c 0 2024-05-13T22:48:27.560 INFO:tasks.workunit.client.1.smithi130.stdout:9/215: write d1/f11 [4858331,124200] 0 2024-05-13T22:48:27.567 INFO:tasks.workunit.client.1.smithi130.stdout:1/206: rename db/d1b/c23 to db/d1b/d1f/c45 0 2024-05-13T22:48:27.571 INFO:tasks.workunit.client.1.smithi130.stdout:7/175: mkdir d7/da/d1d/d3d 0 2024-05-13T22:48:27.577 INFO:tasks.workunit.client.1.smithi130.stdout:6/252: chown d4/d15/d1f/f20 876731 1 2024-05-13T22:48:27.577 INFO:tasks.workunit.client.1.smithi130.stdout:9/216: dread d1/d7/d23/d3d/f40 [0,4194304] 0 2024-05-13T22:48:27.578 INFO:tasks.workunit.client.1.smithi130.stdout:6/253: dread - d4/d15/d1f/d46/d50/f53 zero size 2024-05-13T22:48:27.579 INFO:tasks.workunit.client.1.smithi130.stdout:0/204: dread d0/d7/fb [0,4194304] 0 2024-05-13T22:48:27.580 INFO:tasks.workunit.client.1.smithi130.stdout:0/205: read - d0/d7/dd/d2e/f3f zero size 2024-05-13T22:48:27.580 INFO:tasks.workunit.client.1.smithi130.stdout:0/206: dread - d0/d7/f16 zero size 2024-05-13T22:48:27.582 INFO:tasks.workunit.client.1.smithi130.stdout:2/239: dread d4/db/f22 [0,4194304] 0 2024-05-13T22:48:27.583 INFO:tasks.workunit.client.1.smithi130.stdout:4/239: unlink d8/dc/d35/f31 0 2024-05-13T22:48:27.591 INFO:tasks.workunit.client.1.smithi130.stdout:1/207: mkdir db/d1b/d1f/d2a/d38/d46 0 2024-05-13T22:48:27.594 INFO:tasks.workunit.client.1.smithi130.stdout:2/240: unlink d4/db/dc/d18/d2a/d30/l5a 0 2024-05-13T22:48:27.598 INFO:tasks.workunit.client.1.smithi130.stdout:7/176: mknod d7/d8/c3e 0 2024-05-13T22:48:27.601 INFO:tasks.workunit.client.1.smithi130.stdout:5/217: sync 2024-05-13T22:48:27.601 INFO:tasks.workunit.client.1.smithi130.stdout:3/185: sync 2024-05-13T22:48:27.601 INFO:tasks.workunit.client.1.smithi130.stdout:9/217: mkdir d1/d7/d23/d3d/d43 0 2024-05-13T22:48:27.601 INFO:tasks.workunit.client.1.smithi130.stdout:9/218: readlink d1/db/l26 0 2024-05-13T22:48:27.602 INFO:tasks.workunit.client.1.smithi130.stdout:9/219: read d1/d15/f16 [3502296,23972] 0 2024-05-13T22:48:27.605 INFO:tasks.workunit.client.1.smithi130.stdout:1/208: dread db/f16 [0,4194304] 0 2024-05-13T22:48:27.606 INFO:tasks.workunit.client.1.smithi130.stdout:1/209: write db/d1b/d1f/d2a/d35/f31 [712149,6654] 0 2024-05-13T22:48:27.608 INFO:tasks.workunit.client.1.smithi130.stdout:7/177: chown d7/d8/d9/c2e 416739918 1 2024-05-13T22:48:27.608 INFO:tasks.workunit.client.1.smithi130.stdout:1/210: truncate db/d1b/d1f/d2a/f34 913360 0 2024-05-13T22:48:27.613 INFO:tasks.workunit.client.1.smithi130.stdout:6/254: dwrite d4/d15/d1d/f22 [0,4194304] 0 2024-05-13T22:48:27.613 INFO:tasks.workunit.client.1.smithi130.stdout:2/241: truncate d4/d49/d48/d4a/f52 5196108 0 2024-05-13T22:48:27.614 INFO:tasks.workunit.client.1.smithi130.stdout:2/242: write d4/db/dc/d18/d2a/d30/f47 [963215,12840] 0 2024-05-13T22:48:27.615 INFO:tasks.workunit.client.1.smithi130.stdout:2/243: fdatasync d4/f1f 0 2024-05-13T22:48:27.619 INFO:tasks.workunit.client.1.smithi130.stdout:2/244: dread d4/f1e [0,4194304] 0 2024-05-13T22:48:27.620 INFO:tasks.workunit.client.1.smithi130.stdout:1/211: unlink db/l3f 0 2024-05-13T22:48:27.623 INFO:tasks.workunit.client.1.smithi130.stdout:6/255: rename d4/ff to d4/d15/d1f/d46/d50/f54 0 2024-05-13T22:48:27.637 INFO:tasks.workunit.client.1.smithi130.stdout:1/212: dread db/f22 [0,4194304] 0 2024-05-13T22:48:27.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:27 smithi130 ceph-mon[36591]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:48:27.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:27 smithi130 ceph-mon[36591]: Upgrade: Need to upgrade myself (mgr.smithi130.uoqoxl) 2024-05-13T22:48:27.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:27 smithi130 ceph-mon[36591]: Upgrade: Need to upgrade myself (mgr.smithi130.uoqoxl) 2024-05-13T22:48:27.651 INFO:tasks.workunit.client.1.smithi130.stdout:8/192: sync 2024-05-13T22:48:27.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:27 smithi102 ceph-mon[28599]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:48:27.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:27 smithi102 ceph-mon[28599]: Upgrade: Need to upgrade myself (mgr.smithi130.uoqoxl) 2024-05-13T22:48:27.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:27 smithi102 ceph-mon[28599]: Upgrade: Need to upgrade myself (mgr.smithi130.uoqoxl) 2024-05-13T22:48:27.704 INFO:tasks.workunit.client.0.smithi102.stdout:4/973: sync 2024-05-13T22:48:27.704 INFO:tasks.workunit.client.0.smithi102.stdout:7/907: sync 2024-05-13T22:48:27.705 INFO:tasks.workunit.client.0.smithi102.stdout:9/943: sync 2024-05-13T22:48:27.706 INFO:tasks.workunit.client.1.smithi130.stdout:4/240: getdents d8/df 0 2024-05-13T22:48:27.710 INFO:tasks.workunit.client.1.smithi130.stdout:3/186: write d8/dc/d15/f18 [2227826,120898] 0 2024-05-13T22:48:27.712 INFO:tasks.workunit.client.1.smithi130.stdout:9/220: write d1/f3 [4379854,5573] 0 2024-05-13T22:48:27.713 INFO:tasks.workunit.client.1.smithi130.stdout:2/245: mknod d4/d49/c62 0 2024-05-13T22:48:27.713 INFO:tasks.workunit.client.1.smithi130.stdout:0/207: truncate d0/f8 5025374 0 2024-05-13T22:48:27.715 INFO:tasks.workunit.client.0.smithi102.stdout:4/974: mknod d3/d19/d23/d39/da7/da9/d135/c151 0 2024-05-13T22:48:27.718 INFO:tasks.workunit.client.1.smithi130.stdout:6/256: creat d4/d15/d1f/f55 x:0 0 0 2024-05-13T22:48:27.719 INFO:tasks.workunit.client.0.smithi102.stdout:4/975: fdatasync d3/d22/f74 0 2024-05-13T22:48:27.719 INFO:tasks.workunit.client.0.smithi102.stdout:4/976: chown d3/d19/d23/f37 1698391 1 2024-05-13T22:48:27.720 INFO:tasks.workunit.client.1.smithi130.stdout:0/208: dwrite d0/d7/f22 [0,4194304] 0 2024-05-13T22:48:27.721 INFO:tasks.workunit.client.1.smithi130.stdout:1/213: write db/f32 [4873649,91558] 0 2024-05-13T22:48:27.721 INFO:tasks.workunit.client.0.smithi102.stdout:7/908: getdents d6/d15 0 2024-05-13T22:48:27.723 INFO:tasks.workunit.client.0.smithi102.stdout:7/909: chown d6/d15/d4d/d55/d6f/d49/d95/fb7 2 1 2024-05-13T22:48:27.723 INFO:tasks.workunit.client.1.smithi130.stdout:8/193: unlink d1/dc/f11 0 2024-05-13T22:48:27.724 INFO:tasks.workunit.client.1.smithi130.stdout:7/178: dwrite d7/d8/f2d [0,4194304] 0 2024-05-13T22:48:27.724 INFO:tasks.workunit.client.1.smithi130.stdout:8/194: write d1/dc/f1f [568899,56952] 0 2024-05-13T22:48:27.725 INFO:tasks.workunit.client.1.smithi130.stdout:4/241: creat d8/dc/d26/f55 x:0 0 0 2024-05-13T22:48:27.727 INFO:tasks.workunit.client.0.smithi102.stdout:4/977: mkdir d3/d22/dab/df9/d152 0 2024-05-13T22:48:27.728 INFO:tasks.workunit.client.0.smithi102.stdout:9/944: write d1/d6/d3b/d5b/d127/f135 [465210,38487] 0 2024-05-13T22:48:27.728 INFO:tasks.workunit.client.0.smithi102.stdout:4/978: chown d3/d19/d23/d3a/ff6 387310 1 2024-05-13T22:48:27.733 INFO:tasks.workunit.client.1.smithi130.stdout:5/218: getdents d6/dd/d18 0 2024-05-13T22:48:27.733 INFO:tasks.workunit.client.1.smithi130.stdout:5/219: readlink d6/l3c 0 2024-05-13T22:48:27.735 INFO:tasks.workunit.client.0.smithi102.stdout:7/910: mkdir d6/d15/d4d/d55/d6f/d11b 0 2024-05-13T22:48:27.736 INFO:tasks.workunit.client.1.smithi130.stdout:9/221: creat d1/d15/f44 x:0 0 0 2024-05-13T22:48:27.737 INFO:tasks.workunit.client.1.smithi130.stdout:5/220: dread d6/dd/f3e [0,4194304] 0 2024-05-13T22:48:27.739 INFO:tasks.workunit.client.1.smithi130.stdout:2/246: creat d4/db/f63 x:0 0 0 2024-05-13T22:48:27.741 INFO:tasks.workunit.client.0.smithi102.stdout:9/945: rename d1/d6/d3b/d5b/c8d to d1/d6/d45/da1/d10a/c148 0 2024-05-13T22:48:27.750 INFO:tasks.workunit.client.1.smithi130.stdout:1/214: rename db/d1b/d1f/d2a/d35/l44 to db/d1b/d1f/d2a/d38/d46/l47 0 2024-05-13T22:48:27.750 INFO:tasks.workunit.client.1.smithi130.stdout:8/195: creat d1/d2/db/f40 x:0 0 0 2024-05-13T22:48:27.751 INFO:tasks.workunit.client.1.smithi130.stdout:7/179: rename d7/d8/d39/l3c to d7/d8/d39/l3f 0 2024-05-13T22:48:27.752 INFO:tasks.workunit.client.1.smithi130.stdout:9/222: mkdir d1/d7/d23/d28/d32/d45 0 2024-05-13T22:48:27.753 INFO:tasks.workunit.client.1.smithi130.stdout:7/180: fdatasync d7/da/d1f/f20 0 2024-05-13T22:48:27.754 INFO:tasks.workunit.client.1.smithi130.stdout:5/221: creat d6/d1a/f48 x:0 0 0 2024-05-13T22:48:27.754 INFO:tasks.workunit.client.1.smithi130.stdout:3/187: dwrite f2 [0,4194304] 0 2024-05-13T22:48:27.755 INFO:tasks.workunit.client.1.smithi130.stdout:2/247: unlink d4/d49/c39 0 2024-05-13T22:48:27.756 INFO:tasks.workunit.client.1.smithi130.stdout:6/257: creat d4/d15/d1d/d28/d4e/f56 x:0 0 0 2024-05-13T22:48:27.757 INFO:tasks.workunit.client.0.smithi102.stdout:4/979: dread d3/d4/d6/de/d1f/f53 [0,4194304] 0 2024-05-13T22:48:27.758 INFO:tasks.workunit.client.0.smithi102.stdout:4/980: chown d3/d19/d23/d39/d77/df5 1212674170 1 2024-05-13T22:48:27.761 INFO:tasks.workunit.client.1.smithi130.stdout:8/196: creat d1/d2/db/f41 x:0 0 0 2024-05-13T22:48:27.761 INFO:tasks.workunit.client.1.smithi130.stdout:7/181: dread d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:27.762 INFO:tasks.workunit.client.1.smithi130.stdout:7/182: stat d7/d8/d9/c1a 0 2024-05-13T22:48:27.763 INFO:tasks.workunit.client.0.smithi102.stdout:4/981: rename d3/d4/l5 to d3/d4/d5f/d12d/l153 0 2024-05-13T22:48:27.765 INFO:tasks.workunit.client.0.smithi102.stdout:4/982: truncate d3/d19/d23/f37 1694956 0 2024-05-13T22:48:27.768 INFO:tasks.workunit.client.0.smithi102.stdout:9/946: write d1/d6/d45/f12f [1060825,40041] 0 2024-05-13T22:48:27.769 INFO:tasks.workunit.client.0.smithi102.stdout:4/983: dread d3/d19/d23/d3a/f56 [0,4194304] 0 2024-05-13T22:48:27.771 INFO:tasks.workunit.client.0.smithi102.stdout:4/984: dread - d3/d19/d23/d4e/d68/dcf/f137 zero size 2024-05-13T22:48:27.772 INFO:tasks.workunit.client.0.smithi102.stdout:4/985: chown d3/d19/d23/d39/da7/d7d/fe1 38203399 1 2024-05-13T22:48:27.775 INFO:tasks.workunit.client.1.smithi130.stdout:1/215: creat db/d1b/d1f/d2a/d42/f48 x:0 0 0 2024-05-13T22:48:27.777 INFO:tasks.workunit.client.1.smithi130.stdout:1/216: stat db/d1b/d1f/d2a/d42/f48 0 2024-05-13T22:48:27.777 INFO:tasks.workunit.client.1.smithi130.stdout:6/258: dwrite d4/f29 [0,4194304] 0 2024-05-13T22:48:27.778 INFO:tasks.workunit.client.1.smithi130.stdout:7/183: mkdir d7/da/d1f/d40 0 2024-05-13T22:48:27.779 INFO:tasks.workunit.client.1.smithi130.stdout:7/184: write d7/da/d1f/f30 [1397125,11903] 0 2024-05-13T22:48:27.779 INFO:tasks.workunit.client.1.smithi130.stdout:6/259: write d4/d15/d1d/f22 [5085621,103795] 0 2024-05-13T22:48:27.779 INFO:tasks.workunit.client.0.smithi102.stdout:4/986: creat d3/d19/f154 x:0 0 0 2024-05-13T22:48:27.780 INFO:tasks.workunit.client.0.smithi102.stdout:4/987: readlink d3/d4/d6/de/d1f/l8e 0 2024-05-13T22:48:27.783 INFO:tasks.workunit.client.1.smithi130.stdout:2/248: write d4/f1e [1910388,16864] 0 2024-05-13T22:48:27.786 INFO:tasks.workunit.client.1.smithi130.stdout:0/209: read d0/f8 [1756075,81107] 0 2024-05-13T22:48:27.787 INFO:tasks.workunit.client.1.smithi130.stdout:4/242: getdents d8/dc/d13 0 2024-05-13T22:48:27.788 INFO:tasks.workunit.client.1.smithi130.stdout:4/243: chown d8/f1a 946 1 2024-05-13T22:48:27.790 INFO:tasks.workunit.client.0.smithi102.stdout:9/947: dwrite d1/d6/db/f66 [0,4194304] 0 2024-05-13T22:48:27.791 INFO:tasks.workunit.client.1.smithi130.stdout:8/197: mkdir d1/d2/db/d15/d18/d42 0 2024-05-13T22:48:27.792 INFO:tasks.workunit.client.1.smithi130.stdout:9/223: truncate d1/d15/f1d 3391347 0 2024-05-13T22:48:27.796 INFO:tasks.workunit.client.1.smithi130.stdout:4/244: dwrite d8/dc/d26/f30 [0,4194304] 0 2024-05-13T22:48:27.798 INFO:tasks.workunit.client.1.smithi130.stdout:7/185: write d7/d8/f1c [437493,74599] 0 2024-05-13T22:48:27.798 INFO:tasks.workunit.client.1.smithi130.stdout:2/249: rmdir d4/d49/d48/d4a 39 2024-05-13T22:48:27.799 INFO:tasks.workunit.client.1.smithi130.stdout:3/188: link d8/l13 d8/dc/d19/d2c/l3a 0 2024-05-13T22:48:27.801 INFO:tasks.workunit.client.1.smithi130.stdout:0/210: mknod d0/d7/dd/d17/c43 0 2024-05-13T22:48:27.805 INFO:tasks.workunit.client.1.smithi130.stdout:4/245: unlink f4 0 2024-05-13T22:48:27.806 INFO:tasks.workunit.client.1.smithi130.stdout:7/186: creat d7/da/d1f/d2c/f41 x:0 0 0 2024-05-13T22:48:27.808 INFO:tasks.workunit.client.1.smithi130.stdout:0/211: fsync d0/f42 0 2024-05-13T22:48:27.809 INFO:tasks.workunit.client.1.smithi130.stdout:5/222: truncate d6/d7/f3b 1064021 0 2024-05-13T22:48:27.810 INFO:tasks.workunit.client.1.smithi130.stdout:8/198: link d1/d2/f39 d1/d2/db/d15/d18/d42/f43 0 2024-05-13T22:48:27.811 INFO:tasks.workunit.client.1.smithi130.stdout:6/260: creat d4/f57 x:0 0 0 2024-05-13T22:48:27.812 INFO:tasks.workunit.client.1.smithi130.stdout:6/261: stat d4/d15/d1d/l1e 0 2024-05-13T22:48:27.814 INFO:tasks.workunit.client.1.smithi130.stdout:2/250: mkdir d4/d49/d48/d4a/d64 0 2024-05-13T22:48:27.816 INFO:tasks.workunit.client.1.smithi130.stdout:3/189: link d8/d1a/d1c/f25 d8/dc/d19/d2c/f3b 0 2024-05-13T22:48:27.816 INFO:tasks.workunit.client.1.smithi130.stdout:4/246: fsync d8/dc/d35/d33/d44/f4b 0 2024-05-13T22:48:27.817 INFO:tasks.workunit.client.1.smithi130.stdout:5/223: dread d6/d27/f29 [0,4194304] 0 2024-05-13T22:48:27.817 INFO:tasks.workunit.client.1.smithi130.stdout:1/217: truncate f8 974564 0 2024-05-13T22:48:27.818 INFO:tasks.workunit.client.1.smithi130.stdout:0/212: symlink d0/d7/dd/d17/l44 0 2024-05-13T22:48:27.820 INFO:tasks.workunit.client.1.smithi130.stdout:4/247: truncate d8/dc/d35/d33/f4f 310042 0 2024-05-13T22:48:27.821 INFO:tasks.workunit.client.1.smithi130.stdout:7/187: dwrite d7/da/f35 [0,4194304] 0 2024-05-13T22:48:27.821 INFO:tasks.workunit.client.1.smithi130.stdout:8/199: rename d1/d2/db/d15/d18/d42/f43 to d1/dc/f44 0 2024-05-13T22:48:27.823 INFO:tasks.workunit.client.1.smithi130.stdout:6/262: creat d4/d15/d1f/d46/f58 x:0 0 0 2024-05-13T22:48:27.823 INFO:tasks.workunit.client.1.smithi130.stdout:2/251: rmdir d4/db/dc/d18/d2a 39 2024-05-13T22:48:27.823 INFO:tasks.workunit.client.1.smithi130.stdout:0/213: read d0/f8 [3534413,116434] 0 2024-05-13T22:48:27.824 INFO:tasks.workunit.client.1.smithi130.stdout:5/224: mknod d6/d2f/c49 0 2024-05-13T22:48:27.825 INFO:tasks.workunit.client.1.smithi130.stdout:5/225: readlink d6/l3c 0 2024-05-13T22:48:27.826 INFO:tasks.workunit.client.1.smithi130.stdout:9/224: dread d1/d15/f1d [0,4194304] 0 2024-05-13T22:48:27.827 INFO:tasks.workunit.client.1.smithi130.stdout:4/248: mkdir d8/dc/d35/d33/d44/d56 0 2024-05-13T22:48:27.828 INFO:tasks.workunit.client.1.smithi130.stdout:1/218: rename db/d1b/d1f/d2a/d42 to db/d1b/d1f/d49 0 2024-05-13T22:48:27.829 INFO:tasks.workunit.client.1.smithi130.stdout:4/249: dread - d8/dc/d26/f55 zero size 2024-05-13T22:48:27.829 INFO:tasks.workunit.client.1.smithi130.stdout:0/214: mkdir d0/d7/dd/d17/d1f/d2c/d45 0 2024-05-13T22:48:27.831 INFO:tasks.workunit.client.1.smithi130.stdout:2/252: read - d4/db/dc/d18/d2a/f2f zero size 2024-05-13T22:48:27.831 INFO:tasks.workunit.client.1.smithi130.stdout:0/215: stat d0/d7/d29/c3b 0 2024-05-13T22:48:27.832 INFO:tasks.workunit.client.1.smithi130.stdout:9/225: write d1/d15/f1d [791352,37057] 0 2024-05-13T22:48:27.833 INFO:tasks.workunit.client.1.smithi130.stdout:3/190: creat d8/d1a/f3c x:0 0 0 2024-05-13T22:48:27.833 INFO:tasks.workunit.client.1.smithi130.stdout:5/226: rename d6/d7/d32/l26 to d6/d7/d32/l4a 0 2024-05-13T22:48:27.833 INFO:tasks.workunit.client.1.smithi130.stdout:4/250: dread d8/dc/d29/f3a [0,4194304] 0 2024-05-13T22:48:27.835 INFO:tasks.workunit.client.1.smithi130.stdout:7/188: dread d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:27.835 INFO:tasks.workunit.client.1.smithi130.stdout:0/216: chown d0/d7/d29/d3c 173 1 2024-05-13T22:48:27.835 INFO:tasks.workunit.client.1.smithi130.stdout:6/263: link d4/d15/d1d/f38 d4/d15/d1f/d46/d50/f59 0 2024-05-13T22:48:27.837 INFO:tasks.workunit.client.1.smithi130.stdout:2/253: readlink d4/db/dc/l40 0 2024-05-13T22:48:27.837 INFO:tasks.workunit.client.1.smithi130.stdout:0/217: readlink d0/d7/dd/d17/d1f/d2c/l2d 0 2024-05-13T22:48:27.839 INFO:tasks.workunit.client.1.smithi130.stdout:3/191: creat d8/d35/f3d x:0 0 0 2024-05-13T22:48:27.840 INFO:tasks.workunit.client.1.smithi130.stdout:2/254: chown d4/db/dc/d18/d2a/l3e 12 1 2024-05-13T22:48:27.840 INFO:tasks.workunit.client.1.smithi130.stdout:6/264: symlink d4/d15/d1d/l5a 0 2024-05-13T22:48:27.840 INFO:tasks.workunit.client.1.smithi130.stdout:0/218: write d0/d7/d29/f40 [163450,6735] 0 2024-05-13T22:48:27.840 INFO:tasks.workunit.client.1.smithi130.stdout:9/226: dread d1/d7/f27 [0,4194304] 0 2024-05-13T22:48:27.854 INFO:tasks.workunit.client.1.smithi130.stdout:3/192: dread d8/f34 [0,4194304] 0 2024-05-13T22:48:27.855 INFO:tasks.workunit.client.1.smithi130.stdout:3/193: write d8/dc/f22 [123924,63753] 0 2024-05-13T22:48:27.865 INFO:tasks.workunit.client.0.smithi102.stdout:1/852: sync 2024-05-13T22:48:27.866 INFO:tasks.workunit.client.1.smithi130.stdout:7/189: mkdir d7/da/d1d/d42 0 2024-05-13T22:48:27.867 INFO:tasks.workunit.client.1.smithi130.stdout:5/227: truncate d6/d1a/f2a 1386582 0 2024-05-13T22:48:27.870 INFO:tasks.workunit.client.1.smithi130.stdout:9/227: rmdir d1/d7 39 2024-05-13T22:48:27.874 INFO:tasks.workunit.client.0.smithi102.stdout:1/853: dread d2/d4/d5/fce [0,4194304] 0 2024-05-13T22:48:27.876 INFO:tasks.workunit.client.1.smithi130.stdout:2/255: mknod d4/db/d26/c65 0 2024-05-13T22:48:27.877 INFO:tasks.workunit.client.1.smithi130.stdout:9/228: dwrite d1/f4 [0,4194304] 0 2024-05-13T22:48:27.877 INFO:tasks.workunit.client.1.smithi130.stdout:6/265: dwrite d4/d15/d1f/d46/d50/f54 [0,4194304] 0 2024-05-13T22:48:27.879 INFO:tasks.workunit.client.1.smithi130.stdout:1/219: dread db/d1b/d1f/d2a/f2f [0,4194304] 0 2024-05-13T22:48:27.880 INFO:tasks.workunit.client.1.smithi130.stdout:7/190: mkdir d7/da/d1d/d42/d43 0 2024-05-13T22:48:27.883 INFO:tasks.workunit.client.0.smithi102.stdout:1/854: dwrite d2/d4/d5/d19/dd3/fe8 [0,4194304] 0 2024-05-13T22:48:27.883 INFO:tasks.workunit.client.1.smithi130.stdout:3/194: creat d8/dc/f3e x:0 0 0 2024-05-13T22:48:27.886 INFO:tasks.workunit.client.1.smithi130.stdout:1/220: fsync db/d1b/d1f/d49/f48 0 2024-05-13T22:48:27.886 INFO:tasks.workunit.client.1.smithi130.stdout:3/195: write f2 [554241,92043] 0 2024-05-13T22:48:27.886 INFO:tasks.workunit.client.0.smithi102.stdout:1/855: fdatasync d2/f4b 0 2024-05-13T22:48:27.888 INFO:tasks.workunit.client.1.smithi130.stdout:1/221: dread - db/d1b/d1f/d2a/f41 zero size 2024-05-13T22:48:27.898 INFO:tasks.workunit.client.0.smithi102.stdout:1/856: symlink d2/d4/d5/d19/ded/d10d/l116 0 2024-05-13T22:48:27.898 INFO:tasks.workunit.client.1.smithi130.stdout:4/251: dwrite d8/f1e [4194304,4194304] 0 2024-05-13T22:48:27.900 INFO:tasks.workunit.client.0.smithi102.stdout:1/857: dread d2/d45/f58 [0,4194304] 0 2024-05-13T22:48:27.900 INFO:tasks.workunit.client.1.smithi130.stdout:7/191: rename d7/da/d1f/d2c/f41 to d7/da/d1d/d42/f44 0 2024-05-13T22:48:27.901 INFO:tasks.workunit.client.0.smithi102.stdout:1/858: truncate d2/ff 4660710 0 2024-05-13T22:48:27.901 INFO:tasks.workunit.client.1.smithi130.stdout:3/196: symlink d8/l3f 0 2024-05-13T22:48:27.902 INFO:tasks.workunit.client.0.smithi102.stdout:1/859: write d2/d4/d5/f15 [761449,31346] 0 2024-05-13T22:48:27.902 INFO:tasks.workunit.client.1.smithi130.stdout:0/219: truncate d0/d7/f23 172367 0 2024-05-13T22:48:27.905 INFO:tasks.workunit.client.0.smithi102.stdout:1/860: unlink d2/d10/cb1 0 2024-05-13T22:48:27.908 INFO:tasks.workunit.client.1.smithi130.stdout:2/256: write d4/db/dc/f10 [1511367,34445] 0 2024-05-13T22:48:27.912 INFO:tasks.workunit.client.1.smithi130.stdout:9/229: rename d1/d15/f44 to d1/d7/d23/d28/d2d/f46 0 2024-05-13T22:48:27.913 INFO:tasks.workunit.client.0.smithi102.stdout:1/861: unlink d2/d4/d5/d19/ded/c3e 0 2024-05-13T22:48:27.918 INFO:tasks.workunit.client.0.smithi102.stdout:1/862: chown d2/d4/d5/fa 75970226 1 2024-05-13T22:48:27.936 INFO:tasks.workunit.client.0.smithi102.stdout:1/863: dwrite d2/d4/d5/d19/ded/fd1 [0,4194304] 0 2024-05-13T22:48:27.940 INFO:tasks.workunit.client.0.smithi102.stdout:1/864: dread d2/f4b [0,4194304] 0 2024-05-13T22:48:27.946 INFO:tasks.workunit.client.0.smithi102.stdout:1/865: unlink d2/d4/d5/d19/dbe/fc1 0 2024-05-13T22:48:27.946 INFO:tasks.workunit.client.1.smithi130.stdout:7/192: creat d7/da/d1d/d42/f45 x:0 0 0 2024-05-13T22:48:27.946 INFO:tasks.workunit.client.1.smithi130.stdout:1/222: mkdir db/d4a 0 2024-05-13T22:48:27.947 INFO:tasks.workunit.client.1.smithi130.stdout:9/230: truncate d1/d7/d23/d28/d2d/f34 1134740 0 2024-05-13T22:48:27.947 INFO:tasks.workunit.client.1.smithi130.stdout:3/197: symlink d8/d35/l40 0 2024-05-13T22:48:27.947 INFO:tasks.workunit.client.1.smithi130.stdout:2/257: dread d4/db/d26/f55 [0,4194304] 0 2024-05-13T22:48:27.948 INFO:tasks.workunit.client.0.smithi102.stdout:1/866: creat d2/d45/d79/f117 x:0 0 0 2024-05-13T22:48:27.948 INFO:tasks.workunit.client.1.smithi130.stdout:0/220: creat d0/d7/dd/f46 x:0 0 0 2024-05-13T22:48:27.950 INFO:tasks.workunit.client.1.smithi130.stdout:9/231: fdatasync d1/d7/d23/d28/d2d/f42 0 2024-05-13T22:48:27.951 INFO:tasks.workunit.client.1.smithi130.stdout:6/266: dwrite d4/d15/d1f/f20 [0,4194304] 0 2024-05-13T22:48:27.951 INFO:tasks.workunit.client.0.smithi102.stdout:1/867: read d2/d4/fb5 [234141,18245] 0 2024-05-13T22:48:27.953 INFO:tasks.workunit.client.1.smithi130.stdout:3/198: truncate d8/dc/d19/f30 159790 0 2024-05-13T22:48:27.954 INFO:tasks.workunit.client.1.smithi130.stdout:3/199: read d8/dc/d19/d2c/f3b [3191570,106513] 0 2024-05-13T22:48:27.961 INFO:tasks.workunit.client.1.smithi130.stdout:3/200: dwrite d8/dc/f37 [0,4194304] 0 2024-05-13T22:48:27.961 INFO:tasks.workunit.client.1.smithi130.stdout:7/193: write d7/d8/f19 [3737850,117283] 0 2024-05-13T22:48:27.962 INFO:tasks.workunit.client.1.smithi130.stdout:9/232: rmdir d1/db 39 2024-05-13T22:48:27.962 INFO:tasks.workunit.client.1.smithi130.stdout:7/194: stat d7/da/d1d/c3a 0 2024-05-13T22:48:27.963 INFO:tasks.workunit.client.1.smithi130.stdout:1/223: truncate db/f17 1623756 0 2024-05-13T22:48:27.968 INFO:tasks.workunit.client.1.smithi130.stdout:3/201: fdatasync d8/d1a/d1c/f25 0 2024-05-13T22:48:27.969 INFO:tasks.workunit.client.1.smithi130.stdout:9/233: mknod d1/d7/d23/d28/c47 0 2024-05-13T22:48:27.972 INFO:tasks.workunit.client.1.smithi130.stdout:3/202: write d8/dc/d15/f18 [2947175,78017] 0 2024-05-13T22:48:27.972 INFO:tasks.workunit.client.1.smithi130.stdout:7/195: mkdir d7/da/d1d/d3d/d46 0 2024-05-13T22:48:27.974 INFO:tasks.workunit.client.1.smithi130.stdout:2/258: getdents d4/d49/d48 0 2024-05-13T22:48:27.974 INFO:tasks.workunit.client.1.smithi130.stdout:0/221: dwrite d0/f42 [0,4194304] 0 2024-05-13T22:48:27.975 INFO:tasks.workunit.client.0.smithi102.stdout:1/868: dread d2/d10/d62/f88 [0,4194304] 0 2024-05-13T22:48:27.976 INFO:tasks.workunit.client.1.smithi130.stdout:3/203: truncate d8/dc/d19/f30 1094772 0 2024-05-13T22:48:27.976 INFO:tasks.workunit.client.1.smithi130.stdout:2/259: read d4/db/d26/f2d [2503581,105611] 0 2024-05-13T22:48:27.976 INFO:tasks.workunit.client.0.smithi102.stdout:1/869: chown d2/d45/d79 5887607 1 2024-05-13T22:48:27.977 INFO:tasks.workunit.client.1.smithi130.stdout:9/234: rename d1/d7/fa to d1/d7/d23/d28/d2d/f48 0 2024-05-13T22:48:27.978 INFO:tasks.workunit.client.1.smithi130.stdout:7/196: write d7/da/d1d/d42/f44 [945809,6424] 0 2024-05-13T22:48:27.978 INFO:tasks.workunit.client.1.smithi130.stdout:0/222: creat d0/d7/dd/f47 x:0 0 0 2024-05-13T22:48:27.980 INFO:tasks.workunit.client.1.smithi130.stdout:2/260: mkdir d4/db/dc/d18/d2a/d66 0 2024-05-13T22:48:27.981 INFO:tasks.workunit.client.1.smithi130.stdout:0/223: read d0/d7/dd/d2e/f2f [449033,21134] 0 2024-05-13T22:48:27.982 INFO:tasks.workunit.client.0.smithi102.stdout:1/870: mkdir d2/d4/d5/d19/ded/d118 0 2024-05-13T22:48:27.983 INFO:tasks.workunit.client.1.smithi130.stdout:9/235: mknod d1/d7/d23/d28/d32/c49 0 2024-05-13T22:48:27.986 INFO:tasks.workunit.client.0.smithi102.stdout:8/990: sync 2024-05-13T22:48:27.988 INFO:tasks.workunit.client.1.smithi130.stdout:0/224: symlink d0/d7/dd/d2e/l48 0 2024-05-13T22:48:27.990 INFO:tasks.workunit.client.1.smithi130.stdout:7/197: dread d7/da/d1d/d42/f44 [0,4194304] 0 2024-05-13T22:48:27.991 INFO:tasks.workunit.client.1.smithi130.stdout:9/236: rename d1/d7/c1c to d1/d7/d23/d28/d2d/d2f/c4a 0 2024-05-13T22:48:27.991 INFO:tasks.workunit.client.1.smithi130.stdout:1/224: dread db/f14 [4194304,4194304] 0 2024-05-13T22:48:27.992 INFO:tasks.workunit.client.1.smithi130.stdout:3/204: getdents d8/dc/d19 0 2024-05-13T22:48:27.993 INFO:tasks.workunit.client.0.smithi102.stdout:1/871: dread d2/d4/fb8 [0,4194304] 0 2024-05-13T22:48:27.997 INFO:tasks.workunit.client.0.smithi102.stdout:8/991: dread d2/f32 [0,4194304] 0 2024-05-13T22:48:27.998 INFO:tasks.workunit.client.1.smithi130.stdout:0/225: symlink d0/d7/d29/d3c/l49 0 2024-05-13T22:48:27.999 INFO:tasks.workunit.client.1.smithi130.stdout:7/198: dwrite d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:27.999 INFO:tasks.workunit.client.0.smithi102.stdout:1/872: unlink d2/d10/d62/d86/ceb 0 2024-05-13T22:48:28.000 INFO:tasks.workunit.client.1.smithi130.stdout:0/226: fsync d0/d7/d29/f37 0 2024-05-13T22:48:28.001 INFO:tasks.workunit.client.1.smithi130.stdout:1/225: dwrite db/f1a [4194304,4194304] 0 2024-05-13T22:48:28.002 INFO:tasks.workunit.client.1.smithi130.stdout:3/205: mknod d8/dc/c41 0 2024-05-13T22:48:28.005 INFO:tasks.workunit.client.1.smithi130.stdout:3/206: chown d8/dc/d19/d27/l31 0 1 2024-05-13T22:48:28.005 INFO:tasks.workunit.client.1.smithi130.stdout:1/226: truncate db/d1b/d1f/d2a/f41 556395 0 2024-05-13T22:48:28.006 INFO:tasks.workunit.client.1.smithi130.stdout:2/261: truncate d4/db/f2c 3600298 0 2024-05-13T22:48:28.007 INFO:tasks.workunit.client.0.smithi102.stdout:8/992: creat d2/d7/d27/d64/d80/dc8/d93/dab/f13d x:0 0 0 2024-05-13T22:48:28.008 INFO:tasks.workunit.client.0.smithi102.stdout:8/993: fdatasync d2/d36/d49/f60 0 2024-05-13T22:48:28.008 INFO:tasks.workunit.client.0.smithi102.stdout:1/873: mkdir d2/d4/d38/db2/dba/d10b/d119 0 2024-05-13T22:48:28.012 INFO:tasks.workunit.client.1.smithi130.stdout:3/207: dwrite d8/dc/d19/f1e [4194304,4194304] 0 2024-05-13T22:48:28.013 INFO:tasks.workunit.client.0.smithi102.stdout:1/874: symlink d2/d45/d96/l11a 0 2024-05-13T22:48:28.014 INFO:tasks.workunit.client.1.smithi130.stdout:7/199: symlink d7/da/d1d/d42/d43/l47 0 2024-05-13T22:48:28.014 INFO:tasks.workunit.client.0.smithi102.stdout:8/994: mknod d2/d36/db3/d124/c13e 0 2024-05-13T22:48:28.015 INFO:tasks.workunit.client.1.smithi130.stdout:7/200: fsync d7/da/f35 0 2024-05-13T22:48:28.017 INFO:tasks.workunit.client.0.smithi102.stdout:8/995: fdatasync d2/d36/d49/d5d/ff0 0 2024-05-13T22:48:28.019 INFO:tasks.workunit.client.1.smithi130.stdout:2/262: creat d4/db/dc/d18/d2a/f67 x:0 0 0 2024-05-13T22:48:28.019 INFO:tasks.workunit.client.1.smithi130.stdout:0/227: symlink d0/d7/dd/d17/d1f/d2c/d45/l4a 0 2024-05-13T22:48:28.021 INFO:tasks.workunit.client.0.smithi102.stdout:8/996: mknod d2/d36/db3/d124/c13f 0 2024-05-13T22:48:28.022 INFO:tasks.workunit.client.1.smithi130.stdout:1/227: creat db/d4a/f4b x:0 0 0 2024-05-13T22:48:28.024 INFO:tasks.workunit.client.1.smithi130.stdout:2/263: read d4/db/f28 [5179561,63839] 0 2024-05-13T22:48:28.024 INFO:tasks.workunit.client.1.smithi130.stdout:3/208: mkdir d8/dc/d42 0 2024-05-13T22:48:28.026 INFO:tasks.workunit.client.0.smithi102.stdout:8/997: creat d2/d7/d27/d64/d80/dc8/d93/f140 x:0 0 0 2024-05-13T22:48:28.027 INFO:tasks.workunit.client.0.smithi102.stdout:8/998: stat d2/d7/c81 0 2024-05-13T22:48:28.028 INFO:tasks.workunit.client.1.smithi130.stdout:2/264: creat d4/d56/f68 x:0 0 0 2024-05-13T22:48:28.028 INFO:tasks.workunit.client.1.smithi130.stdout:1/228: mkdir db/d1b/d1f/d2a/d4c 0 2024-05-13T22:48:28.029 INFO:tasks.workunit.client.1.smithi130.stdout:2/265: write d4/d49/f35 [229283,121247] 0 2024-05-13T22:48:28.030 INFO:tasks.workunit.client.1.smithi130.stdout:0/228: getdents d0/d7 0 2024-05-13T22:48:28.032 INFO:tasks.workunit.client.0.smithi102.stdout:8/999: rename d2/d7/f69 to d2/d83/f141 0 2024-05-13T22:48:28.034 INFO:tasks.workunit.client.1.smithi130.stdout:2/266: mkdir d4/db/dc/d18/d2a/d30/d69 0 2024-05-13T22:48:28.036 INFO:tasks.workunit.client.1.smithi130.stdout:0/229: rename d0/d7/d10 to d0/d7/dd/d2e/d4b 0 2024-05-13T22:48:28.037 INFO:tasks.workunit.client.0.smithi102.stdout:1/875: dread d2/d45/f84 [0,4194304] 0 2024-05-13T22:48:28.043 INFO:tasks.workunit.client.1.smithi130.stdout:5/228: sync 2024-05-13T22:48:28.043 INFO:tasks.workunit.client.1.smithi130.stdout:8/200: sync 2024-05-13T22:48:28.043 INFO:tasks.workunit.client.0.smithi102.stdout:1/876: dread d2/d4/d26/d40/f72 [0,4194304] 0 2024-05-13T22:48:28.044 INFO:tasks.workunit.client.1.smithi130.stdout:8/201: readlink d1/d2/db/d15/d16/l20 0 2024-05-13T22:48:28.045 INFO:tasks.workunit.client.1.smithi130.stdout:2/267: creat d4/d49/d48/d4a/d64/f6a x:0 0 0 2024-05-13T22:48:28.046 INFO:tasks.workunit.client.1.smithi130.stdout:3/209: dread d8/dc/f12 [0,4194304] 0 2024-05-13T22:48:28.046 INFO:tasks.workunit.client.1.smithi130.stdout:3/210: stat d8/f2b 0 2024-05-13T22:48:28.046 INFO:tasks.workunit.client.1.smithi130.stdout:9/237: truncate d1/d7/d23/d28/d2d/f3a 1534211 0 2024-05-13T22:48:28.047 INFO:tasks.workunit.client.0.smithi102.stdout:1/877: mkdir d2/d4/d5/d19/dd3/d11b 0 2024-05-13T22:48:28.048 INFO:tasks.workunit.client.1.smithi130.stdout:3/211: read d8/dc/f37 [1929625,89580] 0 2024-05-13T22:48:28.048 INFO:tasks.workunit.client.1.smithi130.stdout:9/238: stat d1/d7/d23/d28/d2d/d2f/c31 0 2024-05-13T22:48:28.049 INFO:tasks.workunit.client.1.smithi130.stdout:9/239: read d1/d7/d23/d3d/f40 [3275604,83632] 0 2024-05-13T22:48:28.049 INFO:tasks.workunit.client.1.smithi130.stdout:5/229: mkdir d6/d7/d24/d4b 0 2024-05-13T22:48:28.050 INFO:tasks.workunit.client.1.smithi130.stdout:5/230: write d6/d7/f45 [3921108,78372] 0 2024-05-13T22:48:28.051 INFO:tasks.workunit.client.1.smithi130.stdout:8/202: creat d1/d2/db/f45 x:0 0 0 2024-05-13T22:48:28.051 INFO:tasks.workunit.client.1.smithi130.stdout:5/231: write d6/dd/d18/fc [1400299,72698] 0 2024-05-13T22:48:28.052 INFO:tasks.workunit.client.1.smithi130.stdout:9/240: dread d1/d15/f25 [0,4194304] 0 2024-05-13T22:48:28.053 INFO:tasks.workunit.client.1.smithi130.stdout:2/268: unlink d4/db/f1c 0 2024-05-13T22:48:28.054 INFO:tasks.workunit.client.1.smithi130.stdout:9/241: write d1/d7/d23/d28/d2d/d2f/f3c [885700,58920] 0 2024-05-13T22:48:28.057 INFO:tasks.workunit.client.0.smithi102.stdout:1/878: creat d2/d10/f11c x:0 0 0 2024-05-13T22:48:28.058 INFO:tasks.workunit.client.1.smithi130.stdout:2/269: dread d4/f1e [0,4194304] 0 2024-05-13T22:48:28.058 INFO:tasks.workunit.client.1.smithi130.stdout:2/270: read d4/db/d26/f55 [242611,113354] 0 2024-05-13T22:48:28.059 INFO:tasks.workunit.client.1.smithi130.stdout:2/271: stat d4/f5f 0 2024-05-13T22:48:28.060 INFO:tasks.workunit.client.1.smithi130.stdout:7/201: truncate d7/da/f35 1159521 0 2024-05-13T22:48:28.061 INFO:tasks.workunit.client.1.smithi130.stdout:5/232: symlink d6/d27/l4c 0 2024-05-13T22:48:28.065 INFO:tasks.workunit.client.1.smithi130.stdout:1/229: getdents db/d1b/d1f/d2a 0 2024-05-13T22:48:28.067 INFO:tasks.workunit.client.1.smithi130.stdout:2/272: mknod d4/d49/d48/c6b 0 2024-05-13T22:48:28.068 INFO:tasks.workunit.client.1.smithi130.stdout:8/203: dread d1/d2/db/d15/f22 [0,4194304] 0 2024-05-13T22:48:28.069 INFO:tasks.workunit.client.1.smithi130.stdout:8/204: readlink d1/d2/db/d15/d16/l20 0 2024-05-13T22:48:28.070 INFO:tasks.workunit.client.1.smithi130.stdout:7/202: mknod d7/da/d1d/d3d/c48 0 2024-05-13T22:48:28.070 INFO:tasks.workunit.client.1.smithi130.stdout:8/205: fsync d1/dc/f21 0 2024-05-13T22:48:28.075 INFO:tasks.workunit.client.1.smithi130.stdout:9/242: link d1/d7/f1a d1/d7/d23/d3d/d43/f4b 0 2024-05-13T22:48:28.076 INFO:tasks.workunit.client.1.smithi130.stdout:9/243: chown d1/d7/d23/d28/l2e 646 1 2024-05-13T22:48:28.076 INFO:tasks.workunit.client.1.smithi130.stdout:1/230: creat db/d1b/d1f/d2a/d35/f4d x:0 0 0 2024-05-13T22:48:28.078 INFO:tasks.workunit.client.1.smithi130.stdout:1/231: write db/d1b/d1f/d2a/f34 [1859098,91262] 0 2024-05-13T22:48:28.078 INFO:tasks.workunit.client.1.smithi130.stdout:0/230: truncate d0/d7/d29/f37 424299 0 2024-05-13T22:48:28.079 INFO:tasks.workunit.client.1.smithi130.stdout:1/232: write db/d1b/f20 [1097134,86940] 0 2024-05-13T22:48:28.081 INFO:tasks.workunit.client.1.smithi130.stdout:3/212: getdents d8 0 2024-05-13T22:48:28.082 INFO:tasks.workunit.client.1.smithi130.stdout:2/273: symlink d4/db/dc/d18/l6c 0 2024-05-13T22:48:28.083 INFO:tasks.workunit.client.1.smithi130.stdout:2/274: chown d4/db/dc/d18/c31 22 1 2024-05-13T22:48:28.083 INFO:tasks.workunit.client.1.smithi130.stdout:2/275: dread - d4/db/f41 zero size 2024-05-13T22:48:28.084 INFO:tasks.workunit.client.1.smithi130.stdout:2/276: stat d4/d49/f35 0 2024-05-13T22:48:28.085 INFO:tasks.workunit.client.1.smithi130.stdout:5/233: rename d6/d27 to d6/d7/d19/d4d 0 2024-05-13T22:48:28.087 INFO:tasks.workunit.client.1.smithi130.stdout:7/203: symlink d7/l49 0 2024-05-13T22:48:28.088 INFO:tasks.workunit.client.1.smithi130.stdout:9/244: symlink d1/d7/d23/d28/d32/d45/l4c 0 2024-05-13T22:48:28.088 INFO:tasks.workunit.client.1.smithi130.stdout:9/245: readlink d1/l14 0 2024-05-13T22:48:28.089 INFO:tasks.workunit.client.1.smithi130.stdout:3/213: symlink d8/d1a/d1c/l43 0 2024-05-13T22:48:28.089 INFO:tasks.workunit.client.1.smithi130.stdout:3/214: stat d8/dc/d19/d27/c2a 0 2024-05-13T22:48:28.091 INFO:tasks.workunit.client.1.smithi130.stdout:0/231: creat d0/d7/dd/d17/d1f/f4c x:0 0 0 2024-05-13T22:48:28.095 INFO:tasks.workunit.client.1.smithi130.stdout:2/277: stat d4/d49/d48/f5d 0 2024-05-13T22:48:28.095 INFO:tasks.workunit.client.1.smithi130.stdout:2/278: fsync d4/d56/f68 0 2024-05-13T22:48:28.096 INFO:tasks.workunit.client.1.smithi130.stdout:9/246: symlink d1/d7/d23/d28/d2d/d2f/l4d 0 2024-05-13T22:48:28.096 INFO:tasks.workunit.client.1.smithi130.stdout:2/279: chown d4/f1b 2292 1 2024-05-13T22:48:28.097 INFO:tasks.workunit.client.1.smithi130.stdout:3/215: mknod d8/dc/d19/c44 0 2024-05-13T22:48:28.098 INFO:tasks.workunit.client.1.smithi130.stdout:1/233: dread db/f1a [0,4194304] 0 2024-05-13T22:48:28.099 INFO:tasks.workunit.client.1.smithi130.stdout:0/232: symlink d0/d7/dd/d17/d1f/d2c/l4d 0 2024-05-13T22:48:28.099 INFO:tasks.workunit.client.1.smithi130.stdout:3/216: write d8/d35/f2e [817074,61758] 0 2024-05-13T22:48:28.104 INFO:tasks.workunit.client.1.smithi130.stdout:1/234: mknod db/d1b/d1f/d2a/d4c/c4e 0 2024-05-13T22:48:28.104 INFO:tasks.workunit.client.1.smithi130.stdout:0/233: dread - d0/d7/dd/d2e/f3f zero size 2024-05-13T22:48:28.104 INFO:tasks.workunit.client.1.smithi130.stdout:5/234: dread d6/dd/f13 [0,4194304] 0 2024-05-13T22:48:28.106 INFO:tasks.workunit.client.1.smithi130.stdout:2/280: dwrite d4/f5 [4194304,4194304] 0 2024-05-13T22:48:28.107 INFO:tasks.workunit.client.1.smithi130.stdout:7/204: dread d7/d8/f13 [0,4194304] 0 2024-05-13T22:48:28.110 INFO:tasks.workunit.client.1.smithi130.stdout:1/235: unlink db/d1b/d1f/c3c 0 2024-05-13T22:48:28.112 INFO:tasks.workunit.client.1.smithi130.stdout:5/235: write d6/d7/d32/f44 [721379,115719] 0 2024-05-13T22:48:28.113 INFO:tasks.workunit.client.1.smithi130.stdout:3/217: mknod d8/dc/d19/c45 0 2024-05-13T22:48:28.114 INFO:tasks.workunit.client.1.smithi130.stdout:5/236: stat d6/dd/f33 0 2024-05-13T22:48:28.117 INFO:tasks.workunit.client.1.smithi130.stdout:7/205: getdents d7/da/d1d/d42/d43 0 2024-05-13T22:48:28.118 INFO:tasks.workunit.client.1.smithi130.stdout:3/218: creat d8/dc/d19/d2c/f46 x:0 0 0 2024-05-13T22:48:28.119 INFO:tasks.workunit.client.1.smithi130.stdout:5/237: mknod d6/dd/d18/c4e 0 2024-05-13T22:48:28.119 INFO:tasks.workunit.client.1.smithi130.stdout:1/236: truncate db/d1b/d1f/d2a/d35/f4d 863971 0 2024-05-13T22:48:28.120 INFO:tasks.workunit.client.1.smithi130.stdout:2/281: dwrite d4/db/dc/d18/f23 [0,4194304] 0 2024-05-13T22:48:28.121 INFO:tasks.workunit.client.1.smithi130.stdout:7/206: write d7/d8/f13 [1330314,63587] 0 2024-05-13T22:48:28.123 INFO:tasks.workunit.client.1.smithi130.stdout:5/238: unlink d6/dd/d18/f3a 0 2024-05-13T22:48:28.124 INFO:tasks.workunit.client.1.smithi130.stdout:9/247: dwrite d1/f33 [0,4194304] 0 2024-05-13T22:48:28.126 INFO:tasks.workunit.client.1.smithi130.stdout:1/237: rename db/d1b/d1f/d2a/f34 to db/d1b/d1f/d2a/f4f 0 2024-05-13T22:48:28.128 INFO:tasks.workunit.client.1.smithi130.stdout:1/238: write db/f32 [3860486,14254] 0 2024-05-13T22:48:28.133 INFO:tasks.workunit.client.1.smithi130.stdout:4/252: sync 2024-05-13T22:48:28.133 INFO:tasks.workunit.client.1.smithi130.stdout:6/267: sync 2024-05-13T22:48:28.147 INFO:tasks.workunit.client.1.smithi130.stdout:5/239: symlink d6/d1a/l4f 0 2024-05-13T22:48:28.149 INFO:tasks.workunit.client.1.smithi130.stdout:3/219: truncate f2 650617 0 2024-05-13T22:48:28.150 INFO:tasks.workunit.client.1.smithi130.stdout:5/240: symlink d6/dd/d18/l50 0 2024-05-13T22:48:28.151 INFO:tasks.workunit.client.1.smithi130.stdout:4/253: mkdir d8/dc/d35/d33/d3f/d57 0 2024-05-13T22:48:28.155 INFO:tasks.workunit.client.1.smithi130.stdout:5/241: rename d6/dd/ff to d6/d7/d32/f51 0 2024-05-13T22:48:28.156 INFO:tasks.workunit.client.1.smithi130.stdout:3/220: dread d8/f34 [0,4194304] 0 2024-05-13T22:48:28.157 INFO:tasks.workunit.client.1.smithi130.stdout:0/234: dread d0/f1e [0,4194304] 0 2024-05-13T22:48:28.158 INFO:tasks.workunit.client.1.smithi130.stdout:5/242: creat d6/d7/d19/f52 x:0 0 0 2024-05-13T22:48:28.163 INFO:tasks.workunit.client.1.smithi130.stdout:4/254: dwrite d8/df/f20 [0,4194304] 0 2024-05-13T22:48:28.168 INFO:tasks.workunit.client.1.smithi130.stdout:5/243: getdents d6/dd/d18 0 2024-05-13T22:48:28.178 INFO:tasks.workunit.client.1.smithi130.stdout:4/255: mknod d8/dc/d35/d33/d3f/d57/c58 0 2024-05-13T22:48:28.178 INFO:tasks.workunit.client.1.smithi130.stdout:3/221: dwrite d8/d35/f2e [0,4194304] 0 2024-05-13T22:48:28.181 INFO:tasks.workunit.client.1.smithi130.stdout:0/235: dwrite d0/f21 [0,4194304] 0 2024-05-13T22:48:28.181 INFO:tasks.workunit.client.1.smithi130.stdout:1/239: dwrite db/f17 [0,4194304] 0 2024-05-13T22:48:28.184 INFO:tasks.workunit.client.1.smithi130.stdout:9/248: dwrite d1/db/fc [0,4194304] 0 2024-05-13T22:48:28.184 INFO:tasks.workunit.client.1.smithi130.stdout:4/256: write f7 [2294211,44937] 0 2024-05-13T22:48:28.185 INFO:tasks.workunit.client.1.smithi130.stdout:3/222: stat f2 0 2024-05-13T22:48:28.186 INFO:tasks.workunit.client.1.smithi130.stdout:0/236: write d0/d7/d29/f37 [584517,126180] 0 2024-05-13T22:48:28.193 INFO:tasks.workunit.client.1.smithi130.stdout:3/223: dread - d8/dc/f3e zero size 2024-05-13T22:48:28.193 INFO:tasks.workunit.client.1.smithi130.stdout:1/240: symlink db/l50 0 2024-05-13T22:48:28.193 INFO:tasks.workunit.client.1.smithi130.stdout:5/244: dwrite d6/dd/f37 [0,4194304] 0 2024-05-13T22:48:28.193 INFO:tasks.workunit.client.1.smithi130.stdout:4/257: rmdir d8/dc/d13 39 2024-05-13T22:48:28.195 INFO:tasks.workunit.client.1.smithi130.stdout:0/237: creat d0/d7/dd/d2e/f4e x:0 0 0 2024-05-13T22:48:28.197 INFO:tasks.workunit.client.1.smithi130.stdout:5/245: rmdir d6/d7/d19 39 2024-05-13T22:48:28.197 INFO:tasks.workunit.client.1.smithi130.stdout:9/249: write d1/d7/d23/d28/d2d/f46 [774403,84437] 0 2024-05-13T22:48:28.198 INFO:tasks.workunit.client.1.smithi130.stdout:4/258: creat d8/df/f59 x:0 0 0 2024-05-13T22:48:28.198 INFO:tasks.workunit.client.1.smithi130.stdout:0/238: rmdir d0/d7 39 2024-05-13T22:48:28.199 INFO:tasks.workunit.client.1.smithi130.stdout:1/241: dread db/f17 [0,4194304] 0 2024-05-13T22:48:28.199 INFO:tasks.workunit.client.1.smithi130.stdout:3/224: creat d8/d1a/f47 x:0 0 0 2024-05-13T22:48:28.204 INFO:tasks.workunit.client.1.smithi130.stdout:4/259: fsync d8/dc/d29/f3a 0 2024-05-13T22:48:28.209 INFO:tasks.workunit.client.1.smithi130.stdout:8/206: sync 2024-05-13T22:48:28.210 INFO:tasks.workunit.client.1.smithi130.stdout:0/239: rename d0/c20 to d0/d7/dd/d17/c4f 0 2024-05-13T22:48:28.215 INFO:tasks.workunit.client.1.smithi130.stdout:1/242: dwrite db/d1b/f20 [0,4194304] 0 2024-05-13T22:48:28.215 INFO:tasks.workunit.client.1.smithi130.stdout:4/260: readlink d8/dc/d35/d33/d3f/l46 0 2024-05-13T22:48:28.215 INFO:tasks.workunit.client.1.smithi130.stdout:3/225: dwrite d8/dc/f3e [0,4194304] 0 2024-05-13T22:48:28.221 INFO:tasks.workunit.client.1.smithi130.stdout:1/243: readlink db/d1b/d1f/l24 0 2024-05-13T22:48:28.222 INFO:tasks.workunit.client.1.smithi130.stdout:9/250: dwrite d1/d15/f3b [0,4194304] 0 2024-05-13T22:48:28.223 INFO:tasks.workunit.client.1.smithi130.stdout:8/207: rename d1/d2/db/d15/f29 to d1/d2/db/f46 0 2024-05-13T22:48:28.228 INFO:tasks.workunit.client.1.smithi130.stdout:8/208: dread - d1/dc/f26 zero size 2024-05-13T22:48:28.230 INFO:tasks.workunit.client.1.smithi130.stdout:0/240: getdents d0/d7/d29/d3c 0 2024-05-13T22:48:28.231 INFO:tasks.workunit.client.1.smithi130.stdout:9/251: fdatasync d1/d7/d23/d3d/f40 0 2024-05-13T22:48:28.232 INFO:tasks.workunit.client.1.smithi130.stdout:5/246: dwrite d6/d7/d19/f2c [0,4194304] 0 2024-05-13T22:48:28.232 INFO:tasks.workunit.client.1.smithi130.stdout:3/226: getdents d8/dc/d19 0 2024-05-13T22:48:28.233 INFO:tasks.workunit.client.0.smithi102.stdout:0/970: sync 2024-05-13T22:48:28.233 INFO:tasks.workunit.client.0.smithi102.stdout:7/911: sync 2024-05-13T22:48:28.235 INFO:tasks.workunit.client.0.smithi102.stdout:7/912: creat d6/da9/f11c x:0 0 0 2024-05-13T22:48:28.236 INFO:tasks.workunit.client.0.smithi102.stdout:0/971: mknod d1/de/d32/d24/d2d/d38/d3d/d81/da5/c131 0 2024-05-13T22:48:28.238 INFO:tasks.workunit.client.1.smithi130.stdout:9/252: write d1/db/f35 [498524,54297] 0 2024-05-13T22:48:28.238 INFO:tasks.workunit.client.1.smithi130.stdout:4/261: dwrite d8/f1e [0,4194304] 0 2024-05-13T22:48:28.241 INFO:tasks.workunit.client.1.smithi130.stdout:3/227: truncate d8/f2b 1265863 0 2024-05-13T22:48:28.242 INFO:tasks.workunit.client.1.smithi130.stdout:5/247: mkdir d6/d7/d19/d4d/d53 0 2024-05-13T22:48:28.242 INFO:tasks.workunit.client.1.smithi130.stdout:1/244: dwrite db/f14 [4194304,4194304] 0 2024-05-13T22:48:28.242 INFO:tasks.workunit.client.1.smithi130.stdout:3/228: dread - d8/d1a/f3c zero size 2024-05-13T22:48:28.244 INFO:tasks.workunit.client.1.smithi130.stdout:4/262: mknod d8/dc/d35/d33/d3f/c5a 0 2024-05-13T22:48:28.245 INFO:tasks.workunit.client.1.smithi130.stdout:0/241: rename d0/d7/dd/d17/d1f/f4c to d0/d7/dd/d17/d1f/d2c/f50 0 2024-05-13T22:48:28.248 INFO:tasks.workunit.client.1.smithi130.stdout:9/253: symlink d1/d7/d23/d28/d32/l4e 0 2024-05-13T22:48:28.248 INFO:tasks.workunit.client.0.smithi102.stdout:0/972: dread d1/fad [0,4194304] 0 2024-05-13T22:48:28.250 INFO:tasks.workunit.client.1.smithi130.stdout:3/229: rename d8/d1a/f3c to d8/d35/f48 0 2024-05-13T22:48:28.252 INFO:tasks.workunit.client.1.smithi130.stdout:4/263: fdatasync d8/dc/d26/f52 0 2024-05-13T22:48:28.256 INFO:tasks.workunit.client.1.smithi130.stdout:0/242: read d0/f4 [493936,29965] 0 2024-05-13T22:48:28.256 INFO:tasks.workunit.client.0.smithi102.stdout:0/973: creat d1/de/d32/f132 x:0 0 0 2024-05-13T22:48:28.257 INFO:tasks.workunit.client.1.smithi130.stdout:3/230: creat d8/d1a/d1c/f49 x:0 0 0 2024-05-13T22:48:28.261 INFO:tasks.workunit.client.1.smithi130.stdout:5/248: creat d6/f54 x:0 0 0 2024-05-13T22:48:28.262 INFO:tasks.workunit.client.1.smithi130.stdout:0/243: symlink d0/d7/dd/d17/l51 0 2024-05-13T22:48:28.262 INFO:tasks.workunit.client.1.smithi130.stdout:8/209: dwrite d1/d2/f39 [0,4194304] 0 2024-05-13T22:48:28.265 INFO:tasks.workunit.client.1.smithi130.stdout:9/254: creat d1/f4f x:0 0 0 2024-05-13T22:48:28.268 INFO:tasks.workunit.client.1.smithi130.stdout:8/210: symlink d1/dc/l47 0 2024-05-13T22:48:28.269 INFO:tasks.workunit.client.1.smithi130.stdout:9/255: write d1/db/f35 [231402,128953] 0 2024-05-13T22:48:28.270 INFO:tasks.workunit.client.1.smithi130.stdout:4/264: dwrite d8/dc/d13/f47 [0,4194304] 0 2024-05-13T22:48:28.271 INFO:tasks.workunit.client.1.smithi130.stdout:0/244: mkdir d0/d7/dd/d17/d52 0 2024-05-13T22:48:28.272 INFO:tasks.workunit.client.0.smithi102.stdout:7/913: write d6/d13/d58/fac [4048,56258] 0 2024-05-13T22:48:28.273 INFO:tasks.workunit.client.1.smithi130.stdout:9/256: readlink d1/l14 0 2024-05-13T22:48:28.273 INFO:tasks.workunit.client.1.smithi130.stdout:4/265: chown d8/cb 902 1 2024-05-13T22:48:28.274 INFO:tasks.workunit.client.1.smithi130.stdout:3/231: read d8/f2b [491978,52199] 0 2024-05-13T22:48:28.276 INFO:tasks.workunit.client.1.smithi130.stdout:3/232: write d8/dc/f22 [1115450,63663] 0 2024-05-13T22:48:28.277 INFO:tasks.workunit.client.1.smithi130.stdout:5/249: dwrite d6/d7/d19/d4d/f2d [0,4194304] 0 2024-05-13T22:48:28.278 INFO:tasks.workunit.client.1.smithi130.stdout:5/250: chown d6/d2f/f39 4694 1 2024-05-13T22:48:28.282 INFO:tasks.workunit.client.0.smithi102.stdout:4/988: sync 2024-05-13T22:48:28.282 INFO:tasks.workunit.client.0.smithi102.stdout:9/948: sync 2024-05-13T22:48:28.283 INFO:tasks.workunit.client.0.smithi102.stdout:1/879: sync 2024-05-13T22:48:28.283 INFO:tasks.workunit.client.1.smithi130.stdout:1/245: truncate db/f32 3352309 0 2024-05-13T22:48:28.285 INFO:tasks.workunit.client.0.smithi102.stdout:9/949: chown d1/d6/d21/d12e/d75/d96/d102/d6c/d10f/f119 3 1 2024-05-13T22:48:28.287 INFO:tasks.workunit.client.1.smithi130.stdout:9/257: mknod d1/c50 0 2024-05-13T22:48:28.288 INFO:tasks.workunit.client.0.smithi102.stdout:9/950: read d1/d6/d21/d12e/d75/db9/ff0 [190193,127760] 0 2024-05-13T22:48:28.290 INFO:tasks.workunit.client.0.smithi102.stdout:0/974: write d1/de/d32/d24/d2d/d38/d3d/f9a [1900668,8999] 0 2024-05-13T22:48:28.291 INFO:tasks.workunit.client.1.smithi130.stdout:5/251: mknod d6/dd/d18/c55 0 2024-05-13T22:48:28.293 INFO:tasks.workunit.client.0.smithi102.stdout:1/880: dwrite d2/d4/d5/d19/ded/d6a/dbd/ffe [0,4194304] 0 2024-05-13T22:48:28.294 INFO:tasks.workunit.client.1.smithi130.stdout:0/245: creat d0/f53 x:0 0 0 2024-05-13T22:48:28.295 INFO:tasks.workunit.client.1.smithi130.stdout:0/246: write d0/f42 [4415151,80610] 0 2024-05-13T22:48:28.300 INFO:tasks.workunit.client.0.smithi102.stdout:0/975: rmdir d1/de/d42/d61 39 2024-05-13T22:48:28.301 INFO:tasks.workunit.client.0.smithi102.stdout:1/881: dwrite d2/d4/d5/fe2 [0,4194304] 0 2024-05-13T22:48:28.303 INFO:tasks.workunit.client.0.smithi102.stdout:9/951: mkdir d1/d6/d45/d2d/d149 0 2024-05-13T22:48:28.303 INFO:tasks.workunit.client.0.smithi102.stdout:1/882: write d2/d10/fee [222074,95619] 0 2024-05-13T22:48:28.305 INFO:tasks.workunit.client.1.smithi130.stdout:5/252: mkdir d6/d7/d24/d56 0 2024-05-13T22:48:28.306 INFO:tasks.workunit.client.1.smithi130.stdout:3/233: dread d8/dc/d19/f1e [0,4194304] 0 2024-05-13T22:48:28.306 INFO:tasks.workunit.client.0.smithi102.stdout:7/914: dread d6/f7 [0,4194304] 0 2024-05-13T22:48:28.308 INFO:tasks.workunit.client.1.smithi130.stdout:3/234: truncate d8/d35/f3d 614524 0 2024-05-13T22:48:28.311 INFO:tasks.workunit.client.1.smithi130.stdout:4/266: link d8/dc/d35/d33/d44/f4b d8/dc/f5b 0 2024-05-13T22:48:28.312 INFO:tasks.workunit.client.1.smithi130.stdout:4/267: write f6 [3170747,98589] 0 2024-05-13T22:48:28.313 INFO:tasks.workunit.client.1.smithi130.stdout:0/247: rmdir d0/d7/dd 39 2024-05-13T22:48:28.318 INFO:tasks.workunit.client.0.smithi102.stdout:4/989: write d3/d22/f36 [1543624,38987] 0 2024-05-13T22:48:28.318 INFO:tasks.workunit.client.0.smithi102.stdout:0/976: read d1/de/d42/d61/fb5 [85436,9400] 0 2024-05-13T22:48:28.319 INFO:tasks.workunit.client.0.smithi102.stdout:4/990: stat d3/d19/d23/d4e/l9b 0 2024-05-13T22:48:28.320 INFO:tasks.workunit.client.0.smithi102.stdout:4/991: chown d3/d19/dad/dc9/d123 222104 1 2024-05-13T22:48:28.320 INFO:tasks.workunit.client.0.smithi102.stdout:0/977: truncate d1/d68/de2/f126 633682 0 2024-05-13T22:48:28.321 INFO:tasks.workunit.client.1.smithi130.stdout:9/258: dread d1/d7/d23/d3d/d43/f4b [0,4194304] 0 2024-05-13T22:48:28.321 INFO:tasks.workunit.client.0.smithi102.stdout:4/992: chown d3/d19/d23/d7e/f12e 3982271 1 2024-05-13T22:48:28.325 INFO:tasks.workunit.client.1.smithi130.stdout:1/246: dread db/d1b/d1f/d2a/f4f [0,4194304] 0 2024-05-13T22:48:28.325 INFO:tasks.workunit.client.0.smithi102.stdout:9/952: creat d1/d6/d45/d3e/f14a x:0 0 0 2024-05-13T22:48:28.327 INFO:tasks.workunit.client.1.smithi130.stdout:4/268: rmdir d8/dc/d29 39 2024-05-13T22:48:28.327 INFO:tasks.workunit.client.0.smithi102.stdout:7/915: dread d6/d15/d4d/fde [0,4194304] 0 2024-05-13T22:48:28.328 INFO:tasks.workunit.client.0.smithi102.stdout:0/978: creat d1/de/d32/d24/d2d/d38/d3d/d10a/f133 x:0 0 0 2024-05-13T22:48:28.333 INFO:tasks.workunit.client.0.smithi102.stdout:7/916: mkdir d6/d13/d2c/dcb/d11d 0 2024-05-13T22:48:28.334 INFO:tasks.workunit.client.1.smithi130.stdout:9/259: mkdir d1/d15/d51 0 2024-05-13T22:48:28.337 INFO:tasks.workunit.client.0.smithi102.stdout:4/993: symlink d3/d19/d23/d39/da7/da9/dd3/l155 0 2024-05-13T22:48:28.339 INFO:tasks.workunit.client.1.smithi130.stdout:3/235: dwrite d8/d1a/d1c/f25 [0,4194304] 0 2024-05-13T22:48:28.340 INFO:tasks.workunit.client.0.smithi102.stdout:7/917: symlink d6/d15/d4c/l11e 0 2024-05-13T22:48:28.340 INFO:tasks.workunit.client.0.smithi102.stdout:9/953: dread d1/d6/d45/d3e/f73 [0,4194304] 0 2024-05-13T22:48:28.341 INFO:tasks.workunit.client.1.smithi130.stdout:0/248: fdatasync d0/d7/d29/f33 0 2024-05-13T22:48:28.343 INFO:tasks.workunit.client.1.smithi130.stdout:5/253: creat d6/d7/f57 x:0 0 0 2024-05-13T22:48:28.346 INFO:tasks.workunit.client.0.smithi102.stdout:9/954: creat d1/d6/d3b/d5b/d127/f14b x:0 0 0 2024-05-13T22:48:28.347 INFO:tasks.workunit.client.0.smithi102.stdout:0/979: dwrite d1/d4d/d79/f80 [0,4194304] 0 2024-05-13T22:48:28.347 INFO:tasks.workunit.client.0.smithi102.stdout:4/994: symlink d3/d19/d23/l156 0 2024-05-13T22:48:28.349 INFO:tasks.workunit.client.1.smithi130.stdout:1/247: symlink db/d1b/d1f/d2a/d4c/l51 0 2024-05-13T22:48:28.349 INFO:tasks.workunit.client.0.smithi102.stdout:7/918: mknod d6/d13/d2c/d39/d114/c11f 0 2024-05-13T22:48:28.350 INFO:tasks.workunit.client.1.smithi130.stdout:0/249: dwrite d0/f1e [0,4194304] 0 2024-05-13T22:48:28.351 INFO:tasks.workunit.client.1.smithi130.stdout:4/269: symlink d8/dc/d29/l5c 0 2024-05-13T22:48:28.354 INFO:tasks.workunit.client.1.smithi130.stdout:3/236: rename d8/dc/d19/d2c/f3b to d8/dc/d42/f4a 0 2024-05-13T22:48:28.357 INFO:tasks.workunit.client.1.smithi130.stdout:5/254: dwrite d6/d7/f57 [0,4194304] 0 2024-05-13T22:48:28.357 INFO:tasks.workunit.client.0.smithi102.stdout:7/919: rename d6/d15/d4d/d55/d6f/d49/f100 to d6/d15/d4d/d55/d6f/d49/d97/f120 0 2024-05-13T22:48:28.358 INFO:tasks.workunit.client.1.smithi130.stdout:9/260: dwrite d1/d7/d23/d3d/d43/f4b [0,4194304] 0 2024-05-13T22:48:28.359 INFO:tasks.workunit.client.0.smithi102.stdout:0/980: mkdir d1/de/d32/d24/d2d/d38/d134 0 2024-05-13T22:48:28.365 INFO:tasks.workunit.client.0.smithi102.stdout:4/995: mkdir d3/d19/d23/d39/da7/da9/dd3/d157 0 2024-05-13T22:48:28.366 INFO:tasks.workunit.client.1.smithi130.stdout:0/250: dread - d0/d7/f16 zero size 2024-05-13T22:48:28.366 INFO:tasks.workunit.client.1.smithi130.stdout:9/261: truncate d1/f3 4752319 0 2024-05-13T22:48:28.366 INFO:tasks.workunit.client.1.smithi130.stdout:3/237: creat d8/f4b x:0 0 0 2024-05-13T22:48:28.369 INFO:tasks.workunit.client.1.smithi130.stdout:6/268: sync 2024-05-13T22:48:28.369 INFO:tasks.workunit.client.1.smithi130.stdout:7/207: sync 2024-05-13T22:48:28.370 INFO:tasks.workunit.client.1.smithi130.stdout:1/248: write db/d1b/d1f/d2a/f2f [2109036,55653] 0 2024-05-13T22:48:28.370 INFO:tasks.workunit.client.1.smithi130.stdout:4/270: truncate d8/dc/d35/d33/d44/f54 80879 0 2024-05-13T22:48:28.371 INFO:tasks.workunit.client.1.smithi130.stdout:3/238: read d8/dc/d15/f18 [2441697,40759] 0 2024-05-13T22:48:28.372 INFO:tasks.workunit.client.0.smithi102.stdout:9/955: dread d1/d6/d21/d71/f9d [0,4194304] 0 2024-05-13T22:48:28.373 INFO:tasks.workunit.client.1.smithi130.stdout:2/282: sync 2024-05-13T22:48:28.376 INFO:tasks.workunit.client.1.smithi130.stdout:9/262: mknod d1/d7/d23/d28/d2d/c52 0 2024-05-13T22:48:28.377 INFO:tasks.workunit.client.1.smithi130.stdout:0/251: dwrite d0/f42 [0,4194304] 0 2024-05-13T22:48:28.379 INFO:tasks.workunit.client.0.smithi102.stdout:7/920: creat d6/d13/d2c/f121 x:0 0 0 2024-05-13T22:48:28.379 INFO:tasks.workunit.client.1.smithi130.stdout:5/255: dread f4 [0,4194304] 0 2024-05-13T22:48:28.382 INFO:tasks.workunit.client.0.smithi102.stdout:9/956: rename d1/d6/d45/d3e/dbe/ld5 to d1/d6/d21/d12e/d75/db9/l14c 0 2024-05-13T22:48:28.386 INFO:tasks.workunit.client.1.smithi130.stdout:4/271: fdatasync d8/dc/d29/f3a 0 2024-05-13T22:48:28.388 INFO:tasks.workunit.client.0.smithi102.stdout:4/996: creat d3/d4/d6/de/d115/f158 x:0 0 0 2024-05-13T22:48:28.389 INFO:tasks.workunit.client.1.smithi130.stdout:1/249: write db/d1b/d1f/d2a/d35/f40 [1642664,19001] 0 2024-05-13T22:48:28.389 INFO:tasks.workunit.client.1.smithi130.stdout:1/250: stat db/f14 0 2024-05-13T22:48:28.390 INFO:tasks.workunit.client.1.smithi130.stdout:3/239: mknod d8/dc/d15/c4c 0 2024-05-13T22:48:28.392 INFO:tasks.workunit.client.1.smithi130.stdout:1/251: write db/d1b/d1f/d49/f48 [587656,44047] 0 2024-05-13T22:48:28.393 INFO:tasks.workunit.client.0.smithi102.stdout:9/957: symlink d1/d6/d45/d2d/l14d 0 2024-05-13T22:48:28.394 INFO:tasks.workunit.client.1.smithi130.stdout:0/252: write d0/d7/dd/d17/d1f/f25 [3695109,41303] 0 2024-05-13T22:48:28.395 INFO:tasks.workunit.client.1.smithi130.stdout:0/253: dread - d0/d7/dd/d2e/f4e zero size 2024-05-13T22:48:28.397 INFO:tasks.workunit.client.0.smithi102.stdout:4/997: rename d3/d19/d23/d4e/d68/dcf/cd1 to d3/d19/d23/d3a/da4/c159 0 2024-05-13T22:48:28.397 INFO:tasks.workunit.client.0.smithi102.stdout:4/998: chown d3/d22/dab/df9/d152 225230488 1 2024-05-13T22:48:28.398 INFO:tasks.workunit.client.1.smithi130.stdout:8/211: sync 2024-05-13T22:48:28.399 INFO:tasks.workunit.client.1.smithi130.stdout:8/212: chown d1/d2/db/f3b 603030613 1 2024-05-13T22:48:28.399 INFO:tasks.workunit.client.1.smithi130.stdout:9/263: rename d1/d7/d23/d3d/d43/f4b to d1/d7/d23/d28/d32/d45/f53 0 2024-05-13T22:48:28.401 INFO:tasks.workunit.client.0.smithi102.stdout:4/999: creat d3/d4/d5f/d12d/f15a x:0 0 0 2024-05-13T22:48:28.402 INFO:tasks.workunit.client.1.smithi130.stdout:7/208: mkdir d7/d8/d9/d4a 0 2024-05-13T22:48:28.403 INFO:tasks.workunit.client.1.smithi130.stdout:7/209: dread - d7/da/d1d/d42/f45 zero size 2024-05-13T22:48:28.406 INFO:tasks.workunit.client.1.smithi130.stdout:4/272: truncate d8/dc/f38 1464965 0 2024-05-13T22:48:28.406 INFO:tasks.workunit.client.1.smithi130.stdout:7/210: chown d7/da/d1d/l26 9793094 1 2024-05-13T22:48:28.406 INFO:tasks.workunit.client.1.smithi130.stdout:4/273: readlink d8/l2d 0 2024-05-13T22:48:28.407 INFO:tasks.workunit.client.1.smithi130.stdout:2/283: unlink d4/d49/d48/f4c 0 2024-05-13T22:48:28.407 INFO:tasks.workunit.client.1.smithi130.stdout:3/240: creat d8/dc/d15/f4d x:0 0 0 2024-05-13T22:48:28.408 INFO:tasks.workunit.client.0.smithi102.stdout:0/981: dwrite d1/de/d32/d24/f83 [0,4194304] 0 2024-05-13T22:48:28.409 INFO:tasks.workunit.client.0.smithi102.stdout:0/982: chown d1/de/d42/d61/ff0 44832450 1 2024-05-13T22:48:28.409 INFO:tasks.workunit.client.1.smithi130.stdout:0/254: mknod d0/d7/d29/c54 0 2024-05-13T22:48:28.416 INFO:tasks.workunit.client.1.smithi130.stdout:5/256: symlink d6/l58 0 2024-05-13T22:48:28.422 INFO:tasks.workunit.client.1.smithi130.stdout:7/211: rmdir d7/da/d1d/d42 39 2024-05-13T22:48:28.424 INFO:tasks.workunit.client.1.smithi130.stdout:2/284: chown d4/d49/d48/d4a/f52 140641125 1 2024-05-13T22:48:28.425 INFO:tasks.workunit.client.1.smithi130.stdout:3/241: mknod d8/d1a/d1c/c4e 0 2024-05-13T22:48:28.425 INFO:tasks.workunit.client.1.smithi130.stdout:4/274: dread d8/dc/d35/d33/d44/f54 [0,4194304] 0 2024-05-13T22:48:28.426 INFO:tasks.workunit.client.1.smithi130.stdout:3/242: fsync d8/f33 0 2024-05-13T22:48:28.426 INFO:tasks.workunit.client.1.smithi130.stdout:1/252: mknod db/c52 0 2024-05-13T22:48:28.427 INFO:tasks.workunit.client.1.smithi130.stdout:0/255: rmdir d0/d7 39 2024-05-13T22:48:28.428 INFO:tasks.workunit.client.1.smithi130.stdout:6/269: link d4/c4c d4/d15/d1f/d46/d52/c5b 0 2024-05-13T22:48:28.429 INFO:tasks.workunit.client.1.smithi130.stdout:6/270: dread - d4/d15/d1f/d46/f58 zero size 2024-05-13T22:48:28.429 INFO:tasks.workunit.client.1.smithi130.stdout:6/271: truncate d4/f37 32098 0 2024-05-13T22:48:28.430 INFO:tasks.workunit.client.1.smithi130.stdout:9/264: creat d1/d15/d51/f54 x:0 0 0 2024-05-13T22:48:28.433 INFO:tasks.workunit.client.1.smithi130.stdout:2/285: rename d4/db/f3a to d4/d56/f6d 0 2024-05-13T22:48:28.433 INFO:tasks.workunit.client.0.smithi102.stdout:0/983: write d1/d4d/f5b [1272186,39101] 0 2024-05-13T22:48:28.435 INFO:tasks.workunit.client.1.smithi130.stdout:4/275: symlink d8/dc/d35/d33/d44/l5d 0 2024-05-13T22:48:28.436 INFO:tasks.workunit.client.1.smithi130.stdout:1/253: creat db/d1b/d1f/f53 x:0 0 0 2024-05-13T22:48:28.437 INFO:tasks.workunit.client.1.smithi130.stdout:1/254: fdatasync db/d4a/f4b 0 2024-05-13T22:48:28.437 INFO:tasks.workunit.client.1.smithi130.stdout:5/257: dread d6/d7/f25 [0,4194304] 0 2024-05-13T22:48:28.438 INFO:tasks.workunit.client.1.smithi130.stdout:8/213: creat d1/d2/f48 x:0 0 0 2024-05-13T22:48:28.439 INFO:tasks.workunit.client.1.smithi130.stdout:6/272: mknod d4/d15/d1d/d28/d4e/c5c 0 2024-05-13T22:48:28.439 INFO:tasks.workunit.client.1.smithi130.stdout:0/256: write d0/d7/dd/d2e/f4e [25087,73232] 0 2024-05-13T22:48:28.441 INFO:tasks.workunit.client.1.smithi130.stdout:6/273: write d4/d15/d1f/d46/d50/f54 [648778,71094] 0 2024-05-13T22:48:28.442 INFO:tasks.workunit.client.1.smithi130.stdout:3/243: write d8/dc/f21 [609072,59562] 0 2024-05-13T22:48:28.443 INFO:tasks.workunit.client.1.smithi130.stdout:3/244: chown d8/dc/c41 18725 1 2024-05-13T22:48:28.444 INFO:tasks.workunit.client.0.smithi102.stdout:0/984: write d1/de/f27 [928120,34498] 0 2024-05-13T22:48:28.446 INFO:tasks.workunit.client.1.smithi130.stdout:8/214: symlink d1/d2/db/d15/d18/d42/l49 0 2024-05-13T22:48:28.447 INFO:tasks.workunit.client.1.smithi130.stdout:2/286: creat d4/db/dc/d18/d2a/d30/d69/f6e x:0 0 0 2024-05-13T22:48:28.451 INFO:tasks.workunit.client.0.smithi102.stdout:0/985: rename d1/de/d42/dea/d100 to d1/de/d32/d8a/d116/d11b/d135 0 2024-05-13T22:48:28.451 INFO:tasks.workunit.client.1.smithi130.stdout:2/287: dread d4/db/dc/d18/d2a/d30/f47 [0,4194304] 0 2024-05-13T22:48:28.454 INFO:tasks.workunit.client.1.smithi130.stdout:9/265: creat d1/d7/f55 x:0 0 0 2024-05-13T22:48:28.454 INFO:tasks.workunit.client.0.smithi102.stdout:0/986: symlink d1/de/d32/l136 0 2024-05-13T22:48:28.455 INFO:tasks.workunit.client.1.smithi130.stdout:8/215: creat d1/d2/db/f4a x:0 0 0 2024-05-13T22:48:28.457 INFO:tasks.workunit.client.1.smithi130.stdout:0/257: rename d0/d7/f28 to d0/d7/dd/d2e/d4b/f55 0 2024-05-13T22:48:28.458 INFO:tasks.workunit.client.1.smithi130.stdout:0/258: chown d0/d7/dd/d17/d1f 3 1 2024-05-13T22:48:28.458 INFO:tasks.workunit.client.0.smithi102.stdout:0/987: fsync d1/de/d32/f9c 0 2024-05-13T22:48:28.460 INFO:tasks.workunit.client.1.smithi130.stdout:6/274: mknod d4/d15/d1f/d46/d52/c5d 0 2024-05-13T22:48:28.463 INFO:tasks.workunit.client.1.smithi130.stdout:2/288: creat d4/db/dc/d18/d2a/d30/d69/f6f x:0 0 0 2024-05-13T22:48:28.463 INFO:tasks.workunit.client.0.smithi102.stdout:0/988: creat d1/de/d32/da6/f137 x:0 0 0 2024-05-13T22:48:28.464 INFO:tasks.workunit.client.1.smithi130.stdout:1/255: truncate db/f14 3479014 0 2024-05-13T22:48:28.465 INFO:tasks.workunit.client.1.smithi130.stdout:7/212: getdents d7/da/d1d/d42 0 2024-05-13T22:48:28.467 INFO:tasks.workunit.client.1.smithi130.stdout:8/216: unlink d1/d2/db/f3d 0 2024-05-13T22:48:28.468 INFO:tasks.workunit.client.0.smithi102.stdout:0/989: truncate d1/de/d42/d61/fb5 1312927 0 2024-05-13T22:48:28.468 INFO:tasks.workunit.client.1.smithi130.stdout:5/258: dwrite d6/d7/f3b [0,4194304] 0 2024-05-13T22:48:28.473 INFO:tasks.workunit.client.1.smithi130.stdout:1/256: dread db/d1b/d1f/d2a/d35/f3d [0,4194304] 0 2024-05-13T22:48:28.474 INFO:tasks.workunit.client.1.smithi130.stdout:1/257: chown db/d1b/d1f/d2a 128648661 1 2024-05-13T22:48:28.474 INFO:tasks.workunit.client.0.smithi102.stdout:0/990: rename d1/de/d32/d8d/ld0 to d1/d4d/d79/d9f/d104/l138 0 2024-05-13T22:48:28.475 INFO:tasks.workunit.client.0.smithi102.stdout:0/991: stat d1/d4d/d79 0 2024-05-13T22:48:28.475 INFO:tasks.workunit.client.0.smithi102.stdout:0/992: stat d1/de/d32/f9c 0 2024-05-13T22:48:28.475 INFO:tasks.workunit.client.1.smithi130.stdout:4/276: getdents d8/dc/d26 0 2024-05-13T22:48:28.477 INFO:tasks.workunit.client.1.smithi130.stdout:0/259: mknod d0/d7/dd/d17/d1f/d2c/d45/c56 0 2024-05-13T22:48:28.480 INFO:tasks.workunit.client.1.smithi130.stdout:2/289: dread d4/f5 [0,4194304] 0 2024-05-13T22:48:28.482 INFO:tasks.workunit.client.0.smithi102.stdout:0/993: creat d1/de/d32/d24/d2d/d38/d134/f139 x:0 0 0 2024-05-13T22:48:28.483 INFO:tasks.workunit.client.0.smithi102.stdout:0/994: dread - d1/de/d42/d61/f114 zero size 2024-05-13T22:48:28.487 INFO:tasks.workunit.client.0.smithi102.stdout:0/995: stat d1/de/d32/l43 0 2024-05-13T22:48:28.489 INFO:tasks.workunit.client.1.smithi130.stdout:9/266: truncate d1/f11 4238058 0 2024-05-13T22:48:28.490 INFO:tasks.workunit.client.1.smithi130.stdout:5/259: creat d6/d2f/f59 x:0 0 0 2024-05-13T22:48:28.492 INFO:tasks.workunit.client.1.smithi130.stdout:5/260: write d6/d7/d19/f52 [722015,73278] 0 2024-05-13T22:48:28.492 INFO:tasks.workunit.client.1.smithi130.stdout:1/258: creat db/d1b/d1f/d49/f54 x:0 0 0 2024-05-13T22:48:28.494 INFO:tasks.workunit.client.1.smithi130.stdout:0/260: write d0/f27 [4800016,92918] 0 2024-05-13T22:48:28.495 INFO:tasks.workunit.client.1.smithi130.stdout:7/213: creat d7/da/d1d/d3d/d46/f4b x:0 0 0 2024-05-13T22:48:28.497 INFO:tasks.workunit.client.1.smithi130.stdout:4/277: dread d8/dc/f5b [0,4194304] 0 2024-05-13T22:48:28.499 INFO:tasks.workunit.client.1.smithi130.stdout:3/245: rename d8/dc/d19/f1e to d8/d1a/f4f 0 2024-05-13T22:48:28.501 INFO:tasks.workunit.client.1.smithi130.stdout:5/261: symlink d6/d7/d24/l5a 0 2024-05-13T22:48:28.501 INFO:tasks.workunit.client.1.smithi130.stdout:5/262: write d6/d1a/f40 [4681914,68769] 0 2024-05-13T22:48:28.502 INFO:tasks.workunit.client.1.smithi130.stdout:1/259: write db/d1b/d1f/d2a/f4f [2254159,127314] 0 2024-05-13T22:48:28.504 INFO:tasks.workunit.client.1.smithi130.stdout:1/260: write db/d1b/d1f/f53 [984044,10256] 0 2024-05-13T22:48:28.504 INFO:tasks.workunit.client.1.smithi130.stdout:8/217: creat d1/d2/db/d15/d18/f4b x:0 0 0 2024-05-13T22:48:28.505 INFO:tasks.workunit.client.1.smithi130.stdout:0/261: mkdir d0/d7/dd/d17/d1f/d2c/d45/d57 0 2024-05-13T22:48:28.506 INFO:tasks.workunit.client.1.smithi130.stdout:4/278: truncate d8/dc/d35/d33/d44/f54 537129 0 2024-05-13T22:48:28.507 INFO:tasks.workunit.client.1.smithi130.stdout:4/279: write d8/fa [1611402,69325] 0 2024-05-13T22:48:28.507 INFO:tasks.workunit.client.1.smithi130.stdout:6/275: rename d4/d15/f3a to d4/d15/d1d/d28/f5e 0 2024-05-13T22:48:28.507 INFO:tasks.workunit.client.1.smithi130.stdout:4/280: fsync d8/df/f40 0 2024-05-13T22:48:28.509 INFO:tasks.workunit.client.1.smithi130.stdout:6/276: readlink d4/l14 0 2024-05-13T22:48:28.512 INFO:tasks.workunit.client.1.smithi130.stdout:0/262: chown d0/d7/fb 30743927 1 2024-05-13T22:48:28.513 INFO:tasks.workunit.client.1.smithi130.stdout:0/263: write d0/d7/dd/d2e/f3f [634585,71979] 0 2024-05-13T22:48:28.515 INFO:tasks.workunit.client.1.smithi130.stdout:5/263: rename d6/d7/f57 to d6/d7/d24/f5b 0 2024-05-13T22:48:28.515 INFO:tasks.workunit.client.1.smithi130.stdout:2/290: link d4/db/f2c d4/db/dc/d18/d58/f70 0 2024-05-13T22:48:28.516 INFO:tasks.workunit.client.1.smithi130.stdout:8/218: fsync d1/d2/db/d15/d18/f4b 0 2024-05-13T22:48:28.516 INFO:tasks.workunit.client.1.smithi130.stdout:2/291: write d4/f19 [42015,31116] 0 2024-05-13T22:48:28.516 INFO:tasks.workunit.client.1.smithi130.stdout:7/214: creat d7/d8/f4c x:0 0 0 2024-05-13T22:48:28.519 INFO:tasks.workunit.client.1.smithi130.stdout:3/246: symlink d8/dc/d19/d2c/l50 0 2024-05-13T22:48:28.522 INFO:tasks.workunit.client.1.smithi130.stdout:0/264: unlink d0/d7/dd/d17/l44 0 2024-05-13T22:48:28.522 INFO:tasks.workunit.client.1.smithi130.stdout:8/219: dwrite d1/d2/f3 [4194304,4194304] 0 2024-05-13T22:48:28.523 INFO:tasks.workunit.client.1.smithi130.stdout:5/264: fsync f3 0 2024-05-13T22:48:28.523 INFO:tasks.workunit.client.1.smithi130.stdout:8/220: dread - d1/d2/db/f3b zero size 2024-05-13T22:48:28.525 INFO:tasks.workunit.client.1.smithi130.stdout:4/281: rename d8/dc/d29/l34 to d8/df/l5e 0 2024-05-13T22:48:28.525 INFO:tasks.workunit.client.1.smithi130.stdout:6/277: symlink d4/l5f 0 2024-05-13T22:48:28.527 INFO:tasks.workunit.client.1.smithi130.stdout:5/265: dread d6/d7/d32/f51 [0,4194304] 0 2024-05-13T22:48:28.528 INFO:tasks.workunit.client.1.smithi130.stdout:7/215: unlink d7/da/d1d/d42/f45 0 2024-05-13T22:48:28.530 INFO:tasks.workunit.client.1.smithi130.stdout:3/247: dread - d8/dc/d19/d27/f28 zero size 2024-05-13T22:48:28.533 INFO:tasks.workunit.client.1.smithi130.stdout:1/261: dwrite db/d1b/d1f/d2a/d35/f3d [0,4194304] 0 2024-05-13T22:48:28.534 INFO:tasks.workunit.client.1.smithi130.stdout:5/266: fdatasync d6/d7/d24/f5b 0 2024-05-13T22:48:28.536 INFO:tasks.workunit.client.1.smithi130.stdout:0/265: creat d0/d7/dd/d17/d1f/d2c/d45/d57/f58 x:0 0 0 2024-05-13T22:48:28.536 INFO:tasks.workunit.client.1.smithi130.stdout:7/216: dread d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:28.537 INFO:tasks.workunit.client.1.smithi130.stdout:3/248: fdatasync d8/dc/d15/f18 0 2024-05-13T22:48:28.537 INFO:tasks.workunit.client.1.smithi130.stdout:7/217: fdatasync d7/d8/f4c 0 2024-05-13T22:48:28.538 INFO:tasks.workunit.client.1.smithi130.stdout:3/249: stat d8/dc/f12 0 2024-05-13T22:48:28.540 INFO:tasks.workunit.client.1.smithi130.stdout:2/292: rename d4/db/dc/f14 to d4/db/dc/d18/d2a/f71 0 2024-05-13T22:48:28.541 INFO:tasks.workunit.client.1.smithi130.stdout:1/262: creat db/d1b/d1f/d49/f55 x:0 0 0 2024-05-13T22:48:28.542 INFO:tasks.workunit.client.1.smithi130.stdout:5/267: unlink d6/l58 0 2024-05-13T22:48:28.543 INFO:tasks.workunit.client.1.smithi130.stdout:0/266: symlink d0/d7/dd/l59 0 2024-05-13T22:48:28.549 INFO:tasks.workunit.client.1.smithi130.stdout:8/221: creat d1/d2/db/d15/d18/f4c x:0 0 0 2024-05-13T22:48:28.554 INFO:tasks.workunit.client.1.smithi130.stdout:5/268: dread d6/d7/d19/d4d/f29 [0,4194304] 0 2024-05-13T22:48:28.554 INFO:tasks.workunit.client.1.smithi130.stdout:4/282: dwrite d8/f1a [4194304,4194304] 0 2024-05-13T22:48:28.555 INFO:tasks.workunit.client.1.smithi130.stdout:7/218: dread d7/da/f35 [0,4194304] 0 2024-05-13T22:48:28.555 INFO:tasks.workunit.client.1.smithi130.stdout:0/267: fsync d0/d7/dd/d17/d1f/d2c/d45/d57/f58 0 2024-05-13T22:48:28.555 INFO:tasks.workunit.client.1.smithi130.stdout:6/278: dread d4/d15/d1d/d28/f47 [0,4194304] 0 2024-05-13T22:48:28.556 INFO:tasks.workunit.client.1.smithi130.stdout:2/293: dwrite d4/db/dc/f43 [0,4194304] 0 2024-05-13T22:48:28.557 INFO:tasks.workunit.client.1.smithi130.stdout:1/263: rmdir db/d1b/d1f 39 2024-05-13T22:48:28.557 INFO:tasks.workunit.client.1.smithi130.stdout:4/283: dread - d8/df/f40 zero size 2024-05-13T22:48:28.557 INFO:tasks.workunit.client.1.smithi130.stdout:5/269: fsync d6/d1a/f40 0 2024-05-13T22:48:28.560 INFO:tasks.workunit.client.1.smithi130.stdout:8/222: truncate d1/d2/db/d15/d18/f4b 680261 0 2024-05-13T22:48:28.562 INFO:tasks.workunit.client.1.smithi130.stdout:8/223: write d1/dc/f44 [2809611,18605] 0 2024-05-13T22:48:28.563 INFO:tasks.workunit.client.1.smithi130.stdout:1/264: fdatasync db/d1b/d1f/d2a/d35/f3d 0 2024-05-13T22:48:28.564 INFO:tasks.workunit.client.1.smithi130.stdout:8/224: write d1/d2/db/d15/d18/f4c [4392,8407] 0 2024-05-13T22:48:28.570 INFO:tasks.workunit.client.1.smithi130.stdout:8/225: truncate d1/d2/f48 319101 0 2024-05-13T22:48:28.570 INFO:tasks.workunit.client.1.smithi130.stdout:0/268: chown d0/d7/dd/d2e 141196 1 2024-05-13T22:48:28.571 INFO:tasks.workunit.client.1.smithi130.stdout:1/265: mkdir db/d1b/d1f/d49/d56 0 2024-05-13T22:48:28.571 INFO:tasks.workunit.client.1.smithi130.stdout:2/294: read d4/f19 [64928,20351] 0 2024-05-13T22:48:28.571 INFO:tasks.workunit.client.1.smithi130.stdout:6/279: fdatasync d4/d15/d1f/f55 0 2024-05-13T22:48:28.571 INFO:tasks.workunit.client.1.smithi130.stdout:7/219: readlink d7/da/d1d/d42/d43/l47 0 2024-05-13T22:48:28.572 INFO:tasks.workunit.client.1.smithi130.stdout:3/250: dread d8/dc/f36 [4194304,4194304] 0 2024-05-13T22:48:28.578 INFO:tasks.workunit.client.1.smithi130.stdout:8/226: rename d1/d2/db/d15/f30 to d1/dc/f4d 0 2024-05-13T22:48:28.579 INFO:tasks.workunit.client.1.smithi130.stdout:2/295: mknod d4/d49/c72 0 2024-05-13T22:48:28.581 INFO:tasks.workunit.client.1.smithi130.stdout:2/296: truncate d4/d49/d48/f5d 461816 0 2024-05-13T22:48:28.582 INFO:tasks.workunit.client.1.smithi130.stdout:0/269: mkdir d0/d7/dd/d17/d1f/d2c/d5a 0 2024-05-13T22:48:28.584 INFO:tasks.workunit.client.1.smithi130.stdout:6/280: mkdir d4/d15/d1d/d60 0 2024-05-13T22:48:28.586 INFO:tasks.workunit.client.1.smithi130.stdout:1/266: symlink db/d1b/d1f/d2a/l57 0 2024-05-13T22:48:28.587 INFO:tasks.workunit.client.1.smithi130.stdout:1/267: truncate db/d1b/d1f/d49/f54 71936 0 2024-05-13T22:48:28.587 INFO:tasks.workunit.client.1.smithi130.stdout:3/251: symlink d8/d1a/d1c/l51 0 2024-05-13T22:48:28.588 INFO:tasks.workunit.client.1.smithi130.stdout:4/284: dread d8/df/f3c [0,4194304] 0 2024-05-13T22:48:28.588 INFO:tasks.workunit.client.1.smithi130.stdout:3/252: dread - d8/d35/f48 zero size 2024-05-13T22:48:28.596 INFO:tasks.workunit.client.1.smithi130.stdout:2/297: unlink d4/d49/d48/f5d 0 2024-05-13T22:48:28.598 INFO:tasks.workunit.client.1.smithi130.stdout:6/281: creat d4/d15/d1d/f61 x:0 0 0 2024-05-13T22:48:28.602 INFO:tasks.workunit.client.1.smithi130.stdout:3/253: creat d8/d35/f52 x:0 0 0 2024-05-13T22:48:28.603 INFO:tasks.workunit.client.1.smithi130.stdout:4/285: dread f6 [0,4194304] 0 2024-05-13T22:48:28.606 INFO:tasks.workunit.client.1.smithi130.stdout:6/282: mkdir d4/d16/d62 0 2024-05-13T22:48:28.611 INFO:tasks.workunit.client.1.smithi130.stdout:0/270: dread d0/f4 [0,4194304] 0 2024-05-13T22:48:28.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:28 smithi130 ceph-mon[36591]: pgmap v31: 65 pgs: 65 active+clean; 2.0 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 111 MiB/s wr, 326 op/s 2024-05-13T22:48:28.654 INFO:tasks.workunit.client.0.smithi102.stdout:1/883: sync 2024-05-13T22:48:28.655 INFO:tasks.workunit.client.1.smithi130.stdout:1/268: mkdir db/d58 0 2024-05-13T22:48:28.656 INFO:tasks.workunit.client.1.smithi130.stdout:1/269: write db/d1b/d1f/f53 [1630832,28014] 0 2024-05-13T22:48:28.657 INFO:tasks.workunit.client.1.smithi130.stdout:1/270: write db/d1b/d1f/d2a/d35/f3d [915490,39015] 0 2024-05-13T22:48:28.658 INFO:tasks.workunit.client.1.smithi130.stdout:8/227: creat d1/d2/db/d15/d18/f4e x:0 0 0 2024-05-13T22:48:28.659 INFO:tasks.workunit.client.1.smithi130.stdout:7/220: getdents d7/da/d1d/d42 0 2024-05-13T22:48:28.661 INFO:tasks.workunit.client.1.smithi130.stdout:3/254: symlink d8/dc/d42/l53 0 2024-05-13T22:48:28.664 INFO:tasks.workunit.client.1.smithi130.stdout:6/283: mkdir d4/d15/d1d/d63 0 2024-05-13T22:48:28.664 INFO:tasks.workunit.client.1.smithi130.stdout:7/221: dwrite d7/d8/f1c [0,4194304] 0 2024-05-13T22:48:28.665 INFO:tasks.workunit.client.1.smithi130.stdout:6/284: chown d4/d15/d1d/d28/f47 2234 1 2024-05-13T22:48:28.670 INFO:tasks.workunit.client.1.smithi130.stdout:1/271: mknod db/d1b/d1f/d49/c59 0 2024-05-13T22:48:28.676 INFO:tasks.workunit.client.1.smithi130.stdout:3/255: symlink d8/dc/d19/d2c/l54 0 2024-05-13T22:48:28.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:28 smithi102 ceph-mon[28599]: pgmap v31: 65 pgs: 65 active+clean; 2.0 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 111 MiB/s wr, 326 op/s 2024-05-13T22:48:28.677 INFO:tasks.workunit.client.1.smithi130.stdout:7/222: fdatasync d7/d8/f19 0 2024-05-13T22:48:28.678 INFO:tasks.workunit.client.1.smithi130.stdout:7/223: chown d7/l49 24050977 1 2024-05-13T22:48:28.679 INFO:tasks.workunit.client.1.smithi130.stdout:6/285: mkdir d4/d15/d1f/d64 0 2024-05-13T22:48:28.681 INFO:tasks.workunit.client.1.smithi130.stdout:0/271: mkdir d0/d7/dd/d17/d52/d5b 0 2024-05-13T22:48:28.682 INFO:tasks.workunit.client.1.smithi130.stdout:2/298: write d4/f5f [1185215,55788] 0 2024-05-13T22:48:28.683 INFO:tasks.workunit.client.1.smithi130.stdout:8/228: mknod d1/d2/c4f 0 2024-05-13T22:48:28.686 INFO:tasks.workunit.client.1.smithi130.stdout:6/286: dwrite d4/d15/d1d/f22 [4194304,4194304] 0 2024-05-13T22:48:28.688 INFO:tasks.workunit.client.1.smithi130.stdout:7/224: rename d7/d8/f13 to d7/da/d1d/d42/d43/f4d 0 2024-05-13T22:48:28.693 INFO:tasks.workunit.client.0.smithi102.stdout:1/884: rmdir d2/d4/d26/da2 39 2024-05-13T22:48:28.695 INFO:tasks.workunit.client.1.smithi130.stdout:0/272: symlink d0/d7/dd/d2e/l5c 0 2024-05-13T22:48:28.696 INFO:tasks.workunit.client.1.smithi130.stdout:0/273: write d0/d7/dd/d17/d1f/f25 [6429712,120403] 0 2024-05-13T22:48:28.698 INFO:tasks.workunit.client.1.smithi130.stdout:3/256: mkdir d8/d1a/d55 0 2024-05-13T22:48:28.700 INFO:tasks.workunit.client.1.smithi130.stdout:6/287: creat d4/d15/d1f/d46/f65 x:0 0 0 2024-05-13T22:48:28.701 INFO:tasks.workunit.client.1.smithi130.stdout:1/272: creat db/d1b/d1f/d2a/f5a x:0 0 0 2024-05-13T22:48:28.707 INFO:tasks.workunit.client.0.smithi102.stdout:1/885: link d2/d4/d38/db2/dba/cf5 d2/d10/c11d 0 2024-05-13T22:48:28.709 INFO:tasks.workunit.client.0.smithi102.stdout:1/886: chown d2/d4/c7d 36015 1 2024-05-13T22:48:28.711 INFO:tasks.workunit.client.0.smithi102.stdout:1/887: fdatasync d2/d10/d62/fa8 0 2024-05-13T22:48:28.714 INFO:tasks.workunit.client.0.smithi102.stdout:1/888: symlink d2/d10/l11e 0 2024-05-13T22:48:28.715 INFO:tasks.workunit.client.1.smithi130.stdout:0/274: symlink d0/d7/dd/d17/d1f/d2c/l5d 0 2024-05-13T22:48:28.716 INFO:tasks.workunit.client.1.smithi130.stdout:1/273: mkdir db/d1b/d1f/d2a/d4c/d5b 0 2024-05-13T22:48:28.717 INFO:tasks.workunit.client.0.smithi102.stdout:1/889: creat d2/d10/f11f x:0 0 0 2024-05-13T22:48:28.717 INFO:tasks.workunit.client.1.smithi130.stdout:1/274: chown db/d1b/d1f/d49 11 1 2024-05-13T22:48:28.717 INFO:tasks.workunit.client.1.smithi130.stdout:8/229: dread d1/dc/f4d [0,4194304] 0 2024-05-13T22:48:28.718 INFO:tasks.workunit.client.1.smithi130.stdout:6/288: creat d4/d15/d1d/d60/f66 x:0 0 0 2024-05-13T22:48:28.718 INFO:tasks.workunit.client.1.smithi130.stdout:8/230: fsync d1/dc/f3a 0 2024-05-13T22:48:28.720 INFO:tasks.workunit.client.0.smithi102.stdout:1/890: rename d2/d4/d5/d19/l6b to d2/d45/df4/l120 0 2024-05-13T22:48:28.721 INFO:tasks.workunit.client.1.smithi130.stdout:0/275: rename d0/d7/d29/c34 to d0/d7/d29/c5e 0 2024-05-13T22:48:28.723 INFO:tasks.workunit.client.1.smithi130.stdout:1/275: dwrite db/d4a/f4b [0,4194304] 0 2024-05-13T22:48:28.724 INFO:tasks.workunit.client.1.smithi130.stdout:1/276: chown db/d1b/f20 3 1 2024-05-13T22:48:28.724 INFO:tasks.workunit.client.1.smithi130.stdout:3/257: rename d8/l13 to d8/d1a/l56 0 2024-05-13T22:48:28.725 INFO:tasks.workunit.client.0.smithi102.stdout:1/891: write d2/d4/d5/d19/dc9/ff7 [679800,109198] 0 2024-05-13T22:48:28.726 INFO:tasks.workunit.client.1.smithi130.stdout:6/289: creat d4/d15/d1f/d46/d50/f67 x:0 0 0 2024-05-13T22:48:28.727 INFO:tasks.workunit.client.1.smithi130.stdout:6/290: read - d4/d15/d1f/f55 zero size 2024-05-13T22:48:28.728 INFO:tasks.workunit.client.1.smithi130.stdout:6/291: write d4/f2e [456122,62520] 0 2024-05-13T22:48:28.732 INFO:tasks.workunit.client.1.smithi130.stdout:0/276: fdatasync d0/d7/dd/f41 0 2024-05-13T22:48:28.732 INFO:tasks.workunit.client.1.smithi130.stdout:0/277: chown d0/d7/dd 1719011551 1 2024-05-13T22:48:28.736 INFO:tasks.workunit.client.0.smithi102.stdout:1/892: truncate d2/fc2 3900825 0 2024-05-13T22:48:28.738 INFO:tasks.workunit.client.1.smithi130.stdout:8/231: symlink d1/d2/db/d15/d18/l50 0 2024-05-13T22:48:28.738 INFO:tasks.workunit.client.1.smithi130.stdout:2/299: dwrite d4/db/dc/d18/d58/f70 [0,4194304] 0 2024-05-13T22:48:28.739 INFO:tasks.workunit.client.1.smithi130.stdout:2/300: fsync d4/db/dc/f43 0 2024-05-13T22:48:28.740 INFO:tasks.workunit.client.1.smithi130.stdout:2/301: chown d4/db/dc/d18/d2a/d30/f47 313 1 2024-05-13T22:48:28.744 INFO:tasks.workunit.client.1.smithi130.stdout:1/277: creat db/d1b/d1f/d2a/d38/f5c x:0 0 0 2024-05-13T22:48:28.745 INFO:tasks.workunit.client.1.smithi130.stdout:0/278: rmdir d0/d7/dd 39 2024-05-13T22:48:28.748 INFO:tasks.workunit.client.0.smithi102.stdout:1/893: creat d2/d45/f121 x:0 0 0 2024-05-13T22:48:28.752 INFO:tasks.workunit.client.1.smithi130.stdout:8/232: rename d1/dc/f21 to d1/d2/db/f51 0 2024-05-13T22:48:28.753 INFO:tasks.workunit.client.1.smithi130.stdout:8/233: chown d1/d2/db/d15/d18/d1a 209845 1 2024-05-13T22:48:28.755 INFO:tasks.workunit.client.0.smithi102.stdout:1/894: unlink l1 0 2024-05-13T22:48:28.756 INFO:tasks.workunit.client.1.smithi130.stdout:5/270: sync 2024-05-13T22:48:28.756 INFO:tasks.workunit.client.1.smithi130.stdout:4/286: sync 2024-05-13T22:48:28.756 INFO:tasks.workunit.client.1.smithi130.stdout:9/267: sync 2024-05-13T22:48:28.757 INFO:tasks.workunit.client.1.smithi130.stdout:1/278: mknod db/d1b/d1f/d2a/d4c/c5d 0 2024-05-13T22:48:28.758 INFO:tasks.workunit.client.1.smithi130.stdout:4/287: chown d8/dc/d35/d33/d44/c4a 6315 1 2024-05-13T22:48:28.760 INFO:tasks.workunit.client.0.smithi102.stdout:1/895: creat d2/d45/d96/d113/f122 x:0 0 0 2024-05-13T22:48:28.762 INFO:tasks.workunit.client.1.smithi130.stdout:5/271: dwrite d6/dd/f37 [0,4194304] 0 2024-05-13T22:48:28.765 INFO:tasks.workunit.client.1.smithi130.stdout:9/268: fdatasync d1/d7/f27 0 2024-05-13T22:48:28.766 INFO:tasks.workunit.client.1.smithi130.stdout:1/279: mkdir db/d1b/d1f/d5e 0 2024-05-13T22:48:28.768 INFO:tasks.workunit.client.1.smithi130.stdout:1/280: write db/d1b/d1f/d2a/d35/f3d [3732732,48224] 0 2024-05-13T22:48:28.768 INFO:tasks.workunit.client.1.smithi130.stdout:3/258: link d8/d1a/l39 d8/d1a/d55/l57 0 2024-05-13T22:48:28.770 INFO:tasks.workunit.client.1.smithi130.stdout:2/302: creat d4/d49/d48/f73 x:0 0 0 2024-05-13T22:48:28.773 INFO:tasks.workunit.client.1.smithi130.stdout:6/292: getdents d4/d15/d1f/d46/d52 0 2024-05-13T22:48:28.774 INFO:tasks.workunit.client.1.smithi130.stdout:6/293: chown d4/d15/d1d/d28/d4e 816076816 1 2024-05-13T22:48:28.774 INFO:tasks.workunit.client.0.smithi102.stdout:1/896: write d2/d10/d62/d7b/dc4/fcf [454564,49201] 0 2024-05-13T22:48:28.778 INFO:tasks.workunit.client.1.smithi130.stdout:9/269: rename d1/c1f to d1/d7/d23/c56 0 2024-05-13T22:48:28.779 INFO:tasks.workunit.client.1.smithi130.stdout:0/279: dread d0/d7/dd/d17/d1f/f25 [4194304,4194304] 0 2024-05-13T22:48:28.781 INFO:tasks.workunit.client.1.smithi130.stdout:4/288: fdatasync d8/dc/d35/d33/d44/f54 0 2024-05-13T22:48:28.781 INFO:tasks.workunit.client.1.smithi130.stdout:4/289: stat d8/df/c11 0 2024-05-13T22:48:28.782 INFO:tasks.workunit.client.1.smithi130.stdout:4/290: stat d8/l9 0 2024-05-13T22:48:28.782 INFO:tasks.workunit.client.1.smithi130.stdout:3/259: creat d8/dc/d15/f58 x:0 0 0 2024-05-13T22:48:28.783 INFO:tasks.workunit.client.1.smithi130.stdout:1/281: fdatasync db/d1b/d1f/d2a/f41 0 2024-05-13T22:48:28.784 INFO:tasks.workunit.client.0.smithi102.stdout:1/897: dwrite d2/d4/d26/d40/f72 [0,4194304] 0 2024-05-13T22:48:28.784 INFO:tasks.workunit.client.1.smithi130.stdout:2/303: fdatasync d4/d49/d48/d4a/f52 0 2024-05-13T22:48:28.785 INFO:tasks.workunit.client.1.smithi130.stdout:8/234: creat d1/d2/db/d15/d18/f52 x:0 0 0 2024-05-13T22:48:28.792 INFO:tasks.workunit.client.1.smithi130.stdout:1/282: creat db/d1b/d1f/d2a/d38/d46/f5f x:0 0 0 2024-05-13T22:48:28.793 INFO:tasks.workunit.client.1.smithi130.stdout:6/294: symlink d4/d15/d1f/d64/l68 0 2024-05-13T22:48:28.795 INFO:tasks.workunit.client.1.smithi130.stdout:2/304: dwrite d4/d49/f35 [0,4194304] 0 2024-05-13T22:48:28.795 INFO:tasks.workunit.client.1.smithi130.stdout:0/280: mkdir d0/d7/dd/d17/d52/d5b/d5f 0 2024-05-13T22:48:28.796 INFO:tasks.workunit.client.1.smithi130.stdout:3/260: symlink d8/dc/d19/l59 0 2024-05-13T22:48:28.800 INFO:tasks.workunit.client.1.smithi130.stdout:9/270: creat d1/d7/f57 x:0 0 0 2024-05-13T22:48:28.800 INFO:tasks.workunit.client.1.smithi130.stdout:0/281: stat d0/d7/dd/d17/d52/d5b 0 2024-05-13T22:48:28.800 INFO:tasks.workunit.client.1.smithi130.stdout:8/235: dwrite d1/d2/db/d15/d18/f31 [0,4194304] 0 2024-05-13T22:48:28.804 INFO:tasks.workunit.client.1.smithi130.stdout:6/295: truncate d4/d15/d1f/f2f 350124 0 2024-05-13T22:48:28.805 INFO:tasks.workunit.client.1.smithi130.stdout:2/305: dread d4/d49/f35 [0,4194304] 0 2024-05-13T22:48:28.806 INFO:tasks.workunit.client.1.smithi130.stdout:0/282: mkdir d0/d7/dd/d17/d52/d5b/d60 0 2024-05-13T22:48:28.807 INFO:tasks.workunit.client.1.smithi130.stdout:3/261: symlink d8/d1a/d55/l5a 0 2024-05-13T22:48:28.807 INFO:tasks.workunit.client.1.smithi130.stdout:8/236: mkdir d1/d2/db/d15/d18/d42/d53 0 2024-05-13T22:48:28.809 INFO:tasks.workunit.client.1.smithi130.stdout:9/271: dread d1/d7/f1a [0,4194304] 0 2024-05-13T22:48:28.809 INFO:tasks.workunit.client.1.smithi130.stdout:2/306: write d4/db/dc/d18/f1d [2902147,125485] 0 2024-05-13T22:48:28.809 INFO:tasks.workunit.client.1.smithi130.stdout:0/283: write d0/d7/d29/f37 [416620,23430] 0 2024-05-13T22:48:28.809 INFO:tasks.workunit.client.1.smithi130.stdout:1/283: link db/d1b/d1f/d2a/d38/f5c db/d1b/d1f/d2a/d38/f60 0 2024-05-13T22:48:28.812 INFO:tasks.workunit.client.1.smithi130.stdout:3/262: creat d8/f5b x:0 0 0 2024-05-13T22:48:28.815 INFO:tasks.workunit.client.1.smithi130.stdout:1/284: dread - db/d1b/d1f/d49/f55 zero size 2024-05-13T22:48:28.815 INFO:tasks.workunit.client.1.smithi130.stdout:2/307: readlink d4/db/dc/d18/d2a/d30/l38 0 2024-05-13T22:48:28.816 INFO:tasks.workunit.client.1.smithi130.stdout:8/237: write d1/dc/f24 [3818114,18856] 0 2024-05-13T22:48:28.817 INFO:tasks.workunit.client.1.smithi130.stdout:6/296: dwrite d4/d15/d1f/f20 [0,4194304] 0 2024-05-13T22:48:28.820 INFO:tasks.workunit.client.1.smithi130.stdout:3/263: symlink d8/dc/d15/l5c 0 2024-05-13T22:48:28.820 INFO:tasks.workunit.client.1.smithi130.stdout:2/308: chown d4/d49/c72 1343 1 2024-05-13T22:48:28.822 INFO:tasks.workunit.client.1.smithi130.stdout:0/284: read d0/d7/dd/d17/d1f/d2c/f30 [2187998,111201] 0 2024-05-13T22:48:28.823 INFO:tasks.workunit.client.1.smithi130.stdout:8/238: unlink d1/dc/f24 0 2024-05-13T22:48:28.830 INFO:tasks.workunit.client.1.smithi130.stdout:8/239: truncate d1/d2/db/d15/d16/f2c 569673 0 2024-05-13T22:48:28.831 INFO:tasks.workunit.client.1.smithi130.stdout:6/297: dwrite d4/d15/d1f/f55 [0,4194304] 0 2024-05-13T22:48:28.832 INFO:tasks.workunit.client.1.smithi130.stdout:2/309: mknod d4/db/dc/c74 0 2024-05-13T22:48:28.833 INFO:tasks.workunit.client.1.smithi130.stdout:1/285: dwrite db/d1b/d1f/d2a/d35/f4d [0,4194304] 0 2024-05-13T22:48:28.833 INFO:tasks.workunit.client.1.smithi130.stdout:3/264: rename d8/f5b to d8/d1a/d1c/f5d 0 2024-05-13T22:48:28.833 INFO:tasks.workunit.client.1.smithi130.stdout:9/272: link d1/d7/c9 d1/d7/d23/d28/d2d/c58 0 2024-05-13T22:48:28.834 INFO:tasks.workunit.client.1.smithi130.stdout:0/285: truncate d0/d7/f23 356592 0 2024-05-13T22:48:28.834 INFO:tasks.workunit.client.1.smithi130.stdout:3/265: stat d8/d35/f52 0 2024-05-13T22:48:28.835 INFO:tasks.workunit.client.1.smithi130.stdout:6/298: creat d4/d16/f69 x:0 0 0 2024-05-13T22:48:28.837 INFO:tasks.workunit.client.1.smithi130.stdout:9/273: write d1/db/f35 [1063639,38509] 0 2024-05-13T22:48:28.841 INFO:tasks.workunit.client.1.smithi130.stdout:3/266: creat d8/f5e x:0 0 0 2024-05-13T22:48:28.845 INFO:tasks.workunit.client.1.smithi130.stdout:8/240: dwrite d1/d2/db/d15/d18/f4e [0,4194304] 0 2024-05-13T22:48:28.849 INFO:tasks.workunit.client.1.smithi130.stdout:3/267: fsync d8/dc/d19/f30 0 2024-05-13T22:48:28.851 INFO:tasks.workunit.client.1.smithi130.stdout:9/274: dwrite d1/d7/f55 [0,4194304] 0 2024-05-13T22:48:28.852 INFO:tasks.workunit.client.1.smithi130.stdout:6/299: rename d4/c4a to d4/d15/d1d/d28/c6a 0 2024-05-13T22:48:28.852 INFO:tasks.workunit.client.1.smithi130.stdout:9/275: stat d1/d7/f57 0 2024-05-13T22:48:28.852 INFO:tasks.workunit.client.1.smithi130.stdout:3/268: chown d8/dc/d19/d2c/f46 906841689 1 2024-05-13T22:48:28.854 INFO:tasks.workunit.client.1.smithi130.stdout:8/241: mkdir d1/d2/db/d15/d16/d54 0 2024-05-13T22:48:28.856 INFO:tasks.workunit.client.1.smithi130.stdout:3/269: write d8/dc/f22 [406943,54051] 0 2024-05-13T22:48:28.856 INFO:tasks.workunit.client.1.smithi130.stdout:9/276: write d1/d7/d23/d28/d2d/f3a [239033,29313] 0 2024-05-13T22:48:28.856 INFO:tasks.workunit.client.1.smithi130.stdout:9/277: readlink d1/l14 0 2024-05-13T22:48:28.857 INFO:tasks.workunit.client.1.smithi130.stdout:0/286: rename d0/d7/dd/f47 to d0/d7/dd/d17/d1f/d2c/f61 0 2024-05-13T22:48:28.858 INFO:tasks.workunit.client.1.smithi130.stdout:3/270: write d8/dc/f12 [3261413,92609] 0 2024-05-13T22:48:28.859 INFO:tasks.workunit.client.1.smithi130.stdout:9/278: dread - d1/d7/d23/d28/d2d/f42 zero size 2024-05-13T22:48:28.861 INFO:tasks.workunit.client.1.smithi130.stdout:6/300: symlink d4/d15/d1d/l6b 0 2024-05-13T22:48:28.861 INFO:tasks.workunit.client.1.smithi130.stdout:9/279: read d1/db/fc [3670846,82622] 0 2024-05-13T22:48:28.863 INFO:tasks.workunit.client.1.smithi130.stdout:7/225: sync 2024-05-13T22:48:28.865 INFO:tasks.workunit.client.1.smithi130.stdout:6/301: write d4/d16/f69 [174006,111780] 0 2024-05-13T22:48:28.866 INFO:tasks.workunit.client.1.smithi130.stdout:9/280: write d1/d7/f57 [530251,4184] 0 2024-05-13T22:48:28.867 INFO:tasks.workunit.client.1.smithi130.stdout:3/271: rmdir d8/dc/d42 39 2024-05-13T22:48:28.868 INFO:tasks.workunit.client.1.smithi130.stdout:7/226: stat d7/da/d1f/f20 0 2024-05-13T22:48:28.870 INFO:tasks.workunit.client.1.smithi130.stdout:6/302: readlink d4/d15/d1d/l1e 0 2024-05-13T22:48:28.871 INFO:tasks.workunit.client.1.smithi130.stdout:0/287: rename d0/d7/dd/d17/d1f/d2c/f31 to d0/d7/dd/d17/d1f/d2c/d45/d57/f62 0 2024-05-13T22:48:28.872 INFO:tasks.workunit.client.1.smithi130.stdout:2/310: dwrite d4/f1b [0,4194304] 0 2024-05-13T22:48:28.872 INFO:tasks.workunit.client.1.smithi130.stdout:3/272: truncate d8/f4b 517913 0 2024-05-13T22:48:28.873 INFO:tasks.workunit.client.1.smithi130.stdout:8/242: unlink d1/dc/f1f 0 2024-05-13T22:48:28.874 INFO:tasks.workunit.client.1.smithi130.stdout:7/227: truncate d7/d8/f4c 466573 0 2024-05-13T22:48:28.874 INFO:tasks.workunit.client.1.smithi130.stdout:0/288: read - d0/f53 zero size 2024-05-13T22:48:28.875 INFO:tasks.workunit.client.1.smithi130.stdout:9/281: chown d1/d7/d23/d28/d32/c49 0 1 2024-05-13T22:48:28.875 INFO:tasks.workunit.client.1.smithi130.stdout:6/303: write d4/d15/d1d/f22 [3093984,6594] 0 2024-05-13T22:48:28.875 INFO:tasks.workunit.client.1.smithi130.stdout:1/286: dread db/d1b/d1f/d2a/d35/f40 [0,4194304] 0 2024-05-13T22:48:28.880 INFO:tasks.workunit.client.1.smithi130.stdout:7/228: dread d7/d8/f4c [0,4194304] 0 2024-05-13T22:48:28.882 INFO:tasks.workunit.client.1.smithi130.stdout:3/273: rename d8/d1a/d1c/l43 to d8/l5f 0 2024-05-13T22:48:28.882 INFO:tasks.workunit.client.1.smithi130.stdout:6/304: mknod d4/d16/c6c 0 2024-05-13T22:48:28.883 INFO:tasks.workunit.client.1.smithi130.stdout:1/287: rmdir db/d1b/d1f/d2a/d38 39 2024-05-13T22:48:28.885 INFO:tasks.workunit.client.1.smithi130.stdout:0/289: mknod d0/d7/dd/d17/d1f/d2c/c63 0 2024-05-13T22:48:28.886 INFO:tasks.workunit.client.1.smithi130.stdout:7/229: read d7/da/d1d/d42/d43/f4d [4346734,127741] 0 2024-05-13T22:48:28.886 INFO:tasks.workunit.client.1.smithi130.stdout:2/311: dwrite d4/d56/f61 [0,4194304] 0 2024-05-13T22:48:28.890 INFO:tasks.workunit.client.1.smithi130.stdout:6/305: mknod d4/d15/d1f/c6d 0 2024-05-13T22:48:28.890 INFO:tasks.workunit.client.1.smithi130.stdout:3/274: truncate d8/dc/d15/f18 899780 0 2024-05-13T22:48:28.893 INFO:tasks.workunit.client.1.smithi130.stdout:8/243: link d1/d2/db/f40 d1/d2/db/d15/d16/d54/f55 0 2024-05-13T22:48:28.894 INFO:tasks.workunit.client.1.smithi130.stdout:0/290: symlink d0/d7/d29/l64 0 2024-05-13T22:48:28.896 INFO:tasks.workunit.client.1.smithi130.stdout:1/288: rename db/f22 to db/d1b/d1f/d49/f61 0 2024-05-13T22:48:28.898 INFO:tasks.workunit.client.1.smithi130.stdout:0/291: dread d0/f27 [4194304,4194304] 0 2024-05-13T22:48:28.898 INFO:tasks.workunit.client.1.smithi130.stdout:2/312: mknod d4/d49/c75 0 2024-05-13T22:48:28.899 INFO:tasks.workunit.client.1.smithi130.stdout:7/230: unlink d7/d8/d9/c2e 0 2024-05-13T22:48:28.905 INFO:tasks.workunit.client.1.smithi130.stdout:9/282: link d1/d7/d23/d28/c41 d1/d7/d23/d28/d32/c59 0 2024-05-13T22:48:28.908 INFO:tasks.workunit.client.0.smithi102.stdout:7/921: sync 2024-05-13T22:48:28.908 INFO:tasks.workunit.client.1.smithi130.stdout:0/292: creat d0/d7/dd/d2e/d4b/f65 x:0 0 0 2024-05-13T22:48:28.911 INFO:tasks.workunit.client.1.smithi130.stdout:5/272: sync 2024-05-13T22:48:28.911 INFO:tasks.workunit.client.1.smithi130.stdout:5/273: chown d6/d7/d19/f1c 120 1 2024-05-13T22:48:28.912 INFO:tasks.workunit.client.1.smithi130.stdout:5/274: read - d6/d7/d24/f41 zero size 2024-05-13T22:48:28.913 INFO:tasks.workunit.client.0.smithi102.stdout:7/922: mkdir d6/d22/de5/d122 0 2024-05-13T22:48:28.915 INFO:tasks.workunit.client.0.smithi102.stdout:7/923: creat d6/d22/dd2/f123 x:0 0 0 2024-05-13T22:48:28.915 INFO:tasks.workunit.client.1.smithi130.stdout:6/306: write d4/d15/d1f/d46/d50/f59 [516693,5619] 0 2024-05-13T22:48:28.916 INFO:tasks.workunit.client.1.smithi130.stdout:6/307: readlink d4/d16/l36 0 2024-05-13T22:48:28.916 INFO:tasks.workunit.client.1.smithi130.stdout:8/244: mkdir d1/d2/db/d15/d18/d42/d53/d56 0 2024-05-13T22:48:28.916 INFO:tasks.workunit.client.0.smithi102.stdout:7/924: write d6/d13/d58/fac [391466,29870] 0 2024-05-13T22:48:28.919 INFO:tasks.workunit.client.1.smithi130.stdout:2/313: write d4/f7 [6232310,7925] 0 2024-05-13T22:48:28.920 INFO:tasks.workunit.client.1.smithi130.stdout:2/314: write d4/db/dc/f4e [255674,43236] 0 2024-05-13T22:48:28.922 INFO:tasks.workunit.client.1.smithi130.stdout:3/275: truncate d8/dc/f12 3611028 0 2024-05-13T22:48:28.924 INFO:tasks.workunit.client.1.smithi130.stdout:3/276: write d8/f5e [280350,35300] 0 2024-05-13T22:48:28.924 INFO:tasks.workunit.client.1.smithi130.stdout:7/231: mknod d7/da/d1d/c4e 0 2024-05-13T22:48:28.924 INFO:tasks.workunit.client.1.smithi130.stdout:3/277: chown d8/dc/c41 2 1 2024-05-13T22:48:28.925 INFO:tasks.workunit.client.1.smithi130.stdout:5/275: symlink d6/d7/d19/d4d/l5c 0 2024-05-13T22:48:28.926 INFO:tasks.workunit.client.1.smithi130.stdout:7/232: dread d7/da/f33 [0,4194304] 0 2024-05-13T22:48:28.926 INFO:tasks.workunit.client.1.smithi130.stdout:7/233: write d7/da/d1f/f20 [5107263,39374] 0 2024-05-13T22:48:28.927 INFO:tasks.workunit.client.1.smithi130.stdout:8/245: symlink d1/d2/db/d15/d18/d42/l57 0 2024-05-13T22:48:28.928 INFO:tasks.workunit.client.0.smithi102.stdout:7/925: dread d6/dd0/d80/fa5 [0,4194304] 0 2024-05-13T22:48:28.928 INFO:tasks.workunit.client.1.smithi130.stdout:1/289: creat db/d1b/f62 x:0 0 0 2024-05-13T22:48:28.929 INFO:tasks.workunit.client.0.smithi102.stdout:7/926: truncate d6/d22/dd2/f123 107777 0 2024-05-13T22:48:28.930 INFO:tasks.workunit.client.1.smithi130.stdout:2/315: write d4/f5 [4793221,71279] 0 2024-05-13T22:48:28.932 INFO:tasks.workunit.client.1.smithi130.stdout:0/293: creat d0/d7/dd/d17/d1f/d2c/d5a/f66 x:0 0 0 2024-05-13T22:48:28.933 INFO:tasks.workunit.client.1.smithi130.stdout:0/294: fsync d0/d7/dd/f46 0 2024-05-13T22:48:28.937 INFO:tasks.workunit.client.0.smithi102.stdout:7/927: rename d6/d13/cc4 to d6/d15/d4d/d55/d6f/c124 0 2024-05-13T22:48:28.937 INFO:tasks.workunit.client.1.smithi130.stdout:2/316: dwrite d4/db/dc/d18/f23 [0,4194304] 0 2024-05-13T22:48:28.939 INFO:tasks.workunit.client.1.smithi130.stdout:9/283: dwrite d1/f1e [0,4194304] 0 2024-05-13T22:48:28.939 INFO:tasks.workunit.client.1.smithi130.stdout:6/308: rmdir d4/d16/d62 0 2024-05-13T22:48:28.940 INFO:tasks.workunit.client.1.smithi130.stdout:2/317: chown d4/d49/c72 224 1 2024-05-13T22:48:28.941 INFO:tasks.workunit.client.1.smithi130.stdout:8/246: mkdir d1/d2/db/d15/d16/d54/d58 0 2024-05-13T22:48:28.942 INFO:tasks.workunit.client.1.smithi130.stdout:8/247: chown d1/d2/db/d15/d18/d1a 37560002 1 2024-05-13T22:48:28.943 INFO:tasks.workunit.client.1.smithi130.stdout:0/295: unlink d0/d7/d29/d3c/l49 0 2024-05-13T22:48:28.948 INFO:tasks.workunit.client.1.smithi130.stdout:7/234: link d7/da/d1d/d42/d43/f4d d7/da/d1d/d3d/d46/f4f 0 2024-05-13T22:48:28.948 INFO:tasks.workunit.client.1.smithi130.stdout:9/284: dwrite d1/d15/f25 [0,4194304] 0 2024-05-13T22:48:28.949 INFO:tasks.workunit.client.1.smithi130.stdout:1/290: mknod db/d1b/d1f/d5e/c63 0 2024-05-13T22:48:28.950 INFO:tasks.workunit.client.1.smithi130.stdout:1/291: readlink db/d1b/d1f/d2a/d4c/l51 0 2024-05-13T22:48:28.951 INFO:tasks.workunit.client.0.smithi102.stdout:0/996: sync 2024-05-13T22:48:28.951 INFO:tasks.workunit.client.0.smithi102.stdout:9/958: sync 2024-05-13T22:48:28.953 INFO:tasks.workunit.client.0.smithi102.stdout:7/928: getdents d6/d15/d4d 0 2024-05-13T22:48:28.955 INFO:tasks.workunit.client.1.smithi130.stdout:2/318: mkdir d4/d49/d48/d4a/d76 0 2024-05-13T22:48:28.956 INFO:tasks.workunit.client.0.smithi102.stdout:9/959: fdatasync d1/d6/d21/d12e/d75/d96/d102/f111 0 2024-05-13T22:48:28.957 INFO:tasks.workunit.client.0.smithi102.stdout:9/960: stat d1/d6/d45/d3e/dbe/c109 0 2024-05-13T22:48:28.958 INFO:tasks.workunit.client.1.smithi130.stdout:0/296: mkdir d0/d7/dd/d17/d52/d67 0 2024-05-13T22:48:28.959 INFO:tasks.workunit.client.0.smithi102.stdout:0/997: creat d1/de/d32/d24/d2d/d38/d3d/d81/f13a x:0 0 0 2024-05-13T22:48:28.961 INFO:tasks.workunit.client.0.smithi102.stdout:7/929: mkdir d6/da9/d125 0 2024-05-13T22:48:28.961 INFO:tasks.workunit.client.0.smithi102.stdout:7/930: read - d6/dd0/df2/d105/f107 zero size 2024-05-13T22:48:28.964 INFO:tasks.workunit.client.0.smithi102.stdout:9/961: rename d1/d6/d3b to d1/d6/d21/d12e/d75/d96/dc6/dd9/d125/d14e 0 2024-05-13T22:48:28.966 INFO:tasks.workunit.client.1.smithi130.stdout:1/292: creat db/d1b/d1f/d49/f64 x:0 0 0 2024-05-13T22:48:28.967 INFO:tasks.workunit.client.0.smithi102.stdout:0/998: creat d1/de/d32/d8a/d116/f13b x:0 0 0 2024-05-13T22:48:28.969 INFO:tasks.workunit.client.1.smithi130.stdout:2/319: rename d4/f21 to d4/db/dc/d18/d2a/d30/d69/f77 0 2024-05-13T22:48:28.971 INFO:tasks.workunit.client.1.smithi130.stdout:2/320: write d4/db/f41 [864637,45107] 0 2024-05-13T22:48:28.972 INFO:tasks.workunit.client.1.smithi130.stdout:1/293: dwrite db/d1b/d1f/d49/f55 [0,4194304] 0 2024-05-13T22:48:28.972 INFO:tasks.workunit.client.1.smithi130.stdout:8/248: symlink d1/d2/db/d15/d16/d54/d58/l59 0 2024-05-13T22:48:28.974 INFO:tasks.workunit.client.1.smithi130.stdout:2/321: write d4/db/dc/d18/f23 [3112878,111814] 0 2024-05-13T22:48:28.975 INFO:tasks.workunit.client.1.smithi130.stdout:8/249: rename d1/d2/db/d15 to d1/d2/db/d15/d18/d1a/d5a 22 2024-05-13T22:48:28.978 INFO:tasks.workunit.client.0.smithi102.stdout:0/999: truncate d1/de/d42/d61/ff0 150580 0 2024-05-13T22:48:28.980 INFO:tasks.workunit.client.0.smithi102.stdout:9/962: creat d1/d6/d45/d3e/d13a/f14f x:0 0 0 2024-05-13T22:48:28.982 INFO:tasks.workunit.client.1.smithi130.stdout:0/297: mkdir d0/d7/dd/d17/d52/d67/d68 0 2024-05-13T22:48:28.984 INFO:tasks.workunit.client.0.smithi102.stdout:9/963: rename d1/c22 to d1/d6/d45/d147/c150 0 2024-05-13T22:48:28.987 INFO:tasks.workunit.client.1.smithi130.stdout:7/235: dwrite d7/da/f35 [0,4194304] 0 2024-05-13T22:48:28.987 INFO:tasks.workunit.client.1.smithi130.stdout:2/322: mknod d4/d49/c78 0 2024-05-13T22:48:28.988 INFO:tasks.workunit.client.0.smithi102.stdout:9/964: dread - d1/d6/d21/d12e/d75/fe3 zero size 2024-05-13T22:48:28.988 INFO:tasks.workunit.client.1.smithi130.stdout:2/323: fsync d4/db/dc/d18/d2a/f67 0 2024-05-13T22:48:28.989 INFO:tasks.workunit.client.0.smithi102.stdout:7/931: write d6/d22/f24 [3047755,123544] 0 2024-05-13T22:48:28.990 INFO:tasks.workunit.client.1.smithi130.stdout:8/250: creat d1/d2/db/d15/d16/f5b x:0 0 0 2024-05-13T22:48:28.994 INFO:tasks.workunit.client.1.smithi130.stdout:0/298: mkdir d0/d7/dd/d69 0 2024-05-13T22:48:28.995 INFO:tasks.workunit.client.1.smithi130.stdout:9/285: write d1/f11 [2314438,29023] 0 2024-05-13T22:48:28.996 INFO:tasks.workunit.client.1.smithi130.stdout:8/251: dread d1/d2/db/d15/d18/f4e [0,4194304] 0 2024-05-13T22:48:29.001 INFO:tasks.workunit.client.1.smithi130.stdout:7/236: dread d7/d8/f4c [0,4194304] 0 2024-05-13T22:48:29.003 INFO:tasks.workunit.client.1.smithi130.stdout:0/299: write d0/d7/dd/d17/d1f/d2c/f30 [2440202,48578] 0 2024-05-13T22:48:29.004 INFO:tasks.workunit.client.0.smithi102.stdout:7/932: creat d6/d15/d4d/d55/d6f/f126 x:0 0 0 2024-05-13T22:48:29.006 INFO:tasks.workunit.client.1.smithi130.stdout:4/291: sync 2024-05-13T22:48:29.007 INFO:tasks.workunit.client.1.smithi130.stdout:1/294: dread db/f32 [0,4194304] 0 2024-05-13T22:48:29.008 INFO:tasks.workunit.client.0.smithi102.stdout:9/965: link d1/d6/d45/d2d/dac/f100 d1/d6/d45/d147/f151 0 2024-05-13T22:48:29.010 INFO:tasks.workunit.client.1.smithi130.stdout:9/286: rename d1/d7/d23/d28/d2d/d2f/f3c to d1/d7/d23/d3d/f5a 0 2024-05-13T22:48:29.012 INFO:tasks.workunit.client.0.smithi102.stdout:7/933: unlink d6/de1/de6/f113 0 2024-05-13T22:48:29.013 INFO:tasks.workunit.client.1.smithi130.stdout:2/324: write d4/d49/d48/d4a/f52 [1098343,33724] 0 2024-05-13T22:48:29.014 INFO:tasks.workunit.client.0.smithi102.stdout:9/966: rename d1/d6/d21/d12e/d75/d96/d102 to d1/d6/d21/d12e/d64/d152 0 2024-05-13T22:48:29.017 INFO:tasks.workunit.client.1.smithi130.stdout:7/237: dread d7/da/f33 [0,4194304] 0 2024-05-13T22:48:29.017 INFO:tasks.workunit.client.0.smithi102.stdout:1/898: sync 2024-05-13T22:48:29.018 INFO:tasks.workunit.client.1.smithi130.stdout:0/300: creat d0/d7/dd/d17/d52/d67/f6a x:0 0 0 2024-05-13T22:48:29.019 INFO:tasks.workunit.client.0.smithi102.stdout:1/899: read d2/d4/d5/d19/ded/fd1 [1726000,17591] 0 2024-05-13T22:48:29.019 INFO:tasks.workunit.client.1.smithi130.stdout:0/301: dread - d0/d7/dd/d17/d1f/d2c/d5a/f66 zero size 2024-05-13T22:48:29.022 INFO:tasks.workunit.client.1.smithi130.stdout:0/302: dread d0/d7/dd/f32 [0,4194304] 0 2024-05-13T22:48:29.022 INFO:tasks.workunit.client.0.smithi102.stdout:9/967: dread d1/d6/d21/d12e/d64/d152/d6c/f86 [0,4194304] 0 2024-05-13T22:48:29.022 INFO:tasks.workunit.client.1.smithi130.stdout:4/292: rmdir d8/dc 39 2024-05-13T22:48:29.031 INFO:tasks.workunit.client.1.smithi130.stdout:9/287: dread d1/f8 [0,4194304] 0 2024-05-13T22:48:29.034 INFO:tasks.workunit.client.1.smithi130.stdout:9/288: dread d1/d15/f25 [0,4194304] 0 2024-05-13T22:48:29.037 INFO:tasks.workunit.client.0.smithi102.stdout:9/968: dread d1/d6/f60 [4194304,4194304] 0 2024-05-13T22:48:29.038 INFO:tasks.workunit.client.1.smithi130.stdout:0/303: mkdir d0/d7/dd/d17/d1f/d2c/d45/d6b 0 2024-05-13T22:48:29.039 INFO:tasks.workunit.client.0.smithi102.stdout:7/934: getdents d6/d13/d2c/d39/db3 0 2024-05-13T22:48:29.039 INFO:tasks.workunit.client.0.smithi102.stdout:1/900: write d2/d45/d96/fd0 [750181,6128] 0 2024-05-13T22:48:29.041 INFO:tasks.workunit.client.0.smithi102.stdout:9/969: chown d1/d6/d21/d12e/d75/d96/dc6/dd9/le6 199 1 2024-05-13T22:48:29.042 INFO:tasks.workunit.client.0.smithi102.stdout:7/935: mknod d6/d22/de5/c127 0 2024-05-13T22:48:29.043 INFO:tasks.workunit.client.0.smithi102.stdout:7/936: dread - d6/d13/d58/f10a zero size 2024-05-13T22:48:29.045 INFO:tasks.workunit.client.1.smithi130.stdout:2/325: write d4/db/dc/f15 [4939757,49915] 0 2024-05-13T22:48:29.045 INFO:tasks.workunit.client.1.smithi130.stdout:2/326: readlink d4/d49/l3c 0 2024-05-13T22:48:29.046 INFO:tasks.workunit.client.1.smithi130.stdout:7/238: symlink d7/d8/d9/l50 0 2024-05-13T22:48:29.046 INFO:tasks.workunit.client.0.smithi102.stdout:1/901: rmdir d2/d4/d9e 39 2024-05-13T22:48:29.048 INFO:tasks.workunit.client.0.smithi102.stdout:7/937: creat d6/d22/dd2/f128 x:0 0 0 2024-05-13T22:48:29.049 INFO:tasks.workunit.client.0.smithi102.stdout:7/938: stat d6/d15/f18 0 2024-05-13T22:48:29.049 INFO:tasks.workunit.client.1.smithi130.stdout:9/289: symlink d1/d7/d23/d28/d32/l5b 0 2024-05-13T22:48:29.051 INFO:tasks.workunit.client.1.smithi130.stdout:4/293: creat d8/dc/d35/d33/d3f/d57/f5f x:0 0 0 2024-05-13T22:48:29.051 INFO:tasks.workunit.client.1.smithi130.stdout:4/294: dread - d8/df/f59 zero size 2024-05-13T22:48:29.052 INFO:tasks.workunit.client.0.smithi102.stdout:1/902: mkdir d2/d45/d96/d123 0 2024-05-13T22:48:29.053 INFO:tasks.workunit.client.1.smithi130.stdout:3/278: sync 2024-05-13T22:48:29.054 INFO:tasks.workunit.client.1.smithi130.stdout:3/279: dread - d8/d1a/f47 zero size 2024-05-13T22:48:29.054 INFO:tasks.workunit.client.1.smithi130.stdout:4/295: dread f6 [0,4194304] 0 2024-05-13T22:48:29.055 INFO:tasks.workunit.client.0.smithi102.stdout:9/970: dwrite d1/d6/d45/d2d/dac/db0/fbb [0,4194304] 0 2024-05-13T22:48:29.055 INFO:tasks.workunit.client.1.smithi130.stdout:0/304: rmdir d0/d7/dd/d17 39 2024-05-13T22:48:29.058 INFO:tasks.workunit.client.1.smithi130.stdout:8/252: creat d1/dc/f5c x:0 0 0 2024-05-13T22:48:29.059 INFO:tasks.workunit.client.1.smithi130.stdout:4/296: dread d8/dc/d35/d33/d44/f4b [0,4194304] 0 2024-05-13T22:48:29.059 INFO:tasks.workunit.client.0.smithi102.stdout:7/939: mkdir d6/d13/d2c/dcb/d11d/d129 0 2024-05-13T22:48:29.060 INFO:tasks.workunit.client.0.smithi102.stdout:7/940: chown d6/d15/d4d/fde 437940 1 2024-05-13T22:48:29.066 INFO:tasks.workunit.client.0.smithi102.stdout:9/971: symlink d1/d6/db/d30/d120/l153 0 2024-05-13T22:48:29.068 INFO:tasks.workunit.client.1.smithi130.stdout:9/290: creat d1/d7/d23/d3d/f5c x:0 0 0 2024-05-13T22:48:29.068 INFO:tasks.workunit.client.1.smithi130.stdout:3/280: mknod d8/c60 0 2024-05-13T22:48:29.068 INFO:tasks.workunit.client.1.smithi130.stdout:3/281: readlink d8/d35/l40 0 2024-05-13T22:48:29.069 INFO:tasks.workunit.client.1.smithi130.stdout:9/291: chown d1/d7/d23/d28/c47 2 1 2024-05-13T22:48:29.070 INFO:tasks.workunit.client.1.smithi130.stdout:9/292: stat d1/c21 0 2024-05-13T22:48:29.070 INFO:tasks.workunit.client.1.smithi130.stdout:0/305: mknod d0/d7/d29/c6c 0 2024-05-13T22:48:29.071 INFO:tasks.workunit.client.1.smithi130.stdout:8/253: chown d1/dc/l47 848 1 2024-05-13T22:48:29.072 INFO:tasks.workunit.client.1.smithi130.stdout:7/239: write d7/d8/f4c [1374968,106992] 0 2024-05-13T22:48:29.073 INFO:tasks.workunit.client.1.smithi130.stdout:4/297: mkdir d8/dc/d35/d33/d3f/d57/d60 0 2024-05-13T22:48:29.074 INFO:tasks.workunit.client.1.smithi130.stdout:1/295: getdents db/d1b/d1f/d49 0 2024-05-13T22:48:29.076 INFO:tasks.workunit.client.1.smithi130.stdout:1/296: chown db/d1b/f20 63 1 2024-05-13T22:48:29.076 INFO:tasks.workunit.client.1.smithi130.stdout:0/306: dread d0/d7/d29/f40 [0,4194304] 0 2024-05-13T22:48:29.076 INFO:tasks.workunit.client.0.smithi102.stdout:7/941: dwrite d6/d15/d4d/fb9 [0,4194304] 0 2024-05-13T22:48:29.078 INFO:tasks.workunit.client.1.smithi130.stdout:5/276: sync 2024-05-13T22:48:29.078 INFO:tasks.workunit.client.1.smithi130.stdout:6/309: sync 2024-05-13T22:48:29.079 INFO:tasks.workunit.client.1.smithi130.stdout:3/282: mkdir d8/dc/d19/d2c/d61 0 2024-05-13T22:48:29.079 INFO:tasks.workunit.client.1.smithi130.stdout:7/240: dwrite d7/d8/f2d [0,4194304] 0 2024-05-13T22:48:29.081 INFO:tasks.workunit.client.1.smithi130.stdout:6/310: truncate d4/d16/f69 552749 0 2024-05-13T22:48:29.082 INFO:tasks.workunit.client.0.smithi102.stdout:1/903: dread d2/d10/f44 [0,4194304] 0 2024-05-13T22:48:29.083 INFO:tasks.workunit.client.1.smithi130.stdout:9/293: symlink d1/d7/d23/d28/d2d/d2f/l5d 0 2024-05-13T22:48:29.084 INFO:tasks.workunit.client.1.smithi130.stdout:9/294: readlink d1/db/l26 0 2024-05-13T22:48:29.085 INFO:tasks.workunit.client.0.smithi102.stdout:9/972: dwrite d1/d6/d21/d12e/d75/db9/ff0 [0,4194304] 0 2024-05-13T22:48:29.085 INFO:tasks.workunit.client.0.smithi102.stdout:7/942: rename d6/d13/d2c/da1/dc1/fa7 to d6/d15/d4d/d55/d6f/d11b/f12a 0 2024-05-13T22:48:29.090 INFO:tasks.workunit.client.0.smithi102.stdout:7/943: write d6/dd0/df2/d105/f107 [967252,90736] 0 2024-05-13T22:48:29.092 INFO:tasks.workunit.client.1.smithi130.stdout:2/327: creat d4/db/dc/f79 x:0 0 0 2024-05-13T22:48:29.092 INFO:tasks.workunit.client.1.smithi130.stdout:2/328: readlink d4/l57 0 2024-05-13T22:48:29.093 INFO:tasks.workunit.client.1.smithi130.stdout:4/298: mkdir d8/dc/d35/d33/d44/d61 0 2024-05-13T22:48:29.094 INFO:tasks.workunit.client.1.smithi130.stdout:2/329: chown d4/db/dc/d18/d2a/d30/d69/f6e 286 1 2024-05-13T22:48:29.095 INFO:tasks.workunit.client.1.smithi130.stdout:1/297: creat db/d1b/d1f/d2a/d4c/f65 x:0 0 0 2024-05-13T22:48:29.097 INFO:tasks.workunit.client.0.smithi102.stdout:9/973: mknod d1/d6/d45/d3e/dbe/c154 0 2024-05-13T22:48:29.098 INFO:tasks.workunit.client.1.smithi130.stdout:3/283: rmdir d8/dc/d15 39 2024-05-13T22:48:29.102 INFO:tasks.workunit.client.1.smithi130.stdout:0/307: dwrite d0/d7/dd/d17/d1f/d2c/f30 [4194304,4194304] 0 2024-05-13T22:48:29.106 INFO:tasks.workunit.client.1.smithi130.stdout:9/295: creat d1/db/f5e x:0 0 0 2024-05-13T22:48:29.108 INFO:tasks.workunit.client.1.smithi130.stdout:8/254: mkdir d1/dc/d5d 0 2024-05-13T22:48:29.109 INFO:tasks.workunit.client.1.smithi130.stdout:8/255: chown d1/d2/db/d15/d16/l20 369453 1 2024-05-13T22:48:29.109 INFO:tasks.workunit.client.0.smithi102.stdout:7/944: write d6/d13/d2c/d39/db3/fcf [7376,87367] 0 2024-05-13T22:48:29.114 INFO:tasks.workunit.client.1.smithi130.stdout:2/330: rename d4/db/dc/l40 to d4/d49/l7a 0 2024-05-13T22:48:29.114 INFO:tasks.workunit.client.0.smithi102.stdout:7/945: creat d6/d22/de5/d122/f12b x:0 0 0 2024-05-13T22:48:29.115 INFO:tasks.workunit.client.0.smithi102.stdout:7/946: chown d6/d13/d2c/f3a 16281605 1 2024-05-13T22:48:29.115 INFO:tasks.workunit.client.1.smithi130.stdout:3/284: fsync d8/d1a/d1c/f5d 0 2024-05-13T22:48:29.117 INFO:tasks.workunit.client.1.smithi130.stdout:7/241: symlink d7/da/l51 0 2024-05-13T22:48:29.117 INFO:tasks.workunit.client.1.smithi130.stdout:6/311: dread d4/d16/f2a [0,4194304] 0 2024-05-13T22:48:29.118 INFO:tasks.workunit.client.0.smithi102.stdout:7/947: rmdir d6/d13/d2c/da1/d103 39 2024-05-13T22:48:29.119 INFO:tasks.workunit.client.0.smithi102.stdout:9/974: dread d1/d6/f1d [0,4194304] 0 2024-05-13T22:48:29.119 INFO:tasks.workunit.client.1.smithi130.stdout:0/308: write d0/d7/fb [5137242,17540] 0 2024-05-13T22:48:29.119 INFO:tasks.workunit.client.1.smithi130.stdout:4/299: dwrite d8/dc/d26/f36 [0,4194304] 0 2024-05-13T22:48:29.121 INFO:tasks.workunit.client.1.smithi130.stdout:6/312: dread - d4/d15/d1f/d46/d50/f53 zero size 2024-05-13T22:48:29.125 INFO:tasks.workunit.client.1.smithi130.stdout:9/296: unlink d1/d7/d23/d28/d2d/d2f/l4d 0 2024-05-13T22:48:29.126 INFO:tasks.workunit.client.1.smithi130.stdout:9/297: readlink d1/d7/l18 0 2024-05-13T22:48:29.126 INFO:tasks.workunit.client.1.smithi130.stdout:1/298: mkdir db/d1b/d1f/d2a/d38/d66 0 2024-05-13T22:48:29.129 INFO:tasks.workunit.client.1.smithi130.stdout:7/242: symlink d7/da/d1d/d42/l52 0 2024-05-13T22:48:29.130 INFO:tasks.workunit.client.1.smithi130.stdout:4/300: mknod d8/dc/d29/c62 0 2024-05-13T22:48:29.131 INFO:tasks.workunit.client.1.smithi130.stdout:6/313: rename d4/d15/c1c to d4/d15/d1f/d46/d50/c6e 0 2024-05-13T22:48:29.132 INFO:tasks.workunit.client.1.smithi130.stdout:5/277: truncate d6/d7/f45 1913742 0 2024-05-13T22:48:29.133 INFO:tasks.workunit.client.1.smithi130.stdout:6/314: readlink d4/d16/l36 0 2024-05-13T22:48:29.134 INFO:tasks.workunit.client.0.smithi102.stdout:9/975: write d1/d6/d21/d12e/d75/d96/dc6/feb [916233,64876] 0 2024-05-13T22:48:29.135 INFO:tasks.workunit.client.1.smithi130.stdout:0/309: dread d0/d7/fb [4194304,4194304] 0 2024-05-13T22:48:29.136 INFO:tasks.workunit.client.1.smithi130.stdout:2/331: fdatasync d4/db/f2e 0 2024-05-13T22:48:29.136 INFO:tasks.workunit.client.1.smithi130.stdout:1/299: dwrite db/f1d [0,4194304] 0 2024-05-13T22:48:29.137 INFO:tasks.workunit.client.1.smithi130.stdout:3/285: rmdir d8/dc/d19/d27 39 2024-05-13T22:48:29.137 INFO:tasks.workunit.client.0.smithi102.stdout:9/976: dread d1/f5c [0,4194304] 0 2024-05-13T22:48:29.139 INFO:tasks.workunit.client.0.smithi102.stdout:7/948: dwrite d6/d15/d4d/d55/f8e [4194304,4194304] 0 2024-05-13T22:48:29.139 INFO:tasks.workunit.client.1.smithi130.stdout:2/332: read - d4/db/dc/f79 zero size 2024-05-13T22:48:29.140 INFO:tasks.workunit.client.0.smithi102.stdout:7/949: readlink d6/d15/d4c/l11e 0 2024-05-13T22:48:29.140 INFO:tasks.workunit.client.1.smithi130.stdout:0/310: readlink d0/d7/dd/d17/d1f/d2c/l4d 0 2024-05-13T22:48:29.141 INFO:tasks.workunit.client.1.smithi130.stdout:4/301: mknod d8/dc/d35/d33/d3f/c63 0 2024-05-13T22:48:29.142 INFO:tasks.workunit.client.1.smithi130.stdout:1/300: read db/d1b/d1f/d2a/f2f [1969076,17322] 0 2024-05-13T22:48:29.142 INFO:tasks.workunit.client.1.smithi130.stdout:9/298: dread d1/db/fc [0,4194304] 0 2024-05-13T22:48:29.145 INFO:tasks.workunit.client.1.smithi130.stdout:5/278: truncate d6/dd/f33 1958138 0 2024-05-13T22:48:29.145 INFO:tasks.workunit.client.1.smithi130.stdout:1/301: write db/d1b/d1f/d2a/f2f [4648659,4845] 0 2024-05-13T22:48:29.147 INFO:tasks.workunit.client.0.smithi102.stdout:9/977: creat d1/d6/d45/d3e/d79/f155 x:0 0 0 2024-05-13T22:48:29.147 INFO:tasks.workunit.client.1.smithi130.stdout:5/279: write d6/d7/d32/f44 [65010,113179] 0 2024-05-13T22:48:29.148 INFO:tasks.workunit.client.1.smithi130.stdout:5/280: dread - d6/f54 zero size 2024-05-13T22:48:29.149 INFO:tasks.workunit.client.0.smithi102.stdout:7/950: dwrite d6/d22/dd2/f128 [0,4194304] 0 2024-05-13T22:48:29.150 INFO:tasks.workunit.client.1.smithi130.stdout:0/311: dwrite d0/d7/dd/d17/d1f/d2c/f30 [4194304,4194304] 0 2024-05-13T22:48:29.153 INFO:tasks.workunit.client.0.smithi102.stdout:7/951: chown d6/d15/d4d/d55/d6f/d65/f86 8869 1 2024-05-13T22:48:29.156 INFO:tasks.workunit.client.1.smithi130.stdout:2/333: fsync d4/fa 0 2024-05-13T22:48:29.160 INFO:tasks.workunit.client.0.smithi102.stdout:9/978: mknod d1/d6/d21/d12e/d64/d152/c156 0 2024-05-13T22:48:29.161 INFO:tasks.workunit.client.1.smithi130.stdout:7/243: dwrite d7/da/d1d/d42/d43/f4d [0,4194304] 0 2024-05-13T22:48:29.164 INFO:tasks.workunit.client.1.smithi130.stdout:7/244: write d7/d8/f19 [2642429,80948] 0 2024-05-13T22:48:29.165 INFO:tasks.workunit.client.0.smithi102.stdout:9/979: creat d1/d6/d21/d12e/d81/f157 x:0 0 0 2024-05-13T22:48:29.166 INFO:tasks.workunit.client.0.smithi102.stdout:9/980: write d1/d6/d21/d12e/d81/d116/f11e [5091561,4670] 0 2024-05-13T22:48:29.168 INFO:tasks.workunit.client.0.smithi102.stdout:9/981: write d1/d6/d45/fe5 [1960167,104072] 0 2024-05-13T22:48:29.169 INFO:tasks.workunit.client.0.smithi102.stdout:9/982: chown d1/d6/d45/f4f 225913751 1 2024-05-13T22:48:29.172 INFO:tasks.workunit.client.0.smithi102.stdout:7/952: link d6/d22/c3d d6/dd0/df2/d105/c12c 0 2024-05-13T22:48:29.174 INFO:tasks.workunit.client.0.smithi102.stdout:9/983: truncate d1/f17 2607381 0 2024-05-13T22:48:29.175 INFO:tasks.workunit.client.1.smithi130.stdout:0/312: mknod d0/d7/dd/d17/d1f/d2c/d5a/c6d 0 2024-05-13T22:48:29.175 INFO:tasks.workunit.client.1.smithi130.stdout:0/313: chown d0/f42 3 1 2024-05-13T22:48:29.178 INFO:tasks.workunit.client.1.smithi130.stdout:3/286: rename d8/dc/d19/d27/l31 to d8/dc/d19/d2c/d61/l62 0 2024-05-13T22:48:29.182 INFO:tasks.workunit.client.0.smithi102.stdout:9/984: fsync d1/d6/d45/da1/f12a 0 2024-05-13T22:48:29.182 INFO:tasks.workunit.client.1.smithi130.stdout:3/287: read d8/f2b [2671305,3812] 0 2024-05-13T22:48:29.182 INFO:tasks.workunit.client.1.smithi130.stdout:9/299: dwrite d1/d7/d23/d28/d2d/f48 [0,4194304] 0 2024-05-13T22:48:29.185 INFO:tasks.workunit.client.1.smithi130.stdout:4/302: symlink d8/dc/d35/d33/d44/d61/l64 0 2024-05-13T22:48:29.186 INFO:tasks.workunit.client.0.smithi102.stdout:9/985: dwrite d1/d6/d21/d12e/d75/d96/dc6/dd9/d125/d14e/d5b/f113 [0,4194304] 0 2024-05-13T22:48:29.187 INFO:tasks.workunit.client.1.smithi130.stdout:1/302: symlink db/d1b/d1f/d2a/d38/d66/l67 0 2024-05-13T22:48:29.187 INFO:tasks.workunit.client.1.smithi130.stdout:4/303: dread - d8/f12 zero size 2024-05-13T22:48:29.188 INFO:tasks.workunit.client.1.smithi130.stdout:1/303: stat db/l50 0 2024-05-13T22:48:29.188 INFO:tasks.workunit.client.1.smithi130.stdout:1/304: stat db/l36 0 2024-05-13T22:48:29.190 INFO:tasks.workunit.client.1.smithi130.stdout:2/334: dread d4/db/d26/f2d [0,4194304] 0 2024-05-13T22:48:29.191 INFO:tasks.workunit.client.0.smithi102.stdout:7/953: write d6/d15/d4d/d55/d6f/d49/d95/fb7 [94365,62453] 0 2024-05-13T22:48:29.192 INFO:tasks.workunit.client.1.smithi130.stdout:0/314: truncate d0/f8 3624101 0 2024-05-13T22:48:29.195 INFO:tasks.workunit.client.1.smithi130.stdout:2/335: dread d4/db/dc/f10 [0,4194304] 0 2024-05-13T22:48:29.196 INFO:tasks.workunit.client.0.smithi102.stdout:9/986: link d1/d6/d21/d12e/d64/d152/d6c/f86 d1/d6/db/d30/d122/f158 0 2024-05-13T22:48:29.198 INFO:tasks.workunit.client.1.smithi130.stdout:6/315: getdents d4/d15/d1d/d60 0 2024-05-13T22:48:29.198 INFO:tasks.workunit.client.1.smithi130.stdout:8/256: sync 2024-05-13T22:48:29.205 INFO:tasks.workunit.client.0.smithi102.stdout:9/987: rename d1/d6/db/d30/l7d to d1/d6/d21/d12e/d75/d96/l159 0 2024-05-13T22:48:29.207 INFO:tasks.workunit.client.1.smithi130.stdout:0/315: mknod d0/d7/dd/d17/d1f/d2c/d5a/c6e 0 2024-05-13T22:48:29.207 INFO:tasks.workunit.client.1.smithi130.stdout:1/305: rmdir db/d1b 39 2024-05-13T22:48:29.210 INFO:tasks.workunit.client.1.smithi130.stdout:2/336: rename d4/db/f27 to d4/db/dc/d18/d58/f7b 0 2024-05-13T22:48:29.212 INFO:tasks.workunit.client.0.smithi102.stdout:9/988: mknod d1/d6/db/d145/c15a 0 2024-05-13T22:48:29.212 INFO:tasks.workunit.client.0.smithi102.stdout:9/989: readlink d1/d6/d21/d71/lc9 0 2024-05-13T22:48:29.215 INFO:tasks.workunit.client.0.smithi102.stdout:7/954: write d6/d15/d4d/d55/d82/feb [162425,63016] 0 2024-05-13T22:48:29.216 INFO:tasks.workunit.client.1.smithi130.stdout:8/257: symlink d1/d2/db/d15/l5e 0 2024-05-13T22:48:29.217 INFO:tasks.workunit.client.0.smithi102.stdout:7/955: chown d6/d15/d4d/d55/d6f/d49/d97/f120 787048 1 2024-05-13T22:48:29.218 INFO:tasks.workunit.client.1.smithi130.stdout:7/245: link d7/d8/d9/l34 d7/da/d1d/d42/l53 0 2024-05-13T22:48:29.219 INFO:tasks.workunit.client.1.smithi130.stdout:7/246: stat d7/da/d1d/d42/l53 0 2024-05-13T22:48:29.219 INFO:tasks.workunit.client.1.smithi130.stdout:4/304: write d8/dc/d29/f3a [1485540,56893] 0 2024-05-13T22:48:29.222 INFO:tasks.workunit.client.1.smithi130.stdout:6/316: dread d4/d15/d1f/f2d [0,4194304] 0 2024-05-13T22:48:29.226 INFO:tasks.workunit.client.1.smithi130.stdout:6/317: truncate d4/d15/d1f/d46/d50/f67 633591 0 2024-05-13T22:48:29.227 INFO:tasks.workunit.client.1.smithi130.stdout:4/305: dread d8/fa [0,4194304] 0 2024-05-13T22:48:29.228 INFO:tasks.workunit.client.1.smithi130.stdout:0/316: rename d0/d7/dd/d17/d1f/d2c/f61 to d0/d7/dd/d17/d52/d5b/f6f 0 2024-05-13T22:48:29.228 INFO:tasks.workunit.client.1.smithi130.stdout:1/306: dread db/f17 [0,4194304] 0 2024-05-13T22:48:29.229 INFO:tasks.workunit.client.1.smithi130.stdout:0/317: fdatasync d0/d7/dd/d2e/f3f 0 2024-05-13T22:48:29.229 INFO:tasks.workunit.client.1.smithi130.stdout:2/337: write d4/db/f22 [8519201,58251] 0 2024-05-13T22:48:29.230 INFO:tasks.workunit.client.1.smithi130.stdout:5/281: dread d6/dd/f33 [0,4194304] 0 2024-05-13T22:48:29.230 INFO:tasks.workunit.client.1.smithi130.stdout:9/300: creat d1/f5f x:0 0 0 2024-05-13T22:48:29.231 INFO:tasks.workunit.client.1.smithi130.stdout:9/301: fsync d1/d7/f57 0 2024-05-13T22:48:29.231 INFO:tasks.workunit.client.1.smithi130.stdout:5/282: fsync d6/d7/d19/f2c 0 2024-05-13T22:48:29.232 INFO:tasks.workunit.client.1.smithi130.stdout:0/318: truncate d0/d7/dd/f46 329642 0 2024-05-13T22:48:29.233 INFO:tasks.workunit.client.1.smithi130.stdout:9/302: chown d1/d7/d23/d3d/f40 57990 1 2024-05-13T22:48:29.234 INFO:tasks.workunit.client.0.smithi102.stdout:7/956: dwrite d6/d13/d2c/f5c [0,4194304] 0 2024-05-13T22:48:29.237 INFO:tasks.workunit.client.1.smithi130.stdout:3/288: dwrite d8/dc/d15/f18 [0,4194304] 0 2024-05-13T22:48:29.239 INFO:tasks.workunit.client.0.smithi102.stdout:7/957: rename d6/dd0/d80/fd8 to d6/da9/f12d 0 2024-05-13T22:48:29.240 INFO:tasks.workunit.client.1.smithi130.stdout:6/318: dwrite d4/d15/d1d/d28/f47 [0,4194304] 0 2024-05-13T22:48:29.246 INFO:tasks.workunit.client.1.smithi130.stdout:9/303: symlink d1/d7/d23/d28/d2d/d2f/l60 0 2024-05-13T22:48:29.250 INFO:tasks.workunit.client.1.smithi130.stdout:1/307: read db/d1b/d1f/d2a/f2f [2362241,681] 0 2024-05-13T22:48:29.252 INFO:tasks.workunit.client.1.smithi130.stdout:8/258: creat d1/d2/db/d15/d18/f5f x:0 0 0 2024-05-13T22:48:29.252 INFO:tasks.workunit.client.1.smithi130.stdout:3/289: mknod d8/d1a/d1c/c63 0 2024-05-13T22:48:29.254 INFO:tasks.workunit.client.1.smithi130.stdout:2/338: symlink d4/db/dc/d18/d2a/d66/l7c 0 2024-05-13T22:48:29.254 INFO:tasks.workunit.client.1.smithi130.stdout:7/247: link d7/d8/c3e d7/da/d1d/d3d/c54 0 2024-05-13T22:48:29.255 INFO:tasks.workunit.client.1.smithi130.stdout:5/283: creat d6/d7/d24/d56/f5d x:0 0 0 2024-05-13T22:48:29.256 INFO:tasks.workunit.client.1.smithi130.stdout:0/319: creat d0/d7/dd/d17/d52/d5b/d5f/f70 x:0 0 0 2024-05-13T22:48:29.258 INFO:tasks.workunit.client.1.smithi130.stdout:1/308: dwrite db/d4a/f4b [0,4194304] 0 2024-05-13T22:48:29.259 INFO:tasks.workunit.client.1.smithi130.stdout:4/306: link d8/dc/d35/d33/d3f/d57/c58 d8/dc/d35/d33/c65 0 2024-05-13T22:48:29.260 INFO:tasks.workunit.client.1.smithi130.stdout:0/320: write d0/d7/dd/d17/d52/d67/f6a [909420,127080] 0 2024-05-13T22:48:29.262 INFO:tasks.workunit.client.1.smithi130.stdout:1/309: write db/d1b/d1f/d2a/d35/f4d [2224315,29194] 0 2024-05-13T22:48:29.262 INFO:tasks.workunit.client.1.smithi130.stdout:8/259: unlink d1/d2/db/f3b 0 2024-05-13T22:48:29.263 INFO:tasks.workunit.client.1.smithi130.stdout:3/290: write d8/f34 [141078,62719] 0 2024-05-13T22:48:29.263 INFO:tasks.workunit.client.1.smithi130.stdout:0/321: truncate d0/d7/dd/f46 539277 0 2024-05-13T22:48:29.265 INFO:tasks.workunit.client.1.smithi130.stdout:2/339: creat d4/db/dc/d18/d2a/f7d x:0 0 0 2024-05-13T22:48:29.265 INFO:tasks.workunit.client.1.smithi130.stdout:2/340: read d4/f8 [2743850,33301] 0 2024-05-13T22:48:29.268 INFO:tasks.workunit.client.0.smithi102.stdout:7/958: dwrite d6/d13/d58/dd1/f12 [4194304,4194304] 0 2024-05-13T22:48:29.273 INFO:tasks.workunit.client.1.smithi130.stdout:4/307: write d8/dc/d26/f2b [31727,44157] 0 2024-05-13T22:48:29.276 INFO:tasks.workunit.client.1.smithi130.stdout:4/308: chown d8/dc/d35/d33/d44/d56 956 1 2024-05-13T22:48:29.278 INFO:tasks.workunit.client.0.smithi102.stdout:7/959: creat d6/d15/d4d/d55/d6f/f12e x:0 0 0 2024-05-13T22:48:29.278 INFO:tasks.workunit.client.1.smithi130.stdout:2/341: fsync d4/db/dc/d18/d2a/f7d 0 2024-05-13T22:48:29.281 INFO:tasks.workunit.client.1.smithi130.stdout:7/248: symlink d7/da/d1f/d40/l55 0 2024-05-13T22:48:29.282 INFO:tasks.workunit.client.1.smithi130.stdout:4/309: rename d8/l15 to d8/df/l66 0 2024-05-13T22:48:29.283 INFO:tasks.workunit.client.1.smithi130.stdout:9/304: dwrite d1/d15/f16 [0,4194304] 0 2024-05-13T22:48:29.283 INFO:tasks.workunit.client.0.smithi102.stdout:7/960: unlink d6/d15/d4d/d55/d6f/d65/ded/f10e 0 2024-05-13T22:48:29.285 INFO:tasks.workunit.client.1.smithi130.stdout:8/260: creat d1/d2/db/d15/d18/f60 x:0 0 0 2024-05-13T22:48:29.288 INFO:tasks.workunit.client.1.smithi130.stdout:9/305: write d1/d7/d23/d28/d2d/f48 [1673995,45211] 0 2024-05-13T22:48:29.290 INFO:tasks.workunit.client.1.smithi130.stdout:2/342: creat d4/db/dc/d18/f7e x:0 0 0 2024-05-13T22:48:29.293 INFO:tasks.workunit.client.1.smithi130.stdout:3/291: dread d8/dc/d19/f20 [0,4194304] 0 2024-05-13T22:48:29.294 INFO:tasks.workunit.client.1.smithi130.stdout:1/310: dwrite db/d1b/d1f/d2a/d38/f60 [0,4194304] 0 2024-05-13T22:48:29.294 INFO:tasks.workunit.client.1.smithi130.stdout:7/249: creat d7/da/f56 x:0 0 0 2024-05-13T22:48:29.296 INFO:tasks.workunit.client.1.smithi130.stdout:4/310: rename d8/dc/d13/l25 to d8/dc/d35/d33/d44/d61/l67 0 2024-05-13T22:48:29.296 INFO:tasks.workunit.client.1.smithi130.stdout:2/343: write d4/db/dc/d18/f23 [3829336,111077] 0 2024-05-13T22:48:29.298 INFO:tasks.workunit.client.1.smithi130.stdout:7/250: stat d7/da/d1d/d42/f44 0 2024-05-13T22:48:29.300 INFO:tasks.workunit.client.1.smithi130.stdout:4/311: stat d8/dc/d13/l45 0 2024-05-13T22:48:29.301 INFO:tasks.workunit.client.1.smithi130.stdout:3/292: readlink d8/d35/l1f 0 2024-05-13T22:48:29.301 INFO:tasks.workunit.client.1.smithi130.stdout:0/322: truncate d0/d7/f22 3014214 0 2024-05-13T22:48:29.301 INFO:tasks.workunit.client.1.smithi130.stdout:9/306: mkdir d1/d7/d23/d61 0 2024-05-13T22:48:29.302 INFO:tasks.workunit.client.1.smithi130.stdout:1/311: mknod db/d1b/d1f/d2a/d38/d46/c68 0 2024-05-13T22:48:29.304 INFO:tasks.workunit.client.1.smithi130.stdout:3/293: write d8/f4b [255456,58490] 0 2024-05-13T22:48:29.305 INFO:tasks.workunit.client.1.smithi130.stdout:9/307: chown d1/f5f 624623 1 2024-05-13T22:48:29.305 INFO:tasks.workunit.client.1.smithi130.stdout:1/312: chown db/f17 681695 1 2024-05-13T22:48:29.305 INFO:tasks.workunit.client.1.smithi130.stdout:7/251: symlink d7/da/d1f/d2c/l57 0 2024-05-13T22:48:29.306 INFO:tasks.workunit.client.1.smithi130.stdout:8/261: dwrite d1/d2/f3 [4194304,4194304] 0 2024-05-13T22:48:29.310 INFO:tasks.workunit.client.1.smithi130.stdout:3/294: creat d8/f64 x:0 0 0 2024-05-13T22:48:29.310 INFO:tasks.workunit.client.0.smithi102.stdout:7/961: dread d6/d22/f33 [0,4194304] 0 2024-05-13T22:48:29.311 INFO:tasks.workunit.client.1.smithi130.stdout:1/313: rmdir db/d1b/d1f/d49 39 2024-05-13T22:48:29.312 INFO:tasks.workunit.client.0.smithi102.stdout:7/962: stat d6/d13/d58/fc9 0 2024-05-13T22:48:29.312 INFO:tasks.workunit.client.0.smithi102.stdout:7/963: chown d6/d13/d2c 25 1 2024-05-13T22:48:29.313 INFO:tasks.workunit.client.0.smithi102.stdout:7/964: dread - d6/d13/d2c/f121 zero size 2024-05-13T22:48:29.313 INFO:tasks.workunit.client.1.smithi130.stdout:9/308: rename d1/f33 to d1/d7/d23/d28/d32/f62 0 2024-05-13T22:48:29.313 INFO:tasks.workunit.client.0.smithi102.stdout:7/965: chown d6/d15/d4c/dfe 0 1 2024-05-13T22:48:29.317 INFO:tasks.workunit.client.1.smithi130.stdout:4/312: getdents d8/dc/d35/d33 0 2024-05-13T22:48:29.317 INFO:tasks.workunit.client.1.smithi130.stdout:1/314: unlink db/c18 0 2024-05-13T22:48:29.318 INFO:tasks.workunit.client.0.smithi102.stdout:7/966: mknod d6/d13/d2c/dcb/c12f 0 2024-05-13T22:48:29.318 INFO:tasks.workunit.client.1.smithi130.stdout:8/262: creat d1/d2/db/d15/d18/d42/d53/d56/f61 x:0 0 0 2024-05-13T22:48:29.320 INFO:tasks.workunit.client.0.smithi102.stdout:7/967: symlink d6/d13/d2c/dcb/d11d/d129/l130 0 2024-05-13T22:48:29.322 INFO:tasks.workunit.client.1.smithi130.stdout:2/344: dwrite d4/db/dc/f37 [0,4194304] 0 2024-05-13T22:48:29.322 INFO:tasks.workunit.client.0.smithi102.stdout:7/968: creat d6/d15/d4d/d55/d6f/d65/ded/f131 x:0 0 0 2024-05-13T22:48:29.324 INFO:tasks.workunit.client.0.smithi102.stdout:7/969: dread d6/f85 [0,4194304] 0 2024-05-13T22:48:29.327 INFO:tasks.workunit.client.0.smithi102.stdout:7/970: mknod d6/d13/d58/dd1/c132 0 2024-05-13T22:48:29.330 INFO:tasks.workunit.client.1.smithi130.stdout:7/252: dwrite d7/d8/f4c [0,4194304] 0 2024-05-13T22:48:29.331 INFO:tasks.workunit.client.1.smithi130.stdout:0/323: dwrite d0/d7/dd/d17/d52/d67/f6a [0,4194304] 0 2024-05-13T22:48:29.332 INFO:tasks.workunit.client.1.smithi130.stdout:1/315: creat db/d1b/d1f/d2a/d35/f69 x:0 0 0 2024-05-13T22:48:29.333 INFO:tasks.workunit.client.1.smithi130.stdout:7/253: fdatasync d7/da/f35 0 2024-05-13T22:48:29.335 INFO:tasks.workunit.client.1.smithi130.stdout:9/309: dread d1/d15/f16 [0,4194304] 0 2024-05-13T22:48:29.335 INFO:tasks.workunit.client.1.smithi130.stdout:3/295: dwrite d8/dc/d15/f4d [0,4194304] 0 2024-05-13T22:48:29.336 INFO:tasks.workunit.client.0.smithi102.stdout:7/971: dwrite d6/d13/d58/fac [0,4194304] 0 2024-05-13T22:48:29.336 INFO:tasks.workunit.client.1.smithi130.stdout:7/254: chown d7/da/d1f/f20 2962005 1 2024-05-13T22:48:29.341 INFO:tasks.workunit.client.1.smithi130.stdout:4/313: rename d8/dc/d35/d33/d3f/l46 to d8/dc/l68 0 2024-05-13T22:48:29.343 INFO:tasks.workunit.client.0.smithi102.stdout:7/972: creat d6/d15/d4d/d55/d6f/d49/f133 x:0 0 0 2024-05-13T22:48:29.343 INFO:tasks.workunit.client.1.smithi130.stdout:9/310: dread d1/d7/d23/d3d/f40 [0,4194304] 0 2024-05-13T22:48:29.344 INFO:tasks.workunit.client.0.smithi102.stdout:7/973: chown d6/d15/d4d/d55/lf1 31187 1 2024-05-13T22:48:29.344 INFO:tasks.workunit.client.1.smithi130.stdout:0/324: mknod d0/d7/dd/d17/d52/c71 0 2024-05-13T22:48:29.344 INFO:tasks.workunit.client.1.smithi130.stdout:9/311: write d1/d15/f1d [3041379,37501] 0 2024-05-13T22:48:29.347 INFO:tasks.workunit.client.1.smithi130.stdout:9/312: fsync d1/db/fc 0 2024-05-13T22:48:29.348 INFO:tasks.workunit.client.1.smithi130.stdout:9/313: dread - d1/db/f5e zero size 2024-05-13T22:48:29.349 INFO:tasks.workunit.client.1.smithi130.stdout:9/314: fdatasync d1/f4f 0 2024-05-13T22:48:29.352 INFO:tasks.workunit.client.1.smithi130.stdout:4/314: dwrite d8/dc/d29/f3a [0,4194304] 0 2024-05-13T22:48:29.353 INFO:tasks.workunit.client.1.smithi130.stdout:4/315: chown d8/dc/d35/d33/f4f 906013 1 2024-05-13T22:48:29.354 INFO:tasks.workunit.client.1.smithi130.stdout:4/316: chown d8/dc/d35 2353 1 2024-05-13T22:48:29.354 INFO:tasks.workunit.client.1.smithi130.stdout:4/317: fsync d8/df/f59 0 2024-05-13T22:48:29.368 INFO:tasks.workunit.client.1.smithi130.stdout:8/263: rename d1/d2/fd to d1/d2/db/d15/d18/d42/f62 0 2024-05-13T22:48:29.371 INFO:tasks.workunit.client.1.smithi130.stdout:0/325: mknod d0/d7/dd/d17/d1f/d2c/c72 0 2024-05-13T22:48:29.371 INFO:tasks.workunit.client.1.smithi130.stdout:0/326: stat d0/d7/dd/d2e/f2f 0 2024-05-13T22:48:29.375 INFO:tasks.workunit.client.1.smithi130.stdout:3/296: mknod d8/d1a/c65 0 2024-05-13T22:48:29.376 INFO:tasks.workunit.client.1.smithi130.stdout:1/316: creat db/f6a x:0 0 0 2024-05-13T22:48:29.377 INFO:tasks.workunit.client.1.smithi130.stdout:0/327: dwrite d0/f21 [0,4194304] 0 2024-05-13T22:48:29.379 INFO:tasks.workunit.client.1.smithi130.stdout:0/328: write d0/d7/dd/d2e/f3f [586870,64485] 0 2024-05-13T22:48:29.379 INFO:tasks.workunit.client.0.smithi102.stdout:7/974: dread d6/d15/d4d/d55/d6f/d11b/f12a [0,4194304] 0 2024-05-13T22:48:29.380 INFO:tasks.workunit.client.1.smithi130.stdout:7/255: rename d7/da/d1f/f20 to d7/da/d1d/d42/f58 0 2024-05-13T22:48:29.383 INFO:tasks.workunit.client.1.smithi130.stdout:7/256: dread d7/d8/f4c [0,4194304] 0 2024-05-13T22:48:29.390 INFO:tasks.workunit.client.1.smithi130.stdout:1/317: chown ca 7 1 2024-05-13T22:48:29.390 INFO:tasks.workunit.client.1.smithi130.stdout:1/318: stat db/d4a 0 2024-05-13T22:48:29.390 INFO:tasks.workunit.client.1.smithi130.stdout:3/297: truncate d8/d1a/d1c/f5d 326159 0 2024-05-13T22:48:29.390 INFO:tasks.workunit.client.1.smithi130.stdout:3/298: stat d8/dc/f36 0 2024-05-13T22:48:29.391 INFO:tasks.workunit.client.1.smithi130.stdout:3/299: chown d8/dc/d19/f30 117843 1 2024-05-13T22:48:29.392 INFO:tasks.workunit.client.1.smithi130.stdout:6/319: sync 2024-05-13T22:48:29.392 INFO:tasks.workunit.client.1.smithi130.stdout:3/300: write d8/dc/f21 [759244,85266] 0 2024-05-13T22:48:29.394 INFO:tasks.workunit.client.1.smithi130.stdout:3/301: truncate d8/dc/d15/f58 621333 0 2024-05-13T22:48:29.395 INFO:tasks.workunit.client.1.smithi130.stdout:3/302: read - d8/f64 zero size 2024-05-13T22:48:29.402 INFO:tasks.workunit.client.1.smithi130.stdout:9/315: truncate d1/d7/d23/d28/d2d/f46 608504 0 2024-05-13T22:48:29.403 INFO:tasks.workunit.client.1.smithi130.stdout:6/320: creat d4/d15/d1d/f6f x:0 0 0 2024-05-13T22:48:29.404 INFO:tasks.workunit.client.1.smithi130.stdout:9/316: stat d1/f1e 0 2024-05-13T22:48:29.405 INFO:tasks.workunit.client.1.smithi130.stdout:4/318: truncate d8/dc/d35/d33/d44/f4b 3583982 0 2024-05-13T22:48:29.405 INFO:tasks.workunit.client.1.smithi130.stdout:3/303: truncate d8/dc/f37 4048561 0 2024-05-13T22:48:29.407 INFO:tasks.workunit.client.0.smithi102.stdout:7/975: write d6/d15/d4d/d55/d6f/fdb [5141710,5141] 0 2024-05-13T22:48:29.409 INFO:tasks.workunit.client.1.smithi130.stdout:8/264: rename d1/d2/f48 to d1/d2/db/d15/d18/f63 0 2024-05-13T22:48:29.411 INFO:tasks.workunit.client.1.smithi130.stdout:6/321: read d4/d15/d1f/d46/d50/f54 [24970,75912] 0 2024-05-13T22:48:29.411 INFO:tasks.workunit.client.0.smithi102.stdout:7/976: creat d6/dd0/f134 x:0 0 0 2024-05-13T22:48:29.412 INFO:tasks.workunit.client.1.smithi130.stdout:1/319: dwrite db/d1b/d1f/d49/f48 [0,4194304] 0 2024-05-13T22:48:29.412 INFO:tasks.workunit.client.0.smithi102.stdout:7/977: dread - d6/d22/dd2/ff4 zero size 2024-05-13T22:48:29.413 INFO:tasks.workunit.client.1.smithi130.stdout:7/257: dread d7/da/d1f/f30 [0,4194304] 0 2024-05-13T22:48:29.413 INFO:tasks.workunit.client.1.smithi130.stdout:0/329: getdents d0/d7/dd/d17 0 2024-05-13T22:48:29.413 INFO:tasks.workunit.client.1.smithi130.stdout:3/304: chown d8/dc/d19/d2c/f46 4253 1 2024-05-13T22:48:29.415 INFO:tasks.workunit.client.1.smithi130.stdout:8/265: write d1/d2/db/f41 [459512,51997] 0 2024-05-13T22:48:29.416 INFO:tasks.workunit.client.0.smithi102.stdout:7/978: fdatasync d6/d13/d2c/f3a 0 2024-05-13T22:48:29.417 INFO:tasks.workunit.client.1.smithi130.stdout:9/317: write d1/d7/d23/d3d/f5a [1633378,38764] 0 2024-05-13T22:48:29.418 INFO:tasks.workunit.client.1.smithi130.stdout:1/320: creat db/d4a/f6b x:0 0 0 2024-05-13T22:48:29.420 INFO:tasks.workunit.client.1.smithi130.stdout:9/318: chown d1/db/cf 2480322 1 2024-05-13T22:48:29.424 INFO:tasks.workunit.client.1.smithi130.stdout:8/266: write d1/d2/db/d15/d18/f4c [685084,18340] 0 2024-05-13T22:48:29.424 INFO:tasks.workunit.client.1.smithi130.stdout:0/330: symlink d0/d7/dd/d17/d52/d5b/l73 0 2024-05-13T22:48:29.425 INFO:tasks.workunit.client.1.smithi130.stdout:4/319: dwrite d8/df/f3c [0,4194304] 0 2024-05-13T22:48:29.426 INFO:tasks.workunit.client.0.smithi102.stdout:7/979: link d6/d13/d2c/da1/dc1/f66 d6/de1/f135 0 2024-05-13T22:48:29.427 INFO:tasks.workunit.client.1.smithi130.stdout:8/267: rename d1/d2/db to d1/d2/db/d15/d18/d42/d53/d56/d64 22 2024-05-13T22:48:29.428 INFO:tasks.workunit.client.1.smithi130.stdout:6/322: mkdir d4/d15/d1f/d3c/d70 0 2024-05-13T22:48:29.431 INFO:tasks.workunit.client.1.smithi130.stdout:4/320: read d8/df/f20 [200147,38639] 0 2024-05-13T22:48:29.434 INFO:tasks.workunit.client.1.smithi130.stdout:0/331: creat d0/d7/d29/f74 x:0 0 0 2024-05-13T22:48:29.434 INFO:tasks.workunit.client.1.smithi130.stdout:9/319: dread d1/db/f35 [0,4194304] 0 2024-05-13T22:48:29.436 INFO:tasks.workunit.client.0.smithi102.stdout:7/980: creat d6/d15/d4d/d55/d6f/f136 x:0 0 0 2024-05-13T22:48:29.440 INFO:tasks.workunit.client.1.smithi130.stdout:5/284: sync 2024-05-13T22:48:29.440 INFO:tasks.workunit.client.1.smithi130.stdout:2/345: sync 2024-05-13T22:48:29.442 INFO:tasks.workunit.client.0.smithi102.stdout:7/981: link d6/d15/d4c/f8c d6/d15/d4d/d55/d6f/d11b/f137 0 2024-05-13T22:48:29.444 INFO:tasks.workunit.client.1.smithi130.stdout:9/320: mkdir d1/d7/d23/d28/d2d/d2f/d63 0 2024-05-13T22:48:29.445 INFO:tasks.workunit.client.1.smithi130.stdout:9/321: chown d1/d15/d51/f54 62033119 1 2024-05-13T22:48:29.446 INFO:tasks.workunit.client.0.smithi102.stdout:7/982: symlink d6/d15/d4d/d55/d7f/l138 0 2024-05-13T22:48:29.446 INFO:tasks.workunit.client.1.smithi130.stdout:7/258: dread d7/da/d1d/d42/f58 [0,4194304] 0 2024-05-13T22:48:29.450 INFO:tasks.workunit.client.0.smithi102.stdout:7/983: mkdir d6/dd0/df2/d105/d139 0 2024-05-13T22:48:29.451 INFO:tasks.workunit.client.1.smithi130.stdout:1/321: write f8 [1069300,21815] 0 2024-05-13T22:48:29.451 INFO:tasks.workunit.client.0.smithi102.stdout:7/984: write d6/d15/d4c/f10d [398131,30405] 0 2024-05-13T22:48:29.453 INFO:tasks.workunit.client.1.smithi130.stdout:2/346: creat d4/db/dc/d18/d2a/d66/f7f x:0 0 0 2024-05-13T22:48:29.454 INFO:tasks.workunit.client.1.smithi130.stdout:4/321: dread d8/dc/d29/f2c [0,4194304] 0 2024-05-13T22:48:29.455 INFO:tasks.workunit.client.1.smithi130.stdout:2/347: fdatasync d4/db/dc/f3b 0 2024-05-13T22:48:29.455 INFO:tasks.workunit.client.1.smithi130.stdout:8/268: creat d1/f65 x:0 0 0 2024-05-13T22:48:29.456 INFO:tasks.workunit.client.1.smithi130.stdout:3/305: dwrite d8/d1a/d1c/f5d [0,4194304] 0 2024-05-13T22:48:29.456 INFO:tasks.workunit.client.1.smithi130.stdout:9/322: creat d1/d7/d23/d28/f64 x:0 0 0 2024-05-13T22:48:29.457 INFO:tasks.workunit.client.0.smithi102.stdout:7/985: symlink d6/d13/d2c/da1/l13a 0 2024-05-13T22:48:29.462 INFO:tasks.workunit.client.1.smithi130.stdout:4/322: symlink d8/dc/d35/d33/d3f/l69 0 2024-05-13T22:48:29.463 INFO:tasks.workunit.client.1.smithi130.stdout:6/323: truncate d4/d15/f31 3888851 0 2024-05-13T22:48:29.464 INFO:tasks.workunit.client.1.smithi130.stdout:0/332: write d0/d7/f22 [2402570,93497] 0 2024-05-13T22:48:29.466 INFO:tasks.workunit.client.1.smithi130.stdout:7/259: symlink d7/d8/d9/l59 0 2024-05-13T22:48:29.467 INFO:tasks.workunit.client.1.smithi130.stdout:8/269: mknod d1/d2/db/d15/d18/d1a/c66 0 2024-05-13T22:48:29.468 INFO:tasks.workunit.client.0.smithi102.stdout:7/986: getdents d6/d13/d2c/d39/d114 0 2024-05-13T22:48:29.469 INFO:tasks.workunit.client.1.smithi130.stdout:8/270: truncate d1/d2/db/f40 372119 0 2024-05-13T22:48:29.469 INFO:tasks.workunit.client.1.smithi130.stdout:8/271: chown d1/d2 760452727 1 2024-05-13T22:48:29.470 INFO:tasks.workunit.client.1.smithi130.stdout:5/285: link d6/d1a/f40 d6/d7/f5e 0 2024-05-13T22:48:29.471 INFO:tasks.workunit.client.0.smithi102.stdout:7/987: mkdir d6/d22/d13b 0 2024-05-13T22:48:29.471 INFO:tasks.workunit.client.1.smithi130.stdout:1/322: mknod db/d1b/d1f/d49/d56/c6c 0 2024-05-13T22:48:29.472 INFO:tasks.workunit.client.1.smithi130.stdout:4/323: symlink d8/dc/d13/l6a 0 2024-05-13T22:48:29.473 INFO:tasks.workunit.client.1.smithi130.stdout:1/323: fdatasync db/d1b/d1f/d49/f48 0 2024-05-13T22:48:29.474 INFO:tasks.workunit.client.1.smithi130.stdout:0/333: creat d0/d7/dd/d2e/f75 x:0 0 0 2024-05-13T22:48:29.475 INFO:tasks.workunit.client.1.smithi130.stdout:1/324: dread - db/d1b/d1f/d49/f64 zero size 2024-05-13T22:48:29.475 INFO:tasks.workunit.client.1.smithi130.stdout:9/323: dwrite d1/d7/d23/d28/d32/d45/f53 [0,4194304] 0 2024-05-13T22:48:29.476 INFO:tasks.workunit.client.1.smithi130.stdout:7/260: symlink d7/da/d1f/d40/l5a 0 2024-05-13T22:48:29.478 INFO:tasks.workunit.client.1.smithi130.stdout:2/348: creat d4/db/dc/d18/f80 x:0 0 0 2024-05-13T22:48:29.480 INFO:tasks.workunit.client.1.smithi130.stdout:4/324: symlink d8/dc/d35/d33/d3f/l6b 0 2024-05-13T22:48:29.481 INFO:tasks.workunit.client.1.smithi130.stdout:1/325: fdatasync db/d1b/d1f/d2a/d35/f31 0 2024-05-13T22:48:29.481 INFO:tasks.workunit.client.1.smithi130.stdout:5/286: write d6/dd/f13 [4149378,130117] 0 2024-05-13T22:48:29.483 INFO:tasks.workunit.client.1.smithi130.stdout:0/334: mknod d0/d7/dd/d17/d52/d5b/c76 0 2024-05-13T22:48:29.483 INFO:tasks.workunit.client.1.smithi130.stdout:1/326: readlink db/d1b/d1f/d2a/d38/d66/l67 0 2024-05-13T22:48:29.484 INFO:tasks.workunit.client.0.smithi102.stdout:7/988: write d6/d13/d2c/f3a [1844674,91109] 0 2024-05-13T22:48:29.485 INFO:tasks.workunit.client.1.smithi130.stdout:7/261: mkdir d7/d8/d5b 0 2024-05-13T22:48:29.487 INFO:tasks.workunit.client.1.smithi130.stdout:5/287: unlink d6/d7/d32/f44 0 2024-05-13T22:48:29.488 INFO:tasks.workunit.client.1.smithi130.stdout:9/324: dread d1/d7/f57 [0,4194304] 0 2024-05-13T22:48:29.490 INFO:tasks.workunit.client.1.smithi130.stdout:7/262: stat d7/da/d1f/d40/l55 0 2024-05-13T22:48:29.494 INFO:tasks.workunit.client.1.smithi130.stdout:9/325: stat d1/d7/d23/d28/d32/l3e 0 2024-05-13T22:48:29.494 INFO:tasks.workunit.client.1.smithi130.stdout:1/327: chown db/d1b/d1f/d2a/c2d 30302276 1 2024-05-13T22:48:29.497 INFO:tasks.workunit.client.1.smithi130.stdout:2/349: write d4/fa [3398063,7687] 0 2024-05-13T22:48:29.498 INFO:tasks.workunit.client.1.smithi130.stdout:3/306: dwrite d8/dc/f37 [4194304,4194304] 0 2024-05-13T22:48:29.498 INFO:tasks.workunit.client.1.smithi130.stdout:4/325: link d8/dc/d13/l6a d8/dc/d35/d33/d44/d56/l6c 0 2024-05-13T22:48:29.500 INFO:tasks.workunit.client.1.smithi130.stdout:7/263: creat d7/da/f5c x:0 0 0 2024-05-13T22:48:29.501 INFO:tasks.workunit.client.1.smithi130.stdout:1/328: dread - db/d1b/d1f/d49/f64 zero size 2024-05-13T22:48:29.501 INFO:tasks.workunit.client.1.smithi130.stdout:0/335: dwrite d0/d7/f22 [0,4194304] 0 2024-05-13T22:48:29.502 INFO:tasks.workunit.client.1.smithi130.stdout:9/326: mkdir d1/d7/d23/d28/d32/d65 0 2024-05-13T22:48:29.503 INFO:tasks.workunit.client.1.smithi130.stdout:1/329: fdatasync db/d4a/f4b 0 2024-05-13T22:48:29.504 INFO:tasks.workunit.client.1.smithi130.stdout:3/307: chown d8/dc/d19/d2c/l32 312164622 1 2024-05-13T22:48:29.504 INFO:tasks.workunit.client.0.smithi102.stdout:7/989: dwrite d6/dd0/d80/fe2 [4194304,4194304] 0 2024-05-13T22:48:29.506 INFO:tasks.workunit.client.1.smithi130.stdout:5/288: dwrite d6/f54 [0,4194304] 0 2024-05-13T22:48:29.506 INFO:tasks.workunit.client.1.smithi130.stdout:2/350: truncate d4/db/f28 5873367 0 2024-05-13T22:48:29.508 INFO:tasks.workunit.client.1.smithi130.stdout:8/272: dread d1/d2/db/f8 [0,4194304] 0 2024-05-13T22:48:29.509 INFO:tasks.workunit.client.1.smithi130.stdout:0/336: chown d0/d7/d29/f37 30829 1 2024-05-13T22:48:29.510 INFO:tasks.workunit.client.1.smithi130.stdout:0/337: dread - d0/f53 zero size 2024-05-13T22:48:29.511 INFO:tasks.workunit.client.1.smithi130.stdout:1/330: chown f8 40 1 2024-05-13T22:48:29.514 INFO:tasks.workunit.client.1.smithi130.stdout:7/264: dread d7/da/f35 [0,4194304] 0 2024-05-13T22:48:29.515 INFO:tasks.workunit.client.1.smithi130.stdout:9/327: symlink d1/d7/d23/d3d/d43/l66 0 2024-05-13T22:48:29.515 INFO:tasks.workunit.client.1.smithi130.stdout:3/308: creat d8/dc/d15/f66 x:0 0 0 2024-05-13T22:48:29.516 INFO:tasks.workunit.client.1.smithi130.stdout:2/351: symlink d4/d49/l81 0 2024-05-13T22:48:29.519 INFO:tasks.workunit.client.1.smithi130.stdout:1/331: write db/d1b/f62 [402659,116468] 0 2024-05-13T22:48:29.520 INFO:tasks.workunit.client.1.smithi130.stdout:9/328: write d1/f5f [599706,124236] 0 2024-05-13T22:48:29.521 INFO:tasks.workunit.client.0.smithi102.stdout:7/990: dwrite d6/d13/f7d [0,4194304] 0 2024-05-13T22:48:29.521 INFO:tasks.workunit.client.1.smithi130.stdout:7/265: rmdir d7/d8/d39 39 2024-05-13T22:48:29.525 INFO:tasks.workunit.client.1.smithi130.stdout:8/273: mknod d1/d2/c67 0 2024-05-13T22:48:29.525 INFO:tasks.workunit.client.0.smithi102.stdout:7/991: unlink d6/d15/d4d/d55/d6f/f7c 0 2024-05-13T22:48:29.526 INFO:tasks.workunit.client.1.smithi130.stdout:5/289: creat d6/f5f x:0 0 0 2024-05-13T22:48:29.528 INFO:tasks.workunit.client.1.smithi130.stdout:9/329: chown d1/d7 24494 1 2024-05-13T22:48:29.530 INFO:tasks.workunit.client.1.smithi130.stdout:2/352: unlink d4/db/dc/f43 0 2024-05-13T22:48:29.530 INFO:tasks.workunit.client.1.smithi130.stdout:4/326: dwrite d8/df/f24 [0,4194304] 0 2024-05-13T22:48:29.530 INFO:tasks.workunit.client.1.smithi130.stdout:5/290: chown d6/d7/f43 13072238 1 2024-05-13T22:48:29.531 INFO:tasks.workunit.client.0.smithi102.stdout:7/992: dwrite d6/d15/d4d/d55/d82/feb [0,4194304] 0 2024-05-13T22:48:29.533 INFO:tasks.workunit.client.1.smithi130.stdout:9/330: chown d1/d7/d23 107 1 2024-05-13T22:48:29.534 INFO:tasks.workunit.client.0.smithi102.stdout:7/993: fdatasync d6/d13/d2c/da1/dc1/f5f 0 2024-05-13T22:48:29.535 INFO:tasks.workunit.client.1.smithi130.stdout:7/266: fsync d7/da/d1f/d2c/f38 0 2024-05-13T22:48:29.539 INFO:tasks.workunit.client.1.smithi130.stdout:0/338: getdents d0/d7/dd/d17/d1f/d2c/d5a 0 2024-05-13T22:48:29.541 INFO:tasks.workunit.client.1.smithi130.stdout:8/274: write d1/d2/db/d15/d16/f5b [551544,91632] 0 2024-05-13T22:48:29.541 INFO:tasks.workunit.client.0.smithi102.stdout:7/994: dwrite d6/d15/d4d/d55/d6f/d65/ded/f131 [0,4194304] 0 2024-05-13T22:48:29.542 INFO:tasks.workunit.client.1.smithi130.stdout:3/309: dwrite d8/f64 [0,4194304] 0 2024-05-13T22:48:29.543 INFO:tasks.workunit.client.1.smithi130.stdout:9/331: creat d1/d7/d23/d28/f67 x:0 0 0 2024-05-13T22:48:29.545 INFO:tasks.workunit.client.1.smithi130.stdout:1/332: dwrite db/d1b/d1f/d2a/f5a [0,4194304] 0 2024-05-13T22:48:29.549 INFO:tasks.workunit.client.1.smithi130.stdout:1/333: chown db/d1b/d1f/d49/f55 540118 1 2024-05-13T22:48:29.549 INFO:tasks.workunit.client.1.smithi130.stdout:7/267: mkdir d7/da/d1d/d5d 0 2024-05-13T22:48:29.550 INFO:tasks.workunit.client.1.smithi130.stdout:3/310: rmdir d8/d35 39 2024-05-13T22:48:29.552 INFO:tasks.workunit.client.1.smithi130.stdout:0/339: mkdir d0/d7/dd/d2e/d77 0 2024-05-13T22:48:29.552 INFO:tasks.workunit.client.1.smithi130.stdout:3/311: fsync d8/dc/f37 0 2024-05-13T22:48:29.554 INFO:tasks.workunit.client.1.smithi130.stdout:4/327: link d8/dc/d26/f55 d8/dc/d35/d33/d3f/f6d 0 2024-05-13T22:48:29.558 INFO:tasks.workunit.client.1.smithi130.stdout:9/332: creat d1/d7/d23/d28/d2d/d2f/d63/f68 x:0 0 0 2024-05-13T22:48:29.560 INFO:tasks.workunit.client.1.smithi130.stdout:5/291: read d6/d7/d24/f5b [2595256,108728] 0 2024-05-13T22:48:29.561 INFO:tasks.workunit.client.1.smithi130.stdout:1/334: mkdir db/d1b/d1f/d2a/d38/d46/d6d 0 2024-05-13T22:48:29.561 INFO:tasks.workunit.client.1.smithi130.stdout:2/353: dwrite d4/db/d26/f2d [4194304,4194304] 0 2024-05-13T22:48:29.562 INFO:tasks.workunit.client.1.smithi130.stdout:3/312: symlink d8/dc/d19/d27/l67 0 2024-05-13T22:48:29.563 INFO:tasks.workunit.client.1.smithi130.stdout:2/354: chown d4/db/d26 335337529 1 2024-05-13T22:48:29.563 INFO:tasks.workunit.client.1.smithi130.stdout:4/328: mkdir d8/dc/d35/d33/d6e 0 2024-05-13T22:48:29.564 INFO:tasks.workunit.client.1.smithi130.stdout:0/340: mkdir d0/d7/dd/d17/d1f/d2c/d45/d57/d78 0 2024-05-13T22:48:29.567 INFO:tasks.workunit.client.1.smithi130.stdout:7/268: dwrite d7/da/f35 [0,4194304] 0 2024-05-13T22:48:29.569 INFO:tasks.workunit.client.1.smithi130.stdout:9/333: truncate d1/d7/d23/d28/d2d/f34 1210100 0 2024-05-13T22:48:29.571 INFO:tasks.workunit.client.1.smithi130.stdout:1/335: chown db/l29 280611148 1 2024-05-13T22:48:29.576 INFO:tasks.workunit.client.1.smithi130.stdout:5/292: creat d6/d7/d19/d4d/f60 x:0 0 0 2024-05-13T22:48:29.576 INFO:tasks.workunit.client.1.smithi130.stdout:3/313: dread d8/dc/d15/f18 [0,4194304] 0 2024-05-13T22:48:29.579 INFO:tasks.workunit.client.1.smithi130.stdout:2/355: rmdir d4/db 39 2024-05-13T22:48:29.580 INFO:tasks.workunit.client.1.smithi130.stdout:5/293: chown d6/d7/f28 5706521 1 2024-05-13T22:48:29.580 INFO:tasks.workunit.client.0.smithi102.stdout:7/995: rename d6/d15/d4d/d55 to d6/d13/d2c/d39/d114/d13c 0 2024-05-13T22:48:29.581 INFO:tasks.workunit.client.1.smithi130.stdout:4/329: dwrite d8/f43 [0,4194304] 0 2024-05-13T22:48:29.581 INFO:tasks.workunit.client.1.smithi130.stdout:3/314: write d8/dc/f3e [307592,2966] 0 2024-05-13T22:48:29.583 INFO:tasks.workunit.client.1.smithi130.stdout:7/269: dwrite d7/d8/f2d [4194304,4194304] 0 2024-05-13T22:48:29.587 INFO:tasks.workunit.client.1.smithi130.stdout:7/270: fsync d7/da/d1f/d2c/f31 0 2024-05-13T22:48:29.588 INFO:tasks.workunit.client.0.smithi102.stdout:7/996: dwrite d6/d13/d58/f10a [0,4194304] 0 2024-05-13T22:48:29.595 INFO:tasks.workunit.client.0.smithi102.stdout:9/990: sync 2024-05-13T22:48:29.595 INFO:tasks.workunit.client.0.smithi102.stdout:1/904: sync 2024-05-13T22:48:29.595 INFO:tasks.workunit.client.1.smithi130.stdout:2/356: creat d4/d49/d48/d4a/f82 x:0 0 0 2024-05-13T22:48:29.597 INFO:tasks.workunit.client.1.smithi130.stdout:9/334: dwrite d1/d7/d23/d3d/f40 [0,4194304] 0 2024-05-13T22:48:29.600 INFO:tasks.workunit.client.1.smithi130.stdout:1/336: mknod db/c6e 0 2024-05-13T22:48:29.600 INFO:tasks.workunit.client.1.smithi130.stdout:6/324: sync 2024-05-13T22:48:29.603 INFO:tasks.workunit.client.1.smithi130.stdout:5/294: dwrite d6/d7/f46 [0,4194304] 0 2024-05-13T22:48:29.603 INFO:tasks.workunit.client.1.smithi130.stdout:9/335: write d1/d7/d23/d3d/f40 [1632848,12924] 0 2024-05-13T22:48:29.603 INFO:tasks.workunit.client.1.smithi130.stdout:0/341: creat d0/d7/dd/f79 x:0 0 0 2024-05-13T22:48:29.604 INFO:tasks.workunit.client.1.smithi130.stdout:3/315: dread d8/dc/f22 [0,4194304] 0 2024-05-13T22:48:29.606 INFO:tasks.workunit.client.1.smithi130.stdout:6/325: readlink d4/d15/d1d/l5a 0 2024-05-13T22:48:29.606 INFO:tasks.workunit.client.1.smithi130.stdout:1/337: chown db/f1a 6059179 1 2024-05-13T22:48:29.608 INFO:tasks.workunit.client.1.smithi130.stdout:6/326: chown d4/l41 71281 1 2024-05-13T22:48:29.610 INFO:tasks.workunit.client.1.smithi130.stdout:4/330: getdents d8/dc/d35/d33/d3f/d57/d60 0 2024-05-13T22:48:29.611 INFO:tasks.workunit.client.1.smithi130.stdout:2/357: dwrite d4/f5f [0,4194304] 0 2024-05-13T22:48:29.613 INFO:tasks.workunit.client.1.smithi130.stdout:1/338: dread db/d1b/d1f/d2a/f41 [0,4194304] 0 2024-05-13T22:48:29.614 INFO:tasks.workunit.client.1.smithi130.stdout:7/271: unlink d7/d8/d9/lf 0 2024-05-13T22:48:29.619 INFO:tasks.workunit.client.1.smithi130.stdout:5/295: mknod d6/d7/d24/c61 0 2024-05-13T22:48:29.620 INFO:tasks.workunit.client.1.smithi130.stdout:5/296: chown d6/d7/l20 17 1 2024-05-13T22:48:29.623 INFO:tasks.workunit.client.1.smithi130.stdout:6/327: rmdir d4/d15/d1d/d60 39 2024-05-13T22:48:29.624 INFO:tasks.workunit.client.1.smithi130.stdout:6/328: read d4/f29 [3076951,6242] 0 2024-05-13T22:48:29.627 INFO:tasks.workunit.client.1.smithi130.stdout:7/272: truncate d7/da/f33 1375976 0 2024-05-13T22:48:29.628 INFO:tasks.workunit.client.1.smithi130.stdout:7/273: fsync d7/da/d1d/d42/d43/f4d 0 2024-05-13T22:48:29.629 INFO:tasks.workunit.client.1.smithi130.stdout:9/336: mkdir d1/d7/d69 0 2024-05-13T22:48:29.630 INFO:tasks.workunit.client.1.smithi130.stdout:3/316: mknod d8/dc/c68 0 2024-05-13T22:48:29.635 INFO:tasks.workunit.client.1.smithi130.stdout:6/329: rename d4/d15/d1f/d46/f65 to d4/d15/d1d/d28/f71 0 2024-05-13T22:48:29.636 INFO:tasks.workunit.client.0.smithi102.stdout:7/997: fsync d6/d22/f33 0 2024-05-13T22:48:29.636 INFO:tasks.workunit.client.1.smithi130.stdout:1/339: mkdir db/d1b/d1f/d2a/d38/d46/d6d/d6f 0 2024-05-13T22:48:29.639 INFO:tasks.workunit.client.0.smithi102.stdout:9/991: truncate d1/d6/d21/d12e/d81/f97 482158 0 2024-05-13T22:48:29.642 INFO:tasks.workunit.client.1.smithi130.stdout:9/337: creat d1/d7/d23/d28/d2d/d2f/f6a x:0 0 0 2024-05-13T22:48:29.645 INFO:tasks.workunit.client.0.smithi102.stdout:9/992: rmdir d1/d6/d21/d12e/d75/d96/dc6/dd9/d125/d14e 39 2024-05-13T22:48:29.646 INFO:tasks.workunit.client.0.smithi102.stdout:9/993: chown d1/d6/d45/d3e/d79/fae 735 1 2024-05-13T22:48:29.648 INFO:tasks.workunit.client.1.smithi130.stdout:2/358: link d4/f3f d4/d49/d42/d4b/f83 0 2024-05-13T22:48:29.651 INFO:tasks.workunit.client.0.smithi102.stdout:9/994: rename d1/d6/d21/d12e/d81/d116 to d1/d6/d45/d3e/d79/d15b 0 2024-05-13T22:48:29.652 INFO:tasks.workunit.client.1.smithi130.stdout:6/330: mkdir d4/d15/d1d/d28/d72 0 2024-05-13T22:48:29.654 INFO:tasks.workunit.client.1.smithi130.stdout:7/274: creat d7/da/d1d/d5d/f5e x:0 0 0 2024-05-13T22:48:29.655 INFO:tasks.workunit.client.0.smithi102.stdout:9/995: symlink d1/d6/d21/d12e/d64/d152/d6c/l15c 0 2024-05-13T22:48:29.657 INFO:tasks.workunit.client.1.smithi130.stdout:9/338: mkdir d1/d7/d23/d28/d2d/d2f/d63/d6b 0 2024-05-13T22:48:29.657 INFO:tasks.workunit.client.1.smithi130.stdout:9/339: readlink d1/d7/d23/d28/l2e 0 2024-05-13T22:48:29.659 INFO:tasks.workunit.client.1.smithi130.stdout:5/297: rename d6/c47 to d6/c62 0 2024-05-13T22:48:29.659 INFO:tasks.workunit.client.0.smithi102.stdout:1/905: dread d2/d45/d79/d92/f61 [0,4194304] 0 2024-05-13T22:48:29.661 INFO:tasks.workunit.client.0.smithi102.stdout:1/906: chown d2/d45/d96/d113/f122 5660 1 2024-05-13T22:48:29.661 INFO:tasks.workunit.client.1.smithi130.stdout:2/359: write d4/d49/f35 [4690339,10824] 0 2024-05-13T22:48:29.663 INFO:tasks.workunit.client.1.smithi130.stdout:6/331: fsync d4/d15/d1d/d60/f66 0 2024-05-13T22:48:29.664 INFO:tasks.workunit.client.1.smithi130.stdout:7/275: symlink d7/da/d1d/d5d/l5f 0 2024-05-13T22:48:29.665 INFO:tasks.workunit.client.0.smithi102.stdout:7/998: dread d6/d13/d2c/d39/d114/d13c/d6f/d65/f86 [0,4194304] 0 2024-05-13T22:48:29.667 INFO:tasks.workunit.client.1.smithi130.stdout:7/276: dread d7/da/d1f/f30 [0,4194304] 0 2024-05-13T22:48:29.668 INFO:tasks.workunit.client.0.smithi102.stdout:1/907: dwrite d2/d10/fee [0,4194304] 0 2024-05-13T22:48:29.669 INFO:tasks.workunit.client.1.smithi130.stdout:5/298: symlink d6/d7/d19/l63 0 2024-05-13T22:48:29.671 INFO:tasks.workunit.client.0.smithi102.stdout:9/996: creat d1/d6/d21/d12e/d75/d96/f15d x:0 0 0 2024-05-13T22:48:29.673 INFO:tasks.workunit.client.1.smithi130.stdout:2/360: read d4/db/dc/f37 [814435,61257] 0 2024-05-13T22:48:29.675 INFO:tasks.workunit.client.1.smithi130.stdout:9/340: mknod d1/d7/d23/d28/d2d/d2f/d63/d6b/c6c 0 2024-05-13T22:48:29.679 INFO:tasks.workunit.client.1.smithi130.stdout:8/275: sync 2024-05-13T22:48:29.682 INFO:tasks.workunit.client.0.smithi102.stdout:7/999: link d6/d13/d2c/d39/d114/d13c/d6f/d65/ded/f131 d6/d13/d2c/d39/d114/d13c/d6f/d102/f13d 0 2024-05-13T22:48:29.686 INFO:tasks.workunit.client.1.smithi130.stdout:7/277: rename d7/d8/d9/l32 to d7/da/d1d/d3d/d46/l60 0 2024-05-13T22:48:29.687 INFO:tasks.workunit.client.1.smithi130.stdout:7/278: read - d7/da/f56 zero size 2024-05-13T22:48:29.689 INFO:tasks.workunit.client.1.smithi130.stdout:5/299: symlink d6/l64 0 2024-05-13T22:48:29.689 INFO:tasks.workunit.client.0.smithi102.stdout:9/997: getdents d1/d6/db/d30/d47 0 2024-05-13T22:48:29.690 INFO:tasks.workunit.client.1.smithi130.stdout:8/276: symlink d1/d2/db/d15/d16/d54/l68 0 2024-05-13T22:48:29.692 INFO:tasks.workunit.client.1.smithi130.stdout:2/361: mknod d4/d49/d48/d4a/d76/c84 0 2024-05-13T22:48:29.694 INFO:tasks.workunit.client.1.smithi130.stdout:5/300: dwrite d6/d7/f28 [0,4194304] 0 2024-05-13T22:48:29.694 INFO:tasks.workunit.client.1.smithi130.stdout:2/362: dread - d4/db/dc/d18/d2a/d30/d69/f6e zero size 2024-05-13T22:48:29.696 INFO:tasks.workunit.client.1.smithi130.stdout:5/301: write d6/d7/d24/f41 [867120,38872] 0 2024-05-13T22:48:29.697 INFO:tasks.workunit.client.0.smithi102.stdout:9/998: mknod d1/d6/d21/d12e/d75/d96/c15e 0 2024-05-13T22:48:29.697 INFO:tasks.workunit.client.0.smithi102.stdout:9/999: dread - d1/d6/d45/d3e/f14a zero size 2024-05-13T22:48:29.700 INFO:tasks.workunit.client.1.smithi130.stdout:7/279: dwrite d7/da/d1d/d42/f58 [4194304,4194304] 0 2024-05-13T22:48:29.701 INFO:tasks.workunit.client.1.smithi130.stdout:7/280: chown d7/da/d1f/d2c 0 1 2024-05-13T22:48:29.702 INFO:tasks.workunit.client.1.smithi130.stdout:8/277: rename d1/c34 to d1/d2/db/d15/d18/d42/d53/c69 0 2024-05-13T22:48:29.702 INFO:tasks.workunit.client.1.smithi130.stdout:7/281: chown d7/da/d1d/d42/l52 34 1 2024-05-13T22:48:29.705 INFO:tasks.workunit.client.1.smithi130.stdout:2/363: rename c1 to d4/d49/d48/d4a/d76/c85 0 2024-05-13T22:48:29.707 INFO:tasks.workunit.client.1.smithi130.stdout:8/278: symlink d1/d2/db/d15/d18/d1a/l6a 0 2024-05-13T22:48:29.710 INFO:tasks.workunit.client.1.smithi130.stdout:7/282: dwrite d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:29.714 INFO:tasks.workunit.client.1.smithi130.stdout:5/302: rename d6/f54 to d6/d7/d24/f65 0 2024-05-13T22:48:29.714 INFO:tasks.workunit.client.1.smithi130.stdout:0/342: dwrite d0/d7/dd/d2e/f2f [0,4194304] 0 2024-05-13T22:48:29.717 INFO:tasks.workunit.client.1.smithi130.stdout:7/283: rmdir d7/da/d1d/d3d 39 2024-05-13T22:48:29.719 INFO:tasks.workunit.client.1.smithi130.stdout:5/303: mknod d6/d1a/c66 0 2024-05-13T22:48:29.720 INFO:tasks.workunit.client.1.smithi130.stdout:0/343: chown d0/fe 37645 1 2024-05-13T22:48:29.722 INFO:tasks.workunit.client.1.smithi130.stdout:5/304: rmdir d6/d7/d24 39 2024-05-13T22:48:29.723 INFO:tasks.workunit.client.1.smithi130.stdout:8/279: dwrite d1/d2/db/d15/d18/f31 [4194304,4194304] 0 2024-05-13T22:48:29.725 INFO:tasks.workunit.client.1.smithi130.stdout:8/280: chown d1/d2/db/d15/d16/d54/f55 41105 1 2024-05-13T22:48:29.725 INFO:tasks.workunit.client.1.smithi130.stdout:7/284: mknod d7/da/d1d/c61 0 2024-05-13T22:48:29.726 INFO:tasks.workunit.client.1.smithi130.stdout:7/285: chown d7/da/f35 1 1 2024-05-13T22:48:29.727 INFO:tasks.workunit.client.1.smithi130.stdout:0/344: dread d0/d7/dd/d17/d1f/f25 [4194304,4194304] 0 2024-05-13T22:48:29.728 INFO:tasks.workunit.client.1.smithi130.stdout:5/305: write d6/d7/f25 [3202209,127495] 0 2024-05-13T22:48:29.729 INFO:tasks.workunit.client.1.smithi130.stdout:5/306: chown d6/dd/f3e 495 1 2024-05-13T22:48:29.735 INFO:tasks.workunit.client.1.smithi130.stdout:0/345: rename d0/d7/d29/f33 to d0/d7/dd/d17/d1f/d2c/f7a 0 2024-05-13T22:48:29.736 INFO:tasks.workunit.client.1.smithi130.stdout:7/286: dwrite d7/da/d1f/d2c/f31 [0,4194304] 0 2024-05-13T22:48:29.737 INFO:tasks.workunit.client.1.smithi130.stdout:8/281: link d1/d2/c67 d1/d2/db/d15/d16/d54/d58/c6b 0 2024-05-13T22:48:29.737 INFO:tasks.workunit.client.1.smithi130.stdout:4/331: sync 2024-05-13T22:48:29.741 INFO:tasks.workunit.client.1.smithi130.stdout:0/346: rmdir d0/d7/dd/d17/d1f/d2c/d45 39 2024-05-13T22:48:29.742 INFO:tasks.workunit.client.1.smithi130.stdout:3/317: dwrite d8/f16 [0,4194304] 0 2024-05-13T22:48:29.747 INFO:tasks.workunit.client.1.smithi130.stdout:4/332: mkdir d8/dc/d35/d33/d3f/d57/d6f 0 2024-05-13T22:48:29.749 INFO:tasks.workunit.client.1.smithi130.stdout:8/282: mknod d1/d2/db/d15/d18/d42/c6c 0 2024-05-13T22:48:29.752 INFO:tasks.workunit.client.1.smithi130.stdout:0/347: mkdir d0/d7/d29/d7b 0 2024-05-13T22:48:29.753 INFO:tasks.workunit.client.1.smithi130.stdout:3/318: creat d8/dc/d19/d2c/d61/f69 x:0 0 0 2024-05-13T22:48:29.755 INFO:tasks.workunit.client.0.smithi102.stdout:1/908: write d2/d45/d79/d92/f61 [2691053,56378] 0 2024-05-13T22:48:29.755 INFO:tasks.workunit.client.1.smithi130.stdout:2/364: dread d4/db/dc/d18/d58/f7b [0,4194304] 0 2024-05-13T22:48:29.756 INFO:tasks.workunit.client.0.smithi102.stdout:1/909: write d2/d10/fa0 [59297,31231] 0 2024-05-13T22:48:29.757 INFO:tasks.workunit.client.1.smithi130.stdout:7/287: creat d7/da/f62 x:0 0 0 2024-05-13T22:48:29.757 INFO:tasks.workunit.client.0.smithi102.stdout:1/910: write d2/d45/fd6 [5520779,112193] 0 2024-05-13T22:48:29.764 INFO:tasks.workunit.client.1.smithi130.stdout:9/341: dwrite d1/d15/f25 [0,4194304] 0 2024-05-13T22:48:29.766 INFO:tasks.workunit.client.0.smithi102.stdout:1/911: rmdir d2/d45/df4 39 2024-05-13T22:48:29.766 INFO:tasks.workunit.client.0.smithi102.stdout:1/912: dread - d2/d10/d62/d86/f109 zero size 2024-05-13T22:48:29.767 INFO:tasks.workunit.client.1.smithi130.stdout:2/365: creat d4/d49/d48/d4a/d64/f86 x:0 0 0 2024-05-13T22:48:29.773 INFO:tasks.workunit.client.1.smithi130.stdout:9/342: creat d1/d15/d51/f6d x:0 0 0 2024-05-13T22:48:29.773 INFO:tasks.workunit.client.1.smithi130.stdout:6/332: sync 2024-05-13T22:48:29.773 INFO:tasks.workunit.client.1.smithi130.stdout:1/340: sync 2024-05-13T22:48:29.773 INFO:tasks.workunit.client.0.smithi102.stdout:1/913: dwrite d2/d10/d62/f88 [4194304,4194304] 0 2024-05-13T22:48:29.776 INFO:tasks.workunit.client.1.smithi130.stdout:2/366: creat d4/db/dc/d18/d2a/d30/d69/f87 x:0 0 0 2024-05-13T22:48:29.778 INFO:tasks.workunit.client.1.smithi130.stdout:2/367: readlink d4/l57 0 2024-05-13T22:48:29.778 INFO:tasks.workunit.client.1.smithi130.stdout:0/348: creat d0/d7/dd/d17/d1f/d2c/d45/d6b/f7c x:0 0 0 2024-05-13T22:48:29.778 INFO:tasks.workunit.client.1.smithi130.stdout:4/333: dread d8/df/f20 [0,4194304] 0 2024-05-13T22:48:29.779 INFO:tasks.workunit.client.0.smithi102.stdout:1/914: creat d2/d4/d26/d40/f124 x:0 0 0 2024-05-13T22:48:29.779 INFO:tasks.workunit.client.0.smithi102.stdout:1/915: fdatasync d2/d10/fa0 0 2024-05-13T22:48:29.780 INFO:tasks.workunit.client.1.smithi130.stdout:4/334: write d8/f12 [651738,60399] 0 2024-05-13T22:48:29.780 INFO:tasks.workunit.client.1.smithi130.stdout:9/343: dwrite d1/d15/f16 [0,4194304] 0 2024-05-13T22:48:29.784 INFO:tasks.workunit.client.1.smithi130.stdout:1/341: creat db/d4a/f70 x:0 0 0 2024-05-13T22:48:29.785 INFO:tasks.workunit.client.1.smithi130.stdout:5/307: getdents d6/d1a 0 2024-05-13T22:48:29.786 INFO:tasks.workunit.client.1.smithi130.stdout:2/368: dread d4/db/dc/d18/d2a/d30/f34 [0,4194304] 0 2024-05-13T22:48:29.788 INFO:tasks.workunit.client.1.smithi130.stdout:4/335: dread - d8/dc/d35/d33/d3f/f6d zero size 2024-05-13T22:48:29.789 INFO:tasks.workunit.client.0.smithi102.stdout:1/916: mkdir d2/d45/df4/d125 0 2024-05-13T22:48:29.789 INFO:tasks.workunit.client.1.smithi130.stdout:3/319: dread d8/dc/f12 [0,4194304] 0 2024-05-13T22:48:29.790 INFO:tasks.workunit.client.1.smithi130.stdout:1/342: chown db/d1b/d1f/d2a/f4f 1 1 2024-05-13T22:48:29.790 INFO:tasks.workunit.client.0.smithi102.stdout:1/917: stat d2/d4/d5/d19/dd3/d11b 0 2024-05-13T22:48:29.791 INFO:tasks.workunit.client.0.smithi102.stdout:1/918: chown d2/d4/fb5 261048366 1 2024-05-13T22:48:29.791 INFO:tasks.workunit.client.1.smithi130.stdout:4/336: dread - d8/dc/f28 zero size 2024-05-13T22:48:29.792 INFO:tasks.workunit.client.1.smithi130.stdout:1/343: dread - db/d1b/d1f/d2a/d35/f69 zero size 2024-05-13T22:48:29.794 INFO:tasks.workunit.client.1.smithi130.stdout:9/344: dwrite d1/f11 [0,4194304] 0 2024-05-13T22:48:29.794 INFO:tasks.workunit.client.1.smithi130.stdout:7/288: getdents d7/da/d1f/d40 0 2024-05-13T22:48:29.794 INFO:tasks.workunit.client.1.smithi130.stdout:3/320: write d8/dc/d15/f66 [663908,22531] 0 2024-05-13T22:48:29.798 INFO:tasks.workunit.client.1.smithi130.stdout:6/333: creat d4/f73 x:0 0 0 2024-05-13T22:48:29.798 INFO:tasks.workunit.client.1.smithi130.stdout:8/283: write d1/d2/db/f40 [1230737,38796] 0 2024-05-13T22:48:29.804 INFO:tasks.workunit.client.1.smithi130.stdout:7/289: creat d7/da/d1f/d2c/f63 x:0 0 0 2024-05-13T22:48:29.805 INFO:tasks.workunit.client.1.smithi130.stdout:3/321: mknod d8/d1a/d1c/c6a 0 2024-05-13T22:48:29.806 INFO:tasks.workunit.client.1.smithi130.stdout:2/369: dwrite d4/db/dc/d18/d58/f59 [0,4194304] 0 2024-05-13T22:48:29.808 INFO:tasks.workunit.client.1.smithi130.stdout:7/290: read d7/d8/f2d [5629524,13055] 0 2024-05-13T22:48:29.808 INFO:tasks.workunit.client.1.smithi130.stdout:1/344: readlink db/d1b/d1f/d2a/l3e 0 2024-05-13T22:48:29.809 INFO:tasks.workunit.client.0.smithi102.stdout:1/919: link d2/d4/d26/da2/cdc d2/d4/d9e/c126 0 2024-05-13T22:48:29.811 INFO:tasks.workunit.client.1.smithi130.stdout:6/334: rmdir d4/d15 39 2024-05-13T22:48:29.812 INFO:tasks.workunit.client.1.smithi130.stdout:9/345: dwrite d1/d7/d23/d28/d2d/f48 [0,4194304] 0 2024-05-13T22:48:29.815 INFO:tasks.workunit.client.1.smithi130.stdout:8/284: dread d1/d2/db/d15/d18/f4c [0,4194304] 0 2024-05-13T22:48:29.818 INFO:tasks.workunit.client.1.smithi130.stdout:3/322: chown d8/dc/d19/d2c/l3a 128230 1 2024-05-13T22:48:29.819 INFO:tasks.workunit.client.1.smithi130.stdout:9/346: dread d1/f5f [0,4194304] 0 2024-05-13T22:48:29.819 INFO:tasks.workunit.client.1.smithi130.stdout:8/285: truncate d1/d2/db/d15/d18/d42/f62 5018396 0 2024-05-13T22:48:29.819 INFO:tasks.workunit.client.0.smithi102.stdout:1/920: rename d2/d4/d26/d40/d10e to d2/d4/d5/d19/dbe/d127 0 2024-05-13T22:48:29.820 INFO:tasks.workunit.client.0.smithi102.stdout:1/921: dread - d2/d10/f11c zero size 2024-05-13T22:48:29.821 INFO:tasks.workunit.client.1.smithi130.stdout:7/291: creat d7/da/d1d/d42/f64 x:0 0 0 2024-05-13T22:48:29.824 INFO:tasks.workunit.client.1.smithi130.stdout:1/345: creat db/d1b/d1f/d2a/d38/d46/f71 x:0 0 0 2024-05-13T22:48:29.826 INFO:tasks.workunit.client.1.smithi130.stdout:5/308: dread d6/d1a/f2a [0,4194304] 0 2024-05-13T22:48:29.827 INFO:tasks.workunit.client.1.smithi130.stdout:9/347: dwrite d1/d7/d23/d28/d32/d45/f53 [0,4194304] 0 2024-05-13T22:48:29.830 INFO:tasks.workunit.client.1.smithi130.stdout:3/323: unlink d8/dc/d19/d27/c2a 0 2024-05-13T22:48:29.833 INFO:tasks.workunit.client.1.smithi130.stdout:6/335: rename d4/f37 to d4/d15/d1f/f74 0 2024-05-13T22:48:29.833 INFO:tasks.workunit.client.1.smithi130.stdout:4/337: getdents d8/dc 0 2024-05-13T22:48:29.836 INFO:tasks.workunit.client.1.smithi130.stdout:3/324: read d8/dc/d19/f30 [517892,5668] 0 2024-05-13T22:48:29.836 INFO:tasks.workunit.client.1.smithi130.stdout:5/309: rename d6/d7/d19/d4d/l4c to d6/d1a/l67 0 2024-05-13T22:48:29.837 INFO:tasks.workunit.client.1.smithi130.stdout:1/346: mkdir db/d1b/d1f/d72 0 2024-05-13T22:48:29.839 INFO:tasks.workunit.client.1.smithi130.stdout:9/348: dwrite d1/d15/f3b [0,4194304] 0 2024-05-13T22:48:29.841 INFO:tasks.workunit.client.1.smithi130.stdout:3/325: fdatasync d8/dc/f22 0 2024-05-13T22:48:29.843 INFO:tasks.workunit.client.1.smithi130.stdout:6/336: dread d4/d15/d1d/d28/f30 [0,4194304] 0 2024-05-13T22:48:29.844 INFO:tasks.workunit.client.1.smithi130.stdout:1/347: rename db/d1b/f62 to db/d1b/d1f/d2a/d38/d46/d6d/f73 0 2024-05-13T22:48:29.846 INFO:tasks.workunit.client.1.smithi130.stdout:5/310: rename d6/d7/d24/f41 to d6/d7/d24/f68 0 2024-05-13T22:48:29.848 INFO:tasks.workunit.client.1.smithi130.stdout:6/337: mkdir d4/d15/d1d/d28/d75 0 2024-05-13T22:48:29.850 INFO:tasks.workunit.client.1.smithi130.stdout:9/349: dwrite d1/f3 [0,4194304] 0 2024-05-13T22:48:29.851 INFO:tasks.workunit.client.1.smithi130.stdout:3/326: rename d8/d35/f2e to d8/dc/d19/d2c/f6b 0 2024-05-13T22:48:29.852 INFO:tasks.workunit.client.0.smithi102.stdout:1/922: truncate d2/d4/d5/d19/ded/d6a/dbd/ffe 1596720 0 2024-05-13T22:48:29.852 INFO:tasks.workunit.client.1.smithi130.stdout:1/348: dread db/d1b/d1f/d2a/d38/d46/d6d/f73 [0,4194304] 0 2024-05-13T22:48:29.853 INFO:tasks.workunit.client.0.smithi102.stdout:1/923: chown d2/d10/c5c 4677757 1 2024-05-13T22:48:29.853 INFO:tasks.workunit.client.1.smithi130.stdout:9/350: chown d1/d7/d23/d28/d2d/f3a 3774003 1 2024-05-13T22:48:29.855 INFO:tasks.workunit.client.1.smithi130.stdout:6/338: creat d4/d15/d1f/d46/d52/f76 x:0 0 0 2024-05-13T22:48:29.856 INFO:tasks.workunit.client.0.smithi102.stdout:1/924: unlink d2/d10/d62/d7b/f82 0 2024-05-13T22:48:29.856 INFO:tasks.workunit.client.1.smithi130.stdout:5/311: dread d6/d7/f28 [0,4194304] 0 2024-05-13T22:48:29.856 INFO:tasks.workunit.client.1.smithi130.stdout:1/349: write db/d1b/d1f/f53 [1081446,17229] 0 2024-05-13T22:48:29.857 INFO:tasks.workunit.client.1.smithi130.stdout:3/327: dread d8/dc/d19/f30 [0,4194304] 0 2024-05-13T22:48:29.858 INFO:tasks.workunit.client.1.smithi130.stdout:1/350: chown db/d1b/d1f/d2a/d38 0 1 2024-05-13T22:48:29.860 INFO:tasks.workunit.client.1.smithi130.stdout:9/351: write d1/d7/d23/d28/d32/f62 [520065,75366] 0 2024-05-13T22:48:29.861 INFO:tasks.workunit.client.1.smithi130.stdout:6/339: creat d4/d15/d1d/d60/f77 x:0 0 0 2024-05-13T22:48:29.862 INFO:tasks.workunit.client.1.smithi130.stdout:6/340: write d4/d15/d1d/d28/f34 [2622739,24365] 0 2024-05-13T22:48:29.863 INFO:tasks.workunit.client.1.smithi130.stdout:3/328: creat d8/d1a/d1c/f6c x:0 0 0 2024-05-13T22:48:29.863 INFO:tasks.workunit.client.0.smithi102.stdout:1/925: dwrite d2/d4/d26/fe6 [4194304,4194304] 0 2024-05-13T22:48:29.866 INFO:tasks.workunit.client.1.smithi130.stdout:1/351: fsync db/f32 0 2024-05-13T22:48:29.866 INFO:tasks.workunit.client.1.smithi130.stdout:4/338: truncate d8/dc/d26/f36 2682668 0 2024-05-13T22:48:29.867 INFO:tasks.workunit.client.1.smithi130.stdout:4/339: write d8/dc/d29/f3a [3813510,34057] 0 2024-05-13T22:48:29.878 INFO:tasks.workunit.client.1.smithi130.stdout:6/341: rmdir d4/d15/d1f/d46 39 2024-05-13T22:48:29.880 INFO:tasks.workunit.client.1.smithi130.stdout:1/352: mkdir db/d58/d74 0 2024-05-13T22:48:29.882 INFO:tasks.workunit.client.1.smithi130.stdout:6/342: symlink d4/d15/d1d/d63/l78 0 2024-05-13T22:48:29.883 INFO:tasks.workunit.client.1.smithi130.stdout:6/343: write d4/d16/f69 [1497365,51005] 0 2024-05-13T22:48:29.883 INFO:tasks.workunit.client.1.smithi130.stdout:1/353: mkdir db/d1b/d1f/d72/d75 0 2024-05-13T22:48:29.883 INFO:tasks.workunit.client.1.smithi130.stdout:6/344: chown d4/d15/d1d 287431817 1 2024-05-13T22:48:29.884 INFO:tasks.workunit.client.1.smithi130.stdout:1/354: truncate db/f1d 4880217 0 2024-05-13T22:48:29.886 INFO:tasks.workunit.client.0.smithi102.stdout:1/926: write d2/d4/d5/f36 [1913620,116075] 0 2024-05-13T22:48:29.886 INFO:tasks.workunit.client.1.smithi130.stdout:1/355: dread db/d1b/d1f/d49/f54 [0,4194304] 0 2024-05-13T22:48:29.889 INFO:tasks.workunit.client.0.smithi102.stdout:1/927: rmdir d2/d4/d38/db2 39 2024-05-13T22:48:29.890 INFO:tasks.workunit.client.1.smithi130.stdout:1/356: mknod db/d1b/d1f/d2a/d38/c76 0 2024-05-13T22:48:29.893 INFO:tasks.workunit.client.1.smithi130.stdout:3/329: truncate d8/dc/d15/f66 268001 0 2024-05-13T22:48:29.895 INFO:tasks.workunit.client.1.smithi130.stdout:1/357: symlink db/d1b/d1f/d2a/d38/d46/l77 0 2024-05-13T22:48:29.896 INFO:tasks.workunit.client.1.smithi130.stdout:4/340: dwrite d8/dc/d35/d33/d44/f54 [0,4194304] 0 2024-05-13T22:48:29.896 INFO:tasks.workunit.client.1.smithi130.stdout:1/358: fsync db/d4a/f70 0 2024-05-13T22:48:29.898 INFO:tasks.workunit.client.1.smithi130.stdout:6/345: write d4/d15/d1f/f2f [93370,66569] 0 2024-05-13T22:48:29.899 INFO:tasks.workunit.client.1.smithi130.stdout:1/359: symlink db/d1b/d1f/d2a/d38/l78 0 2024-05-13T22:48:29.903 INFO:tasks.workunit.client.1.smithi130.stdout:4/341: dread d8/dc/d35/d33/f4f [0,4194304] 0 2024-05-13T22:48:29.904 INFO:tasks.workunit.client.1.smithi130.stdout:1/360: creat db/d1b/d1f/d72/f79 x:0 0 0 2024-05-13T22:48:29.904 INFO:tasks.workunit.client.1.smithi130.stdout:3/330: link d8/d1a/c65 d8/d35/c6d 0 2024-05-13T22:48:29.908 INFO:tasks.workunit.client.1.smithi130.stdout:4/342: read d8/fa [112317,19617] 0 2024-05-13T22:48:29.909 INFO:tasks.workunit.client.1.smithi130.stdout:3/331: dread d8/dc/d15/f4d [0,4194304] 0 2024-05-13T22:48:29.909 INFO:tasks.workunit.client.0.smithi102.stdout:1/928: dwrite d2/d10/d62/fa8 [0,4194304] 0 2024-05-13T22:48:29.910 INFO:tasks.workunit.client.1.smithi130.stdout:4/343: write d8/dc/d35/f50 [1001564,57315] 0 2024-05-13T22:48:29.911 INFO:tasks.workunit.client.1.smithi130.stdout:6/346: getdents d4/d15/d1d 0 2024-05-13T22:48:29.917 INFO:tasks.workunit.client.1.smithi130.stdout:3/332: link d8/f34 d8/d1a/d1c/f6e 0 2024-05-13T22:48:29.919 INFO:tasks.workunit.client.1.smithi130.stdout:6/347: dwrite d4/d15/d1d/f61 [0,4194304] 0 2024-05-13T22:48:29.921 INFO:tasks.workunit.client.1.smithi130.stdout:3/333: mknod d8/c6f 0 2024-05-13T22:48:29.924 INFO:tasks.workunit.client.1.smithi130.stdout:6/348: creat d4/d15/d1f/d46/d50/f79 x:0 0 0 2024-05-13T22:48:29.925 INFO:tasks.workunit.client.1.smithi130.stdout:4/344: dread d8/dc/f5b [0,4194304] 0 2024-05-13T22:48:29.926 INFO:tasks.workunit.client.1.smithi130.stdout:4/345: truncate d8/dc/d26/f2b 595453 0 2024-05-13T22:48:29.926 INFO:tasks.workunit.client.1.smithi130.stdout:6/349: symlink d4/d15/d1d/l7a 0 2024-05-13T22:48:29.927 INFO:tasks.workunit.client.0.smithi102.stdout:1/929: dwrite d2/d4/d5/d19/ded/f35 [0,4194304] 0 2024-05-13T22:48:29.927 INFO:tasks.workunit.client.1.smithi130.stdout:4/346: truncate d8/f43 5106462 0 2024-05-13T22:48:29.928 INFO:tasks.workunit.client.1.smithi130.stdout:4/347: write d8/df/f3c [5106243,69648] 0 2024-05-13T22:48:29.929 INFO:tasks.workunit.client.0.smithi102.stdout:1/930: readlink d2/d4/d5/d19/l41 0 2024-05-13T22:48:29.930 INFO:tasks.workunit.client.1.smithi130.stdout:6/350: mkdir d4/d15/d1f/d46/d52/d7b 0 2024-05-13T22:48:29.931 INFO:tasks.workunit.client.1.smithi130.stdout:6/351: write d4/d15/d1f/f26 [1664585,62483] 0 2024-05-13T22:48:29.931 INFO:tasks.workunit.client.0.smithi102.stdout:1/931: truncate d2/d10/d62/d7b/dc4/fcc 5046187 0 2024-05-13T22:48:29.932 INFO:tasks.workunit.client.1.smithi130.stdout:4/348: truncate d8/dc/f38 2440024 0 2024-05-13T22:48:29.933 INFO:tasks.workunit.client.1.smithi130.stdout:4/349: write d8/df/f24 [3595034,937] 0 2024-05-13T22:48:29.934 INFO:tasks.workunit.client.1.smithi130.stdout:0/349: sync 2024-05-13T22:48:29.934 INFO:tasks.workunit.client.1.smithi130.stdout:4/350: write d8/dc/d35/f50 [281215,26355] 0 2024-05-13T22:48:29.934 INFO:tasks.workunit.client.0.smithi102.stdout:1/932: write d2/d4/d38/db2/f102 [295003,21599] 0 2024-05-13T22:48:29.935 INFO:tasks.workunit.client.1.smithi130.stdout:6/352: symlink d4/d15/d1f/d46/d52/l7c 0 2024-05-13T22:48:29.935 INFO:tasks.workunit.client.0.smithi102.stdout:1/933: chown d2/d45/da4 581770071 1 2024-05-13T22:48:29.935 INFO:tasks.workunit.client.1.smithi130.stdout:4/351: write d8/dc/f28 [855580,66432] 0 2024-05-13T22:48:29.941 INFO:tasks.workunit.client.0.smithi102.stdout:1/934: getdents d2/d45/da4/def 0 2024-05-13T22:48:29.941 INFO:tasks.workunit.client.1.smithi130.stdout:0/350: dread d0/d7/dd/f32 [0,4194304] 0 2024-05-13T22:48:29.943 INFO:tasks.workunit.client.1.smithi130.stdout:0/351: write d0/d7/dd/f79 [849489,110074] 0 2024-05-13T22:48:29.943 INFO:tasks.workunit.client.0.smithi102.stdout:1/935: mkdir d2/d10/d62/d7b/dc4/d128 0 2024-05-13T22:48:29.944 INFO:tasks.workunit.client.0.smithi102.stdout:1/936: dread - d2/d4/d26/f8e zero size 2024-05-13T22:48:29.947 INFO:tasks.workunit.client.1.smithi130.stdout:4/352: dwrite d8/dc/d13/f47 [0,4194304] 0 2024-05-13T22:48:29.947 INFO:tasks.workunit.client.0.smithi102.stdout:1/937: mknod d2/d4/d26/c129 0 2024-05-13T22:48:29.951 INFO:tasks.workunit.client.1.smithi130.stdout:0/352: creat d0/d7/dd/d17/d1f/f7d x:0 0 0 2024-05-13T22:48:29.952 INFO:tasks.workunit.client.0.smithi102.stdout:1/938: dread d2/d4/d5/d19/ded/f35 [0,4194304] 0 2024-05-13T22:48:29.956 INFO:tasks.workunit.client.0.smithi102.stdout:1/939: symlink d2/d4/d9e/l12a 0 2024-05-13T22:48:29.957 INFO:tasks.workunit.client.1.smithi130.stdout:0/353: dwrite d0/d7/dd/f46 [0,4194304] 0 2024-05-13T22:48:29.957 INFO:tasks.workunit.client.1.smithi130.stdout:0/354: stat d0/d7/d29/d7b 0 2024-05-13T22:48:29.958 INFO:tasks.workunit.client.1.smithi130.stdout:6/353: mkdir d4/d15/d1d/d28/d75/d7d 0 2024-05-13T22:48:29.959 INFO:tasks.workunit.client.0.smithi102.stdout:1/940: creat d2/d10/d62/d86/f12b x:0 0 0 2024-05-13T22:48:29.960 INFO:tasks.workunit.client.0.smithi102.stdout:1/941: chown d2/d4/d26/c129 58 1 2024-05-13T22:48:29.962 INFO:tasks.workunit.client.1.smithi130.stdout:0/355: mknod d0/d7/dd/d17/d1f/d2c/d45/c7e 0 2024-05-13T22:48:29.963 INFO:tasks.workunit.client.0.smithi102.stdout:1/942: fsync d2/d4/d26/d40/f72 0 2024-05-13T22:48:29.963 INFO:tasks.workunit.client.1.smithi130.stdout:6/354: mknod d4/d15/d1f/d46/d52/c7e 0 2024-05-13T22:48:29.964 INFO:tasks.workunit.client.1.smithi130.stdout:6/355: stat d4/d15/d1f/f20 0 2024-05-13T22:48:29.965 INFO:tasks.workunit.client.0.smithi102.stdout:1/943: rmdir d2/d4/d26/d40/d5e 39 2024-05-13T22:48:29.966 INFO:tasks.workunit.client.1.smithi130.stdout:0/356: mknod d0/d7/dd/d69/c7f 0 2024-05-13T22:48:29.967 INFO:tasks.workunit.client.1.smithi130.stdout:6/356: creat d4/d15/d1f/d64/f7f x:0 0 0 2024-05-13T22:48:29.969 INFO:tasks.workunit.client.1.smithi130.stdout:6/357: creat d4/d15/d1d/d28/d75/f80 x:0 0 0 2024-05-13T22:48:29.971 INFO:tasks.workunit.client.0.smithi102.stdout:1/944: dwrite d2/d4/d26/da2/ffc [0,4194304] 0 2024-05-13T22:48:29.987 INFO:tasks.workunit.client.1.smithi130.stdout:4/353: truncate d8/df/f24 3723258 0 2024-05-13T22:48:29.991 INFO:tasks.workunit.client.1.smithi130.stdout:6/358: truncate d4/d15/d1d/d28/f47 1142722 0 2024-05-13T22:48:29.991 INFO:tasks.workunit.client.0.smithi102.stdout:1/945: write d2/d4/d5/d19/dc9/fd7 [674829,68576] 0 2024-05-13T22:48:29.992 INFO:tasks.workunit.client.0.smithi102.stdout:1/946: write d2/d4/d5/df0/ff9 [562868,126660] 0 2024-05-13T22:48:29.993 INFO:tasks.workunit.client.1.smithi130.stdout:4/354: dwrite d8/dc/d26/f30 [0,4194304] 0 2024-05-13T22:48:29.995 INFO:tasks.workunit.client.0.smithi102.stdout:1/947: rmdir d2/d45/d79/d92 39 2024-05-13T22:48:30.001 INFO:tasks.workunit.client.1.smithi130.stdout:6/359: dwrite d4/f57 [0,4194304] 0 2024-05-13T22:48:30.002 INFO:tasks.workunit.client.0.smithi102.stdout:1/948: dwrite d2/d4/d5/d19/f53 [0,4194304] 0 2024-05-13T22:48:30.006 INFO:tasks.workunit.client.0.smithi102.stdout:1/949: creat d2/d10/d62/f12c x:0 0 0 2024-05-13T22:48:30.007 INFO:tasks.workunit.client.1.smithi130.stdout:4/355: dwrite d8/dc/d26/f53 [4194304,4194304] 0 2024-05-13T22:48:30.010 INFO:tasks.workunit.client.0.smithi102.stdout:1/950: truncate d2/d10/d62/d7b/f95 1266977 0 2024-05-13T22:48:30.011 INFO:tasks.workunit.client.0.smithi102.stdout:1/951: fsync d2/d4/d26/f9a 0 2024-05-13T22:48:30.015 INFO:tasks.workunit.client.1.smithi130.stdout:6/360: dwrite d4/d15/d1f/d46/d50/f53 [0,4194304] 0 2024-05-13T22:48:30.016 INFO:tasks.workunit.client.0.smithi102.stdout:1/952: creat d2/d4/d5/f12d x:0 0 0 2024-05-13T22:48:30.017 INFO:tasks.workunit.client.0.smithi102.stdout:1/953: chown d2/d10/f31 44795 1 2024-05-13T22:48:30.020 INFO:tasks.workunit.client.0.smithi102.stdout:1/954: dread - d2/d45/d79/d92/fb7 zero size 2024-05-13T22:48:30.022 INFO:tasks.workunit.client.1.smithi130.stdout:4/356: dwrite d8/dc/d35/f50 [0,4194304] 0 2024-05-13T22:48:30.023 INFO:tasks.workunit.client.0.smithi102.stdout:1/955: truncate d2/d4/d5/d19/ded/f4d 3626603 0 2024-05-13T22:48:30.024 INFO:tasks.workunit.client.1.smithi130.stdout:4/357: truncate d8/df/f40 606202 0 2024-05-13T22:48:30.025 INFO:tasks.workunit.client.1.smithi130.stdout:6/361: dwrite d4/d15/d1d/f22 [0,4194304] 0 2024-05-13T22:48:30.025 INFO:tasks.workunit.client.0.smithi102.stdout:1/956: readlink d2/d45/df4/l120 0 2024-05-13T22:48:30.026 INFO:tasks.workunit.client.0.smithi102.stdout:1/957: chown d2/d10/d62/f12c 302069 1 2024-05-13T22:48:30.028 INFO:tasks.workunit.client.1.smithi130.stdout:6/362: creat d4/d15/d1f/d64/f81 x:0 0 0 2024-05-13T22:48:30.032 INFO:tasks.workunit.client.0.smithi102.stdout:1/958: creat d2/d4/d38/db2/dba/d112/f12e x:0 0 0 2024-05-13T22:48:30.034 INFO:tasks.workunit.client.1.smithi130.stdout:4/358: rmdir d8/dc/d35/d33/d3f/d57/d60 0 2024-05-13T22:48:30.036 INFO:tasks.workunit.client.1.smithi130.stdout:6/363: symlink d4/d15/d1f/d46/d52/d7b/l82 0 2024-05-13T22:48:30.037 INFO:tasks.workunit.client.1.smithi130.stdout:2/370: sync 2024-05-13T22:48:30.039 INFO:tasks.workunit.client.1.smithi130.stdout:6/364: symlink d4/d15/d1d/d28/d72/l83 0 2024-05-13T22:48:30.040 INFO:tasks.workunit.client.1.smithi130.stdout:6/365: readlink d4/d15/d1f/d64/l68 0 2024-05-13T22:48:30.041 INFO:tasks.workunit.client.0.smithi102.stdout:1/959: dread d2/d10/f85 [0,4194304] 0 2024-05-13T22:48:30.048 INFO:tasks.workunit.client.1.smithi130.stdout:6/366: rename d4/d15/d1d/f61 to d4/d15/d1f/f84 0 2024-05-13T22:48:30.048 INFO:tasks.workunit.client.1.smithi130.stdout:2/371: dwrite d4/db/dc/d18/d2a/f7d [0,4194304] 0 2024-05-13T22:48:30.049 INFO:tasks.workunit.client.1.smithi130.stdout:2/372: chown d4/d49/d42/d4b/c5e 998 1 2024-05-13T22:48:30.052 INFO:tasks.workunit.client.1.smithi130.stdout:2/373: mkdir d4/db/dc/d18/d2a/d30/d88 0 2024-05-13T22:48:30.052 INFO:tasks.workunit.client.1.smithi130.stdout:6/367: truncate d4/d15/f31 4046061 0 2024-05-13T22:48:30.053 INFO:tasks.workunit.client.1.smithi130.stdout:2/374: fdatasync d4/f1b 0 2024-05-13T22:48:30.062 INFO:tasks.workunit.client.1.smithi130.stdout:4/359: dread d8/f1a [0,4194304] 0 2024-05-13T22:48:30.064 INFO:tasks.workunit.client.1.smithi130.stdout:4/360: symlink d8/dc/d35/d33/d44/l70 0 2024-05-13T22:48:30.075 INFO:tasks.workunit.client.1.smithi130.stdout:2/375: getdents d4/db/dc/d18/d2a/d30 0 2024-05-13T22:48:30.076 INFO:tasks.workunit.client.1.smithi130.stdout:6/368: rename d4/d16/c1b to d4/d15/d1d/d63/c85 0 2024-05-13T22:48:30.078 INFO:tasks.workunit.client.1.smithi130.stdout:6/369: mknod d4/d15/d1f/d46/d52/c86 0 2024-05-13T22:48:30.079 INFO:tasks.workunit.client.1.smithi130.stdout:2/376: dread d4/db/dc/d18/d2a/d30/d69/f77 [0,4194304] 0 2024-05-13T22:48:30.080 INFO:tasks.workunit.client.1.smithi130.stdout:2/377: rmdir d4/d49/d42/d4b 39 2024-05-13T22:48:30.082 INFO:tasks.workunit.client.1.smithi130.stdout:6/370: dread d4/d16/f69 [0,4194304] 0 2024-05-13T22:48:30.084 INFO:tasks.workunit.client.1.smithi130.stdout:6/371: dread - d4/d15/d1d/d60/f77 zero size 2024-05-13T22:48:30.086 INFO:tasks.workunit.client.1.smithi130.stdout:6/372: read d4/d15/d1d/f22 [236536,65558] 0 2024-05-13T22:48:30.090 INFO:tasks.workunit.client.1.smithi130.stdout:6/373: dread d4/d15/d1d/f38 [0,4194304] 0 2024-05-13T22:48:30.130 INFO:tasks.workunit.client.1.smithi130.stdout:8/286: sync 2024-05-13T22:48:30.130 INFO:tasks.workunit.client.1.smithi130.stdout:5/312: sync 2024-05-13T22:48:30.130 INFO:tasks.workunit.client.1.smithi130.stdout:9/352: sync 2024-05-13T22:48:30.130 INFO:tasks.workunit.client.1.smithi130.stdout:7/292: sync 2024-05-13T22:48:30.130 INFO:tasks.workunit.client.1.smithi130.stdout:1/361: sync 2024-05-13T22:48:30.130 INFO:tasks.workunit.client.1.smithi130.stdout:3/334: sync 2024-05-13T22:48:30.132 INFO:tasks.workunit.client.1.smithi130.stdout:7/293: chown d7/da/d1d/l28 187 1 2024-05-13T22:48:30.132 INFO:tasks.workunit.client.1.smithi130.stdout:8/287: write d1/d2/f3 [1843150,45935] 0 2024-05-13T22:48:30.133 INFO:tasks.workunit.client.1.smithi130.stdout:9/353: mknod d1/d15/c6e 0 2024-05-13T22:48:30.134 INFO:tasks.workunit.client.1.smithi130.stdout:9/354: chown d1/d15/d51 77884 1 2024-05-13T22:48:30.135 INFO:tasks.workunit.client.1.smithi130.stdout:5/313: write d6/d7/d24/f68 [1038233,63338] 0 2024-05-13T22:48:30.137 INFO:tasks.workunit.client.1.smithi130.stdout:7/294: mkdir d7/da/d1f/d2c/d65 0 2024-05-13T22:48:30.138 INFO:tasks.workunit.client.1.smithi130.stdout:8/288: creat d1/d2/db/d15/d18/d42/f6d x:0 0 0 2024-05-13T22:48:30.138 INFO:tasks.workunit.client.1.smithi130.stdout:7/295: chown d7/d8/f2d 0 1 2024-05-13T22:48:30.138 INFO:tasks.workunit.client.1.smithi130.stdout:3/335: creat d8/f70 x:0 0 0 2024-05-13T22:48:30.141 INFO:tasks.workunit.client.1.smithi130.stdout:9/355: rmdir d1/d15/d51 39 2024-05-13T22:48:30.145 INFO:tasks.workunit.client.1.smithi130.stdout:8/289: mknod d1/d2/db/c6e 0 2024-05-13T22:48:30.148 INFO:tasks.workunit.client.1.smithi130.stdout:9/356: mknod d1/d7/d23/d28/d2d/d2f/d63/d6b/c6f 0 2024-05-13T22:48:30.150 INFO:tasks.workunit.client.1.smithi130.stdout:1/362: dread db/d1b/d1f/d2a/d38/f60 [0,4194304] 0 2024-05-13T22:48:30.151 INFO:tasks.workunit.client.1.smithi130.stdout:9/357: fsync d1/d7/d23/d28/d2d/d2f/d63/f68 0 2024-05-13T22:48:30.152 INFO:tasks.workunit.client.1.smithi130.stdout:8/290: dread d1/d2/db/f46 [0,4194304] 0 2024-05-13T22:48:30.153 INFO:tasks.workunit.client.1.smithi130.stdout:3/336: truncate d8/dc/f37 8320827 0 2024-05-13T22:48:30.154 INFO:tasks.workunit.client.1.smithi130.stdout:9/358: write d1/d7/d23/d28/d2d/f3a [2502009,25951] 0 2024-05-13T22:48:30.154 INFO:tasks.workunit.client.1.smithi130.stdout:3/337: chown d8/dc/f12 1 1 2024-05-13T22:48:30.155 INFO:tasks.workunit.client.1.smithi130.stdout:7/296: write d7/d8/f2d [6452042,81824] 0 2024-05-13T22:48:30.158 INFO:tasks.workunit.client.1.smithi130.stdout:5/314: truncate d6/dd/d18/f15 3877601 0 2024-05-13T22:48:30.159 INFO:tasks.workunit.client.1.smithi130.stdout:3/338: mknod d8/d1a/d55/c71 0 2024-05-13T22:48:30.162 INFO:tasks.workunit.client.1.smithi130.stdout:1/363: link db/d4a/f70 db/d1b/d1f/f7a 0 2024-05-13T22:48:30.163 INFO:tasks.workunit.client.1.smithi130.stdout:9/359: mknod d1/d7/d69/c70 0 2024-05-13T22:48:30.163 INFO:tasks.workunit.client.1.smithi130.stdout:1/364: chown db/d1b/d1f/d72/d75 993283008 1 2024-05-13T22:48:30.164 INFO:tasks.workunit.client.1.smithi130.stdout:1/365: chown db/d4a 466780315 1 2024-05-13T22:48:30.165 INFO:tasks.workunit.client.1.smithi130.stdout:8/291: dwrite d1/d2/db/f8 [0,4194304] 0 2024-05-13T22:48:30.169 INFO:tasks.workunit.client.1.smithi130.stdout:8/292: read d1/d2/db/d15/d18/f31 [889905,100677] 0 2024-05-13T22:48:30.170 INFO:tasks.workunit.client.1.smithi130.stdout:7/297: rename d7/da/d1f/d2c to d7/d8/d9/d66 0 2024-05-13T22:48:30.171 INFO:tasks.workunit.client.1.smithi130.stdout:3/339: dwrite d8/f2b [0,4194304] 0 2024-05-13T22:48:30.171 INFO:tasks.workunit.client.1.smithi130.stdout:5/315: link d6/d7/d19/d4d/l42 d6/d7/d32/l69 0 2024-05-13T22:48:30.172 INFO:tasks.workunit.client.1.smithi130.stdout:0/357: sync 2024-05-13T22:48:30.173 INFO:tasks.workunit.client.1.smithi130.stdout:5/316: write d6/d7/d32/f3d [51275,104089] 0 2024-05-13T22:48:30.175 INFO:tasks.workunit.client.1.smithi130.stdout:3/340: write d8/d1a/d1c/f25 [4114063,120732] 0 2024-05-13T22:48:30.176 INFO:tasks.workunit.client.1.smithi130.stdout:9/360: fdatasync d1/d7/d23/d28/d2d/f46 0 2024-05-13T22:48:30.176 INFO:tasks.workunit.client.1.smithi130.stdout:7/298: truncate d7/da/d1f/f30 4499792 0 2024-05-13T22:48:30.178 INFO:tasks.workunit.client.1.smithi130.stdout:9/361: dread - d1/d7/d23/d3d/f5c zero size 2024-05-13T22:48:30.179 INFO:tasks.workunit.client.1.smithi130.stdout:9/362: fsync d1/d7/d23/d28/f67 0 2024-05-13T22:48:30.184 INFO:tasks.workunit.client.1.smithi130.stdout:8/293: mknod d1/d2/db/d15/d18/c6f 0 2024-05-13T22:48:30.187 INFO:tasks.workunit.client.1.smithi130.stdout:9/363: dwrite d1/d7/d23/d28/d2d/f34 [0,4194304] 0 2024-05-13T22:48:30.187 INFO:tasks.workunit.client.1.smithi130.stdout:8/294: chown d1/d2/db/d15/d18/d42/d53/d56/f61 253991369 1 2024-05-13T22:48:30.187 INFO:tasks.workunit.client.1.smithi130.stdout:9/364: readlink d1/d15/l24 0 2024-05-13T22:48:30.188 INFO:tasks.workunit.client.1.smithi130.stdout:3/341: rename d8/dc/d19/d2c/l50 to d8/dc/d42/l72 0 2024-05-13T22:48:30.194 INFO:tasks.workunit.client.1.smithi130.stdout:7/299: rename d7/da/d1f to d7/da/d1d/d3d/d46/d67 0 2024-05-13T22:48:30.194 INFO:tasks.workunit.client.1.smithi130.stdout:8/295: link d1/d2/db/d15/d18/l2e d1/d2/db/d15/d18/d42/d53/l70 0 2024-05-13T22:48:30.194 INFO:tasks.workunit.client.1.smithi130.stdout:3/342: write d8/d1a/f47 [20208,25635] 0 2024-05-13T22:48:30.196 INFO:tasks.workunit.client.1.smithi130.stdout:7/300: chown d7/d8/f1c 791871 1 2024-05-13T22:48:30.197 INFO:tasks.workunit.client.1.smithi130.stdout:9/365: rename d1/db/cf to d1/d7/d23/c71 0 2024-05-13T22:48:30.197 INFO:tasks.workunit.client.1.smithi130.stdout:7/301: chown d7/d8/l14 15744 1 2024-05-13T22:48:30.198 INFO:tasks.workunit.client.1.smithi130.stdout:0/358: dwrite d0/f27 [0,4194304] 0 2024-05-13T22:48:30.198 INFO:tasks.workunit.client.1.smithi130.stdout:8/296: truncate d1/d2/db/d15/d16/f1c 1708211 0 2024-05-13T22:48:30.200 INFO:tasks.workunit.client.1.smithi130.stdout:9/366: dread - d1/d7/d23/d28/d2d/d2f/d63/f68 zero size 2024-05-13T22:48:30.202 INFO:tasks.workunit.client.1.smithi130.stdout:7/302: mknod d7/da/d1d/c68 0 2024-05-13T22:48:30.202 INFO:tasks.workunit.client.1.smithi130.stdout:7/303: fdatasync d7/d8/d9/d66/f63 0 2024-05-13T22:48:30.204 INFO:tasks.workunit.client.1.smithi130.stdout:7/304: truncate d7/da/f56 906742 0 2024-05-13T22:48:30.205 INFO:tasks.workunit.client.1.smithi130.stdout:0/359: creat d0/d7/dd/d17/d52/f80 x:0 0 0 2024-05-13T22:48:30.207 INFO:tasks.workunit.client.1.smithi130.stdout:0/360: read d0/d7/dd/f79 [672741,45608] 0 2024-05-13T22:48:30.207 INFO:tasks.workunit.client.1.smithi130.stdout:5/317: truncate d6/dd/f13 928409 0 2024-05-13T22:48:30.207 INFO:tasks.workunit.client.1.smithi130.stdout:7/305: mknod d7/da/d1d/d3d/d46/c69 0 2024-05-13T22:48:30.210 INFO:tasks.workunit.client.1.smithi130.stdout:8/297: rename d1/dc/f3f to d1/d2/db/d15/d18/d42/d53/d56/f71 0 2024-05-13T22:48:30.211 INFO:tasks.workunit.client.1.smithi130.stdout:5/318: mkdir d6/d7/d19/d4d/d6a 0 2024-05-13T22:48:30.212 INFO:tasks.workunit.client.1.smithi130.stdout:3/343: truncate d8/f4b 174884 0 2024-05-13T22:48:30.213 INFO:tasks.workunit.client.1.smithi130.stdout:3/344: dread - d8/d1a/d1c/f6c zero size 2024-05-13T22:48:30.215 INFO:tasks.workunit.client.1.smithi130.stdout:0/361: rename d0/d7/dd/d17/d1f/d2c/l2d to d0/d7/dd/d17/d52/d67/l81 0 2024-05-13T22:48:30.215 INFO:tasks.workunit.client.1.smithi130.stdout:3/345: truncate d8/d1a/d1c/f6c 800178 0 2024-05-13T22:48:30.215 INFO:tasks.workunit.client.1.smithi130.stdout:8/298: mknod d1/d2/db/d15/d18/d1a/c72 0 2024-05-13T22:48:30.216 INFO:tasks.workunit.client.1.smithi130.stdout:5/319: mknod d6/d7/d24/d56/c6b 0 2024-05-13T22:48:30.217 INFO:tasks.workunit.client.1.smithi130.stdout:7/306: creat d7/f6a x:0 0 0 2024-05-13T22:48:30.220 INFO:tasks.workunit.client.1.smithi130.stdout:7/307: unlink d7/d8/c27 0 2024-05-13T22:48:30.221 INFO:tasks.workunit.client.1.smithi130.stdout:9/367: dwrite d1/f5f [0,4194304] 0 2024-05-13T22:48:30.221 INFO:tasks.workunit.client.1.smithi130.stdout:8/299: mknod d1/d2/db/d15/d18/c73 0 2024-05-13T22:48:30.223 INFO:tasks.workunit.client.1.smithi130.stdout:0/362: rmdir d0/d7/d29/d3c 0 2024-05-13T22:48:30.224 INFO:tasks.workunit.client.1.smithi130.stdout:7/308: read d7/d8/f1c [2000069,54668] 0 2024-05-13T22:48:30.225 INFO:tasks.workunit.client.1.smithi130.stdout:3/346: link d8/dc/f22 d8/dc/f73 0 2024-05-13T22:48:30.228 INFO:tasks.workunit.client.1.smithi130.stdout:5/320: dwrite d6/f1f [0,4194304] 0 2024-05-13T22:48:30.229 INFO:tasks.workunit.client.1.smithi130.stdout:0/363: dread - d0/d7/dd/d17/d52/f80 zero size 2024-05-13T22:48:30.229 INFO:tasks.workunit.client.1.smithi130.stdout:3/347: truncate d8/dc/f73 2107411 0 2024-05-13T22:48:30.231 INFO:tasks.workunit.client.1.smithi130.stdout:8/300: dread d1/d2/db/d15/d18/f63 [0,4194304] 0 2024-05-13T22:48:30.236 INFO:tasks.workunit.client.1.smithi130.stdout:5/321: readlink d6/d7/d32/l4a 0 2024-05-13T22:48:30.236 INFO:tasks.workunit.client.1.smithi130.stdout:8/301: truncate d1/d2/db/f40 2014861 0 2024-05-13T22:48:30.237 INFO:tasks.workunit.client.1.smithi130.stdout:0/364: mknod d0/d7/dd/d17/d52/d5b/c82 0 2024-05-13T22:48:30.240 INFO:tasks.workunit.client.1.smithi130.stdout:9/368: dread d1/f3 [4194304,4194304] 0 2024-05-13T22:48:30.241 INFO:tasks.workunit.client.1.smithi130.stdout:3/348: dwrite d8/d1a/d1c/f49 [0,4194304] 0 2024-05-13T22:48:30.241 INFO:tasks.workunit.client.1.smithi130.stdout:7/309: creat d7/da/d1d/f6b x:0 0 0 2024-05-13T22:48:30.242 INFO:tasks.workunit.client.1.smithi130.stdout:5/322: rename d6/d7/d24/f5b to d6/d2f/f6c 0 2024-05-13T22:48:30.244 INFO:tasks.workunit.client.1.smithi130.stdout:7/310: dread - d7/da/d1d/d3d/d46/f4b zero size 2024-05-13T22:48:30.244 INFO:tasks.workunit.client.1.smithi130.stdout:0/365: dread d0/d7/f23 [0,4194304] 0 2024-05-13T22:48:30.247 INFO:tasks.workunit.client.1.smithi130.stdout:9/369: stat d1/d7/d23/d28/d2d/d2f/c4a 0 2024-05-13T22:48:30.247 INFO:tasks.workunit.client.1.smithi130.stdout:0/366: read d0/d7/f23 [141155,52112] 0 2024-05-13T22:48:30.248 INFO:tasks.workunit.client.1.smithi130.stdout:6/374: sync 2024-05-13T22:48:30.249 INFO:tasks.workunit.client.1.smithi130.stdout:2/378: sync 2024-05-13T22:48:30.249 INFO:tasks.workunit.client.1.smithi130.stdout:4/361: sync 2024-05-13T22:48:30.250 INFO:tasks.workunit.client.1.smithi130.stdout:3/349: dwrite d8/f16 [0,4194304] 0 2024-05-13T22:48:30.251 INFO:tasks.workunit.client.1.smithi130.stdout:0/367: stat d0/d7/dd/d2e/d77 0 2024-05-13T22:48:30.252 INFO:tasks.workunit.client.1.smithi130.stdout:2/379: chown d4/f9 51 1 2024-05-13T22:48:30.253 INFO:tasks.workunit.client.1.smithi130.stdout:7/311: creat d7/da/d1d/d3d/d46/d67/f6c x:0 0 0 2024-05-13T22:48:30.255 INFO:tasks.workunit.client.1.smithi130.stdout:7/312: write d7/da/f35 [3785895,97274] 0 2024-05-13T22:48:30.257 INFO:tasks.workunit.client.1.smithi130.stdout:3/350: creat d8/dc/d42/f74 x:0 0 0 2024-05-13T22:48:30.257 INFO:tasks.workunit.client.1.smithi130.stdout:6/375: creat d4/d15/d1f/d46/d52/f87 x:0 0 0 2024-05-13T22:48:30.258 INFO:tasks.workunit.client.1.smithi130.stdout:0/368: fsync d0/d7/dd/d17/d52/d67/f6a 0 2024-05-13T22:48:30.259 INFO:tasks.workunit.client.1.smithi130.stdout:6/376: readlink d4/d15/d1d/l7a 0 2024-05-13T22:48:30.261 INFO:tasks.workunit.client.1.smithi130.stdout:0/369: write d0/f1e [937098,58400] 0 2024-05-13T22:48:30.261 INFO:tasks.workunit.client.1.smithi130.stdout:8/302: truncate d1/d2/f25 769619 0 2024-05-13T22:48:30.263 INFO:tasks.workunit.client.1.smithi130.stdout:9/370: write d1/d7/f27 [830551,118291] 0 2024-05-13T22:48:30.263 INFO:tasks.workunit.client.1.smithi130.stdout:3/351: symlink d8/dc/d19/d2c/d61/l75 0 2024-05-13T22:48:30.264 INFO:tasks.workunit.client.1.smithi130.stdout:6/377: mknod d4/d15/d1f/d3c/c88 0 2024-05-13T22:48:30.266 INFO:tasks.workunit.client.1.smithi130.stdout:4/362: rmdir d8/dc/d35/d33/d6e 0 2024-05-13T22:48:30.268 INFO:tasks.workunit.client.1.smithi130.stdout:7/313: creat d7/d8/d5b/f6d x:0 0 0 2024-05-13T22:48:30.269 INFO:tasks.workunit.client.1.smithi130.stdout:5/323: dwrite d6/d2f/f39 [4194304,4194304] 0 2024-05-13T22:48:30.269 INFO:tasks.workunit.client.1.smithi130.stdout:3/352: creat d8/dc/d15/f76 x:0 0 0 2024-05-13T22:48:30.270 INFO:tasks.workunit.client.1.smithi130.stdout:9/371: chown d1/d7/d23/c56 6042351 1 2024-05-13T22:48:30.271 INFO:tasks.workunit.client.1.smithi130.stdout:5/324: readlink d6/dd/d18/l14 0 2024-05-13T22:48:30.271 INFO:tasks.workunit.client.1.smithi130.stdout:8/303: fdatasync d1/dc/f26 0 2024-05-13T22:48:30.272 INFO:tasks.workunit.client.1.smithi130.stdout:4/363: write f6 [2403725,114368] 0 2024-05-13T22:48:30.272 INFO:tasks.workunit.client.1.smithi130.stdout:7/314: dread - d7/da/d1d/d42/f64 zero size 2024-05-13T22:48:30.272 INFO:tasks.workunit.client.1.smithi130.stdout:6/378: rename d4/fc to d4/d15/d1f/d46/f89 0 2024-05-13T22:48:30.275 INFO:tasks.workunit.client.1.smithi130.stdout:9/372: chown d1/d7/d23/d28/d32/l3e 1 1 2024-05-13T22:48:30.275 INFO:tasks.workunit.client.1.smithi130.stdout:6/379: readlink d4/d15/d1f/d64/l68 0 2024-05-13T22:48:30.275 INFO:tasks.workunit.client.1.smithi130.stdout:7/315: readlink d7/da/d1d/d42/d43/l47 0 2024-05-13T22:48:30.277 INFO:tasks.workunit.client.1.smithi130.stdout:6/380: write d4/d15/d1d/d60/f77 [19237,82585] 0 2024-05-13T22:48:30.281 INFO:tasks.workunit.client.1.smithi130.stdout:8/304: rmdir d1/d2/db/d15/d18/d42 39 2024-05-13T22:48:30.282 INFO:tasks.workunit.client.1.smithi130.stdout:1/366: sync 2024-05-13T22:48:30.283 INFO:tasks.workunit.client.1.smithi130.stdout:4/364: unlink d8/dc/d35/d33/d3f/c63 0 2024-05-13T22:48:30.284 INFO:tasks.workunit.client.1.smithi130.stdout:8/305: read d1/d2/db/f46 [2356176,80295] 0 2024-05-13T22:48:30.285 INFO:tasks.workunit.client.1.smithi130.stdout:9/373: dwrite d1/d7/f57 [0,4194304] 0 2024-05-13T22:48:30.285 INFO:tasks.workunit.client.1.smithi130.stdout:3/353: rmdir d8/dc/d19/d2c 39 2024-05-13T22:48:30.285 INFO:tasks.workunit.client.1.smithi130.stdout:2/380: dread d4/db/dc/f20 [4194304,4194304] 0 2024-05-13T22:48:30.286 INFO:tasks.workunit.client.1.smithi130.stdout:4/365: chown d8/dc/d26/c39 39130 1 2024-05-13T22:48:30.287 INFO:tasks.workunit.client.1.smithi130.stdout:6/381: creat d4/d15/d1f/d46/d50/f8a x:0 0 0 2024-05-13T22:48:30.287 INFO:tasks.workunit.client.1.smithi130.stdout:9/374: stat d1/d15/f37 0 2024-05-13T22:48:30.287 INFO:tasks.workunit.client.1.smithi130.stdout:1/367: stat db/d1b/d1f/d2a/d38/d46/l77 0 2024-05-13T22:48:30.289 INFO:tasks.workunit.client.1.smithi130.stdout:8/306: chown d1/d2/db/d15/d18/d1a/l6a 14 1 2024-05-13T22:48:30.291 INFO:tasks.workunit.client.1.smithi130.stdout:3/354: fdatasync f2 0 2024-05-13T22:48:30.292 INFO:tasks.workunit.client.1.smithi130.stdout:7/316: truncate d7/da/f33 819088 0 2024-05-13T22:48:30.294 INFO:tasks.workunit.client.1.smithi130.stdout:1/368: write db/d1b/d1f/d2a/d35/f31 [1675566,13539] 0 2024-05-13T22:48:30.296 INFO:tasks.workunit.client.1.smithi130.stdout:5/325: write f4 [4231260,14510] 0 2024-05-13T22:48:30.297 INFO:tasks.workunit.client.1.smithi130.stdout:1/369: read - db/d1b/d1f/f7a zero size 2024-05-13T22:48:30.298 INFO:tasks.workunit.client.1.smithi130.stdout:0/370: dwrite d0/d7/dd/d2e/d4b/f15 [0,4194304] 0 2024-05-13T22:48:30.298 INFO:tasks.workunit.client.1.smithi130.stdout:4/366: symlink d8/dc/d35/d33/d3f/d57/l71 0 2024-05-13T22:48:30.299 INFO:tasks.workunit.client.1.smithi130.stdout:3/355: chown d8/dc/d19/d2c 5340912 1 2024-05-13T22:48:30.301 INFO:tasks.workunit.client.1.smithi130.stdout:4/367: chown c0 827902 1 2024-05-13T22:48:30.304 INFO:tasks.workunit.client.1.smithi130.stdout:5/326: symlink d6/dd/l6d 0 2024-05-13T22:48:30.307 INFO:tasks.workunit.client.1.smithi130.stdout:3/356: truncate d8/dc/f3e 4850508 0 2024-05-13T22:48:30.307 INFO:tasks.workunit.client.1.smithi130.stdout:2/381: creat d4/d49/d42/f89 x:0 0 0 2024-05-13T22:48:30.311 INFO:tasks.workunit.client.1.smithi130.stdout:7/317: dwrite d7/da/d1d/d42/d43/f4d [0,4194304] 0 2024-05-13T22:48:30.312 INFO:tasks.workunit.client.1.smithi130.stdout:6/382: dwrite d4/d15/d1f/d46/d50/f59 [0,4194304] 0 2024-05-13T22:48:30.313 INFO:tasks.workunit.client.1.smithi130.stdout:5/327: chown d6/d7/fa 197468666 1 2024-05-13T22:48:30.313 INFO:tasks.workunit.client.1.smithi130.stdout:8/307: write d1/d2/f6 [4694113,16555] 0 2024-05-13T22:48:30.314 INFO:tasks.workunit.client.1.smithi130.stdout:9/375: link d1/c21 d1/d15/c72 0 2024-05-13T22:48:30.319 INFO:tasks.workunit.client.1.smithi130.stdout:9/376: write d1/d7/f27 [281539,52675] 0 2024-05-13T22:48:30.319 INFO:tasks.workunit.client.1.smithi130.stdout:0/371: creat d0/d7/dd/d17/d1f/d2c/d45/d57/d78/f83 x:0 0 0 2024-05-13T22:48:30.319 INFO:tasks.workunit.client.1.smithi130.stdout:1/370: dread db/d1b/d1f/d2a/d38/f5c [0,4194304] 0 2024-05-13T22:48:30.320 INFO:tasks.workunit.client.1.smithi130.stdout:7/318: rmdir d7/da/d1d/d42 39 2024-05-13T22:48:30.323 INFO:tasks.workunit.client.1.smithi130.stdout:0/372: stat d0/d7/d29/l64 0 2024-05-13T22:48:30.323 INFO:tasks.workunit.client.1.smithi130.stdout:9/377: fdatasync d1/d7/d23/d28/d32/f62 0 2024-05-13T22:48:30.325 INFO:tasks.workunit.client.1.smithi130.stdout:7/319: write d7/da/d1d/d3d/d46/f4b [155119,114870] 0 2024-05-13T22:48:30.326 INFO:tasks.workunit.client.1.smithi130.stdout:2/382: dwrite d4/f1f [0,4194304] 0 2024-05-13T22:48:30.332 INFO:tasks.workunit.client.1.smithi130.stdout:4/368: rmdir d8/dc/d35/d33/d3f/d57/d6f 0 2024-05-13T22:48:30.334 INFO:tasks.workunit.client.1.smithi130.stdout:0/373: fsync d0/d7/dd/f32 0 2024-05-13T22:48:30.334 INFO:tasks.workunit.client.1.smithi130.stdout:1/371: creat db/d1b/d1f/d2a/d38/d46/d6d/f7b x:0 0 0 2024-05-13T22:48:30.336 INFO:tasks.workunit.client.1.smithi130.stdout:2/383: rmdir d4/db/dc/d18/d2a 39 2024-05-13T22:48:30.336 INFO:tasks.workunit.client.1.smithi130.stdout:4/369: dread - d8/df/f59 zero size 2024-05-13T22:48:30.337 INFO:tasks.workunit.client.1.smithi130.stdout:3/357: rename d8/dc/d19/d2c/d61/l75 to d8/dc/l77 0 2024-05-13T22:48:30.339 INFO:tasks.workunit.client.1.smithi130.stdout:9/378: dwrite d1/d7/d23/d28/d32/f62 [4194304,4194304] 0 2024-05-13T22:48:30.339 INFO:tasks.workunit.client.1.smithi130.stdout:3/358: stat d8/f16 0 2024-05-13T22:48:30.340 INFO:tasks.workunit.client.1.smithi130.stdout:2/384: mknod d4/d49/c8a 0 2024-05-13T22:48:30.340 INFO:tasks.workunit.client.1.smithi130.stdout:7/320: write d7/da/d1d/d42/f58 [3685716,95709] 0 2024-05-13T22:48:30.341 INFO:tasks.workunit.client.1.smithi130.stdout:8/308: rename d1/d2/db/d15/d18/f52 to d1/d2/db/d15/f74 0 2024-05-13T22:48:30.343 INFO:tasks.workunit.client.1.smithi130.stdout:1/372: truncate db/d1b/d1f/d2a/f2f 1432364 0 2024-05-13T22:48:30.345 INFO:tasks.workunit.client.1.smithi130.stdout:0/374: creat d0/d7/dd/d17/d1f/d2c/d45/d57/d78/f84 x:0 0 0 2024-05-13T22:48:30.347 INFO:tasks.workunit.client.1.smithi130.stdout:2/385: read - d4/d49/d48/d4a/d64/f6a zero size 2024-05-13T22:48:30.347 INFO:tasks.workunit.client.1.smithi130.stdout:0/375: readlink d0/d7/dd/d17/l51 0 2024-05-13T22:48:30.347 INFO:tasks.workunit.client.1.smithi130.stdout:3/359: fdatasync f2 0 2024-05-13T22:48:30.347 INFO:tasks.workunit.client.1.smithi130.stdout:7/321: creat d7/da/d1d/d3d/d46/f6e x:0 0 0 2024-05-13T22:48:30.350 INFO:tasks.workunit.client.1.smithi130.stdout:2/386: write d4/db/f2c [4269052,96173] 0 2024-05-13T22:48:30.351 INFO:tasks.workunit.client.1.smithi130.stdout:9/379: rmdir d1/d7/d23/d28/d2d/d2f 39 2024-05-13T22:48:30.353 INFO:tasks.workunit.client.1.smithi130.stdout:5/328: dread d6/d7/f3b [0,4194304] 0 2024-05-13T22:48:30.356 INFO:tasks.workunit.client.1.smithi130.stdout:1/373: mknod db/d1b/d1f/c7c 0 2024-05-13T22:48:30.358 INFO:tasks.workunit.client.1.smithi130.stdout:5/329: dread - d6/d7/d24/d56/f5d zero size 2024-05-13T22:48:30.359 INFO:tasks.workunit.client.1.smithi130.stdout:4/370: getdents d8/dc/d35/d33/d44 0 2024-05-13T22:48:30.360 INFO:tasks.workunit.client.1.smithi130.stdout:7/322: truncate d7/da/d1d/d5d/f5e 626679 0 2024-05-13T22:48:30.361 INFO:tasks.workunit.client.1.smithi130.stdout:6/383: truncate d4/d15/d1f/f2d 601519 0 2024-05-13T22:48:30.361 INFO:tasks.workunit.client.1.smithi130.stdout:8/309: dwrite d1/d2/f39 [0,4194304] 0 2024-05-13T22:48:30.363 INFO:tasks.workunit.client.1.smithi130.stdout:1/374: read - db/d1b/d1f/d2a/d38/d46/f71 zero size 2024-05-13T22:48:30.367 INFO:tasks.workunit.client.1.smithi130.stdout:8/310: fdatasync d1/d2/db/d15/d16/f1c 0 2024-05-13T22:48:30.367 INFO:tasks.workunit.client.1.smithi130.stdout:0/376: dwrite d0/d7/f22 [0,4194304] 0 2024-05-13T22:48:30.372 INFO:tasks.workunit.client.1.smithi130.stdout:9/380: truncate d1/d15/f16 963474 0 2024-05-13T22:48:30.374 INFO:tasks.workunit.client.1.smithi130.stdout:9/381: dread - d1/d7/d23/d28/f67 zero size 2024-05-13T22:48:30.375 INFO:tasks.workunit.client.1.smithi130.stdout:2/387: rename d4/db/dc/d18/d2a/l5c to d4/d49/d48/d4a/l8b 0 2024-05-13T22:48:30.376 INFO:tasks.workunit.client.1.smithi130.stdout:3/360: dwrite d8/f4b [0,4194304] 0 2024-05-13T22:48:30.377 INFO:tasks.workunit.client.1.smithi130.stdout:7/323: dwrite d7/da/d1d/d3d/d46/f4f [0,4194304] 0 2024-05-13T22:48:30.379 INFO:tasks.workunit.client.1.smithi130.stdout:5/330: unlink d6/d7/d19/d4d/f29 0 2024-05-13T22:48:30.379 INFO:tasks.workunit.client.1.smithi130.stdout:6/384: mkdir d4/d15/d8b 0 2024-05-13T22:48:30.381 INFO:tasks.workunit.client.1.smithi130.stdout:4/371: creat d8/dc/d35/d33/d44/d56/f72 x:0 0 0 2024-05-13T22:48:30.388 INFO:tasks.workunit.client.1.smithi130.stdout:4/372: read d8/dc/d26/f52 [135982,125574] 0 2024-05-13T22:48:30.388 INFO:tasks.workunit.client.1.smithi130.stdout:6/385: truncate d4/d15/d1f/d64/f81 189096 0 2024-05-13T22:48:30.391 INFO:tasks.workunit.client.1.smithi130.stdout:8/311: chown d1/d2/db/d15/d18/d1a/c28 42 1 2024-05-13T22:48:30.391 INFO:tasks.workunit.client.1.smithi130.stdout:9/382: read - d1/d7/d23/d28/d2d/d2f/f3f zero size 2024-05-13T22:48:30.392 INFO:tasks.workunit.client.1.smithi130.stdout:2/388: rename d4/db/dc/f10 to d4/d49/d48/d4a/f8c 0 2024-05-13T22:48:30.397 INFO:tasks.workunit.client.1.smithi130.stdout:3/361: creat d8/f78 x:0 0 0 2024-05-13T22:48:30.398 INFO:tasks.workunit.client.1.smithi130.stdout:6/386: symlink d4/d15/d1d/d28/d75/l8c 0 2024-05-13T22:48:30.398 INFO:tasks.workunit.client.1.smithi130.stdout:5/331: dwrite d6/dd/f37 [0,4194304] 0 2024-05-13T22:48:30.399 INFO:tasks.workunit.client.1.smithi130.stdout:9/383: mkdir d1/d7/d23/d28/d73 0 2024-05-13T22:48:30.399 INFO:tasks.workunit.client.1.smithi130.stdout:8/312: write d1/dc/f3a [183628,89858] 0 2024-05-13T22:48:30.401 INFO:tasks.workunit.client.1.smithi130.stdout:4/373: dread d8/dc/d35/d33/d44/f54 [0,4194304] 0 2024-05-13T22:48:30.404 INFO:tasks.workunit.client.1.smithi130.stdout:6/387: creat d4/d15/d1d/d28/f8d x:0 0 0 2024-05-13T22:48:30.406 INFO:tasks.workunit.client.1.smithi130.stdout:6/388: truncate d4/d15/d1f/d46/d50/f79 815882 0 2024-05-13T22:48:30.406 INFO:tasks.workunit.client.1.smithi130.stdout:8/313: unlink d1/d2/db/d15/d16/d54/d58/l59 0 2024-05-13T22:48:30.414 INFO:tasks.workunit.client.1.smithi130.stdout:4/374: symlink d8/dc/d35/d33/d44/l73 0 2024-05-13T22:48:30.414 INFO:tasks.workunit.client.1.smithi130.stdout:6/389: symlink d4/d15/d1d/d63/l8e 0 2024-05-13T22:48:30.414 INFO:tasks.workunit.client.1.smithi130.stdout:8/314: readlink d1/d2/db/d15/d18/d42/l57 0 2024-05-13T22:48:30.416 INFO:tasks.workunit.client.1.smithi130.stdout:1/375: dread db/f32 [0,4194304] 0 2024-05-13T22:48:30.418 INFO:tasks.workunit.client.1.smithi130.stdout:3/362: getdents d8/dc/d19/d2c 0 2024-05-13T22:48:30.425 INFO:tasks.workunit.client.1.smithi130.stdout:7/324: dread d7/d8/f4c [0,4194304] 0 2024-05-13T22:48:30.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:30 smithi102 ceph-mon[28599]: pgmap v32: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 64 MiB/s rd, 209 MiB/s wr, 498 op/s 2024-05-13T22:48:30.432 INFO:tasks.workunit.client.1.smithi130.stdout:5/332: dread d6/dd/d18/fc [0,4194304] 0 2024-05-13T22:48:30.477 INFO:tasks.workunit.client.1.smithi130.stdout:6/390: creat d4/d15/d1d/d28/f8f x:0 0 0 2024-05-13T22:48:30.478 INFO:tasks.workunit.client.1.smithi130.stdout:6/391: readlink d4/d15/d1d/l7a 0 2024-05-13T22:48:30.478 INFO:tasks.workunit.client.1.smithi130.stdout:6/392: chown d4/f12 292508785 1 2024-05-13T22:48:30.479 INFO:tasks.workunit.client.1.smithi130.stdout:4/375: rename d8/f1a to d8/dc/d35/d33/d44/f74 0 2024-05-13T22:48:30.480 INFO:tasks.workunit.client.1.smithi130.stdout:9/384: creat d1/d7/d23/d28/d2d/f74 x:0 0 0 2024-05-13T22:48:30.480 INFO:tasks.workunit.client.1.smithi130.stdout:9/385: chown d1/d7/f57 41 1 2024-05-13T22:48:30.488 INFO:tasks.workunit.client.1.smithi130.stdout:5/333: mkdir d6/dd/d18/d6e 0 2024-05-13T22:48:30.491 INFO:tasks.workunit.client.1.smithi130.stdout:4/376: creat d8/dc/d26/f75 x:0 0 0 2024-05-13T22:48:30.492 INFO:tasks.workunit.client.1.smithi130.stdout:8/315: mknod d1/d2/db/d15/d18/c75 0 2024-05-13T22:48:30.495 INFO:tasks.workunit.client.1.smithi130.stdout:9/386: unlink d1/d15/c2b 0 2024-05-13T22:48:30.495 INFO:tasks.workunit.client.1.smithi130.stdout:9/387: chown d1/d7/l18 5038 1 2024-05-13T22:48:30.496 INFO:tasks.workunit.client.1.smithi130.stdout:0/377: dwrite d0/f8 [0,4194304] 0 2024-05-13T22:48:30.498 INFO:tasks.workunit.client.1.smithi130.stdout:1/376: truncate db/f16 3681171 0 2024-05-13T22:48:30.499 INFO:tasks.workunit.client.1.smithi130.stdout:3/363: symlink d8/dc/d19/l79 0 2024-05-13T22:48:30.500 INFO:tasks.workunit.client.1.smithi130.stdout:2/389: write d4/db/dc/d18/d2a/d30/d69/f77 [145088,112154] 0 2024-05-13T22:48:30.501 INFO:tasks.workunit.client.1.smithi130.stdout:5/334: rmdir d6/d7/d24/d56 39 2024-05-13T22:48:30.509 INFO:tasks.workunit.client.1.smithi130.stdout:9/388: unlink d1/d7/d23/d28/d2d/c52 0 2024-05-13T22:48:30.510 INFO:tasks.workunit.client.1.smithi130.stdout:7/325: creat d7/da/d1d/f6f x:0 0 0 2024-05-13T22:48:30.512 INFO:tasks.workunit.client.1.smithi130.stdout:5/335: mknod d6/d7/d32/c6f 0 2024-05-13T22:48:30.513 INFO:tasks.workunit.client.1.smithi130.stdout:1/377: dread db/d1b/d1f/d49/f48 [0,4194304] 0 2024-05-13T22:48:30.517 INFO:tasks.workunit.client.1.smithi130.stdout:0/378: link d0/d7/dd/d17/d52/d5b/d5f/f70 d0/d7/dd/d17/d1f/d2c/d45/d57/f85 0 2024-05-13T22:48:30.519 INFO:tasks.workunit.client.1.smithi130.stdout:2/390: mknod d4/d49/d48/c8d 0 2024-05-13T22:48:30.521 INFO:tasks.workunit.client.1.smithi130.stdout:7/326: mkdir d7/da/d1d/d5d/d70 0 2024-05-13T22:48:30.521 INFO:tasks.workunit.client.1.smithi130.stdout:7/327: fsync d7/da/d1d/d42/f64 0 2024-05-13T22:48:30.522 INFO:tasks.workunit.client.1.smithi130.stdout:5/336: readlink d6/d1a/l67 0 2024-05-13T22:48:30.523 INFO:tasks.workunit.client.1.smithi130.stdout:1/378: creat db/d1b/d1f/d49/d56/f7d x:0 0 0 2024-05-13T22:48:30.524 INFO:tasks.workunit.client.1.smithi130.stdout:8/316: creat d1/d2/db/d15/d18/f76 x:0 0 0 2024-05-13T22:48:30.525 INFO:tasks.workunit.client.1.smithi130.stdout:1/379: dread - db/d4a/f70 zero size 2024-05-13T22:48:30.525 INFO:tasks.workunit.client.1.smithi130.stdout:7/328: write d7/d8/d9/d66/f63 [738481,71620] 0 2024-05-13T22:48:30.529 INFO:tasks.workunit.client.1.smithi130.stdout:2/391: rmdir d4/d49/d42 39 2024-05-13T22:48:30.532 INFO:tasks.workunit.client.1.smithi130.stdout:6/393: dwrite d4/d15/f31 [0,4194304] 0 2024-05-13T22:48:30.533 INFO:tasks.workunit.client.1.smithi130.stdout:9/389: getdents d1/d7/d23/d28/d32/d45 0 2024-05-13T22:48:30.533 INFO:tasks.workunit.client.1.smithi130.stdout:3/364: dread d8/dc/f3e [0,4194304] 0 2024-05-13T22:48:30.534 INFO:tasks.workunit.client.1.smithi130.stdout:5/337: dread d6/f1f [0,4194304] 0 2024-05-13T22:48:30.534 INFO:tasks.workunit.client.1.smithi130.stdout:1/380: creat db/d1b/d1f/d49/d56/f7e x:0 0 0 2024-05-13T22:48:30.538 INFO:tasks.workunit.client.1.smithi130.stdout:6/394: readlink d4/d15/d1d/d28/d72/l83 0 2024-05-13T22:48:30.538 INFO:tasks.workunit.client.1.smithi130.stdout:5/338: read d6/f1f [1551381,72743] 0 2024-05-13T22:48:30.539 INFO:tasks.workunit.client.1.smithi130.stdout:9/390: chown d1/d7/d69/c70 31 1 2024-05-13T22:48:30.540 INFO:tasks.workunit.client.1.smithi130.stdout:4/377: dwrite d8/df/f24 [4194304,4194304] 0 2024-05-13T22:48:30.540 INFO:tasks.workunit.client.1.smithi130.stdout:0/379: truncate d0/d7/dd/d2e/f2f 2558687 0 2024-05-13T22:48:30.542 INFO:tasks.workunit.client.1.smithi130.stdout:2/392: creat d4/d56/f8e x:0 0 0 2024-05-13T22:48:30.543 INFO:tasks.workunit.client.1.smithi130.stdout:5/339: write d6/d7/d19/f2c [179310,41221] 0 2024-05-13T22:48:30.549 INFO:tasks.workunit.client.1.smithi130.stdout:9/391: stat d1/d7/d23/d28/d2d/d2f/d63/f68 0 2024-05-13T22:48:30.550 INFO:tasks.workunit.client.1.smithi130.stdout:3/365: creat d8/d1a/f7a x:0 0 0 2024-05-13T22:48:30.551 INFO:tasks.workunit.client.1.smithi130.stdout:4/378: mknod d8/dc/d35/d33/d44/c76 0 2024-05-13T22:48:30.555 INFO:tasks.workunit.client.1.smithi130.stdout:5/340: truncate d6/d1a/f40 4596451 0 2024-05-13T22:48:30.558 INFO:tasks.workunit.client.1.smithi130.stdout:1/381: dwrite db/d1b/d1f/d49/d56/f7d [0,4194304] 0 2024-05-13T22:48:30.559 INFO:tasks.workunit.client.1.smithi130.stdout:0/380: rename d0/d7/c3d to d0/d7/dd/d17/d1f/d2c/d45/d57/d78/c86 0 2024-05-13T22:48:30.572 INFO:tasks.workunit.client.1.smithi130.stdout:8/317: dread d1/d2/db/d15/d18/f4e [0,4194304] 0 2024-05-13T22:48:30.572 INFO:tasks.workunit.client.1.smithi130.stdout:2/393: dwrite d4/f19 [0,4194304] 0 2024-05-13T22:48:30.573 INFO:tasks.workunit.client.1.smithi130.stdout:6/395: dwrite d4/f12 [4194304,4194304] 0 2024-05-13T22:48:30.575 INFO:tasks.workunit.client.1.smithi130.stdout:5/341: truncate d6/dd/f3e 549898 0 2024-05-13T22:48:30.576 INFO:tasks.workunit.client.1.smithi130.stdout:9/392: dread d1/d15/f3b [0,4194304] 0 2024-05-13T22:48:30.580 INFO:tasks.workunit.client.1.smithi130.stdout:1/382: mkdir db/d1b/d1f/d49/d56/d7f 0 2024-05-13T22:48:30.581 INFO:tasks.workunit.client.1.smithi130.stdout:4/379: dwrite d8/dc/d26/f2b [0,4194304] 0 2024-05-13T22:48:30.582 INFO:tasks.workunit.client.1.smithi130.stdout:7/329: dread d7/da/f35 [0,4194304] 0 2024-05-13T22:48:30.586 INFO:tasks.workunit.client.1.smithi130.stdout:8/318: chown d1/d2/db/d15/d16/f5b 627735 1 2024-05-13T22:48:30.590 INFO:tasks.workunit.client.1.smithi130.stdout:2/394: creat d4/db/d26/f8f x:0 0 0 2024-05-13T22:48:30.590 INFO:tasks.workunit.client.1.smithi130.stdout:1/383: mknod db/c80 0 2024-05-13T22:48:30.592 INFO:tasks.workunit.client.1.smithi130.stdout:7/330: write d7/da/d1d/d3d/d46/f4b [1205359,60556] 0 2024-05-13T22:48:30.595 INFO:tasks.workunit.client.1.smithi130.stdout:3/366: dwrite d8/dc/d15/f66 [0,4194304] 0 2024-05-13T22:48:30.596 INFO:tasks.workunit.client.1.smithi130.stdout:0/381: dwrite d0/d7/dd/d17/d1f/d2c/d45/d57/d78/f84 [0,4194304] 0 2024-05-13T22:48:30.599 INFO:tasks.workunit.client.1.smithi130.stdout:9/393: creat d1/d7/d23/d3d/d43/f75 x:0 0 0 2024-05-13T22:48:30.601 INFO:tasks.workunit.client.1.smithi130.stdout:7/331: truncate d7/da/f62 516286 0 2024-05-13T22:48:30.602 INFO:tasks.workunit.client.1.smithi130.stdout:0/382: dread - d0/d7/d29/f74 zero size 2024-05-13T22:48:30.603 INFO:tasks.workunit.client.1.smithi130.stdout:2/395: mkdir d4/db/dc/d18/d2a/d30/d88/d90 0 2024-05-13T22:48:30.607 INFO:tasks.workunit.client.1.smithi130.stdout:7/332: readlink d7/da/d1d/l28 0 2024-05-13T22:48:30.610 INFO:tasks.workunit.client.1.smithi130.stdout:0/383: stat d0/d7/dd/d17/d1f/d2c/d45/c56 0 2024-05-13T22:48:30.617 INFO:tasks.workunit.client.1.smithi130.stdout:9/394: rename d1/db/l17 to d1/d7/d23/d3d/l76 0 2024-05-13T22:48:30.617 INFO:tasks.workunit.client.1.smithi130.stdout:1/384: dread db/d1b/d1f/d49/d56/f7d [0,4194304] 0 2024-05-13T22:48:30.619 INFO:tasks.workunit.client.1.smithi130.stdout:2/396: mkdir d4/db/dc/d18/d2a/d30/d88/d90/d91 0 2024-05-13T22:48:30.620 INFO:tasks.workunit.client.1.smithi130.stdout:9/395: write d1/d7/d23/d3d/f5c [731945,15484] 0 2024-05-13T22:48:30.621 INFO:tasks.workunit.client.1.smithi130.stdout:3/367: dwrite d8/d1a/f47 [0,4194304] 0 2024-05-13T22:48:30.622 INFO:tasks.workunit.client.1.smithi130.stdout:8/319: dwrite d1/d2/db/d15/d18/f76 [0,4194304] 0 2024-05-13T22:48:30.625 INFO:tasks.workunit.client.1.smithi130.stdout:4/380: dwrite d8/dc/d26/f52 [0,4194304] 0 2024-05-13T22:48:30.625 INFO:tasks.workunit.client.1.smithi130.stdout:2/397: dread - d4/d49/d48/d4a/f82 zero size 2024-05-13T22:48:30.632 INFO:tasks.workunit.client.1.smithi130.stdout:7/333: dwrite d7/da/d1d/d42/f44 [0,4194304] 0 2024-05-13T22:48:30.634 INFO:tasks.workunit.client.1.smithi130.stdout:1/385: unlink db/f14 0 2024-05-13T22:48:30.634 INFO:tasks.workunit.client.1.smithi130.stdout:2/398: write d4/d56/f61 [5215745,28815] 0 2024-05-13T22:48:30.635 INFO:tasks.workunit.client.0.smithi102.stdout:1/960: sync 2024-05-13T22:48:30.637 INFO:tasks.workunit.client.1.smithi130.stdout:0/384: rename d0/d7/dd/f79 to d0/d7/dd/d17/d52/f87 0 2024-05-13T22:48:30.637 INFO:tasks.workunit.client.0.smithi102.stdout:1/961: truncate d2/d45/d79/f117 163358 0 2024-05-13T22:48:30.638 INFO:tasks.workunit.client.1.smithi130.stdout:5/342: dread d6/d7/d24/f65 [0,4194304] 0 2024-05-13T22:48:30.640 INFO:tasks.workunit.client.1.smithi130.stdout:4/381: mkdir d8/dc/d35/d33/d44/d77 0 2024-05-13T22:48:30.640 INFO:tasks.workunit.client.1.smithi130.stdout:7/334: stat d7/da/d1d/d3d/d46/d67/d40/l55 0 2024-05-13T22:48:30.641 INFO:tasks.workunit.client.1.smithi130.stdout:3/368: mkdir d8/dc/d7b 0 2024-05-13T22:48:30.641 INFO:tasks.workunit.client.1.smithi130.stdout:9/396: dread d1/d7/d23/d28/d32/f62 [4194304,4194304] 0 2024-05-13T22:48:30.641 INFO:tasks.workunit.client.1.smithi130.stdout:2/399: read - d4/d49/d48/d4a/d64/f6a zero size 2024-05-13T22:48:30.643 INFO:tasks.workunit.client.1.smithi130.stdout:4/382: write d8/dc/d29/f3a [5138312,79411] 0 2024-05-13T22:48:30.643 INFO:tasks.workunit.client.1.smithi130.stdout:0/385: truncate d0/d7/d29/f37 1666513 0 2024-05-13T22:48:30.644 INFO:tasks.workunit.client.1.smithi130.stdout:7/335: write d7/da/d1d/d42/f64 [301820,59404] 0 2024-05-13T22:48:30.645 INFO:tasks.workunit.client.1.smithi130.stdout:9/397: chown d1/f3 111367 1 2024-05-13T22:48:30.645 INFO:tasks.workunit.client.0.smithi102.stdout:1/962: dwrite d2/d10/d62/d7b/fd9 [0,4194304] 0 2024-05-13T22:48:30.647 INFO:tasks.workunit.client.1.smithi130.stdout:7/336: chown d7/da/d1d/c61 32718 1 2024-05-13T22:48:30.648 INFO:tasks.workunit.client.1.smithi130.stdout:1/386: creat db/d1b/d1f/d72/d75/f81 x:0 0 0 2024-05-13T22:48:30.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:30 smithi130 ceph-mon[36591]: pgmap v32: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 64 MiB/s rd, 209 MiB/s wr, 498 op/s 2024-05-13T22:48:30.650 INFO:tasks.workunit.client.1.smithi130.stdout:1/387: chown db/d1b/d1f/d2a/d35/f3b 6597 1 2024-05-13T22:48:30.651 INFO:tasks.workunit.client.0.smithi102.stdout:1/963: symlink d2/d4/d5/d19/ded/dff/l12f 0 2024-05-13T22:48:30.654 INFO:tasks.workunit.client.1.smithi130.stdout:5/343: fsync d6/dd/f13 0 2024-05-13T22:48:30.656 INFO:tasks.workunit.client.0.smithi102.stdout:1/964: rename d2/d4/c7d to d2/d4/d5/d19/ded/d6a/dbd/c130 0 2024-05-13T22:48:30.656 INFO:tasks.workunit.client.0.smithi102.stdout:1/965: dread - d2/d10/d62/d86/f109 zero size 2024-05-13T22:48:30.657 INFO:tasks.workunit.client.1.smithi130.stdout:8/320: link d1/d2/c1d d1/d2/db/d15/d18/d1a/c77 0 2024-05-13T22:48:30.657 INFO:tasks.workunit.client.0.smithi102.stdout:1/966: readlink d2/d4/d5/d19/ded/d10d/l116 0 2024-05-13T22:48:30.658 INFO:tasks.workunit.client.1.smithi130.stdout:8/321: write d1/d2/db/d15/d16/f2c [996044,52622] 0 2024-05-13T22:48:30.658 INFO:tasks.workunit.client.0.smithi102.stdout:1/967: dread d2/f4b [0,4194304] 0 2024-05-13T22:48:30.659 INFO:tasks.workunit.client.1.smithi130.stdout:8/322: fsync d1/d2/db/d15/d18/f5f 0 2024-05-13T22:48:30.659 INFO:tasks.workunit.client.1.smithi130.stdout:8/323: chown d1/d2/f39 821880 1 2024-05-13T22:48:30.660 INFO:tasks.workunit.client.1.smithi130.stdout:8/324: chown d1/d2/db/d15/d18 17 1 2024-05-13T22:48:30.660 INFO:tasks.workunit.client.1.smithi130.stdout:9/398: creat d1/d7/d23/d28/d2d/d2f/d63/f77 x:0 0 0 2024-05-13T22:48:30.661 INFO:tasks.workunit.client.1.smithi130.stdout:9/399: read d1/f5f [331898,40804] 0 2024-05-13T22:48:30.664 INFO:tasks.workunit.client.1.smithi130.stdout:0/386: rename d0/d7/dd/d17/d1f/d2c/d45/d57/d78 to d0/d7/dd/d69/d88 0 2024-05-13T22:48:30.665 INFO:tasks.workunit.client.1.smithi130.stdout:0/387: write d0/d7/dd/d69/d88/f83 [1026296,90535] 0 2024-05-13T22:48:30.666 INFO:tasks.workunit.client.0.smithi102.stdout:1/968: rename d2/d4/d5/f12d to d2/d4/d5/d19/dd3/f131 0 2024-05-13T22:48:30.666 INFO:tasks.workunit.client.1.smithi130.stdout:0/388: fsync d0/f21 0 2024-05-13T22:48:30.667 INFO:tasks.workunit.client.0.smithi102.stdout:1/969: fsync d2/d45/f121 0 2024-05-13T22:48:30.667 INFO:tasks.workunit.client.1.smithi130.stdout:0/389: chown d0/d7 1 1 2024-05-13T22:48:30.669 INFO:tasks.workunit.client.0.smithi102.stdout:1/970: getdents d2/d4/d5/d19/ded/d6a/dbd/dec 0 2024-05-13T22:48:30.672 INFO:tasks.workunit.client.1.smithi130.stdout:8/325: mknod d1/d2/db/d15/d18/d42/d53/c78 0 2024-05-13T22:48:30.672 INFO:tasks.workunit.client.0.smithi102.stdout:1/971: creat d2/d4/d5/df0/f132 x:0 0 0 2024-05-13T22:48:30.673 INFO:tasks.workunit.client.1.smithi130.stdout:9/400: fdatasync d1/f8 0 2024-05-13T22:48:30.673 INFO:tasks.workunit.client.1.smithi130.stdout:4/383: dwrite d8/dc/d35/d33/d3f/f6d [0,4194304] 0 2024-05-13T22:48:30.674 INFO:tasks.workunit.client.1.smithi130.stdout:1/388: mkdir db/d82 0 2024-05-13T22:48:30.675 INFO:tasks.workunit.client.1.smithi130.stdout:9/401: read d1/d15/f37 [1810046,23163] 0 2024-05-13T22:48:30.677 INFO:tasks.workunit.client.1.smithi130.stdout:3/369: creat d8/dc/d19/f7c x:0 0 0 2024-05-13T22:48:30.678 INFO:tasks.workunit.client.1.smithi130.stdout:2/400: truncate d4/db/f22 8267999 0 2024-05-13T22:48:30.678 INFO:tasks.workunit.client.0.smithi102.stdout:1/972: fsync d2/d10/d62/d7b/dc4/fc8 0 2024-05-13T22:48:30.680 INFO:tasks.workunit.client.1.smithi130.stdout:3/370: fsync d8/d1a/f7a 0 2024-05-13T22:48:30.683 INFO:tasks.workunit.client.1.smithi130.stdout:9/402: write d1/d7/f55 [3320691,74080] 0 2024-05-13T22:48:30.683 INFO:tasks.workunit.client.1.smithi130.stdout:7/337: creat d7/f71 x:0 0 0 2024-05-13T22:48:30.684 INFO:tasks.workunit.client.1.smithi130.stdout:2/401: read - d4/db/dc/d18/d2a/f2f zero size 2024-05-13T22:48:30.685 INFO:tasks.workunit.client.1.smithi130.stdout:5/344: mknod d6/d7/d19/d4d/c70 0 2024-05-13T22:48:30.686 INFO:tasks.workunit.client.1.smithi130.stdout:0/390: rename d0/d7/dd/d2e/f2f to d0/d7/dd/d17/d1f/d2c/d45/d57/f89 0 2024-05-13T22:48:30.688 INFO:tasks.workunit.client.1.smithi130.stdout:7/338: dread - d7/da/f5c zero size 2024-05-13T22:48:30.688 INFO:tasks.workunit.client.1.smithi130.stdout:3/371: symlink d8/d1a/d55/l7d 0 2024-05-13T22:48:30.689 INFO:tasks.workunit.client.1.smithi130.stdout:8/326: dwrite d1/d2/db/d15/d16/f2c [0,4194304] 0 2024-05-13T22:48:30.692 INFO:tasks.workunit.client.1.smithi130.stdout:4/384: dwrite f6 [4194304,4194304] 0 2024-05-13T22:48:30.692 INFO:tasks.workunit.client.0.smithi102.stdout:1/973: write d2/d4/d38/f89 [589511,118120] 0 2024-05-13T22:48:30.693 INFO:tasks.workunit.client.1.smithi130.stdout:3/372: write d8/dc/d15/f58 [507726,60000] 0 2024-05-13T22:48:30.695 INFO:tasks.workunit.client.1.smithi130.stdout:1/389: rename db/d1b/f20 to db/d1b/d1f/d49/d56/d7f/f83 0 2024-05-13T22:48:30.700 INFO:tasks.workunit.client.1.smithi130.stdout:5/345: mkdir d6/d7/d24/d71 0 2024-05-13T22:48:30.700 INFO:tasks.workunit.client.0.smithi102.stdout:1/974: symlink d2/d4/l133 0 2024-05-13T22:48:30.700 INFO:tasks.workunit.client.1.smithi130.stdout:5/346: write d6/d7/f25 [299143,86238] 0 2024-05-13T22:48:30.701 INFO:tasks.workunit.client.1.smithi130.stdout:9/403: creat d1/d15/f78 x:0 0 0 2024-05-13T22:48:30.704 INFO:tasks.workunit.client.0.smithi102.stdout:1/975: creat d2/d4/d38/db2/dba/d10b/f134 x:0 0 0 2024-05-13T22:48:30.705 INFO:tasks.workunit.client.1.smithi130.stdout:0/391: stat d0/d7/dd/d17/d1f/d2c/d45/d57/f89 0 2024-05-13T22:48:30.706 INFO:tasks.workunit.client.1.smithi130.stdout:0/392: fsync d0/d7/d29/f37 0 2024-05-13T22:48:30.708 INFO:tasks.workunit.client.1.smithi130.stdout:2/402: rename d4/f1f to d4/d49/d48/d4a/d76/f92 0 2024-05-13T22:48:30.711 INFO:tasks.workunit.client.1.smithi130.stdout:1/390: unlink db/f32 0 2024-05-13T22:48:30.712 INFO:tasks.workunit.client.1.smithi130.stdout:1/391: chown db/d1b/d1f/d2a/d38/d46/d6d 28600 1 2024-05-13T22:48:30.714 INFO:tasks.workunit.client.0.smithi102.stdout:1/976: rmdir d2/d4/d5/d19/dbe/d127 0 2024-05-13T22:48:30.715 INFO:tasks.workunit.client.0.smithi102.stdout:1/977: readlink d2/d4/d38/db2/dba/l106 0 2024-05-13T22:48:30.715 INFO:tasks.workunit.client.1.smithi130.stdout:9/404: mkdir d1/d7/d69/d79 0 2024-05-13T22:48:30.717 INFO:tasks.workunit.client.1.smithi130.stdout:1/392: dwrite db/d1b/d1f/d2a/d35/f4d [0,4194304] 0 2024-05-13T22:48:30.718 INFO:tasks.workunit.client.1.smithi130.stdout:8/327: rename d1/d2/db/d15/d18/d42/l57 to d1/d2/db/d15/d18/d42/d53/l79 0 2024-05-13T22:48:30.720 INFO:tasks.workunit.client.1.smithi130.stdout:2/403: mkdir d4/db/dc/d18/d2a/d30/d88/d90/d93 0 2024-05-13T22:48:30.721 INFO:tasks.workunit.client.0.smithi102.stdout:1/978: dwrite d2/d4/d26/d40/f72 [0,4194304] 0 2024-05-13T22:48:30.723 INFO:tasks.workunit.client.1.smithi130.stdout:1/393: mknod db/d1b/d1f/d49/d56/c84 0 2024-05-13T22:48:30.725 INFO:tasks.workunit.client.1.smithi130.stdout:0/393: rename d0/d7/dd/d17/d52/c71 to d0/d7/dd/d17/d1f/d2c/d45/d6b/c8a 0 2024-05-13T22:48:30.725 INFO:tasks.workunit.client.1.smithi130.stdout:8/328: dwrite d1/d2/db/d15/d18/f76 [4194304,4194304] 0 2024-05-13T22:48:30.726 INFO:tasks.workunit.client.0.smithi102.stdout:1/979: symlink d2/d4/d38/db2/l135 0 2024-05-13T22:48:30.727 INFO:tasks.workunit.client.1.smithi130.stdout:0/394: chown d0/d7/dd/d17/d1f/f25 25 1 2024-05-13T22:48:30.728 INFO:tasks.workunit.client.1.smithi130.stdout:3/373: link d8/dc/d19/d2c/l3a d8/dc/d7b/l7e 0 2024-05-13T22:48:30.730 INFO:tasks.workunit.client.0.smithi102.stdout:1/980: stat d2/d4/c43 0 2024-05-13T22:48:30.731 INFO:tasks.workunit.client.0.smithi102.stdout:1/981: fsync d2/d45/d96/d113/f122 0 2024-05-13T22:48:30.732 INFO:tasks.workunit.client.0.smithi102.stdout:1/982: write d2/d4/d5/df0/f132 [830176,34124] 0 2024-05-13T22:48:30.733 INFO:tasks.workunit.client.1.smithi130.stdout:8/329: rename d1/dc/f3a to d1/d2/db/d15/d16/d54/d58/f7a 0 2024-05-13T22:48:30.734 INFO:tasks.workunit.client.1.smithi130.stdout:5/347: getdents d6 0 2024-05-13T22:48:30.737 INFO:tasks.workunit.client.1.smithi130.stdout:7/339: dread d7/d8/d9/d66/f31 [0,4194304] 0 2024-05-13T22:48:30.738 INFO:tasks.workunit.client.1.smithi130.stdout:7/340: read - d7/da/d1d/d3d/d46/f6e zero size 2024-05-13T22:48:30.739 INFO:tasks.workunit.client.1.smithi130.stdout:7/341: chown d7/da/d1d/f6b 1 1 2024-05-13T22:48:30.739 INFO:tasks.workunit.client.0.smithi102.stdout:1/983: link d2/d45/f121 d2/d10/d62/d7b/dc4/d128/f136 0 2024-05-13T22:48:30.740 INFO:tasks.workunit.client.1.smithi130.stdout:7/342: write d7/d8/d9/d66/f63 [246117,114941] 0 2024-05-13T22:48:30.741 INFO:tasks.workunit.client.1.smithi130.stdout:8/330: fsync d1/d2/db/d15/f22 0 2024-05-13T22:48:30.743 INFO:tasks.workunit.client.1.smithi130.stdout:8/331: chown d1/d2/db/d15/ca 4 1 2024-05-13T22:48:30.748 INFO:tasks.workunit.client.1.smithi130.stdout:9/405: getdents d1/d7/d69 0 2024-05-13T22:48:30.750 INFO:tasks.workunit.client.1.smithi130.stdout:5/348: dread - d6/d7/d19/d4d/f60 zero size 2024-05-13T22:48:30.751 INFO:tasks.workunit.client.1.smithi130.stdout:8/332: dwrite d1/d2/db/d15/d18/f4b [0,4194304] 0 2024-05-13T22:48:30.752 INFO:tasks.workunit.client.1.smithi130.stdout:3/374: rename d8/f64 to d8/dc/d19/f7f 0 2024-05-13T22:48:30.758 INFO:tasks.workunit.client.1.smithi130.stdout:9/406: rmdir d1/d7/d23/d28/d32/d45 39 2024-05-13T22:48:30.760 INFO:tasks.workunit.client.1.smithi130.stdout:5/349: mkdir d6/d1a/d72 0 2024-05-13T22:48:30.760 INFO:tasks.workunit.client.1.smithi130.stdout:5/350: readlink d6/d7/l20 0 2024-05-13T22:48:30.762 INFO:tasks.workunit.client.1.smithi130.stdout:3/375: creat d8/d35/f80 x:0 0 0 2024-05-13T22:48:30.762 INFO:tasks.workunit.client.1.smithi130.stdout:3/376: stat d8/d1a 0 2024-05-13T22:48:30.763 INFO:tasks.workunit.client.1.smithi130.stdout:3/377: write d8/dc/d19/d27/f28 [605015,47326] 0 2024-05-13T22:48:30.763 INFO:tasks.workunit.client.1.smithi130.stdout:8/333: rename d1/d2/db/d15/d18/d42/d53/c78 to d1/d2/db/d15/c7b 0 2024-05-13T22:48:30.767 INFO:tasks.workunit.client.1.smithi130.stdout:9/407: dread - d1/d15/d51/f54 zero size 2024-05-13T22:48:30.767 INFO:tasks.workunit.client.1.smithi130.stdout:7/343: link d7/da/d1d/d3d/d46/d67/c25 d7/c72 0 2024-05-13T22:48:30.769 INFO:tasks.workunit.client.0.smithi102.stdout:1/984: write d2/d45/d79/d92/f64 [1594144,23704] 0 2024-05-13T22:48:30.769 INFO:tasks.workunit.client.1.smithi130.stdout:0/395: dread d0/f4 [0,4194304] 0 2024-05-13T22:48:30.772 INFO:tasks.workunit.client.1.smithi130.stdout:3/378: link f3 d8/dc/d42/f81 0 2024-05-13T22:48:30.774 INFO:tasks.workunit.client.1.smithi130.stdout:1/394: dwrite db/d1b/d1f/f7a [0,4194304] 0 2024-05-13T22:48:30.775 INFO:tasks.workunit.client.0.smithi102.stdout:1/985: rename d2/d4/d5/d19/ded/d6a/dbd to d2/d45/df4/d137 0 2024-05-13T22:48:30.775 INFO:tasks.workunit.client.0.smithi102.stdout:1/986: stat d2/d4/d26/l4f 0 2024-05-13T22:48:30.776 INFO:tasks.workunit.client.1.smithi130.stdout:9/408: creat d1/d7/d23/f7a x:0 0 0 2024-05-13T22:48:30.777 INFO:tasks.workunit.client.1.smithi130.stdout:8/334: dread d1/d2/db/d15/d18/f76 [0,4194304] 0 2024-05-13T22:48:30.777 INFO:tasks.workunit.client.1.smithi130.stdout:3/379: truncate d8/d1a/d1c/f25 4477758 0 2024-05-13T22:48:30.778 INFO:tasks.workunit.client.0.smithi102.stdout:1/987: stat d2/d4/d26/d40/d5e/le4 0 2024-05-13T22:48:30.779 INFO:tasks.workunit.client.1.smithi130.stdout:0/396: truncate d0/d7/dd/d17/d1f/d2c/d5a/f66 522809 0 2024-05-13T22:48:30.780 INFO:tasks.workunit.client.1.smithi130.stdout:7/344: dwrite d7/d8/f2d [4194304,4194304] 0 2024-05-13T22:48:30.780 INFO:tasks.workunit.client.1.smithi130.stdout:5/351: write d6/dd/d18/f15 [2877600,73878] 0 2024-05-13T22:48:30.781 INFO:tasks.workunit.client.1.smithi130.stdout:1/395: symlink db/d1b/d1f/d2a/d35/l85 0 2024-05-13T22:48:30.785 INFO:tasks.workunit.client.1.smithi130.stdout:5/352: creat d6/f73 x:0 0 0 2024-05-13T22:48:30.785 INFO:tasks.workunit.client.1.smithi130.stdout:9/409: truncate d1/d7/d23/d3d/d43/f75 422868 0 2024-05-13T22:48:30.786 INFO:tasks.workunit.client.1.smithi130.stdout:7/345: write d7/da/d1d/d3d/d46/f6e [344351,51969] 0 2024-05-13T22:48:30.786 INFO:tasks.workunit.client.1.smithi130.stdout:0/397: symlink d0/l8b 0 2024-05-13T22:48:30.791 INFO:tasks.workunit.client.1.smithi130.stdout:0/398: fsync d0/d7/dd/f41 0 2024-05-13T22:48:30.792 INFO:tasks.workunit.client.1.smithi130.stdout:7/346: creat d7/da/d1d/d42/f73 x:0 0 0 2024-05-13T22:48:30.793 INFO:tasks.workunit.client.1.smithi130.stdout:7/347: dread - d7/da/d1d/f6f zero size 2024-05-13T22:48:30.795 INFO:tasks.workunit.client.1.smithi130.stdout:9/410: rename d1/d7/d23/d28/d2d/f42 to d1/d15/f7b 0 2024-05-13T22:48:30.795 INFO:tasks.workunit.client.1.smithi130.stdout:8/335: dwrite d1/dc/f44 [4194304,4194304] 0 2024-05-13T22:48:30.799 INFO:tasks.workunit.client.1.smithi130.stdout:7/348: creat d7/da/d1d/d42/f74 x:0 0 0 2024-05-13T22:48:30.801 INFO:tasks.workunit.client.1.smithi130.stdout:1/396: write db/d1b/d1f/d49/f61 [1548530,69936] 0 2024-05-13T22:48:30.801 INFO:tasks.workunit.client.1.smithi130.stdout:9/411: creat d1/d7/d23/d28/f7c x:0 0 0 2024-05-13T22:48:30.802 INFO:tasks.workunit.client.1.smithi130.stdout:0/399: dwrite d0/f42 [4194304,4194304] 0 2024-05-13T22:48:30.804 INFO:tasks.workunit.client.1.smithi130.stdout:7/349: write d7/da/d1d/d3d/d46/f4b [2154643,72570] 0 2024-05-13T22:48:30.804 INFO:tasks.workunit.client.1.smithi130.stdout:3/380: dread d8/dc/f12 [0,4194304] 0 2024-05-13T22:48:30.808 INFO:tasks.workunit.client.1.smithi130.stdout:8/336: link d1/d2/db/d15/d18/f63 d1/d2/db/d15/d18/d42/d53/d56/f7c 0 2024-05-13T22:48:30.809 INFO:tasks.workunit.client.1.smithi130.stdout:1/397: write db/f1a [8419022,79498] 0 2024-05-13T22:48:30.811 INFO:tasks.workunit.client.1.smithi130.stdout:6/396: sync 2024-05-13T22:48:30.811 INFO:tasks.workunit.client.1.smithi130.stdout:3/381: creat d8/dc/d19/d27/f82 x:0 0 0 2024-05-13T22:48:30.813 INFO:tasks.workunit.client.1.smithi130.stdout:8/337: write d1/d2/db/d15/d18/d42/f62 [3954268,116830] 0 2024-05-13T22:48:30.813 INFO:tasks.workunit.client.1.smithi130.stdout:1/398: chown db/d1b/d1f/d2a/f41 277927877 1 2024-05-13T22:48:30.815 INFO:tasks.workunit.client.1.smithi130.stdout:0/400: link d0/d7/dd/d17/d52/d67/f6a d0/d7/dd/d2e/f8c 0 2024-05-13T22:48:30.815 INFO:tasks.workunit.client.1.smithi130.stdout:5/353: truncate d6/d7/d19/d4d/f34 882875 0 2024-05-13T22:48:30.815 INFO:tasks.workunit.client.1.smithi130.stdout:8/338: readlink d1/d2/db/d15/d16/l20 0 2024-05-13T22:48:30.816 INFO:tasks.workunit.client.1.smithi130.stdout:6/397: chown d4/d15/d1f/d46/d50/c6e 854 1 2024-05-13T22:48:30.819 INFO:tasks.workunit.client.1.smithi130.stdout:0/401: unlink d0/d7/dd/d17/d1f/d2c/d45/c7e 0 2024-05-13T22:48:30.820 INFO:tasks.workunit.client.1.smithi130.stdout:6/398: symlink d4/d15/d1f/d64/l90 0 2024-05-13T22:48:30.821 INFO:tasks.workunit.client.1.smithi130.stdout:1/399: dread db/d1b/d1f/d2a/d38/d46/d6d/f73 [0,4194304] 0 2024-05-13T22:48:30.821 INFO:tasks.workunit.client.1.smithi130.stdout:8/339: fdatasync d1/d2/db/d15/d16/d54/d58/f7a 0 2024-05-13T22:48:30.822 INFO:tasks.workunit.client.1.smithi130.stdout:5/354: mknod d6/d1a/d72/c74 0 2024-05-13T22:48:30.823 INFO:tasks.workunit.client.1.smithi130.stdout:5/355: readlink d6/d1a/l4f 0 2024-05-13T22:48:30.823 INFO:tasks.workunit.client.1.smithi130.stdout:0/402: write d0/d7/dd/d17/d1f/d2c/f7a [465011,1288] 0 2024-05-13T22:48:30.825 INFO:tasks.workunit.client.1.smithi130.stdout:1/400: chown db/c11 174 1 2024-05-13T22:48:30.826 INFO:tasks.workunit.client.1.smithi130.stdout:1/401: write db/d1b/d1f/d2a/d35/f31 [2624367,13473] 0 2024-05-13T22:48:30.826 INFO:tasks.workunit.client.1.smithi130.stdout:8/340: mknod d1/d2/db/d15/d16/d54/c7d 0 2024-05-13T22:48:30.827 INFO:tasks.workunit.client.1.smithi130.stdout:1/402: read f8 [1027262,16380] 0 2024-05-13T22:48:30.829 INFO:tasks.workunit.client.1.smithi130.stdout:9/412: dread d1/d7/f55 [0,4194304] 0 2024-05-13T22:48:30.829 INFO:tasks.workunit.client.1.smithi130.stdout:1/403: write db/d1b/d1f/d2a/d35/f4d [3589297,30301] 0 2024-05-13T22:48:30.829 INFO:tasks.workunit.client.1.smithi130.stdout:9/413: chown d1/d7/c20 0 1 2024-05-13T22:48:30.832 INFO:tasks.workunit.client.1.smithi130.stdout:0/403: mkdir d0/d7/dd/d2e/d77/d8d 0 2024-05-13T22:48:30.832 INFO:tasks.workunit.client.1.smithi130.stdout:9/414: chown d1/d7/d23/d28/d2d/f74 82712895 1 2024-05-13T22:48:30.837 INFO:tasks.workunit.client.1.smithi130.stdout:7/350: dread d7/d8/f1c [0,4194304] 0 2024-05-13T22:48:30.837 INFO:tasks.workunit.client.1.smithi130.stdout:6/399: getdents d4/d15/d1f/d64 0 2024-05-13T22:48:30.838 INFO:tasks.workunit.client.1.smithi130.stdout:3/382: dwrite f2 [0,4194304] 0 2024-05-13T22:48:30.840 INFO:tasks.workunit.client.1.smithi130.stdout:6/400: dread - d4/d15/d1d/d60/f66 zero size 2024-05-13T22:48:30.846 INFO:tasks.workunit.client.1.smithi130.stdout:0/404: dwrite d0/d7/dd/d2e/d4b/f65 [0,4194304] 0 2024-05-13T22:48:30.846 INFO:tasks.workunit.client.1.smithi130.stdout:6/401: readlink d4/d15/d1d/l5a 0 2024-05-13T22:48:30.846 INFO:tasks.workunit.client.1.smithi130.stdout:7/351: rename l6 to d7/da/d1d/l75 0 2024-05-13T22:48:30.848 INFO:tasks.workunit.client.1.smithi130.stdout:7/352: write d7/da/d1d/d42/f74 [126671,22147] 0 2024-05-13T22:48:30.850 INFO:tasks.workunit.client.1.smithi130.stdout:6/402: unlink d4/d15/d1f/f26 0 2024-05-13T22:48:30.851 INFO:tasks.workunit.client.1.smithi130.stdout:0/405: mknod d0/c8e 0 2024-05-13T22:48:30.853 INFO:tasks.workunit.client.1.smithi130.stdout:0/406: stat d0/d7/dd/d2e/d4b 0 2024-05-13T22:48:30.853 INFO:tasks.workunit.client.1.smithi130.stdout:6/403: write d4/d15/d1f/f55 [634723,122741] 0 2024-05-13T22:48:30.853 INFO:tasks.workunit.client.1.smithi130.stdout:7/353: unlink d7/da/d1d/d42/f44 0 2024-05-13T22:48:30.856 INFO:tasks.workunit.client.1.smithi130.stdout:3/383: dwrite d8/dc/f22 [0,4194304] 0 2024-05-13T22:48:30.858 INFO:tasks.workunit.client.1.smithi130.stdout:0/407: write d0/d7/dd/d17/d1f/d2c/d45/d57/f62 [141841,94425] 0 2024-05-13T22:48:30.859 INFO:tasks.workunit.client.1.smithi130.stdout:0/408: fdatasync d0/f42 0 2024-05-13T22:48:30.859 INFO:tasks.workunit.client.1.smithi130.stdout:7/354: rename d7/d8/d9/l34 to d7/da/d1d/d3d/d46/d67/l76 0 2024-05-13T22:48:30.860 INFO:tasks.workunit.client.1.smithi130.stdout:3/384: mkdir d8/dc/d19/d83 0 2024-05-13T22:48:30.861 INFO:tasks.workunit.client.1.smithi130.stdout:3/385: stat d8/f78 0 2024-05-13T22:48:30.863 INFO:tasks.workunit.client.1.smithi130.stdout:3/386: fsync d8/f5e 0 2024-05-13T22:48:30.865 INFO:tasks.workunit.client.1.smithi130.stdout:6/404: dwrite d4/d15/d1f/d46/d52/f87 [0,4194304] 0 2024-05-13T22:48:30.870 INFO:tasks.workunit.client.1.smithi130.stdout:7/355: getdents d7/da/d1d/d42 0 2024-05-13T22:48:30.873 INFO:tasks.workunit.client.1.smithi130.stdout:7/356: creat d7/da/d1d/d3d/d46/d67/d40/f77 x:0 0 0 2024-05-13T22:48:30.874 INFO:tasks.workunit.client.1.smithi130.stdout:7/357: chown d7/da/d1d/d5d 60 1 2024-05-13T22:48:30.875 INFO:tasks.workunit.client.1.smithi130.stdout:6/405: write d4/d16/f39 [1175605,102067] 0 2024-05-13T22:48:30.876 INFO:tasks.workunit.client.1.smithi130.stdout:7/358: write d7/da/d1d/d3d/d46/f4b [2387999,1842] 0 2024-05-13T22:48:30.879 INFO:tasks.workunit.client.1.smithi130.stdout:0/409: dwrite d0/f4 [0,4194304] 0 2024-05-13T22:48:30.880 INFO:tasks.workunit.client.1.smithi130.stdout:7/359: rmdir d7/d8 39 2024-05-13T22:48:30.917 INFO:tasks.workunit.client.1.smithi130.stdout:2/404: sync 2024-05-13T22:48:30.918 INFO:tasks.workunit.client.1.smithi130.stdout:4/385: sync 2024-05-13T22:48:30.935 INFO:tasks.workunit.client.1.smithi130.stdout:6/406: dwrite d4/d15/d1d/d28/f5e [0,4194304] 0 2024-05-13T22:48:30.936 INFO:tasks.workunit.client.1.smithi130.stdout:2/405: creat d4/d49/d42/d4b/f94 x:0 0 0 2024-05-13T22:48:30.937 INFO:tasks.workunit.client.1.smithi130.stdout:2/406: dread - d4/d49/d48/d4a/f82 zero size 2024-05-13T22:48:30.938 INFO:tasks.workunit.client.1.smithi130.stdout:0/410: rename d0/d7/dd/d17/d1f/f25 to d0/d7/f8f 0 2024-05-13T22:48:30.939 INFO:tasks.workunit.client.1.smithi130.stdout:7/360: creat d7/d8/f78 x:0 0 0 2024-05-13T22:48:30.941 INFO:tasks.workunit.client.1.smithi130.stdout:7/361: dread - d7/d8/d5b/f6d zero size 2024-05-13T22:48:30.941 INFO:tasks.workunit.client.1.smithi130.stdout:4/386: rename d8/f1e to d8/dc/d13/f78 0 2024-05-13T22:48:30.944 INFO:tasks.workunit.client.1.smithi130.stdout:2/407: dwrite d4/f19 [0,4194304] 0 2024-05-13T22:48:30.949 INFO:tasks.workunit.client.1.smithi130.stdout:2/408: chown d4/d56/f61 331 1 2024-05-13T22:48:30.949 INFO:tasks.workunit.client.1.smithi130.stdout:6/407: getdents d4/d15/d1f 0 2024-05-13T22:48:30.954 INFO:tasks.workunit.client.1.smithi130.stdout:2/409: write d4/db/dc/d18/d2a/d30/d69/f6f [817393,76014] 0 2024-05-13T22:48:30.955 INFO:tasks.workunit.client.1.smithi130.stdout:4/387: dwrite d8/df/f59 [0,4194304] 0 2024-05-13T22:48:30.960 INFO:tasks.workunit.client.1.smithi130.stdout:0/411: dwrite d0/d7/dd/d17/d1f/d2c/d45/d57/f89 [0,4194304] 0 2024-05-13T22:48:30.962 INFO:tasks.workunit.client.1.smithi130.stdout:6/408: dwrite d4/d15/d1d/d28/f8f [0,4194304] 0 2024-05-13T22:48:30.972 INFO:tasks.workunit.client.1.smithi130.stdout:7/362: dread d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:31.004 INFO:tasks.workunit.client.1.smithi130.stdout:5/356: sync 2024-05-13T22:48:31.004 INFO:tasks.workunit.client.1.smithi130.stdout:5/357: fdatasync d6/d7/f43 0 2024-05-13T22:48:31.039 INFO:tasks.workunit.client.1.smithi130.stdout:4/388: mknod d8/dc/d35/d33/d44/d61/c79 0 2024-05-13T22:48:31.043 INFO:tasks.workunit.client.1.smithi130.stdout:6/409: unlink d4/l14 0 2024-05-13T22:48:31.050 INFO:tasks.workunit.client.1.smithi130.stdout:6/410: dwrite d4/d15/d1f/f20 [0,4194304] 0 2024-05-13T22:48:31.050 INFO:tasks.workunit.client.1.smithi130.stdout:2/410: creat d4/d49/f95 x:0 0 0 2024-05-13T22:48:31.053 INFO:tasks.workunit.client.1.smithi130.stdout:0/412: unlink d0/d7/dd/d17/d52/f87 0 2024-05-13T22:48:31.054 INFO:tasks.workunit.client.1.smithi130.stdout:2/411: truncate d4/db/dc/d18/d2a/d30/d69/f77 515551 0 2024-05-13T22:48:31.055 INFO:tasks.workunit.client.1.smithi130.stdout:2/412: chown d4/db/f63 3 1 2024-05-13T22:48:31.062 INFO:tasks.workunit.client.1.smithi130.stdout:7/363: fsync d7/da/f33 0 2024-05-13T22:48:31.068 INFO:tasks.workunit.client.1.smithi130.stdout:8/341: sync 2024-05-13T22:48:31.071 INFO:tasks.workunit.client.1.smithi130.stdout:7/364: dwrite d7/da/d1d/d3d/d46/f4b [0,4194304] 0 2024-05-13T22:48:31.072 INFO:tasks.workunit.client.1.smithi130.stdout:6/411: dwrite d4/d15/d1f/f84 [0,4194304] 0 2024-05-13T22:48:31.081 INFO:tasks.workunit.client.1.smithi130.stdout:7/365: dwrite d7/da/d1d/d42/d43/f4d [0,4194304] 0 2024-05-13T22:48:31.091 INFO:tasks.workunit.client.1.smithi130.stdout:7/366: creat d7/d8/d39/f79 x:0 0 0 2024-05-13T22:48:31.092 INFO:tasks.workunit.client.1.smithi130.stdout:6/412: read d4/d15/d1d/f3d [205907,125346] 0 2024-05-13T22:48:31.094 INFO:tasks.workunit.client.1.smithi130.stdout:9/415: sync 2024-05-13T22:48:31.094 INFO:tasks.workunit.client.1.smithi130.stdout:3/387: sync 2024-05-13T22:48:31.094 INFO:tasks.workunit.client.1.smithi130.stdout:1/404: sync 2024-05-13T22:48:31.102 INFO:tasks.workunit.client.1.smithi130.stdout:9/416: creat d1/d15/f7d x:0 0 0 2024-05-13T22:48:31.102 INFO:tasks.workunit.client.1.smithi130.stdout:7/367: dwrite d7/da/d1d/d42/f74 [0,4194304] 0 2024-05-13T22:48:31.104 INFO:tasks.workunit.client.1.smithi130.stdout:6/413: dread d4/d15/d1f/f84 [0,4194304] 0 2024-05-13T22:48:31.105 INFO:tasks.workunit.client.0.smithi102.stdout:1/988: sync 2024-05-13T22:48:31.107 INFO:tasks.workunit.client.1.smithi130.stdout:1/405: dwrite db/f1a [4194304,4194304] 0 2024-05-13T22:48:31.110 INFO:tasks.workunit.client.1.smithi130.stdout:7/368: creat d7/d8/d9/d66/f7a x:0 0 0 2024-05-13T22:48:31.112 INFO:tasks.workunit.client.1.smithi130.stdout:9/417: mknod d1/d7/d23/d61/c7e 0 2024-05-13T22:48:31.113 INFO:tasks.workunit.client.1.smithi130.stdout:1/406: chown db/f17 9612444 1 2024-05-13T22:48:31.114 INFO:tasks.workunit.client.1.smithi130.stdout:3/388: dwrite d8/dc/d19/d2c/d61/f69 [0,4194304] 0 2024-05-13T22:48:31.114 INFO:tasks.workunit.client.1.smithi130.stdout:2/413: dread d4/db/dc/f20 [0,4194304] 0 2024-05-13T22:48:31.117 INFO:tasks.workunit.client.1.smithi130.stdout:9/418: creat d1/d15/d51/f7f x:0 0 0 2024-05-13T22:48:31.118 INFO:tasks.workunit.client.1.smithi130.stdout:3/389: fdatasync d8/dc/f22 0 2024-05-13T22:48:31.123 INFO:tasks.workunit.client.1.smithi130.stdout:6/414: dwrite d4/d15/d1d/d28/f30 [0,4194304] 0 2024-05-13T22:48:31.126 INFO:tasks.workunit.client.1.smithi130.stdout:9/419: fsync d1/d7/d23/d28/d2d/f3a 0 2024-05-13T22:48:31.127 INFO:tasks.workunit.client.1.smithi130.stdout:1/407: dwrite db/d1b/d1f/d2a/d35/f4d [0,4194304] 0 2024-05-13T22:48:31.128 INFO:tasks.workunit.client.1.smithi130.stdout:9/420: write d1/d7/d23/d28/d2d/f34 [3550396,116248] 0 2024-05-13T22:48:31.128 INFO:tasks.workunit.client.1.smithi130.stdout:2/414: symlink d4/db/dc/d18/d2a/d30/d88/l96 0 2024-05-13T22:48:31.132 INFO:tasks.workunit.client.1.smithi130.stdout:6/415: mknod d4/d15/d1f/d3c/c91 0 2024-05-13T22:48:31.138 INFO:tasks.workunit.client.1.smithi130.stdout:1/408: dwrite db/d1b/d1f/d72/f79 [0,4194304] 0 2024-05-13T22:48:31.139 INFO:tasks.workunit.client.1.smithi130.stdout:9/421: dwrite d1/f4 [0,4194304] 0 2024-05-13T22:48:31.143 INFO:tasks.workunit.client.1.smithi130.stdout:6/416: symlink d4/d15/d1d/d60/l92 0 2024-05-13T22:48:31.146 INFO:tasks.workunit.client.1.smithi130.stdout:6/417: chown d4/d15/d1f/f84 25 1 2024-05-13T22:48:31.151 INFO:tasks.workunit.client.1.smithi130.stdout:1/409: dwrite db/d4a/f6b [0,4194304] 0 2024-05-13T22:48:31.154 INFO:tasks.workunit.client.1.smithi130.stdout:1/410: write db/d1b/d1f/d2a/d35/f31 [2298074,56987] 0 2024-05-13T22:48:31.194 INFO:tasks.workunit.client.1.smithi130.stdout:0/413: write d0/d7/d29/f40 [838635,45339] 0 2024-05-13T22:48:31.195 INFO:tasks.workunit.client.1.smithi130.stdout:4/389: truncate d8/df/f59 3904731 0 2024-05-13T22:48:31.195 INFO:tasks.workunit.client.1.smithi130.stdout:0/414: fdatasync d0/fe 0 2024-05-13T22:48:31.198 INFO:tasks.workunit.client.1.smithi130.stdout:0/415: mknod d0/d7/dd/d17/d52/d67/c90 0 2024-05-13T22:48:31.200 INFO:tasks.workunit.client.1.smithi130.stdout:4/390: truncate d8/dc/f38 453753 0 2024-05-13T22:48:31.200 INFO:tasks.workunit.client.1.smithi130.stdout:4/391: readlink d8/df/l51 0 2024-05-13T22:48:31.202 INFO:tasks.workunit.client.1.smithi130.stdout:0/416: mknod d0/d7/dd/d17/d52/d5b/d5f/c91 0 2024-05-13T22:48:31.204 INFO:tasks.workunit.client.1.smithi130.stdout:4/392: mknod d8/dc/d35/d33/d44/d77/c7a 0 2024-05-13T22:48:31.205 INFO:tasks.workunit.client.1.smithi130.stdout:0/417: unlink d0/d7/dd/d17/d52/d67/f6a 0 2024-05-13T22:48:31.206 INFO:tasks.workunit.client.1.smithi130.stdout:0/418: dread - d0/d7/dd/d17/d52/f80 zero size 2024-05-13T22:48:31.208 INFO:tasks.workunit.client.1.smithi130.stdout:4/393: dwrite d8/dc/d35/f50 [0,4194304] 0 2024-05-13T22:48:31.213 INFO:tasks.workunit.client.1.smithi130.stdout:4/394: dwrite d8/dc/d35/f50 [0,4194304] 0 2024-05-13T22:48:31.219 INFO:tasks.workunit.client.1.smithi130.stdout:4/395: dwrite d8/dc/d26/f55 [0,4194304] 0 2024-05-13T22:48:31.219 INFO:tasks.workunit.client.1.smithi130.stdout:5/358: creat d6/f75 x:0 0 0 2024-05-13T22:48:31.224 INFO:tasks.workunit.client.1.smithi130.stdout:5/359: chown d6/dd/f37 0 1 2024-05-13T22:48:31.228 INFO:tasks.workunit.client.1.smithi130.stdout:5/360: chown d6/d7/d32/f3d 28 1 2024-05-13T22:48:31.228 INFO:tasks.workunit.client.1.smithi130.stdout:8/342: mkdir d1/d2/db/d15/d7e 0 2024-05-13T22:48:31.234 INFO:tasks.workunit.client.1.smithi130.stdout:8/343: chown d1/d2/db/d15/d18/d42/d53/l79 57500 1 2024-05-13T22:48:31.234 INFO:tasks.workunit.client.1.smithi130.stdout:3/390: truncate d8/dc/d15/f66 238313 0 2024-05-13T22:48:31.235 INFO:tasks.workunit.client.1.smithi130.stdout:5/361: dwrite d6/d7/d32/f3d [0,4194304] 0 2024-05-13T22:48:31.237 INFO:tasks.workunit.client.1.smithi130.stdout:2/415: link d4/db/dc/d18/d58/f7b d4/db/dc/d18/d2a/f97 0 2024-05-13T22:48:31.238 INFO:tasks.workunit.client.1.smithi130.stdout:2/416: fsync d4/d56/f61 0 2024-05-13T22:48:31.240 INFO:tasks.workunit.client.1.smithi130.stdout:2/417: write d4/db/dc/d18/f36 [1508610,35915] 0 2024-05-13T22:48:31.241 INFO:tasks.workunit.client.1.smithi130.stdout:8/344: dwrite d1/d2/db/d15/d18/f76 [0,4194304] 0 2024-05-13T22:48:31.242 INFO:tasks.workunit.client.1.smithi130.stdout:5/362: mkdir d6/dd/d18/d6e/d76 0 2024-05-13T22:48:31.243 INFO:tasks.workunit.client.1.smithi130.stdout:6/418: truncate d4/d15/d1f/d46/d50/f53 407720 0 2024-05-13T22:48:31.243 INFO:tasks.workunit.client.1.smithi130.stdout:5/363: stat d6/dd/c1d 0 2024-05-13T22:48:31.245 INFO:tasks.workunit.client.1.smithi130.stdout:5/364: chown d6/d7/d19/d4d/c70 137408538 1 2024-05-13T22:48:31.250 INFO:tasks.workunit.client.1.smithi130.stdout:6/419: chown d4/d15/d1d/c21 22367457 1 2024-05-13T22:48:31.250 INFO:tasks.workunit.client.1.smithi130.stdout:5/365: unlink d6/d7/d19/d4d/c70 0 2024-05-13T22:48:31.255 INFO:tasks.workunit.client.1.smithi130.stdout:2/418: dwrite d4/db/dc/d18/d2a/f97 [0,4194304] 0 2024-05-13T22:48:31.255 INFO:tasks.workunit.client.1.smithi130.stdout:7/369: rename d7/d8/f2d to d7/d8/d39/f7b 0 2024-05-13T22:48:31.255 INFO:tasks.workunit.client.1.smithi130.stdout:8/345: dwrite d1/d2/db/d15/f74 [0,4194304] 0 2024-05-13T22:48:31.257 INFO:tasks.workunit.client.1.smithi130.stdout:4/396: creat d8/f7b x:0 0 0 2024-05-13T22:48:31.259 INFO:tasks.workunit.client.1.smithi130.stdout:8/346: stat d1/d2/db/c2b 0 2024-05-13T22:48:31.265 INFO:tasks.workunit.client.1.smithi130.stdout:9/422: rename d1/d7/d23/d3d/f5a to d1/d7/d69/d79/f80 0 2024-05-13T22:48:31.265 INFO:tasks.workunit.client.1.smithi130.stdout:6/420: dwrite d4/d15/d1f/d46/d50/f67 [0,4194304] 0 2024-05-13T22:48:31.265 INFO:tasks.workunit.client.1.smithi130.stdout:8/347: stat d1/d2/db/d15/d18/d42/f62 0 2024-05-13T22:48:31.268 INFO:tasks.workunit.client.1.smithi130.stdout:4/397: symlink d8/dc/d35/d33/d44/d61/l7c 0 2024-05-13T22:48:31.270 INFO:tasks.workunit.client.1.smithi130.stdout:9/423: symlink d1/d7/d23/d61/l81 0 2024-05-13T22:48:31.271 INFO:tasks.workunit.client.1.smithi130.stdout:9/424: fdatasync d1/d7/d23/d28/f67 0 2024-05-13T22:48:31.274 INFO:tasks.workunit.client.1.smithi130.stdout:1/411: rename db/d1b/d1f/d2a/d35/f31 to db/d82/f86 0 2024-05-13T22:48:31.280 INFO:tasks.workunit.client.1.smithi130.stdout:9/425: creat d1/d7/d23/d3d/d43/f82 x:0 0 0 2024-05-13T22:48:31.280 INFO:tasks.workunit.client.1.smithi130.stdout:9/426: chown d1/d7/d69/d79/f80 202132 1 2024-05-13T22:48:31.282 INFO:tasks.workunit.client.1.smithi130.stdout:7/370: rename d7/da/f56 to d7/d8/d39/f7c 0 2024-05-13T22:48:31.283 INFO:tasks.workunit.client.1.smithi130.stdout:1/412: mkdir db/d1b/d1f/d2a/d38/d46/d6d/d87 0 2024-05-13T22:48:31.284 INFO:tasks.workunit.client.1.smithi130.stdout:0/419: rmdir d0 39 2024-05-13T22:48:31.288 INFO:tasks.workunit.client.1.smithi130.stdout:9/427: creat d1/db/f83 x:0 0 0 2024-05-13T22:48:31.289 INFO:tasks.workunit.client.1.smithi130.stdout:9/428: truncate d1/d15/d51/f6d 131546 0 2024-05-13T22:48:31.289 INFO:tasks.workunit.client.1.smithi130.stdout:6/421: rename d4/d15/d1f/c6d to d4/d15/d1d/d28/d75/d7d/c93 0 2024-05-13T22:48:31.290 INFO:tasks.workunit.client.1.smithi130.stdout:7/371: truncate d7/d8/d39/f7b 4321658 0 2024-05-13T22:48:31.291 INFO:tasks.workunit.client.1.smithi130.stdout:1/413: mkdir db/d1b/d1f/d49/d56/d88 0 2024-05-13T22:48:31.292 INFO:tasks.workunit.client.1.smithi130.stdout:8/348: creat d1/d2/f7f x:0 0 0 2024-05-13T22:48:31.293 INFO:tasks.workunit.client.1.smithi130.stdout:8/349: write d1/d2/db/d15/d18/d42/f62 [3811220,35671] 0 2024-05-13T22:48:31.293 INFO:tasks.workunit.client.1.smithi130.stdout:8/350: fsync d1/d2/f6 0 2024-05-13T22:48:31.297 INFO:tasks.workunit.client.1.smithi130.stdout:9/429: write d1/d7/f55 [3667633,57205] 0 2024-05-13T22:48:31.298 INFO:tasks.workunit.client.1.smithi130.stdout:9/430: chown d1/f4 673 1 2024-05-13T22:48:31.300 INFO:tasks.workunit.client.1.smithi130.stdout:4/398: rename d8/dc/d26/f53 to d8/dc/d13/f7d 0 2024-05-13T22:48:31.301 INFO:tasks.workunit.client.1.smithi130.stdout:3/391: truncate d8/d1a/f4f 4104358 0 2024-05-13T22:48:31.302 INFO:tasks.workunit.client.1.smithi130.stdout:7/372: creat d7/da/d1d/d42/f7d x:0 0 0 2024-05-13T22:48:31.311 INFO:tasks.workunit.client.1.smithi130.stdout:9/431: creat d1/d7/d23/d28/d2d/d2f/d63/d6b/f84 x:0 0 0 2024-05-13T22:48:31.312 INFO:tasks.workunit.client.1.smithi130.stdout:9/432: write d1/db/fc [3244085,104523] 0 2024-05-13T22:48:31.314 INFO:tasks.workunit.client.0.smithi102.stdout:1/989: sync 2024-05-13T22:48:31.315 INFO:tasks.workunit.client.1.smithi130.stdout:6/422: creat d4/d15/d8b/f94 x:0 0 0 2024-05-13T22:48:31.315 INFO:tasks.workunit.client.1.smithi130.stdout:5/366: truncate f4 1197144 0 2024-05-13T22:48:31.315 INFO:tasks.workunit.client.1.smithi130.stdout:3/392: creat d8/dc/d19/d27/f84 x:0 0 0 2024-05-13T22:48:31.316 INFO:tasks.workunit.client.1.smithi130.stdout:5/367: truncate d6/f73 574485 0 2024-05-13T22:48:31.318 INFO:tasks.workunit.client.1.smithi130.stdout:6/423: read d4/f57 [2447301,125707] 0 2024-05-13T22:48:31.321 INFO:tasks.workunit.client.1.smithi130.stdout:2/419: dwrite d4/d49/d42/d4b/f83 [0,4194304] 0 2024-05-13T22:48:31.321 INFO:tasks.workunit.client.1.smithi130.stdout:4/399: dread d8/f1f [0,4194304] 0 2024-05-13T22:48:31.322 INFO:tasks.workunit.client.1.smithi130.stdout:9/433: mkdir d1/d15/d51/d85 0 2024-05-13T22:48:31.323 INFO:tasks.workunit.client.1.smithi130.stdout:2/420: write d4/d49/d42/d4b/f94 [28500,17005] 0 2024-05-13T22:48:31.324 INFO:tasks.workunit.client.1.smithi130.stdout:4/400: chown d8/dc/d35/d33/d3f/l69 539 1 2024-05-13T22:48:31.325 INFO:tasks.workunit.client.1.smithi130.stdout:9/434: dread - d1/d7/d23/d28/f67 zero size 2024-05-13T22:48:31.325 INFO:tasks.workunit.client.1.smithi130.stdout:1/414: creat db/d1b/d1f/f89 x:0 0 0 2024-05-13T22:48:31.325 INFO:tasks.workunit.client.1.smithi130.stdout:9/435: dread - d1/f4f zero size 2024-05-13T22:48:31.326 INFO:tasks.workunit.client.1.smithi130.stdout:7/373: mkdir d7/d8/d7e 0 2024-05-13T22:48:31.328 INFO:tasks.workunit.client.1.smithi130.stdout:5/368: mknod d6/dd/c77 0 2024-05-13T22:48:31.336 INFO:tasks.workunit.client.1.smithi130.stdout:3/393: mkdir d8/dc/d85 0 2024-05-13T22:48:31.337 INFO:tasks.workunit.client.1.smithi130.stdout:2/421: rmdir d4/d49/d48/d4a/d64 39 2024-05-13T22:48:31.338 INFO:tasks.workunit.client.1.smithi130.stdout:9/436: dread d1/d7/d23/d28/d2d/f48 [0,4194304] 0 2024-05-13T22:48:31.339 INFO:tasks.workunit.client.1.smithi130.stdout:2/422: stat d4/db/dc/f4e 0 2024-05-13T22:48:31.342 INFO:tasks.workunit.client.1.smithi130.stdout:5/369: dread d6/dd/d18/f15 [0,4194304] 0 2024-05-13T22:48:31.342 INFO:tasks.workunit.client.1.smithi130.stdout:9/437: read d1/d7/f1a [348234,11028] 0 2024-05-13T22:48:31.343 INFO:tasks.workunit.client.1.smithi130.stdout:5/370: chown d6/d7/l20 8333 1 2024-05-13T22:48:31.344 INFO:tasks.workunit.client.1.smithi130.stdout:6/424: truncate d4/d15/d1f/d46/f58 782983 0 2024-05-13T22:48:31.344 INFO:tasks.workunit.client.1.smithi130.stdout:5/371: write d6/d7/f25 [1078430,70879] 0 2024-05-13T22:48:31.345 INFO:tasks.workunit.client.1.smithi130.stdout:6/425: chown d4/d15/d1f/d46/d52/c86 1543648 1 2024-05-13T22:48:31.345 INFO:tasks.workunit.client.1.smithi130.stdout:3/394: truncate d8/dc/d19/f20 2721431 0 2024-05-13T22:48:31.348 INFO:tasks.workunit.client.1.smithi130.stdout:2/423: rmdir d4/d56 39 2024-05-13T22:48:31.350 INFO:tasks.workunit.client.1.smithi130.stdout:8/351: write d1/d2/db/f8 [2112971,81202] 0 2024-05-13T22:48:31.350 INFO:tasks.workunit.client.1.smithi130.stdout:2/424: readlink d4/db/dc/d18/d2a/d30/l38 0 2024-05-13T22:48:31.351 INFO:tasks.workunit.client.1.smithi130.stdout:0/420: fsync d0/d7/dd/d2e/f4e 0 2024-05-13T22:48:31.352 INFO:tasks.workunit.client.1.smithi130.stdout:2/425: readlink d4/d49/l3c 0 2024-05-13T22:48:31.354 INFO:tasks.workunit.client.1.smithi130.stdout:6/426: dwrite d4/d15/d1f/d46/d52/f76 [0,4194304] 0 2024-05-13T22:48:31.354 INFO:tasks.workunit.client.1.smithi130.stdout:8/352: write d1/d2/db/f45 [84792,128734] 0 2024-05-13T22:48:31.361 INFO:tasks.workunit.client.1.smithi130.stdout:2/426: dwrite d4/d49/f35 [0,4194304] 0 2024-05-13T22:48:31.363 INFO:tasks.workunit.client.1.smithi130.stdout:5/372: dread d6/d7/d19/f2c [0,4194304] 0 2024-05-13T22:48:31.365 INFO:tasks.workunit.client.0.smithi102.stdout:1/990: dwrite d2/d45/d79/f81 [0,4194304] 0 2024-05-13T22:48:31.372 INFO:tasks.workunit.client.0.smithi102.stdout:1/991: dwrite d2/d10/d62/d7b/dc4/fcf [0,4194304] 0 2024-05-13T22:48:31.374 INFO:tasks.workunit.client.1.smithi130.stdout:3/395: symlink d8/dc/d7b/l86 0 2024-05-13T22:48:31.374 INFO:tasks.workunit.client.0.smithi102.stdout:1/992: dread - d2/d4/d5/d19/ded/ff6 zero size 2024-05-13T22:48:31.375 INFO:tasks.workunit.client.1.smithi130.stdout:3/396: dread - d8/dc/d19/f7c zero size 2024-05-13T22:48:31.378 INFO:tasks.workunit.client.1.smithi130.stdout:6/427: symlink d4/l95 0 2024-05-13T22:48:31.378 INFO:tasks.workunit.client.0.smithi102.stdout:1/993: write d2/d4/d5/d19/ded/fd1 [5062678,92684] 0 2024-05-13T22:48:31.379 INFO:tasks.workunit.client.1.smithi130.stdout:7/374: write d7/da/f33 [793909,130517] 0 2024-05-13T22:48:31.380 INFO:tasks.workunit.client.1.smithi130.stdout:9/438: creat d1/d15/f86 x:0 0 0 2024-05-13T22:48:31.381 INFO:tasks.workunit.client.1.smithi130.stdout:7/375: write d7/da/d1d/d3d/d46/f4f [2009850,72388] 0 2024-05-13T22:48:31.381 INFO:tasks.workunit.client.1.smithi130.stdout:9/439: fdatasync d1/d7/f57 0 2024-05-13T22:48:31.391 INFO:tasks.workunit.client.1.smithi130.stdout:3/397: rename d8/dc/d15/f2f to d8/dc/d19/d2c/f87 0 2024-05-13T22:48:31.391 INFO:tasks.workunit.client.1.smithi130.stdout:5/373: unlink d6/d7/d32/l69 0 2024-05-13T22:48:31.397 INFO:tasks.workunit.client.1.smithi130.stdout:0/421: getdents d0/d7/dd/d69/d88 0 2024-05-13T22:48:31.399 INFO:tasks.workunit.client.1.smithi130.stdout:6/428: creat d4/d15/d1f/d3c/f96 x:0 0 0 2024-05-13T22:48:31.400 INFO:tasks.workunit.client.1.smithi130.stdout:3/398: symlink d8/dc/d7b/l88 0 2024-05-13T22:48:31.402 INFO:tasks.workunit.client.1.smithi130.stdout:3/399: stat d8/dc/d19/d27/f84 0 2024-05-13T22:48:31.402 INFO:tasks.workunit.client.1.smithi130.stdout:0/422: dread d0/d7/dd/d17/d1f/d2c/d45/d57/f89 [0,4194304] 0 2024-05-13T22:48:31.402 INFO:tasks.workunit.client.1.smithi130.stdout:7/376: symlink d7/d8/d9/l7f 0 2024-05-13T22:48:31.404 INFO:tasks.workunit.client.0.smithi102.stdout:1/994: dread d2/d10/d62/d7b/dc4/fcc [0,4194304] 0 2024-05-13T22:48:31.406 INFO:tasks.workunit.client.1.smithi130.stdout:6/429: unlink d4/d15/d1f/d46/d52/f76 0 2024-05-13T22:48:31.407 INFO:tasks.workunit.client.1.smithi130.stdout:5/374: rename d6/d7/d24/f68 to d6/d7/d19/d4d/d6a/f78 0 2024-05-13T22:48:31.410 INFO:tasks.workunit.client.1.smithi130.stdout:7/377: dwrite d7/da/d1d/d42/f64 [0,4194304] 0 2024-05-13T22:48:31.410 INFO:tasks.workunit.client.1.smithi130.stdout:8/353: dwrite d1/d2/db/d15/d18/f63 [0,4194304] 0 2024-05-13T22:48:31.410 INFO:tasks.workunit.client.1.smithi130.stdout:5/375: chown d6/d7/d24/d4b 28665390 1 2024-05-13T22:48:31.410 INFO:tasks.workunit.client.1.smithi130.stdout:3/400: fdatasync d8/dc/d42/f81 0 2024-05-13T22:48:31.418 INFO:tasks.workunit.client.1.smithi130.stdout:0/423: creat d0/d7/dd/d17/d1f/d2c/d45/d57/f92 x:0 0 0 2024-05-13T22:48:31.419 INFO:tasks.workunit.client.1.smithi130.stdout:9/440: link d1/db/cd d1/d7/d69/d79/c87 0 2024-05-13T22:48:31.423 INFO:tasks.workunit.client.1.smithi130.stdout:6/430: dwrite d4/d15/d1f/d46/d50/f51 [0,4194304] 0 2024-05-13T22:48:31.427 INFO:tasks.workunit.client.1.smithi130.stdout:5/376: write d6/d7/d19/d4d/d6a/f78 [1037850,101869] 0 2024-05-13T22:48:31.427 INFO:tasks.workunit.client.1.smithi130.stdout:0/424: dwrite d0/f8 [0,4194304] 0 2024-05-13T22:48:31.438 INFO:tasks.workunit.client.0.smithi102.stdout:1/995: truncate d2/d10/d62/d7b/fd9 3654309 0 2024-05-13T22:48:31.439 INFO:tasks.workunit.client.0.smithi102.stdout:1/996: fdatasync d2/d45/d79/faa 0 2024-05-13T22:48:31.439 INFO:tasks.workunit.client.1.smithi130.stdout:9/441: symlink d1/d15/d51/d85/l88 0 2024-05-13T22:48:31.442 INFO:tasks.workunit.client.1.smithi130.stdout:3/401: creat d8/d1a/f89 x:0 0 0 2024-05-13T22:48:31.442 INFO:tasks.workunit.client.1.smithi130.stdout:8/354: creat d1/d2/f80 x:0 0 0 2024-05-13T22:48:31.443 INFO:tasks.workunit.client.1.smithi130.stdout:8/355: fsync d1/dc/f26 0 2024-05-13T22:48:31.444 INFO:tasks.workunit.client.1.smithi130.stdout:8/356: readlink d1/d2/db/d15/d18/d1a/l6a 0 2024-05-13T22:48:31.447 INFO:tasks.workunit.client.0.smithi102.stdout:1/997: creat d2/d4/d38/db2/dba/d10b/d119/f138 x:0 0 0 2024-05-13T22:48:31.448 INFO:tasks.workunit.client.0.smithi102.stdout:1/998: dread - d2/d4/d26/d40/f124 zero size 2024-05-13T22:48:31.448 INFO:tasks.workunit.client.1.smithi130.stdout:9/442: symlink d1/d7/d23/d3d/d43/l89 0 2024-05-13T22:48:31.450 INFO:tasks.workunit.client.1.smithi130.stdout:8/357: rename d1/d2/db/d15/d18/d1a/c66 to d1/d2/db/d15/d7e/c81 0 2024-05-13T22:48:31.451 INFO:tasks.workunit.client.1.smithi130.stdout:8/358: fdatasync d1/d2/db/d15/d18/f5f 0 2024-05-13T22:48:31.452 INFO:tasks.workunit.client.0.smithi102.stdout:1/999: unlink d2/d10/f11f 0 2024-05-13T22:48:31.453 INFO:tasks.workunit.client.1.smithi130.stdout:9/443: dread d1/d7/d23/d28/d2d/f46 [0,4194304] 0 2024-05-13T22:48:31.455 INFO:tasks.workunit.client.1.smithi130.stdout:7/378: truncate d7/d8/f19 6728 0 2024-05-13T22:48:31.456 INFO:tasks.workunit.client.1.smithi130.stdout:7/379: rename d7 to d7/d8/d9/d66/d65/d80 22 2024-05-13T22:48:31.457 INFO:tasks.workunit.client.1.smithi130.stdout:7/380: fsync d7/da/d1d/d3d/d46/d67/f6c 0 2024-05-13T22:48:31.457 INFO:tasks.workunit.client.0.smithi102.stderr:+ rm -rf -- ./tmp.ZPN04XmeAi 2024-05-13T22:48:31.457 INFO:tasks.workunit.client.1.smithi130.stdout:7/381: dread - d7/d8/d39/f79 zero size 2024-05-13T22:48:31.458 INFO:tasks.workunit.client.1.smithi130.stdout:8/359: mknod d1/dc/d5d/c82 0 2024-05-13T22:48:31.459 INFO:tasks.workunit.client.1.smithi130.stdout:9/444: getdents d1/d7/d23/d28/d32/d65 0 2024-05-13T22:48:31.460 INFO:tasks.workunit.client.1.smithi130.stdout:9/445: fdatasync d1/d7/d23/d28/f39 0 2024-05-13T22:48:31.461 INFO:tasks.workunit.client.1.smithi130.stdout:8/360: symlink d1/d2/db/d15/d18/d42/d53/d56/l83 0 2024-05-13T22:48:31.462 INFO:tasks.workunit.client.1.smithi130.stdout:5/377: truncate d6/dd/d18/f15 78315 0 2024-05-13T22:48:31.465 INFO:tasks.workunit.client.1.smithi130.stdout:7/382: link d7/d8/d5b/f6d d7/d8/d9/d66/f81 0 2024-05-13T22:48:31.465 INFO:tasks.workunit.client.1.smithi130.stdout:7/383: stat d7/d8/d39/f79 0 2024-05-13T22:48:31.466 INFO:tasks.workunit.client.1.smithi130.stdout:9/446: chown d1/d7/d23/d28/l38 145419234 1 2024-05-13T22:48:31.468 INFO:tasks.workunit.client.1.smithi130.stdout:8/361: symlink d1/d2/db/l84 0 2024-05-13T22:48:31.471 INFO:tasks.workunit.client.1.smithi130.stdout:0/425: dread d0/d7/dd/d2e/d4b/f15 [0,4194304] 0 2024-05-13T22:48:31.471 INFO:tasks.workunit.client.1.smithi130.stdout:5/378: dread d6/dd/f3e [0,4194304] 0 2024-05-13T22:48:31.471 INFO:tasks.workunit.client.1.smithi130.stdout:7/384: unlink d7/da/d1d/d5d/f5e 0 2024-05-13T22:48:31.475 INFO:tasks.workunit.client.1.smithi130.stdout:9/447: dread d1/d7/d23/d28/d2d/f48 [0,4194304] 0 2024-05-13T22:48:31.476 INFO:tasks.workunit.client.1.smithi130.stdout:8/362: fsync d1/d2/db/d15/d18/d42/d53/d56/f71 0 2024-05-13T22:48:31.477 INFO:tasks.workunit.client.1.smithi130.stdout:8/363: chown d1/dc/f26 46 1 2024-05-13T22:48:31.477 INFO:tasks.workunit.client.1.smithi130.stdout:8/364: read - d1/d2/db/f4a zero size 2024-05-13T22:48:31.480 INFO:tasks.workunit.client.1.smithi130.stdout:5/379: mkdir d6/dd/d79 0 2024-05-13T22:48:31.483 INFO:tasks.workunit.client.1.smithi130.stdout:0/426: creat d0/d7/dd/d17/d52/d5b/d5f/f93 x:0 0 0 2024-05-13T22:48:31.489 INFO:tasks.workunit.client.1.smithi130.stdout:0/427: stat d0/d7/dd/c3e 0 2024-05-13T22:48:31.490 INFO:tasks.workunit.client.1.smithi130.stdout:7/385: symlink d7/d8/d7e/l82 0 2024-05-13T22:48:31.494 INFO:tasks.workunit.client.1.smithi130.stdout:5/380: creat d6/d1a/f7a x:0 0 0 2024-05-13T22:48:31.496 INFO:tasks.workunit.client.1.smithi130.stdout:7/386: read d7/d8/d39/f7b [2669700,85068] 0 2024-05-13T22:48:31.498 INFO:tasks.workunit.client.1.smithi130.stdout:0/428: symlink d0/d7/dd/d2e/l94 0 2024-05-13T22:48:31.500 INFO:tasks.workunit.client.1.smithi130.stdout:9/448: getdents d1/d7 0 2024-05-13T22:48:31.501 INFO:tasks.workunit.client.1.smithi130.stdout:8/365: getdents d1/d2/db/d15/d16/d54/d58 0 2024-05-13T22:48:31.502 INFO:tasks.workunit.client.1.smithi130.stdout:5/381: dread d6/dd/f13 [0,4194304] 0 2024-05-13T22:48:31.504 INFO:tasks.workunit.client.1.smithi130.stdout:9/449: dread d1/d7/d23/d28/d2d/f48 [0,4194304] 0 2024-05-13T22:48:31.504 INFO:tasks.workunit.client.1.smithi130.stdout:0/429: symlink d0/d7/dd/d17/d52/d67/l95 0 2024-05-13T22:48:31.506 INFO:tasks.workunit.client.1.smithi130.stdout:8/366: write d1/f10 [1909522,114977] 0 2024-05-13T22:48:31.507 INFO:tasks.workunit.client.1.smithi130.stdout:9/450: creat d1/d7/d69/d79/f8a x:0 0 0 2024-05-13T22:48:31.509 INFO:tasks.workunit.client.1.smithi130.stdout:0/430: unlink d0/f53 0 2024-05-13T22:48:31.510 INFO:tasks.workunit.client.1.smithi130.stdout:7/387: creat d7/f83 x:0 0 0 2024-05-13T22:48:31.511 INFO:tasks.workunit.client.1.smithi130.stdout:7/388: write d7/da/d1d/d42/f58 [4383144,44120] 0 2024-05-13T22:48:31.511 INFO:tasks.workunit.client.1.smithi130.stdout:9/451: creat d1/d7/d23/d3d/f8b x:0 0 0 2024-05-13T22:48:31.512 INFO:tasks.workunit.client.1.smithi130.stdout:5/382: rename f3 to d6/d7/d24/f7b 0 2024-05-13T22:48:31.513 INFO:tasks.workunit.client.1.smithi130.stdout:9/452: symlink d1/d7/d69/l8c 0 2024-05-13T22:48:31.517 INFO:tasks.workunit.client.1.smithi130.stdout:8/367: rename d1/dc/d5d/c82 to d1/d2/db/d15/d18/c85 0 2024-05-13T22:48:31.517 INFO:tasks.workunit.client.1.smithi130.stdout:7/389: dwrite d7/f6a [0,4194304] 0 2024-05-13T22:48:31.519 INFO:tasks.workunit.client.1.smithi130.stdout:5/383: fdatasync d6/d1a/f40 0 2024-05-13T22:48:31.523 INFO:tasks.workunit.client.1.smithi130.stdout:0/431: rename d0/d7/dd/d69/d88 to d0/d7/dd/d17/d52/d67/d68/d96 0 2024-05-13T22:48:31.525 INFO:tasks.workunit.client.1.smithi130.stdout:9/453: dwrite d1/d7/d23/d28/f67 [0,4194304] 0 2024-05-13T22:48:31.528 INFO:tasks.workunit.client.1.smithi130.stdout:8/368: dwrite d1/d2/db/f45 [0,4194304] 0 2024-05-13T22:48:31.534 INFO:tasks.workunit.client.1.smithi130.stdout:7/390: unlink d7/da/d1d/d3d/d46/d67/l76 0 2024-05-13T22:48:31.536 INFO:tasks.workunit.client.1.smithi130.stdout:8/369: rmdir d1/d2 39 2024-05-13T22:48:31.538 INFO:tasks.workunit.client.1.smithi130.stdout:7/391: dread d7/d8/d39/f7c [0,4194304] 0 2024-05-13T22:48:31.538 INFO:tasks.workunit.client.1.smithi130.stdout:9/454: mkdir d1/d7/d23/d28/d73/d8d 0 2024-05-13T22:48:31.539 INFO:tasks.workunit.client.1.smithi130.stdout:9/455: readlink d1/d7/l18 0 2024-05-13T22:48:31.540 INFO:tasks.workunit.client.1.smithi130.stdout:7/392: write d7/d8/f78 [892074,54388] 0 2024-05-13T22:48:31.540 INFO:tasks.workunit.client.1.smithi130.stdout:5/384: link d6/dd/d18/f36 d6/f7c 0 2024-05-13T22:48:31.543 INFO:tasks.workunit.client.1.smithi130.stdout:9/456: chown d1/d7/d23/d28/d2d/d2f/c4a 0 1 2024-05-13T22:48:31.543 INFO:tasks.workunit.client.1.smithi130.stdout:8/370: dwrite d1/f10 [0,4194304] 0 2024-05-13T22:48:31.543 INFO:tasks.workunit.client.1.smithi130.stdout:7/393: write d7/da/d1d/d3d/d46/f6e [641532,126950] 0 2024-05-13T22:48:31.548 INFO:tasks.workunit.client.1.smithi130.stdout:5/385: creat d6/d2f/f7d x:0 0 0 2024-05-13T22:48:31.549 INFO:tasks.workunit.client.1.smithi130.stdout:9/457: creat d1/d15/d51/f8e x:0 0 0 2024-05-13T22:48:31.554 INFO:tasks.workunit.client.1.smithi130.stdout:8/371: rmdir d1/d2/db/d15/d18/d42/d53 39 2024-05-13T22:48:31.555 INFO:tasks.workunit.client.1.smithi130.stdout:8/372: write d1/d2/f39 [6444190,11611] 0 2024-05-13T22:48:31.559 INFO:tasks.workunit.client.1.smithi130.stdout:7/394: symlink d7/da/l84 0 2024-05-13T22:48:31.560 INFO:tasks.workunit.client.1.smithi130.stdout:9/458: rmdir d1/d7/d23 39 2024-05-13T22:48:31.560 INFO:tasks.workunit.client.1.smithi130.stdout:7/395: truncate d7/da/d1d/f6b 833917 0 2024-05-13T22:48:31.561 INFO:tasks.workunit.client.1.smithi130.stdout:0/432: dread d0/d7/dd/d2e/d4b/f55 [0,4194304] 0 2024-05-13T22:48:31.563 INFO:tasks.workunit.client.1.smithi130.stdout:5/386: mknod d6/c7e 0 2024-05-13T22:48:31.567 INFO:tasks.workunit.client.1.smithi130.stdout:8/373: rename d1/l14 to d1/d2/db/d15/l86 0 2024-05-13T22:48:31.567 INFO:tasks.workunit.client.1.smithi130.stdout:0/433: dwrite d0/d7/dd/d17/d52/d67/d68/d96/f83 [0,4194304] 0 2024-05-13T22:48:31.568 INFO:tasks.workunit.client.1.smithi130.stdout:0/434: chown d0/d7/dd/d17/d52/d67 1179582591 1 2024-05-13T22:48:31.571 INFO:tasks.workunit.client.1.smithi130.stdout:7/396: creat d7/da/d1d/d5d/d70/f85 x:0 0 0 2024-05-13T22:48:31.572 INFO:tasks.workunit.client.1.smithi130.stdout:5/387: mkdir d6/dd/d18/d6e/d76/d7f 0 2024-05-13T22:48:31.573 INFO:tasks.workunit.client.1.smithi130.stdout:7/397: chown d7/da/d1d/d3d/d46/d67/d40 73176364 1 2024-05-13T22:48:31.574 INFO:tasks.workunit.client.1.smithi130.stdout:5/388: truncate d6/d2f/f59 247867 0 2024-05-13T22:48:31.575 INFO:tasks.workunit.client.1.smithi130.stdout:7/398: write d7/da/f33 [1627758,102635] 0 2024-05-13T22:48:31.575 INFO:tasks.workunit.client.1.smithi130.stdout:8/374: rename d1/dc/f4d to d1/dc/d5d/f87 0 2024-05-13T22:48:31.576 INFO:tasks.workunit.client.1.smithi130.stdout:5/389: chown d6/dd/f3e 82 1 2024-05-13T22:48:31.578 INFO:tasks.workunit.client.1.smithi130.stdout:7/399: dread - d7/d8/d5b/f6d zero size 2024-05-13T22:48:31.578 INFO:tasks.workunit.client.1.smithi130.stdout:9/459: link d1/d7/d23/d3d/d43/f82 d1/d7/d23/d28/d73/d8d/f8f 0 2024-05-13T22:48:31.578 INFO:tasks.workunit.client.1.smithi130.stdout:5/390: stat d6/d7/f22 0 2024-05-13T22:48:31.579 INFO:tasks.workunit.client.1.smithi130.stdout:9/460: write d1/d7/d23/d28/d32/f62 [1501895,118315] 0 2024-05-13T22:48:31.583 INFO:tasks.workunit.client.1.smithi130.stdout:7/400: fdatasync d7/d8/f4c 0 2024-05-13T22:48:31.584 INFO:tasks.workunit.client.1.smithi130.stdout:7/401: write d7/da/f33 [2494339,89633] 0 2024-05-13T22:48:31.585 INFO:tasks.workunit.client.1.smithi130.stdout:4/401: sync 2024-05-13T22:48:31.586 INFO:tasks.workunit.client.1.smithi130.stdout:4/402: chown d8/f1f 30 1 2024-05-13T22:48:31.586 INFO:tasks.workunit.client.1.smithi130.stdout:4/403: dread - d8/dc/d26/f75 zero size 2024-05-13T22:48:31.587 INFO:tasks.workunit.client.1.smithi130.stdout:5/391: dread d6/dd/d18/f15 [0,4194304] 0 2024-05-13T22:48:31.588 INFO:tasks.workunit.client.1.smithi130.stdout:5/392: dread d6/dd/f13 [0,4194304] 0 2024-05-13T22:48:31.590 INFO:tasks.workunit.client.1.smithi130.stdout:0/435: rename d0/d7/dd/d17/d52/d67/l81 to d0/d7/dd/d17/d52/d5b/d5f/l97 0 2024-05-13T22:48:31.597 INFO:tasks.workunit.client.1.smithi130.stdout:5/393: dread d6/dd/f33 [0,4194304] 0 2024-05-13T22:48:31.597 INFO:tasks.workunit.client.1.smithi130.stdout:5/394: chown d6/d1a/d72/c74 454305 1 2024-05-13T22:48:31.602 INFO:tasks.workunit.client.1.smithi130.stdout:4/404: rename d8/dc/d13/l45 to d8/dc/d13/l7e 0 2024-05-13T22:48:31.603 INFO:tasks.workunit.client.1.smithi130.stdout:1/415: sync 2024-05-13T22:48:31.604 INFO:tasks.workunit.client.1.smithi130.stdout:2/427: sync 2024-05-13T22:48:31.606 INFO:tasks.workunit.client.1.smithi130.stdout:1/416: read db/d1b/d1f/d2a/f41 [25387,91195] 0 2024-05-13T22:48:31.606 INFO:tasks.workunit.client.1.smithi130.stdout:0/436: mknod d0/d7/dd/d17/d1f/d2c/d45/d6b/c98 0 2024-05-13T22:48:31.608 INFO:tasks.workunit.client.1.smithi130.stdout:9/461: mknod d1/d7/d23/d28/d73/d8d/c90 0 2024-05-13T22:48:31.608 INFO:tasks.workunit.client.1.smithi130.stdout:4/405: dwrite d8/dc/d29/f3a [0,4194304] 0 2024-05-13T22:48:31.609 INFO:tasks.workunit.client.1.smithi130.stdout:7/402: mkdir d7/da/d86 0 2024-05-13T22:48:31.609 INFO:tasks.workunit.client.1.smithi130.stdout:4/406: chown d8/dc/d35/d33/d44 3 1 2024-05-13T22:48:31.613 INFO:tasks.workunit.client.1.smithi130.stdout:1/417: dwrite db/d4a/f4b [0,4194304] 0 2024-05-13T22:48:31.618 INFO:tasks.workunit.client.1.smithi130.stdout:0/437: chown d0/d7/dd/d2e/d77/d8d 1470876 1 2024-05-13T22:48:31.622 INFO:tasks.workunit.client.1.smithi130.stdout:8/375: truncate d1/f10 3202560 0 2024-05-13T22:48:31.629 INFO:tasks.workunit.client.1.smithi130.stdout:0/438: symlink d0/d7/dd/d2e/d77/l99 0 2024-05-13T22:48:31.643 INFO:tasks.workunit.client.1.smithi130.stdout:9/462: symlink d1/d7/d23/d28/d2d/l91 0 2024-05-13T22:48:31.643 INFO:tasks.workunit.client.1.smithi130.stdout:4/407: link d8/dc/d13/f7d d8/dc/d35/d33/d44/d56/f7f 0 2024-05-13T22:48:31.644 INFO:tasks.workunit.client.1.smithi130.stdout:8/376: creat d1/d2/db/d15/d16/f88 x:0 0 0 2024-05-13T22:48:31.644 INFO:tasks.workunit.client.1.smithi130.stdout:5/395: write d6/dd/f33 [225009,79977] 0 2024-05-13T22:48:31.644 INFO:tasks.workunit.client.1.smithi130.stdout:2/428: rename d4/db/dc/d18/f80 to d4/db/dc/d18/f98 0 2024-05-13T22:48:31.644 INFO:tasks.workunit.client.1.smithi130.stdout:0/439: rename d0/f4 to d0/d7/dd/d17/d52/f9a 0 2024-05-13T22:48:31.648 INFO:tasks.workunit.client.1.smithi130.stdout:5/396: fsync d6/dd/f3e 0 2024-05-13T22:48:31.648 INFO:tasks.workunit.client.1.smithi130.stdout:0/440: dread - d0/d7/dd/d17/d52/d5b/d5f/f93 zero size 2024-05-13T22:48:31.649 INFO:tasks.workunit.client.1.smithi130.stdout:4/408: dread d8/f12 [0,4194304] 0 2024-05-13T22:48:31.650 INFO:tasks.workunit.client.1.smithi130.stdout:8/377: mknod d1/d2/db/d15/c89 0 2024-05-13T22:48:31.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:31 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:48:31.651 INFO:tasks.workunit.client.1.smithi130.stdout:2/429: mkdir d4/db/dc/d18/d2a/d30/d88/d99 0 2024-05-13T22:48:31.651 INFO:tasks.workunit.client.1.smithi130.stdout:9/463: dwrite d1/f1e [4194304,4194304] 0 2024-05-13T22:48:31.655 INFO:tasks.workunit.client.1.smithi130.stdout:0/441: rmdir d0/d7/dd/d2e/d4b 39 2024-05-13T22:48:31.656 INFO:tasks.workunit.client.1.smithi130.stdout:0/442: chown d0/f21 230 1 2024-05-13T22:48:31.658 INFO:tasks.workunit.client.1.smithi130.stdout:0/443: truncate d0/d7/dd/d17/d52/d67/d68/d96/f83 4598617 0 2024-05-13T22:48:31.659 INFO:tasks.workunit.client.1.smithi130.stdout:2/430: rmdir d4/d49/d42/d4b 39 2024-05-13T22:48:31.661 INFO:tasks.workunit.client.1.smithi130.stdout:9/464: mkdir d1/d15/d51/d85/d92 0 2024-05-13T22:48:31.665 INFO:tasks.workunit.client.1.smithi130.stdout:0/444: mknod d0/d7/dd/d69/c9b 0 2024-05-13T22:48:31.667 INFO:tasks.workunit.client.1.smithi130.stdout:9/465: dwrite d1/d7/d23/d28/d32/f62 [0,4194304] 0 2024-05-13T22:48:31.667 INFO:tasks.workunit.client.1.smithi130.stdout:9/466: dread - d1/d15/d51/f7f zero size 2024-05-13T22:48:31.668 INFO:tasks.workunit.client.1.smithi130.stdout:4/409: rename d8/dc/d13 to d8/dc/d80 0 2024-05-13T22:48:31.671 INFO:tasks.workunit.client.1.smithi130.stdout:2/431: mkdir d4/db/dc/d18/d2a/d30/d88/d99/d9a 0 2024-05-13T22:48:31.673 INFO:tasks.workunit.client.1.smithi130.stdout:4/410: mkdir d8/df/d81 0 2024-05-13T22:48:31.675 INFO:tasks.workunit.client.1.smithi130.stdout:9/467: dwrite d1/d7/d23/f7a [0,4194304] 0 2024-05-13T22:48:31.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:31 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:48:31.677 INFO:tasks.workunit.client.1.smithi130.stdout:2/432: creat d4/d49/d48/d4a/d76/f9b x:0 0 0 2024-05-13T22:48:31.679 INFO:tasks.workunit.client.1.smithi130.stdout:0/445: creat d0/f9c x:0 0 0 2024-05-13T22:48:31.681 INFO:tasks.workunit.client.1.smithi130.stdout:9/468: chown d1/d7/d23/d28/d32/c59 19412899 1 2024-05-13T22:48:31.682 INFO:tasks.workunit.client.1.smithi130.stdout:9/469: chown d1/d7/d23/d28/d32/c59 0 1 2024-05-13T22:48:31.684 INFO:tasks.workunit.client.1.smithi130.stdout:5/397: dread d6/f1f [0,4194304] 0 2024-05-13T22:48:31.687 INFO:tasks.workunit.client.1.smithi130.stdout:9/470: creat d1/d7/d23/d28/d2d/d2f/f93 x:0 0 0 2024-05-13T22:48:31.688 INFO:tasks.workunit.client.1.smithi130.stdout:0/446: creat d0/d7/dd/d2e/d77/d8d/f9d x:0 0 0 2024-05-13T22:48:31.689 INFO:tasks.workunit.client.1.smithi130.stdout:0/447: truncate d0/d7/f16 464654 0 2024-05-13T22:48:31.692 INFO:tasks.workunit.client.1.smithi130.stdout:9/471: rmdir d1/d7/d23/d28/d2d/d2f/d63/d6b 39 2024-05-13T22:48:31.694 INFO:tasks.workunit.client.1.smithi130.stdout:8/378: write d1/f10 [737357,75701] 0 2024-05-13T22:48:31.694 INFO:tasks.workunit.client.1.smithi130.stdout:8/379: dread - d1/dc/f26 zero size 2024-05-13T22:48:31.694 INFO:tasks.workunit.client.1.smithi130.stdout:2/433: getdents d4/d49/d42 0 2024-05-13T22:48:31.695 INFO:tasks.workunit.client.1.smithi130.stdout:3/402: sync 2024-05-13T22:48:31.695 INFO:tasks.workunit.client.1.smithi130.stdout:6/431: sync 2024-05-13T22:48:31.696 INFO:tasks.workunit.client.1.smithi130.stdout:2/434: chown d4/db/dc/d18/d2a/d30/d69/f6e 29 1 2024-05-13T22:48:31.697 INFO:tasks.workunit.client.1.smithi130.stdout:6/432: write d4/f12 [8850551,82845] 0 2024-05-13T22:48:31.697 INFO:tasks.workunit.client.1.smithi130.stdout:2/435: chown d4/d49/d48/d4a 1120565 1 2024-05-13T22:48:31.699 INFO:tasks.workunit.client.1.smithi130.stdout:9/472: write d1/d7/d23/d28/d2d/d2f/d63/d6b/f84 [960459,119776] 0 2024-05-13T22:48:31.699 INFO:tasks.workunit.client.1.smithi130.stdout:2/436: fdatasync d4/db/dc/d18/d2a/d66/f7f 0 2024-05-13T22:48:31.701 INFO:tasks.workunit.client.1.smithi130.stdout:9/473: write d1/d7/d23/d28/d2d/f34 [2180855,122477] 0 2024-05-13T22:48:31.702 INFO:tasks.workunit.client.1.smithi130.stdout:3/403: mknod d8/dc/d42/c8a 0 2024-05-13T22:48:31.703 INFO:tasks.workunit.client.1.smithi130.stdout:3/404: fsync d8/d1a/f47 0 2024-05-13T22:48:31.706 INFO:tasks.workunit.client.1.smithi130.stdout:6/433: symlink d4/d15/d1f/d46/d52/d7b/l97 0 2024-05-13T22:48:31.706 INFO:tasks.workunit.client.1.smithi130.stdout:4/411: write d8/dc/d26/f36 [1521465,69400] 0 2024-05-13T22:48:31.707 INFO:tasks.workunit.client.1.smithi130.stdout:0/448: getdents d0/d7/dd/d17/d52/d5b 0 2024-05-13T22:48:31.709 INFO:tasks.workunit.client.1.smithi130.stdout:5/398: write d6/dd/d18/f36 [1206953,69280] 0 2024-05-13T22:48:31.709 INFO:tasks.workunit.client.1.smithi130.stdout:4/412: truncate d8/dc/d35/d33/d3f/d57/f5f 998045 0 2024-05-13T22:48:31.710 INFO:tasks.workunit.client.1.smithi130.stdout:2/437: creat d4/db/dc/d18/d2a/d30/f9c x:0 0 0 2024-05-13T22:48:31.710 INFO:tasks.workunit.client.1.smithi130.stdout:5/399: write d6/d1a/f48 [19901,18587] 0 2024-05-13T22:48:31.712 INFO:tasks.workunit.client.1.smithi130.stdout:3/405: symlink d8/d1a/d55/l8b 0 2024-05-13T22:48:31.716 INFO:tasks.workunit.client.1.smithi130.stdout:8/380: rename d1/d2/db/d15/d18/d1a/c72 to d1/d2/db/d15/d18/c8a 0 2024-05-13T22:48:31.719 INFO:tasks.workunit.client.1.smithi130.stdout:6/434: mknod d4/d15/d1f/c98 0 2024-05-13T22:48:31.719 INFO:tasks.workunit.client.1.smithi130.stdout:0/449: mkdir d0/d7/dd/d2e/d9e 0 2024-05-13T22:48:31.721 INFO:tasks.workunit.client.1.smithi130.stdout:9/474: rename d1/db/l26 to d1/d15/d51/l94 0 2024-05-13T22:48:31.723 INFO:tasks.workunit.client.1.smithi130.stdout:8/381: dwrite d1/d2/db/d15/d18/f3e [0,4194304] 0 2024-05-13T22:48:31.723 INFO:tasks.workunit.client.1.smithi130.stdout:0/450: truncate d0/d7/dd/d17/d52/f80 824117 0 2024-05-13T22:48:31.725 INFO:tasks.workunit.client.1.smithi130.stdout:8/382: chown d1/d2/db/c6e 2540184 1 2024-05-13T22:48:31.726 INFO:tasks.workunit.client.1.smithi130.stdout:8/383: readlink d1/d2/db/l84 0 2024-05-13T22:48:31.729 INFO:tasks.workunit.client.1.smithi130.stdout:4/413: rename d8/dc/d80/f47 to d8/dc/d35/d33/d3f/f82 0 2024-05-13T22:48:31.729 INFO:tasks.workunit.client.1.smithi130.stdout:9/475: dwrite d1/d7/d23/d28/d32/f62 [8388608,4194304] 0 2024-05-13T22:48:31.734 INFO:tasks.workunit.client.1.smithi130.stdout:3/406: write d8/dc/d15/f66 [641996,125174] 0 2024-05-13T22:48:31.736 INFO:tasks.workunit.client.1.smithi130.stdout:0/451: mkdir d0/d7/dd/d2e/d77/d8d/d9f 0 2024-05-13T22:48:31.738 INFO:tasks.workunit.client.1.smithi130.stdout:5/400: rename d6/d7/d24/f65 to d6/dd/d18/d6e/d76/f80 0 2024-05-13T22:48:31.743 INFO:tasks.workunit.client.1.smithi130.stdout:8/384: rename d1/d2/db/d15/d18/c73 to d1/d2/db/d15/c8b 0 2024-05-13T22:48:31.745 INFO:tasks.workunit.client.1.smithi130.stdout:4/414: unlink d8/dc/d35/d33/d44/d61/l67 0 2024-05-13T22:48:31.747 INFO:tasks.workunit.client.1.smithi130.stdout:9/476: mknod d1/d7/d23/d28/d32/d65/c95 0 2024-05-13T22:48:31.752 INFO:tasks.workunit.client.1.smithi130.stdout:3/407: rename d8/dc/f73 to d8/dc/d42/f8c 0 2024-05-13T22:48:31.752 INFO:tasks.workunit.client.1.smithi130.stdout:0/452: write d0/d7/dd/d17/d1f/d2c/d45/d57/f89 [273999,91045] 0 2024-05-13T22:48:31.753 INFO:tasks.workunit.client.1.smithi130.stdout:6/435: dread d4/d15/d1d/d28/f5e [0,4194304] 0 2024-05-13T22:48:31.754 INFO:tasks.workunit.client.1.smithi130.stdout:4/415: mknod d8/df/d81/c83 0 2024-05-13T22:48:31.756 INFO:tasks.workunit.client.1.smithi130.stdout:9/477: mkdir d1/d96 0 2024-05-13T22:48:31.757 INFO:tasks.workunit.client.1.smithi130.stdout:5/401: link f4 d6/d7/d24/d71/f81 0 2024-05-13T22:48:31.758 INFO:tasks.workunit.client.1.smithi130.stdout:6/436: rmdir d4/d15/d1f 39 2024-05-13T22:48:31.759 INFO:tasks.workunit.client.1.smithi130.stdout:3/408: unlink d8/d1a/c65 0 2024-05-13T22:48:31.760 INFO:tasks.workunit.client.1.smithi130.stdout:0/453: mkdir d0/da0 0 2024-05-13T22:48:31.765 INFO:tasks.workunit.client.1.smithi130.stdout:4/416: dread d8/dc/d26/f30 [0,4194304] 0 2024-05-13T22:48:31.768 INFO:tasks.workunit.client.1.smithi130.stdout:5/402: rename d6/d7/d24 to d6/dd/d18/d6e/d82 0 2024-05-13T22:48:31.773 INFO:tasks.workunit.client.1.smithi130.stdout:0/454: unlink d0/d7/dd/d17/d1f/d2c/f30 0 2024-05-13T22:48:31.777 INFO:tasks.workunit.client.1.smithi130.stdout:9/478: truncate d1/f3 694095 0 2024-05-13T22:48:31.783 INFO:tasks.workunit.client.1.smithi130.stdout:4/417: mkdir d8/dc/d84 0 2024-05-13T22:48:31.790 INFO:tasks.workunit.client.1.smithi130.stdout:6/437: unlink d4/d15/d1f/d46/d52/c5d 0 2024-05-13T22:48:31.805 INFO:tasks.workunit.client.1.smithi130.stdout:3/409: creat d8/dc/d19/f8d x:0 0 0 2024-05-13T22:48:31.805 INFO:tasks.workunit.client.1.smithi130.stdout:3/410: stat d8/d1a/d55/l5a 0 2024-05-13T22:48:31.805 INFO:tasks.workunit.client.1.smithi130.stdout:7/403: sync 2024-05-13T22:48:31.808 INFO:tasks.workunit.client.1.smithi130.stdout:5/403: rename d6/dd/d18/d6e/d82/d56/c6b to d6/dd/d18/d6e/d82/d4b/c83 0 2024-05-13T22:48:31.810 INFO:tasks.workunit.client.1.smithi130.stdout:6/438: truncate d4/d15/d1d/f6f 687591 0 2024-05-13T22:48:31.811 INFO:tasks.workunit.client.1.smithi130.stdout:0/455: truncate d0/d7/d29/f37 790152 0 2024-05-13T22:48:31.815 INFO:tasks.workunit.client.1.smithi130.stdout:0/456: dwrite d0/f8 [0,4194304] 0 2024-05-13T22:48:31.816 INFO:tasks.workunit.client.1.smithi130.stdout:0/457: readlink d0/d7/dd/d17/d1f/d2c/d45/l4a 0 2024-05-13T22:48:31.822 INFO:tasks.workunit.client.1.smithi130.stdout:0/458: dwrite d0/d7/dd/d17/d1f/d2c/f7a [0,4194304] 0 2024-05-13T22:48:31.824 INFO:tasks.workunit.client.1.smithi130.stdout:9/479: truncate d1/d7/d23/d3d/d43/f75 335392 0 2024-05-13T22:48:31.825 INFO:tasks.workunit.client.1.smithi130.stdout:9/480: read - d1/d15/d51/f7f zero size 2024-05-13T22:48:31.827 INFO:tasks.workunit.client.1.smithi130.stdout:6/439: creat d4/d15/d1d/d28/d4e/f99 x:0 0 0 2024-05-13T22:48:31.833 INFO:tasks.workunit.client.1.smithi130.stdout:4/418: mkdir d8/dc/d80/d85 0 2024-05-13T22:48:31.834 INFO:tasks.workunit.client.1.smithi130.stdout:4/419: write d8/df/f3c [2614985,28179] 0 2024-05-13T22:48:31.835 INFO:tasks.workunit.client.1.smithi130.stdout:4/420: chown d8/df/f20 10987632 1 2024-05-13T22:48:31.836 INFO:tasks.workunit.client.1.smithi130.stdout:5/404: mkdir d6/dd/d18/d6e/d82/d84 0 2024-05-13T22:48:31.837 INFO:tasks.workunit.client.1.smithi130.stdout:5/405: readlink d6/d7/d19/l63 0 2024-05-13T22:48:31.837 INFO:tasks.workunit.client.1.smithi130.stdout:5/406: stat d6/d7 0 2024-05-13T22:48:31.838 INFO:tasks.workunit.client.1.smithi130.stdout:9/481: rename d1/d7/d23/d28/d2d/f3a to d1/d7/d23/d28/f97 0 2024-05-13T22:48:31.839 INFO:tasks.workunit.client.1.smithi130.stdout:9/482: write d1/d7/d23/d3d/f5c [813965,117203] 0 2024-05-13T22:48:31.840 INFO:tasks.workunit.client.1.smithi130.stdout:6/440: fdatasync d4/d15/d1d/d28/f71 0 2024-05-13T22:48:31.843 INFO:tasks.workunit.client.1.smithi130.stdout:4/421: stat d8/dc/d35 0 2024-05-13T22:48:31.849 INFO:tasks.workunit.client.1.smithi130.stdout:9/483: rmdir d1/d7/d23/d28/d32 39 2024-05-13T22:48:31.859 INFO:tasks.workunit.client.1.smithi130.stdout:4/422: dwrite d8/dc/d29/f3a [4194304,4194304] 0 2024-05-13T22:48:31.861 INFO:tasks.workunit.client.1.smithi130.stdout:6/441: mknod d4/c9a 0 2024-05-13T22:48:31.862 INFO:tasks.workunit.client.1.smithi130.stdout:3/411: link d8/d35/l1f d8/dc/d19/d2c/d61/l8e 0 2024-05-13T22:48:31.862 INFO:tasks.workunit.client.1.smithi130.stdout:6/442: write d4/f12 [8658242,90941] 0 2024-05-13T22:48:31.863 INFO:tasks.workunit.client.1.smithi130.stdout:6/443: stat d4/d15/d1d/d60/f66 0 2024-05-13T22:48:31.866 INFO:tasks.workunit.client.1.smithi130.stdout:6/444: read d4/d16/f39 [1062583,80847] 0 2024-05-13T22:48:31.873 INFO:tasks.workunit.client.1.smithi130.stdout:5/407: dwrite d6/d7/d19/d4d/f34 [0,4194304] 0 2024-05-13T22:48:31.874 INFO:tasks.workunit.client.1.smithi130.stdout:3/412: mkdir d8/dc/d15/d8f 0 2024-05-13T22:48:31.875 INFO:tasks.workunit.client.1.smithi130.stdout:3/413: write f2 [73204,5816] 0 2024-05-13T22:48:31.876 INFO:tasks.workunit.client.1.smithi130.stdout:7/404: getdents d7/da/d1d/d42/d43 0 2024-05-13T22:48:31.879 INFO:tasks.workunit.client.1.smithi130.stdout:9/484: symlink d1/d7/d23/d28/d32/l98 0 2024-05-13T22:48:31.885 INFO:tasks.workunit.client.1.smithi130.stdout:3/414: creat d8/dc/d7b/f90 x:0 0 0 2024-05-13T22:48:31.885 INFO:tasks.workunit.client.1.smithi130.stdout:6/445: mknod d4/d15/c9b 0 2024-05-13T22:48:31.887 INFO:tasks.workunit.client.1.smithi130.stdout:4/423: creat d8/dc/f86 x:0 0 0 2024-05-13T22:48:31.890 INFO:tasks.workunit.client.1.smithi130.stdout:6/446: mknod d4/d15/d1d/d28/d4e/c9c 0 2024-05-13T22:48:31.890 INFO:tasks.workunit.client.1.smithi130.stdout:9/485: symlink d1/d15/l99 0 2024-05-13T22:48:31.893 INFO:tasks.workunit.client.1.smithi130.stdout:7/405: rename d7/da/d1d/d3d/d46/c69 to d7/c87 0 2024-05-13T22:48:31.894 INFO:tasks.workunit.client.1.smithi130.stdout:3/415: dwrite d8/dc/f22 [0,4194304] 0 2024-05-13T22:48:31.895 INFO:tasks.workunit.client.1.smithi130.stdout:1/418: sync 2024-05-13T22:48:31.897 INFO:tasks.workunit.client.1.smithi130.stdout:9/486: truncate d1/d7/d23/d28/d2d/d2f/d63/d6b/f84 1931146 0 2024-05-13T22:48:31.899 INFO:tasks.workunit.client.1.smithi130.stdout:7/406: mkdir d7/da/d1d/d3d/d46/d67/d88 0 2024-05-13T22:48:31.899 INFO:tasks.workunit.client.1.smithi130.stdout:6/447: dwrite d4/d15/d1d/d28/f34 [0,4194304] 0 2024-05-13T22:48:31.903 INFO:tasks.workunit.client.1.smithi130.stdout:6/448: chown d4/d15/d1f/d46/d50/f59 50048468 1 2024-05-13T22:48:31.905 INFO:tasks.workunit.client.1.smithi130.stdout:4/424: dwrite d8/dc/d26/f30 [0,4194304] 0 2024-05-13T22:48:31.906 INFO:tasks.workunit.client.1.smithi130.stdout:7/407: creat d7/da/d1d/d5d/f89 x:0 0 0 2024-05-13T22:48:31.906 INFO:tasks.workunit.client.1.smithi130.stdout:1/419: dwrite db/d1b/d1f/d2a/f41 [0,4194304] 0 2024-05-13T22:48:31.908 INFO:tasks.workunit.client.1.smithi130.stdout:1/420: write db/d1b/d1f/d2a/f4f [1566991,37925] 0 2024-05-13T22:48:31.909 INFO:tasks.workunit.client.1.smithi130.stdout:5/408: dread d6/dd/f37 [0,4194304] 0 2024-05-13T22:48:31.915 INFO:tasks.workunit.client.1.smithi130.stdout:7/408: rmdir d7/da 39 2024-05-13T22:48:31.921 INFO:tasks.workunit.client.1.smithi130.stdout:4/425: dread d8/dc/f28 [0,4194304] 0 2024-05-13T22:48:31.922 INFO:tasks.workunit.client.1.smithi130.stdout:4/426: chown d8/dc/d26/f75 22293636 1 2024-05-13T22:48:31.923 INFO:tasks.workunit.client.1.smithi130.stdout:4/427: readlink d8/dc/d35/d33/d3f/l69 0 2024-05-13T22:48:31.928 INFO:tasks.workunit.client.1.smithi130.stdout:1/421: creat db/d1b/d1f/d2a/d38/d46/d6d/d87/f8a x:0 0 0 2024-05-13T22:48:31.930 INFO:tasks.workunit.client.1.smithi130.stdout:5/409: creat d6/dd/d18/d6e/d76/d7f/f85 x:0 0 0 2024-05-13T22:48:31.931 INFO:tasks.workunit.client.1.smithi130.stdout:5/410: chown d6/dd/d18/fc 2835562 1 2024-05-13T22:48:31.931 INFO:tasks.workunit.client.1.smithi130.stdout:3/416: truncate d8/dc/d42/f4a 217701 0 2024-05-13T22:48:31.932 INFO:tasks.workunit.client.1.smithi130.stdout:4/428: mknod d8/dc/d35/d33/d44/d77/c87 0 2024-05-13T22:48:31.938 INFO:tasks.workunit.client.1.smithi130.stdout:7/409: creat d7/da/d86/f8a x:0 0 0 2024-05-13T22:48:31.938 INFO:tasks.workunit.client.1.smithi130.stdout:3/417: dwrite d8/dc/d19/d27/f84 [0,4194304] 0 2024-05-13T22:48:31.939 INFO:tasks.workunit.client.1.smithi130.stdout:6/449: dwrite d4/f29 [0,4194304] 0 2024-05-13T22:48:31.940 INFO:tasks.workunit.client.1.smithi130.stdout:4/429: symlink d8/dc/d84/l88 0 2024-05-13T22:48:31.940 INFO:tasks.workunit.client.1.smithi130.stdout:3/418: chown d8/f5e 65038755 1 2024-05-13T22:48:31.942 INFO:tasks.workunit.client.1.smithi130.stdout:4/430: write d8/dc/d26/f36 [38312,126842] 0 2024-05-13T22:48:31.942 INFO:tasks.workunit.client.1.smithi130.stdout:7/410: fsync d7/d8/d9/d66/f31 0 2024-05-13T22:48:31.943 INFO:tasks.workunit.client.1.smithi130.stdout:4/431: write d8/dc/d26/f36 [2705246,53962] 0 2024-05-13T22:48:31.950 INFO:tasks.workunit.client.1.smithi130.stdout:7/411: dwrite d7/da/d1d/d5d/d70/f85 [0,4194304] 0 2024-05-13T22:48:31.957 INFO:tasks.workunit.client.1.smithi130.stdout:6/450: truncate d4/d15/d1f/f2d 1110146 0 2024-05-13T22:48:31.961 INFO:tasks.workunit.client.1.smithi130.stdout:2/438: sync 2024-05-13T22:48:31.961 INFO:tasks.workunit.client.1.smithi130.stdout:0/459: sync 2024-05-13T22:48:31.962 INFO:tasks.workunit.client.1.smithi130.stdout:8/385: sync 2024-05-13T22:48:31.964 INFO:tasks.workunit.client.1.smithi130.stdout:3/419: getdents d8/d1a/d1c 0 2024-05-13T22:48:31.967 INFO:tasks.workunit.client.1.smithi130.stdout:2/439: stat d4/db/dc/d18/d2a/d30/f47 0 2024-05-13T22:48:31.968 INFO:tasks.workunit.client.1.smithi130.stdout:5/411: dread d6/d2f/f6c [0,4194304] 0 2024-05-13T22:48:31.969 INFO:tasks.workunit.client.1.smithi130.stdout:3/420: dread - d8/dc/d15/f76 zero size 2024-05-13T22:48:31.969 INFO:tasks.workunit.client.1.smithi130.stdout:7/412: dwrite d7/da/f5c [0,4194304] 0 2024-05-13T22:48:31.973 INFO:tasks.workunit.client.1.smithi130.stdout:1/422: dwrite db/d1b/d1f/d49/f54 [0,4194304] 0 2024-05-13T22:48:31.973 INFO:tasks.workunit.client.1.smithi130.stdout:4/432: dwrite d8/dc/d80/f78 [4194304,4194304] 0 2024-05-13T22:48:31.976 INFO:tasks.workunit.client.1.smithi130.stdout:2/440: chown d4/db/dc/d18/d2a/f97 14 1 2024-05-13T22:48:31.980 INFO:tasks.workunit.client.1.smithi130.stdout:9/487: sync 2024-05-13T22:48:31.983 INFO:tasks.workunit.client.1.smithi130.stdout:7/413: mkdir d7/d8/d7e/d8b 0 2024-05-13T22:48:31.985 INFO:tasks.workunit.client.1.smithi130.stdout:6/451: dwrite d4/d15/d1f/d46/d50/f59 [0,4194304] 0 2024-05-13T22:48:31.989 INFO:tasks.workunit.client.1.smithi130.stdout:7/414: write d7/da/d1d/d5d/d70/f85 [1723849,129566] 0 2024-05-13T22:48:31.990 INFO:tasks.workunit.client.1.smithi130.stdout:8/386: link d1/d2/db/d15/d16/d54/d58/f7a d1/d2/f8c 0 2024-05-13T22:48:31.991 INFO:tasks.workunit.client.1.smithi130.stdout:0/460: dwrite d0/d7/dd/d2e/d77/d8d/f9d [0,4194304] 0 2024-05-13T22:48:31.993 INFO:tasks.workunit.client.1.smithi130.stdout:5/412: dwrite d6/d7/f28 [0,4194304] 0 2024-05-13T22:48:31.993 INFO:tasks.workunit.client.1.smithi130.stdout:7/415: write d7/f6a [501625,52515] 0 2024-05-13T22:48:31.994 INFO:tasks.workunit.client.1.smithi130.stdout:8/387: chown d1/d2/db/d15/d16/f2c 992200 1 2024-05-13T22:48:31.994 INFO:tasks.workunit.client.1.smithi130.stdout:7/416: read - d7/d8/d9/d66/f7a zero size 2024-05-13T22:48:31.996 INFO:tasks.workunit.client.1.smithi130.stdout:3/421: rmdir d8/dc/d85 0 2024-05-13T22:48:31.998 INFO:tasks.workunit.client.1.smithi130.stdout:6/452: creat d4/d15/d1f/d46/d52/d7b/f9d x:0 0 0 2024-05-13T22:48:31.999 INFO:tasks.workunit.client.1.smithi130.stdout:6/453: stat d4/d15/d1f/d46/f89 0 2024-05-13T22:48:32.000 INFO:tasks.workunit.client.1.smithi130.stdout:6/454: write d4/f49 [761720,66853] 0 2024-05-13T22:48:32.001 INFO:tasks.workunit.client.1.smithi130.stdout:8/388: dread d1/d2/db/d15/d16/d54/f55 [0,4194304] 0 2024-05-13T22:48:32.011 INFO:tasks.workunit.client.1.smithi130.stdout:7/417: creat d7/da/d1d/d5d/d70/f8c x:0 0 0 2024-05-13T22:48:32.015 INFO:tasks.workunit.client.1.smithi130.stdout:0/461: rename d0/d7/c38 to d0/d7/dd/d2e/d77/d8d/d9f/ca1 0 2024-05-13T22:48:32.017 INFO:tasks.workunit.client.1.smithi130.stdout:0/462: stat d0/d7/dd/d17/d1f/d2c/d5a/c6d 0 2024-05-13T22:48:32.017 INFO:tasks.workunit.client.1.smithi130.stdout:7/418: dwrite d7/da/d1d/f6b [0,4194304] 0 2024-05-13T22:48:32.020 INFO:tasks.workunit.client.1.smithi130.stdout:4/433: write d8/dc/d35/d33/f4f [295458,34692] 0 2024-05-13T22:48:32.024 INFO:tasks.workunit.client.1.smithi130.stdout:0/463: creat d0/d7/dd/d17/d52/d67/d68/d96/fa2 x:0 0 0 2024-05-13T22:48:32.024 INFO:tasks.workunit.client.1.smithi130.stdout:2/441: write d4/db/dc/d18/d2a/d30/f47 [838250,74969] 0 2024-05-13T22:48:32.027 INFO:tasks.workunit.client.1.smithi130.stdout:3/422: rename d8/dc/d42/f4a to d8/d1a/d1c/f91 0 2024-05-13T22:48:32.033 INFO:tasks.workunit.client.1.smithi130.stdout:1/423: truncate db/d1b/d1f/d2a/f4f 933009 0 2024-05-13T22:48:32.094 INFO:tasks.workunit.client.1.smithi130.stdout:6/455: rename d4/d15/d1d/d28/d72 to d4/d15/d1d/d28/d4e/d9e 0 2024-05-13T22:48:32.096 INFO:tasks.workunit.client.1.smithi130.stdout:9/488: dwrite d1/d7/d23/d3d/f40 [0,4194304] 0 2024-05-13T22:48:32.096 INFO:tasks.workunit.client.1.smithi130.stdout:7/419: truncate d7/da/d1d/d3d/d46/f6e 191913 0 2024-05-13T22:48:32.097 INFO:tasks.workunit.client.1.smithi130.stdout:6/456: write d4/d15/d1f/d46/d50/f79 [1689024,81855] 0 2024-05-13T22:48:32.098 INFO:tasks.workunit.client.1.smithi130.stdout:1/424: rename db/c25 to db/d1b/d1f/d49/c8b 0 2024-05-13T22:48:32.099 INFO:tasks.workunit.client.1.smithi130.stdout:8/389: link d1/d2/db/d15/d18/d42/d53/l70 d1/d2/db/d15/d18/d42/d53/d56/l8d 0 2024-05-13T22:48:32.100 INFO:tasks.workunit.client.1.smithi130.stdout:2/442: creat d4/d49/d48/f9d x:0 0 0 2024-05-13T22:48:32.102 INFO:tasks.workunit.client.1.smithi130.stdout:3/423: creat d8/f92 x:0 0 0 2024-05-13T22:48:32.104 INFO:tasks.workunit.client.1.smithi130.stdout:9/489: dwrite d1/d15/d51/f54 [0,4194304] 0 2024-05-13T22:48:32.107 INFO:tasks.workunit.client.1.smithi130.stdout:4/434: link d8/dc/c27 d8/dc/d35/d33/d3f/c89 0 2024-05-13T22:48:32.110 INFO:tasks.workunit.client.1.smithi130.stdout:6/457: creat d4/d16/f9f x:0 0 0 2024-05-13T22:48:32.111 INFO:tasks.workunit.client.1.smithi130.stdout:6/458: write d4/d15/d1f/d46/d52/f87 [4215025,8013] 0 2024-05-13T22:48:32.112 INFO:tasks.workunit.client.1.smithi130.stdout:4/435: dread d8/dc/d35/d33/f4f [0,4194304] 0 2024-05-13T22:48:32.113 INFO:tasks.workunit.client.1.smithi130.stdout:1/425: symlink db/d58/l8c 0 2024-05-13T22:48:32.113 INFO:tasks.workunit.client.1.smithi130.stdout:4/436: fsync d8/dc/d80/f78 0 2024-05-13T22:48:32.115 INFO:tasks.workunit.client.1.smithi130.stdout:4/437: write d8/df/f3c [2774937,69989] 0 2024-05-13T22:48:32.117 INFO:tasks.workunit.client.1.smithi130.stdout:8/390: mknod d1/d2/db/d15/d18/d42/d53/d56/c8e 0 2024-05-13T22:48:32.124 INFO:tasks.workunit.client.1.smithi130.stdout:7/420: link d7/d8/f78 d7/da/d1d/d5d/d70/f8d 0 2024-05-13T22:48:32.134 INFO:tasks.workunit.client.1.smithi130.stdout:6/459: dwrite d4/d15/d1f/f55 [0,4194304] 0 2024-05-13T22:48:32.137 INFO:tasks.workunit.client.1.smithi130.stdout:1/426: fdatasync db/d1b/d1f/d2a/d38/f5c 0 2024-05-13T22:48:32.138 INFO:tasks.workunit.client.1.smithi130.stdout:4/438: dread d8/dc/d80/f7d [0,4194304] 0 2024-05-13T22:48:32.141 INFO:tasks.workunit.client.1.smithi130.stdout:8/391: dread d1/d2/db/d15/d16/d54/f55 [0,4194304] 0 2024-05-13T22:48:32.142 INFO:tasks.workunit.client.1.smithi130.stdout:3/424: symlink d8/dc/l93 0 2024-05-13T22:48:32.143 INFO:tasks.workunit.client.1.smithi130.stdout:8/392: write d1/d2/db/d15/d18/d42/d53/d56/f7c [3488412,26083] 0 2024-05-13T22:48:32.143 INFO:tasks.workunit.client.1.smithi130.stdout:3/425: chown d8/dc/l17 0 1 2024-05-13T22:48:32.145 INFO:tasks.workunit.client.1.smithi130.stdout:9/490: creat d1/d7/d69/d79/f9a x:0 0 0 2024-05-13T22:48:32.145 INFO:tasks.workunit.client.1.smithi130.stdout:5/413: sync 2024-05-13T22:48:32.152 INFO:tasks.workunit.client.1.smithi130.stdout:2/443: truncate d4/d56/f6d 2081653 0 2024-05-13T22:48:32.154 INFO:tasks.workunit.client.1.smithi130.stdout:2/444: truncate d4/d56/f68 200958 0 2024-05-13T22:48:32.154 INFO:tasks.workunit.client.1.smithi130.stdout:6/460: mkdir d4/d15/d8b/da0 0 2024-05-13T22:48:32.159 INFO:tasks.workunit.client.1.smithi130.stdout:4/439: creat d8/dc/d35/f8a x:0 0 0 2024-05-13T22:48:32.164 INFO:tasks.workunit.client.1.smithi130.stdout:3/426: unlink d8/f70 0 2024-05-13T22:48:32.164 INFO:tasks.workunit.client.1.smithi130.stdout:3/427: chown d8/dc/d19 258 1 2024-05-13T22:48:32.169 INFO:tasks.workunit.client.1.smithi130.stdout:8/393: creat d1/d2/db/d15/d16/f8f x:0 0 0 2024-05-13T22:48:32.169 INFO:tasks.workunit.client.1.smithi130.stdout:1/427: write f8 [587044,11823] 0 2024-05-13T22:48:32.170 INFO:tasks.workunit.client.1.smithi130.stdout:1/428: readlink db/d1b/d1f/d2a/l3e 0 2024-05-13T22:48:32.172 INFO:tasks.workunit.client.1.smithi130.stdout:0/464: sync 2024-05-13T22:48:32.172 INFO:tasks.workunit.client.1.smithi130.stdout:8/394: dread d1/d2/f8c [0,4194304] 0 2024-05-13T22:48:32.174 INFO:tasks.workunit.client.1.smithi130.stdout:8/395: read - d1/d2/db/d15/d16/f8f zero size 2024-05-13T22:48:32.174 INFO:tasks.workunit.client.1.smithi130.stdout:1/429: dread db/d4a/f4b [0,4194304] 0 2024-05-13T22:48:32.175 INFO:tasks.workunit.client.1.smithi130.stdout:0/465: truncate d0/d7/dd/d17/d1f/d2c/f7a 5120794 0 2024-05-13T22:48:32.175 INFO:tasks.workunit.client.1.smithi130.stdout:8/396: chown d1/d2/db/c2b 249107225 1 2024-05-13T22:48:32.180 INFO:tasks.workunit.client.1.smithi130.stdout:0/466: dwrite d0/d7/dd/d17/d1f/d2c/d45/d57/f89 [0,4194304] 0 2024-05-13T22:48:32.189 INFO:tasks.workunit.client.1.smithi130.stdout:3/428: rename d8/d1a/f7a to d8/d1a/d1c/f94 0 2024-05-13T22:48:32.193 INFO:tasks.workunit.client.1.smithi130.stdout:7/421: truncate d7/da/f5c 3559594 0 2024-05-13T22:48:32.199 INFO:tasks.workunit.client.1.smithi130.stdout:8/397: dread d1/f33 [0,4194304] 0 2024-05-13T22:48:32.201 INFO:tasks.workunit.client.1.smithi130.stdout:8/398: dread d1/d2/db/d15/d16/d54/d58/f7a [0,4194304] 0 2024-05-13T22:48:32.205 INFO:tasks.workunit.client.1.smithi130.stdout:0/467: symlink d0/d7/dd/d17/d52/la3 0 2024-05-13T22:48:32.207 INFO:tasks.workunit.client.1.smithi130.stdout:8/399: dwrite d1/d2/db/d15/d16/f8f [0,4194304] 0 2024-05-13T22:48:32.213 INFO:tasks.workunit.client.1.smithi130.stdout:5/414: creat d6/f86 x:0 0 0 2024-05-13T22:48:32.216 INFO:tasks.workunit.client.1.smithi130.stdout:9/491: rename d1/f8 to d1/d7/d23/d28/d32/d45/f9b 0 2024-05-13T22:48:32.223 INFO:tasks.workunit.client.1.smithi130.stdout:6/461: dwrite d4/d15/d1d/d28/f47 [0,4194304] 0 2024-05-13T22:48:32.224 INFO:tasks.workunit.client.1.smithi130.stdout:0/468: symlink d0/d7/dd/d2e/d77/d8d/d9f/la4 0 2024-05-13T22:48:32.226 INFO:tasks.workunit.client.1.smithi130.stdout:8/400: creat d1/d2/db/d15/d18/d1a/f90 x:0 0 0 2024-05-13T22:48:32.227 INFO:tasks.workunit.client.1.smithi130.stdout:4/440: creat d8/f8b x:0 0 0 2024-05-13T22:48:32.229 INFO:tasks.workunit.client.1.smithi130.stdout:4/441: read d8/dc/d35/d33/d44/f4b [1704391,14213] 0 2024-05-13T22:48:32.229 INFO:tasks.workunit.client.1.smithi130.stdout:4/442: chown d8/dc/d80/c32 644450176 1 2024-05-13T22:48:32.235 INFO:tasks.workunit.client.1.smithi130.stdout:2/445: getdents d4/db/d26 0 2024-05-13T22:48:32.237 INFO:tasks.workunit.client.1.smithi130.stdout:6/462: dread - d4/d15/d1f/d3c/f96 zero size 2024-05-13T22:48:32.238 INFO:tasks.workunit.client.1.smithi130.stdout:0/469: mknod d0/d7/dd/d17/d52/ca5 0 2024-05-13T22:48:32.239 INFO:tasks.workunit.client.1.smithi130.stdout:6/463: dread - d4/d15/d1d/d60/f66 zero size 2024-05-13T22:48:32.245 INFO:tasks.workunit.client.1.smithi130.stdout:9/492: unlink d1/d7/f1a 0 2024-05-13T22:48:32.247 INFO:tasks.workunit.client.1.smithi130.stdout:7/422: dwrite d7/da/d1d/d3d/d46/f6e [0,4194304] 0 2024-05-13T22:48:32.249 INFO:tasks.workunit.client.1.smithi130.stdout:0/470: mknod d0/d7/dd/d17/d52/d67/ca6 0 2024-05-13T22:48:32.249 INFO:tasks.workunit.client.1.smithi130.stdout:5/415: creat d6/d7/d19/d4d/f87 x:0 0 0 2024-05-13T22:48:32.254 INFO:tasks.workunit.client.1.smithi130.stdout:3/429: rename d8/d1a/d1c/c4e to d8/dc/c95 0 2024-05-13T22:48:32.258 INFO:tasks.workunit.client.1.smithi130.stdout:9/493: creat d1/d7/d23/d28/d73/d8d/f9c x:0 0 0 2024-05-13T22:48:32.260 INFO:tasks.workunit.client.1.smithi130.stdout:4/443: truncate d8/dc/d35/f50 2977728 0 2024-05-13T22:48:32.265 INFO:tasks.workunit.client.1.smithi130.stdout:8/401: link d1/d2/db/c6e d1/d2/db/d15/c91 0 2024-05-13T22:48:32.269 INFO:tasks.workunit.client.1.smithi130.stdout:7/423: dread d7/da/d1d/d3d/d46/f4b [0,4194304] 0 2024-05-13T22:48:32.269 INFO:tasks.workunit.client.1.smithi130.stdout:0/471: truncate d0/d7/f23 212069 0 2024-05-13T22:48:32.271 INFO:tasks.workunit.client.1.smithi130.stdout:5/416: dwrite d6/d7/d19/f2c [0,4194304] 0 2024-05-13T22:48:32.272 INFO:tasks.workunit.client.1.smithi130.stdout:2/446: rename d4/db/dc/d18/d2a/d30/d88/d90/d91 to d4/db/d26/d9e 0 2024-05-13T22:48:32.273 INFO:tasks.workunit.client.1.smithi130.stdout:5/417: write d6/dd/d18/d6e/d76/d7f/f85 [7715,59394] 0 2024-05-13T22:48:32.276 INFO:tasks.workunit.client.1.smithi130.stdout:0/472: dwrite d0/d7/dd/d17/d52/d67/d68/d96/f84 [0,4194304] 0 2024-05-13T22:48:32.285 INFO:tasks.workunit.client.1.smithi130.stdout:7/424: creat d7/da/d1d/d42/d43/f8e x:0 0 0 2024-05-13T22:48:32.288 INFO:tasks.workunit.client.1.smithi130.stdout:7/425: dread d7/da/d1d/d3d/d46/f4b [0,4194304] 0 2024-05-13T22:48:32.290 INFO:tasks.workunit.client.1.smithi130.stdout:1/430: sync 2024-05-13T22:48:32.291 INFO:tasks.workunit.client.1.smithi130.stdout:8/402: rename d1/d2/db/f40 to d1/d2/db/d15/f92 0 2024-05-13T22:48:32.295 INFO:tasks.workunit.client.1.smithi130.stdout:2/447: write d4/db/dc/d18/d2a/f44 [1499412,41845] 0 2024-05-13T22:48:32.296 INFO:tasks.workunit.client.1.smithi130.stdout:7/426: dwrite d7/da/d1d/d3d/d46/d67/f6c [0,4194304] 0 2024-05-13T22:48:32.298 INFO:tasks.workunit.client.1.smithi130.stdout:7/427: stat d7/da/d1d/d3d/d46/d67/f6c 0 2024-05-13T22:48:32.300 INFO:tasks.workunit.client.1.smithi130.stdout:5/418: fsync d6/d7/fa 0 2024-05-13T22:48:32.301 INFO:tasks.workunit.client.1.smithi130.stdout:5/419: readlink d6/dd/d18/l2e 0 2024-05-13T22:48:32.304 INFO:tasks.workunit.client.1.smithi130.stdout:6/464: link d4/d15/d1f/l43 d4/d15/d1d/d28/la1 0 2024-05-13T22:48:32.305 INFO:tasks.workunit.client.1.smithi130.stdout:6/465: fsync d4/d15/d1f/d46/d50/f8a 0 2024-05-13T22:48:32.307 INFO:tasks.workunit.client.1.smithi130.stdout:8/403: mknod d1/d2/db/c93 0 2024-05-13T22:48:32.307 INFO:tasks.workunit.client.1.smithi130.stdout:3/430: creat d8/dc/d19/f96 x:0 0 0 2024-05-13T22:48:32.309 INFO:tasks.workunit.client.1.smithi130.stdout:9/494: creat d1/f9d x:0 0 0 2024-05-13T22:48:32.310 INFO:tasks.workunit.client.1.smithi130.stdout:9/495: write d1/d15/f1d [538070,70462] 0 2024-05-13T22:48:32.311 INFO:tasks.workunit.client.1.smithi130.stdout:0/473: mkdir d0/d7/dd/d2e/d9e/da7 0 2024-05-13T22:48:32.314 INFO:tasks.workunit.client.1.smithi130.stdout:8/404: dwrite d1/d2/db/d15/d18/f3e [0,4194304] 0 2024-05-13T22:48:32.314 INFO:tasks.workunit.client.1.smithi130.stdout:9/496: write d1/d7/d23/d28/d2d/d2f/f6a [361310,27438] 0 2024-05-13T22:48:32.314 INFO:tasks.workunit.client.1.smithi130.stdout:4/444: truncate d8/dc/d35/f50 2007106 0 2024-05-13T22:48:32.315 INFO:tasks.workunit.client.1.smithi130.stdout:8/405: write d1/d2/db/d15/d18/d1a/f90 [648185,55156] 0 2024-05-13T22:48:32.316 INFO:tasks.workunit.client.1.smithi130.stdout:1/431: mknod db/d1b/d1f/d49/d56/d88/c8d 0 2024-05-13T22:48:32.317 INFO:tasks.workunit.client.1.smithi130.stdout:1/432: chown db/d1b/d1f/d2a/d38/c76 14 1 2024-05-13T22:48:32.318 INFO:tasks.workunit.client.1.smithi130.stdout:1/433: chown db/d1b/d1f/d2a/d38/d46/d6d/f73 2 1 2024-05-13T22:48:32.322 INFO:tasks.workunit.client.1.smithi130.stdout:7/428: mkdir d7/d8f 0 2024-05-13T22:48:32.322 INFO:tasks.workunit.client.1.smithi130.stdout:7/429: stat d7/da/d1d/d5d/d70/f85 0 2024-05-13T22:48:32.325 INFO:tasks.workunit.client.1.smithi130.stdout:5/420: symlink d6/l88 0 2024-05-13T22:48:32.333 INFO:tasks.workunit.client.1.smithi130.stdout:8/406: unlink d1/d2/db/f51 0 2024-05-13T22:48:32.335 INFO:tasks.workunit.client.1.smithi130.stdout:8/407: fsync d1/d2/db/d15/d16/f5b 0 2024-05-13T22:48:32.335 INFO:tasks.workunit.client.1.smithi130.stdout:3/431: link d8/dc/d7b/f90 d8/dc/d19/d27/f97 0 2024-05-13T22:48:32.335 INFO:tasks.workunit.client.1.smithi130.stdout:4/445: dwrite d8/dc/d29/f2c [4194304,4194304] 0 2024-05-13T22:48:32.336 INFO:tasks.workunit.client.1.smithi130.stdout:7/430: mknod d7/d8/d7e/c90 0 2024-05-13T22:48:32.336 INFO:tasks.workunit.client.1.smithi130.stdout:8/408: fsync d1/d2/db/d15/d16/f88 0 2024-05-13T22:48:32.337 INFO:tasks.workunit.client.1.smithi130.stdout:4/446: chown d8/df/d81/c83 23478 1 2024-05-13T22:48:32.337 INFO:tasks.workunit.client.1.smithi130.stdout:0/474: mkdir d0/d7/dd/d2e/da8 0 2024-05-13T22:48:32.338 INFO:tasks.workunit.client.1.smithi130.stdout:4/447: chown d8/dc/d35/d33/d44/d61 480880 1 2024-05-13T22:48:32.340 INFO:tasks.workunit.client.1.smithi130.stdout:0/475: chown d0/d7/dd/d17/d1f/d2c/d5a/c6d 1260504502 1 2024-05-13T22:48:32.340 INFO:tasks.workunit.client.1.smithi130.stdout:9/497: mkdir d1/d15/d9e 0 2024-05-13T22:48:32.343 INFO:tasks.workunit.client.1.smithi130.stdout:8/409: dwrite d1/f65 [0,4194304] 0 2024-05-13T22:48:32.345 INFO:tasks.workunit.client.1.smithi130.stdout:5/421: mknod d6/dd/d18/d6e/d82/d84/c89 0 2024-05-13T22:48:32.351 INFO:tasks.workunit.client.1.smithi130.stdout:0/476: rename d0/d7/dd/d2e/d77 to d0/d7/dd/d69/da9 0 2024-05-13T22:48:32.354 INFO:tasks.workunit.client.1.smithi130.stdout:8/410: creat d1/d2/db/d15/d16/d54/d58/f94 x:0 0 0 2024-05-13T22:48:32.355 INFO:tasks.workunit.client.1.smithi130.stdout:8/411: write d1/d2/db/d15/d18/f60 [690628,116363] 0 2024-05-13T22:48:32.355 INFO:tasks.workunit.client.1.smithi130.stdout:7/431: creat d7/d8/d9/d66/d65/f91 x:0 0 0 2024-05-13T22:48:32.361 INFO:tasks.workunit.client.1.smithi130.stdout:7/432: dwrite d7/f71 [0,4194304] 0 2024-05-13T22:48:32.365 INFO:tasks.workunit.client.1.smithi130.stdout:8/412: creat d1/d2/db/d15/d7e/f95 x:0 0 0 2024-05-13T22:48:32.366 INFO:tasks.workunit.client.1.smithi130.stdout:3/432: write d8/f34 [2090403,70155] 0 2024-05-13T22:48:32.372 INFO:tasks.workunit.client.1.smithi130.stdout:4/448: getdents d8/dc/d29 0 2024-05-13T22:48:32.372 INFO:tasks.workunit.client.1.smithi130.stdout:4/449: fsync d8/f8b 0 2024-05-13T22:48:32.376 INFO:tasks.workunit.client.1.smithi130.stdout:0/477: creat d0/faa x:0 0 0 2024-05-13T22:48:32.377 INFO:tasks.workunit.client.1.smithi130.stdout:3/433: symlink d8/dc/l98 0 2024-05-13T22:48:32.378 INFO:tasks.workunit.client.1.smithi130.stdout:0/478: fdatasync d0/d7/dd/d17/d52/d67/d68/d96/f84 0 2024-05-13T22:48:32.379 INFO:tasks.workunit.client.1.smithi130.stdout:4/450: dwrite d8/dc/d26/f30 [0,4194304] 0 2024-05-13T22:48:32.380 INFO:tasks.workunit.client.1.smithi130.stdout:5/422: write d6/d7/f5e [3128136,54003] 0 2024-05-13T22:48:32.380 INFO:tasks.workunit.client.1.smithi130.stdout:3/434: write d8/dc/d19/f96 [187899,93751] 0 2024-05-13T22:48:32.381 INFO:tasks.workunit.client.1.smithi130.stdout:0/479: chown d0/d7/dd/d69/da9/d8d/d9f 30881 1 2024-05-13T22:48:32.382 INFO:tasks.workunit.client.1.smithi130.stdout:3/435: write d8/d1a/d1c/f49 [4763130,76428] 0 2024-05-13T22:48:32.389 INFO:tasks.workunit.client.1.smithi130.stdout:9/498: dread d1/d7/d23/d28/d32/d45/f9b [0,4194304] 0 2024-05-13T22:48:32.389 INFO:tasks.workunit.client.1.smithi130.stdout:9/499: chown d1/d7/d23/d28/d73/d8d/f8f 0 1 2024-05-13T22:48:32.390 INFO:tasks.workunit.client.1.smithi130.stdout:9/500: stat d1/db/f5e 0 2024-05-13T22:48:32.390 INFO:tasks.workunit.client.1.smithi130.stdout:3/436: creat d8/dc/d7b/f99 x:0 0 0 2024-05-13T22:48:32.391 INFO:tasks.workunit.client.1.smithi130.stdout:3/437: fsync d8/dc/f21 0 2024-05-13T22:48:32.395 INFO:tasks.workunit.client.1.smithi130.stdout:0/480: rename d0/d7/dd/d17/d52/d5b/l73 to d0/d7/dd/d69/da9/d8d/lab 0 2024-05-13T22:48:32.397 INFO:tasks.workunit.client.1.smithi130.stdout:5/423: mkdir d6/d7/d19/d4d/d8a 0 2024-05-13T22:48:32.399 INFO:tasks.workunit.client.1.smithi130.stdout:0/481: dread d0/d7/dd/d69/da9/d8d/f9d [0,4194304] 0 2024-05-13T22:48:32.399 INFO:tasks.workunit.client.1.smithi130.stdout:9/501: rmdir d1/db 39 2024-05-13T22:48:32.404 INFO:tasks.workunit.client.1.smithi130.stdout:5/424: dwrite d6/d1a/f2a [0,4194304] 0 2024-05-13T22:48:32.405 INFO:tasks.workunit.client.1.smithi130.stdout:3/438: creat d8/dc/d15/f9a x:0 0 0 2024-05-13T22:48:32.409 INFO:tasks.workunit.client.1.smithi130.stdout:9/502: dwrite d1/d15/f1d [0,4194304] 0 2024-05-13T22:48:32.410 INFO:tasks.workunit.client.1.smithi130.stdout:9/503: write d1/d15/f16 [1334408,73726] 0 2024-05-13T22:48:32.413 INFO:tasks.workunit.client.1.smithi130.stdout:0/482: mkdir d0/d7/dd/d69/dac 0 2024-05-13T22:48:32.414 INFO:tasks.workunit.client.1.smithi130.stdout:4/451: dwrite d8/dc/f5b [0,4194304] 0 2024-05-13T22:48:32.416 INFO:tasks.workunit.client.1.smithi130.stdout:3/439: creat d8/d1a/d55/f9b x:0 0 0 2024-05-13T22:48:32.423 INFO:tasks.workunit.client.1.smithi130.stdout:5/425: creat d6/dd/d18/d6e/f8b x:0 0 0 2024-05-13T22:48:32.424 INFO:tasks.workunit.client.1.smithi130.stdout:3/440: write d8/dc/d19/f30 [1450730,77766] 0 2024-05-13T22:48:32.424 INFO:tasks.workunit.client.1.smithi130.stdout:0/483: rmdir d0/d7/dd/d17/d1f 39 2024-05-13T22:48:32.425 INFO:tasks.workunit.client.1.smithi130.stdout:5/426: chown d6/dd/d18/l10 1516023866 1 2024-05-13T22:48:32.426 INFO:tasks.workunit.client.1.smithi130.stdout:3/441: fsync d8/d1a/d1c/f6e 0 2024-05-13T22:48:32.427 INFO:tasks.workunit.client.1.smithi130.stdout:3/442: readlink d8/l3f 0 2024-05-13T22:48:32.428 INFO:tasks.workunit.client.1.smithi130.stdout:4/452: unlink d8/dc/d80/l7e 0 2024-05-13T22:48:32.431 INFO:tasks.workunit.client.1.smithi130.stdout:0/484: write d0/d7/fb [3624832,73161] 0 2024-05-13T22:48:32.434 INFO:tasks.workunit.client.1.smithi130.stdout:9/504: write d1/d7/d23/d3d/d43/f82 [358186,69460] 0 2024-05-13T22:48:32.435 INFO:tasks.workunit.client.1.smithi130.stdout:3/443: mkdir d8/dc/d19/d2c/d61/d9c 0 2024-05-13T22:48:32.437 INFO:tasks.workunit.client.1.smithi130.stdout:4/453: mkdir d8/dc/d35/d33/d44/d56/d8c 0 2024-05-13T22:48:32.438 INFO:tasks.workunit.client.1.smithi130.stdout:4/454: write d8/dc/d80/f78 [6989766,82161] 0 2024-05-13T22:48:32.438 INFO:tasks.workunit.client.1.smithi130.stdout:2/448: sync 2024-05-13T22:48:32.439 INFO:tasks.workunit.client.1.smithi130.stdout:5/427: mknod d6/dd/d79/c8c 0 2024-05-13T22:48:32.440 INFO:tasks.workunit.client.1.smithi130.stdout:4/455: truncate d8/f7b 867089 0 2024-05-13T22:48:32.440 INFO:tasks.workunit.client.1.smithi130.stdout:5/428: stat d6/d7/f43 0 2024-05-13T22:48:32.440 INFO:tasks.workunit.client.1.smithi130.stdout:4/456: fsync d8/f8b 0 2024-05-13T22:48:32.441 INFO:tasks.workunit.client.1.smithi130.stdout:2/449: write d4/db/dc/d18/d2a/d30/d69/f87 [755570,35143] 0 2024-05-13T22:48:32.441 INFO:tasks.workunit.client.1.smithi130.stdout:5/429: dread - d6/d7/d19/d4d/f87 zero size 2024-05-13T22:48:32.442 INFO:tasks.workunit.client.1.smithi130.stdout:2/450: readlink d4/d49/l3c 0 2024-05-13T22:48:32.442 INFO:tasks.workunit.client.1.smithi130.stdout:3/444: rmdir d8/dc/d19/d27 39 2024-05-13T22:48:32.443 INFO:tasks.workunit.client.1.smithi130.stdout:4/457: chown d8/dc/d35/d33/d44/l5d 215302313 1 2024-05-13T22:48:32.443 INFO:tasks.workunit.client.1.smithi130.stdout:5/430: stat d6/d7/f46 0 2024-05-13T22:48:32.445 INFO:tasks.workunit.client.1.smithi130.stdout:0/485: creat d0/fad x:0 0 0 2024-05-13T22:48:32.448 INFO:tasks.workunit.client.1.smithi130.stdout:4/458: dwrite f6 [0,4194304] 0 2024-05-13T22:48:32.455 INFO:tasks.workunit.client.1.smithi130.stdout:2/451: mknod d4/db/dc/d18/d2a/d30/d88/d90/c9f 0 2024-05-13T22:48:32.458 INFO:tasks.workunit.client.1.smithi130.stdout:9/505: creat d1/d7/d23/d28/d2d/d2f/d63/f9f x:0 0 0 2024-05-13T22:48:32.466 INFO:tasks.workunit.client.1.smithi130.stdout:4/459: rename d8/dc/d35/d33/d3f/d57/f5f to d8/dc/d35/f8d 0 2024-05-13T22:48:32.470 INFO:tasks.workunit.client.1.smithi130.stdout:3/445: dwrite d8/dc/d7b/f90 [0,4194304] 0 2024-05-13T22:48:32.470 INFO:tasks.workunit.client.1.smithi130.stdout:0/486: write d0/d7/dd/d2e/d4b/f55 [3376567,31039] 0 2024-05-13T22:48:32.472 INFO:tasks.workunit.client.1.smithi130.stdout:3/446: chown d8/dc/d15/l5c 1935 1 2024-05-13T22:48:32.473 INFO:tasks.workunit.client.1.smithi130.stdout:9/506: mknod d1/d15/d9e/ca0 0 2024-05-13T22:48:32.480 INFO:tasks.workunit.client.1.smithi130.stdout:5/431: dread d6/d7/d19/f1c [0,4194304] 0 2024-05-13T22:48:32.481 INFO:tasks.workunit.client.1.smithi130.stdout:2/452: dread d4/d49/d48/d4a/f8c [4194304,4194304] 0 2024-05-13T22:48:32.482 INFO:tasks.workunit.client.1.smithi130.stdout:2/453: chown d4/d49/d42/l45 35 1 2024-05-13T22:48:32.482 INFO:tasks.workunit.client.1.smithi130.stdout:2/454: write d4/f8 [3209304,12868] 0 2024-05-13T22:48:32.486 INFO:tasks.workunit.client.1.smithi130.stdout:9/507: mkdir d1/d7/d23/d3d/da1 0 2024-05-13T22:48:32.488 INFO:tasks.workunit.client.1.smithi130.stdout:4/460: getdents d8/dc/d35/d33/d44/d56/d8c 0 2024-05-13T22:48:32.492 INFO:tasks.workunit.client.1.smithi130.stdout:5/432: symlink d6/dd/l8d 0 2024-05-13T22:48:32.495 INFO:tasks.workunit.client.1.smithi130.stdout:2/455: dread d4/f19 [0,4194304] 0 2024-05-13T22:48:32.495 INFO:tasks.workunit.client.1.smithi130.stdout:9/508: mkdir d1/d7/d69/da2 0 2024-05-13T22:48:32.496 INFO:tasks.workunit.client.1.smithi130.stdout:2/456: dread - d4/db/dc/d18/f7e zero size 2024-05-13T22:48:32.496 INFO:tasks.workunit.client.1.smithi130.stdout:2/457: chown d4/db/d26/d9e 104 1 2024-05-13T22:48:32.498 INFO:tasks.workunit.client.1.smithi130.stdout:0/487: rename d0/faa to d0/d7/dd/d17/fae 0 2024-05-13T22:48:32.500 INFO:tasks.workunit.client.1.smithi130.stdout:0/488: fdatasync d0/f8 0 2024-05-13T22:48:32.502 INFO:tasks.workunit.client.1.smithi130.stdout:3/447: rename d8/dc/d19/f7c to d8/dc/d19/d2c/d61/d9c/f9d 0 2024-05-13T22:48:32.503 INFO:tasks.workunit.client.1.smithi130.stdout:9/509: dwrite d1/d7/d23/d28/d2d/d2f/d63/f9f [0,4194304] 0 2024-05-13T22:48:32.504 INFO:tasks.workunit.client.1.smithi130.stdout:4/461: dwrite d8/f12 [0,4194304] 0 2024-05-13T22:48:32.505 INFO:tasks.workunit.client.1.smithi130.stdout:9/510: dread - d1/d7/d23/d28/f64 zero size 2024-05-13T22:48:32.507 INFO:tasks.workunit.client.1.smithi130.stdout:6/466: sync 2024-05-13T22:48:32.513 INFO:tasks.workunit.client.1.smithi130.stdout:5/433: getdents d6/dd/d18/d6e/d82/d84 0 2024-05-13T22:48:32.513 INFO:tasks.workunit.client.1.smithi130.stdout:0/489: truncate d0/d7/dd/d17/d1f/d2c/d45/d57/f92 164776 0 2024-05-13T22:48:32.514 INFO:tasks.workunit.client.1.smithi130.stdout:4/462: dwrite d8/dc/d29/f3a [8388608,4194304] 0 2024-05-13T22:48:32.516 INFO:tasks.workunit.client.1.smithi130.stdout:4/463: fdatasync d8/dc/f86 0 2024-05-13T22:48:32.519 INFO:tasks.workunit.client.1.smithi130.stdout:4/464: fsync d8/df/f40 0 2024-05-13T22:48:32.519 INFO:tasks.workunit.client.1.smithi130.stdout:3/448: creat d8/dc/d19/f9e x:0 0 0 2024-05-13T22:48:32.520 INFO:tasks.workunit.client.1.smithi130.stdout:6/467: dwrite d4/d15/d1d/d28/d4e/f99 [0,4194304] 0 2024-05-13T22:48:32.521 INFO:tasks.workunit.client.1.smithi130.stdout:5/434: mknod d6/dd/d79/c8e 0 2024-05-13T22:48:32.522 INFO:tasks.workunit.client.1.smithi130.stdout:5/435: chown d6/dd/f37 7751 1 2024-05-13T22:48:32.523 INFO:tasks.workunit.client.1.smithi130.stdout:2/458: rename d4/db/c3d to d4/db/dc/ca0 0 2024-05-13T22:48:32.524 INFO:tasks.workunit.client.1.smithi130.stdout:0/490: mkdir d0/d7/dd/d17/d52/d5b/daf 0 2024-05-13T22:48:32.525 INFO:tasks.workunit.client.1.smithi130.stdout:2/459: chown d4/db/f63 493237 1 2024-05-13T22:48:32.531 INFO:tasks.workunit.client.1.smithi130.stdout:6/468: mkdir d4/d15/d1d/d60/da2 0 2024-05-13T22:48:32.531 INFO:tasks.workunit.client.1.smithi130.stdout:4/465: mknod d8/dc/d35/d33/d44/d56/d8c/c8e 0 2024-05-13T22:48:32.531 INFO:tasks.workunit.client.1.smithi130.stdout:0/491: mkdir d0/d7/dd/d17/d1f/db0 0 2024-05-13T22:48:32.532 INFO:tasks.workunit.client.1.smithi130.stdout:6/469: chown d4/d15/d1d/d28/f71 40538 1 2024-05-13T22:48:32.533 INFO:tasks.workunit.client.1.smithi130.stdout:3/449: fdatasync d8/dc/d19/f20 0 2024-05-13T22:48:32.534 INFO:tasks.workunit.client.1.smithi130.stdout:3/450: fdatasync d8/dc/f22 0 2024-05-13T22:48:32.535 INFO:tasks.workunit.client.1.smithi130.stdout:4/466: symlink d8/dc/d35/d33/d44/d77/l8f 0 2024-05-13T22:48:32.535 INFO:tasks.workunit.client.1.smithi130.stdout:2/460: getdents d4/db/dc/d18/d2a/d30/d88/d99/d9a 0 2024-05-13T22:48:32.536 INFO:tasks.workunit.client.1.smithi130.stdout:4/467: write d8/f12 [4474104,22034] 0 2024-05-13T22:48:32.536 INFO:tasks.workunit.client.1.smithi130.stdout:0/492: fsync d0/d7/dd/d17/d1f/d2c/f50 0 2024-05-13T22:48:32.548 INFO:tasks.workunit.client.1.smithi130.stdout:3/451: dwrite d8/dc/f3e [4194304,4194304] 0 2024-05-13T22:48:32.552 INFO:tasks.workunit.client.1.smithi130.stdout:6/470: creat d4/d15/d8b/da0/fa3 x:0 0 0 2024-05-13T22:48:32.553 INFO:tasks.workunit.client.1.smithi130.stdout:6/471: stat d4/d15/d1f/d46/d50/f59 0 2024-05-13T22:48:32.554 INFO:tasks.workunit.client.1.smithi130.stdout:6/472: write d4/d15/d1f/d46/d50/f51 [1006708,7094] 0 2024-05-13T22:48:32.555 INFO:tasks.workunit.client.1.smithi130.stdout:4/468: rmdir d8/dc/d35/d33/d3f/d57 39 2024-05-13T22:48:32.560 INFO:tasks.workunit.client.1.smithi130.stdout:5/436: getdents d6/d1a/d72 0 2024-05-13T22:48:32.561 INFO:tasks.workunit.client.1.smithi130.stdout:2/461: fdatasync d4/db/f22 0 2024-05-13T22:48:32.562 INFO:tasks.workunit.client.1.smithi130.stdout:2/462: fdatasync d4/d49/d42/f89 0 2024-05-13T22:48:32.567 INFO:tasks.workunit.client.1.smithi130.stdout:6/473: write d4/d16/f2a [977196,115692] 0 2024-05-13T22:48:32.576 INFO:tasks.workunit.client.1.smithi130.stdout:4/469: getdents d8/dc/d29 0 2024-05-13T22:48:32.580 INFO:tasks.workunit.client.1.smithi130.stdout:3/452: creat d8/dc/f9f x:0 0 0 2024-05-13T22:48:32.581 INFO:tasks.workunit.client.1.smithi130.stdout:3/453: chown d8/f5e 23262 1 2024-05-13T22:48:32.582 INFO:tasks.workunit.client.1.smithi130.stdout:0/493: link d0/d7/d29/l39 d0/d7/dd/d69/da9/d8d/lb1 0 2024-05-13T22:48:32.591 INFO:tasks.workunit.client.1.smithi130.stdout:4/470: dwrite d8/dc/d80/f7d [4194304,4194304] 0 2024-05-13T22:48:32.593 INFO:tasks.workunit.client.1.smithi130.stdout:5/437: dwrite d6/d7/f45 [0,4194304] 0 2024-05-13T22:48:32.596 INFO:tasks.workunit.client.1.smithi130.stdout:6/474: getdents d4/d15/d1f 0 2024-05-13T22:48:32.598 INFO:tasks.workunit.client.1.smithi130.stdout:3/454: dwrite d8/d35/f3d [0,4194304] 0 2024-05-13T22:48:32.602 INFO:tasks.workunit.client.1.smithi130.stdout:5/438: mknod d6/dd/d18/d6e/d82/d56/c8f 0 2024-05-13T22:48:32.605 INFO:tasks.workunit.client.1.smithi130.stdout:0/494: truncate d0/d7/dd/f46 1843886 0 2024-05-13T22:48:32.606 INFO:tasks.workunit.client.1.smithi130.stdout:0/495: fdatasync d0/d7/dd/d2e/d4b/f15 0 2024-05-13T22:48:32.608 INFO:tasks.workunit.client.1.smithi130.stdout:3/455: stat d8/d35/f80 0 2024-05-13T22:48:32.610 INFO:tasks.workunit.client.1.smithi130.stdout:4/471: dread d8/df/f20 [0,4194304] 0 2024-05-13T22:48:32.613 INFO:tasks.workunit.client.1.smithi130.stdout:4/472: dread f7 [4194304,4194304] 0 2024-05-13T22:48:32.614 INFO:tasks.workunit.client.1.smithi130.stdout:6/475: mkdir d4/d15/d1d/d60/da2/da4 0 2024-05-13T22:48:32.614 INFO:tasks.workunit.client.1.smithi130.stdout:0/496: dwrite d0/d7/f22 [0,4194304] 0 2024-05-13T22:48:32.615 INFO:tasks.workunit.client.1.smithi130.stdout:7/433: sync 2024-05-13T22:48:32.615 INFO:tasks.workunit.client.1.smithi130.stdout:1/434: sync 2024-05-13T22:48:32.615 INFO:tasks.workunit.client.1.smithi130.stdout:8/413: sync 2024-05-13T22:48:32.617 INFO:tasks.workunit.client.1.smithi130.stdout:8/414: fdatasync d1/d2/db/d15/d18/f76 0 2024-05-13T22:48:32.618 INFO:tasks.workunit.client.1.smithi130.stdout:3/456: creat d8/d1a/d1c/fa0 x:0 0 0 2024-05-13T22:48:32.620 INFO:tasks.workunit.client.1.smithi130.stdout:4/473: readlink d8/df/l5e 0 2024-05-13T22:48:32.623 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:32 smithi102 ceph-mon[28599]: pgmap v33: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 43 MiB/s rd, 164 MiB/s wr, 325 op/s 2024-05-13T22:48:32.623 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:32 smithi102 ceph-mon[28599]: Upgrade: Updating mgr.smithi102.xcijrj 2024-05-13T22:48:32.624 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:32 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:32.624 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi102.xcijrj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:48:32.624 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:32 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi102.xcijrj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:48:32.624 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:48:32.624 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:32 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:48:32.624 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:32 smithi102 ceph-mon[28599]: Deploying daemon mgr.smithi102.xcijrj on smithi102 2024-05-13T22:48:32.624 INFO:tasks.workunit.client.1.smithi130.stdout:0/497: unlink d0/f21 0 2024-05-13T22:48:32.627 INFO:tasks.workunit.client.1.smithi130.stdout:4/474: dwrite d8/dc/d35/d33/d44/f54 [0,4194304] 0 2024-05-13T22:48:32.629 INFO:tasks.workunit.client.1.smithi130.stdout:7/434: mkdir d7/da/d1d/d3d/d46/d92 0 2024-05-13T22:48:32.629 INFO:tasks.workunit.client.1.smithi130.stdout:7/435: chown d7/d8/d7e/d8b 35 1 2024-05-13T22:48:32.633 INFO:tasks.workunit.client.1.smithi130.stdout:8/415: unlink d1/dc/f26 0 2024-05-13T22:48:32.642 INFO:tasks.workunit.client.1.smithi130.stdout:5/439: write d6/d7/f22 [657093,7990] 0 2024-05-13T22:48:32.642 INFO:tasks.workunit.client.1.smithi130.stdout:5/440: chown d6/d7/f5e 7123 1 2024-05-13T22:48:32.644 INFO:tasks.workunit.client.1.smithi130.stdout:5/441: read d6/dd/f3e [266371,112157] 0 2024-05-13T22:48:32.652 INFO:tasks.workunit.client.1.smithi130.stdout:6/476: creat d4/d15/d1d/fa5 x:0 0 0 2024-05-13T22:48:32.656 INFO:tasks.workunit.client.1.smithi130.stdout:1/435: creat db/d1b/d1f/d2a/f8e x:0 0 0 2024-05-13T22:48:32.663 INFO:tasks.workunit.client.1.smithi130.stdout:7/436: symlink d7/da/d1d/l93 0 2024-05-13T22:48:32.663 INFO:tasks.workunit.client.1.smithi130.stdout:3/457: dwrite f3 [0,4194304] 0 2024-05-13T22:48:32.664 INFO:tasks.workunit.client.1.smithi130.stdout:7/437: chown d7/d8/d9/l59 8307 1 2024-05-13T22:48:32.666 INFO:tasks.workunit.client.1.smithi130.stdout:7/438: dread - d7/d8/d39/f79 zero size 2024-05-13T22:48:32.673 INFO:tasks.workunit.client.1.smithi130.stdout:1/436: rename db/c6e to db/d58/c8f 0 2024-05-13T22:48:32.676 INFO:tasks.workunit.client.1.smithi130.stdout:5/442: symlink d6/dd/d18/d6e/d82/l90 0 2024-05-13T22:48:32.677 INFO:tasks.workunit.client.1.smithi130.stdout:5/443: fsync d6/dd/d18/f36 0 2024-05-13T22:48:32.677 INFO:tasks.workunit.client.1.smithi130.stdout:7/439: creat d7/d8/d39/f94 x:0 0 0 2024-05-13T22:48:32.679 INFO:tasks.workunit.client.1.smithi130.stdout:5/444: read d6/d7/f28 [3624089,129130] 0 2024-05-13T22:48:32.680 INFO:tasks.workunit.client.1.smithi130.stdout:1/437: dwrite db/d1b/d1f/d49/f61 [0,4194304] 0 2024-05-13T22:48:32.681 INFO:tasks.workunit.client.1.smithi130.stdout:7/440: truncate d7/da/d1d/d3d/d46/d67/d40/f77 299849 0 2024-05-13T22:48:32.682 INFO:tasks.workunit.client.1.smithi130.stdout:0/498: getdents d0/d7/dd/d69/da9 0 2024-05-13T22:48:32.682 INFO:tasks.workunit.client.1.smithi130.stdout:9/511: sync 2024-05-13T22:48:32.683 INFO:tasks.workunit.client.1.smithi130.stdout:8/416: write d1/d2/db/d15/d18/f4c [1136286,109009] 0 2024-05-13T22:48:32.683 INFO:tasks.workunit.client.1.smithi130.stdout:7/441: dread - d7/f83 zero size 2024-05-13T22:48:32.684 INFO:tasks.workunit.client.1.smithi130.stdout:9/512: dread - d1/d7/d69/d79/f9a zero size 2024-05-13T22:48:32.684 INFO:tasks.workunit.client.1.smithi130.stdout:8/417: readlink d1/d2/db/d15/d18/l2e 0 2024-05-13T22:48:32.688 INFO:tasks.workunit.client.1.smithi130.stdout:1/438: symlink db/d58/l90 0 2024-05-13T22:48:32.688 INFO:tasks.workunit.client.1.smithi130.stdout:5/445: read d6/d7/fa [146499,60601] 0 2024-05-13T22:48:32.692 INFO:tasks.workunit.client.1.smithi130.stdout:7/442: dwrite d7/da/d86/f8a [0,4194304] 0 2024-05-13T22:48:32.693 INFO:tasks.workunit.client.1.smithi130.stdout:9/513: dread d1/d7/d23/d3d/f40 [0,4194304] 0 2024-05-13T22:48:32.693 INFO:tasks.workunit.client.1.smithi130.stdout:8/418: dread d1/d2/db/d15/d16/f8f [0,4194304] 0 2024-05-13T22:48:32.698 INFO:tasks.workunit.client.1.smithi130.stdout:1/439: mknod db/d1b/d1f/d49/d56/d7f/c91 0 2024-05-13T22:48:32.700 INFO:tasks.workunit.client.1.smithi130.stdout:1/440: write db/d1b/d1f/d49/f48 [2325225,87536] 0 2024-05-13T22:48:32.703 INFO:tasks.workunit.client.1.smithi130.stdout:5/446: dread d6/dd/f1b [0,4194304] 0 2024-05-13T22:48:32.703 INFO:tasks.workunit.client.1.smithi130.stdout:7/443: unlink d7/da/f35 0 2024-05-13T22:48:32.704 INFO:tasks.workunit.client.1.smithi130.stdout:3/458: truncate d8/dc/f36 615653 0 2024-05-13T22:48:32.707 INFO:tasks.workunit.client.1.smithi130.stdout:6/477: dwrite d4/d15/d1f/d46/d50/f53 [0,4194304] 0 2024-05-13T22:48:32.707 INFO:tasks.workunit.client.1.smithi130.stdout:9/514: dread d1/d7/d69/d79/f80 [0,4194304] 0 2024-05-13T22:48:32.714 INFO:tasks.workunit.client.1.smithi130.stdout:0/499: dwrite d0/d7/dd/d17/d1f/d2c/d45/d57/f89 [0,4194304] 0 2024-05-13T22:48:32.716 INFO:tasks.workunit.client.1.smithi130.stdout:7/444: truncate d7/da/d1d/d42/f73 221145 0 2024-05-13T22:48:32.716 INFO:tasks.workunit.client.1.smithi130.stdout:2/463: sync 2024-05-13T22:48:32.716 INFO:tasks.workunit.client.1.smithi130.stdout:2/464: chown d4 50082 1 2024-05-13T22:48:32.717 INFO:tasks.workunit.client.1.smithi130.stdout:2/465: write d4/d49/d42/f89 [332286,86946] 0 2024-05-13T22:48:32.719 INFO:tasks.workunit.client.1.smithi130.stdout:5/447: truncate d6/d7/fa 543688 0 2024-05-13T22:48:32.721 INFO:tasks.workunit.client.1.smithi130.stdout:2/466: dread d4/db/dc/d18/d2a/d30/d69/f77 [0,4194304] 0 2024-05-13T22:48:32.725 INFO:tasks.workunit.client.1.smithi130.stdout:8/419: rename d1/d2/db/d15/d16/d54/d58/c6b to d1/d2/db/d15/d18/d42/d53/c96 0 2024-05-13T22:48:32.765 INFO:tasks.workunit.client.1.smithi130.stdout:1/441: creat db/d1b/d1f/d2a/d38/d46/d6d/d6f/f92 x:0 0 0 2024-05-13T22:48:32.768 INFO:tasks.workunit.client.1.smithi130.stdout:9/515: mknod d1/ca3 0 2024-05-13T22:48:32.768 INFO:tasks.workunit.client.1.smithi130.stdout:6/478: dwrite d4/d15/d1d/d60/f77 [0,4194304] 0 2024-05-13T22:48:32.770 INFO:tasks.workunit.client.1.smithi130.stdout:9/516: chown d1/d15/f16 38 1 2024-05-13T22:48:32.774 INFO:tasks.workunit.client.1.smithi130.stdout:7/445: dwrite d7/d8/f4c [0,4194304] 0 2024-05-13T22:48:32.778 INFO:tasks.workunit.client.1.smithi130.stdout:8/420: truncate d1/d2/db/d15/d18/f76 6288551 0 2024-05-13T22:48:32.781 INFO:tasks.workunit.client.1.smithi130.stdout:3/459: rename d8/d1a/d1c/f91 to d8/fa1 0 2024-05-13T22:48:32.782 INFO:tasks.workunit.client.1.smithi130.stdout:3/460: fsync d8/d1a/d1c/f6e 0 2024-05-13T22:48:32.783 INFO:tasks.workunit.client.1.smithi130.stdout:7/446: symlink d7/da/d1d/d3d/d46/d67/l95 0 2024-05-13T22:48:32.783 INFO:tasks.workunit.client.1.smithi130.stdout:3/461: write d8/dc/d19/d2c/f46 [580318,18551] 0 2024-05-13T22:48:32.785 INFO:tasks.workunit.client.1.smithi130.stdout:1/442: link db/d1b/d1f/d2a/c37 db/d1b/d1f/d5e/c93 0 2024-05-13T22:48:32.786 INFO:tasks.workunit.client.1.smithi130.stdout:8/421: creat d1/d2/db/d15/d7e/f97 x:0 0 0 2024-05-13T22:48:32.788 INFO:tasks.workunit.client.1.smithi130.stdout:5/448: rename d6/f75 to d6/d7/d32/f91 0 2024-05-13T22:48:32.789 INFO:tasks.workunit.client.1.smithi130.stdout:1/443: dread - db/d1b/d1f/d2a/d38/d46/f71 zero size 2024-05-13T22:48:32.789 INFO:tasks.workunit.client.1.smithi130.stdout:2/467: dread d4/fd [0,4194304] 0 2024-05-13T22:48:32.791 INFO:tasks.workunit.client.1.smithi130.stdout:9/517: dread d1/f3 [0,4194304] 0 2024-05-13T22:48:32.792 INFO:tasks.workunit.client.1.smithi130.stdout:3/462: dwrite d8/d1a/d55/f9b [0,4194304] 0 2024-05-13T22:48:32.792 INFO:tasks.workunit.client.1.smithi130.stdout:6/479: rename d4/d15/d1f/d46/d50/f59 to d4/d15/d1d/d28/d75/fa6 0 2024-05-13T22:48:32.793 INFO:tasks.workunit.client.1.smithi130.stdout:7/447: rename d7/da to d7/da/d86/d96 22 2024-05-13T22:48:32.794 INFO:tasks.workunit.client.1.smithi130.stdout:8/422: creat d1/d2/db/d15/d16/d54/d58/f98 x:0 0 0 2024-05-13T22:48:32.796 INFO:tasks.workunit.client.1.smithi130.stdout:8/423: write d1/d2/db/f45 [2224086,1709] 0 2024-05-13T22:48:32.798 INFO:tasks.workunit.client.1.smithi130.stdout:1/444: dread db/d1b/d1f/d49/f61 [0,4194304] 0 2024-05-13T22:48:32.804 INFO:tasks.workunit.client.1.smithi130.stdout:6/480: creat d4/d15/d1f/d46/d50/fa7 x:0 0 0 2024-05-13T22:48:32.807 INFO:tasks.workunit.client.1.smithi130.stdout:4/475: sync 2024-05-13T22:48:32.810 INFO:tasks.workunit.client.1.smithi130.stdout:8/424: symlink d1/dc/d5d/l99 0 2024-05-13T22:48:32.812 INFO:tasks.workunit.client.1.smithi130.stdout:7/448: dwrite d7/da/d1d/d3d/d46/f4b [0,4194304] 0 2024-05-13T22:48:32.814 INFO:tasks.workunit.client.1.smithi130.stdout:7/449: write d7/f6a [2616885,85935] 0 2024-05-13T22:48:32.815 INFO:tasks.workunit.client.1.smithi130.stdout:5/449: dwrite d6/d7/f28 [0,4194304] 0 2024-05-13T22:48:32.818 INFO:tasks.workunit.client.1.smithi130.stdout:3/463: symlink d8/dc/d19/la2 0 2024-05-13T22:48:32.822 INFO:tasks.workunit.client.1.smithi130.stdout:6/481: rename d4/d15/d8b to d4/d15/d1d/d28/d75/da8 0 2024-05-13T22:48:32.823 INFO:tasks.workunit.client.1.smithi130.stdout:5/450: dwrite d6/dd/d18/f36 [0,4194304] 0 2024-05-13T22:48:32.824 INFO:tasks.workunit.client.1.smithi130.stdout:0/500: sync 2024-05-13T22:48:32.825 INFO:tasks.workunit.client.1.smithi130.stdout:2/468: dwrite d4/d49/d42/f53 [0,4194304] 0 2024-05-13T22:48:32.829 INFO:tasks.workunit.client.1.smithi130.stdout:2/469: dread - d4/db/f2e zero size 2024-05-13T22:48:32.829 INFO:tasks.workunit.client.1.smithi130.stdout:9/518: link d1/f11 d1/d7/d23/d28/d2d/d2f/d63/d6b/fa4 0 2024-05-13T22:48:32.830 INFO:tasks.workunit.client.1.smithi130.stdout:9/519: truncate d1/d15/d51/f7f 248894 0 2024-05-13T22:48:32.831 INFO:tasks.workunit.client.1.smithi130.stdout:8/425: rmdir d1 39 2024-05-13T22:48:32.832 INFO:tasks.workunit.client.1.smithi130.stdout:9/520: read - d1/d15/d51/f8e zero size 2024-05-13T22:48:32.841 INFO:tasks.workunit.client.1.smithi130.stdout:7/450: rmdir d7 39 2024-05-13T22:48:32.845 INFO:tasks.workunit.client.1.smithi130.stdout:3/464: mkdir d8/dc/d7b/da3 0 2024-05-13T22:48:32.846 INFO:tasks.workunit.client.1.smithi130.stdout:4/476: mknod d8/dc/d80/d85/c90 0 2024-05-13T22:48:32.849 INFO:tasks.workunit.client.1.smithi130.stdout:6/482: symlink d4/d15/d1f/d46/la9 0 2024-05-13T22:48:32.851 INFO:tasks.workunit.client.1.smithi130.stdout:9/521: unlink d1/d7/d23/d28/d32/c59 0 2024-05-13T22:48:32.853 INFO:tasks.workunit.client.1.smithi130.stdout:4/477: dwrite d8/dc/d35/f8a [0,4194304] 0 2024-05-13T22:48:32.854 INFO:tasks.workunit.client.1.smithi130.stdout:0/501: fdatasync d0/f1e 0 2024-05-13T22:48:32.856 INFO:tasks.workunit.client.1.smithi130.stdout:0/502: chown d0/d7/dd/d17/d52/d5b/d60 43337 1 2024-05-13T22:48:32.856 INFO:tasks.workunit.client.1.smithi130.stdout:2/470: symlink d4/db/dc/d18/la1 0 2024-05-13T22:48:32.859 INFO:tasks.workunit.client.1.smithi130.stdout:8/426: dwrite d1/d2/db/d15/d18/d42/d53/d56/f7c [0,4194304] 0 2024-05-13T22:48:32.864 INFO:tasks.workunit.client.1.smithi130.stdout:1/445: truncate db/f1a 2734409 0 2024-05-13T22:48:32.864 INFO:tasks.workunit.client.1.smithi130.stdout:4/478: fsync f7 0 2024-05-13T22:48:32.867 INFO:tasks.workunit.client.1.smithi130.stdout:9/522: fdatasync d1/d7/d23/d28/d2d/f46 0 2024-05-13T22:48:32.867 INFO:tasks.workunit.client.1.smithi130.stdout:2/471: dwrite d4/d56/f68 [0,4194304] 0 2024-05-13T22:48:32.870 INFO:tasks.workunit.client.1.smithi130.stdout:5/451: dread d6/dd/d18/fc [0,4194304] 0 2024-05-13T22:48:32.873 INFO:tasks.workunit.client.1.smithi130.stdout:7/451: rename d7/d8/d9/d66/f38 to d7/d8/d5b/f97 0 2024-05-13T22:48:32.874 INFO:tasks.workunit.client.1.smithi130.stdout:8/427: dwrite d1/d2/db/d15/d7e/f95 [0,4194304] 0 2024-05-13T22:48:32.881 INFO:tasks.workunit.client.1.smithi130.stdout:1/446: creat db/d4a/f94 x:0 0 0 2024-05-13T22:48:32.893 INFO:tasks.workunit.client.1.smithi130.stdout:9/523: dwrite d1/db/f83 [0,4194304] 0 2024-05-13T22:48:32.896 INFO:tasks.workunit.client.1.smithi130.stdout:5/452: mkdir d6/d1a/d92 0 2024-05-13T22:48:32.899 INFO:tasks.workunit.client.1.smithi130.stdout:4/479: rename d8/dc/d80/f7d to d8/df/d81/f91 0 2024-05-13T22:48:32.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:32 smithi130 ceph-mon[36591]: pgmap v33: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 43 MiB/s rd, 164 MiB/s wr, 325 op/s 2024-05-13T22:48:32.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:32 smithi130 ceph-mon[36591]: Upgrade: Updating mgr.smithi102.xcijrj 2024-05-13T22:48:32.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:32 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:32.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi102.xcijrj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:48:32.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:32 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi102.xcijrj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:48:32.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:48:32.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:32 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:48:32.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:32 smithi130 ceph-mon[36591]: Deploying daemon mgr.smithi102.xcijrj on smithi102 2024-05-13T22:48:32.906 INFO:tasks.workunit.client.1.smithi130.stdout:9/524: creat d1/d7/d23/d28/d73/d8d/fa5 x:0 0 0 2024-05-13T22:48:32.906 INFO:tasks.workunit.client.1.smithi130.stdout:2/472: write d4/f5 [377290,25838] 0 2024-05-13T22:48:32.907 INFO:tasks.workunit.client.1.smithi130.stdout:5/453: symlink d6/d1a/l93 0 2024-05-13T22:48:32.908 INFO:tasks.workunit.client.1.smithi130.stdout:7/452: mknod d7/da/d1d/d42/c98 0 2024-05-13T22:48:32.911 INFO:tasks.workunit.client.1.smithi130.stdout:8/428: rename d1/d2/db/d15/d16/d54/d58/f7a to d1/d2/db/d15/d18/d42/d53/d56/f9a 0 2024-05-13T22:48:32.912 INFO:tasks.workunit.client.1.smithi130.stdout:6/483: rename d4/d15/d1d/d63 to d4/d15/d1d/daa 0 2024-05-13T22:48:32.913 INFO:tasks.workunit.client.1.smithi130.stdout:4/480: readlink d8/dc/l68 0 2024-05-13T22:48:32.916 INFO:tasks.workunit.client.1.smithi130.stdout:9/525: chown d1/d7/d23/d28/d32/d45/f53 68745 1 2024-05-13T22:48:32.918 INFO:tasks.workunit.client.1.smithi130.stdout:5/454: symlink d6/dd/d18/d6e/d76/d7f/l94 0 2024-05-13T22:48:32.919 INFO:tasks.workunit.client.1.smithi130.stdout:1/447: dwrite db/d4a/f4b [0,4194304] 0 2024-05-13T22:48:32.919 INFO:tasks.workunit.client.1.smithi130.stdout:7/453: write d7/d8/d39/f7c [1555666,57006] 0 2024-05-13T22:48:32.921 INFO:tasks.workunit.client.1.smithi130.stdout:6/484: symlink d4/d15/d1d/d28/lab 0 2024-05-13T22:48:32.923 INFO:tasks.workunit.client.1.smithi130.stdout:4/481: mknod d8/dc/d35/d33/d44/d61/c92 0 2024-05-13T22:48:32.926 INFO:tasks.workunit.client.1.smithi130.stdout:2/473: mknod d4/d49/d48/d4a/ca2 0 2024-05-13T22:48:32.927 INFO:tasks.workunit.client.1.smithi130.stdout:9/526: mknod d1/d7/d23/d28/d73/ca6 0 2024-05-13T22:48:32.928 INFO:tasks.workunit.client.1.smithi130.stdout:4/482: dwrite d8/f8b [0,4194304] 0 2024-05-13T22:48:32.929 INFO:tasks.workunit.client.1.smithi130.stdout:1/448: rmdir db/d1b/d1f/d2a/d38/d66 39 2024-05-13T22:48:32.930 INFO:tasks.workunit.client.1.smithi130.stdout:7/454: creat d7/d8/d9/d66/f99 x:0 0 0 2024-05-13T22:48:32.931 INFO:tasks.workunit.client.1.smithi130.stdout:8/429: mkdir d1/d9b 0 2024-05-13T22:48:32.932 INFO:tasks.workunit.client.1.smithi130.stdout:8/430: fdatasync d1/d2/db/d15/d18/f5f 0 2024-05-13T22:48:32.933 INFO:tasks.workunit.client.1.smithi130.stdout:4/483: read d8/dc/d80/f78 [271121,57113] 0 2024-05-13T22:48:32.934 INFO:tasks.workunit.client.1.smithi130.stdout:4/484: chown d8/dc/d35/d33 827217355 1 2024-05-13T22:48:32.934 INFO:tasks.workunit.client.1.smithi130.stdout:2/474: creat d4/d49/d42/d4b/fa3 x:0 0 0 2024-05-13T22:48:32.937 INFO:tasks.workunit.client.1.smithi130.stdout:5/455: mkdir d6/d7/d95 0 2024-05-13T22:48:32.938 INFO:tasks.workunit.client.1.smithi130.stdout:5/456: write d6/dd/d18/d6e/d82/d56/f5d [24914,103051] 0 2024-05-13T22:48:32.939 INFO:tasks.workunit.client.1.smithi130.stdout:1/449: creat db/d4a/f95 x:0 0 0 2024-05-13T22:48:32.939 INFO:tasks.workunit.client.1.smithi130.stdout:9/527: read d1/d7/f57 [1418979,117334] 0 2024-05-13T22:48:32.939 INFO:tasks.workunit.client.1.smithi130.stdout:3/465: sync 2024-05-13T22:48:32.941 INFO:tasks.workunit.client.1.smithi130.stdout:7/455: write d7/da/d1d/d42/f64 [2634612,76165] 0 2024-05-13T22:48:32.948 INFO:tasks.workunit.client.1.smithi130.stdout:6/485: dread d4/f12 [0,4194304] 0 2024-05-13T22:48:32.953 INFO:tasks.workunit.client.1.smithi130.stdout:3/466: dread f2 [0,4194304] 0 2024-05-13T22:48:32.954 INFO:tasks.workunit.client.1.smithi130.stdout:1/450: creat db/d1b/d1f/d49/d56/d7f/f96 x:0 0 0 2024-05-13T22:48:32.961 INFO:tasks.workunit.client.1.smithi130.stdout:1/451: dwrite db/d1b/d1f/d49/f54 [0,4194304] 0 2024-05-13T22:48:32.966 INFO:tasks.workunit.client.1.smithi130.stdout:7/456: dread d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:32.967 INFO:tasks.workunit.client.1.smithi130.stdout:4/485: creat d8/dc/d35/d33/d3f/d57/f93 x:0 0 0 2024-05-13T22:48:32.973 INFO:tasks.workunit.client.1.smithi130.stdout:4/486: dwrite d8/dc/f5b [0,4194304] 0 2024-05-13T22:48:32.975 INFO:tasks.workunit.client.1.smithi130.stdout:1/452: unlink db/d1b/c28 0 2024-05-13T22:48:32.977 INFO:tasks.workunit.client.1.smithi130.stdout:8/431: dread d1/f33 [0,4194304] 0 2024-05-13T22:48:32.978 INFO:tasks.workunit.client.1.smithi130.stdout:2/475: rename d4/db/dc/d18/d2a/d30/d69/f87 to d4/fa4 0 2024-05-13T22:48:32.981 INFO:tasks.workunit.client.1.smithi130.stdout:8/432: stat d1/d2/db/d15/d16/d54/d58 0 2024-05-13T22:48:32.981 INFO:tasks.workunit.client.1.smithi130.stdout:2/476: stat d4/db/dc/d18/d2a/f2f 0 2024-05-13T22:48:32.982 INFO:tasks.workunit.client.1.smithi130.stdout:6/486: mkdir d4/d15/d1d/d60/da2/da4/dac 0 2024-05-13T22:48:32.985 INFO:tasks.workunit.client.1.smithi130.stdout:4/487: mkdir d8/dc/d35/d33/d44/d77/d94 0 2024-05-13T22:48:32.985 INFO:tasks.workunit.client.1.smithi130.stdout:9/528: dwrite d1/d7/d23/d28/d2d/f48 [4194304,4194304] 0 2024-05-13T22:48:32.986 INFO:tasks.workunit.client.1.smithi130.stdout:6/487: dread - d4/d15/d1f/d46/d50/f8a zero size 2024-05-13T22:48:32.987 INFO:tasks.workunit.client.1.smithi130.stdout:8/433: truncate d1/d2/db/f45 5088643 0 2024-05-13T22:48:32.988 INFO:tasks.workunit.client.1.smithi130.stdout:1/453: dwrite db/d1b/d1f/d49/f48 [4194304,4194304] 0 2024-05-13T22:48:32.988 INFO:tasks.workunit.client.1.smithi130.stdout:4/488: readlink d8/dc/d35/d33/d44/l5d 0 2024-05-13T22:48:32.991 INFO:tasks.workunit.client.1.smithi130.stdout:8/434: dread - d1/d2/db/d15/d16/d54/d58/f94 zero size 2024-05-13T22:48:32.993 INFO:tasks.workunit.client.1.smithi130.stdout:2/477: dwrite d4/db/f2c [0,4194304] 0 2024-05-13T22:48:32.994 INFO:tasks.workunit.client.1.smithi130.stdout:2/478: fsync d4/db/dc/f3b 0 2024-05-13T22:48:32.997 INFO:tasks.workunit.client.1.smithi130.stdout:7/457: creat d7/da/d1d/f9a x:0 0 0 2024-05-13T22:48:32.999 INFO:tasks.workunit.client.1.smithi130.stdout:8/435: dwrite d1/d2/db/f45 [0,4194304] 0 2024-05-13T22:48:33.002 INFO:tasks.workunit.client.1.smithi130.stdout:3/467: getdents d8 0 2024-05-13T22:48:33.003 INFO:tasks.workunit.client.1.smithi130.stdout:6/488: symlink d4/d15/d1d/d28/d75/d7d/lad 0 2024-05-13T22:48:33.009 INFO:tasks.workunit.client.1.smithi130.stdout:9/529: rename d1/f5f to d1/d7/d23/d28/d32/d65/fa7 0 2024-05-13T22:48:33.009 INFO:tasks.workunit.client.1.smithi130.stdout:6/489: dwrite d4/d15/d1d/d28/f30 [0,4194304] 0 2024-05-13T22:48:33.022 INFO:tasks.workunit.client.1.smithi130.stdout:8/436: fsync d1/dc/d5d/f87 0 2024-05-13T22:48:33.030 INFO:tasks.workunit.client.1.smithi130.stdout:1/454: truncate db/d1b/d1f/d2a/d38/f60 2833603 0 2024-05-13T22:48:33.031 INFO:tasks.workunit.client.1.smithi130.stdout:1/455: stat db/d1b/d1f/d2a/d35/f40 0 2024-05-13T22:48:33.032 INFO:tasks.workunit.client.1.smithi130.stdout:7/458: mknod d7/d8/d9/c9b 0 2024-05-13T22:48:33.032 INFO:tasks.workunit.client.1.smithi130.stdout:7/459: stat d7/da/d1d/d42 0 2024-05-13T22:48:33.034 INFO:tasks.workunit.client.1.smithi130.stdout:8/437: creat d1/d2/db/d15/d7e/f9c x:0 0 0 2024-05-13T22:48:33.035 INFO:tasks.workunit.client.1.smithi130.stdout:6/490: unlink d4/c4c 0 2024-05-13T22:48:33.037 INFO:tasks.workunit.client.1.smithi130.stdout:7/460: dwrite d7/da/d1d/d42/d43/f4d [0,4194304] 0 2024-05-13T22:48:33.041 INFO:tasks.workunit.client.1.smithi130.stdout:2/479: creat d4/fa5 x:0 0 0 2024-05-13T22:48:33.041 INFO:tasks.workunit.client.1.smithi130.stdout:0/503: sync 2024-05-13T22:48:33.046 INFO:tasks.workunit.client.1.smithi130.stdout:3/468: link d8/d1a/d55/l5a d8/dc/d42/la4 0 2024-05-13T22:48:33.050 INFO:tasks.workunit.client.1.smithi130.stdout:6/491: creat d4/d15/d1d/d28/d75/da8/da0/fae x:0 0 0 2024-05-13T22:48:33.052 INFO:tasks.workunit.client.1.smithi130.stdout:7/461: symlink d7/da/d86/l9c 0 2024-05-13T22:48:33.053 INFO:tasks.workunit.client.1.smithi130.stdout:1/456: symlink db/d1b/d1f/d2a/l97 0 2024-05-13T22:48:33.053 INFO:tasks.workunit.client.1.smithi130.stdout:7/462: fsync d7/da/d1d/d42/d43/f8e 0 2024-05-13T22:48:33.055 INFO:tasks.workunit.client.1.smithi130.stdout:0/504: chown d0/d7/dd/d17/d52/d5b/d5f/l97 11 1 2024-05-13T22:48:33.056 INFO:tasks.workunit.client.1.smithi130.stdout:2/480: rename d4/d49/d48/c8d to d4/db/dc/d18/d2a/d30/d69/ca6 0 2024-05-13T22:48:33.056 INFO:tasks.workunit.client.1.smithi130.stdout:4/489: dwrite d8/df/f59 [0,4194304] 0 2024-05-13T22:48:33.058 INFO:tasks.workunit.client.1.smithi130.stdout:6/492: creat d4/d15/d1f/faf x:0 0 0 2024-05-13T22:48:33.059 INFO:tasks.workunit.client.1.smithi130.stdout:7/463: symlink d7/d8/d5b/l9d 0 2024-05-13T22:48:33.060 INFO:tasks.workunit.client.1.smithi130.stdout:9/530: truncate d1/d7/d23/d28/d32/d65/fa7 2163355 0 2024-05-13T22:48:33.061 INFO:tasks.workunit.client.1.smithi130.stdout:4/490: dread d8/df/f40 [0,4194304] 0 2024-05-13T22:48:33.061 INFO:tasks.workunit.client.1.smithi130.stdout:2/481: write d4/fa4 [1318195,82467] 0 2024-05-13T22:48:33.061 INFO:tasks.workunit.client.1.smithi130.stdout:3/469: rename d8/d1a/d1c/c6a to d8/dc/d15/ca5 0 2024-05-13T22:48:33.063 INFO:tasks.workunit.client.1.smithi130.stdout:7/464: read d7/d8/f1c [3179018,65858] 0 2024-05-13T22:48:33.064 INFO:tasks.workunit.client.1.smithi130.stdout:3/470: dread - d8/d35/f80 zero size 2024-05-13T22:48:33.064 INFO:tasks.workunit.client.1.smithi130.stdout:2/482: chown d4/d49/f35 1095 1 2024-05-13T22:48:33.066 INFO:tasks.workunit.client.1.smithi130.stdout:8/438: dread d1/d2/db/d15/d18/f76 [0,4194304] 0 2024-05-13T22:48:33.066 INFO:tasks.workunit.client.1.smithi130.stdout:7/465: chown d7/da/d1d/d3d/d46/d67/d88 123634005 1 2024-05-13T22:48:33.066 INFO:tasks.workunit.client.1.smithi130.stdout:0/505: mknod d0/d7/dd/d17/d1f/d2c/d45/d57/cb2 0 2024-05-13T22:48:33.069 INFO:tasks.workunit.client.1.smithi130.stdout:2/483: truncate d4/d56/f8e 1006258 0 2024-05-13T22:48:33.072 INFO:tasks.workunit.client.1.smithi130.stdout:2/484: write d4/f7 [20769,129513] 0 2024-05-13T22:48:33.078 INFO:tasks.workunit.client.1.smithi130.stdout:3/471: dwrite d8/d1a/f89 [0,4194304] 0 2024-05-13T22:48:33.079 INFO:tasks.workunit.client.1.smithi130.stdout:9/531: creat d1/d7/d69/d79/fa8 x:0 0 0 2024-05-13T22:48:33.082 INFO:tasks.workunit.client.1.smithi130.stdout:4/491: dread d8/df/f3c [0,4194304] 0 2024-05-13T22:48:33.083 INFO:tasks.workunit.client.1.smithi130.stdout:7/466: creat d7/d8/d9/d66/d65/f9e x:0 0 0 2024-05-13T22:48:33.083 INFO:tasks.workunit.client.1.smithi130.stdout:9/532: chown d1/d7/d23/d28/d2d/f74 4157 1 2024-05-13T22:48:33.084 INFO:tasks.workunit.client.1.smithi130.stdout:5/457: sync 2024-05-13T22:48:33.085 INFO:tasks.workunit.client.1.smithi130.stdout:4/492: write d8/df/f40 [1146040,70642] 0 2024-05-13T22:48:33.087 INFO:tasks.workunit.client.1.smithi130.stdout:0/506: mknod d0/d7/dd/d17/d1f/d2c/d45/cb3 0 2024-05-13T22:48:33.087 INFO:tasks.workunit.client.1.smithi130.stdout:2/485: dwrite d4/d49/d48/d4a/d76/f9b [0,4194304] 0 2024-05-13T22:48:33.088 INFO:tasks.workunit.client.1.smithi130.stdout:9/533: truncate d1/d15/d51/f6d 962763 0 2024-05-13T22:48:33.090 INFO:tasks.workunit.client.1.smithi130.stdout:6/493: dread d4/d16/f2a [0,4194304] 0 2024-05-13T22:48:33.090 INFO:tasks.workunit.client.1.smithi130.stdout:8/439: creat d1/d2/db/d15/d18/d1a/f9d x:0 0 0 2024-05-13T22:48:33.099 INFO:tasks.workunit.client.1.smithi130.stdout:7/467: write d7/da/d1d/d42/f58 [6768314,130489] 0 2024-05-13T22:48:33.099 INFO:tasks.workunit.client.1.smithi130.stdout:7/468: readlink d7/d8/l1b 0 2024-05-13T22:48:33.102 INFO:tasks.workunit.client.1.smithi130.stdout:0/507: unlink d0/d7/d29/c3b 0 2024-05-13T22:48:33.103 INFO:tasks.workunit.client.1.smithi130.stdout:0/508: stat d0/d7/dd/d17/c2a 0 2024-05-13T22:48:33.103 INFO:tasks.workunit.client.1.smithi130.stdout:2/486: mknod d4/db/d26/d9e/ca7 0 2024-05-13T22:48:33.104 INFO:tasks.workunit.client.1.smithi130.stdout:0/509: write d0/d7/d29/f40 [635516,82746] 0 2024-05-13T22:48:33.104 INFO:tasks.workunit.client.1.smithi130.stdout:0/510: readlink d0/d7/dd/d17/l51 0 2024-05-13T22:48:33.105 INFO:tasks.workunit.client.1.smithi130.stdout:0/511: write d0/d7/dd/d2e/f4e [1083453,90608] 0 2024-05-13T22:48:33.108 INFO:tasks.workunit.client.1.smithi130.stdout:1/457: rename db/l36 to db/d1b/d1f/d2a/d38/l98 0 2024-05-13T22:48:33.108 INFO:tasks.workunit.client.1.smithi130.stdout:6/494: rmdir d4/d15/d1d/d28/d75/d7d 39 2024-05-13T22:48:33.113 INFO:tasks.workunit.client.1.smithi130.stdout:5/458: creat d6/d1a/d92/f96 x:0 0 0 2024-05-13T22:48:33.114 INFO:tasks.workunit.client.1.smithi130.stdout:5/459: readlink d6/dd/l6d 0 2024-05-13T22:48:33.127 INFO:tasks.workunit.client.1.smithi130.stdout:4/493: truncate d8/df/f3c 5152808 0 2024-05-13T22:48:33.128 INFO:tasks.workunit.client.1.smithi130.stdout:5/460: symlink d6/dd/l97 0 2024-05-13T22:48:33.129 INFO:tasks.workunit.client.1.smithi130.stdout:9/534: truncate d1/d15/d51/f54 1092449 0 2024-05-13T22:48:33.130 INFO:tasks.workunit.client.1.smithi130.stdout:2/487: mkdir d4/da8 0 2024-05-13T22:48:33.130 INFO:tasks.workunit.client.1.smithi130.stdout:9/535: dread - d1/d15/d51/f8e zero size 2024-05-13T22:48:33.131 INFO:tasks.workunit.client.1.smithi130.stdout:7/469: dwrite d7/d8/f19 [0,4194304] 0 2024-05-13T22:48:33.131 INFO:tasks.workunit.client.1.smithi130.stdout:0/512: rename d0/d7/dd/d69/da9/d8d/d9f/ca1 to d0/d7/dd/d17/d52/d5b/cb4 0 2024-05-13T22:48:33.137 INFO:tasks.workunit.client.1.smithi130.stdout:3/472: link d8/d1a/f4f d8/dc/d19/d2c/fa6 0 2024-05-13T22:48:33.145 INFO:tasks.workunit.client.1.smithi130.stdout:1/458: write db/d1b/d1f/d2a/f2f [2317166,30797] 0 2024-05-13T22:48:33.145 INFO:tasks.workunit.client.1.smithi130.stdout:8/440: getdents d1 0 2024-05-13T22:48:33.147 INFO:tasks.workunit.client.1.smithi130.stdout:8/441: write d1/d2/db/d15/d18/f60 [1380550,83245] 0 2024-05-13T22:48:33.149 INFO:tasks.workunit.client.1.smithi130.stdout:2/488: creat d4/db/dc/d18/d2a/d66/fa9 x:0 0 0 2024-05-13T22:48:33.151 INFO:tasks.workunit.client.1.smithi130.stdout:4/494: write d8/dc/f28 [1810851,45001] 0 2024-05-13T22:48:33.152 INFO:tasks.workunit.client.1.smithi130.stdout:8/442: dread d1/d2/db/d15/d7e/f95 [0,4194304] 0 2024-05-13T22:48:33.152 INFO:tasks.workunit.client.1.smithi130.stdout:9/536: dread d1/d15/f3b [0,4194304] 0 2024-05-13T22:48:33.154 INFO:tasks.workunit.client.1.smithi130.stdout:8/443: read - d1/d2/db/d15/d18/d1a/f9d zero size 2024-05-13T22:48:33.154 INFO:tasks.workunit.client.1.smithi130.stdout:3/473: symlink d8/dc/d15/la7 0 2024-05-13T22:48:33.157 INFO:tasks.workunit.client.1.smithi130.stdout:1/459: symlink db/d1b/d1f/d2a/d38/d46/d6d/l99 0 2024-05-13T22:48:33.158 INFO:tasks.workunit.client.1.smithi130.stdout:7/470: write d7/da/d1d/d5d/d70/f8d [1504781,34403] 0 2024-05-13T22:48:33.161 INFO:tasks.workunit.client.1.smithi130.stdout:0/513: mknod d0/d7/dd/d69/dac/cb5 0 2024-05-13T22:48:33.164 INFO:tasks.workunit.client.1.smithi130.stdout:2/489: creat d4/db/dc/d18/d2a/d30/d88/faa x:0 0 0 2024-05-13T22:48:33.165 INFO:tasks.workunit.client.1.smithi130.stdout:9/537: dread d1/d15/f1d [0,4194304] 0 2024-05-13T22:48:33.166 INFO:tasks.workunit.client.1.smithi130.stdout:7/471: dwrite d7/da/d1d/d42/d43/f8e [0,4194304] 0 2024-05-13T22:48:33.171 INFO:tasks.workunit.client.1.smithi130.stdout:1/460: rename db/d82/f86 to db/d1b/d1f/d49/d56/d88/f9a 0 2024-05-13T22:48:33.174 INFO:tasks.workunit.client.1.smithi130.stdout:0/514: creat d0/d7/dd/d17/d1f/d2c/d45/d57/fb6 x:0 0 0 2024-05-13T22:48:33.175 INFO:tasks.workunit.client.1.smithi130.stdout:0/515: fsync d0/d7/d29/f40 0 2024-05-13T22:48:33.176 INFO:tasks.workunit.client.1.smithi130.stdout:4/495: unlink d8/dc/c27 0 2024-05-13T22:48:33.178 INFO:tasks.workunit.client.1.smithi130.stdout:2/490: unlink d4/db/dc/d18/d2a/d30/d88/l96 0 2024-05-13T22:48:33.180 INFO:tasks.workunit.client.1.smithi130.stdout:9/538: read d1/d7/d23/d28/d32/d65/fa7 [519758,14958] 0 2024-05-13T22:48:33.182 INFO:tasks.workunit.client.1.smithi130.stdout:5/461: getdents d6/dd/d18/d6e/d82/d4b 0 2024-05-13T22:48:33.184 INFO:tasks.workunit.client.1.smithi130.stdout:5/462: read d6/dd/f13 [813125,79039] 0 2024-05-13T22:48:33.186 INFO:tasks.workunit.client.1.smithi130.stdout:7/472: dwrite d7/da/d1d/d3d/d46/d67/f30 [0,4194304] 0 2024-05-13T22:48:33.188 INFO:tasks.workunit.client.1.smithi130.stdout:0/516: unlink d0/d7/dd/d17/d1f/d2c/d45/d57/f92 0 2024-05-13T22:48:33.189 INFO:tasks.workunit.client.1.smithi130.stdout:0/517: chown d0/d7/dd/d17/c35 29 1 2024-05-13T22:48:33.189 INFO:tasks.workunit.client.1.smithi130.stdout:2/491: creat d4/db/dc/d18/d2a/fab x:0 0 0 2024-05-13T22:48:33.190 INFO:tasks.workunit.client.1.smithi130.stdout:9/539: creat d1/d7/d69/d79/fa9 x:0 0 0 2024-05-13T22:48:33.191 INFO:tasks.workunit.client.1.smithi130.stdout:8/444: link d1/d2/db/d15/d18/l50 d1/d9b/l9e 0 2024-05-13T22:48:33.191 INFO:tasks.workunit.client.1.smithi130.stdout:0/518: stat d0/d7/dd/d17/d52/d67/c90 0 2024-05-13T22:48:33.192 INFO:tasks.workunit.client.1.smithi130.stdout:3/474: link d8/d1a/d55/l7d d8/dc/d15/d8f/la8 0 2024-05-13T22:48:33.192 INFO:tasks.workunit.client.1.smithi130.stdout:8/445: chown d1/c32 96 1 2024-05-13T22:48:33.193 INFO:tasks.workunit.client.1.smithi130.stdout:3/475: write d8/dc/d42/f8c [4369344,52658] 0 2024-05-13T22:48:33.194 INFO:tasks.workunit.client.1.smithi130.stdout:1/461: symlink db/d58/d74/l9b 0 2024-05-13T22:48:33.195 INFO:tasks.workunit.client.1.smithi130.stdout:5/463: creat d6/d1a/d92/f98 x:0 0 0 2024-05-13T22:48:33.203 INFO:tasks.workunit.client.1.smithi130.stdout:9/540: fdatasync d1/d15/f1d 0 2024-05-13T22:48:33.204 INFO:tasks.workunit.client.1.smithi130.stdout:9/541: chown d1/d7/d23 16 1 2024-05-13T22:48:33.204 INFO:tasks.workunit.client.1.smithi130.stdout:9/542: chown d1/d15/l24 2327 1 2024-05-13T22:48:33.206 INFO:tasks.workunit.client.1.smithi130.stdout:2/492: mknod d4/db/d26/d9e/cac 0 2024-05-13T22:48:33.206 INFO:tasks.workunit.client.1.smithi130.stdout:2/493: write d4/f5 [5743438,21570] 0 2024-05-13T22:48:33.211 INFO:tasks.workunit.client.1.smithi130.stdout:5/464: creat d6/dd/d18/d6e/d82/d4b/f99 x:0 0 0 2024-05-13T22:48:33.211 INFO:tasks.workunit.client.1.smithi130.stdout:6/495: truncate d4/d15/d1f/d46/d52/f87 1744671 0 2024-05-13T22:48:33.212 INFO:tasks.workunit.client.1.smithi130.stdout:8/446: dread d1/d2/db/d15/f74 [0,4194304] 0 2024-05-13T22:48:33.212 INFO:tasks.workunit.client.1.smithi130.stdout:6/496: chown d4/d16/c6c 399207699 1 2024-05-13T22:48:33.214 INFO:tasks.workunit.client.1.smithi130.stdout:7/473: symlink d7/l9f 0 2024-05-13T22:48:33.221 INFO:tasks.workunit.client.1.smithi130.stdout:9/543: unlink d1/d7/d23/d28/d2d/d2f/f3f 0 2024-05-13T22:48:33.226 INFO:tasks.workunit.client.1.smithi130.stdout:2/494: creat d4/db/d26/d9e/fad x:0 0 0 2024-05-13T22:48:33.232 INFO:tasks.workunit.client.1.smithi130.stdout:5/465: creat d6/d7/d19/f9a x:0 0 0 2024-05-13T22:48:33.234 INFO:tasks.workunit.client.1.smithi130.stdout:6/497: readlink d4/d15/d1f/l3b 0 2024-05-13T22:48:33.236 INFO:tasks.workunit.client.1.smithi130.stdout:8/447: mkdir d1/d2/db/d15/d7e/d9f 0 2024-05-13T22:48:33.238 INFO:tasks.workunit.client.1.smithi130.stdout:8/448: chown d1/d2/db/d15/d18/d42/d53/d56/f7c 2635701 1 2024-05-13T22:48:33.239 INFO:tasks.workunit.client.1.smithi130.stdout:4/496: creat d8/dc/d35/d33/d3f/f95 x:0 0 0 2024-05-13T22:48:33.240 INFO:tasks.workunit.client.1.smithi130.stdout:0/519: dwrite d0/d7/dd/f46 [0,4194304] 0 2024-05-13T22:48:33.240 INFO:tasks.workunit.client.1.smithi130.stdout:7/474: unlink d7/d8/d9/d66/f63 0 2024-05-13T22:48:33.242 INFO:tasks.workunit.client.1.smithi130.stdout:0/520: chown d0/d7/dd/d2e/d4b/f15 0 1 2024-05-13T22:48:33.242 INFO:tasks.workunit.client.1.smithi130.stdout:9/544: mknod d1/d7/d23/caa 0 2024-05-13T22:48:33.245 INFO:tasks.workunit.client.1.smithi130.stdout:5/466: creat d6/dd/d79/f9b x:0 0 0 2024-05-13T22:48:33.246 INFO:tasks.workunit.client.1.smithi130.stdout:0/521: fsync d0/d7/dd/d17/d52/f80 0 2024-05-13T22:48:33.247 INFO:tasks.workunit.client.1.smithi130.stdout:6/498: chown d4/d15/d1f/d64 18591215 1 2024-05-13T22:48:33.251 INFO:tasks.workunit.client.1.smithi130.stdout:4/497: dwrite d8/df/f40 [0,4194304] 0 2024-05-13T22:48:33.253 INFO:tasks.workunit.client.1.smithi130.stdout:7/475: dwrite d7/da/d1d/d3d/d46/d67/f30 [4194304,4194304] 0 2024-05-13T22:48:33.254 INFO:tasks.workunit.client.1.smithi130.stdout:4/498: chown d8/dc/d35/d33/d44/f54 0 1 2024-05-13T22:48:33.255 INFO:tasks.workunit.client.1.smithi130.stdout:8/449: creat d1/d2/db/d15/d7e/d9f/fa0 x:0 0 0 2024-05-13T22:48:33.259 INFO:tasks.workunit.client.1.smithi130.stdout:7/476: fdatasync d7/da/d1d/d3d/d46/f4f 0 2024-05-13T22:48:33.260 INFO:tasks.workunit.client.1.smithi130.stdout:5/467: creat d6/d7/d19/d4d/d6a/f9c x:0 0 0 2024-05-13T22:48:33.261 INFO:tasks.workunit.client.1.smithi130.stdout:8/450: readlink d1/d2/db/d15/d18/l2e 0 2024-05-13T22:48:33.262 INFO:tasks.workunit.client.1.smithi130.stdout:7/477: read - d7/f83 zero size 2024-05-13T22:48:33.262 INFO:tasks.workunit.client.1.smithi130.stdout:6/499: dwrite d4/d15/d1f/d46/d50/fa7 [0,4194304] 0 2024-05-13T22:48:33.264 INFO:tasks.workunit.client.1.smithi130.stdout:7/478: dread - d7/d8/d39/f79 zero size 2024-05-13T22:48:33.264 INFO:tasks.workunit.client.1.smithi130.stdout:5/468: dread d6/d7/d32/f51 [0,4194304] 0 2024-05-13T22:48:33.265 INFO:tasks.workunit.client.1.smithi130.stdout:6/500: fsync d4/d15/d1f/d46/d50/f51 0 2024-05-13T22:48:33.270 INFO:tasks.workunit.client.1.smithi130.stdout:4/499: mkdir d8/dc/d80/d96 0 2024-05-13T22:48:33.271 INFO:tasks.workunit.client.1.smithi130.stdout:4/500: chown d8/dc/d35/d33/d44/d77/l8f 3 1 2024-05-13T22:48:33.273 INFO:tasks.workunit.client.1.smithi130.stdout:0/522: getdents d0/d7/dd/d2e/d9e/da7 0 2024-05-13T22:48:33.275 INFO:tasks.workunit.client.1.smithi130.stdout:9/545: link d1/d7/d69/d79/f80 d1/d7/d23/d28/fab 0 2024-05-13T22:48:33.276 INFO:tasks.workunit.client.1.smithi130.stdout:9/546: write d1/f1e [1265553,57529] 0 2024-05-13T22:48:33.278 INFO:tasks.workunit.client.1.smithi130.stdout:8/451: creat d1/d2/db/d15/d7e/fa1 x:0 0 0 2024-05-13T22:48:33.279 INFO:tasks.workunit.client.1.smithi130.stdout:7/479: mknod d7/d8/d5b/ca0 0 2024-05-13T22:48:33.280 INFO:tasks.workunit.client.1.smithi130.stdout:8/452: write d1/d2/db/d15/d18/f4c [689827,81184] 0 2024-05-13T22:48:33.281 INFO:tasks.workunit.client.1.smithi130.stdout:7/480: fsync d7/d8/d9/d66/d65/f91 0 2024-05-13T22:48:33.283 INFO:tasks.workunit.client.1.smithi130.stdout:5/469: mknod d6/dd/d18/d6e/d82/d4b/c9d 0 2024-05-13T22:48:33.284 INFO:tasks.workunit.client.1.smithi130.stdout:5/470: read d6/d2f/f59 [178429,92850] 0 2024-05-13T22:48:33.289 INFO:tasks.workunit.client.1.smithi130.stdout:9/547: rmdir d1/d7/d23/d28/d2d 39 2024-05-13T22:48:33.290 INFO:tasks.workunit.client.1.smithi130.stdout:0/523: dread d0/d7/dd/d17/d52/d67/d68/d96/f83 [0,4194304] 0 2024-05-13T22:48:33.291 INFO:tasks.workunit.client.1.smithi130.stdout:0/524: chown d0/d7/dd/d17/d1f/d2c/f7a 137 1 2024-05-13T22:48:33.291 INFO:tasks.workunit.client.1.smithi130.stdout:0/525: stat d0/d7/fb 0 2024-05-13T22:48:33.298 INFO:tasks.workunit.client.1.smithi130.stdout:7/481: symlink d7/d8/d9/d66/la1 0 2024-05-13T22:48:33.298 INFO:tasks.workunit.client.1.smithi130.stdout:7/482: fsync d7/d8/d39/f79 0 2024-05-13T22:48:33.299 INFO:tasks.workunit.client.1.smithi130.stdout:6/501: creat d4/d15/d1d/d28/d75/d7d/fb0 x:0 0 0 2024-05-13T22:48:33.299 INFO:tasks.workunit.client.1.smithi130.stdout:5/471: creat d6/d1a/f9e x:0 0 0 2024-05-13T22:48:33.300 INFO:tasks.workunit.client.1.smithi130.stdout:6/502: dread - d4/d15/d1d/d28/d75/f80 zero size 2024-05-13T22:48:33.301 INFO:tasks.workunit.client.1.smithi130.stdout:6/503: write d4/d15/d1d/fa5 [190072,26942] 0 2024-05-13T22:48:33.301 INFO:tasks.workunit.client.1.smithi130.stdout:0/526: chown d0/c1 486980 1 2024-05-13T22:48:33.303 INFO:tasks.workunit.client.1.smithi130.stdout:8/453: truncate d1/d2/f8c 677987 0 2024-05-13T22:48:33.305 INFO:tasks.workunit.client.1.smithi130.stdout:7/483: mknod d7/d8/d39/ca2 0 2024-05-13T22:48:33.309 INFO:tasks.workunit.client.1.smithi130.stdout:3/476: sync 2024-05-13T22:48:33.310 INFO:tasks.workunit.client.1.smithi130.stdout:3/477: write d8/dc/d19/d2c/d61/f69 [3109779,105273] 0 2024-05-13T22:48:33.317 INFO:tasks.workunit.client.1.smithi130.stdout:4/501: getdents d8/dc 0 2024-05-13T22:48:33.318 INFO:tasks.workunit.client.1.smithi130.stdout:9/548: rename d1/db/f83 to d1/d7/fac 0 2024-05-13T22:48:33.319 INFO:tasks.workunit.client.1.smithi130.stdout:9/549: write d1/d7/d23/d28/d73/d8d/f8f [481859,6526] 0 2024-05-13T22:48:33.321 INFO:tasks.workunit.client.1.smithi130.stdout:5/472: write f4 [40832,112555] 0 2024-05-13T22:48:33.322 INFO:tasks.workunit.client.1.smithi130.stdout:5/473: fdatasync d6/d1a/f48 0 2024-05-13T22:48:33.324 INFO:tasks.workunit.client.1.smithi130.stdout:7/484: mknod d7/da/ca3 0 2024-05-13T22:48:33.327 INFO:tasks.workunit.client.1.smithi130.stdout:4/502: readlink d8/df/l66 0 2024-05-13T22:48:33.329 INFO:tasks.workunit.client.1.smithi130.stdout:7/485: dwrite d7/d8/d5b/f97 [4194304,4194304] 0 2024-05-13T22:48:33.331 INFO:tasks.workunit.client.1.smithi130.stdout:6/504: creat d4/fb1 x:0 0 0 2024-05-13T22:48:33.335 INFO:tasks.workunit.client.1.smithi130.stdout:0/527: creat d0/fb7 x:0 0 0 2024-05-13T22:48:33.336 INFO:tasks.workunit.client.1.smithi130.stdout:5/474: creat d6/d1a/d92/f9f x:0 0 0 2024-05-13T22:48:33.337 INFO:tasks.workunit.client.1.smithi130.stdout:8/454: write d1/d2/f8c [400245,97736] 0 2024-05-13T22:48:33.340 INFO:tasks.workunit.client.1.smithi130.stdout:4/503: fsync d8/fa 0 2024-05-13T22:48:33.344 INFO:tasks.workunit.client.1.smithi130.stdout:0/528: dwrite d0/d7/dd/d17/d1f/d2c/d45/d6b/f7c [0,4194304] 0 2024-05-13T22:48:33.344 INFO:tasks.workunit.client.1.smithi130.stdout:5/475: dread d6/f7c [0,4194304] 0 2024-05-13T22:48:33.345 INFO:tasks.workunit.client.1.smithi130.stdout:5/476: fdatasync d6/d1a/f40 0 2024-05-13T22:48:33.352 INFO:tasks.workunit.client.1.smithi130.stdout:5/477: chown d6/d7/d19/d4d/d8a 533 1 2024-05-13T22:48:33.352 INFO:tasks.workunit.client.1.smithi130.stdout:3/478: rename c6 to d8/dc/d19/ca9 0 2024-05-13T22:48:33.352 INFO:tasks.workunit.client.1.smithi130.stdout:3/479: write d8/d35/f80 [964205,23608] 0 2024-05-13T22:48:33.352 INFO:tasks.workunit.client.1.smithi130.stdout:2/495: sync 2024-05-13T22:48:33.353 INFO:tasks.workunit.client.1.smithi130.stdout:1/462: sync 2024-05-13T22:48:33.354 INFO:tasks.workunit.client.1.smithi130.stdout:0/529: dwrite d0/d7/dd/d17/d1f/d2c/d5a/f66 [0,4194304] 0 2024-05-13T22:48:33.355 INFO:tasks.workunit.client.1.smithi130.stdout:0/530: stat d0/d7/dd/d69 0 2024-05-13T22:48:33.356 INFO:tasks.workunit.client.1.smithi130.stdout:0/531: chown d0/d7/d29/l39 11 1 2024-05-13T22:48:33.356 INFO:tasks.workunit.client.1.smithi130.stdout:7/486: symlink d7/d8/la4 0 2024-05-13T22:48:33.361 INFO:tasks.workunit.client.1.smithi130.stdout:7/487: dread d7/da/d1d/d3d/d46/f4f [0,4194304] 0 2024-05-13T22:48:33.361 INFO:tasks.workunit.client.1.smithi130.stdout:6/505: dread d4/f9 [0,4194304] 0 2024-05-13T22:48:33.364 INFO:tasks.workunit.client.1.smithi130.stdout:5/478: dread d6/d2f/f59 [0,4194304] 0 2024-05-13T22:48:33.365 INFO:tasks.workunit.client.1.smithi130.stdout:5/479: truncate d6/d7/d19/f9a 472619 0 2024-05-13T22:48:33.377 INFO:tasks.workunit.client.1.smithi130.stdout:4/504: creat d8/dc/d35/d33/d44/d77/d94/f97 x:0 0 0 2024-05-13T22:48:33.378 INFO:tasks.workunit.client.1.smithi130.stdout:9/550: getdents d1/d7 0 2024-05-13T22:48:33.379 INFO:tasks.workunit.client.1.smithi130.stdout:7/488: creat d7/d8/d9/d66/fa5 x:0 0 0 2024-05-13T22:48:33.379 INFO:tasks.workunit.client.1.smithi130.stdout:4/505: dread d8/f43 [4194304,4194304] 0 2024-05-13T22:48:33.380 INFO:tasks.workunit.client.1.smithi130.stdout:9/551: write d1/db/f5e [554262,40893] 0 2024-05-13T22:48:33.380 INFO:tasks.workunit.client.1.smithi130.stdout:6/506: creat d4/d15/d1f/d46/fb2 x:0 0 0 2024-05-13T22:48:33.382 INFO:tasks.workunit.client.1.smithi130.stdout:7/489: truncate d7/d8/d9/d66/d65/f91 241486 0 2024-05-13T22:48:33.383 INFO:tasks.workunit.client.1.smithi130.stdout:8/455: link d1/d2/db/d15/d18/l2e d1/d2/db/d15/d7e/la2 0 2024-05-13T22:48:33.384 INFO:tasks.workunit.client.1.smithi130.stdout:7/490: chown d7/da/d1d/d3d/d46/d92 15 1 2024-05-13T22:48:33.385 INFO:tasks.workunit.client.1.smithi130.stdout:1/463: unlink db/d1b/d1f/d2a/c37 0 2024-05-13T22:48:33.386 INFO:tasks.workunit.client.1.smithi130.stdout:2/496: symlink d4/db/dc/d18/d2a/d30/d88/d90/d93/lae 0 2024-05-13T22:48:33.388 INFO:tasks.workunit.client.1.smithi130.stdout:6/507: dwrite d4/d15/d1d/d28/f30 [0,4194304] 0 2024-05-13T22:48:33.401 INFO:tasks.workunit.client.1.smithi130.stdout:3/480: dwrite d8/d1a/d1c/f25 [0,4194304] 0 2024-05-13T22:48:33.402 INFO:tasks.workunit.client.1.smithi130.stdout:4/506: creat d8/dc/d35/d33/d44/d61/f98 x:0 0 0 2024-05-13T22:48:33.407 INFO:tasks.workunit.client.1.smithi130.stdout:7/491: unlink d7/d8/d9/d66/f99 0 2024-05-13T22:48:33.411 INFO:tasks.workunit.client.1.smithi130.stdout:1/464: unlink db/d4a/f6b 0 2024-05-13T22:48:33.411 INFO:tasks.workunit.client.1.smithi130.stdout:8/456: rename d1/d2/db/d15/d18 to d1/d2/db/d15/d18/da3 22 2024-05-13T22:48:33.414 INFO:tasks.workunit.client.1.smithi130.stdout:0/532: rmdir d0/d7/dd/d17/d52/d5b/daf 0 2024-05-13T22:48:33.416 INFO:tasks.workunit.client.1.smithi130.stdout:0/533: dread d0/d7/f16 [0,4194304] 0 2024-05-13T22:48:33.416 INFO:tasks.workunit.client.1.smithi130.stdout:0/534: chown d0/d7/d29/d7b 1191 1 2024-05-13T22:48:33.417 INFO:tasks.workunit.client.1.smithi130.stdout:0/535: chown d0/d7/dd/d17/d1f 4703 1 2024-05-13T22:48:33.417 INFO:tasks.workunit.client.1.smithi130.stdout:9/552: creat d1/d7/d69/da2/fad x:0 0 0 2024-05-13T22:48:33.418 INFO:tasks.workunit.client.1.smithi130.stdout:0/536: write d0/d7/dd/d17/d1f/f7d [988725,76656] 0 2024-05-13T22:48:33.419 INFO:tasks.workunit.client.1.smithi130.stdout:5/480: mknod d6/d7/d19/d4d/d53/ca0 0 2024-05-13T22:48:33.420 INFO:tasks.workunit.client.1.smithi130.stdout:0/537: dread - d0/d7/dd/d17/d1f/d2c/d45/d57/f85 zero size 2024-05-13T22:48:33.423 INFO:tasks.workunit.client.1.smithi130.stdout:3/481: fdatasync d8/dc/d19/d2c/fa6 0 2024-05-13T22:48:33.434 INFO:tasks.workunit.client.1.smithi130.stdout:4/507: dread d8/f1f [0,4194304] 0 2024-05-13T22:48:33.437 INFO:tasks.workunit.client.1.smithi130.stdout:7/492: rename d7/d8 to d7/da/d1d/d5d/da6 0 2024-05-13T22:48:33.439 INFO:tasks.workunit.client.1.smithi130.stdout:2/497: creat d4/d49/d48/d4a/d64/faf x:0 0 0 2024-05-13T22:48:33.439 INFO:tasks.workunit.client.1.smithi130.stdout:2/498: write d4/d49/d48/d4a/f82 [232024,85506] 0 2024-05-13T22:48:33.443 INFO:tasks.workunit.client.1.smithi130.stdout:8/457: mkdir d1/d2/db/d15/d18/d42/d53/da4 0 2024-05-13T22:48:33.443 INFO:tasks.workunit.client.1.smithi130.stdout:8/458: write d1/d2/f6 [2686969,111405] 0 2024-05-13T22:48:33.447 INFO:tasks.workunit.client.1.smithi130.stdout:6/508: mknod d4/d15/d1d/d60/da2/da4/dac/cb3 0 2024-05-13T22:48:33.447 INFO:tasks.workunit.client.1.smithi130.stdout:6/509: stat d4/d16/c6c 0 2024-05-13T22:48:33.450 INFO:tasks.workunit.client.1.smithi130.stdout:5/481: write d6/f7c [5031874,21170] 0 2024-05-13T22:48:33.452 INFO:tasks.workunit.client.1.smithi130.stdout:6/510: dwrite d4/d15/d1d/fa5 [0,4194304] 0 2024-05-13T22:48:33.458 INFO:tasks.workunit.client.1.smithi130.stdout:0/538: dwrite d0/fe [4194304,4194304] 0 2024-05-13T22:48:33.463 INFO:tasks.workunit.client.1.smithi130.stdout:4/508: dwrite d8/f1b [0,4194304] 0 2024-05-13T22:48:33.467 INFO:tasks.workunit.client.1.smithi130.stdout:6/511: fsync d4/f9 0 2024-05-13T22:48:33.468 INFO:tasks.workunit.client.1.smithi130.stdout:6/512: chown d4/c13 7217 1 2024-05-13T22:48:33.470 INFO:tasks.workunit.client.1.smithi130.stdout:7/493: unlink d7/da/d1d/d42/f7d 0 2024-05-13T22:48:33.475 INFO:tasks.workunit.client.1.smithi130.stdout:5/482: dread d6/d7/d32/f3d [0,4194304] 0 2024-05-13T22:48:33.477 INFO:tasks.workunit.client.1.smithi130.stdout:6/513: dread d4/d15/d1f/d46/d50/f79 [0,4194304] 0 2024-05-13T22:48:33.478 INFO:tasks.workunit.client.1.smithi130.stdout:1/465: rename db/d1b/d1f/d2a/d38/d46/d6d/d87 to db/d9c 0 2024-05-13T22:48:33.483 INFO:tasks.workunit.client.1.smithi130.stdout:1/466: dwrite db/d1b/d1f/f7a [0,4194304] 0 2024-05-13T22:48:33.484 INFO:tasks.workunit.client.1.smithi130.stdout:0/539: creat d0/d7/dd/d2e/d9e/da7/fb8 x:0 0 0 2024-05-13T22:48:33.488 INFO:tasks.workunit.client.1.smithi130.stdout:7/494: unlink d7/da/d1d/d5d/da6/d7e/l82 0 2024-05-13T22:48:33.488 INFO:tasks.workunit.client.1.smithi130.stdout:7/495: fsync d7/da/d1d/f6f 0 2024-05-13T22:48:33.497 INFO:tasks.workunit.client.1.smithi130.stdout:5/483: write d6/dd/d18/f15 [935743,115346] 0 2024-05-13T22:48:33.498 INFO:tasks.workunit.client.1.smithi130.stdout:5/484: fdatasync d6/dd/d18/d6e/d82/d4b/f99 0 2024-05-13T22:48:33.499 INFO:tasks.workunit.client.1.smithi130.stdout:7/496: rmdir d7/da 39 2024-05-13T22:48:33.505 INFO:tasks.workunit.client.1.smithi130.stdout:1/467: mknod db/d1b/d1f/d2a/d4c/d5b/c9d 0 2024-05-13T22:48:33.509 INFO:tasks.workunit.client.1.smithi130.stdout:6/514: truncate d4/f12 5768650 0 2024-05-13T22:48:33.511 INFO:tasks.workunit.client.1.smithi130.stdout:4/509: unlink d8/dc/d35/d33/d3f/d57/c58 0 2024-05-13T22:48:33.513 INFO:tasks.workunit.client.1.smithi130.stdout:2/499: rename d4/d49/d42/d4b/c5e to d4/d49/cb0 0 2024-05-13T22:48:33.513 INFO:tasks.workunit.client.1.smithi130.stdout:7/497: dwrite d7/da/d1d/d5d/da6/d9/d66/f31 [0,4194304] 0 2024-05-13T22:48:33.518 INFO:tasks.workunit.client.1.smithi130.stdout:4/510: unlink d8/dc/d35/d33/d44/d56/d8c/c8e 0 2024-05-13T22:48:33.522 INFO:tasks.workunit.client.1.smithi130.stdout:7/498: mkdir d7/da/d1d/d42/d43/da7 0 2024-05-13T22:48:33.522 INFO:tasks.workunit.client.1.smithi130.stdout:0/540: link d0/f42 d0/d7/dd/d17/d1f/d2c/d5a/fb9 0 2024-05-13T22:48:33.524 INFO:tasks.workunit.client.1.smithi130.stdout:1/468: rename db/d1b/d1f/d2a/d38/f5c to db/d1b/d1f/d2a/d38/d46/d6d/f9e 0 2024-05-13T22:48:33.525 INFO:tasks.workunit.client.1.smithi130.stdout:1/469: chown db/d1b/d1f/d2a/d35/f40 935911771 1 2024-05-13T22:48:33.525 INFO:tasks.workunit.client.1.smithi130.stdout:4/511: symlink d8/dc/d35/d33/d44/l99 0 2024-05-13T22:48:33.526 INFO:tasks.workunit.client.1.smithi130.stdout:1/470: write db/d4a/f4b [2995929,58033] 0 2024-05-13T22:48:33.526 INFO:tasks.workunit.client.1.smithi130.stdout:5/485: link d6/c7e d6/ca1 0 2024-05-13T22:48:33.527 INFO:tasks.workunit.client.1.smithi130.stdout:5/486: fdatasync d6/d7/f43 0 2024-05-13T22:48:33.527 INFO:tasks.workunit.client.1.smithi130.stdout:9/553: sync 2024-05-13T22:48:33.527 INFO:tasks.workunit.client.1.smithi130.stdout:3/482: sync 2024-05-13T22:48:33.527 INFO:tasks.workunit.client.1.smithi130.stdout:8/459: sync 2024-05-13T22:48:33.527 INFO:tasks.workunit.client.1.smithi130.stdout:2/500: mknod d4/db/dc/d18/d2a/d30/d88/d99/d9a/cb1 0 2024-05-13T22:48:33.528 INFO:tasks.workunit.client.1.smithi130.stdout:5/487: readlink d6/dd/l6d 0 2024-05-13T22:48:33.529 INFO:tasks.workunit.client.1.smithi130.stdout:3/483: write d8/d1a/d1c/fa0 [121512,102865] 0 2024-05-13T22:48:33.529 INFO:tasks.workunit.client.1.smithi130.stdout:2/501: write d4/f7 [4753767,63969] 0 2024-05-13T22:48:33.530 INFO:tasks.workunit.client.1.smithi130.stdout:4/512: creat d8/dc/f9a x:0 0 0 2024-05-13T22:48:33.531 INFO:tasks.workunit.client.1.smithi130.stdout:8/460: write d1/d2/db/d15/d7e/fa1 [615401,111278] 0 2024-05-13T22:48:33.532 INFO:tasks.workunit.client.1.smithi130.stdout:2/502: chown d4/db 406014089 1 2024-05-13T22:48:33.533 INFO:tasks.workunit.client.1.smithi130.stdout:9/554: readlink d1/d7/d23/d28/d2d/d2f/l60 0 2024-05-13T22:48:33.535 INFO:tasks.workunit.client.1.smithi130.stdout:0/541: symlink d0/d7/dd/d17/lba 0 2024-05-13T22:48:33.535 INFO:tasks.workunit.client.1.smithi130.stdout:9/555: truncate d1/d7/d23/d28/d2d/d2f/f93 446498 0 2024-05-13T22:48:33.536 INFO:tasks.workunit.client.1.smithi130.stdout:5/488: symlink d6/d1a/d92/la2 0 2024-05-13T22:48:33.537 INFO:tasks.workunit.client.1.smithi130.stdout:0/542: chown d0/d7/dd/d69/da9/d8d/d9f 339621 1 2024-05-13T22:48:33.537 INFO:tasks.workunit.client.1.smithi130.stdout:8/461: dread d1/d2/db/d15/d18/f63 [0,4194304] 0 2024-05-13T22:48:33.537 INFO:tasks.workunit.client.1.smithi130.stdout:9/556: stat d1/d15/l24 0 2024-05-13T22:48:33.538 INFO:tasks.workunit.client.1.smithi130.stdout:3/484: readlink d8/dc/d15/d8f/la8 0 2024-05-13T22:48:33.539 INFO:tasks.workunit.client.1.smithi130.stdout:7/499: link d7/da/d1d/d5d/da6/f1c d7/da/d1d/d5d/da6/d39/fa8 0 2024-05-13T22:48:33.541 INFO:tasks.workunit.client.1.smithi130.stdout:2/503: mknod d4/d49/d42/d4b/cb2 0 2024-05-13T22:48:33.544 INFO:tasks.workunit.client.1.smithi130.stdout:9/557: dwrite d1/f1e [0,4194304] 0 2024-05-13T22:48:33.548 INFO:tasks.workunit.client.1.smithi130.stdout:0/543: chown d0/d7/dd/d2e/l48 5216 1 2024-05-13T22:48:33.548 INFO:tasks.workunit.client.1.smithi130.stdout:6/515: dread d4/d15/d1d/f22 [4194304,4194304] 0 2024-05-13T22:48:33.550 INFO:tasks.workunit.client.1.smithi130.stdout:2/504: dwrite d4/f1b [0,4194304] 0 2024-05-13T22:48:33.551 INFO:tasks.workunit.client.1.smithi130.stdout:8/462: symlink d1/d2/db/d15/d18/d1a/la5 0 2024-05-13T22:48:33.552 INFO:tasks.workunit.client.1.smithi130.stdout:3/485: rmdir d8/dc/d19/d2c/d61/d9c 39 2024-05-13T22:48:33.554 INFO:tasks.workunit.client.1.smithi130.stdout:2/505: write d4/d49/d48/d4a/d64/f6a [597089,119846] 0 2024-05-13T22:48:33.556 INFO:tasks.workunit.client.1.smithi130.stdout:9/558: dwrite d1/d15/d51/f6d [0,4194304] 0 2024-05-13T22:48:33.558 INFO:tasks.workunit.client.1.smithi130.stdout:9/559: write d1/d7/d23/d28/f67 [4476329,86500] 0 2024-05-13T22:48:33.560 INFO:tasks.workunit.client.1.smithi130.stdout:1/471: getdents db/d58 0 2024-05-13T22:48:33.573 INFO:tasks.workunit.client.1.smithi130.stdout:0/544: write d0/d7/dd/d17/d52/d5b/f6f [957137,33591] 0 2024-05-13T22:48:33.576 INFO:tasks.workunit.client.1.smithi130.stdout:0/545: dread d0/d7/dd/d17/d52/f80 [0,4194304] 0 2024-05-13T22:48:33.579 INFO:tasks.workunit.client.1.smithi130.stdout:8/463: dwrite d1/d2/db/d15/f74 [0,4194304] 0 2024-05-13T22:48:33.580 INFO:tasks.workunit.client.1.smithi130.stdout:3/486: creat d8/d1a/d1c/faa x:0 0 0 2024-05-13T22:48:33.581 INFO:tasks.workunit.client.1.smithi130.stdout:3/487: fsync d8/d1a/d1c/f25 0 2024-05-13T22:48:33.584 INFO:tasks.workunit.client.1.smithi130.stdout:2/506: fsync d4/fd 0 2024-05-13T22:48:33.595 INFO:tasks.workunit.client.1.smithi130.stdout:7/500: dwrite d7/da/d1d/d5d/da6/d5b/f6d [0,4194304] 0 2024-05-13T22:48:33.595 INFO:tasks.workunit.client.1.smithi130.stdout:1/472: mkdir db/d82/d9f 0 2024-05-13T22:48:33.596 INFO:tasks.workunit.client.1.smithi130.stdout:1/473: chown db/d1b/d1f/l43 233218 1 2024-05-13T22:48:33.603 INFO:tasks.workunit.client.1.smithi130.stdout:5/489: rename d6/f86 to d6/d7/d19/d4d/fa3 0 2024-05-13T22:48:33.608 INFO:tasks.workunit.client.1.smithi130.stdout:5/490: dwrite d6/f73 [0,4194304] 0 2024-05-13T22:48:33.610 INFO:tasks.workunit.client.1.smithi130.stdout:0/546: dread - d0/d7/dd/d2e/f75 zero size 2024-05-13T22:48:33.611 INFO:tasks.workunit.client.1.smithi130.stdout:3/488: dread d8/d1a/d1c/f6e [0,4194304] 0 2024-05-13T22:48:33.611 INFO:tasks.workunit.client.1.smithi130.stdout:3/489: write d8/f33 [227312,22521] 0 2024-05-13T22:48:33.612 INFO:tasks.workunit.client.1.smithi130.stdout:3/490: dread - d8/dc/d7b/f99 zero size 2024-05-13T22:48:33.616 INFO:tasks.workunit.client.1.smithi130.stdout:2/507: creat d4/db/dc/d18/d2a/d30/fb3 x:0 0 0 2024-05-13T22:48:33.618 INFO:tasks.workunit.client.1.smithi130.stdout:9/560: mknod d1/d7/d23/d3d/da1/cae 0 2024-05-13T22:48:33.619 INFO:tasks.workunit.client.1.smithi130.stdout:9/561: read - d1/d7/d23/d28/f7c zero size 2024-05-13T22:48:33.623 INFO:tasks.workunit.client.1.smithi130.stdout:7/501: mkdir d7/da/d1d/d5d/da6/d39/da9 0 2024-05-13T22:48:33.628 INFO:tasks.workunit.client.1.smithi130.stdout:4/513: chown d8/dc/d35/d33 2297341 1 2024-05-13T22:48:33.634 INFO:tasks.workunit.client.1.smithi130.stdout:3/491: truncate d8/fa 102390 0 2024-05-13T22:48:33.638 INFO:tasks.workunit.client.1.smithi130.stdout:9/562: mknod d1/d15/d51/d85/caf 0 2024-05-13T22:48:33.639 INFO:tasks.workunit.client.1.smithi130.stdout:0/547: dread d0/f1e [0,4194304] 0 2024-05-13T22:48:33.640 INFO:tasks.workunit.client.1.smithi130.stdout:9/563: write d1/d7/d23/d28/d2d/d2f/d63/d6b/f84 [1897520,68025] 0 2024-05-13T22:48:33.646 INFO:tasks.workunit.client.1.smithi130.stdout:9/564: dwrite d1/d7/d23/d28/f39 [0,4194304] 0 2024-05-13T22:48:33.648 INFO:tasks.workunit.client.1.smithi130.stdout:2/508: dread d4/db/dc/f15 [0,4194304] 0 2024-05-13T22:48:33.649 INFO:tasks.workunit.client.1.smithi130.stdout:5/491: link d6/d7/f25 d6/d2f/fa4 0 2024-05-13T22:48:33.650 INFO:tasks.workunit.client.1.smithi130.stdout:3/492: symlink d8/d1a/d1c/lab 0 2024-05-13T22:48:33.650 INFO:tasks.workunit.client.1.smithi130.stdout:5/492: chown d6/d7/d19/d4d/d8a 304977 1 2024-05-13T22:48:33.651 INFO:tasks.workunit.client.1.smithi130.stdout:5/493: stat d6/dd/d18/fc 0 2024-05-13T22:48:33.651 INFO:tasks.workunit.client.1.smithi130.stdout:3/493: write d8/dc/d7b/f99 [693264,7491] 0 2024-05-13T22:48:33.654 INFO:tasks.workunit.client.1.smithi130.stdout:7/502: fdatasync d7/da/d1d/d5d/da6/f1c 0 2024-05-13T22:48:33.655 INFO:tasks.workunit.client.1.smithi130.stdout:7/503: stat d7/da/d1d/d5d/da6/d9/l16 0 2024-05-13T22:48:33.655 INFO:tasks.workunit.client.1.smithi130.stdout:7/504: chown d7/da/d1d/d3d/d46/d67/f30 15065699 1 2024-05-13T22:48:33.661 INFO:tasks.workunit.client.1.smithi130.stdout:6/516: rename d4/d15/d1f/d46/d50/f53 to d4/fb4 0 2024-05-13T22:48:33.664 INFO:tasks.workunit.client.1.smithi130.stdout:9/565: dread d1/d7/d23/d3d/d43/f75 [0,4194304] 0 2024-05-13T22:48:33.665 INFO:tasks.workunit.client.1.smithi130.stdout:3/494: mkdir d8/dc/d15/dac 0 2024-05-13T22:48:33.667 INFO:tasks.workunit.client.1.smithi130.stdout:1/474: truncate db/d1b/d1f/d2a/f41 882963 0 2024-05-13T22:48:33.668 INFO:tasks.workunit.client.1.smithi130.stdout:1/475: stat db/d1b/d1f/d2a/d38 0 2024-05-13T22:48:33.668 INFO:tasks.workunit.client.1.smithi130.stdout:7/505: creat d7/da/d1d/d5d/da6/d9/d66/d65/faa x:0 0 0 2024-05-13T22:48:33.669 INFO:tasks.workunit.client.1.smithi130.stdout:7/506: chown d7/da/d1d/d42/c98 503473 1 2024-05-13T22:48:33.673 INFO:tasks.workunit.client.1.smithi130.stdout:7/507: dwrite d7/da/d1d/d5d/d70/f85 [0,4194304] 0 2024-05-13T22:48:33.674 INFO:tasks.workunit.client.1.smithi130.stdout:4/514: rename d8/dc/d35/d33/d3f/f95 to d8/df/d81/f9b 0 2024-05-13T22:48:33.675 INFO:tasks.workunit.client.1.smithi130.stdout:6/517: fsync d4/d15/d1f/d46/d50/f79 0 2024-05-13T22:48:33.677 INFO:tasks.workunit.client.1.smithi130.stdout:8/464: sync 2024-05-13T22:48:33.681 INFO:tasks.workunit.client.1.smithi130.stdout:0/548: creat d0/d7/dd/d17/fbb x:0 0 0 2024-05-13T22:48:33.682 INFO:tasks.workunit.client.1.smithi130.stdout:1/476: truncate db/d1b/d1f/d2a/d35/f3b 29106 0 2024-05-13T22:48:33.690 INFO:tasks.workunit.client.1.smithi130.stdout:1/477: read db/d1b/d1f/d72/f79 [1306315,130601] 0 2024-05-13T22:48:33.690 INFO:tasks.workunit.client.1.smithi130.stdout:2/509: rename d4/db/dc/d18/d2a/d30/f34 to d4/d49/d48/d4a/d64/fb4 0 2024-05-13T22:48:33.691 INFO:tasks.workunit.client.1.smithi130.stdout:4/515: rmdir d8/df 39 2024-05-13T22:48:33.692 INFO:tasks.workunit.client.1.smithi130.stdout:2/510: truncate d4/db/dc/d18/f7e 174304 0 2024-05-13T22:48:33.693 INFO:tasks.workunit.client.1.smithi130.stdout:5/494: write d6/dd/d18/d6e/d82/f7b [1239477,84117] 0 2024-05-13T22:48:33.694 INFO:tasks.workunit.client.1.smithi130.stdout:6/518: readlink d4/d15/d1d/l1e 0 2024-05-13T22:48:33.699 INFO:tasks.workunit.client.1.smithi130.stdout:0/549: chown d0/fad 28122 1 2024-05-13T22:48:33.700 INFO:tasks.workunit.client.1.smithi130.stdout:0/550: read d0/d7/dd/d2e/d4b/f65 [1353425,79853] 0 2024-05-13T22:48:33.701 INFO:tasks.workunit.client.1.smithi130.stdout:0/551: chown d0/d7/dd/d17/d1f/d2c/d45/d57/fb6 1276 1 2024-05-13T22:48:33.703 INFO:tasks.workunit.client.1.smithi130.stdout:9/566: rename d1/d15/f3b to d1/d15/d51/d85/fb0 0 2024-05-13T22:48:33.703 INFO:tasks.workunit.client.1.smithi130.stdout:9/567: readlink d1/d7/d69/l8c 0 2024-05-13T22:48:33.704 INFO:tasks.workunit.client.1.smithi130.stdout:7/508: mknod d7/da/d1d/d5d/da6/d9/d4a/cab 0 2024-05-13T22:48:33.706 INFO:tasks.workunit.client.1.smithi130.stdout:2/511: rmdir d4/db/dc/d18/d2a/d30/d69 39 2024-05-13T22:48:33.706 INFO:tasks.workunit.client.1.smithi130.stdout:2/512: chown d4/d56 132475204 1 2024-05-13T22:48:33.711 INFO:tasks.workunit.client.1.smithi130.stdout:6/519: write d4/d15/d1f/f20 [2051322,54466] 0 2024-05-13T22:48:33.712 INFO:tasks.workunit.client.1.smithi130.stdout:5/495: dread d6/dd/d18/f2b [0,4194304] 0 2024-05-13T22:48:33.712 INFO:tasks.workunit.client.1.smithi130.stdout:1/478: dread db/d1b/d1f/d2a/d38/f60 [0,4194304] 0 2024-05-13T22:48:33.713 INFO:tasks.workunit.client.1.smithi130.stdout:5/496: truncate d6/dd/d18/fc 5150661 0 2024-05-13T22:48:33.714 INFO:tasks.workunit.client.1.smithi130.stdout:1/479: truncate db/d1b/d1f/d2a/d35/f69 674173 0 2024-05-13T22:48:33.716 INFO:tasks.workunit.client.1.smithi130.stdout:8/465: rename d1/d2/db/d15/d18/d42/d53/d56/f7c to d1/d2/db/fa6 0 2024-05-13T22:48:33.717 INFO:tasks.workunit.client.1.smithi130.stdout:8/466: fdatasync d1/d2/db/d15/d16/f1c 0 2024-05-13T22:48:33.718 INFO:tasks.workunit.client.1.smithi130.stdout:9/568: creat d1/d7/d23/d3d/d43/fb1 x:0 0 0 2024-05-13T22:48:33.719 INFO:tasks.workunit.client.1.smithi130.stdout:7/509: symlink d7/da/d1d/d3d/d46/lac 0 2024-05-13T22:48:33.720 INFO:tasks.workunit.client.1.smithi130.stdout:0/552: write d0/d7/dd/d17/d52/d5b/d5f/f70 [91430,111690] 0 2024-05-13T22:48:33.724 INFO:tasks.workunit.client.1.smithi130.stdout:6/520: unlink d4/d15/d1d/d60/f77 0 2024-05-13T22:48:33.724 INFO:tasks.workunit.client.1.smithi130.stdout:6/521: chown d4/d15/d1d/d28/l40 4997 1 2024-05-13T22:48:33.725 INFO:tasks.workunit.client.1.smithi130.stdout:6/522: chown d4/d15/d1f/d46/f58 707 1 2024-05-13T22:48:33.726 INFO:tasks.workunit.client.1.smithi130.stdout:6/523: truncate d4/f73 71485 0 2024-05-13T22:48:33.727 INFO:tasks.workunit.client.1.smithi130.stdout:3/495: rename d8/c60 to d8/dc/d19/d2c/d61/d9c/cad 0 2024-05-13T22:48:33.729 INFO:tasks.workunit.client.1.smithi130.stdout:8/467: dread d1/d2/db/d15/d16/f8f [0,4194304] 0 2024-05-13T22:48:33.729 INFO:tasks.workunit.client.1.smithi130.stdout:8/468: fsync d1/f65 0 2024-05-13T22:48:33.730 INFO:tasks.workunit.client.1.smithi130.stdout:0/553: read d0/d7/dd/d17/d52/d67/d68/d96/f83 [4340543,34033] 0 2024-05-13T22:48:33.730 INFO:tasks.workunit.client.1.smithi130.stdout:0/554: chown d0/d7/dd/d69/da9/l99 9443163 1 2024-05-13T22:48:33.731 INFO:tasks.workunit.client.1.smithi130.stdout:0/555: write d0/d7/d29/f40 [367809,5289] 0 2024-05-13T22:48:33.732 INFO:tasks.workunit.client.1.smithi130.stdout:0/556: stat d0/d7/dd/d17/d52/d5b 0 2024-05-13T22:48:33.733 INFO:tasks.workunit.client.1.smithi130.stdout:0/557: truncate d0/d7/d29/f40 1346633 0 2024-05-13T22:48:33.733 INFO:tasks.workunit.client.1.smithi130.stdout:1/480: mkdir db/d82/d9f/da0 0 2024-05-13T22:48:33.737 INFO:tasks.workunit.client.1.smithi130.stdout:2/513: rename d4/d56/f68 to d4/d49/d42/d4b/fb5 0 2024-05-13T22:48:33.738 INFO:tasks.workunit.client.1.smithi130.stdout:7/510: creat d7/da/d1d/d3d/d46/d92/fad x:0 0 0 2024-05-13T22:48:33.738 INFO:tasks.workunit.client.1.smithi130.stdout:2/514: truncate d4/db/dc/d18/d2a/fab 877183 0 2024-05-13T22:48:33.739 INFO:tasks.workunit.client.1.smithi130.stdout:2/515: write d4/db/f2c [2701325,67012] 0 2024-05-13T22:48:33.744 INFO:tasks.workunit.client.1.smithi130.stdout:2/516: dwrite d4/d49/d48/d4a/f52 [0,4194304] 0 2024-05-13T22:48:33.745 INFO:tasks.workunit.client.1.smithi130.stdout:2/517: dread - d4/db/d26/f8f zero size 2024-05-13T22:48:33.745 INFO:tasks.workunit.client.1.smithi130.stdout:0/558: creat d0/d7/dd/d69/da9/fbc x:0 0 0 2024-05-13T22:48:33.746 INFO:tasks.workunit.client.1.smithi130.stdout:2/518: write d4/f3f [5229218,122127] 0 2024-05-13T22:48:33.746 INFO:tasks.workunit.client.1.smithi130.stdout:6/524: creat d4/d15/d1f/d3c/fb5 x:0 0 0 2024-05-13T22:48:33.748 INFO:tasks.workunit.client.1.smithi130.stdout:0/559: write d0/d7/f22 [3556862,116756] 0 2024-05-13T22:48:33.749 INFO:tasks.workunit.client.1.smithi130.stdout:4/516: getdents d8/dc/d35/d33/d3f 0 2024-05-13T22:48:33.751 INFO:tasks.workunit.client.1.smithi130.stdout:2/519: chown d4/db/dc/d18/d58/f59 1 1 2024-05-13T22:48:33.752 INFO:tasks.workunit.client.1.smithi130.stdout:0/560: read d0/d7/dd/d17/d1f/f7d [614040,41214] 0 2024-05-13T22:48:33.753 INFO:tasks.workunit.client.1.smithi130.stdout:0/561: dread - d0/d7/dd/d17/d52/d5b/d5f/f93 zero size 2024-05-13T22:48:33.754 INFO:tasks.workunit.client.1.smithi130.stdout:9/569: rename d1/d7/d23/d3d/da1 to d1/d7/d23/d28/d73/db2 0 2024-05-13T22:48:33.756 INFO:tasks.workunit.client.1.smithi130.stdout:9/570: write d1/d7/d23/d3d/d43/fb1 [14857,112518] 0 2024-05-13T22:48:33.762 INFO:tasks.workunit.client.1.smithi130.stdout:8/469: fdatasync d1/d2/db/fa6 0 2024-05-13T22:48:33.763 INFO:tasks.workunit.client.1.smithi130.stdout:8/470: stat d1/d2/db/d15/d18/d1a/l6a 0 2024-05-13T22:48:33.763 INFO:tasks.workunit.client.1.smithi130.stdout:8/471: stat d1/d2/db/d15/d18 0 2024-05-13T22:48:33.766 INFO:tasks.workunit.client.1.smithi130.stdout:5/497: getdents d6/dd/d18/d6e/d76 0 2024-05-13T22:48:33.768 INFO:tasks.workunit.client.1.smithi130.stdout:1/481: dread db/d1b/d1f/d49/f61 [0,4194304] 0 2024-05-13T22:48:33.769 INFO:tasks.workunit.client.1.smithi130.stdout:1/482: stat db/d58 0 2024-05-13T22:48:33.774 INFO:tasks.workunit.client.1.smithi130.stdout:6/525: creat d4/d15/d1f/d46/d50/fb6 x:0 0 0 2024-05-13T22:48:33.775 INFO:tasks.workunit.client.1.smithi130.stdout:2/520: dread - d4/db/dc/d18/d2a/f50 zero size 2024-05-13T22:48:33.780 INFO:tasks.workunit.client.1.smithi130.stdout:8/472: dread d1/d2/db/d15/f22 [0,4194304] 0 2024-05-13T22:48:33.784 INFO:tasks.workunit.client.1.smithi130.stdout:9/571: symlink d1/d7/d23/d28/d32/lb3 0 2024-05-13T22:48:33.785 INFO:tasks.workunit.client.1.smithi130.stdout:2/521: creat d4/db/dc/d18/d2a/d30/d88/d99/d9a/fb6 x:0 0 0 2024-05-13T22:48:33.789 INFO:tasks.workunit.client.1.smithi130.stdout:5/498: link d6/d7/d19/d4d/f34 d6/d7/d19/d4d/d8a/fa5 0 2024-05-13T22:48:33.798 INFO:tasks.workunit.client.1.smithi130.stdout:4/517: link d8/dc/d35/d33/d44/f74 d8/dc/d84/f9c 0 2024-05-13T22:48:33.801 INFO:tasks.workunit.client.1.smithi130.stdout:6/526: rename d4/d15/d1d/c21 to d4/d15/d1f/d3c/d70/cb7 0 2024-05-13T22:48:33.803 INFO:tasks.workunit.client.1.smithi130.stdout:0/562: creat d0/d7/dd/d17/d1f/d2c/fbd x:0 0 0 2024-05-13T22:48:33.810 INFO:tasks.workunit.client.1.smithi130.stdout:2/522: creat d4/db/d26/fb7 x:0 0 0 2024-05-13T22:48:33.814 INFO:tasks.workunit.client.1.smithi130.stdout:1/483: truncate f8 334192 0 2024-05-13T22:48:33.816 INFO:tasks.workunit.client.1.smithi130.stdout:3/496: truncate d8/d1a/d55/f9b 1820960 0 2024-05-13T22:48:33.822 INFO:tasks.workunit.client.1.smithi130.stdout:7/511: link d7/da/d1d/d42/l53 d7/da/d1d/d42/lae 0 2024-05-13T22:48:33.823 INFO:tasks.workunit.client.1.smithi130.stdout:5/499: symlink d6/d7/d19/d4d/d8a/la6 0 2024-05-13T22:48:33.824 INFO:tasks.workunit.client.1.smithi130.stdout:5/500: dread - d6/d7/d19/d4d/f60 zero size 2024-05-13T22:48:33.828 INFO:tasks.workunit.client.1.smithi130.stdout:8/473: creat d1/d2/fa7 x:0 0 0 2024-05-13T22:48:33.829 INFO:tasks.workunit.client.1.smithi130.stdout:8/474: write d1/d2/f8c [940253,76635] 0 2024-05-13T22:48:33.835 INFO:tasks.workunit.client.1.smithi130.stdout:4/518: rename l2 to d8/dc/d80/d96/l9d 0 2024-05-13T22:48:33.837 INFO:tasks.workunit.client.1.smithi130.stdout:6/527: mkdir d4/d15/d1d/db8 0 2024-05-13T22:48:33.837 INFO:tasks.workunit.client.1.smithi130.stdout:8/475: dwrite d1/d2/db/d15/d18/d1a/f9d [0,4194304] 0 2024-05-13T22:48:33.837 INFO:tasks.workunit.client.1.smithi130.stdout:0/563: dwrite d0/d7/f16 [0,4194304] 0 2024-05-13T22:48:33.839 INFO:tasks.workunit.client.1.smithi130.stdout:8/476: chown d1/d2/db/f45 394336 1 2024-05-13T22:48:33.840 INFO:tasks.workunit.client.1.smithi130.stdout:1/484: mkdir db/d1b/d1f/d2a/d38/d66/da1 0 2024-05-13T22:48:33.841 INFO:tasks.workunit.client.1.smithi130.stdout:1/485: chown db/d1b/d1f/d2a/d4c/f65 3012 1 2024-05-13T22:48:33.842 INFO:tasks.workunit.client.1.smithi130.stdout:3/497: mknod d8/d1a/cae 0 2024-05-13T22:48:33.856 INFO:tasks.workunit.client.1.smithi130.stdout:7/512: rename d7/da/d1d/d5d/da6/d5b/f97 to d7/da/d1d/d3d/d46/d92/faf 0 2024-05-13T22:48:33.857 INFO:tasks.workunit.client.1.smithi130.stdout:7/513: chown d7/da/d1d/d5d/da6/l14 279612300 1 2024-05-13T22:48:33.861 INFO:tasks.workunit.client.1.smithi130.stdout:9/572: getdents d1/d7/d23/d28/d73/db2 0 2024-05-13T22:48:33.862 INFO:tasks.workunit.client.1.smithi130.stdout:9/573: write d1/d7/d23/d28/d2d/d2f/d63/f9f [3697110,79037] 0 2024-05-13T22:48:33.864 INFO:tasks.workunit.client.1.smithi130.stdout:8/477: write d1/d2/db/d15/d7e/f95 [4797912,81053] 0 2024-05-13T22:48:33.866 INFO:tasks.workunit.client.1.smithi130.stdout:1/486: truncate db/d1b/d1f/d2a/d38/d46/d6d/f9e 2649859 0 2024-05-13T22:48:33.867 INFO:tasks.workunit.client.1.smithi130.stdout:1/487: stat db/d4a/f4b 0 2024-05-13T22:48:33.872 INFO:tasks.workunit.client.1.smithi130.stdout:6/528: symlink d4/d15/lb9 0 2024-05-13T22:48:33.879 INFO:tasks.workunit.client.1.smithi130.stdout:2/523: getdents d4/db/dc/d18/d2a 0 2024-05-13T22:48:33.880 INFO:tasks.workunit.client.1.smithi130.stdout:9/574: mkdir d1/d15/d51/db4 0 2024-05-13T22:48:33.885 INFO:tasks.workunit.client.1.smithi130.stdout:8/478: rename d1/d2/db/d15/d7e/fa1 to d1/dc/d5d/fa8 0 2024-05-13T22:48:33.887 INFO:tasks.workunit.client.1.smithi130.stdout:8/479: fsync d1/d2/db/d15/d16/d54/d58/f98 0 2024-05-13T22:48:33.888 INFO:tasks.workunit.client.1.smithi130.stdout:3/498: dwrite d8/dc/d19/d2c/d61/d9c/f9d [0,4194304] 0 2024-05-13T22:48:33.888 INFO:tasks.workunit.client.1.smithi130.stdout:1/488: symlink db/d1b/d1f/d2a/d4c/la2 0 2024-05-13T22:48:33.889 INFO:tasks.workunit.client.1.smithi130.stdout:3/499: dread - d8/dc/d19/f9e zero size 2024-05-13T22:48:33.894 INFO:tasks.workunit.client.1.smithi130.stdout:1/489: write db/d1b/d1f/d72/d75/f81 [1026982,56488] 0 2024-05-13T22:48:33.894 INFO:tasks.workunit.client.1.smithi130.stdout:0/564: creat d0/d7/dd/d2e/fbe x:0 0 0 2024-05-13T22:48:33.894 INFO:tasks.workunit.client.1.smithi130.stdout:1/490: stat db/d1b/d1f/d2a/d38/d66 0 2024-05-13T22:48:33.894 INFO:tasks.workunit.client.1.smithi130.stdout:1/491: chown db/d4a 4322 1 2024-05-13T22:48:33.895 INFO:tasks.workunit.client.1.smithi130.stdout:9/575: symlink d1/db/lb5 0 2024-05-13T22:48:33.895 INFO:tasks.workunit.client.1.smithi130.stdout:8/480: dwrite d1/d2/f39 [4194304,4194304] 0 2024-05-13T22:48:33.896 INFO:tasks.workunit.client.1.smithi130.stdout:1/492: chown db/d1b/d1f/d49/d56/d7f/f96 3848033 1 2024-05-13T22:48:33.897 INFO:tasks.workunit.client.1.smithi130.stdout:9/576: dread - d1/d7/d23/d28/d2d/f74 zero size 2024-05-13T22:48:33.897 INFO:tasks.workunit.client.1.smithi130.stdout:1/493: fdatasync db/d4a/f94 0 2024-05-13T22:48:33.902 INFO:tasks.workunit.client.1.smithi130.stdout:4/519: rename d8/dc/d29/f2c to d8/df/d81/f9e 0 2024-05-13T22:48:33.904 INFO:tasks.workunit.client.1.smithi130.stdout:9/577: dwrite d1/d7/d23/d28/f67 [0,4194304] 0 2024-05-13T22:48:33.908 INFO:tasks.workunit.client.1.smithi130.stdout:6/529: fdatasync d4/d15/d1d/f38 0 2024-05-13T22:48:33.909 INFO:tasks.workunit.client.1.smithi130.stdout:2/524: write d4/db/f2e [957528,48747] 0 2024-05-13T22:48:33.917 INFO:tasks.workunit.client.1.smithi130.stdout:1/494: unlink db/d1b/d1f/d2a/d38/d46/c68 0 2024-05-13T22:48:33.922 INFO:tasks.workunit.client.1.smithi130.stdout:5/501: sync 2024-05-13T22:48:33.922 INFO:tasks.workunit.client.1.smithi130.stdout:9/578: symlink d1/d15/d51/d85/lb6 0 2024-05-13T22:48:33.922 INFO:tasks.workunit.client.1.smithi130.stdout:5/502: read - d6/d7/f43 zero size 2024-05-13T22:48:33.923 INFO:tasks.workunit.client.1.smithi130.stdout:7/514: getdents d7/da/d86 0 2024-05-13T22:48:33.925 INFO:tasks.workunit.client.1.smithi130.stdout:3/500: mkdir d8/dc/d7b/da3/daf 0 2024-05-13T22:48:33.925 INFO:tasks.workunit.client.1.smithi130.stdout:3/501: fdatasync d8/dc/d7b/f99 0 2024-05-13T22:48:33.925 INFO:tasks.workunit.client.1.smithi130.stdout:2/525: rmdir d4/d56 39 2024-05-13T22:48:33.926 INFO:tasks.workunit.client.1.smithi130.stdout:2/526: chown d4/d49/c72 12077 1 2024-05-13T22:48:33.926 INFO:tasks.workunit.client.1.smithi130.stdout:6/530: symlink d4/d15/d1f/d3c/d70/lba 0 2024-05-13T22:48:33.927 INFO:tasks.workunit.client.1.smithi130.stdout:0/565: getdents d0/d7/dd/d17/d52/d5b/d60 0 2024-05-13T22:48:33.928 INFO:tasks.workunit.client.1.smithi130.stdout:8/481: symlink d1/d2/db/d15/d18/d42/la9 0 2024-05-13T22:48:33.931 INFO:tasks.workunit.client.1.smithi130.stdout:4/520: truncate d8/dc/d35/d33/f4f 385507 0 2024-05-13T22:48:33.932 INFO:tasks.workunit.client.1.smithi130.stdout:8/482: dread - d1/d2/db/d15/d7e/d9f/fa0 zero size 2024-05-13T22:48:33.934 INFO:tasks.workunit.client.1.smithi130.stdout:3/502: dwrite d8/dc/d19/d2c/d61/f69 [0,4194304] 0 2024-05-13T22:48:33.934 INFO:tasks.workunit.client.1.smithi130.stdout:7/515: fdatasync d7/da/d1d/d5d/da6/f1c 0 2024-05-13T22:48:33.935 INFO:tasks.workunit.client.1.smithi130.stdout:8/483: stat d1/d2/db/d15/d18/d1a/la5 0 2024-05-13T22:48:33.936 INFO:tasks.workunit.client.1.smithi130.stdout:6/531: dwrite d4/d15/d1f/d46/d50/fa7 [0,4194304] 0 2024-05-13T22:48:33.937 INFO:tasks.workunit.client.1.smithi130.stdout:7/516: write d7/da/d1d/d5d/da6/f78 [214472,45302] 0 2024-05-13T22:48:33.938 INFO:tasks.workunit.client.1.smithi130.stdout:8/484: read d1/d2/db/f46 [946351,100966] 0 2024-05-13T22:48:33.938 INFO:tasks.workunit.client.1.smithi130.stdout:7/517: write d7/da/d1d/d5d/da6/d39/f7c [2486979,6193] 0 2024-05-13T22:48:33.940 INFO:tasks.workunit.client.1.smithi130.stdout:7/518: write d7/da/d1d/d5d/da6/d9/d66/f81 [4076860,40914] 0 2024-05-13T22:48:33.948 INFO:tasks.workunit.client.1.smithi130.stdout:9/579: mknod d1/d15/d51/db4/cb7 0 2024-05-13T22:48:33.953 INFO:tasks.workunit.client.1.smithi130.stdout:5/503: rename d6/l64 to d6/dd/d18/d6e/la7 0 2024-05-13T22:48:33.956 INFO:tasks.workunit.client.1.smithi130.stdout:6/532: chown d4/d15/d1f/d46/d52/c5b 1 1 2024-05-13T22:48:33.957 INFO:tasks.workunit.client.1.smithi130.stdout:8/485: symlink d1/d2/db/d15/d7e/d9f/laa 0 2024-05-13T22:48:33.962 INFO:tasks.workunit.client.1.smithi130.stdout:6/533: dwrite d4/d15/d1d/d28/d75/da8/da0/fae [0,4194304] 0 2024-05-13T22:48:33.962 INFO:tasks.workunit.client.1.smithi130.stdout:1/495: rename db/d1b/d1f to db/d1b/da3 0 2024-05-13T22:48:33.963 INFO:tasks.workunit.client.1.smithi130.stdout:6/534: read d4/d15/d1d/fa5 [1986852,18957] 0 2024-05-13T22:48:33.965 INFO:tasks.workunit.client.1.smithi130.stdout:4/521: symlink d8/dc/l9f 0 2024-05-13T22:48:33.969 INFO:tasks.workunit.client.1.smithi130.stdout:2/527: rename d4/d49/d42/d4b/cb2 to d4/db/dc/d18/d2a/d30/d88/d99/d9a/cb8 0 2024-05-13T22:48:33.970 INFO:tasks.workunit.client.1.smithi130.stdout:2/528: dread - d4/db/dc/d18/d2a/d66/f7f zero size 2024-05-13T22:48:33.972 INFO:tasks.workunit.client.1.smithi130.stdout:5/504: mknod d6/d7/d19/ca8 0 2024-05-13T22:48:33.977 INFO:tasks.workunit.client.1.smithi130.stdout:8/486: mkdir d1/d2/db/d15/d16/d54/d58/dab 0 2024-05-13T22:48:33.982 INFO:tasks.workunit.client.1.smithi130.stdout:1/496: mkdir db/d1b/da3/d49/d56/da4 0 2024-05-13T22:48:33.985 INFO:tasks.workunit.client.1.smithi130.stdout:9/580: creat d1/d96/fb8 x:0 0 0 2024-05-13T22:48:33.986 INFO:tasks.workunit.client.1.smithi130.stdout:6/535: truncate d4/f9 2198860 0 2024-05-13T22:48:33.988 INFO:tasks.workunit.client.1.smithi130.stdout:9/581: chown d1/d7/d23/d28/d73/d8d/f8f 1874792 1 2024-05-13T22:48:33.989 INFO:tasks.workunit.client.1.smithi130.stdout:1/497: dwrite db/d9c/f8a [0,4194304] 0 2024-05-13T22:48:33.990 INFO:tasks.workunit.client.1.smithi130.stdout:1/498: dread - db/d1b/da3/d2a/d38/d46/d6d/f7b zero size 2024-05-13T22:48:33.993 INFO:tasks.workunit.client.1.smithi130.stdout:2/529: mkdir d4/db/dc/d18/d2a/d66/db9 0 2024-05-13T22:48:33.999 INFO:tasks.workunit.client.1.smithi130.stdout:6/536: truncate d4/d15/d1f/f2f 483480 0 2024-05-13T22:48:33.999 INFO:tasks.workunit.client.1.smithi130.stdout:5/505: dwrite d6/f5f [0,4194304] 0 2024-05-13T22:48:34.001 INFO:tasks.workunit.client.1.smithi130.stdout:5/506: fsync d6/d1a/f9e 0 2024-05-13T22:48:34.001 INFO:tasks.workunit.client.1.smithi130.stdout:6/537: write d4/d15/d1d/d28/d75/f80 [354940,87535] 0 2024-05-13T22:48:34.002 INFO:tasks.workunit.client.1.smithi130.stdout:4/522: mkdir d8/dc/d35/d33/d44/d61/da0 0 2024-05-13T22:48:34.004 INFO:tasks.workunit.client.1.smithi130.stdout:0/566: rename d0/d7/d29/c5e to d0/d7/dd/d17/d1f/d2c/d5a/cbf 0 2024-05-13T22:48:34.004 INFO:tasks.workunit.client.1.smithi130.stdout:0/567: fsync d0/f1e 0 2024-05-13T22:48:34.011 INFO:tasks.workunit.client.1.smithi130.stdout:3/503: creat d8/d35/fb0 x:0 0 0 2024-05-13T22:48:34.013 INFO:tasks.workunit.client.1.smithi130.stdout:2/530: mknod d4/db/dc/d18/d2a/d30/d88/d99/cba 0 2024-05-13T22:48:34.013 INFO:tasks.workunit.client.1.smithi130.stdout:7/519: getdents d7/da/d1d/d3d/d46 0 2024-05-13T22:48:34.015 INFO:tasks.workunit.client.1.smithi130.stdout:9/582: dread d1/d7/fac [0,4194304] 0 2024-05-13T22:48:34.015 INFO:tasks.workunit.client.1.smithi130.stdout:2/531: truncate d4/db/dc/d18/f98 879214 0 2024-05-13T22:48:34.020 INFO:tasks.workunit.client.1.smithi130.stdout:3/504: rmdir d8/d1a/d55 39 2024-05-13T22:48:34.026 INFO:tasks.workunit.client.1.smithi130.stdout:8/487: rename d1/d2/db/d15/d18/d42/d53/l70 to d1/d2/db/d15/d16/d54/lac 0 2024-05-13T22:48:34.028 INFO:tasks.workunit.client.1.smithi130.stdout:0/568: creat d0/d7/dd/d2e/da8/fc0 x:0 0 0 2024-05-13T22:48:34.030 INFO:tasks.workunit.client.1.smithi130.stdout:3/505: creat d8/dc/d15/fb1 x:0 0 0 2024-05-13T22:48:34.031 INFO:tasks.workunit.client.1.smithi130.stdout:3/506: stat d8/d1a/d1c/fa0 0 2024-05-13T22:48:34.032 INFO:tasks.workunit.client.1.smithi130.stdout:7/520: dread d7/da/d1d/d3d/d46/f4b [0,4194304] 0 2024-05-13T22:48:34.037 INFO:tasks.workunit.client.1.smithi130.stdout:7/521: dwrite d7/da/d1d/d3d/d46/d92/fad [0,4194304] 0 2024-05-13T22:48:34.040 INFO:tasks.workunit.client.1.smithi130.stdout:3/507: readlink d8/l3f 0 2024-05-13T22:48:34.041 INFO:tasks.workunit.client.1.smithi130.stdout:2/532: truncate d4/db/dc/d18/d2a/f7d 3905186 0 2024-05-13T22:48:34.042 INFO:tasks.workunit.client.1.smithi130.stdout:5/507: link d6/dd/d18/d6e/d82/d4b/c9d d6/ca9 0 2024-05-13T22:48:34.044 INFO:tasks.workunit.client.1.smithi130.stdout:2/533: chown d4/db/dc/d18/d2a/f97 159152770 1 2024-05-13T22:48:34.046 INFO:tasks.workunit.client.1.smithi130.stdout:8/488: symlink d1/dc/lad 0 2024-05-13T22:48:34.047 INFO:tasks.workunit.client.1.smithi130.stdout:0/569: dwrite d0/d7/dd/d17/fae [0,4194304] 0 2024-05-13T22:48:34.047 INFO:tasks.workunit.client.1.smithi130.stdout:2/534: read - d4/d49/d42/d4b/fa3 zero size 2024-05-13T22:48:34.048 INFO:tasks.workunit.client.1.smithi130.stdout:1/499: getdents db/d1b/da3/d5e 0 2024-05-13T22:48:34.050 INFO:tasks.workunit.client.1.smithi130.stdout:2/535: chown d4/db/dc/d18/d2a/d66 26919 1 2024-05-13T22:48:34.051 INFO:tasks.workunit.client.1.smithi130.stdout:0/570: fdatasync d0/d7/dd/d17/d1f/d2c/d45/d57/f58 0 2024-05-13T22:48:34.052 INFO:tasks.workunit.client.1.smithi130.stdout:5/508: dwrite d6/d7/f43 [0,4194304] 0 2024-05-13T22:48:34.052 INFO:tasks.workunit.client.1.smithi130.stdout:1/500: chown db/d1b/da3/d2a/d4c/c5d 1603596 1 2024-05-13T22:48:34.056 INFO:tasks.workunit.client.1.smithi130.stdout:2/536: dread d4/db/dc/f4e [0,4194304] 0 2024-05-13T22:48:34.062 INFO:tasks.workunit.client.1.smithi130.stdout:6/538: rename d4/d15/d1f/d46/c48 to d4/d15/cbb 0 2024-05-13T22:48:34.065 INFO:tasks.workunit.client.1.smithi130.stdout:7/522: creat d7/da/d1d/d5d/da6/d39/da9/fb0 x:0 0 0 2024-05-13T22:48:34.070 INFO:tasks.workunit.client.1.smithi130.stdout:3/508: dread d8/dc/f22 [0,4194304] 0 2024-05-13T22:48:34.073 INFO:tasks.workunit.client.1.smithi130.stdout:0/571: creat d0/d7/dd/d17/d52/fc1 x:0 0 0 2024-05-13T22:48:34.074 INFO:tasks.workunit.client.1.smithi130.stdout:5/509: creat d6/d7/d19/d4d/d6a/faa x:0 0 0 2024-05-13T22:48:34.076 INFO:tasks.workunit.client.1.smithi130.stdout:5/510: dread - d6/d1a/f9e zero size 2024-05-13T22:48:34.076 INFO:tasks.workunit.client.1.smithi130.stdout:2/537: chown d4/d49/d48/l60 10084 1 2024-05-13T22:48:34.076 INFO:tasks.workunit.client.1.smithi130.stdout:8/489: mkdir d1/d2/dae 0 2024-05-13T22:48:34.077 INFO:tasks.workunit.client.1.smithi130.stdout:2/538: fsync d4/fa5 0 2024-05-13T22:48:34.078 INFO:tasks.workunit.client.1.smithi130.stdout:4/523: rename f6 to d8/dc/d26/fa1 0 2024-05-13T22:48:34.083 INFO:tasks.workunit.client.1.smithi130.stdout:3/509: dread d8/d1a/d1c/f5d [0,4194304] 0 2024-05-13T22:48:34.086 INFO:tasks.workunit.client.1.smithi130.stdout:0/572: mknod d0/d7/dd/d2e/d9e/da7/cc2 0 2024-05-13T22:48:34.088 INFO:tasks.workunit.client.1.smithi130.stdout:5/511: symlink d6/dd/lab 0 2024-05-13T22:48:34.089 INFO:tasks.workunit.client.1.smithi130.stdout:2/539: mkdir d4/d49/d48/d4a/d64/dbb 0 2024-05-13T22:48:34.092 INFO:tasks.workunit.client.1.smithi130.stdout:5/512: dwrite d6/f7c [0,4194304] 0 2024-05-13T22:48:34.094 INFO:tasks.workunit.client.1.smithi130.stdout:9/583: sync 2024-05-13T22:48:34.098 INFO:tasks.workunit.client.1.smithi130.stdout:1/501: rename db/d1b/da3/d2a/d35/l85 to db/d58/la5 0 2024-05-13T22:48:34.098 INFO:tasks.workunit.client.1.smithi130.stdout:9/584: dread - d1/f4f zero size 2024-05-13T22:48:34.099 INFO:tasks.workunit.client.1.smithi130.stdout:7/523: mkdir d7/da/d1d/db1 0 2024-05-13T22:48:34.100 INFO:tasks.workunit.client.1.smithi130.stdout:3/510: readlink d8/dc/d7b/l7e 0 2024-05-13T22:48:34.100 INFO:tasks.workunit.client.1.smithi130.stdout:7/524: chown d7/da/d1d/d3d/d46/l60 30991 1 2024-05-13T22:48:34.101 INFO:tasks.workunit.client.1.smithi130.stdout:6/539: truncate d4/d15/d1d/d28/d4e/f99 2840624 0 2024-05-13T22:48:34.102 INFO:tasks.workunit.client.1.smithi130.stdout:6/540: stat d4/d15/d1f/d46/d50/f8a 0 2024-05-13T22:48:34.103 INFO:tasks.workunit.client.1.smithi130.stdout:0/573: mkdir d0/d7/dd/d2e/d9e/da7/dc3 0 2024-05-13T22:48:34.105 INFO:tasks.workunit.client.1.smithi130.stdout:4/524: write d8/df/f20 [4976771,2723] 0 2024-05-13T22:48:34.106 INFO:tasks.workunit.client.1.smithi130.stdout:2/540: chown d4/d49/d48/d4a/l8b 4501139 1 2024-05-13T22:48:34.107 INFO:tasks.workunit.client.1.smithi130.stdout:8/490: mknod d1/d2/db/d15/d16/d54/d58/dab/caf 0 2024-05-13T22:48:34.107 INFO:tasks.workunit.client.1.smithi130.stdout:8/491: stat d1/d2/db/d15/d18/f3e 0 2024-05-13T22:48:34.108 INFO:tasks.workunit.client.1.smithi130.stdout:8/492: dread - d1/d2/db/f4a zero size 2024-05-13T22:48:34.109 INFO:tasks.workunit.client.1.smithi130.stdout:8/493: fsync d1/dc/f44 0 2024-05-13T22:48:34.109 INFO:tasks.workunit.client.1.smithi130.stdout:9/585: creat d1/d7/d23/d28/d32/d65/fb9 x:0 0 0 2024-05-13T22:48:34.109 INFO:tasks.workunit.client.1.smithi130.stdout:3/511: mknod d8/d1a/d1c/cb2 0 2024-05-13T22:48:34.111 INFO:tasks.workunit.client.1.smithi130.stdout:3/512: stat d8/dc/d42/f74 0 2024-05-13T22:48:34.112 INFO:tasks.workunit.client.1.smithi130.stdout:3/513: stat c7 0 2024-05-13T22:48:34.113 INFO:tasks.workunit.client.1.smithi130.stdout:1/502: dwrite db/d1b/da3/d49/d56/d7f/f83 [0,4194304] 0 2024-05-13T22:48:34.114 INFO:tasks.workunit.client.1.smithi130.stdout:1/503: chown db/d82 222789565 1 2024-05-13T22:48:34.117 INFO:tasks.workunit.client.1.smithi130.stdout:1/504: read db/d1b/da3/d2a/f5a [1962228,6231] 0 2024-05-13T22:48:34.119 INFO:tasks.workunit.client.1.smithi130.stdout:0/574: dwrite d0/f8 [0,4194304] 0 2024-05-13T22:48:34.121 INFO:tasks.workunit.client.1.smithi130.stdout:7/525: write d7/da/d1d/d5d/da6/d39/f7b [5091574,71554] 0 2024-05-13T22:48:34.121 INFO:tasks.workunit.client.1.smithi130.stdout:8/494: rmdir d1/d2/db/d15/d18/d42/d53 39 2024-05-13T22:48:34.122 INFO:tasks.workunit.client.1.smithi130.stdout:9/586: readlink d1/d7/d23/d28/d32/d45/l4c 0 2024-05-13T22:48:34.127 INFO:tasks.workunit.client.1.smithi130.stdout:6/541: truncate d4/d15/d1f/d46/d50/f51 3567259 0 2024-05-13T22:48:34.128 INFO:tasks.workunit.client.1.smithi130.stdout:8/495: dwrite d1/d2/db/d15/d18/f4c [0,4194304] 0 2024-05-13T22:48:34.130 INFO:tasks.workunit.client.1.smithi130.stdout:4/525: link d8/dc/d35/d33/d44/f54 d8/dc/d35/d33/d3f/fa2 0 2024-05-13T22:48:34.133 INFO:tasks.workunit.client.1.smithi130.stdout:5/513: creat d6/fac x:0 0 0 2024-05-13T22:48:34.134 INFO:tasks.workunit.client.1.smithi130.stdout:5/514: stat d6/d7/d19/d4d/fa3 0 2024-05-13T22:48:34.135 INFO:tasks.workunit.client.1.smithi130.stdout:5/515: chown d6/d1a/f2a 155127856 1 2024-05-13T22:48:34.135 INFO:tasks.workunit.client.1.smithi130.stdout:1/505: symlink db/d9c/la6 0 2024-05-13T22:48:34.135 INFO:tasks.workunit.client.1.smithi130.stdout:5/516: readlink d6/dd/d18/l2e 0 2024-05-13T22:48:34.140 INFO:tasks.workunit.client.1.smithi130.stdout:9/587: rmdir d1/d7/d23 39 2024-05-13T22:48:34.142 INFO:tasks.workunit.client.1.smithi130.stdout:7/526: mknod d7/da/d1d/d5d/da6/d9/d66/cb2 0 2024-05-13T22:48:34.143 INFO:tasks.workunit.client.1.smithi130.stdout:3/514: symlink d8/lb3 0 2024-05-13T22:48:34.144 INFO:tasks.workunit.client.1.smithi130.stdout:6/542: symlink d4/d15/d1f/d46/lbc 0 2024-05-13T22:48:34.145 INFO:tasks.workunit.client.1.smithi130.stdout:8/496: readlink d1/d2/db/d15/d18/d42/d53/d56/l83 0 2024-05-13T22:48:34.145 INFO:tasks.workunit.client.1.smithi130.stdout:2/541: creat d4/db/dc/fbc x:0 0 0 2024-05-13T22:48:34.146 INFO:tasks.workunit.client.1.smithi130.stdout:2/542: stat d4/db/dc/d18/d2a/l2b 0 2024-05-13T22:48:34.146 INFO:tasks.workunit.client.1.smithi130.stdout:4/526: rename d8/dc/d35/d33/d44/f74 to d8/dc/d35/d33/d44/fa3 0 2024-05-13T22:48:34.147 INFO:tasks.workunit.client.1.smithi130.stdout:1/506: chown db/cf 6612268 1 2024-05-13T22:48:34.147 INFO:tasks.workunit.client.1.smithi130.stdout:5/517: stat d6/d7/d19/d4d/l5c 0 2024-05-13T22:48:34.149 INFO:tasks.workunit.client.1.smithi130.stdout:1/507: truncate db/d1b/da3/d2a/d38/d46/d6d/f7b 124438 0 2024-05-13T22:48:34.153 INFO:tasks.workunit.client.1.smithi130.stdout:3/515: truncate d8/dc/d15/f9a 36813 0 2024-05-13T22:48:34.153 INFO:tasks.workunit.client.1.smithi130.stdout:4/527: chown d8/dc/d80/l49 1035392474 1 2024-05-13T22:48:34.156 INFO:tasks.workunit.client.1.smithi130.stdout:0/575: rename d0/d7/dd/d17/d1f/d2c/d45/c56 to d0/d7/cc4 0 2024-05-13T22:48:34.158 INFO:tasks.workunit.client.1.smithi130.stdout:1/508: mknod db/d82/ca7 0 2024-05-13T22:48:34.162 INFO:tasks.workunit.client.1.smithi130.stdout:2/543: symlink d4/db/dc/d18/d2a/d66/db9/lbd 0 2024-05-13T22:48:34.164 INFO:tasks.workunit.client.1.smithi130.stdout:5/518: rename d6/d1a/l93 to d6/d7/d19/d4d/d53/lad 0 2024-05-13T22:48:34.165 INFO:tasks.workunit.client.1.smithi130.stdout:0/576: dwrite d0/d7/dd/d17/d1f/d2c/d45/d6b/f7c [0,4194304] 0 2024-05-13T22:48:34.167 INFO:tasks.workunit.client.1.smithi130.stdout:9/588: mknod d1/d7/cba 0 2024-05-13T22:48:34.167 INFO:tasks.workunit.client.1.smithi130.stdout:5/519: readlink d6/d7/d32/l4a 0 2024-05-13T22:48:34.168 INFO:tasks.workunit.client.1.smithi130.stdout:1/509: dwrite db/d1b/da3/f53 [0,4194304] 0 2024-05-13T22:48:34.169 INFO:tasks.workunit.client.1.smithi130.stdout:7/527: dwrite d7/da/d1d/d5d/da6/d39/fa8 [0,4194304] 0 2024-05-13T22:48:34.173 INFO:tasks.workunit.client.1.smithi130.stdout:8/497: creat d1/d2/db/d15/d18/fb0 x:0 0 0 2024-05-13T22:48:34.177 INFO:tasks.workunit.client.1.smithi130.stdout:0/577: dwrite d0/f8 [0,4194304] 0 2024-05-13T22:48:34.178 INFO:tasks.workunit.client.1.smithi130.stdout:2/544: fsync d4/db/f63 0 2024-05-13T22:48:34.179 INFO:tasks.workunit.client.1.smithi130.stdout:2/545: chown d4/db/dc/d18/d2a/d30/f9c 989746850 1 2024-05-13T22:48:34.181 INFO:tasks.workunit.client.1.smithi130.stdout:7/528: creat d7/da/d1d/d5d/da6/d5b/fb3 x:0 0 0 2024-05-13T22:48:34.183 INFO:tasks.workunit.client.1.smithi130.stdout:7/529: fdatasync d7/da/d1d/d5d/da6/d39/f7b 0 2024-05-13T22:48:34.185 INFO:tasks.workunit.client.1.smithi130.stdout:0/578: dwrite d0/d7/dd/d17/d1f/d2c/d45/d6b/f7c [0,4194304] 0 2024-05-13T22:48:34.187 INFO:tasks.workunit.client.1.smithi130.stdout:5/520: mknod d6/dd/d18/d6e/d82/d71/cae 0 2024-05-13T22:48:34.188 INFO:tasks.workunit.client.1.smithi130.stdout:2/546: creat d4/d49/d48/d4a/d64/fbe x:0 0 0 2024-05-13T22:48:34.189 INFO:tasks.workunit.client.1.smithi130.stdout:0/579: write d0/d7/dd/d17/d1f/d2c/d45/d57/f85 [814150,7823] 0 2024-05-13T22:48:34.189 INFO:tasks.workunit.client.1.smithi130.stdout:5/521: fsync d6/d7/d19/f9a 0 2024-05-13T22:48:34.191 INFO:tasks.workunit.client.1.smithi130.stdout:1/510: truncate db/d1b/da3/d2a/f41 669199 0 2024-05-13T22:48:34.196 INFO:tasks.workunit.client.1.smithi130.stdout:4/528: getdents d8/dc/d80/d96 0 2024-05-13T22:48:34.201 INFO:tasks.workunit.client.1.smithi130.stdout:8/498: dread d1/dc/d5d/f87 [0,4194304] 0 2024-05-13T22:48:34.204 INFO:tasks.workunit.client.1.smithi130.stdout:5/522: dread d6/dd/d18/d6e/d76/f80 [0,4194304] 0 2024-05-13T22:48:34.205 INFO:tasks.workunit.client.1.smithi130.stdout:5/523: chown d6/d7/d19/d4d/f34 548028 1 2024-05-13T22:48:34.206 INFO:tasks.workunit.client.1.smithi130.stdout:5/524: fdatasync d6/d7/d19/f52 0 2024-05-13T22:48:34.207 INFO:tasks.workunit.client.1.smithi130.stdout:5/525: truncate d6/d1a/d92/f96 798509 0 2024-05-13T22:48:34.210 INFO:tasks.workunit.client.1.smithi130.stdout:7/530: link d7/da/f33 d7/da/d1d/d5d/da6/d39/fb4 0 2024-05-13T22:48:34.211 INFO:tasks.workunit.client.1.smithi130.stdout:7/531: truncate d7/da/d1d/f6f 32613 0 2024-05-13T22:48:34.212 INFO:tasks.workunit.client.1.smithi130.stdout:4/529: mknod d8/dc/d35/d33/d44/d61/ca4 0 2024-05-13T22:48:34.212 INFO:tasks.workunit.client.1.smithi130.stdout:8/499: fdatasync d1/d2/db/d15/d16/f8f 0 2024-05-13T22:48:34.213 INFO:tasks.workunit.client.1.smithi130.stdout:9/589: symlink d1/d7/d23/d3d/lbb 0 2024-05-13T22:48:34.213 INFO:tasks.workunit.client.1.smithi130.stdout:8/500: write d1/d2/f6 [4785616,49468] 0 2024-05-13T22:48:34.214 INFO:tasks.workunit.client.1.smithi130.stdout:9/590: dread - d1/d7/d23/d28/d2d/d2f/d63/f77 zero size 2024-05-13T22:48:34.215 INFO:tasks.workunit.client.1.smithi130.stdout:8/501: stat d1/d2/db/d15/d18/d1a 0 2024-05-13T22:48:34.215 INFO:tasks.workunit.client.1.smithi130.stdout:5/526: rmdir d6/d1a/d72 39 2024-05-13T22:48:34.217 INFO:tasks.workunit.client.1.smithi130.stdout:5/527: dread d6/dd/d18/f2b [0,4194304] 0 2024-05-13T22:48:34.219 INFO:tasks.workunit.client.1.smithi130.stdout:1/511: dread db/d1b/da3/d2a/f5a [0,4194304] 0 2024-05-13T22:48:34.219 INFO:tasks.workunit.client.1.smithi130.stdout:6/543: sync 2024-05-13T22:48:34.219 INFO:tasks.workunit.client.1.smithi130.stdout:3/516: dread d8/d35/f3d [0,4194304] 0 2024-05-13T22:48:34.220 INFO:tasks.workunit.client.1.smithi130.stdout:1/512: fsync db/f6a 0 2024-05-13T22:48:34.221 INFO:tasks.workunit.client.1.smithi130.stdout:1/513: write db/d1b/da3/d2a/d38/d46/d6d/f7b [685090,123918] 0 2024-05-13T22:48:34.221 INFO:tasks.workunit.client.1.smithi130.stdout:1/514: chown db/f1d 6474042 1 2024-05-13T22:48:34.223 INFO:tasks.workunit.client.1.smithi130.stdout:1/515: read db/f1d [2044115,105974] 0 2024-05-13T22:48:34.224 INFO:tasks.workunit.client.1.smithi130.stdout:1/516: fdatasync db/d1b/da3/d49/f54 0 2024-05-13T22:48:34.225 INFO:tasks.workunit.client.1.smithi130.stdout:8/502: creat d1/d2/db/d15/d16/fb1 x:0 0 0 2024-05-13T22:48:34.226 INFO:tasks.workunit.client.1.smithi130.stdout:2/547: rename d4/d49 to d4/db/dbf 0 2024-05-13T22:48:34.234 INFO:tasks.workunit.client.1.smithi130.stdout:2/548: dwrite d4/db/dbf/d48/d4a/f52 [4194304,4194304] 0 2024-05-13T22:48:34.236 INFO:tasks.workunit.client.1.smithi130.stdout:0/580: link d0/d7/dd/d17/d1f/d2c/d5a/cbf d0/d7/dd/d2e/d9e/da7/dc3/cc5 0 2024-05-13T22:48:34.238 INFO:tasks.workunit.client.1.smithi130.stdout:1/517: dread db/d1b/da3/d49/f55 [0,4194304] 0 2024-05-13T22:48:34.241 INFO:tasks.workunit.client.1.smithi130.stdout:8/503: rename d1/d2/dae to d1/d2/db/d15/d16/d54/db2 0 2024-05-13T22:48:34.243 INFO:tasks.workunit.client.1.smithi130.stdout:7/532: creat d7/da/d1d/d42/fb5 x:0 0 0 2024-05-13T22:48:34.245 INFO:tasks.workunit.client.1.smithi130.stdout:4/530: link d8/dc/d26/f36 d8/fa5 0 2024-05-13T22:48:34.246 INFO:tasks.workunit.client.1.smithi130.stdout:9/591: write d1/d7/d23/d28/d2d/f46 [1503910,91556] 0 2024-05-13T22:48:34.247 INFO:tasks.workunit.client.1.smithi130.stdout:5/528: mkdir d6/d1a/daf 0 2024-05-13T22:48:34.249 INFO:tasks.workunit.client.1.smithi130.stdout:3/517: creat d8/d1a/d1c/fb4 x:0 0 0 2024-05-13T22:48:34.251 INFO:tasks.workunit.client.1.smithi130.stdout:2/549: mknod d4/db/cc0 0 2024-05-13T22:48:34.252 INFO:tasks.workunit.client.1.smithi130.stdout:6/544: write d4/d16/f39 [62741,37381] 0 2024-05-13T22:48:34.253 INFO:tasks.workunit.client.1.smithi130.stdout:2/550: stat d4/f9 0 2024-05-13T22:48:34.256 INFO:tasks.workunit.client.1.smithi130.stdout:6/545: dread d4/d15/d1f/d46/d50/fa7 [0,4194304] 0 2024-05-13T22:48:34.259 INFO:tasks.workunit.client.1.smithi130.stdout:8/504: rmdir d1/dc/d5d 39 2024-05-13T22:48:34.262 INFO:tasks.workunit.client.1.smithi130.stdout:4/531: mkdir d8/dc/d80/da6 0 2024-05-13T22:48:34.263 INFO:tasks.workunit.client.1.smithi130.stdout:8/505: dread d1/d2/db/d15/f74 [0,4194304] 0 2024-05-13T22:48:34.264 INFO:tasks.workunit.client.1.smithi130.stdout:9/592: chown d1/d15/c72 106430732 1 2024-05-13T22:48:34.264 INFO:tasks.workunit.client.1.smithi130.stdout:5/529: write d6/f1f [641319,74012] 0 2024-05-13T22:48:34.265 INFO:tasks.workunit.client.1.smithi130.stdout:3/518: rename d8/dc/d15/c4c to d8/dc/d7b/da3/cb5 0 2024-05-13T22:48:34.269 INFO:tasks.workunit.client.1.smithi130.stdout:2/551: creat d4/db/dc/d18/d2a/d66/db9/fc1 x:0 0 0 2024-05-13T22:48:34.269 INFO:tasks.workunit.client.1.smithi130.stdout:8/506: dwrite d1/d2/db/d15/d18/d42/f6d [0,4194304] 0 2024-05-13T22:48:34.269 INFO:tasks.workunit.client.1.smithi130.stdout:1/518: write db/d1b/da3/d49/f61 [120959,65860] 0 2024-05-13T22:48:34.270 INFO:tasks.workunit.client.1.smithi130.stdout:0/581: mknod d0/d7/dd/d2e/cc6 0 2024-05-13T22:48:34.271 INFO:tasks.workunit.client.1.smithi130.stdout:1/519: stat db/d1b/da3/d2a/d4c/c5d 0 2024-05-13T22:48:34.272 INFO:tasks.workunit.client.1.smithi130.stdout:1/520: dread - db/f6a zero size 2024-05-13T22:48:34.275 INFO:tasks.workunit.client.1.smithi130.stdout:7/533: link d7/da/d1d/d3d/d46/f4b d7/da/d1d/d42/d43/fb6 0 2024-05-13T22:48:34.275 INFO:tasks.workunit.client.1.smithi130.stdout:1/521: write db/d1b/da3/d2a/d38/d46/f71 [886174,128746] 0 2024-05-13T22:48:34.277 INFO:tasks.workunit.client.1.smithi130.stdout:7/534: fsync d7/da/d1d/d5d/da6/d5b/f6d 0 2024-05-13T22:48:34.278 INFO:tasks.workunit.client.1.smithi130.stdout:3/519: mknod d8/dc/d42/cb6 0 2024-05-13T22:48:34.280 INFO:tasks.workunit.client.1.smithi130.stdout:6/546: mknod d4/d15/d1f/cbd 0 2024-05-13T22:48:34.281 INFO:tasks.workunit.client.1.smithi130.stdout:8/507: symlink d1/d2/db/lb3 0 2024-05-13T22:48:34.281 INFO:tasks.workunit.client.1.smithi130.stdout:2/552: rmdir d4/db/dbf/d48/d4a/d64 39 2024-05-13T22:48:34.283 INFO:tasks.workunit.client.1.smithi130.stdout:0/582: rename d0/d7/dd/f32 to d0/d7/dd/d17/d1f/d2c/d5a/fc7 0 2024-05-13T22:48:34.284 INFO:tasks.workunit.client.1.smithi130.stdout:5/530: mkdir d6/dd/d18/d6e/d82/db0 0 2024-05-13T22:48:34.286 INFO:tasks.workunit.client.1.smithi130.stdout:6/547: unlink d4/d15/c27 0 2024-05-13T22:48:34.287 INFO:tasks.workunit.client.1.smithi130.stdout:2/553: symlink d4/db/dbf/d48/d4a/d76/lc2 0 2024-05-13T22:48:34.289 INFO:tasks.workunit.client.1.smithi130.stdout:5/531: dread d6/dd/d18/d6e/d82/d56/f5d [0,4194304] 0 2024-05-13T22:48:34.289 INFO:tasks.workunit.client.1.smithi130.stdout:3/520: rename d8/d35/f80 to d8/dc/d7b/da3/daf/fb7 0 2024-05-13T22:48:34.290 INFO:tasks.workunit.client.1.smithi130.stdout:8/508: rename d1/d2/db/d15/d18 to d1/d2/db/d15/d18/d1a/db4 22 2024-05-13T22:48:34.290 INFO:tasks.workunit.client.1.smithi130.stdout:0/583: dwrite d0/d7/dd/d17/d1f/d2c/d5a/f66 [0,4194304] 0 2024-05-13T22:48:34.290 INFO:tasks.workunit.client.1.smithi130.stdout:2/554: dread d4/db/dc/d18/f7e [0,4194304] 0 2024-05-13T22:48:34.291 INFO:tasks.workunit.client.1.smithi130.stdout:3/521: fsync d8/d1a/d1c/f6e 0 2024-05-13T22:48:34.293 INFO:tasks.workunit.client.1.smithi130.stdout:7/535: creat d7/da/d1d/d42/d43/da7/fb7 x:0 0 0 2024-05-13T22:48:34.294 INFO:tasks.workunit.client.1.smithi130.stdout:7/536: dread - d7/da/d1d/d5d/da6/d9/d66/f7a zero size 2024-05-13T22:48:34.295 INFO:tasks.workunit.client.1.smithi130.stdout:4/532: getdents d8/dc 0 2024-05-13T22:48:34.299 INFO:tasks.workunit.client.1.smithi130.stdout:6/548: creat d4/d15/d1d/d28/d75/d7d/fbe x:0 0 0 2024-05-13T22:48:34.302 INFO:tasks.workunit.client.1.smithi130.stdout:1/522: link db/d4a/f4b db/d1b/da3/d2a/d38/fa8 0 2024-05-13T22:48:34.303 INFO:tasks.workunit.client.1.smithi130.stdout:5/532: dread d6/d7/f22 [0,4194304] 0 2024-05-13T22:48:34.304 INFO:tasks.workunit.client.1.smithi130.stdout:5/533: write d6/f1f [3915063,72836] 0 2024-05-13T22:48:34.306 INFO:tasks.workunit.client.1.smithi130.stdout:5/534: truncate d6/dd/d18/f36 5132051 0 2024-05-13T22:48:34.310 INFO:tasks.workunit.client.1.smithi130.stdout:0/584: dread d0/d7/f23 [0,4194304] 0 2024-05-13T22:48:34.310 INFO:tasks.workunit.client.1.smithi130.stdout:3/522: creat d8/dc/d7b/da3/daf/fb8 x:0 0 0 2024-05-13T22:48:34.311 INFO:tasks.workunit.client.1.smithi130.stdout:2/555: symlink d4/db/dc/d18/d2a/d30/d88/d99/lc3 0 2024-05-13T22:48:34.314 INFO:tasks.workunit.client.1.smithi130.stdout:7/537: rename d7/da/d86/l9c to d7/da/d86/lb8 0 2024-05-13T22:48:34.315 INFO:tasks.workunit.client.1.smithi130.stdout:0/585: dwrite d0/f8 [4194304,4194304] 0 2024-05-13T22:48:34.315 INFO:tasks.workunit.client.1.smithi130.stdout:4/533: creat d8/dc/d35/d33/d44/d77/d94/fa7 x:0 0 0 2024-05-13T22:48:34.317 INFO:tasks.workunit.client.1.smithi130.stdout:7/538: read d7/da/d1d/d5d/da6/f4c [480068,21698] 0 2024-05-13T22:48:34.317 INFO:tasks.workunit.client.1.smithi130.stdout:4/534: stat d8/dc/d35/d33/d44/f4b 0 2024-05-13T22:48:34.318 INFO:tasks.workunit.client.1.smithi130.stdout:4/535: fsync d8/dc/d26/f75 0 2024-05-13T22:48:34.319 INFO:tasks.workunit.client.1.smithi130.stdout:9/593: getdents d1/d7/d69 0 2024-05-13T22:48:34.320 INFO:tasks.workunit.client.1.smithi130.stdout:4/536: dread - d8/dc/f86 zero size 2024-05-13T22:48:34.320 INFO:tasks.workunit.client.1.smithi130.stdout:5/535: unlink d6/d1a/f9e 0 2024-05-13T22:48:34.321 INFO:tasks.workunit.client.1.smithi130.stdout:4/537: chown d8/dc/d35/l37 11501449 1 2024-05-13T22:48:34.322 INFO:tasks.workunit.client.1.smithi130.stdout:8/509: creat d1/d9b/fb5 x:0 0 0 2024-05-13T22:48:34.323 INFO:tasks.workunit.client.1.smithi130.stdout:2/556: creat d4/db/dc/d18/d2a/d30/d88/d90/fc4 x:0 0 0 2024-05-13T22:48:34.324 INFO:tasks.workunit.client.1.smithi130.stdout:2/557: dread - d4/db/dc/d18/d2a/d66/db9/fc1 zero size 2024-05-13T22:48:34.326 INFO:tasks.workunit.client.1.smithi130.stdout:0/586: creat d0/d7/dd/d69/fc8 x:0 0 0 2024-05-13T22:48:34.327 INFO:tasks.workunit.client.1.smithi130.stdout:1/523: mknod db/d1b/da3/d2a/ca9 0 2024-05-13T22:48:34.328 INFO:tasks.workunit.client.1.smithi130.stdout:7/539: chown d7/da/d1d/d3d/d46/d67/c25 423661 1 2024-05-13T22:48:34.329 INFO:tasks.workunit.client.1.smithi130.stdout:9/594: symlink d1/d7/d23/d3d/d43/lbc 0 2024-05-13T22:48:34.330 INFO:tasks.workunit.client.1.smithi130.stdout:5/536: symlink d6/dd/d18/d6e/d82/d71/lb1 0 2024-05-13T22:48:34.330 INFO:tasks.workunit.client.1.smithi130.stdout:1/524: dread db/d9c/f8a [0,4194304] 0 2024-05-13T22:48:34.331 INFO:tasks.workunit.client.1.smithi130.stdout:5/537: stat d6/dd/d18/l10 0 2024-05-13T22:48:34.333 INFO:tasks.workunit.client.1.smithi130.stdout:5/538: write d6/d1a/f48 [289619,22392] 0 2024-05-13T22:48:34.334 INFO:tasks.workunit.client.1.smithi130.stdout:2/558: mkdir d4/db/dc/d18/d2a/d30/d88/dc5 0 2024-05-13T22:48:34.334 INFO:tasks.workunit.client.1.smithi130.stdout:1/525: read db/d1b/da3/d49/f48 [7214131,60476] 0 2024-05-13T22:48:34.335 INFO:tasks.workunit.client.1.smithi130.stdout:4/538: mkdir d8/da8 0 2024-05-13T22:48:34.336 INFO:tasks.workunit.client.1.smithi130.stdout:8/510: mknod d1/d2/db/d15/d18/d42/cb6 0 2024-05-13T22:48:34.336 INFO:tasks.workunit.client.1.smithi130.stdout:9/595: dwrite d1/f4f [0,4194304] 0 2024-05-13T22:48:34.339 INFO:tasks.workunit.client.1.smithi130.stdout:1/526: write db/d1b/da3/d2a/d38/d46/d6d/f7b [1605398,110806] 0 2024-05-13T22:48:34.341 INFO:tasks.workunit.client.1.smithi130.stdout:1/527: stat db/d58/l90 0 2024-05-13T22:48:34.341 INFO:tasks.workunit.client.1.smithi130.stdout:9/596: readlink d1/d7/d23/d28/d32/l4e 0 2024-05-13T22:48:34.343 INFO:tasks.workunit.client.1.smithi130.stdout:9/597: chown d1/f1e 3624 1 2024-05-13T22:48:34.344 INFO:tasks.workunit.client.1.smithi130.stdout:5/539: dwrite d6/d7/f28 [0,4194304] 0 2024-05-13T22:48:34.346 INFO:tasks.workunit.client.1.smithi130.stdout:4/539: write d8/dc/d26/fa1 [7412411,119223] 0 2024-05-13T22:48:34.347 INFO:tasks.workunit.client.1.smithi130.stdout:8/511: dwrite d1/d2/db/f41 [0,4194304] 0 2024-05-13T22:48:34.348 INFO:tasks.workunit.client.1.smithi130.stdout:3/523: getdents d8/dc/d19/d2c/d61/d9c 0 2024-05-13T22:48:34.349 INFO:tasks.workunit.client.1.smithi130.stdout:8/512: readlink d1/d2/db/d15/d7e/d9f/laa 0 2024-05-13T22:48:34.357 INFO:tasks.workunit.client.1.smithi130.stdout:1/528: mkdir db/d1b/da3/d72/daa 0 2024-05-13T22:48:34.358 INFO:tasks.workunit.client.1.smithi130.stdout:3/524: dwrite d8/dc/d19/f96 [0,4194304] 0 2024-05-13T22:48:34.365 INFO:tasks.workunit.client.1.smithi130.stdout:8/513: mknod d1/d2/db/d15/d16/d54/d58/dab/cb7 0 2024-05-13T22:48:34.369 INFO:tasks.workunit.client.1.smithi130.stdout:3/525: mknod d8/d35/cb9 0 2024-05-13T22:48:34.369 INFO:tasks.workunit.client.1.smithi130.stdout:5/540: dwrite d6/d7/d32/f3d [0,4194304] 0 2024-05-13T22:48:34.371 INFO:tasks.workunit.client.1.smithi130.stdout:8/514: creat d1/d2/db/d15/d16/d54/fb8 x:0 0 0 2024-05-13T22:48:34.375 INFO:tasks.workunit.client.1.smithi130.stdout:2/559: rename d4/db/dc/d18/d2a/d30/d69 to d4/db/dbf/d48/d4a/dc6 0 2024-05-13T22:48:34.376 INFO:tasks.workunit.client.1.smithi130.stdout:4/540: getdents d8/dc/d80/d85 0 2024-05-13T22:48:34.376 INFO:tasks.workunit.client.1.smithi130.stdout:8/515: read d1/d2/db/d15/d18/f4e [4147124,35626] 0 2024-05-13T22:48:34.376 INFO:tasks.workunit.client.1.smithi130.stdout:4/541: dread - d8/dc/f86 zero size 2024-05-13T22:48:34.377 INFO:tasks.workunit.client.1.smithi130.stdout:4/542: chown d8/dc/f86 24896 1 2024-05-13T22:48:34.377 INFO:tasks.workunit.client.1.smithi130.stdout:8/516: write d1/d2/db/d15/d7e/d9f/fa0 [86716,103625] 0 2024-05-13T22:48:34.382 INFO:tasks.workunit.client.1.smithi130.stdout:1/529: creat db/d1b/da3/d2a/d38/fab x:0 0 0 2024-05-13T22:48:34.383 INFO:tasks.workunit.client.1.smithi130.stdout:1/530: chown db/d9c 234 1 2024-05-13T22:48:34.383 INFO:tasks.workunit.client.1.smithi130.stdout:3/526: dread - d8/d35/f48 zero size 2024-05-13T22:48:34.383 INFO:tasks.workunit.client.1.smithi130.stdout:5/541: creat d6/dd/d18/d6e/d76/d7f/fb2 x:0 0 0 2024-05-13T22:48:34.386 INFO:tasks.workunit.client.1.smithi130.stdout:6/549: sync 2024-05-13T22:48:34.386 INFO:tasks.workunit.client.1.smithi130.stdout:4/543: unlink d8/dc/d35/d33/d44/l99 0 2024-05-13T22:48:34.388 INFO:tasks.workunit.client.1.smithi130.stdout:8/517: mkdir d1/d2/db/d15/d16/d54/d58/db9 0 2024-05-13T22:48:34.391 INFO:tasks.workunit.client.1.smithi130.stdout:9/598: write d1/d7/d23/d28/d32/d45/f53 [2687944,23299] 0 2024-05-13T22:48:34.394 INFO:tasks.workunit.client.1.smithi130.stdout:2/560: creat d4/da8/fc7 x:0 0 0 2024-05-13T22:48:34.394 INFO:tasks.workunit.client.1.smithi130.stdout:5/542: creat d6/dd/d79/fb3 x:0 0 0 2024-05-13T22:48:34.395 INFO:tasks.workunit.client.1.smithi130.stdout:2/561: write d4/db/dc/fbc [300801,81894] 0 2024-05-13T22:48:34.395 INFO:tasks.workunit.client.1.smithi130.stdout:5/543: fdatasync d6/d7/d19/f52 0 2024-05-13T22:48:34.395 INFO:tasks.workunit.client.1.smithi130.stdout:2/562: fsync d4/f7 0 2024-05-13T22:48:34.401 INFO:tasks.workunit.client.1.smithi130.stdout:4/544: dwrite d8/f1f [0,4194304] 0 2024-05-13T22:48:34.401 INFO:tasks.workunit.client.1.smithi130.stdout:3/527: mkdir d8/dc/d19/d83/dba 0 2024-05-13T22:48:34.403 INFO:tasks.workunit.client.1.smithi130.stdout:4/545: chown d8/dc/d35/d33/d44/fa3 0 1 2024-05-13T22:48:34.404 INFO:tasks.workunit.client.1.smithi130.stdout:9/599: mknod d1/d7/d23/d28/d73/db2/cbd 0 2024-05-13T22:48:34.404 INFO:tasks.workunit.client.1.smithi130.stdout:9/600: chown d1/d15/f37 322922025 1 2024-05-13T22:48:34.405 INFO:tasks.workunit.client.1.smithi130.stdout:5/544: mknod d6/d2f/cb4 0 2024-05-13T22:48:34.409 INFO:tasks.workunit.client.1.smithi130.stdout:6/550: creat d4/d15/d1f/d3c/fbf x:0 0 0 2024-05-13T22:48:34.411 INFO:tasks.workunit.client.1.smithi130.stdout:8/518: rmdir d1/d2 39 2024-05-13T22:48:34.413 INFO:tasks.workunit.client.1.smithi130.stdout:1/531: creat db/fac x:0 0 0 2024-05-13T22:48:34.414 INFO:tasks.workunit.client.1.smithi130.stdout:3/528: write d8/dc/d7b/da3/daf/fb7 [1781419,23024] 0 2024-05-13T22:48:34.415 INFO:tasks.workunit.client.1.smithi130.stdout:4/546: creat d8/dc/d35/d33/d44/fa9 x:0 0 0 2024-05-13T22:48:34.417 INFO:tasks.workunit.client.1.smithi130.stdout:9/601: rename d1/d15/c72 to d1/d7/d23/d3d/d43/cbe 0 2024-05-13T22:48:34.417 INFO:tasks.workunit.client.1.smithi130.stdout:2/563: dwrite d4/db/dbf/d48/d4a/d64/faf [0,4194304] 0 2024-05-13T22:48:34.420 INFO:tasks.workunit.client.1.smithi130.stdout:6/551: rmdir d4/d15/d1d/d60/da2/da4 39 2024-05-13T22:48:34.425 INFO:tasks.workunit.client.1.smithi130.stdout:2/564: dwrite d4/db/dbf/d48/d4a/f52 [4194304,4194304] 0 2024-05-13T22:48:34.432 INFO:tasks.workunit.client.1.smithi130.stdout:3/529: mkdir d8/dc/d7b/dbb 0 2024-05-13T22:48:34.432 INFO:tasks.workunit.client.1.smithi130.stdout:4/547: unlink d8/dc/d29/c62 0 2024-05-13T22:48:34.433 INFO:tasks.workunit.client.1.smithi130.stdout:9/602: mknod d1/d7/d69/cbf 0 2024-05-13T22:48:34.434 INFO:tasks.workunit.client.1.smithi130.stdout:9/603: write d1/d7/d23/d28/d73/d8d/fa5 [897218,7527] 0 2024-05-13T22:48:34.439 INFO:tasks.workunit.client.1.smithi130.stdout:0/587: sync 2024-05-13T22:48:34.440 INFO:tasks.workunit.client.1.smithi130.stdout:8/519: rmdir d1 39 2024-05-13T22:48:34.442 INFO:tasks.workunit.client.1.smithi130.stdout:1/532: symlink db/d1b/lad 0 2024-05-13T22:48:34.443 INFO:tasks.workunit.client.1.smithi130.stdout:2/565: symlink d4/db/d26/d9e/lc8 0 2024-05-13T22:48:34.443 INFO:tasks.workunit.client.1.smithi130.stdout:0/588: dread d0/d7/dd/d17/d52/d5b/d5f/f70 [0,4194304] 0 2024-05-13T22:48:34.444 INFO:tasks.workunit.client.1.smithi130.stdout:0/589: write d0/f1e [2123885,121333] 0 2024-05-13T22:48:34.446 INFO:tasks.workunit.client.1.smithi130.stdout:9/604: dread d1/db/f35 [0,4194304] 0 2024-05-13T22:48:34.456 INFO:tasks.workunit.client.1.smithi130.stdout:4/548: creat d8/dc/d80/d96/faa x:0 0 0 2024-05-13T22:48:34.457 INFO:tasks.workunit.client.1.smithi130.stdout:6/552: chown d4/d15/d1f/d46/d52/d7b 1011193 1 2024-05-13T22:48:34.458 INFO:tasks.workunit.client.1.smithi130.stdout:4/549: write d8/dc/d35/d33/d44/d77/d94/fa7 [240863,77935] 0 2024-05-13T22:48:34.460 INFO:tasks.workunit.client.1.smithi130.stdout:1/533: rename db/d58/c8f to db/d1b/da3/d49/d56/d88/cae 0 2024-05-13T22:48:34.461 INFO:tasks.workunit.client.1.smithi130.stdout:1/534: read - db/d4a/f94 zero size 2024-05-13T22:48:34.463 INFO:tasks.workunit.client.1.smithi130.stdout:3/530: dwrite d8/f16 [0,4194304] 0 2024-05-13T22:48:34.465 INFO:tasks.workunit.client.1.smithi130.stdout:9/605: chown d1/d7/d23/d3d/d43/cbe 2750917 1 2024-05-13T22:48:34.468 INFO:tasks.workunit.client.1.smithi130.stdout:8/520: write d1/d2/db/d15/d18/d42/d53/d56/f9a [2034480,114686] 0 2024-05-13T22:48:34.471 INFO:tasks.workunit.client.1.smithi130.stdout:2/566: rename d4/db/dbf/d48/d4a/d64 to d4/da8/dc9 0 2024-05-13T22:48:34.474 INFO:tasks.workunit.client.1.smithi130.stdout:3/531: fsync d8/dc/d19/d27/f84 0 2024-05-13T22:48:34.475 INFO:tasks.workunit.client.1.smithi130.stdout:3/532: chown d8/dc/d19/d27/f84 47030 1 2024-05-13T22:48:34.477 INFO:tasks.workunit.client.1.smithi130.stdout:8/521: readlink d1/d2/db/d15/d18/l50 0 2024-05-13T22:48:34.480 INFO:tasks.workunit.client.1.smithi130.stdout:6/553: rename d4/f29 to d4/d15/d1d/d28/d75/da8/da0/fc0 0 2024-05-13T22:48:34.486 INFO:tasks.workunit.client.1.smithi130.stdout:3/533: unlink d8/dc/d19/d2c/d61/f69 0 2024-05-13T22:48:34.488 INFO:tasks.workunit.client.1.smithi130.stdout:3/534: write d8/dc/d19/d27/f84 [3081654,57801] 0 2024-05-13T22:48:34.490 INFO:tasks.workunit.client.1.smithi130.stdout:5/545: truncate d6/dd/d18/fc 448321 0 2024-05-13T22:48:34.494 INFO:tasks.workunit.client.1.smithi130.stdout:9/606: rename d1/d7/d23/d28/f7c to d1/d7/d23/d28/d32/d65/fc0 0 2024-05-13T22:48:34.503 INFO:tasks.workunit.client.1.smithi130.stdout:6/554: dread d4/d15/d1f/f3e [0,4194304] 0 2024-05-13T22:48:34.510 INFO:tasks.workunit.client.1.smithi130.stdout:7/540: sync 2024-05-13T22:48:34.511 INFO:tasks.workunit.client.1.smithi130.stdout:7/541: stat d7/da/d1d/d5d/da6/d9/d4a 0 2024-05-13T22:48:34.511 INFO:tasks.workunit.client.1.smithi130.stdout:2/567: creat d4/db/dc/d18/fca x:0 0 0 2024-05-13T22:48:34.512 INFO:tasks.workunit.client.1.smithi130.stdout:7/542: chown d7/c36 3 1 2024-05-13T22:48:34.512 INFO:tasks.workunit.client.1.smithi130.stdout:4/550: getdents d8/dc/d35/d33/d44/d56/d8c 0 2024-05-13T22:48:34.512 INFO:tasks.workunit.client.1.smithi130.stdout:4/551: fdatasync d8/dc/d26/f75 0 2024-05-13T22:48:34.513 INFO:tasks.workunit.client.1.smithi130.stdout:4/552: chown d8/dc/d26/f30 4 1 2024-05-13T22:48:34.513 INFO:tasks.workunit.client.1.smithi130.stdout:7/543: truncate d7/da/d1d/d5d/da6/d39/f94 648923 0 2024-05-13T22:48:34.514 INFO:tasks.workunit.client.1.smithi130.stdout:4/553: chown d8/dc/d35/d33/d44/d61/f98 14 1 2024-05-13T22:48:34.518 INFO:tasks.workunit.client.1.smithi130.stdout:6/555: creat d4/d15/d1d/d28/d75/d7d/fc1 x:0 0 0 2024-05-13T22:48:34.520 INFO:tasks.workunit.client.1.smithi130.stdout:7/544: dwrite d7/da/d1d/d42/fb5 [0,4194304] 0 2024-05-13T22:48:34.522 INFO:tasks.workunit.client.1.smithi130.stdout:2/568: rmdir d4/db/dbf/d42/d4b 39 2024-05-13T22:48:34.523 INFO:tasks.workunit.client.1.smithi130.stdout:2/569: readlink d4/db/dbf/d48/d4a/l8b 0 2024-05-13T22:48:34.526 INFO:tasks.workunit.client.1.smithi130.stdout:4/554: mkdir d8/dc/d26/dab 0 2024-05-13T22:48:34.526 INFO:tasks.workunit.client.1.smithi130.stdout:7/545: dwrite d7/da/d1d/d5d/da6/d9/d66/fa5 [0,4194304] 0 2024-05-13T22:48:34.528 INFO:tasks.workunit.client.1.smithi130.stdout:7/546: stat d7/da/d1d/d5d/da6/d9/d66/l2f 0 2024-05-13T22:48:34.530 INFO:tasks.workunit.client.1.smithi130.stdout:6/556: creat d4/d15/d1d/daa/fc2 x:0 0 0 2024-05-13T22:48:34.533 INFO:tasks.workunit.client.1.smithi130.stdout:8/522: rename d1/d2/db/d15/d16/d54/db2 to d1/d2/db/d15/d18/dba 0 2024-05-13T22:48:34.535 INFO:tasks.workunit.client.1.smithi130.stdout:2/570: mknod d4/da8/dc9/ccb 0 2024-05-13T22:48:34.536 INFO:tasks.workunit.client.1.smithi130.stdout:2/571: chown d4/db/dc/d18/d2a/d30/d88/d99/d9a/cb1 89089 1 2024-05-13T22:48:34.540 INFO:tasks.workunit.client.1.smithi130.stdout:4/555: rename d8/df/d81/f91 to d8/df/d81/fac 0 2024-05-13T22:48:34.544 INFO:tasks.workunit.client.1.smithi130.stdout:8/523: symlink d1/d2/db/lbb 0 2024-05-13T22:48:34.549 INFO:tasks.workunit.client.1.smithi130.stdout:1/535: getdents db/d1b/da3/d49/d56/d88 0 2024-05-13T22:48:34.549 INFO:tasks.workunit.client.1.smithi130.stdout:1/536: chown db/d1b/l39 1 1 2024-05-13T22:48:34.550 INFO:tasks.workunit.client.1.smithi130.stdout:1/537: stat db/d1b/da3/f89 0 2024-05-13T22:48:34.550 INFO:tasks.workunit.client.1.smithi130.stdout:4/556: fdatasync d8/dc/f38 0 2024-05-13T22:48:34.555 INFO:tasks.workunit.client.1.smithi130.stdout:8/524: read d1/d2/db/d15/d18/f63 [2557942,115534] 0 2024-05-13T22:48:34.555 INFO:tasks.workunit.client.1.smithi130.stdout:8/525: chown d1/d2/db/f41 45211 1 2024-05-13T22:48:34.557 INFO:tasks.workunit.client.1.smithi130.stdout:1/538: rmdir db/d82/d9f 39 2024-05-13T22:48:34.563 INFO:tasks.workunit.client.1.smithi130.stdout:4/557: mkdir d8/df/dad 0 2024-05-13T22:48:34.570 INFO:tasks.workunit.client.1.smithi130.stdout:0/590: sync 2024-05-13T22:48:34.571 INFO:tasks.workunit.client.1.smithi130.stdout:0/591: write d0/d7/dd/d17/d1f/d2c/d45/d57/f58 [873099,109852] 0 2024-05-13T22:48:34.576 INFO:tasks.workunit.client.1.smithi130.stdout:0/592: dwrite d0/f8 [0,4194304] 0 2024-05-13T22:48:34.577 INFO:tasks.workunit.client.1.smithi130.stdout:5/546: truncate d6/dd/d18/d6e/d82/f7b 487472 0 2024-05-13T22:48:34.579 INFO:tasks.workunit.client.1.smithi130.stdout:6/557: getdents d4/d15/d1f 0 2024-05-13T22:48:34.581 INFO:tasks.workunit.client.1.smithi130.stdout:8/526: mknod d1/d2/db/d15/d18/d42/cbc 0 2024-05-13T22:48:34.582 INFO:tasks.workunit.client.1.smithi130.stdout:9/607: truncate d1/d7/d23/d28/d2d/f48 418677 0 2024-05-13T22:48:34.594 INFO:tasks.workunit.client.1.smithi130.stdout:5/547: creat d6/dd/fb5 x:0 0 0 2024-05-13T22:48:34.596 INFO:tasks.workunit.client.1.smithi130.stdout:0/593: creat d0/d7/dd/d17/d52/d5b/d5f/fc9 x:0 0 0 2024-05-13T22:48:34.596 INFO:tasks.workunit.client.1.smithi130.stdout:7/547: truncate d7/da/d1d/d42/f58 5163306 0 2024-05-13T22:48:34.597 INFO:tasks.workunit.client.1.smithi130.stdout:0/594: chown d0/d7/dd/c3e 4626248 1 2024-05-13T22:48:34.598 INFO:tasks.workunit.client.1.smithi130.stdout:6/558: dread - d4/d15/d1d/d28/d4e/f56 zero size 2024-05-13T22:48:34.598 INFO:tasks.workunit.client.1.smithi130.stdout:2/572: truncate d4/db/dc/fbc 201687 0 2024-05-13T22:48:34.599 INFO:tasks.workunit.client.1.smithi130.stdout:6/559: chown d4/d15/d1f/d46/d50/f67 4 1 2024-05-13T22:48:34.601 INFO:tasks.workunit.client.1.smithi130.stdout:7/548: dread d7/da/d1d/d42/fb5 [0,4194304] 0 2024-05-13T22:48:34.602 INFO:tasks.workunit.client.1.smithi130.stdout:0/595: dwrite d0/d7/dd/d17/d52/d67/d68/d96/f84 [0,4194304] 0 2024-05-13T22:48:34.631 INFO:tasks.workunit.client.1.smithi130.stdout:8/527: rename d1/d2/db/d15/d7e/c81 to d1/d2/db/d15/d18/dba/cbd 0 2024-05-13T22:48:34.635 INFO:tasks.workunit.client.1.smithi130.stdout:1/539: dwrite db/d4a/f4b [0,4194304] 0 2024-05-13T22:48:34.636 INFO:tasks.workunit.client.1.smithi130.stdout:9/608: creat d1/d7/d23/d28/d32/fc1 x:0 0 0 2024-05-13T22:48:34.637 INFO:tasks.workunit.client.1.smithi130.stdout:1/540: fdatasync db/d4a/f95 0 2024-05-13T22:48:34.637 INFO:tasks.workunit.client.1.smithi130.stdout:4/558: dwrite d8/dc/d80/f78 [4194304,4194304] 0 2024-05-13T22:48:34.640 INFO:tasks.workunit.client.1.smithi130.stdout:7/549: dread d7/da/d1d/d42/f64 [0,4194304] 0 2024-05-13T22:48:34.642 INFO:tasks.workunit.client.1.smithi130.stdout:8/528: creat d1/d2/db/d15/d18/dba/fbe x:0 0 0 2024-05-13T22:48:34.645 INFO:tasks.workunit.client.1.smithi130.stdout:5/548: dread d6/d7/d19/d4d/f2d [0,4194304] 0 2024-05-13T22:48:34.647 INFO:tasks.workunit.client.1.smithi130.stdout:5/549: read d6/d7/f45 [2222484,95972] 0 2024-05-13T22:48:34.648 INFO:tasks.workunit.client.1.smithi130.stdout:5/550: dread - d6/fac zero size 2024-05-13T22:48:34.649 INFO:tasks.workunit.client.1.smithi130.stdout:9/609: chown d1/d7/d23/d28/c41 321963238 1 2024-05-13T22:48:34.649 INFO:tasks.workunit.client.1.smithi130.stdout:5/551: readlink d6/dd/d18/d6e/d82/l5a 0 2024-05-13T22:48:34.650 INFO:tasks.workunit.client.1.smithi130.stdout:5/552: readlink d6/dd/d18/l14 0 2024-05-13T22:48:34.654 INFO:tasks.workunit.client.1.smithi130.stdout:4/559: unlink d8/f43 0 2024-05-13T22:48:34.657 INFO:tasks.workunit.client.1.smithi130.stdout:0/596: creat d0/d7/fca x:0 0 0 2024-05-13T22:48:34.660 INFO:tasks.workunit.client.1.smithi130.stdout:7/550: rmdir d7/da/d1d/d5d/da6/d39/da9 39 2024-05-13T22:48:34.662 INFO:tasks.workunit.client.1.smithi130.stdout:8/529: creat d1/d2/db/d15/d16/fbf x:0 0 0 2024-05-13T22:48:34.663 INFO:tasks.workunit.client.1.smithi130.stdout:0/597: dwrite d0/d7/dd/d17/d1f/d2c/d45/d57/f62 [0,4194304] 0 2024-05-13T22:48:34.669 INFO:tasks.workunit.client.1.smithi130.stdout:9/610: creat d1/d7/d23/fc2 x:0 0 0 2024-05-13T22:48:34.671 INFO:tasks.workunit.client.1.smithi130.stdout:9/611: write d1/d7/d23/d28/d32/d65/fb9 [809356,93854] 0 2024-05-13T22:48:34.674 INFO:tasks.workunit.client.1.smithi130.stdout:5/553: creat d6/dd/fb6 x:0 0 0 2024-05-13T22:48:34.677 INFO:tasks.workunit.client.1.smithi130.stdout:4/560: creat d8/dc/d35/d33/d44/d56/fae x:0 0 0 2024-05-13T22:48:34.677 INFO:tasks.workunit.client.1.smithi130.stdout:4/561: readlink d8/dc/d35/d33/d44/d77/l8f 0 2024-05-13T22:48:34.681 INFO:tasks.workunit.client.1.smithi130.stdout:8/530: rename d1/d2/db/f4a to d1/d2/db/d15/fc0 0 2024-05-13T22:48:34.682 INFO:tasks.workunit.client.1.smithi130.stdout:8/531: write d1/d2/db/d15/d7e/d9f/fa0 [669766,54211] 0 2024-05-13T22:48:34.686 INFO:tasks.workunit.client.1.smithi130.stdout:3/535: sync 2024-05-13T22:48:34.686 INFO:tasks.workunit.client.1.smithi130.stdout:5/554: fdatasync d6/d7/d19/d4d/d8a/fa5 0 2024-05-13T22:48:34.687 INFO:tasks.workunit.client.1.smithi130.stdout:4/562: creat d8/df/faf x:0 0 0 2024-05-13T22:48:34.691 INFO:tasks.workunit.client.1.smithi130.stdout:9/612: rename d1/d7/d23/d28/d32/d45/l4c to d1/d7/d69/d79/lc3 0 2024-05-13T22:48:34.694 INFO:tasks.workunit.client.1.smithi130.stdout:1/541: dwrite db/f1d [0,4194304] 0 2024-05-13T22:48:34.695 INFO:tasks.workunit.client.1.smithi130.stdout:1/542: write db/f1d [3388352,23503] 0 2024-05-13T22:48:34.700 INFO:tasks.workunit.client.1.smithi130.stdout:5/555: unlink d6/d7/d19/l63 0 2024-05-13T22:48:34.701 INFO:tasks.workunit.client.1.smithi130.stdout:3/536: symlink d8/dc/d19/d2c/lbc 0 2024-05-13T22:48:34.703 INFO:tasks.workunit.client.1.smithi130.stdout:3/537: read d8/dc/d15/f18 [3767299,63985] 0 2024-05-13T22:48:34.704 INFO:tasks.workunit.client.1.smithi130.stdout:4/563: unlink d8/dc/d80/f78 0 2024-05-13T22:48:34.708 INFO:tasks.workunit.client.1.smithi130.stdout:9/613: creat d1/d7/d69/da2/fc4 x:0 0 0 2024-05-13T22:48:34.712 INFO:tasks.workunit.client.1.smithi130.stdout:0/598: mknod d0/da0/ccb 0 2024-05-13T22:48:34.715 INFO:tasks.workunit.client.1.smithi130.stdout:9/614: dread d1/d7/d23/d28/d2d/d2f/f6a [0,4194304] 0 2024-05-13T22:48:34.723 INFO:tasks.workunit.client.1.smithi130.stdout:8/532: write d1/d2/db/d15/d18/d42/d53/d56/f61 [81506,10595] 0 2024-05-13T22:48:34.724 INFO:tasks.workunit.client.1.smithi130.stdout:8/533: dread - d1/d2/db/d15/d16/d54/d58/f98 zero size 2024-05-13T22:48:34.730 INFO:tasks.workunit.client.1.smithi130.stdout:7/551: getdents d7/da/d1d/d5d/da6/d9/d66 0 2024-05-13T22:48:34.731 INFO:tasks.workunit.client.1.smithi130.stdout:7/552: write d7/da/d1d/d5d/da6/d39/f94 [1173187,86038] 0 2024-05-13T22:48:34.731 INFO:tasks.workunit.client.1.smithi130.stdout:4/564: write d8/df/d81/f9e [4245082,10450] 0 2024-05-13T22:48:34.735 INFO:tasks.workunit.client.1.smithi130.stdout:9/615: mknod d1/db/cc5 0 2024-05-13T22:48:34.738 INFO:tasks.workunit.client.1.smithi130.stdout:8/534: mknod d1/d2/db/d15/d16/d54/cc1 0 2024-05-13T22:48:34.742 INFO:tasks.workunit.client.1.smithi130.stdout:4/565: mkdir d8/dc/d80/db0 0 2024-05-13T22:48:34.743 INFO:tasks.workunit.client.1.smithi130.stdout:4/566: fsync d8/df/f40 0 2024-05-13T22:48:34.744 INFO:tasks.workunit.client.1.smithi130.stdout:9/616: creat d1/d7/d23/d3d/fc6 x:0 0 0 2024-05-13T22:48:34.745 INFO:tasks.workunit.client.1.smithi130.stdout:1/543: rmdir db/d1b/da3/d2a/d38/d66/da1 0 2024-05-13T22:48:34.745 INFO:tasks.workunit.client.1.smithi130.stdout:9/617: chown d1/d7/d69/d79/f9a 921991 1 2024-05-13T22:48:34.746 INFO:tasks.workunit.client.1.smithi130.stdout:7/553: dread d7/da/d1d/d5d/da6/d9/d66/f81 [0,4194304] 0 2024-05-13T22:48:34.747 INFO:tasks.workunit.client.1.smithi130.stdout:3/538: creat d8/fbd x:0 0 0 2024-05-13T22:48:34.747 INFO:tasks.workunit.client.1.smithi130.stdout:7/554: readlink d7/da/d1d/d5d/da6/d9/d66/l3b 0 2024-05-13T22:48:34.748 INFO:tasks.workunit.client.1.smithi130.stdout:3/539: dread - d8/d35/f48 zero size 2024-05-13T22:48:34.748 INFO:tasks.workunit.client.1.smithi130.stdout:7/555: chown d7/da/d1d/d42/d43/f4d 9 1 2024-05-13T22:48:34.749 INFO:tasks.workunit.client.1.smithi130.stdout:3/540: write f3 [768129,37109] 0 2024-05-13T22:48:34.749 INFO:tasks.workunit.client.1.smithi130.stdout:0/599: rename d0/d7/dd/d2e/f75 to d0/d7/fcc 0 2024-05-13T22:48:34.753 INFO:tasks.workunit.client.1.smithi130.stdout:9/618: mkdir d1/d7/d23/d28/d2d/d2f/dc7 0 2024-05-13T22:48:34.753 INFO:tasks.workunit.client.1.smithi130.stdout:9/619: write d1/f1e [981452,119044] 0 2024-05-13T22:48:34.760 INFO:tasks.workunit.client.1.smithi130.stdout:3/541: write d8/f4b [1830028,61568] 0 2024-05-13T22:48:34.761 INFO:tasks.workunit.client.1.smithi130.stdout:3/542: fsync d8/dc/d15/f76 0 2024-05-13T22:48:34.761 INFO:tasks.workunit.client.1.smithi130.stdout:7/556: mkdir d7/da/d1d/d3d/db9 0 2024-05-13T22:48:34.762 INFO:tasks.workunit.client.1.smithi130.stdout:3/543: write d8/dc/f9f [143039,33893] 0 2024-05-13T22:48:34.762 INFO:tasks.workunit.client.1.smithi130.stdout:0/600: mkdir d0/d7/dd/d2e/da8/dcd 0 2024-05-13T22:48:34.764 INFO:tasks.workunit.client.1.smithi130.stdout:8/535: dread d1/f33 [0,4194304] 0 2024-05-13T22:48:34.764 INFO:tasks.workunit.client.1.smithi130.stdout:3/544: fsync d8/dc/d15/f58 0 2024-05-13T22:48:34.766 INFO:tasks.workunit.client.1.smithi130.stdout:9/620: mknod d1/d7/d23/d3d/cc8 0 2024-05-13T22:48:34.767 INFO:tasks.workunit.client.1.smithi130.stdout:1/544: rename db/d1b/da3/d2a/d38/l98 to db/d1b/da3/d2a/d38/d46/d6d/laf 0 2024-05-13T22:48:34.768 INFO:tasks.workunit.client.1.smithi130.stdout:1/545: truncate db/d4a/f70 4376073 0 2024-05-13T22:48:34.770 INFO:tasks.workunit.client.1.smithi130.stdout:2/573: sync 2024-05-13T22:48:34.772 INFO:tasks.workunit.client.1.smithi130.stdout:0/601: mknod d0/d7/dd/d69/da9/cce 0 2024-05-13T22:48:34.772 INFO:tasks.workunit.client.1.smithi130.stdout:8/536: mkdir d1/d2/db/d15/d7e/dc2 0 2024-05-13T22:48:34.774 INFO:tasks.workunit.client.1.smithi130.stdout:1/546: dwrite db/d1b/da3/d2a/d38/d46/f71 [0,4194304] 0 2024-05-13T22:48:34.775 INFO:tasks.workunit.client.1.smithi130.stdout:3/545: creat d8/dc/d7b/da3/fbe x:0 0 0 2024-05-13T22:48:34.780 INFO:tasks.workunit.client.1.smithi130.stdout:4/567: getdents d8/dc/d35/d33/d44/d61 0 2024-05-13T22:48:34.781 INFO:tasks.workunit.client.1.smithi130.stdout:2/574: rename d4/db/d26/fb7 to d4/db/d26/fcc 0 2024-05-13T22:48:34.783 INFO:tasks.workunit.client.1.smithi130.stdout:0/602: symlink d0/d7/dd/d17/d1f/lcf 0 2024-05-13T22:48:34.785 INFO:tasks.workunit.client.1.smithi130.stdout:0/603: write d0/d7/dd/d17/fae [3352531,47256] 0 2024-05-13T22:48:34.786 INFO:tasks.workunit.client.1.smithi130.stdout:4/568: dwrite d8/dc/d26/f30 [0,4194304] 0 2024-05-13T22:48:34.791 INFO:tasks.workunit.client.1.smithi130.stdout:1/547: creat db/d82/d9f/fb0 x:0 0 0 2024-05-13T22:48:34.793 INFO:tasks.workunit.client.1.smithi130.stdout:9/621: link d1/d7/d23/d3d/fc6 d1/fc9 0 2024-05-13T22:48:34.793 INFO:tasks.workunit.client.1.smithi130.stdout:3/546: getdents d8/dc/d19/d2c/d61/d9c 0 2024-05-13T22:48:34.794 INFO:tasks.workunit.client.1.smithi130.stdout:7/557: getdents d7/da/d1d/d5d 0 2024-05-13T22:48:34.795 INFO:tasks.workunit.client.1.smithi130.stdout:3/547: write d8/dc/d7b/da3/daf/fb8 [249586,74018] 0 2024-05-13T22:48:34.800 INFO:tasks.workunit.client.1.smithi130.stdout:8/537: mkdir d1/dc/dc3 0 2024-05-13T22:48:34.800 INFO:tasks.workunit.client.1.smithi130.stdout:0/604: dwrite d0/f27 [0,4194304] 0 2024-05-13T22:48:34.802 INFO:tasks.workunit.client.1.smithi130.stdout:8/538: truncate d1/d2/db/d15/d7e/f9c 319889 0 2024-05-13T22:48:34.803 INFO:tasks.workunit.client.1.smithi130.stdout:1/548: rename db/d1b/da3/d2a/d38/d46/d6d/d6f/f92 to db/d58/fb1 0 2024-05-13T22:48:34.804 INFO:tasks.workunit.client.1.smithi130.stdout:1/549: chown db/d1b/da3/d49/d56/c84 636995 1 2024-05-13T22:48:34.808 INFO:tasks.workunit.client.1.smithi130.stdout:1/550: dread db/d1b/da3/d72/d75/f81 [0,4194304] 0 2024-05-13T22:48:34.809 INFO:tasks.workunit.client.1.smithi130.stdout:7/558: write d7/da/d1d/d42/f64 [1720062,98699] 0 2024-05-13T22:48:34.810 INFO:tasks.workunit.client.1.smithi130.stdout:2/575: write d4/db/dc/f4e [1277890,123204] 0 2024-05-13T22:48:34.813 INFO:tasks.workunit.client.1.smithi130.stdout:5/556: sync 2024-05-13T22:48:34.813 INFO:tasks.workunit.client.1.smithi130.stdout:6/560: sync 2024-05-13T22:48:34.815 INFO:tasks.workunit.client.1.smithi130.stdout:4/569: write d8/dc/d26/f36 [619180,128760] 0 2024-05-13T22:48:34.816 INFO:tasks.workunit.client.1.smithi130.stdout:8/539: mknod d1/d2/db/cc4 0 2024-05-13T22:48:34.819 INFO:tasks.workunit.client.1.smithi130.stdout:4/570: read d8/dc/d26/f2b [1584145,110467] 0 2024-05-13T22:48:34.820 INFO:tasks.workunit.client.1.smithi130.stdout:1/551: write db/d1b/da3/d49/d56/f7d [2384576,124222] 0 2024-05-13T22:48:34.821 INFO:tasks.workunit.client.1.smithi130.stdout:0/605: dwrite d0/f9c [0,4194304] 0 2024-05-13T22:48:34.824 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:34 smithi102 ceph-mon[28599]: pgmap v34: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 80 MiB/s rd, 266 MiB/s wr, 533 op/s 2024-05-13T22:48:34.826 INFO:tasks.workunit.client.1.smithi130.stdout:9/622: creat d1/d7/d23/d28/d2d/d2f/d63/fca x:0 0 0 2024-05-13T22:48:34.833 INFO:tasks.workunit.client.1.smithi130.stdout:2/576: mkdir d4/db/dc/d18/d2a/d30/d88/dc5/dcd 0 2024-05-13T22:48:34.837 INFO:tasks.workunit.client.1.smithi130.stdout:7/559: link d7/da/d1d/d5d/da6/d9/d66/fa5 d7/da/d1d/d5d/da6/d9/d4a/fba 0 2024-05-13T22:48:34.838 INFO:tasks.workunit.client.1.smithi130.stdout:2/577: dwrite d4/db/dc/d18/d58/f70 [0,4194304] 0 2024-05-13T22:48:34.839 INFO:tasks.workunit.client.1.smithi130.stdout:4/571: rmdir d8/dc/d35/d33 39 2024-05-13T22:48:34.839 INFO:tasks.workunit.client.1.smithi130.stdout:7/560: read - d7/da/d1d/d5d/da6/d9/d66/f7a zero size 2024-05-13T22:48:34.840 INFO:tasks.workunit.client.1.smithi130.stdout:1/552: symlink db/d1b/da3/d49/d56/d7f/lb2 0 2024-05-13T22:48:34.841 INFO:tasks.workunit.client.1.smithi130.stdout:0/606: symlink d0/d7/dd/d17/d52/d5b/d5f/ld0 0 2024-05-13T22:48:34.843 INFO:tasks.workunit.client.1.smithi130.stdout:3/548: getdents d8/d35 0 2024-05-13T22:48:34.845 INFO:tasks.workunit.client.1.smithi130.stdout:5/557: dwrite d6/dd/f3e [0,4194304] 0 2024-05-13T22:48:34.846 INFO:tasks.workunit.client.1.smithi130.stdout:6/561: creat d4/d15/d1f/d46/d52/fc3 x:0 0 0 2024-05-13T22:48:34.847 INFO:tasks.workunit.client.1.smithi130.stdout:6/562: stat d4/d15/d1f/d64/l68 0 2024-05-13T22:48:34.847 INFO:tasks.workunit.client.1.smithi130.stdout:7/561: write d7/da/d1d/d3d/d46/d67/f30 [445930,7399] 0 2024-05-13T22:48:34.852 INFO:tasks.workunit.client.1.smithi130.stdout:8/540: link d1/d2/db/d15/d16/f8f d1/d2/fc5 0 2024-05-13T22:48:34.857 INFO:tasks.workunit.client.1.smithi130.stdout:2/578: dread d4/f1a [0,4194304] 0 2024-05-13T22:48:34.857 INFO:tasks.workunit.client.1.smithi130.stdout:6/563: dwrite d4/d15/d1f/f20 [4194304,4194304] 0 2024-05-13T22:48:34.862 INFO:tasks.workunit.client.1.smithi130.stdout:3/549: symlink d8/dc/d19/d2c/d61/lbf 0 2024-05-13T22:48:34.863 INFO:tasks.workunit.client.1.smithi130.stdout:0/607: getdents d0/d7/dd/d17/d52/d5b/d60 0 2024-05-13T22:48:34.863 INFO:tasks.workunit.client.1.smithi130.stdout:3/550: read - d8/fbd zero size 2024-05-13T22:48:34.864 INFO:tasks.workunit.client.1.smithi130.stdout:2/579: write d4/db/dc/f15 [175113,46275] 0 2024-05-13T22:48:34.864 INFO:tasks.workunit.client.1.smithi130.stdout:7/562: symlink d7/da/d1d/d42/lbb 0 2024-05-13T22:48:34.865 INFO:tasks.workunit.client.1.smithi130.stdout:4/572: creat d8/dc/d35/d33/d3f/d57/fb1 x:0 0 0 2024-05-13T22:48:34.866 INFO:tasks.workunit.client.1.smithi130.stdout:2/580: chown d4/db/dc/d18/d2a/l2b 233 1 2024-05-13T22:48:34.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:34 smithi130 ceph-mon[36591]: pgmap v34: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 80 MiB/s rd, 266 MiB/s wr, 533 op/s 2024-05-13T22:48:34.907 INFO:tasks.workunit.client.1.smithi130.stdout:7/563: unlink d7/da/d1d/d42/f64 0 2024-05-13T22:48:34.908 INFO:tasks.workunit.client.1.smithi130.stdout:2/581: mknod d4/db/dc/d18/d2a/cce 0 2024-05-13T22:48:34.908 INFO:tasks.workunit.client.1.smithi130.stdout:5/558: link d6/dd/d18/d6e/d82/l5a d6/d7/lb7 0 2024-05-13T22:48:34.908 INFO:tasks.workunit.client.1.smithi130.stdout:8/541: write d1/d2/db/f8 [2057709,16605] 0 2024-05-13T22:48:34.910 INFO:tasks.workunit.client.1.smithi130.stdout:7/564: stat d7/da/d1d/d5d/da6/d9/c9b 0 2024-05-13T22:48:34.910 INFO:tasks.workunit.client.1.smithi130.stdout:2/582: readlink d4/db/dc/d18/d2a/l3e 0 2024-05-13T22:48:34.910 INFO:tasks.workunit.client.1.smithi130.stdout:0/608: symlink d0/d7/d29/d7b/ld1 0 2024-05-13T22:48:34.911 INFO:tasks.workunit.client.1.smithi130.stdout:3/551: creat d8/dc/d15/dac/fc0 x:0 0 0 2024-05-13T22:48:34.912 INFO:tasks.workunit.client.1.smithi130.stdout:8/542: stat d1/d2/db/d15/d7e 0 2024-05-13T22:48:34.915 INFO:tasks.workunit.client.1.smithi130.stdout:1/553: dwrite db/f16 [0,4194304] 0 2024-05-13T22:48:34.916 INFO:tasks.workunit.client.1.smithi130.stdout:4/573: dwrite f7 [0,4194304] 0 2024-05-13T22:48:34.917 INFO:tasks.workunit.client.1.smithi130.stdout:1/554: fsync db/d1b/da3/d49/f61 0 2024-05-13T22:48:34.918 INFO:tasks.workunit.client.1.smithi130.stdout:1/555: write db/d1b/da3/f53 [3980139,29884] 0 2024-05-13T22:48:34.919 INFO:tasks.workunit.client.1.smithi130.stdout:1/556: readlink db/d58/l8c 0 2024-05-13T22:48:34.920 INFO:tasks.workunit.client.1.smithi130.stdout:5/559: mknod d6/d1a/daf/cb8 0 2024-05-13T22:48:34.925 INFO:tasks.workunit.client.1.smithi130.stdout:7/565: mkdir d7/da/d1d/d3d/d46/d67/dbc 0 2024-05-13T22:48:34.925 INFO:tasks.workunit.client.1.smithi130.stdout:0/609: mkdir d0/d7/dd/d69/da9/dd2 0 2024-05-13T22:48:34.929 INFO:tasks.workunit.client.1.smithi130.stdout:8/543: symlink d1/d2/db/d15/d7e/lc6 0 2024-05-13T22:48:34.930 INFO:tasks.workunit.client.1.smithi130.stdout:2/583: write d4/f1a [1070563,95575] 0 2024-05-13T22:48:34.932 INFO:tasks.workunit.client.1.smithi130.stdout:3/552: dwrite d8/f2b [0,4194304] 0 2024-05-13T22:48:34.933 INFO:tasks.workunit.client.1.smithi130.stdout:2/584: chown d4/db/dbf/f35 2 1 2024-05-13T22:48:34.934 INFO:tasks.workunit.client.1.smithi130.stdout:5/560: dwrite d6/dd/d18/d6e/d76/d7f/fb2 [0,4194304] 0 2024-05-13T22:48:34.937 INFO:tasks.workunit.client.1.smithi130.stdout:5/561: write d6/f5f [312167,3821] 0 2024-05-13T22:48:34.938 INFO:tasks.workunit.client.1.smithi130.stdout:7/566: fdatasync d7/da/d1d/d5d/da6/d9/d66/fa5 0 2024-05-13T22:48:34.940 INFO:tasks.workunit.client.1.smithi130.stdout:9/623: sync 2024-05-13T22:48:34.941 INFO:tasks.workunit.client.1.smithi130.stdout:3/553: fsync d8/f33 0 2024-05-13T22:48:34.942 INFO:tasks.workunit.client.1.smithi130.stdout:3/554: readlink d8/dc/d15/l38 0 2024-05-13T22:48:34.944 INFO:tasks.workunit.client.1.smithi130.stdout:2/585: dwrite d4/db/d26/d9e/fad [0,4194304] 0 2024-05-13T22:48:34.945 INFO:tasks.workunit.client.1.smithi130.stdout:3/555: truncate d8/dc/d19/d27/f28 861806 0 2024-05-13T22:48:34.946 INFO:tasks.workunit.client.1.smithi130.stdout:3/556: write d8/f33 [627593,61775] 0 2024-05-13T22:48:34.955 INFO:tasks.workunit.client.1.smithi130.stdout:8/544: mknod d1/d2/cc7 0 2024-05-13T22:48:34.959 INFO:tasks.workunit.client.1.smithi130.stdout:4/574: link d8/l17 d8/dc/d35/d33/d44/d61/da0/lb2 0 2024-05-13T22:48:34.959 INFO:tasks.workunit.client.1.smithi130.stdout:1/557: write db/d1b/da3/d2a/d35/f3b [1059932,104786] 0 2024-05-13T22:48:34.961 INFO:tasks.workunit.client.1.smithi130.stdout:0/610: creat d0/d7/dd/d17/d1f/d2c/fd3 x:0 0 0 2024-05-13T22:48:34.963 INFO:tasks.workunit.client.1.smithi130.stdout:0/611: truncate d0/d7/dd/d17/d1f/d2c/fd3 725398 0 2024-05-13T22:48:34.963 INFO:tasks.workunit.client.1.smithi130.stdout:8/545: mknod d1/d2/db/d15/d18/d42/d53/d56/cc8 0 2024-05-13T22:48:34.965 INFO:tasks.workunit.client.1.smithi130.stdout:4/575: unlink d8/dc/d35/l37 0 2024-05-13T22:48:34.966 INFO:tasks.workunit.client.1.smithi130.stdout:0/612: mknod d0/d7/dd/d17/d1f/cd4 0 2024-05-13T22:48:34.968 INFO:tasks.workunit.client.1.smithi130.stdout:1/558: creat db/d1b/da3/d72/daa/fb3 x:0 0 0 2024-05-13T22:48:34.971 INFO:tasks.workunit.client.1.smithi130.stdout:5/562: rename d6/d7/d19/d4d/l42 to d6/lb9 0 2024-05-13T22:48:34.972 INFO:tasks.workunit.client.1.smithi130.stdout:3/557: getdents d8/dc/d42 0 2024-05-13T22:48:34.972 INFO:tasks.workunit.client.1.smithi130.stdout:0/613: dwrite d0/fad [0,4194304] 0 2024-05-13T22:48:34.974 INFO:tasks.workunit.client.1.smithi130.stdout:0/614: chown d0/c1 18789829 1 2024-05-13T22:48:34.976 INFO:tasks.workunit.client.1.smithi130.stdout:3/558: truncate d8/dc/d15/dac/fc0 807242 0 2024-05-13T22:48:34.980 INFO:tasks.workunit.client.1.smithi130.stdout:2/586: rename d4/d56/f6d to d4/db/dc/d18/d58/fcf 0 2024-05-13T22:48:34.981 INFO:tasks.workunit.client.1.smithi130.stdout:2/587: write d4/db/dc/f20 [803533,4250] 0 2024-05-13T22:48:34.982 INFO:tasks.workunit.client.1.smithi130.stdout:0/615: chown d0/d7/l1d 1879 1 2024-05-13T22:48:34.983 INFO:tasks.workunit.client.1.smithi130.stdout:7/567: write d7/da/f5c [598210,45319] 0 2024-05-13T22:48:34.988 INFO:tasks.workunit.client.1.smithi130.stdout:8/546: dread d1/d2/db/d15/d18/d1a/f9d [0,4194304] 0 2024-05-13T22:48:34.990 INFO:tasks.workunit.client.1.smithi130.stdout:4/576: rename d8/dc/d35/d33/d44/l73 to d8/dc/d35/d33/d44/d56/lb3 0 2024-05-13T22:48:34.992 INFO:tasks.workunit.client.1.smithi130.stdout:4/577: write d8/df/f59 [1111750,29302] 0 2024-05-13T22:48:34.993 INFO:tasks.workunit.client.1.smithi130.stdout:3/559: dwrite d8/d35/f48 [0,4194304] 0 2024-05-13T22:48:34.997 INFO:tasks.workunit.client.1.smithi130.stdout:2/588: symlink d4/db/dc/d18/d2a/d30/d88/dc5/ld0 0 2024-05-13T22:48:34.997 INFO:tasks.workunit.client.1.smithi130.stdout:8/547: dwrite d1/d2/db/d15/d16/fb1 [0,4194304] 0 2024-05-13T22:48:34.999 INFO:tasks.workunit.client.1.smithi130.stdout:1/559: write f8 [514337,97764] 0 2024-05-13T22:48:35.000 INFO:tasks.workunit.client.1.smithi130.stdout:8/548: read d1/dc/f44 [5520090,27766] 0 2024-05-13T22:48:35.000 INFO:tasks.workunit.client.1.smithi130.stdout:0/616: unlink d0/d7/dd/d17/d52/la3 0 2024-05-13T22:48:35.001 INFO:tasks.workunit.client.1.smithi130.stdout:1/560: write db/d1b/da3/d2a/d38/fa8 [1764664,23171] 0 2024-05-13T22:48:35.002 INFO:tasks.workunit.client.1.smithi130.stdout:1/561: write db/f1d [91305,12521] 0 2024-05-13T22:48:35.008 INFO:tasks.workunit.client.1.smithi130.stdout:1/562: dwrite db/f1d [0,4194304] 0 2024-05-13T22:48:35.014 INFO:tasks.workunit.client.1.smithi130.stdout:9/624: sync 2024-05-13T22:48:35.014 INFO:tasks.workunit.client.1.smithi130.stdout:6/564: sync 2024-05-13T22:48:35.015 INFO:tasks.workunit.client.1.smithi130.stdout:7/568: rename d7/da/d1d/d3d/d46/d67 to d7/da/d1d/d5d/da6/d9/d4a/dbd 0 2024-05-13T22:48:35.015 INFO:tasks.workunit.client.1.smithi130.stdout:7/569: stat d7/da/d1d/d5d/da6/f1c 0 2024-05-13T22:48:35.016 INFO:tasks.workunit.client.1.smithi130.stdout:7/570: write d7/da/d1d/d5d/da6/d5b/f6d [295852,52317] 0 2024-05-13T22:48:35.017 INFO:tasks.workunit.client.1.smithi130.stdout:7/571: chown d7/da/d1d/d5d/da6/d5b 3993 1 2024-05-13T22:48:35.018 INFO:tasks.workunit.client.1.smithi130.stdout:7/572: write d7/f6a [4809674,52479] 0 2024-05-13T22:48:35.019 INFO:tasks.workunit.client.1.smithi130.stdout:7/573: chown d7/da/d1d/d5d/da6/l14 0 1 2024-05-13T22:48:35.027 INFO:tasks.workunit.client.1.smithi130.stdout:5/563: link d6/d2f/cb4 d6/dd/d18/d6e/d76/cba 0 2024-05-13T22:48:35.027 INFO:tasks.workunit.client.1.smithi130.stdout:3/560: write d8/dc/d19/d2c/f6b [2917497,95375] 0 2024-05-13T22:48:35.031 INFO:tasks.workunit.client.1.smithi130.stdout:3/561: dread d8/d35/f3d [0,4194304] 0 2024-05-13T22:48:35.035 INFO:tasks.workunit.client.1.smithi130.stdout:0/617: mknod d0/d7/dd/d17/d52/d67/d68/d96/cd5 0 2024-05-13T22:48:35.037 INFO:tasks.workunit.client.1.smithi130.stdout:1/563: symlink db/d1b/da3/d2a/d38/d46/d6d/lb4 0 2024-05-13T22:48:35.041 INFO:tasks.workunit.client.1.smithi130.stdout:8/549: rename d1/d2/db/d15/d16/fbf to d1/d2/db/d15/d7e/d9f/fc9 0 2024-05-13T22:48:35.042 INFO:tasks.workunit.client.1.smithi130.stdout:8/550: chown d1/d2/db/lbb 142 1 2024-05-13T22:48:35.043 INFO:tasks.workunit.client.1.smithi130.stdout:1/564: dwrite db/d1b/da3/d2a/d35/f3b [0,4194304] 0 2024-05-13T22:48:35.046 INFO:tasks.workunit.client.1.smithi130.stdout:2/589: symlink d4/db/dbf/ld1 0 2024-05-13T22:48:35.050 INFO:tasks.workunit.client.1.smithi130.stdout:3/562: creat d8/d1a/d1c/fc1 x:0 0 0 2024-05-13T22:48:35.050 INFO:tasks.workunit.client.1.smithi130.stdout:7/574: dread d7/da/d1d/d3d/d46/f4b [0,4194304] 0 2024-05-13T22:48:35.052 INFO:tasks.workunit.client.1.smithi130.stdout:0/618: symlink d0/d7/dd/d17/d52/d5b/d5f/ld6 0 2024-05-13T22:48:35.053 INFO:tasks.workunit.client.1.smithi130.stdout:0/619: stat d0/d7/d29/d7b 0 2024-05-13T22:48:35.059 INFO:tasks.workunit.client.1.smithi130.stdout:1/565: symlink db/d4a/lb5 0 2024-05-13T22:48:35.060 INFO:tasks.workunit.client.1.smithi130.stdout:1/566: fdatasync db/d4a/f95 0 2024-05-13T22:48:35.065 INFO:tasks.workunit.client.1.smithi130.stdout:2/590: rmdir d4/db/dc 39 2024-05-13T22:48:35.066 INFO:tasks.workunit.client.1.smithi130.stdout:4/578: dwrite d8/dc/d35/d33/d44/d56/f7f [4194304,4194304] 0 2024-05-13T22:48:35.068 INFO:tasks.workunit.client.1.smithi130.stdout:1/567: dwrite db/d1b/da3/d49/d56/d7f/f96 [0,4194304] 0 2024-05-13T22:48:35.076 INFO:tasks.workunit.client.1.smithi130.stdout:7/575: unlink d7/da/d1d/d5d/da6/d39/ca2 0 2024-05-13T22:48:35.076 INFO:tasks.workunit.client.1.smithi130.stdout:2/591: dread d4/fa4 [0,4194304] 0 2024-05-13T22:48:35.078 INFO:tasks.workunit.client.1.smithi130.stdout:0/620: dread d0/d7/dd/d17/d52/f9a [0,4194304] 0 2024-05-13T22:48:35.079 INFO:tasks.workunit.client.1.smithi130.stdout:5/564: truncate d6/dd/d18/d6e/d82/d71/f81 344331 0 2024-05-13T22:48:35.080 INFO:tasks.workunit.client.1.smithi130.stdout:6/565: creat d4/fc4 x:0 0 0 2024-05-13T22:48:35.080 INFO:tasks.workunit.client.1.smithi130.stdout:6/566: chown d4/d15/d1d/d60 77281104 1 2024-05-13T22:48:35.081 INFO:tasks.workunit.client.1.smithi130.stdout:6/567: chown d4/d15/d1f/d46/c4f 11826 1 2024-05-13T22:48:35.083 INFO:tasks.workunit.client.1.smithi130.stdout:4/579: dread - d8/dc/d80/d96/faa zero size 2024-05-13T22:48:35.085 INFO:tasks.workunit.client.1.smithi130.stdout:6/568: chown d4/d15/d1d/d60/da2/da4 6951 1 2024-05-13T22:48:35.085 INFO:tasks.workunit.client.1.smithi130.stdout:6/569: chown d4/d15/d1f/c3f 42 1 2024-05-13T22:48:35.086 INFO:tasks.workunit.client.1.smithi130.stdout:6/570: chown d4/d15/d1d/l1e 95786 1 2024-05-13T22:48:35.086 INFO:tasks.workunit.client.1.smithi130.stdout:3/563: creat d8/dc/d19/d2c/d61/fc2 x:0 0 0 2024-05-13T22:48:35.087 INFO:tasks.workunit.client.1.smithi130.stdout:7/576: mkdir d7/da/d1d/db1/dbe 0 2024-05-13T22:48:35.087 INFO:tasks.workunit.client.1.smithi130.stdout:2/592: mknod d4/db/dbf/d48/cd2 0 2024-05-13T22:48:35.087 INFO:tasks.workunit.client.1.smithi130.stdout:3/564: write d8/dc/d19/f30 [277210,130122] 0 2024-05-13T22:48:35.090 INFO:tasks.workunit.client.1.smithi130.stdout:7/577: stat d7/da/d1d/d5d/da6/d9/d4a/dbd/f6c 0 2024-05-13T22:48:35.091 INFO:tasks.workunit.client.1.smithi130.stdout:3/565: truncate d8/dc/d7b/da3/daf/fb7 2472500 0 2024-05-13T22:48:35.093 INFO:tasks.workunit.client.1.smithi130.stdout:0/621: getdents d0/d7/dd/d2e/d9e/da7/dc3 0 2024-05-13T22:48:35.096 INFO:tasks.workunit.client.1.smithi130.stdout:0/622: symlink d0/d7/dd/d17/d1f/d2c/d45/ld7 0 2024-05-13T22:48:35.100 INFO:tasks.workunit.client.1.smithi130.stdout:0/623: write d0/d7/dd/d17/d52/d67/d68/d96/fa2 [38415,33921] 0 2024-05-13T22:48:35.101 INFO:tasks.workunit.client.1.smithi130.stdout:7/578: dread d7/da/d1d/d5d/da6/d39/fb4 [0,4194304] 0 2024-05-13T22:48:35.101 INFO:tasks.workunit.client.1.smithi130.stdout:2/593: dwrite d4/db/dc/d18/d2a/d30/f9c [0,4194304] 0 2024-05-13T22:48:35.102 INFO:tasks.workunit.client.1.smithi130.stdout:3/566: link d8/dc/c11 d8/dc/d15/cc3 0 2024-05-13T22:48:35.104 INFO:tasks.workunit.client.1.smithi130.stdout:2/594: write d4/db/f2c [834529,90218] 0 2024-05-13T22:48:35.105 INFO:tasks.workunit.client.1.smithi130.stdout:2/595: fdatasync d4/db/dc/d18/d2a/d66/fa9 0 2024-05-13T22:48:35.106 INFO:tasks.workunit.client.1.smithi130.stdout:2/596: read d4/f7 [731546,97948] 0 2024-05-13T22:48:35.107 INFO:tasks.workunit.client.1.smithi130.stdout:5/565: truncate d6/d7/f46 2944419 0 2024-05-13T22:48:35.107 INFO:tasks.workunit.client.1.smithi130.stdout:6/571: write d4/d15/d1d/f22 [1763103,336] 0 2024-05-13T22:48:35.110 INFO:tasks.workunit.client.1.smithi130.stdout:4/580: truncate d8/f1f 1051771 0 2024-05-13T22:48:35.111 INFO:tasks.workunit.client.1.smithi130.stdout:7/579: symlink d7/da/d1d/lbf 0 2024-05-13T22:48:35.114 INFO:tasks.workunit.client.1.smithi130.stdout:2/597: dread d4/db/dbf/d42/d4b/f94 [0,4194304] 0 2024-05-13T22:48:35.115 INFO:tasks.workunit.client.1.smithi130.stdout:3/567: mkdir d8/dc/d19/d83/dba/dc4 0 2024-05-13T22:48:35.117 INFO:tasks.workunit.client.1.smithi130.stdout:4/581: chown d8/dc/d26/fa1 19 1 2024-05-13T22:48:35.122 INFO:tasks.workunit.client.1.smithi130.stdout:7/580: mkdir d7/da/d1d/d42/dc0 0 2024-05-13T22:48:35.126 INFO:tasks.workunit.client.1.smithi130.stdout:7/581: chown d7/f71 63978 1 2024-05-13T22:48:35.126 INFO:tasks.workunit.client.1.smithi130.stdout:5/566: creat d6/dd/d18/d6e/d82/fbb x:0 0 0 2024-05-13T22:48:35.127 INFO:tasks.workunit.client.1.smithi130.stdout:2/598: symlink d4/db/dc/ld3 0 2024-05-13T22:48:35.128 INFO:tasks.workunit.client.1.smithi130.stdout:9/625: sync 2024-05-13T22:48:35.131 INFO:tasks.workunit.client.1.smithi130.stdout:5/567: readlink d6/d1a/d92/la2 0 2024-05-13T22:48:35.131 INFO:tasks.workunit.client.1.smithi130.stdout:0/624: dread d0/d7/fb [4194304,4194304] 0 2024-05-13T22:48:35.133 INFO:tasks.workunit.client.1.smithi130.stdout:3/568: dwrite d8/dc/d15/f18 [0,4194304] 0 2024-05-13T22:48:35.134 INFO:tasks.workunit.client.1.smithi130.stdout:4/582: dwrite d8/dc/f5b [0,4194304] 0 2024-05-13T22:48:35.134 INFO:tasks.workunit.client.1.smithi130.stdout:6/572: dread d4/f57 [0,4194304] 0 2024-05-13T22:48:35.136 INFO:tasks.workunit.client.1.smithi130.stdout:9/626: write d1/d7/d23/d28/d32/d45/f53 [1124122,9112] 0 2024-05-13T22:48:35.136 INFO:tasks.workunit.client.1.smithi130.stdout:7/582: mknod d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/cc1 0 2024-05-13T22:48:35.137 INFO:tasks.workunit.client.1.smithi130.stdout:0/625: fsync d0/d7/dd/d17/d1f/d2c/d45/d57/f62 0 2024-05-13T22:48:35.142 INFO:tasks.workunit.client.1.smithi130.stdout:5/568: mkdir d6/dd/d18/d6e/d82/d4b/dbc 0 2024-05-13T22:48:35.143 INFO:tasks.workunit.client.1.smithi130.stdout:5/569: dread - d6/d7/d19/d4d/d6a/faa zero size 2024-05-13T22:48:35.144 INFO:tasks.workunit.client.1.smithi130.stdout:5/570: readlink d6/d7/d32/l4a 0 2024-05-13T22:48:35.150 INFO:tasks.workunit.client.1.smithi130.stdout:9/627: creat d1/d7/d69/d79/fcb x:0 0 0 2024-05-13T22:48:35.150 INFO:tasks.workunit.client.1.smithi130.stdout:5/571: dread d6/d1a/f48 [0,4194304] 0 2024-05-13T22:48:35.152 INFO:tasks.workunit.client.1.smithi130.stdout:7/583: fdatasync d7/da/d1d/d5d/da6/d39/da9/fb0 0 2024-05-13T22:48:35.152 INFO:tasks.workunit.client.1.smithi130.stdout:7/584: fsync d7/f71 0 2024-05-13T22:48:35.153 INFO:tasks.workunit.client.1.smithi130.stdout:6/573: truncate d4/d15/d1f/f2d 802845 0 2024-05-13T22:48:35.153 INFO:tasks.workunit.client.1.smithi130.stdout:0/626: creat d0/d7/dd/d2e/d9e/da7/fd8 x:0 0 0 2024-05-13T22:48:35.155 INFO:tasks.workunit.client.1.smithi130.stdout:3/569: symlink d8/dc/d19/lc5 0 2024-05-13T22:48:35.159 INFO:tasks.workunit.client.1.smithi130.stdout:4/583: creat d8/dc/d80/db0/fb4 x:0 0 0 2024-05-13T22:48:35.171 INFO:tasks.workunit.client.1.smithi130.stdout:4/584: dread d8/dc/d26/f2b [0,4194304] 0 2024-05-13T22:48:35.172 INFO:tasks.workunit.client.1.smithi130.stdout:4/585: chown d8/dc/d35/d33/d44/l5d 1 1 2024-05-13T22:48:35.173 INFO:tasks.workunit.client.1.smithi130.stdout:2/599: truncate d4/db/d26/d9e/fad 1730355 0 2024-05-13T22:48:35.175 INFO:tasks.workunit.client.1.smithi130.stdout:0/627: rename d0/d7/dd/c14 to d0/cd9 0 2024-05-13T22:48:35.176 INFO:tasks.workunit.client.1.smithi130.stdout:0/628: write d0/d7/dd/f46 [3819312,79267] 0 2024-05-13T22:48:35.179 INFO:tasks.workunit.client.1.smithi130.stdout:3/570: rename d8/dc/d15/dac/fc0 to d8/dc/d42/fc6 0 2024-05-13T22:48:35.180 INFO:tasks.workunit.client.1.smithi130.stdout:3/571: chown d8/d1a/d1c/l23 7 1 2024-05-13T22:48:35.183 INFO:tasks.workunit.client.1.smithi130.stdout:4/586: creat d8/df/dad/fb5 x:0 0 0 2024-05-13T22:48:35.183 INFO:tasks.workunit.client.1.smithi130.stdout:9/628: dwrite d1/fc9 [0,4194304] 0 2024-05-13T22:48:35.184 INFO:tasks.workunit.client.1.smithi130.stdout:5/572: truncate d6/d7/f46 565506 0 2024-05-13T22:48:35.187 INFO:tasks.workunit.client.1.smithi130.stdout:3/572: dread d8/dc/d7b/f99 [0,4194304] 0 2024-05-13T22:48:35.188 INFO:tasks.workunit.client.1.smithi130.stdout:2/600: symlink d4/db/dc/ld4 0 2024-05-13T22:48:35.192 INFO:tasks.workunit.client.1.smithi130.stdout:7/585: dwrite d7/da/d1d/d5d/da6/f4c [0,4194304] 0 2024-05-13T22:48:35.194 INFO:tasks.workunit.client.1.smithi130.stdout:0/629: creat d0/d7/dd/d17/d52/d5b/d60/fda x:0 0 0 2024-05-13T22:48:35.196 INFO:tasks.workunit.client.1.smithi130.stdout:4/587: dwrite d8/df/f59 [4194304,4194304] 0 2024-05-13T22:48:35.199 INFO:tasks.workunit.client.1.smithi130.stdout:9/629: rmdir d1/d7/d23/d28/d73/d8d 39 2024-05-13T22:48:35.199 INFO:tasks.workunit.client.1.smithi130.stdout:9/630: stat d1/d15/d51/d85 0 2024-05-13T22:48:35.201 INFO:tasks.workunit.client.1.smithi130.stdout:3/573: fsync d8/dc/d42/f8c 0 2024-05-13T22:48:35.203 INFO:tasks.workunit.client.1.smithi130.stdout:7/586: rmdir d7/da 39 2024-05-13T22:48:35.203 INFO:tasks.workunit.client.1.smithi130.stdout:3/574: stat d8/dc/d42/f74 0 2024-05-13T22:48:35.203 INFO:tasks.workunit.client.1.smithi130.stdout:2/601: creat d4/db/dc/d18/d2a/d30/d88/dc5/fd5 x:0 0 0 2024-05-13T22:48:35.211 INFO:tasks.workunit.client.1.smithi130.stdout:7/587: dread d7/da/f33 [0,4194304] 0 2024-05-13T22:48:35.211 INFO:tasks.workunit.client.1.smithi130.stdout:2/602: symlink d4/db/dc/d18/d2a/d66/ld6 0 2024-05-13T22:48:35.213 INFO:tasks.workunit.client.1.smithi130.stdout:7/588: write d7/da/d1d/d3d/d46/d92/fad [1277268,112536] 0 2024-05-13T22:48:35.214 INFO:tasks.workunit.client.1.smithi130.stdout:5/573: creat d6/d7/fbd x:0 0 0 2024-05-13T22:48:35.215 INFO:tasks.workunit.client.1.smithi130.stdout:3/575: unlink d8/d1a/d55/l8b 0 2024-05-13T22:48:35.218 INFO:tasks.workunit.client.1.smithi130.stdout:2/603: symlink d4/db/dc/d18/d2a/d30/d88/ld7 0 2024-05-13T22:48:35.219 INFO:tasks.workunit.client.1.smithi130.stdout:2/604: readlink d4/db/dbf/d48/l60 0 2024-05-13T22:48:35.219 INFO:tasks.workunit.client.1.smithi130.stdout:7/589: dwrite d7/da/d1d/d3d/d46/f6e [0,4194304] 0 2024-05-13T22:48:35.221 INFO:tasks.workunit.client.1.smithi130.stdout:5/574: symlink d6/d2f/lbe 0 2024-05-13T22:48:35.223 INFO:tasks.workunit.client.1.smithi130.stdout:3/576: mknod d8/dc/d15/d8f/cc7 0 2024-05-13T22:48:35.226 INFO:tasks.workunit.client.1.smithi130.stdout:8/551: sync 2024-05-13T22:48:35.228 INFO:tasks.workunit.client.1.smithi130.stdout:7/590: mknod d7/da/d1d/d5d/da6/d9/cc2 0 2024-05-13T22:48:35.229 INFO:tasks.workunit.client.1.smithi130.stdout:8/552: chown d1/d2/db/d15/d7e/d9f/laa 5 1 2024-05-13T22:48:35.230 INFO:tasks.workunit.client.1.smithi130.stdout:3/577: dwrite d8/dc/d19/d2c/f46 [0,4194304] 0 2024-05-13T22:48:35.232 INFO:tasks.workunit.client.1.smithi130.stdout:7/591: creat d7/da/d86/fc3 x:0 0 0 2024-05-13T22:48:35.232 INFO:tasks.workunit.client.1.smithi130.stdout:4/588: write d8/dc/d35/d33/d44/f54 [712861,35290] 0 2024-05-13T22:48:35.234 INFO:tasks.workunit.client.1.smithi130.stdout:8/553: creat d1/d2/db/d15/d18/dba/fca x:0 0 0 2024-05-13T22:48:35.235 INFO:tasks.workunit.client.1.smithi130.stdout:0/630: truncate d0/fad 2761758 0 2024-05-13T22:48:35.238 INFO:tasks.workunit.client.1.smithi130.stdout:4/589: dread d8/f1f [0,4194304] 0 2024-05-13T22:48:35.239 INFO:tasks.workunit.client.1.smithi130.stdout:5/575: link d6/dd/d18/d6e/d82/d71/lb1 d6/dd/d18/d6e/d82/d4b/dbc/lbf 0 2024-05-13T22:48:35.249 INFO:tasks.workunit.client.1.smithi130.stdout:8/554: symlink d1/d2/db/d15/d18/lcb 0 2024-05-13T22:48:35.252 INFO:tasks.workunit.client.1.smithi130.stdout:0/631: unlink d0/d7/dd/d69/da9/d8d/lab 0 2024-05-13T22:48:35.253 INFO:tasks.workunit.client.1.smithi130.stdout:0/632: read - d0/d7/dd/d2e/d9e/da7/fb8 zero size 2024-05-13T22:48:35.255 INFO:tasks.workunit.client.1.smithi130.stdout:0/633: write d0/d7/dd/d17/d52/d67/d68/d96/f84 [1759904,5081] 0 2024-05-13T22:48:35.256 INFO:tasks.workunit.client.1.smithi130.stdout:5/576: chown d6/dd/d18/d6e/d82/d4b/dbc/lbf 132220680 1 2024-05-13T22:48:35.258 INFO:tasks.workunit.client.1.smithi130.stdout:8/555: rmdir d1/d2/db/d15/d7e 39 2024-05-13T22:48:35.259 INFO:tasks.workunit.client.1.smithi130.stdout:3/578: truncate d8/d35/f48 2962044 0 2024-05-13T22:48:35.265 INFO:tasks.workunit.client.1.smithi130.stdout:7/592: dread d7/da/d1d/f6b [0,4194304] 0 2024-05-13T22:48:35.266 INFO:tasks.workunit.client.1.smithi130.stdout:7/593: write d7/da/d1d/d5d/da6/f1c [4585015,104450] 0 2024-05-13T22:48:35.269 INFO:tasks.workunit.client.1.smithi130.stdout:3/579: write d8/dc/d15/f4d [4638704,82971] 0 2024-05-13T22:48:35.271 INFO:tasks.workunit.client.1.smithi130.stdout:4/590: getdents d8/dc/d35/d33/d44/d77 0 2024-05-13T22:48:35.272 INFO:tasks.workunit.client.1.smithi130.stdout:0/634: symlink d0/d7/dd/d69/da9/dd2/ldb 0 2024-05-13T22:48:35.272 INFO:tasks.workunit.client.1.smithi130.stdout:5/577: dread d6/dd/f1b [0,4194304] 0 2024-05-13T22:48:35.273 INFO:tasks.workunit.client.1.smithi130.stdout:3/580: symlink d8/dc/d7b/lc8 0 2024-05-13T22:48:35.273 INFO:tasks.workunit.client.1.smithi130.stdout:7/594: mkdir d7/da/d1d/d5d/da6/d9/d4a/dbd/dbc/dc4 0 2024-05-13T22:48:35.274 INFO:tasks.workunit.client.1.smithi130.stdout:7/595: write d7/da/d1d/d5d/da6/d9/d66/f7a [559829,55343] 0 2024-05-13T22:48:35.275 INFO:tasks.workunit.client.1.smithi130.stdout:0/635: symlink d0/d7/dd/d69/da9/ldc 0 2024-05-13T22:48:35.276 INFO:tasks.workunit.client.1.smithi130.stdout:0/636: write d0/f1e [928080,129657] 0 2024-05-13T22:48:35.279 INFO:tasks.workunit.client.1.smithi130.stdout:0/637: dread d0/f8 [0,4194304] 0 2024-05-13T22:48:35.302 INFO:tasks.workunit.client.1.smithi130.stdout:5/578: creat d6/d7/d19/d4d/d6a/fc0 x:0 0 0 2024-05-13T22:48:35.302 INFO:tasks.workunit.client.1.smithi130.stdout:8/556: write d1/d2/fc5 [3639647,72518] 0 2024-05-13T22:48:35.303 INFO:tasks.workunit.client.1.smithi130.stdout:7/596: write d7/da/d1d/d42/f74 [2872669,19902] 0 2024-05-13T22:48:35.304 INFO:tasks.workunit.client.1.smithi130.stdout:3/581: symlink d8/dc/lc9 0 2024-05-13T22:48:35.308 INFO:tasks.workunit.client.1.smithi130.stdout:0/638: dread d0/d7/dd/d2e/f4e [0,4194304] 0 2024-05-13T22:48:35.309 INFO:tasks.workunit.client.1.smithi130.stdout:5/579: mkdir d6/d7/d19/dc1 0 2024-05-13T22:48:35.311 INFO:tasks.workunit.client.1.smithi130.stdout:3/582: dwrite d8/f33 [0,4194304] 0 2024-05-13T22:48:35.313 INFO:tasks.workunit.client.1.smithi130.stdout:7/597: unlink d7/da/f33 0 2024-05-13T22:48:35.315 INFO:tasks.workunit.client.1.smithi130.stdout:0/639: creat d0/d7/dd/d2e/d9e/da7/dc3/fdd x:0 0 0 2024-05-13T22:48:35.319 INFO:tasks.workunit.client.1.smithi130.stdout:5/580: dwrite d6/d7/d19/d4d/d6a/faa [0,4194304] 0 2024-05-13T22:48:35.321 INFO:tasks.workunit.client.1.smithi130.stdout:0/640: fdatasync d0/d7/dd/d17/d1f/d2c/d45/d57/f58 0 2024-05-13T22:48:35.322 INFO:tasks.workunit.client.1.smithi130.stdout:5/581: symlink d6/d7/d19/d4d/d6a/lc2 0 2024-05-13T22:48:35.324 INFO:tasks.workunit.client.1.smithi130.stdout:3/583: link d8/dc/d19/ca9 d8/cca 0 2024-05-13T22:48:35.325 INFO:tasks.workunit.client.1.smithi130.stdout:5/582: chown d6/d7/d19/d4d/d6a 78 1 2024-05-13T22:48:35.325 INFO:tasks.workunit.client.1.smithi130.stdout:7/598: dwrite d7/da/d1d/d3d/d46/f4f [0,4194304] 0 2024-05-13T22:48:35.326 INFO:tasks.workunit.client.1.smithi130.stdout:5/583: fdatasync d6/dd/fb6 0 2024-05-13T22:48:35.330 INFO:tasks.workunit.client.1.smithi130.stdout:7/599: rmdir d7/da/d1d/d5d/da6/d9/d4a/dbd/dbc 39 2024-05-13T22:48:35.331 INFO:tasks.workunit.client.1.smithi130.stdout:0/641: getdents d0/d7/dd/d17/d52/d5b 0 2024-05-13T22:48:35.333 INFO:tasks.workunit.client.1.smithi130.stdout:3/584: dwrite d8/dc/d7b/f90 [0,4194304] 0 2024-05-13T22:48:35.333 INFO:tasks.workunit.client.1.smithi130.stdout:7/600: rmdir d7/da/d1d/db1 39 2024-05-13T22:48:35.335 INFO:tasks.workunit.client.1.smithi130.stdout:1/568: sync 2024-05-13T22:48:35.335 INFO:tasks.workunit.client.1.smithi130.stdout:0/642: write d0/d7/dd/d2e/d4b/f55 [262574,110778] 0 2024-05-13T22:48:35.338 INFO:tasks.workunit.client.1.smithi130.stdout:5/584: dwrite d6/dd/d18/d6e/f8b [0,4194304] 0 2024-05-13T22:48:35.339 INFO:tasks.workunit.client.1.smithi130.stdout:7/601: chown d7/da/d1d/l93 3210 1 2024-05-13T22:48:35.340 INFO:tasks.workunit.client.1.smithi130.stdout:1/569: readlink db/d1b/da3/d2a/l57 0 2024-05-13T22:48:35.341 INFO:tasks.workunit.client.1.smithi130.stdout:1/570: write db/d4a/f70 [1188134,61622] 0 2024-05-13T22:48:35.343 INFO:tasks.workunit.client.1.smithi130.stdout:5/585: truncate d6/d7/d19/d4d/f87 377743 0 2024-05-13T22:48:35.343 INFO:tasks.workunit.client.1.smithi130.stdout:8/557: dread d1/d2/db/d15/d18/f31 [0,4194304] 0 2024-05-13T22:48:35.344 INFO:tasks.workunit.client.1.smithi130.stdout:5/586: stat d6/d7/d19/d4d/d6a/f9c 0 2024-05-13T22:48:35.345 INFO:tasks.workunit.client.1.smithi130.stdout:5/587: chown d6/f7c 7681487 1 2024-05-13T22:48:35.346 INFO:tasks.workunit.client.1.smithi130.stdout:8/558: write d1/d2/db/d15/d18/dba/fca [268529,40076] 0 2024-05-13T22:48:35.349 INFO:tasks.workunit.client.1.smithi130.stdout:7/602: creat d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/fc5 x:0 0 0 2024-05-13T22:48:35.350 INFO:tasks.workunit.client.1.smithi130.stdout:7/603: stat d7/da/d1d/d42/d43 0 2024-05-13T22:48:35.351 INFO:tasks.workunit.client.1.smithi130.stdout:1/571: rename db/d58/d74/l9b to db/d82/d9f/da0/lb6 0 2024-05-13T22:48:35.356 INFO:tasks.workunit.client.1.smithi130.stdout:5/588: dread d6/dd/d18/d6e/d76/f80 [0,4194304] 0 2024-05-13T22:48:35.362 INFO:tasks.workunit.client.1.smithi130.stdout:5/589: dwrite d6/d1a/f40 [0,4194304] 0 2024-05-13T22:48:35.363 INFO:tasks.workunit.client.1.smithi130.stdout:8/559: creat d1/dc/dc3/fcc x:0 0 0 2024-05-13T22:48:35.364 INFO:tasks.workunit.client.1.smithi130.stdout:5/590: chown d6/d1a/d92 7976 1 2024-05-13T22:48:35.364 INFO:tasks.workunit.client.1.smithi130.stdout:3/585: link d8/dc/d15/ca5 d8/dc/d7b/ccb 0 2024-05-13T22:48:35.365 INFO:tasks.workunit.client.1.smithi130.stdout:7/604: mknod d7/da/d1d/d5d/da6/d39/cc6 0 2024-05-13T22:48:35.368 INFO:tasks.workunit.client.1.smithi130.stdout:5/591: mknod d6/d2f/cc3 0 2024-05-13T22:48:35.371 INFO:tasks.workunit.client.1.smithi130.stdout:3/586: dwrite d8/f2b [0,4194304] 0 2024-05-13T22:48:35.371 INFO:tasks.workunit.client.1.smithi130.stdout:8/560: mkdir d1/d2/db/d15/d7e/dcd 0 2024-05-13T22:48:35.375 INFO:tasks.workunit.client.1.smithi130.stdout:5/592: creat d6/d7/d19/fc4 x:0 0 0 2024-05-13T22:48:35.376 INFO:tasks.workunit.client.1.smithi130.stdout:5/593: write d6/d1a/f2a [731804,127220] 0 2024-05-13T22:48:35.377 INFO:tasks.workunit.client.1.smithi130.stdout:7/605: rename d7/da/d1d/d5d/da6/d9/d4a/dbd/dbc/dc4 to d7/da/d1d/d3d/d46/dc7 0 2024-05-13T22:48:35.379 INFO:tasks.workunit.client.1.smithi130.stdout:3/587: mkdir d8/dc/d15/dcc 0 2024-05-13T22:48:35.380 INFO:tasks.workunit.client.1.smithi130.stdout:0/643: dread d0/d7/dd/f41 [0,4194304] 0 2024-05-13T22:48:35.380 INFO:tasks.workunit.client.1.smithi130.stdout:9/631: sync 2024-05-13T22:48:35.380 INFO:tasks.workunit.client.1.smithi130.stdout:6/574: sync 2024-05-13T22:48:35.382 INFO:tasks.workunit.client.1.smithi130.stdout:9/632: write d1/d7/d23/fc2 [221291,54507] 0 2024-05-13T22:48:35.388 INFO:tasks.workunit.client.1.smithi130.stdout:8/561: creat d1/d2/db/d15/d16/d54/d58/db9/fce x:0 0 0 2024-05-13T22:48:35.390 INFO:tasks.workunit.client.1.smithi130.stdout:1/572: rmdir db/d58/d74 0 2024-05-13T22:48:35.397 INFO:tasks.workunit.client.1.smithi130.stdout:5/594: rename d6/d1a/d72/c74 to d6/d1a/d72/cc5 0 2024-05-13T22:48:35.398 INFO:tasks.workunit.client.1.smithi130.stdout:9/633: mknod d1/d7/d23/d28/d2d/d2f/d63/d6b/ccc 0 2024-05-13T22:48:35.399 INFO:tasks.workunit.client.1.smithi130.stdout:1/573: write db/d1b/da3/d49/d56/d7f/f96 [3119894,22617] 0 2024-05-13T22:48:35.400 INFO:tasks.workunit.client.1.smithi130.stdout:7/606: link d7/da/d1d/d5d/da6/d9/cc2 d7/d8f/cc8 0 2024-05-13T22:48:35.404 INFO:tasks.workunit.client.1.smithi130.stdout:6/575: rename d4/d15/d1f/d46/d52/f87 to d4/d15/d1d/d60/da2/da4/dac/fc5 0 2024-05-13T22:48:35.404 INFO:tasks.workunit.client.1.smithi130.stdout:0/644: dread d0/d7/dd/d2e/d4b/f65 [0,4194304] 0 2024-05-13T22:48:35.405 INFO:tasks.workunit.client.1.smithi130.stdout:0/645: chown d0/d7/dd/d69/dac 5318423 1 2024-05-13T22:48:35.405 INFO:tasks.workunit.client.1.smithi130.stdout:8/562: unlink d1/d2/db/d15/d18/d1a/c28 0 2024-05-13T22:48:35.406 INFO:tasks.workunit.client.1.smithi130.stdout:5/595: creat d6/dd/d18/d6e/d82/d56/fc6 x:0 0 0 2024-05-13T22:48:35.407 INFO:tasks.workunit.client.1.smithi130.stdout:1/574: mknod db/d1b/da3/d2a/d35/cb7 0 2024-05-13T22:48:35.409 INFO:tasks.workunit.client.1.smithi130.stdout:6/576: fsync d4/d15/d1f/f84 0 2024-05-13T22:48:35.411 INFO:tasks.workunit.client.1.smithi130.stdout:0/646: creat d0/d7/dd/d2e/d9e/da7/fde x:0 0 0 2024-05-13T22:48:35.412 INFO:tasks.workunit.client.1.smithi130.stdout:0/647: truncate d0/fb7 298281 0 2024-05-13T22:48:35.416 INFO:tasks.workunit.client.1.smithi130.stdout:6/577: unlink d4/d15/d1f/d3c/c91 0 2024-05-13T22:48:35.417 INFO:tasks.workunit.client.1.smithi130.stdout:0/648: mkdir d0/d7/dd/d2e/d9e/da7/dc3/ddf 0 2024-05-13T22:48:35.419 INFO:tasks.workunit.client.1.smithi130.stdout:1/575: dread db/d1b/da3/d2a/d38/d46/d6d/f9e [0,4194304] 0 2024-05-13T22:48:35.420 INFO:tasks.workunit.client.1.smithi130.stdout:1/576: chown db/d82 57031968 1 2024-05-13T22:48:35.421 INFO:tasks.workunit.client.1.smithi130.stdout:7/607: getdents d7/da/d1d/d42 0 2024-05-13T22:48:35.422 INFO:tasks.workunit.client.1.smithi130.stdout:0/649: mknod d0/d7/dd/d17/d52/d67/d68/d96/ce0 0 2024-05-13T22:48:35.423 INFO:tasks.workunit.client.1.smithi130.stdout:5/596: dread d6/d7/d19/f2c [0,4194304] 0 2024-05-13T22:48:35.425 INFO:tasks.workunit.client.1.smithi130.stdout:1/577: creat db/d1b/da3/d49/fb8 x:0 0 0 2024-05-13T22:48:35.427 INFO:tasks.workunit.client.1.smithi130.stdout:5/597: mkdir d6/dd/d18/d6e/dc7 0 2024-05-13T22:48:35.427 INFO:tasks.workunit.client.1.smithi130.stdout:7/608: dread d7/da/d1d/d5d/da6/f4c [0,4194304] 0 2024-05-13T22:48:35.427 INFO:tasks.workunit.client.1.smithi130.stdout:6/578: dread d4/d15/d1f/f3e [0,4194304] 0 2024-05-13T22:48:35.438 INFO:tasks.workunit.client.1.smithi130.stdout:1/578: dwrite db/d1b/da3/d72/f79 [0,4194304] 0 2024-05-13T22:48:35.438 INFO:tasks.workunit.client.1.smithi130.stdout:5/598: rename d6/dd/d18/l16 to d6/dd/d18/d6e/d76/lc8 0 2024-05-13T22:48:35.438 INFO:tasks.workunit.client.1.smithi130.stdout:7/609: mkdir d7/da/d1d/d5d/da6/d9/d4a/dbd/dc9 0 2024-05-13T22:48:35.464 INFO:tasks.workunit.client.1.smithi130.stdout:2/605: sync 2024-05-13T22:48:35.464 INFO:tasks.workunit.client.1.smithi130.stdout:4/591: sync 2024-05-13T22:48:35.465 INFO:tasks.workunit.client.1.smithi130.stdout:7/610: mknod d7/da/d1d/db1/dbe/cca 0 2024-05-13T22:48:35.465 INFO:tasks.workunit.client.1.smithi130.stdout:4/592: stat d8/dc/f9a 0 2024-05-13T22:48:35.467 INFO:tasks.workunit.client.1.smithi130.stdout:1/579: rename db/d1b/da3/d2a/c2c to db/cb9 0 2024-05-13T22:48:35.470 INFO:tasks.workunit.client.1.smithi130.stdout:4/593: chown d8/dc/d29/c4c 117 1 2024-05-13T22:48:35.473 INFO:tasks.workunit.client.1.smithi130.stdout:7/611: truncate d7/da/d1d/d42/f58 3694876 0 2024-05-13T22:48:35.473 INFO:tasks.workunit.client.1.smithi130.stdout:4/594: chown d8/dc/d35/d33/d3f 16 1 2024-05-13T22:48:35.474 INFO:tasks.workunit.client.1.smithi130.stdout:1/580: creat db/d1b/da3/d2a/d35/fba x:0 0 0 2024-05-13T22:48:35.475 INFO:tasks.workunit.client.1.smithi130.stdout:2/606: dwrite d4/db/dbf/d48/d4a/f52 [4194304,4194304] 0 2024-05-13T22:48:35.477 INFO:tasks.workunit.client.1.smithi130.stdout:5/599: rename d6/ca1 to d6/d7/d19/d4d/d6a/cc9 0 2024-05-13T22:48:35.479 INFO:tasks.workunit.client.1.smithi130.stdout:5/600: fsync d6/fac 0 2024-05-13T22:48:35.482 INFO:tasks.workunit.client.1.smithi130.stdout:1/581: mknod db/d1b/da3/d5e/cbb 0 2024-05-13T22:48:35.485 INFO:tasks.workunit.client.1.smithi130.stdout:5/601: write d6/dd/d18/d6e/f8b [3688222,47753] 0 2024-05-13T22:48:35.486 INFO:tasks.workunit.client.1.smithi130.stdout:7/612: link d7/da/d1d/d5d/f89 d7/da/d1d/fcb 0 2024-05-13T22:48:35.488 INFO:tasks.workunit.client.1.smithi130.stdout:8/563: dwrite d1/d2/db/d15/f74 [0,4194304] 0 2024-05-13T22:48:35.489 INFO:tasks.workunit.client.1.smithi130.stdout:5/602: dread d6/d1a/d92/f96 [0,4194304] 0 2024-05-13T22:48:35.490 INFO:tasks.workunit.client.1.smithi130.stdout:1/582: fdatasync db/d1b/da3/d2a/d38/f60 0 2024-05-13T22:48:35.491 INFO:tasks.workunit.client.1.smithi130.stdout:7/613: dread d7/da/d1d/d5d/da6/d9/d66/d65/f91 [0,4194304] 0 2024-05-13T22:48:35.491 INFO:tasks.workunit.client.1.smithi130.stdout:2/607: link d4/db/dc/d18/d2a/d30/f9c d4/db/dc/d18/d2a/d30/fd8 0 2024-05-13T22:48:35.494 INFO:tasks.workunit.client.1.smithi130.stdout:8/564: chown d1/d2/db/d15/f92 1 1 2024-05-13T22:48:35.495 INFO:tasks.workunit.client.1.smithi130.stdout:7/614: read d7/da/d1d/d42/d43/f4d [2949492,13078] 0 2024-05-13T22:48:35.497 INFO:tasks.workunit.client.1.smithi130.stdout:7/615: fsync d7/da/d86/f8a 0 2024-05-13T22:48:35.499 INFO:tasks.workunit.client.1.smithi130.stdout:3/588: sync 2024-05-13T22:48:35.500 INFO:tasks.workunit.client.1.smithi130.stdout:9/634: sync 2024-05-13T22:48:35.500 INFO:tasks.workunit.client.1.smithi130.stdout:1/583: truncate db/d1b/da3/d72/d75/f81 1364507 0 2024-05-13T22:48:35.504 INFO:tasks.workunit.client.1.smithi130.stdout:2/608: symlink d4/db/dbf/d42/ld9 0 2024-05-13T22:48:35.505 INFO:tasks.workunit.client.1.smithi130.stdout:6/579: write d4/d15/d1f/d46/d50/f79 [2503452,110957] 0 2024-05-13T22:48:35.506 INFO:tasks.workunit.client.1.smithi130.stdout:3/589: creat d8/dc/d15/fcd x:0 0 0 2024-05-13T22:48:35.507 INFO:tasks.workunit.client.1.smithi130.stdout:9/635: write d1/d7/d23/d28/d32/d65/fc0 [779053,104405] 0 2024-05-13T22:48:35.508 INFO:tasks.workunit.client.1.smithi130.stdout:1/584: creat db/d1b/da3/d49/d56/fbc x:0 0 0 2024-05-13T22:48:35.509 INFO:tasks.workunit.client.1.smithi130.stdout:9/636: stat d1/d7/d23/d3d/d43/fb1 0 2024-05-13T22:48:35.509 INFO:tasks.workunit.client.1.smithi130.stdout:1/585: readlink db/d1b/l1c 0 2024-05-13T22:48:35.510 INFO:tasks.workunit.client.1.smithi130.stdout:8/565: mkdir d1/d2/db/d15/d7e/dc2/dcf 0 2024-05-13T22:48:35.510 INFO:tasks.workunit.client.1.smithi130.stdout:4/595: dread d8/f1b [0,4194304] 0 2024-05-13T22:48:35.511 INFO:tasks.workunit.client.1.smithi130.stdout:8/566: dread - d1/d2/db/d15/d7e/d9f/fc9 zero size 2024-05-13T22:48:35.511 INFO:tasks.workunit.client.1.smithi130.stdout:0/650: truncate d0/d7/dd/d17/d52/f9a 3507233 0 2024-05-13T22:48:35.512 INFO:tasks.workunit.client.1.smithi130.stdout:2/609: symlink d4/db/d26/d9e/lda 0 2024-05-13T22:48:35.513 INFO:tasks.workunit.client.1.smithi130.stdout:6/580: creat d4/d15/d1d/d60/da2/da4/fc6 x:0 0 0 2024-05-13T22:48:35.514 INFO:tasks.workunit.client.1.smithi130.stdout:3/590: symlink d8/dc/d19/d83/lce 0 2024-05-13T22:48:35.516 INFO:tasks.workunit.client.1.smithi130.stdout:3/591: write d8/dc/d19/d2c/f6b [4120662,8542] 0 2024-05-13T22:48:35.519 INFO:tasks.workunit.client.1.smithi130.stdout:3/592: write d8/f4b [4130720,108842] 0 2024-05-13T22:48:35.520 INFO:tasks.workunit.client.1.smithi130.stdout:4/596: rename d8/df to d8/dc/d35/d33/d44/d77/d94/db6 0 2024-05-13T22:48:35.520 INFO:tasks.workunit.client.1.smithi130.stdout:0/651: mkdir d0/d7/dd/d2e/d9e/da7/de1 0 2024-05-13T22:48:35.523 INFO:tasks.workunit.client.1.smithi130.stdout:8/567: mkdir d1/d2/db/d15/d18/d42/d53/dd0 0 2024-05-13T22:48:35.523 INFO:tasks.workunit.client.1.smithi130.stdout:3/593: readlink d8/dc/d7b/l88 0 2024-05-13T22:48:35.524 INFO:tasks.workunit.client.1.smithi130.stdout:6/581: dwrite d4/d15/d1f/f20 [0,4194304] 0 2024-05-13T22:48:35.526 INFO:tasks.workunit.client.1.smithi130.stdout:9/637: dwrite d1/d7/d23/d28/f97 [12582912,4194304] 0 2024-05-13T22:48:35.526 INFO:tasks.workunit.client.1.smithi130.stdout:0/652: dread - d0/d7/dd/d17/d1f/d2c/fbd zero size 2024-05-13T22:48:35.527 INFO:tasks.workunit.client.1.smithi130.stdout:6/582: readlink d4/d15/d1f/d3c/d70/lba 0 2024-05-13T22:48:35.527 INFO:tasks.workunit.client.1.smithi130.stdout:8/568: write d1/d2/db/d15/d18/fb0 [993842,102552] 0 2024-05-13T22:48:35.529 INFO:tasks.workunit.client.1.smithi130.stdout:0/653: chown d0/d7/dd/d17/d52/d5b/d5f/c91 42 1 2024-05-13T22:48:35.529 INFO:tasks.workunit.client.1.smithi130.stdout:1/586: link db/le db/d1b/da3/d2a/d38/d46/lbd 0 2024-05-13T22:48:35.534 INFO:tasks.workunit.client.1.smithi130.stdout:4/597: rename d8/dc/l4d to d8/dc/d80/d96/lb7 0 2024-05-13T22:48:35.535 INFO:tasks.workunit.client.1.smithi130.stdout:8/569: creat d1/d2/db/d15/d18/dba/fd1 x:0 0 0 2024-05-13T22:48:35.539 INFO:tasks.workunit.client.1.smithi130.stdout:6/583: rename d4/d15/d1f/cbd to d4/d15/d1d/d28/d75/da8/cc7 0 2024-05-13T22:48:35.542 INFO:tasks.workunit.client.1.smithi130.stdout:0/654: mknod d0/d7/dd/d17/d1f/db0/ce2 0 2024-05-13T22:48:35.544 INFO:tasks.workunit.client.1.smithi130.stdout:1/587: link db/d1b/da3/d49/f61 db/d1b/da3/d2a/d38/d46/d6d/fbe 0 2024-05-13T22:48:35.545 INFO:tasks.workunit.client.1.smithi130.stdout:9/638: getdents d1/d7/d23/d3d 0 2024-05-13T22:48:35.554 INFO:tasks.workunit.client.1.smithi130.stdout:0/655: mkdir d0/d7/dd/d17/d1f/d2c/de3 0 2024-05-13T22:48:35.557 INFO:tasks.workunit.client.1.smithi130.stdout:1/588: symlink db/d82/lbf 0 2024-05-13T22:48:35.564 INFO:tasks.workunit.client.1.smithi130.stdout:0/656: symlink d0/d7/dd/d17/d52/d5b/le4 0 2024-05-13T22:48:35.571 INFO:tasks.workunit.client.1.smithi130.stdout:0/657: creat d0/d7/dd/d2e/fe5 x:0 0 0 2024-05-13T22:48:35.571 INFO:tasks.workunit.client.1.smithi130.stdout:6/584: dread d4/d15/d1d/d28/f8f [0,4194304] 0 2024-05-13T22:48:35.573 INFO:tasks.workunit.client.1.smithi130.stdout:0/658: write d0/d7/dd/d2e/fe5 [16536,21647] 0 2024-05-13T22:48:35.577 INFO:tasks.workunit.client.1.smithi130.stdout:5/603: dwrite d6/d2f/fa4 [0,4194304] 0 2024-05-13T22:48:35.579 INFO:tasks.workunit.client.1.smithi130.stdout:6/585: dread d4/d15/d1d/d28/d75/f80 [0,4194304] 0 2024-05-13T22:48:35.580 INFO:tasks.workunit.client.1.smithi130.stdout:7/616: truncate d7/da/d1d/d5d/da6/f4c 2436767 0 2024-05-13T22:48:35.584 INFO:tasks.workunit.client.1.smithi130.stdout:5/604: creat d6/dd/d18/fca x:0 0 0 2024-05-13T22:48:35.585 INFO:tasks.workunit.client.1.smithi130.stdout:5/605: write d6/dd/d18/d6e/d82/d4b/f99 [722050,82909] 0 2024-05-13T22:48:35.586 INFO:tasks.workunit.client.1.smithi130.stdout:5/606: stat d6/d7/f25 0 2024-05-13T22:48:35.587 INFO:tasks.workunit.client.1.smithi130.stdout:0/659: fsync d0/d7/dd/d2e/f8c 0 2024-05-13T22:48:35.595 INFO:tasks.workunit.client.1.smithi130.stdout:5/607: write d6/d7/f22 [1845279,118941] 0 2024-05-13T22:48:35.597 INFO:tasks.workunit.client.1.smithi130.stdout:5/608: fsync d6/dd/d18/d6e/d82/d4b/f99 0 2024-05-13T22:48:35.600 INFO:tasks.workunit.client.1.smithi130.stdout:4/598: rename d8/dc/d80/d96 to d8/dc/d35/d33/d44/db8 0 2024-05-13T22:48:35.601 INFO:tasks.workunit.client.1.smithi130.stdout:8/570: rename d1 to d1/d2/db/d15/d18/d42/d53/dd0/dd2 22 2024-05-13T22:48:35.603 INFO:tasks.workunit.client.1.smithi130.stdout:2/610: dwrite d4/db/d26/d9e/fad [0,4194304] 0 2024-05-13T22:48:35.605 INFO:tasks.workunit.client.1.smithi130.stdout:2/611: read d4/db/d26/f55 [713907,11563] 0 2024-05-13T22:48:35.618 INFO:tasks.workunit.client.1.smithi130.stdout:6/586: read - d4/d15/d1f/d3c/fbf zero size 2024-05-13T22:48:35.618 INFO:tasks.workunit.client.1.smithi130.stdout:7/617: dread d7/da/d1d/d42/d43/f8e [0,4194304] 0 2024-05-13T22:48:35.619 INFO:tasks.workunit.client.1.smithi130.stdout:9/639: truncate d1/d7/d23/d28/d32/d45/f53 856666 0 2024-05-13T22:48:35.620 INFO:tasks.workunit.client.1.smithi130.stdout:4/599: dread d8/dc/d35/f8a [0,4194304] 0 2024-05-13T22:48:35.627 INFO:tasks.workunit.client.1.smithi130.stdout:5/609: mknod d6/dd/d18/d6e/d82/d4b/dbc/ccb 0 2024-05-13T22:48:35.630 INFO:tasks.workunit.client.1.smithi130.stdout:2/612: creat d4/db/dc/d18/d2a/d30/d88/d90/fdb x:0 0 0 2024-05-13T22:48:35.636 INFO:tasks.workunit.client.1.smithi130.stdout:8/571: creat d1/d2/db/d15/d18/d42/d53/dd0/fd3 x:0 0 0 2024-05-13T22:48:35.641 INFO:tasks.workunit.client.1.smithi130.stdout:9/640: rmdir d1/d7/d23/d3d/d43 39 2024-05-13T22:48:35.642 INFO:tasks.workunit.client.1.smithi130.stdout:7/618: truncate d7/da/d1d/d5d/da6/d9/d4a/fba 2448359 0 2024-05-13T22:48:35.662 INFO:tasks.workunit.client.1.smithi130.stdout:3/594: sync 2024-05-13T22:48:35.662 INFO:tasks.workunit.client.1.smithi130.stdout:4/600: mknod d8/dc/d35/d33/d44/d56/cb9 0 2024-05-13T22:48:35.662 INFO:tasks.workunit.client.1.smithi130.stdout:2/613: creat d4/db/dbf/d42/d4b/fdc x:0 0 0 2024-05-13T22:48:35.664 INFO:tasks.workunit.client.1.smithi130.stdout:5/610: dread d6/d1a/f48 [0,4194304] 0 2024-05-13T22:48:35.664 INFO:tasks.workunit.client.1.smithi130.stdout:9/641: unlink d1/d7/d69/d79/f8a 0 2024-05-13T22:48:35.665 INFO:tasks.workunit.client.1.smithi130.stdout:2/614: stat d4/db/dbf/ld1 0 2024-05-13T22:48:35.668 INFO:tasks.workunit.client.1.smithi130.stdout:2/615: fsync d4/db/d26/f2d 0 2024-05-13T22:48:35.671 INFO:tasks.workunit.client.1.smithi130.stdout:7/619: mknod d7/ccc 0 2024-05-13T22:48:35.672 INFO:tasks.workunit.client.1.smithi130.stdout:6/587: dwrite d4/d15/d1f/f74 [0,4194304] 0 2024-05-13T22:48:35.673 INFO:tasks.workunit.client.1.smithi130.stdout:3/595: dwrite d8/d1a/d1c/f25 [0,4194304] 0 2024-05-13T22:48:35.673 INFO:tasks.workunit.client.1.smithi130.stdout:6/588: write d4/fc4 [520926,53420] 0 2024-05-13T22:48:35.677 INFO:tasks.workunit.client.1.smithi130.stdout:9/642: symlink d1/d7/d23/lcd 0 2024-05-13T22:48:35.681 INFO:tasks.workunit.client.1.smithi130.stdout:3/596: dwrite d8/d35/fb0 [0,4194304] 0 2024-05-13T22:48:35.685 INFO:tasks.workunit.client.1.smithi130.stdout:5/611: symlink d6/dd/d18/d6e/d82/d56/lcc 0 2024-05-13T22:48:35.692 INFO:tasks.workunit.client.1.smithi130.stdout:2/616: creat d4/db/dbf/d48/d4a/d76/fdd x:0 0 0 2024-05-13T22:48:35.693 INFO:tasks.workunit.client.1.smithi130.stdout:2/617: readlink d4/db/dc/d18/d2a/d66/l7c 0 2024-05-13T22:48:35.694 INFO:tasks.workunit.client.1.smithi130.stdout:2/618: truncate d4/db/dbf/f95 56442 0 2024-05-13T22:48:35.699 INFO:tasks.workunit.client.1.smithi130.stdout:3/597: creat d8/dc/d7b/fcf x:0 0 0 2024-05-13T22:48:35.704 INFO:tasks.workunit.client.1.smithi130.stdout:5/612: creat d6/dd/fcd x:0 0 0 2024-05-13T22:48:35.705 INFO:tasks.workunit.client.1.smithi130.stdout:3/598: dwrite d8/fbd [0,4194304] 0 2024-05-13T22:48:35.706 INFO:tasks.workunit.client.1.smithi130.stdout:3/599: dread - d8/dc/d15/f76 zero size 2024-05-13T22:48:35.707 INFO:tasks.workunit.client.1.smithi130.stdout:4/601: link d8/dc/d35/d33/d3f/d57/l71 d8/dc/d35/d33/d44/d61/lba 0 2024-05-13T22:48:35.710 INFO:tasks.workunit.client.1.smithi130.stdout:3/600: symlink d8/d35/ld0 0 2024-05-13T22:48:35.717 INFO:tasks.workunit.client.1.smithi130.stdout:2/619: rename d4/db/dc/d18/d2a/d30/d88/d90/d93 to d4/db/dde 0 2024-05-13T22:48:35.718 INFO:tasks.workunit.client.1.smithi130.stdout:2/620: write d4/f1b [310190,65955] 0 2024-05-13T22:48:35.718 INFO:tasks.workunit.client.1.smithi130.stdout:2/621: stat d4/db/d26 0 2024-05-13T22:48:35.718 INFO:tasks.workunit.client.1.smithi130.stdout:2/622: stat d4/db/d26/f55 0 2024-05-13T22:48:35.720 INFO:tasks.workunit.client.1.smithi130.stdout:9/643: link d1/d7/cba d1/d7/d23/d28/cce 0 2024-05-13T22:48:35.722 INFO:tasks.workunit.client.1.smithi130.stdout:4/602: mkdir d8/dbb 0 2024-05-13T22:48:35.725 INFO:tasks.workunit.client.1.smithi130.stdout:4/603: fsync d8/dc/d35/d33/d3f/d57/fb1 0 2024-05-13T22:48:35.726 INFO:tasks.workunit.client.1.smithi130.stdout:2/623: dwrite d4/f5 [4194304,4194304] 0 2024-05-13T22:48:35.726 INFO:tasks.workunit.client.1.smithi130.stdout:7/620: dread d7/da/d1d/d5d/da6/d39/f7b [0,4194304] 0 2024-05-13T22:48:35.726 INFO:tasks.workunit.client.1.smithi130.stdout:9/644: truncate d1/d7/d23/d28/d2d/d2f/f93 1147622 0 2024-05-13T22:48:35.727 INFO:tasks.workunit.client.1.smithi130.stdout:9/645: stat d1/db/cc5 0 2024-05-13T22:48:35.728 INFO:tasks.workunit.client.1.smithi130.stdout:2/624: write d4/db/dbf/d48/d4a/f8c [6821462,84840] 0 2024-05-13T22:48:35.728 INFO:tasks.workunit.client.1.smithi130.stdout:5/613: getdents d6/d7/d19/d4d/d8a 0 2024-05-13T22:48:35.730 INFO:tasks.workunit.client.1.smithi130.stdout:4/604: truncate d8/dc/d35/d33/d44/d77/d94/db6/faf 903880 0 2024-05-13T22:48:35.730 INFO:tasks.workunit.client.1.smithi130.stdout:7/621: truncate d7/da/d1d/d5d/da6/d9/d66/d65/faa 164890 0 2024-05-13T22:48:35.731 INFO:tasks.workunit.client.1.smithi130.stdout:5/614: creat d6/dd/d18/fce x:0 0 0 2024-05-13T22:48:35.732 INFO:tasks.workunit.client.1.smithi130.stdout:9/646: creat d1/d7/d23/d28/d2d/d2f/d63/d6b/fcf x:0 0 0 2024-05-13T22:48:35.734 INFO:tasks.workunit.client.1.smithi130.stdout:9/647: write d1/d7/d23/d28/d2d/d2f/d63/f9f [4592663,83352] 0 2024-05-13T22:48:35.736 INFO:tasks.workunit.client.1.smithi130.stdout:7/622: fdatasync d7/da/d1d/d5d/f89 0 2024-05-13T22:48:35.741 INFO:tasks.workunit.client.1.smithi130.stdout:3/601: link d8/d35/f3d d8/dc/d15/d8f/fd1 0 2024-05-13T22:48:35.745 INFO:tasks.workunit.client.1.smithi130.stdout:5/615: rename d6/dd/d18/d6e/d82/f7b to d6/d1a/d72/fcf 0 2024-05-13T22:48:35.746 INFO:tasks.workunit.client.1.smithi130.stdout:3/602: truncate d8/f5e 1007596 0 2024-05-13T22:48:35.747 INFO:tasks.workunit.client.1.smithi130.stdout:3/603: write d8/f5e [1712397,83223] 0 2024-05-13T22:48:35.752 INFO:tasks.workunit.client.1.smithi130.stdout:5/616: mknod d6/dd/d18/cd0 0 2024-05-13T22:48:35.756 INFO:tasks.workunit.client.1.smithi130.stdout:2/625: getdents d4/db/dc/d18/d2a/d30/d88/dc5 0 2024-05-13T22:48:35.759 INFO:tasks.workunit.client.1.smithi130.stdout:3/604: dwrite d8/f92 [0,4194304] 0 2024-05-13T22:48:35.759 INFO:tasks.workunit.client.1.smithi130.stdout:3/605: chown d8/dc/d19/d2c/f6b 16055326 1 2024-05-13T22:48:35.762 INFO:tasks.workunit.client.1.smithi130.stdout:7/623: creat d7/da/d1d/d5d/da6/fcd x:0 0 0 2024-05-13T22:48:35.762 INFO:tasks.workunit.client.1.smithi130.stdout:1/589: sync 2024-05-13T22:48:35.767 INFO:tasks.workunit.client.1.smithi130.stdout:2/626: fsync d4/f1e 0 2024-05-13T22:48:35.769 INFO:tasks.workunit.client.1.smithi130.stdout:9/648: link d1/d7/d23/d28/d2d/f48 d1/d7/d23/d28/d32/d65/fd0 0 2024-05-13T22:48:35.774 INFO:tasks.workunit.client.1.smithi130.stdout:7/624: mknod d7/da/d86/cce 0 2024-05-13T22:48:35.777 INFO:tasks.workunit.client.1.smithi130.stdout:1/590: symlink db/d1b/da3/d2a/d4c/lc0 0 2024-05-13T22:48:35.780 INFO:tasks.workunit.client.1.smithi130.stdout:7/625: dwrite d7/da/f62 [0,4194304] 0 2024-05-13T22:48:35.780 INFO:tasks.workunit.client.1.smithi130.stdout:3/606: link d8/dc/d15/f4d d8/dc/d7b/da3/daf/fd2 0 2024-05-13T22:48:35.781 INFO:tasks.workunit.client.1.smithi130.stdout:3/607: fdatasync d8/d1a/d1c/f49 0 2024-05-13T22:48:35.788 INFO:tasks.workunit.client.1.smithi130.stdout:1/591: creat db/d58/fc1 x:0 0 0 2024-05-13T22:48:35.790 INFO:tasks.workunit.client.1.smithi130.stdout:9/649: fdatasync d1/d7/d23/d28/d32/d45/f53 0 2024-05-13T22:48:35.793 INFO:tasks.workunit.client.1.smithi130.stdout:7/626: write d7/da/d1d/d5d/da6/d39/f7c [556222,32406] 0 2024-05-13T22:48:35.795 INFO:tasks.workunit.client.1.smithi130.stdout:5/617: getdents d6/d7 0 2024-05-13T22:48:35.796 INFO:tasks.workunit.client.1.smithi130.stdout:3/608: unlink d8/dc/d15/l5c 0 2024-05-13T22:48:35.798 INFO:tasks.workunit.client.1.smithi130.stdout:2/627: link d4/db/dc/d18/d58/f70 d4/db/dbf/d48/d4a/fdf 0 2024-05-13T22:48:35.799 INFO:tasks.workunit.client.1.smithi130.stdout:1/592: chown db/d1b/da3/d2a/d38/d46/d6d/laf 37 1 2024-05-13T22:48:35.799 INFO:tasks.workunit.client.1.smithi130.stdout:2/628: write d4/f5 [5535957,17999] 0 2024-05-13T22:48:35.804 INFO:tasks.workunit.client.1.smithi130.stdout:7/627: fsync d7/da/d1d/d5d/da6/d39/f94 0 2024-05-13T22:48:35.805 INFO:tasks.workunit.client.1.smithi130.stdout:2/629: dwrite d4/db/dc/f3b [0,4194304] 0 2024-05-13T22:48:35.806 INFO:tasks.workunit.client.1.smithi130.stdout:2/630: chown d4/db/dc/d18/d2a/d66/db9/lbd 1049027 1 2024-05-13T22:48:35.807 INFO:tasks.workunit.client.1.smithi130.stdout:3/609: chown d8/fa1 130 1 2024-05-13T22:48:35.812 INFO:tasks.workunit.client.1.smithi130.stdout:9/650: write d1/d7/d23/d28/d2d/d2f/d63/f68 [13476,78286] 0 2024-05-13T22:48:35.821 INFO:tasks.workunit.client.1.smithi130.stdout:3/610: dread d8/dc/d7b/da3/daf/fd2 [0,4194304] 0 2024-05-13T22:48:35.822 INFO:tasks.workunit.client.1.smithi130.stdout:2/631: mknod d4/db/d26/d9e/ce0 0 2024-05-13T22:48:35.827 INFO:tasks.workunit.client.1.smithi130.stdout:7/628: rename d7/da/d1d/l75 to d7/da/d1d/d5d/da6/d9/d4a/dbd/dc9/lcf 0 2024-05-13T22:48:35.827 INFO:tasks.workunit.client.1.smithi130.stdout:5/618: dwrite d6/d7/f46 [0,4194304] 0 2024-05-13T22:48:35.832 INFO:tasks.workunit.client.1.smithi130.stdout:2/632: mknod d4/d56/ce1 0 2024-05-13T22:48:35.843 INFO:tasks.workunit.client.1.smithi130.stdout:1/593: getdents db/d1b/da3/d49 0 2024-05-13T22:48:35.844 INFO:tasks.workunit.client.1.smithi130.stdout:9/651: creat d1/d7/d23/d3d/fd1 x:0 0 0 2024-05-13T22:48:35.844 INFO:tasks.workunit.client.1.smithi130.stdout:5/619: write d6/d7/f46 [4255620,49646] 0 2024-05-13T22:48:35.846 INFO:tasks.workunit.client.1.smithi130.stdout:3/611: rmdir d8/dc/d7b/dbb 0 2024-05-13T22:48:35.847 INFO:tasks.workunit.client.1.smithi130.stdout:3/612: fsync d8/d1a/d1c/f6e 0 2024-05-13T22:48:35.847 INFO:tasks.workunit.client.1.smithi130.stdout:2/633: mknod d4/db/ce2 0 2024-05-13T22:48:35.848 INFO:tasks.workunit.client.1.smithi130.stdout:1/594: mkdir db/d1b/da3/d2a/d4c/dc2 0 2024-05-13T22:48:35.849 INFO:tasks.workunit.client.1.smithi130.stdout:5/620: mknod d6/d1a/daf/cd1 0 2024-05-13T22:48:35.851 INFO:tasks.workunit.client.1.smithi130.stdout:7/629: creat d7/da/fd0 x:0 0 0 2024-05-13T22:48:35.852 INFO:tasks.workunit.client.1.smithi130.stdout:3/613: rename d8/dc/d42 to d8/dc/d7b/da3/dd3 0 2024-05-13T22:48:35.852 INFO:tasks.workunit.client.1.smithi130.stdout:7/630: stat d7/da/d1d/d5d/da6/f1c 0 2024-05-13T22:48:35.854 INFO:tasks.workunit.client.1.smithi130.stdout:1/595: dwrite db/d1b/da3/d2a/d38/fab [0,4194304] 0 2024-05-13T22:48:35.854 INFO:tasks.workunit.client.1.smithi130.stdout:9/652: mkdir d1/d7/dd2 0 2024-05-13T22:48:35.855 INFO:tasks.workunit.client.1.smithi130.stdout:9/653: stat d1/d15/f1d 0 2024-05-13T22:48:35.855 INFO:tasks.workunit.client.1.smithi130.stdout:1/596: chown db/d1b/da3/d49 50233 1 2024-05-13T22:48:35.860 INFO:tasks.workunit.client.1.smithi130.stdout:3/614: symlink d8/dc/d15/d8f/ld4 0 2024-05-13T22:48:35.861 INFO:tasks.workunit.client.1.smithi130.stdout:7/631: creat d7/da/d1d/db1/fd1 x:0 0 0 2024-05-13T22:48:35.861 INFO:tasks.workunit.client.1.smithi130.stdout:7/632: readlink d7/da/d1d/d5d/da6/l14 0 2024-05-13T22:48:35.862 INFO:tasks.workunit.client.1.smithi130.stdout:2/634: write d4/db/dbf/d42/d4b/f4f [619819,38348] 0 2024-05-13T22:48:35.865 INFO:tasks.workunit.client.1.smithi130.stdout:9/654: write d1/d7/d23/d28/d32/d65/fb9 [634385,123732] 0 2024-05-13T22:48:35.866 INFO:tasks.workunit.client.1.smithi130.stdout:9/655: chown d1/d15/d51 247741638 1 2024-05-13T22:48:35.868 INFO:tasks.workunit.client.1.smithi130.stdout:3/615: mknod d8/dc/d19/d2c/cd5 0 2024-05-13T22:48:35.870 INFO:tasks.workunit.client.1.smithi130.stdout:3/616: chown d8/dc/d7b/lc8 46911 1 2024-05-13T22:48:35.870 INFO:tasks.workunit.client.1.smithi130.stdout:0/660: sync 2024-05-13T22:48:35.870 INFO:tasks.workunit.client.1.smithi130.stdout:5/621: write d6/dd/d18/d6e/d82/d71/f81 [1135124,111079] 0 2024-05-13T22:48:35.871 INFO:tasks.workunit.client.1.smithi130.stdout:7/633: dwrite d7/da/d1d/d5d/da6/d9/d4a/dbd/d40/f77 [0,4194304] 0 2024-05-13T22:48:35.874 INFO:tasks.workunit.client.1.smithi130.stdout:5/622: write d6/d7/d19/f52 [999320,15984] 0 2024-05-13T22:48:35.875 INFO:tasks.workunit.client.1.smithi130.stdout:1/597: dread db/d1b/da3/d2a/f4f [0,4194304] 0 2024-05-13T22:48:35.876 INFO:tasks.workunit.client.1.smithi130.stdout:5/623: truncate d6/f1f 5186251 0 2024-05-13T22:48:35.878 INFO:tasks.workunit.client.1.smithi130.stdout:3/617: creat d8/dc/d7b/da3/dd3/fd6 x:0 0 0 2024-05-13T22:48:35.879 INFO:tasks.workunit.client.1.smithi130.stdout:0/661: mkdir d0/d7/dd/d17/d52/d5b/d60/de6 0 2024-05-13T22:48:35.881 INFO:tasks.workunit.client.1.smithi130.stdout:7/634: fsync d7/da/d1d/fcb 0 2024-05-13T22:48:35.889 INFO:tasks.workunit.client.1.smithi130.stdout:2/635: dwrite d4/db/dc/d18/d2a/d30/f9c [4194304,4194304] 0 2024-05-13T22:48:35.889 INFO:tasks.workunit.client.1.smithi130.stdout:3/618: readlink d8/d1a/d55/l57 0 2024-05-13T22:48:35.890 INFO:tasks.workunit.client.1.smithi130.stdout:5/624: dread d6/d1a/d72/fcf [0,4194304] 0 2024-05-13T22:48:35.895 INFO:tasks.workunit.client.1.smithi130.stdout:7/635: dwrite d7/da/d1d/d5d/da6/d5b/fb3 [0,4194304] 0 2024-05-13T22:48:35.895 INFO:tasks.workunit.client.1.smithi130.stdout:9/656: link d1/d7/d23/d28/d2d/f34 d1/d7/d23/d28/d32/d45/fd3 0 2024-05-13T22:48:35.896 INFO:tasks.workunit.client.1.smithi130.stdout:5/625: readlink d6/dd/d18/d6e/d82/d4b/dbc/lbf 0 2024-05-13T22:48:35.897 INFO:tasks.workunit.client.1.smithi130.stdout:5/626: readlink d6/d7/d19/d4d/d6a/lc2 0 2024-05-13T22:48:35.901 INFO:tasks.workunit.client.1.smithi130.stdout:3/619: dread d8/d1a/d1c/fa0 [0,4194304] 0 2024-05-13T22:48:35.901 INFO:tasks.workunit.client.1.smithi130.stdout:7/636: truncate d7/da/d1d/d5d/da6/d9/d66/f7a 1061847 0 2024-05-13T22:48:35.901 INFO:tasks.workunit.client.1.smithi130.stdout:2/636: rmdir d4/db 39 2024-05-13T22:48:35.901 INFO:tasks.workunit.client.1.smithi130.stdout:3/620: fdatasync d8/d1a/d1c/fc1 0 2024-05-13T22:48:35.902 INFO:tasks.workunit.client.1.smithi130.stdout:0/662: dread d0/d7/f16 [0,4194304] 0 2024-05-13T22:48:35.902 INFO:tasks.workunit.client.1.smithi130.stdout:7/637: fdatasync d7/da/d1d/d5d/da6/f78 0 2024-05-13T22:48:35.903 INFO:tasks.workunit.client.1.smithi130.stdout:0/663: chown d0/d7/dd/d2e/d4b 70030975 1 2024-05-13T22:48:35.904 INFO:tasks.workunit.client.1.smithi130.stdout:0/664: chown d0/d7/dd/d69/da9/d8d/f9d 2532233 1 2024-05-13T22:48:35.905 INFO:tasks.workunit.client.1.smithi130.stdout:0/665: readlink d0/d7/dd/d69/da9/d8d/d9f/la4 0 2024-05-13T22:48:35.906 INFO:tasks.workunit.client.1.smithi130.stdout:9/657: rmdir d1/d7/d23/d28/d73/db2 39 2024-05-13T22:48:35.907 INFO:tasks.workunit.client.1.smithi130.stdout:5/627: rename d6/dd/l97 to d6/dd/d18/d6e/dc7/ld2 0 2024-05-13T22:48:35.908 INFO:tasks.workunit.client.1.smithi130.stdout:5/628: chown d6/dd/d18/d6e/d82/d56/lcc 512516049 1 2024-05-13T22:48:35.910 INFO:tasks.workunit.client.1.smithi130.stdout:1/598: truncate db/d1b/da3/d2a/d35/f3d 748143 0 2024-05-13T22:48:35.913 INFO:tasks.workunit.client.1.smithi130.stdout:3/621: creat d8/d1a/d55/fd7 x:0 0 0 2024-05-13T22:48:35.914 INFO:tasks.workunit.client.1.smithi130.stdout:7/638: mknod d7/da/d1d/db1/dbe/cd2 0 2024-05-13T22:48:35.915 INFO:tasks.workunit.client.1.smithi130.stdout:7/639: fsync d7/da/d1d/d5d/da6/d9/d66/f7a 0 2024-05-13T22:48:35.916 INFO:tasks.workunit.client.1.smithi130.stdout:0/666: mkdir d0/d7/dd/d17/d1f/de7 0 2024-05-13T22:48:35.917 INFO:tasks.workunit.client.1.smithi130.stdout:8/572: sync 2024-05-13T22:48:35.917 INFO:tasks.workunit.client.1.smithi130.stdout:6/589: sync 2024-05-13T22:48:35.917 INFO:tasks.workunit.client.1.smithi130.stdout:0/667: write d0/d7/dd/f46 [4475573,11643] 0 2024-05-13T22:48:35.918 INFO:tasks.workunit.client.1.smithi130.stdout:6/590: stat d4/d15/d1d/d28/d4e/c9c 0 2024-05-13T22:48:35.918 INFO:tasks.workunit.client.1.smithi130.stdout:9/658: rename d1/d7/d23/d28/d2d/d2f/l5d to d1/d15/d51/d85/ld4 0 2024-05-13T22:48:35.919 INFO:tasks.workunit.client.1.smithi130.stdout:5/629: mknod d6/dd/d18/d6e/dc7/cd3 0 2024-05-13T22:48:35.919 INFO:tasks.workunit.client.1.smithi130.stdout:0/668: write d0/d7/dd/d17/d1f/d2c/d5a/f66 [2100236,91171] 0 2024-05-13T22:48:35.921 INFO:tasks.workunit.client.1.smithi130.stdout:5/630: dread - d6/dd/d18/d6e/d82/fbb zero size 2024-05-13T22:48:35.921 INFO:tasks.workunit.client.1.smithi130.stdout:2/637: rmdir d4/db/d26/d9e 39 2024-05-13T22:48:35.922 INFO:tasks.workunit.client.1.smithi130.stdout:0/669: truncate d0/fb7 1233556 0 2024-05-13T22:48:35.923 INFO:tasks.workunit.client.1.smithi130.stdout:2/638: fsync d4/db/dc/f15 0 2024-05-13T22:48:35.923 INFO:tasks.workunit.client.1.smithi130.stdout:3/622: creat d8/dc/d19/d2c/fd8 x:0 0 0 2024-05-13T22:48:35.925 INFO:tasks.workunit.client.1.smithi130.stdout:2/639: dread - d4/db/dc/d18/d2a/d66/fa9 zero size 2024-05-13T22:48:35.926 INFO:tasks.workunit.client.1.smithi130.stdout:6/591: dwrite d4/d15/d1d/d28/f30 [0,4194304] 0 2024-05-13T22:48:35.927 INFO:tasks.workunit.client.1.smithi130.stdout:7/640: mkdir d7/da/d1d/db1/dbe/dd3 0 2024-05-13T22:48:35.940 INFO:tasks.workunit.client.1.smithi130.stdout:1/599: mkdir db/d82/dc3 0 2024-05-13T22:48:35.941 INFO:tasks.workunit.client.1.smithi130.stdout:2/640: creat d4/db/dbf/d42/d4b/fe3 x:0 0 0 2024-05-13T22:48:35.942 INFO:tasks.workunit.client.1.smithi130.stdout:6/592: creat d4/d15/d1d/d60/da2/da4/dac/fc8 x:0 0 0 2024-05-13T22:48:35.942 INFO:tasks.workunit.client.1.smithi130.stdout:7/641: mkdir d7/da/d1d/db1/dd4 0 2024-05-13T22:48:35.942 INFO:tasks.workunit.client.1.smithi130.stdout:0/670: write d0/d7/dd/d2e/f4e [1818043,87268] 0 2024-05-13T22:48:35.944 INFO:tasks.workunit.client.1.smithi130.stdout:2/641: fdatasync d4/db/dc/d18/d2a/d30/fd8 0 2024-05-13T22:48:35.945 INFO:tasks.workunit.client.1.smithi130.stdout:3/623: creat d8/dc/d19/d83/dba/dc4/fd9 x:0 0 0 2024-05-13T22:48:35.945 INFO:tasks.workunit.client.1.smithi130.stdout:3/624: write d8/dc/d7b/fcf [451608,78151] 0 2024-05-13T22:48:36.008 INFO:tasks.workunit.client.1.smithi130.stdout:4/605: sync 2024-05-13T22:48:36.043 INFO:tasks.workunit.client.1.smithi130.stdout:5/631: sync 2024-05-13T22:48:36.044 INFO:tasks.workunit.client.1.smithi130.stdout:9/659: rename d1/d7/d23/d28/d73/d8d/f8f to d1/d15/d51/d85/d92/fd5 0 2024-05-13T22:48:36.051 INFO:tasks.workunit.client.1.smithi130.stdout:0/671: rmdir d0/d7/dd/d17/d1f/d2c 39 2024-05-13T22:48:36.064 INFO:tasks.workunit.client.1.smithi130.stdout:9/660: mknod d1/db/cd6 0 2024-05-13T22:48:36.068 INFO:tasks.workunit.client.1.smithi130.stdout:8/573: truncate d1/d2/db/d15/d7e/f95 2133752 0 2024-05-13T22:48:36.069 INFO:tasks.workunit.client.1.smithi130.stdout:8/574: write d1/d2/db/f46 [4227537,81179] 0 2024-05-13T22:48:36.069 INFO:tasks.workunit.client.1.smithi130.stdout:0/672: rmdir d0/d7/dd/d17/d52 39 2024-05-13T22:48:36.083 INFO:tasks.workunit.client.1.smithi130.stdout:4/606: symlink d8/dc/d80/da6/lbc 0 2024-05-13T22:48:36.084 INFO:tasks.workunit.client.1.smithi130.stdout:4/607: write d8/dc/d26/f30 [3011224,80966] 0 2024-05-13T22:48:36.090 INFO:tasks.workunit.client.1.smithi130.stdout:9/661: dread - d1/d7/d69/da2/fc4 zero size 2024-05-13T22:48:36.100 INFO:tasks.workunit.client.1.smithi130.stdout:4/608: rmdir d8/dc/d80/da6 39 2024-05-13T22:48:36.101 INFO:tasks.workunit.client.1.smithi130.stdout:4/609: chown d8/dc/d26/f30 1413999 1 2024-05-13T22:48:36.101 INFO:tasks.workunit.client.1.smithi130.stdout:4/610: write d8/dc/d29/f3a [8497507,5849] 0 2024-05-13T22:48:36.102 INFO:tasks.workunit.client.1.smithi130.stdout:9/662: creat d1/d15/d51/fd7 x:0 0 0 2024-05-13T22:48:36.105 INFO:tasks.workunit.client.1.smithi130.stdout:4/611: write d8/f1b [750783,55091] 0 2024-05-13T22:48:36.107 INFO:tasks.workunit.client.1.smithi130.stdout:4/612: read d8/dc/d35/f8d [758590,31178] 0 2024-05-13T22:48:36.107 INFO:tasks.workunit.client.1.smithi130.stdout:9/663: write d1/d15/d51/d85/d92/fd5 [770345,71018] 0 2024-05-13T22:48:36.108 INFO:tasks.workunit.client.1.smithi130.stdout:9/664: readlink d1/d7/d23/d61/l81 0 2024-05-13T22:48:36.110 INFO:tasks.workunit.client.1.smithi130.stdout:0/673: dread d0/d7/dd/d17/d52/d67/d68/d96/f83 [0,4194304] 0 2024-05-13T22:48:36.111 INFO:tasks.workunit.client.1.smithi130.stdout:0/674: chown d0/da0 0 1 2024-05-13T22:48:36.115 INFO:tasks.workunit.client.1.smithi130.stdout:9/665: read d1/d7/d23/d28/d2d/d2f/f6a [35233,78971] 0 2024-05-13T22:48:36.116 INFO:tasks.workunit.client.1.smithi130.stdout:4/613: mkdir d8/dc/d35/d33/d44/d61/da0/dbd 0 2024-05-13T22:48:36.118 INFO:tasks.workunit.client.1.smithi130.stdout:0/675: creat d0/d7/dd/d69/da9/d8d/d9f/fe8 x:0 0 0 2024-05-13T22:48:36.122 INFO:tasks.workunit.client.1.smithi130.stdout:9/666: dread d1/d7/d23/d3d/d43/f75 [0,4194304] 0 2024-05-13T22:48:36.123 INFO:tasks.workunit.client.1.smithi130.stdout:9/667: chown d1/d15/d51/d85 569 1 2024-05-13T22:48:36.124 INFO:tasks.workunit.client.1.smithi130.stdout:0/676: chown d0/d7/dd/d17/d1f/d2c/d45/d57/f85 4 1 2024-05-13T22:48:36.130 INFO:tasks.workunit.client.1.smithi130.stdout:0/677: dwrite d0/d7/dd/d17/fbb [0,4194304] 0 2024-05-13T22:48:36.133 INFO:tasks.workunit.client.1.smithi130.stdout:9/668: dwrite d1/d7/d23/d28/d32/d45/f9b [0,4194304] 0 2024-05-13T22:48:36.135 INFO:tasks.workunit.client.1.smithi130.stdout:0/678: truncate d0/d7/dd/d2e/d9e/da7/dc3/fdd 589301 0 2024-05-13T22:48:36.136 INFO:tasks.workunit.client.1.smithi130.stdout:9/669: write d1/d7/d23/f7a [4365518,14995] 0 2024-05-13T22:48:36.137 INFO:tasks.workunit.client.1.smithi130.stdout:0/679: stat d0/d7/dd/d17/d1f/d2c/d45/d57/f62 0 2024-05-13T22:48:36.139 INFO:tasks.workunit.client.1.smithi130.stdout:0/680: fdatasync d0/d7/dd/d2e/f8c 0 2024-05-13T22:48:36.143 INFO:tasks.workunit.client.1.smithi130.stdout:9/670: dwrite d1/d7/d23/d28/d32/fc1 [0,4194304] 0 2024-05-13T22:48:36.186 INFO:tasks.workunit.client.1.smithi130.stdout:2/642: write d4/db/dc/f4d [334830,53257] 0 2024-05-13T22:48:36.190 INFO:tasks.workunit.client.1.smithi130.stdout:7/642: dwrite d7/da/d1d/fcb [0,4194304] 0 2024-05-13T22:48:36.194 INFO:tasks.workunit.client.1.smithi130.stdout:2/643: mkdir d4/db/d26/d9e/de4 0 2024-05-13T22:48:36.194 INFO:tasks.workunit.client.1.smithi130.stdout:5/632: write d6/d1a/d72/fcf [724473,27447] 0 2024-05-13T22:48:36.195 INFO:tasks.workunit.client.1.smithi130.stdout:3/625: dwrite d8/d1a/d1c/f6c [0,4194304] 0 2024-05-13T22:48:36.201 INFO:tasks.workunit.client.1.smithi130.stdout:3/626: creat d8/dc/d15/fda x:0 0 0 2024-05-13T22:48:36.202 INFO:tasks.workunit.client.1.smithi130.stdout:2/644: fdatasync d4/db/dbf/d42/d4b/f94 0 2024-05-13T22:48:36.203 INFO:tasks.workunit.client.1.smithi130.stdout:3/627: fdatasync d8/d1a/d1c/fc1 0 2024-05-13T22:48:36.205 INFO:tasks.workunit.client.1.smithi130.stdout:5/633: dwrite d6/dd/d79/f9b [0,4194304] 0 2024-05-13T22:48:36.219 INFO:tasks.workunit.client.1.smithi130.stdout:5/634: dread f4 [0,4194304] 0 2024-05-13T22:48:36.220 INFO:tasks.workunit.client.1.smithi130.stdout:5/635: readlink d6/d1a/l67 0 2024-05-13T22:48:36.230 INFO:tasks.workunit.client.1.smithi130.stdout:8/575: dwrite d1/d2/db/fa6 [0,4194304] 0 2024-05-13T22:48:36.234 INFO:tasks.workunit.client.1.smithi130.stdout:8/576: mknod d1/dc/cd4 0 2024-05-13T22:48:36.241 INFO:tasks.workunit.client.1.smithi130.stdout:4/614: creat d8/dc/d35/d33/fbe x:0 0 0 2024-05-13T22:48:36.242 INFO:tasks.workunit.client.1.smithi130.stdout:5/636: dread d6/f5f [0,4194304] 0 2024-05-13T22:48:36.244 INFO:tasks.workunit.client.1.smithi130.stdout:5/637: read d6/d7/d19/d4d/f34 [979514,1107] 0 2024-05-13T22:48:36.245 INFO:tasks.workunit.client.1.smithi130.stdout:4/615: unlink d8/dc/d84/l88 0 2024-05-13T22:48:36.246 INFO:tasks.workunit.client.1.smithi130.stdout:6/593: symlink d4/d15/d1f/lc9 0 2024-05-13T22:48:36.249 INFO:tasks.workunit.client.1.smithi130.stdout:5/638: dwrite d6/d7/d32/f3d [0,4194304] 0 2024-05-13T22:48:36.252 INFO:tasks.workunit.client.1.smithi130.stdout:4/616: mkdir d8/dc/d35/d33/dbf 0 2024-05-13T22:48:36.255 INFO:tasks.workunit.client.1.smithi130.stdout:5/639: dwrite d6/dd/d18/d6e/f8b [0,4194304] 0 2024-05-13T22:48:36.257 INFO:tasks.workunit.client.1.smithi130.stdout:2/645: creat d4/db/dc/d18/d2a/d30/fe5 x:0 0 0 2024-05-13T22:48:36.261 INFO:tasks.workunit.client.1.smithi130.stdout:0/681: mkdir d0/d7/dd/d17/d1f/d2c/d45/de9 0 2024-05-13T22:48:36.263 INFO:tasks.workunit.client.1.smithi130.stdout:8/577: mknod d1/d2/db/d15/d18/d42/cd5 0 2024-05-13T22:48:36.263 INFO:tasks.workunit.client.1.smithi130.stdout:9/671: write d1/f11 [4807686,17165] 0 2024-05-13T22:48:36.264 INFO:tasks.workunit.client.1.smithi130.stdout:2/646: dwrite d4/db/d26/f2d [0,4194304] 0 2024-05-13T22:48:36.266 INFO:tasks.workunit.client.1.smithi130.stdout:9/672: chown d1/d15/f86 8 1 2024-05-13T22:48:36.267 INFO:tasks.workunit.client.1.smithi130.stdout:2/647: read - d4/db/dc/d18/d2a/d66/f7f zero size 2024-05-13T22:48:36.268 INFO:tasks.workunit.client.1.smithi130.stdout:2/648: readlink d4/db/l54 0 2024-05-13T22:48:36.270 INFO:tasks.workunit.client.1.smithi130.stdout:8/578: creat d1/d2/db/d15/d18/d42/d53/fd6 x:0 0 0 2024-05-13T22:48:36.276 INFO:tasks.workunit.client.1.smithi130.stdout:9/673: dwrite d1/d15/d51/f6d [0,4194304] 0 2024-05-13T22:48:36.277 INFO:tasks.workunit.client.1.smithi130.stdout:2/649: dwrite d4/db/dc/f20 [8388608,4194304] 0 2024-05-13T22:48:36.278 INFO:tasks.workunit.client.1.smithi130.stdout:9/674: write d1/d15/f86 [505421,51921] 0 2024-05-13T22:48:36.280 INFO:tasks.workunit.client.1.smithi130.stdout:0/682: getdents d0/d7/dd/d17/d52/d5b/d60 0 2024-05-13T22:48:36.280 INFO:tasks.workunit.client.1.smithi130.stdout:2/650: chown d4/db/dbf/d42/d4b/f83 0 1 2024-05-13T22:48:36.282 INFO:tasks.workunit.client.1.smithi130.stdout:1/600: rename db/d1b/da3/d2a/d38/f60 to db/d1b/da3/d2a/d35/fc4 0 2024-05-13T22:48:36.286 INFO:tasks.workunit.client.1.smithi130.stdout:8/579: dread d1/d2/db/d15/d7e/d9f/fa0 [0,4194304] 0 2024-05-13T22:48:36.287 INFO:tasks.workunit.client.1.smithi130.stdout:0/683: mknod d0/d7/dd/d2e/da8/cea 0 2024-05-13T22:48:36.289 INFO:tasks.workunit.client.1.smithi130.stdout:2/651: truncate d4/db/f22 3904334 0 2024-05-13T22:48:36.290 INFO:tasks.workunit.client.1.smithi130.stdout:2/652: readlink d4/l57 0 2024-05-13T22:48:36.290 INFO:tasks.workunit.client.1.smithi130.stdout:6/594: rename d4/d15/d1d/l6b to d4/d15/d1f/d3c/lca 0 2024-05-13T22:48:36.292 INFO:tasks.workunit.client.1.smithi130.stdout:8/580: symlink d1/d2/db/d15/d7e/d9f/ld7 0 2024-05-13T22:48:36.293 INFO:tasks.workunit.client.1.smithi130.stdout:8/581: stat d1/d2/db/d15/d18/dba/fca 0 2024-05-13T22:48:36.295 INFO:tasks.workunit.client.1.smithi130.stdout:6/595: stat d4/d15/d1f/d46/d50/c6e 0 2024-05-13T22:48:36.295 INFO:tasks.workunit.client.1.smithi130.stdout:0/684: getdents d0/d7/dd/d2e/d9e/da7/de1 0 2024-05-13T22:48:36.295 INFO:tasks.workunit.client.1.smithi130.stdout:2/653: read d4/db/dbf/d42/f53 [3041178,21785] 0 2024-05-13T22:48:36.296 INFO:tasks.workunit.client.1.smithi130.stdout:2/654: chown d4/db/dc/d18/d2a/d30/d88/d99 5562 1 2024-05-13T22:48:36.297 INFO:tasks.workunit.client.1.smithi130.stdout:4/617: rename d8/dc/d80/da6 to d8/dc/d35/d33/d3f/d57/dc0 0 2024-05-13T22:48:36.297 INFO:tasks.workunit.client.1.smithi130.stdout:7/643: write d7/da/d1d/d5d/da6/d9/d4a/fba [2502161,25223] 0 2024-05-13T22:48:36.298 INFO:tasks.workunit.client.1.smithi130.stdout:4/618: readlink d8/dc/d35/d33/d44/d61/da0/lb2 0 2024-05-13T22:48:36.299 INFO:tasks.workunit.client.1.smithi130.stdout:8/582: creat d1/d2/db/d15/d16/d54/d58/fd8 x:0 0 0 2024-05-13T22:48:36.300 INFO:tasks.workunit.client.1.smithi130.stdout:4/619: dread - d8/dc/d35/d33/fbe zero size 2024-05-13T22:48:36.301 INFO:tasks.workunit.client.1.smithi130.stdout:6/596: symlink d4/d15/d1d/d28/lcb 0 2024-05-13T22:48:36.302 INFO:tasks.workunit.client.1.smithi130.stdout:7/644: mknod d7/da/d1d/d5d/cd5 0 2024-05-13T22:48:36.304 INFO:tasks.workunit.client.1.smithi130.stdout:3/628: write d8/dc/d15/f4d [789629,88369] 0 2024-05-13T22:48:36.305 INFO:tasks.workunit.client.1.smithi130.stdout:8/583: creat d1/d2/db/d15/d18/d42/d53/fd9 x:0 0 0 2024-05-13T22:48:36.307 INFO:tasks.workunit.client.1.smithi130.stdout:2/655: symlink d4/db/dc/le6 0 2024-05-13T22:48:36.310 INFO:tasks.workunit.client.1.smithi130.stdout:8/584: read d1/d2/db/d15/d18/f3e [2698865,23182] 0 2024-05-13T22:48:36.310 INFO:tasks.workunit.client.1.smithi130.stdout:6/597: dwrite d4/f73 [0,4194304] 0 2024-05-13T22:48:36.310 INFO:tasks.workunit.client.1.smithi130.stdout:9/675: dread d1/fc9 [0,4194304] 0 2024-05-13T22:48:36.315 INFO:tasks.workunit.client.1.smithi130.stdout:3/629: fdatasync d8/dc/d19/d2c/f87 0 2024-05-13T22:48:36.316 INFO:tasks.workunit.client.1.smithi130.stdout:2/656: dwrite d4/db/dbf/d42/d4b/f83 [0,4194304] 0 2024-05-13T22:48:36.318 INFO:tasks.workunit.client.1.smithi130.stdout:2/657: truncate d4/db/d26/f8f 594634 0 2024-05-13T22:48:36.320 INFO:tasks.workunit.client.1.smithi130.stdout:1/601: rename db/d1b/da3/d2a/d35/f69 to db/d1b/da3/d2a/d38/fc5 0 2024-05-13T22:48:36.321 INFO:tasks.workunit.client.1.smithi130.stdout:1/602: write db/d1b/da3/d72/daa/fb3 [371035,128287] 0 2024-05-13T22:48:36.322 INFO:tasks.workunit.client.1.smithi130.stdout:1/603: stat db/d1b/da3/d2a/l97 0 2024-05-13T22:48:36.326 INFO:tasks.workunit.client.1.smithi130.stdout:7/645: mknod d7/da/d1d/db1/dd4/cd6 0 2024-05-13T22:48:36.326 INFO:tasks.workunit.client.1.smithi130.stdout:7/646: readlink d7/da/d1d/d5d/l5f 0 2024-05-13T22:48:36.331 INFO:tasks.workunit.client.1.smithi130.stdout:6/598: creat d4/d15/d1d/d60/fcc x:0 0 0 2024-05-13T22:48:36.331 INFO:tasks.workunit.client.1.smithi130.stdout:6/599: dread - d4/d15/d1f/d46/d52/fc3 zero size 2024-05-13T22:48:36.338 INFO:tasks.workunit.client.1.smithi130.stdout:5/640: dwrite d6/d7/f3b [4194304,4194304] 0 2024-05-13T22:48:36.341 INFO:tasks.workunit.client.1.smithi130.stdout:9/676: dread d1/db/fc [0,4194304] 0 2024-05-13T22:48:36.349 INFO:tasks.workunit.client.1.smithi130.stdout:1/604: unlink db/d1b/da3/d49/f55 0 2024-05-13T22:48:36.352 INFO:tasks.workunit.client.1.smithi130.stdout:7/647: dread d7/da/d1d/d42/d43/f4d [0,4194304] 0 2024-05-13T22:48:36.354 INFO:tasks.workunit.client.1.smithi130.stdout:8/585: mkdir d1/d2/db/d15/d7e/dc2/dcf/dda 0 2024-05-13T22:48:36.355 INFO:tasks.workunit.client.1.smithi130.stdout:8/586: write d1/d2/db/d15/d18/f4b [658547,53372] 0 2024-05-13T22:48:36.359 INFO:tasks.workunit.client.1.smithi130.stdout:3/630: mkdir d8/d35/ddb 0 2024-05-13T22:48:36.359 INFO:tasks.workunit.client.1.smithi130.stdout:3/631: readlink d8/dc/d19/d2c/l32 0 2024-05-13T22:48:36.359 INFO:tasks.workunit.client.1.smithi130.stdout:7/648: dread d7/da/d1d/d5d/d70/f8d [0,4194304] 0 2024-05-13T22:48:36.363 INFO:tasks.workunit.client.1.smithi130.stdout:5/641: creat d6/dd/d18/d6e/d82/d56/fd4 x:0 0 0 2024-05-13T22:48:36.367 INFO:tasks.workunit.client.1.smithi130.stdout:0/685: dwrite d0/d7/dd/d2e/f3f [0,4194304] 0 2024-05-13T22:48:36.369 INFO:tasks.workunit.client.1.smithi130.stdout:8/587: dread d1/d2/db/d15/d18/d42/f6d [0,4194304] 0 2024-05-13T22:48:36.369 INFO:tasks.workunit.client.1.smithi130.stdout:0/686: chown d0/d7/dd/d17/d52/d5b/d5f/ld6 57 1 2024-05-13T22:48:36.371 INFO:tasks.workunit.client.1.smithi130.stdout:4/620: write d8/dc/d35/f50 [733317,30350] 0 2024-05-13T22:48:36.376 INFO:tasks.workunit.client.1.smithi130.stdout:4/621: dwrite d8/dc/d35/d33/d44/d61/f98 [0,4194304] 0 2024-05-13T22:48:36.381 INFO:tasks.workunit.client.1.smithi130.stdout:4/622: dread d8/dc/d35/d33/d44/d61/f98 [0,4194304] 0 2024-05-13T22:48:36.383 INFO:tasks.workunit.client.1.smithi130.stdout:3/632: rmdir d8/dc/d7b/da3/daf 39 2024-05-13T22:48:36.383 INFO:tasks.workunit.client.1.smithi130.stdout:4/623: write d8/dc/d35/d33/d44/f54 [1512118,82492] 0 2024-05-13T22:48:36.395 INFO:tasks.workunit.client.1.smithi130.stdout:9/677: link d1/d7/d23/f7a d1/d7/d23/d28/d73/d8d/fd8 0 2024-05-13T22:48:36.403 INFO:tasks.workunit.client.1.smithi130.stdout:2/658: write d4/db/dc/f37 [5045543,83043] 0 2024-05-13T22:48:36.406 INFO:tasks.workunit.client.1.smithi130.stdout:8/588: creat d1/d2/db/d15/d16/d54/d58/fdb x:0 0 0 2024-05-13T22:48:36.417 INFO:tasks.workunit.client.1.smithi130.stdout:7/649: mknod d7/cd7 0 2024-05-13T22:48:36.418 INFO:tasks.workunit.client.1.smithi130.stdout:1/605: write db/f17 [1912728,112583] 0 2024-05-13T22:48:36.419 INFO:tasks.workunit.client.1.smithi130.stdout:1/606: chown db/d1b/da3/d2a/d38/d46/f5f 521 1 2024-05-13T22:48:36.420 INFO:tasks.workunit.client.1.smithi130.stdout:1/607: write db/d1b/da3/d2a/d38/fa8 [2187727,108396] 0 2024-05-13T22:48:36.421 INFO:tasks.workunit.client.1.smithi130.stdout:1/608: stat db/d1b/da3/d2a/d4c/c5d 0 2024-05-13T22:48:36.425 INFO:tasks.workunit.client.1.smithi130.stdout:9/678: write d1/d7/d23/d3d/fc6 [1129594,7484] 0 2024-05-13T22:48:36.426 INFO:tasks.workunit.client.1.smithi130.stdout:3/633: dwrite d8/d1a/d1c/f94 [0,4194304] 0 2024-05-13T22:48:36.427 INFO:tasks.workunit.client.1.smithi130.stdout:0/687: mknod d0/d7/ceb 0 2024-05-13T22:48:36.430 INFO:tasks.workunit.client.1.smithi130.stdout:9/679: write d1/d7/d23/d28/d2d/d2f/d63/d6b/fcf [95326,93063] 0 2024-05-13T22:48:36.433 INFO:tasks.workunit.client.1.smithi130.stdout:5/642: dwrite d6/d7/fa [0,4194304] 0 2024-05-13T22:48:36.434 INFO:tasks.workunit.client.1.smithi130.stdout:9/680: chown d1/d7/d23/fc2 12060 1 2024-05-13T22:48:36.435 INFO:tasks.workunit.client.1.smithi130.stdout:9/681: stat d1/d7/d23/d28/d2d/d2f/l60 0 2024-05-13T22:48:36.441 INFO:tasks.workunit.client.1.smithi130.stdout:7/650: mknod d7/da/d1d/d5d/da6/d9/d66/cd8 0 2024-05-13T22:48:36.446 INFO:tasks.workunit.client.1.smithi130.stdout:3/634: symlink d8/d1a/d55/ldc 0 2024-05-13T22:48:36.447 INFO:tasks.workunit.client.1.smithi130.stdout:3/635: chown d8/dc/c68 152 1 2024-05-13T22:48:36.448 INFO:tasks.workunit.client.1.smithi130.stdout:6/600: link d4/d15/d1f/d64/f81 d4/d15/d1d/fcd 0 2024-05-13T22:48:36.450 INFO:tasks.workunit.client.1.smithi130.stdout:6/601: write d4/d15/d1d/d28/d75/da8/da0/fae [1597586,112210] 0 2024-05-13T22:48:36.458 INFO:tasks.workunit.client.1.smithi130.stdout:9/682: creat d1/db/fd9 x:0 0 0 2024-05-13T22:48:36.484 INFO:tasks.workunit.client.1.smithi130.stdout:2/659: dwrite d4/db/dc/f79 [0,4194304] 0 2024-05-13T22:48:36.485 INFO:tasks.workunit.client.1.smithi130.stdout:2/660: stat d4/db/dc/d18/d2a/d30/d88/d90/fdb 0 2024-05-13T22:48:36.492 INFO:tasks.workunit.client.1.smithi130.stdout:7/651: mknod d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/cd9 0 2024-05-13T22:48:36.493 INFO:tasks.workunit.client.1.smithi130.stdout:1/609: write db/d9c/f8a [5116926,5662] 0 2024-05-13T22:48:36.494 INFO:tasks.workunit.client.1.smithi130.stdout:4/624: creat d8/dc/d35/fc1 x:0 0 0 2024-05-13T22:48:36.495 INFO:tasks.workunit.client.1.smithi130.stdout:4/625: write d8/dc/d35/d33/fbe [760921,58383] 0 2024-05-13T22:48:36.496 INFO:tasks.workunit.client.1.smithi130.stdout:4/626: write d8/f7b [1500700,84656] 0 2024-05-13T22:48:36.500 INFO:tasks.workunit.client.1.smithi130.stdout:6/602: write d4/d15/d1f/d46/d50/f67 [2578670,65226] 0 2024-05-13T22:48:36.504 INFO:tasks.workunit.client.1.smithi130.stdout:8/589: rename d1/f65 to d1/dc/d5d/fdc 0 2024-05-13T22:48:36.504 INFO:tasks.workunit.client.1.smithi130.stdout:6/603: dwrite d4/d15/d1f/d46/d50/f67 [0,4194304] 0 2024-05-13T22:48:36.505 INFO:tasks.workunit.client.1.smithi130.stdout:8/590: fsync d1/d2/db/d15/f22 0 2024-05-13T22:48:36.506 INFO:tasks.workunit.client.1.smithi130.stdout:9/683: mknod d1/d7/d23/d28/d2d/d2f/d63/d6b/cda 0 2024-05-13T22:48:36.507 INFO:tasks.workunit.client.1.smithi130.stdout:8/591: write d1/d2/db/d15/d18/fb0 [175543,30361] 0 2024-05-13T22:48:36.518 INFO:tasks.workunit.client.1.smithi130.stdout:1/610: symlink db/d1b/da3/d72/lc6 0 2024-05-13T22:48:36.523 INFO:tasks.workunit.client.1.smithi130.stdout:4/627: creat d8/dc/d26/fc2 x:0 0 0 2024-05-13T22:48:36.524 INFO:tasks.workunit.client.1.smithi130.stdout:0/688: dwrite d0/d7/dd/d2e/f8c [0,4194304] 0 2024-05-13T22:48:36.526 INFO:tasks.workunit.client.1.smithi130.stdout:0/689: truncate d0/d7/fcc 55594 0 2024-05-13T22:48:36.527 INFO:tasks.workunit.client.1.smithi130.stdout:5/643: creat d6/d1a/fd5 x:0 0 0 2024-05-13T22:48:36.527 INFO:tasks.workunit.client.1.smithi130.stdout:0/690: chown d0/d7/dd/d2e/d9e/da7/fde 696 1 2024-05-13T22:48:36.528 INFO:tasks.workunit.client.1.smithi130.stdout:0/691: fsync d0/d7/dd/d17/d1f/d2c/fd3 0 2024-05-13T22:48:36.529 INFO:tasks.workunit.client.1.smithi130.stdout:3/636: rename d8/dc/d15/d8f to d8/dc/d7b/da3/dd3/ddd 0 2024-05-13T22:48:36.534 INFO:tasks.workunit.client.1.smithi130.stdout:6/604: unlink d4/l5f 0 2024-05-13T22:48:36.537 INFO:tasks.workunit.client.1.smithi130.stdout:8/592: symlink d1/d2/db/d15/d16/ldd 0 2024-05-13T22:48:36.539 INFO:tasks.workunit.client.1.smithi130.stdout:2/661: fsync d4/da8/dc9/fb4 0 2024-05-13T22:48:36.544 INFO:tasks.workunit.client.1.smithi130.stdout:7/652: symlink d7/da/d1d/d5d/da6/d9/lda 0 2024-05-13T22:48:36.545 INFO:tasks.workunit.client.1.smithi130.stdout:1/611: creat db/d1b/da3/d49/d56/d88/fc7 x:0 0 0 2024-05-13T22:48:36.546 INFO:tasks.workunit.client.1.smithi130.stdout:7/653: write d7/da/d1d/f9a [555852,80738] 0 2024-05-13T22:48:36.551 INFO:tasks.workunit.client.1.smithi130.stdout:5/644: mknod d6/dd/d18/d6e/d82/d84/cd6 0 2024-05-13T22:48:36.552 INFO:tasks.workunit.client.1.smithi130.stdout:1/612: dread db/d9c/f8a [4194304,4194304] 0 2024-05-13T22:48:36.555 INFO:tasks.workunit.client.1.smithi130.stdout:7/654: dwrite d7/da/d1d/d5d/da6/d9/d4a/fba [0,4194304] 0 2024-05-13T22:48:36.555 INFO:tasks.workunit.client.1.smithi130.stdout:0/692: rmdir d0/d7/dd/d17/d52/d5b 39 2024-05-13T22:48:36.559 INFO:tasks.workunit.client.1.smithi130.stdout:3/637: fdatasync d8/dc/d19/d27/f82 0 2024-05-13T22:48:36.560 INFO:tasks.workunit.client.1.smithi130.stdout:2/662: dread d4/db/dbf/d48/d4a/f52 [0,4194304] 0 2024-05-13T22:48:36.561 INFO:tasks.workunit.client.1.smithi130.stdout:3/638: dread - d8/d1a/d1c/fc1 zero size 2024-05-13T22:48:36.561 INFO:tasks.workunit.client.1.smithi130.stdout:2/663: chown d4/da8/dc9 0 1 2024-05-13T22:48:36.562 INFO:tasks.workunit.client.1.smithi130.stdout:3/639: write d8/dc/d19/d2c/f87 [606556,11994] 0 2024-05-13T22:48:36.564 INFO:tasks.workunit.client.1.smithi130.stdout:9/684: mkdir d1/d7/d23/d28/d32/d45/ddb 0 2024-05-13T22:48:36.566 INFO:tasks.workunit.client.1.smithi130.stdout:8/593: mkdir d1/dc/d5d/dde 0 2024-05-13T22:48:36.586 INFO:tasks.workunit.client.1.smithi130.stdout:4/628: mknod d8/da8/cc3 0 2024-05-13T22:48:36.591 INFO:tasks.workunit.client.1.smithi130.stdout:5/645: chown d6/c7e 151038948 1 2024-05-13T22:48:36.593 INFO:tasks.workunit.client.1.smithi130.stdout:4/629: dwrite d8/dc/d35/d33/d44/d56/fae [0,4194304] 0 2024-05-13T22:48:36.596 INFO:tasks.workunit.client.1.smithi130.stdout:4/630: chown d8/dc/d35/d33/d44/d77/d94/db6/dad 36736734 1 2024-05-13T22:48:36.597 INFO:tasks.workunit.client.1.smithi130.stdout:4/631: chown d8/dc/d35/d33/d44/d61/f98 2599 1 2024-05-13T22:48:36.598 INFO:tasks.workunit.client.1.smithi130.stdout:7/655: symlink d7/da/d1d/d42/d43/ldb 0 2024-05-13T22:48:36.600 INFO:tasks.workunit.client.1.smithi130.stdout:7/656: write d7/da/d1d/d5d/d70/f8d [243683,19241] 0 2024-05-13T22:48:36.601 INFO:tasks.workunit.client.1.smithi130.stdout:0/693: stat d0/d7/dd/d17/d52/d5b/d60/de6 0 2024-05-13T22:48:36.609 INFO:tasks.workunit.client.1.smithi130.stdout:3/640: dread d8/dc/f3e [0,4194304] 0 2024-05-13T22:48:36.613 INFO:tasks.workunit.client.1.smithi130.stdout:5/646: dread d6/dd/f3e [0,4194304] 0 2024-05-13T22:48:36.627 INFO:tasks.workunit.client.1.smithi130.stdout:3/641: dread d8/dc/d19/d27/f97 [0,4194304] 0 2024-05-13T22:48:36.630 INFO:tasks.workunit.client.1.smithi130.stdout:4/632: mknod d8/dc/d35/d33/d44/d56/cc4 0 2024-05-13T22:48:36.632 INFO:tasks.workunit.client.1.smithi130.stdout:3/642: dwrite d8/dc/d7b/da3/dd3/fd6 [0,4194304] 0 2024-05-13T22:48:36.634 INFO:tasks.workunit.client.1.smithi130.stdout:7/657: creat d7/da/d1d/d5d/da6/d7e/fdc x:0 0 0 2024-05-13T22:48:36.636 INFO:tasks.workunit.client.1.smithi130.stdout:7/658: write d7/da/d1d/d3d/d46/d92/fad [4280565,64778] 0 2024-05-13T22:48:36.636 INFO:tasks.workunit.client.1.smithi130.stdout:0/694: rmdir d0/d7/d29 39 2024-05-13T22:48:36.638 INFO:tasks.workunit.client.1.smithi130.stdout:3/643: dwrite d8/f92 [0,4194304] 0 2024-05-13T22:48:36.638 INFO:tasks.workunit.client.1.smithi130.stdout:0/695: chown d0/d7/dd/d17/d1f/cd4 3 1 2024-05-13T22:48:36.639 INFO:tasks.workunit.client.1.smithi130.stdout:6/605: rename d4/d15/d1f/d3c/c88 to d4/d15/cce 0 2024-05-13T22:48:36.640 INFO:tasks.workunit.client.1.smithi130.stdout:6/606: dread - d4/d15/d1f/d46/fb2 zero size 2024-05-13T22:48:36.641 INFO:tasks.workunit.client.1.smithi130.stdout:6/607: write d4/d15/d1d/d28/f30 [280423,106223] 0 2024-05-13T22:48:36.643 INFO:tasks.workunit.client.1.smithi130.stdout:2/664: mknod d4/db/d26/d9e/de4/ce7 0 2024-05-13T22:48:36.654 INFO:tasks.workunit.client.1.smithi130.stdout:8/594: unlink d1/d2/f25 0 2024-05-13T22:48:36.655 INFO:tasks.workunit.client.1.smithi130.stdout:5/647: creat d6/dd/d18/fd7 x:0 0 0 2024-05-13T22:48:36.658 INFO:tasks.workunit.client.1.smithi130.stdout:1/613: link db/l29 db/d58/lc8 0 2024-05-13T22:48:36.658 INFO:tasks.workunit.client.1.smithi130.stdout:5/648: dread d6/dd/d18/d6e/d76/d7f/f85 [0,4194304] 0 2024-05-13T22:48:36.660 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:36 smithi102 ceph-mon[28599]: pgmap v35: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 61 MiB/s rd, 201 MiB/s wr, 380 op/s 2024-05-13T22:48:36.660 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:36 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:36.660 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:36 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:36.661 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:36 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:48:36.670 INFO:tasks.workunit.client.1.smithi130.stdout:7/659: fdatasync d7/da/d1d/d3d/d46/f4f 0 2024-05-13T22:48:36.671 INFO:tasks.workunit.client.1.smithi130.stdout:7/660: write d7/da/d1d/d5d/da6/d7e/fdc [1007785,113360] 0 2024-05-13T22:48:36.672 INFO:tasks.workunit.client.1.smithi130.stdout:7/661: chown d7/da/d1d/d5d/da6/l14 3552796 1 2024-05-13T22:48:36.674 INFO:tasks.workunit.client.1.smithi130.stdout:0/696: write d0/d7/fcc [346943,38497] 0 2024-05-13T22:48:36.675 INFO:tasks.workunit.client.1.smithi130.stdout:3/644: mknod d8/d1a/d55/cde 0 2024-05-13T22:48:36.675 INFO:tasks.workunit.client.1.smithi130.stdout:0/697: dread - d0/d7/dd/d2e/d9e/da7/fd8 zero size 2024-05-13T22:48:36.678 INFO:tasks.workunit.client.1.smithi130.stdout:0/698: dread d0/d7/f16 [0,4194304] 0 2024-05-13T22:48:36.682 INFO:tasks.workunit.client.1.smithi130.stdout:9/685: write d1/d15/d51/f54 [1823543,9233] 0 2024-05-13T22:48:36.683 INFO:tasks.workunit.client.1.smithi130.stdout:9/686: write d1/f11 [3793268,101083] 0 2024-05-13T22:48:36.692 INFO:tasks.workunit.client.1.smithi130.stdout:5/649: creat d6/d7/d19/fd8 x:0 0 0 2024-05-13T22:48:36.692 INFO:tasks.workunit.client.1.smithi130.stdout:4/633: write d8/dc/d84/f9c [9192918,22609] 0 2024-05-13T22:48:36.695 INFO:tasks.workunit.client.1.smithi130.stdout:1/614: fdatasync db/d1b/da3/d49/f61 0 2024-05-13T22:48:36.700 INFO:tasks.workunit.client.1.smithi130.stdout:7/662: fsync d7/da/d1d/d5d/da6/d39/f79 0 2024-05-13T22:48:36.700 INFO:tasks.workunit.client.1.smithi130.stdout:7/663: readlink d7/da/d1d/d42/lbb 0 2024-05-13T22:48:36.707 INFO:tasks.workunit.client.1.smithi130.stdout:3/645: dwrite d8/d1a/d1c/f5d [0,4194304] 0 2024-05-13T22:48:36.714 INFO:tasks.workunit.client.1.smithi130.stdout:5/650: write d6/dd/f37 [626556,122001] 0 2024-05-13T22:48:36.715 INFO:tasks.workunit.client.1.smithi130.stdout:4/634: stat d8/dc/d35/d33/f4f 0 2024-05-13T22:48:36.721 INFO:tasks.workunit.client.1.smithi130.stdout:7/664: rmdir d7/da/d1d/d5d/da6 39 2024-05-13T22:48:36.723 INFO:tasks.workunit.client.1.smithi130.stdout:0/699: symlink d0/d7/lec 0 2024-05-13T22:48:36.725 INFO:tasks.workunit.client.1.smithi130.stdout:0/700: truncate d0/d7/dd/d69/fc8 254165 0 2024-05-13T22:48:36.725 INFO:tasks.workunit.client.1.smithi130.stdout:0/701: fdatasync d0/d7/dd/d2e/f4e 0 2024-05-13T22:48:36.727 INFO:tasks.workunit.client.1.smithi130.stdout:6/608: creat d4/d15/d1d/d28/d75/fcf x:0 0 0 2024-05-13T22:48:36.729 INFO:tasks.workunit.client.1.smithi130.stdout:2/665: creat d4/db/dbf/fe8 x:0 0 0 2024-05-13T22:48:36.732 INFO:tasks.workunit.client.1.smithi130.stdout:9/687: symlink d1/d7/d23/d28/d2d/ldc 0 2024-05-13T22:48:36.733 INFO:tasks.workunit.client.1.smithi130.stdout:8/595: link d1/d2/db/d15/d18/d1a/l6a d1/d2/db/d15/d18/dba/ldf 0 2024-05-13T22:48:36.735 INFO:tasks.workunit.client.1.smithi130.stdout:8/596: write d1/d2/db/d15/d16/d54/fb8 [942238,65095] 0 2024-05-13T22:48:36.740 INFO:tasks.workunit.client.1.smithi130.stdout:9/688: dwrite d1/d7/d23/d28/d2d/d2f/d63/f77 [0,4194304] 0 2024-05-13T22:48:36.741 INFO:tasks.workunit.client.1.smithi130.stdout:5/651: dread d6/d2f/f6c [0,4194304] 0 2024-05-13T22:48:36.742 INFO:tasks.workunit.client.1.smithi130.stdout:5/652: chown d6/dd/fb5 954342 1 2024-05-13T22:48:36.742 INFO:tasks.workunit.client.1.smithi130.stdout:5/653: readlink d6/d2f/lbe 0 2024-05-13T22:48:36.745 INFO:tasks.workunit.client.1.smithi130.stdout:3/646: truncate d8/d1a/d1c/f6e 2009053 0 2024-05-13T22:48:36.748 INFO:tasks.workunit.client.1.smithi130.stdout:4/635: rename d8/dc/f86 to d8/dc/d35/d33/d44/d56/fc5 0 2024-05-13T22:48:36.752 INFO:tasks.workunit.client.1.smithi130.stdout:1/615: link db/d1b/da3/d2a/d38/d46/d6d/lb4 db/d1b/da3/d49/d56/lc9 0 2024-05-13T22:48:36.753 INFO:tasks.workunit.client.1.smithi130.stdout:1/616: write db/d1b/da3/d49/d56/d88/fc7 [780123,7202] 0 2024-05-13T22:48:36.759 INFO:tasks.workunit.client.1.smithi130.stdout:8/597: rename d1/d2/db/d15/d16/d54/d58/dab/cb7 to d1/d2/db/d15/d16/d54/d58/dab/ce0 0 2024-05-13T22:48:36.759 INFO:tasks.workunit.client.1.smithi130.stdout:8/598: readlink d1/d2/db/d15/d16/d54/l68 0 2024-05-13T22:48:36.769 INFO:tasks.workunit.client.1.smithi130.stdout:5/654: rename d6/f5f to d6/d1a/d72/fd9 0 2024-05-13T22:48:36.770 INFO:tasks.workunit.client.1.smithi130.stdout:4/636: mknod d8/dc/d35/d33/d44/d77/cc6 0 2024-05-13T22:48:36.773 INFO:tasks.workunit.client.1.smithi130.stdout:7/665: symlink d7/da/d1d/d5d/da6/d9/ldd 0 2024-05-13T22:48:36.773 INFO:tasks.workunit.client.1.smithi130.stdout:0/702: link d0/d7/dd/d17/c2a d0/d7/dd/d17/d1f/d2c/d45/de9/ced 0 2024-05-13T22:48:36.775 INFO:tasks.workunit.client.1.smithi130.stdout:0/703: read d0/d7/dd/d2e/d4b/f15 [3290055,80898] 0 2024-05-13T22:48:36.777 INFO:tasks.workunit.client.1.smithi130.stdout:0/704: readlink d0/d7/dd/d69/da9/dd2/ldb 0 2024-05-13T22:48:36.779 INFO:tasks.workunit.client.1.smithi130.stdout:7/666: dwrite d7/f6a [0,4194304] 0 2024-05-13T22:48:36.779 INFO:tasks.workunit.client.1.smithi130.stdout:6/609: dwrite d4/d15/d1f/f2d [0,4194304] 0 2024-05-13T22:48:36.784 INFO:tasks.workunit.client.1.smithi130.stdout:5/655: creat d6/d7/fda x:0 0 0 2024-05-13T22:48:36.787 INFO:tasks.workunit.client.1.smithi130.stdout:4/637: unlink d8/dc/d35/d33/d3f/fa2 0 2024-05-13T22:48:36.791 INFO:tasks.workunit.client.1.smithi130.stdout:9/689: truncate d1/d7/d23/d28/d32/d65/fb9 152025 0 2024-05-13T22:48:36.800 INFO:tasks.workunit.client.1.smithi130.stdout:5/656: creat d6/dd/d18/d6e/d82/d84/fdb x:0 0 0 2024-05-13T22:48:36.800 INFO:tasks.workunit.client.1.smithi130.stdout:5/657: fdatasync d6/d7/f28 0 2024-05-13T22:48:36.801 INFO:tasks.workunit.client.1.smithi130.stdout:5/658: truncate d6/dd/d18/fca 1009201 0 2024-05-13T22:48:36.802 INFO:tasks.workunit.client.1.smithi130.stdout:3/647: rename d8/dc/d7b/l7e to d8/dc/d7b/da3/ldf 0 2024-05-13T22:48:36.804 INFO:tasks.workunit.client.1.smithi130.stdout:9/690: symlink d1/d7/ldd 0 2024-05-13T22:48:36.806 INFO:tasks.workunit.client.1.smithi130.stdout:5/659: mknod d6/d7/d19/d4d/cdc 0 2024-05-13T22:48:36.806 INFO:tasks.workunit.client.1.smithi130.stdout:5/660: chown d6/d2f/f7d 3706 1 2024-05-13T22:48:36.810 INFO:tasks.workunit.client.1.smithi130.stdout:1/617: rename db/d58/lc8 to db/d1b/da3/d49/d56/d7f/lca 0 2024-05-13T22:48:36.811 INFO:tasks.workunit.client.1.smithi130.stdout:3/648: dread d8/dc/f12 [0,4194304] 0 2024-05-13T22:48:36.813 INFO:tasks.workunit.client.1.smithi130.stdout:9/691: mknod d1/d7/cde 0 2024-05-13T22:48:36.816 INFO:tasks.workunit.client.1.smithi130.stdout:3/649: dwrite f3 [0,4194304] 0 2024-05-13T22:48:36.817 INFO:tasks.workunit.client.1.smithi130.stdout:3/650: chown d8/dc/d7b/da3/dd3/fd6 208453 1 2024-05-13T22:48:36.821 INFO:tasks.workunit.client.1.smithi130.stdout:6/610: rename d4/d16/f69 to d4/d15/d1d/daa/fd0 0 2024-05-13T22:48:36.825 INFO:tasks.workunit.client.1.smithi130.stdout:1/618: read db/d1b/da3/d2a/d38/d46/d6d/fbe [3178162,55053] 0 2024-05-13T22:48:36.826 INFO:tasks.workunit.client.1.smithi130.stdout:1/619: fdatasync db/d1b/da3/d2a/d35/fba 0 2024-05-13T22:48:36.831 INFO:tasks.workunit.client.1.smithi130.stdout:9/692: creat d1/d7/d23/d28/d2d/d2f/fdf x:0 0 0 2024-05-13T22:48:36.835 INFO:tasks.workunit.client.1.smithi130.stdout:5/661: link d6/dd/d18/d6e/f8b d6/d1a/daf/fdd 0 2024-05-13T22:48:36.836 INFO:tasks.workunit.client.1.smithi130.stdout:9/693: dwrite d1/db/fd9 [0,4194304] 0 2024-05-13T22:48:36.841 INFO:tasks.workunit.client.1.smithi130.stdout:9/694: dwrite d1/d7/d23/d28/d2d/d2f/d63/d6b/f84 [0,4194304] 0 2024-05-13T22:48:36.841 INFO:tasks.workunit.client.1.smithi130.stdout:7/667: rename d7/da/d1d/d3d/d46/lac to d7/da/d1d/db1/dd4/lde 0 2024-05-13T22:48:36.850 INFO:tasks.workunit.client.1.smithi130.stdout:5/662: symlink d6/d7/d19/d4d/d53/lde 0 2024-05-13T22:48:36.857 INFO:tasks.workunit.client.1.smithi130.stdout:6/611: rename d4/c13 to d4/d15/d1d/d28/d4e/cd1 0 2024-05-13T22:48:36.862 INFO:tasks.workunit.client.1.smithi130.stdout:9/695: creat d1/d7/d69/d79/fe0 x:0 0 0 2024-05-13T22:48:36.862 INFO:tasks.workunit.client.1.smithi130.stdout:1/620: write db/d1b/da3/f89 [158782,80789] 0 2024-05-13T22:48:36.866 INFO:tasks.workunit.client.1.smithi130.stdout:5/663: mkdir d6/d7/d19/d4d/d6a/ddf 0 2024-05-13T22:48:36.869 INFO:tasks.workunit.client.1.smithi130.stdout:6/612: unlink d4/d15/d1d/f22 0 2024-05-13T22:48:36.870 INFO:tasks.workunit.client.1.smithi130.stdout:7/668: mknod d7/da/d1d/d5d/da6/d7e/d8b/cdf 0 2024-05-13T22:48:36.874 INFO:tasks.workunit.client.1.smithi130.stdout:7/669: dread d7/da/d1d/d5d/d70/f8d [0,4194304] 0 2024-05-13T22:48:36.874 INFO:tasks.workunit.client.1.smithi130.stdout:9/696: unlink d1/db/f5e 0 2024-05-13T22:48:36.874 INFO:tasks.workunit.client.1.smithi130.stdout:1/621: creat db/d82/d9f/da0/fcb x:0 0 0 2024-05-13T22:48:36.881 INFO:tasks.workunit.client.1.smithi130.stdout:3/651: dread d8/d1a/d1c/f25 [0,4194304] 0 2024-05-13T22:48:36.884 INFO:tasks.workunit.client.1.smithi130.stdout:3/652: dread d8/fa1 [0,4194304] 0 2024-05-13T22:48:36.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:36 smithi130 ceph-mon[36591]: pgmap v35: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 61 MiB/s rd, 201 MiB/s wr, 380 op/s 2024-05-13T22:48:36.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:36 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:36.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:36 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:36.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:36 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:48:36.944 INFO:tasks.workunit.client.1.smithi130.stdout:5/664: creat d6/dd/d18/d6e/d82/db0/fe0 x:0 0 0 2024-05-13T22:48:36.944 INFO:tasks.workunit.client.1.smithi130.stdout:6/613: rename d4/d15/d1f/d46/d52/l7c to d4/d15/d1d/d28/ld2 0 2024-05-13T22:48:36.945 INFO:tasks.workunit.client.1.smithi130.stdout:3/653: read d8/dc/d15/f9a [14686,86565] 0 2024-05-13T22:48:36.945 INFO:tasks.workunit.client.1.smithi130.stdout:1/622: fsync db/d1b/da3/d49/d56/d7f/f83 0 2024-05-13T22:48:36.945 INFO:tasks.workunit.client.1.smithi130.stdout:9/697: getdents d1/d7/d23/d28/d2d/d2f/dc7 0 2024-05-13T22:48:36.946 INFO:tasks.workunit.client.1.smithi130.stdout:5/665: chown d6/d7/d19/d4d/d6a/fc0 13312 1 2024-05-13T22:48:36.947 INFO:tasks.workunit.client.1.smithi130.stdout:3/654: write d8/dc/d19/d27/f84 [222960,11467] 0 2024-05-13T22:48:36.949 INFO:tasks.workunit.client.1.smithi130.stdout:7/670: dread d7/da/d1d/d3d/d46/d92/faf [0,4194304] 0 2024-05-13T22:48:36.951 INFO:tasks.workunit.client.1.smithi130.stdout:6/614: creat d4/d15/d1f/d64/fd3 x:0 0 0 2024-05-13T22:48:36.955 INFO:tasks.workunit.client.1.smithi130.stdout:7/671: dwrite d7/da/d1d/db1/fd1 [0,4194304] 0 2024-05-13T22:48:36.956 INFO:tasks.workunit.client.1.smithi130.stdout:7/672: read d7/da/f62 [841123,18736] 0 2024-05-13T22:48:36.962 INFO:tasks.workunit.client.1.smithi130.stdout:1/623: unlink db/d1b/da3/d2a/d4c/l51 0 2024-05-13T22:48:36.962 INFO:tasks.workunit.client.1.smithi130.stdout:1/624: stat db/l50 0 2024-05-13T22:48:36.966 INFO:tasks.workunit.client.1.smithi130.stdout:1/625: readlink db/d1b/da3/d2a/d38/d66/l67 0 2024-05-13T22:48:36.966 INFO:tasks.workunit.client.1.smithi130.stdout:1/626: truncate db/d1b/da3/d49/d56/f7d 5052281 0 2024-05-13T22:48:36.966 INFO:tasks.workunit.client.1.smithi130.stdout:5/666: fsync d6/d1a/d72/fd9 0 2024-05-13T22:48:36.967 INFO:tasks.workunit.client.1.smithi130.stdout:3/655: truncate d8/dc/d19/d27/f97 4925571 0 2024-05-13T22:48:36.969 INFO:tasks.workunit.client.1.smithi130.stdout:2/666: sync 2024-05-13T22:48:36.972 INFO:tasks.workunit.client.1.smithi130.stdout:9/698: creat d1/d7/dd2/fe1 x:0 0 0 2024-05-13T22:48:36.973 INFO:tasks.workunit.client.1.smithi130.stdout:9/699: chown d1/d7/d69/da2/fad 1627 1 2024-05-13T22:48:36.977 INFO:tasks.workunit.client.1.smithi130.stdout:7/673: rename d7/da/d1d/c4e to d7/da/d1d/d3d/db9/ce0 0 2024-05-13T22:48:36.977 INFO:tasks.workunit.client.1.smithi130.stdout:7/674: write d7/da/f5c [2336138,57926] 0 2024-05-13T22:48:36.978 INFO:tasks.workunit.client.1.smithi130.stdout:1/627: fsync db/d1b/da3/d2a/d38/d46/f5f 0 2024-05-13T22:48:36.979 INFO:tasks.workunit.client.1.smithi130.stdout:9/700: dread d1/d7/d23/d28/d2d/f46 [0,4194304] 0 2024-05-13T22:48:36.984 INFO:tasks.workunit.client.1.smithi130.stdout:5/667: creat d6/d1a/daf/fe1 x:0 0 0 2024-05-13T22:48:36.985 INFO:tasks.workunit.client.1.smithi130.stdout:3/656: creat d8/dc/d7b/fe0 x:0 0 0 2024-05-13T22:48:36.989 INFO:tasks.workunit.client.1.smithi130.stdout:5/668: dwrite d6/dd/fb5 [0,4194304] 0 2024-05-13T22:48:36.990 INFO:tasks.workunit.client.1.smithi130.stdout:5/669: write d6/d7/f22 [4740569,16746] 0 2024-05-13T22:48:36.990 INFO:tasks.workunit.client.1.smithi130.stdout:2/667: symlink d4/db/dc/d18/d2a/le9 0 2024-05-13T22:48:36.996 INFO:tasks.workunit.client.1.smithi130.stdout:7/675: mkdir d7/da/d1d/db1/de1 0 2024-05-13T22:48:36.997 INFO:tasks.workunit.client.1.smithi130.stdout:7/676: write d7/da/d1d/d5d/d70/f85 [3569023,63524] 0 2024-05-13T22:48:36.998 INFO:tasks.workunit.client.1.smithi130.stdout:9/701: creat d1/d7/d69/da2/fe2 x:0 0 0 2024-05-13T22:48:37.011 INFO:tasks.workunit.client.1.smithi130.stdout:5/670: rename d6/d7/d19/d4d/f34 to d6/dd/d18/d6e/d82/d4b/dbc/fe2 0 2024-05-13T22:48:37.021 INFO:tasks.workunit.client.1.smithi130.stdout:9/702: rmdir d1/d15/d51 39 2024-05-13T22:48:37.023 INFO:tasks.workunit.client.1.smithi130.stdout:6/615: symlink d4/d15/d1d/d28/d75/ld4 0 2024-05-13T22:48:37.024 INFO:tasks.workunit.client.1.smithi130.stdout:6/616: truncate d4/d15/d1d/d60/fcc 1032866 0 2024-05-13T22:48:37.024 INFO:tasks.workunit.client.1.smithi130.stdout:2/668: mknod d4/da8/dc9/dbb/cea 0 2024-05-13T22:48:37.026 INFO:tasks.workunit.client.1.smithi130.stdout:5/671: creat d6/dd/d79/fe3 x:0 0 0 2024-05-13T22:48:37.027 INFO:tasks.workunit.client.1.smithi130.stdout:9/703: creat d1/d7/d23/fe3 x:0 0 0 2024-05-13T22:48:37.029 INFO:tasks.workunit.client.1.smithi130.stdout:3/657: creat d8/dc/d19/fe1 x:0 0 0 2024-05-13T22:48:37.033 INFO:tasks.workunit.client.1.smithi130.stdout:2/669: creat d4/db/d26/d9e/de4/feb x:0 0 0 2024-05-13T22:48:37.035 INFO:tasks.workunit.client.1.smithi130.stdout:1/628: truncate db/d1b/da3/d2a/d38/d46/f71 3743079 0 2024-05-13T22:48:37.039 INFO:tasks.workunit.client.1.smithi130.stdout:2/670: dwrite d4/db/dc/d18/d2a/d30/d88/d90/fc4 [0,4194304] 0 2024-05-13T22:48:37.042 INFO:tasks.workunit.client.1.smithi130.stdout:9/704: mkdir d1/d7/d23/d28/d73/db2/de4 0 2024-05-13T22:48:37.044 INFO:tasks.workunit.client.1.smithi130.stdout:9/705: dread d1/d7/d23/d28/d2d/d2f/f93 [0,4194304] 0 2024-05-13T22:48:37.046 INFO:tasks.workunit.client.1.smithi130.stdout:6/617: creat d4/d15/d1d/d28/d75/fd5 x:0 0 0 2024-05-13T22:48:37.047 INFO:tasks.workunit.client.1.smithi130.stdout:6/618: chown d4/d15/d1d/d28/d75/ld4 1704 1 2024-05-13T22:48:37.049 INFO:tasks.workunit.client.1.smithi130.stdout:9/706: dread d1/d7/d23/d28/d32/d65/fc0 [0,4194304] 0 2024-05-13T22:48:37.052 INFO:tasks.workunit.client.1.smithi130.stdout:2/671: symlink d4/db/dc/d18/d2a/d30/d88/d99/lec 0 2024-05-13T22:48:37.066 INFO:tasks.workunit.client.1.smithi130.stdout:3/658: dwrite d8/dc/d19/d27/f97 [0,4194304] 0 2024-05-13T22:48:37.070 INFO:tasks.workunit.client.1.smithi130.stdout:1/629: creat db/d82/fcc x:0 0 0 2024-05-13T22:48:37.071 INFO:tasks.workunit.client.1.smithi130.stdout:8/599: sync 2024-05-13T22:48:37.072 INFO:tasks.workunit.client.1.smithi130.stdout:2/672: dread - d4/db/dc/d18/d2a/f2f zero size 2024-05-13T22:48:37.075 INFO:tasks.workunit.client.1.smithi130.stdout:1/630: dwrite db/f17 [0,4194304] 0 2024-05-13T22:48:37.078 INFO:tasks.workunit.client.1.smithi130.stdout:6/619: truncate d4/d15/d1f/d46/d50/f54 1274352 0 2024-05-13T22:48:37.082 INFO:tasks.workunit.client.1.smithi130.stdout:9/707: dwrite d1/d15/f1d [0,4194304] 0 2024-05-13T22:48:37.084 INFO:tasks.workunit.client.1.smithi130.stdout:3/659: rename d8/dc/d15/f18 to d8/d1a/d55/fe2 0 2024-05-13T22:48:37.085 INFO:tasks.workunit.client.1.smithi130.stdout:3/660: readlink d8/dc/d19/d2c/l54 0 2024-05-13T22:48:37.094 INFO:tasks.workunit.client.1.smithi130.stdout:2/673: dread d4/db/f41 [0,4194304] 0 2024-05-13T22:48:37.096 INFO:tasks.workunit.client.1.smithi130.stdout:9/708: creat d1/d7/d69/fe5 x:0 0 0 2024-05-13T22:48:37.098 INFO:tasks.workunit.client.1.smithi130.stdout:8/600: rename d1/d2/db/d15/ca to d1/d2/db/d15/d18/d42/d53/da4/ce1 0 2024-05-13T22:48:37.100 INFO:tasks.workunit.client.1.smithi130.stdout:2/674: rmdir d4/db/dc/d18/d2a/d30/d88/d99 39 2024-05-13T22:48:37.101 INFO:tasks.workunit.client.1.smithi130.stdout:6/620: rmdir d4/d15/d1f/d46/d50 39 2024-05-13T22:48:37.103 INFO:tasks.workunit.client.1.smithi130.stdout:8/601: dwrite d1/d2/db/d15/d18/f4c [0,4194304] 0 2024-05-13T22:48:37.109 INFO:tasks.workunit.client.1.smithi130.stdout:9/709: dwrite d1/d15/f7b [0,4194304] 0 2024-05-13T22:48:37.117 INFO:tasks.workunit.client.1.smithi130.stdout:8/602: write d1/d2/db/d15/d7e/d9f/fa0 [1176980,86194] 0 2024-05-13T22:48:37.118 INFO:tasks.workunit.client.1.smithi130.stdout:8/603: fsync d1/dc/f44 0 2024-05-13T22:48:37.119 INFO:tasks.workunit.client.1.smithi130.stdout:8/604: dread - d1/d2/db/d15/d18/d42/d53/dd0/fd3 zero size 2024-05-13T22:48:37.120 INFO:tasks.workunit.client.1.smithi130.stdout:7/677: sync 2024-05-13T22:48:37.120 INFO:tasks.workunit.client.1.smithi130.stdout:0/705: sync 2024-05-13T22:48:37.120 INFO:tasks.workunit.client.1.smithi130.stdout:4/638: sync 2024-05-13T22:48:37.124 INFO:tasks.workunit.client.1.smithi130.stdout:3/661: truncate d8/dc/f3e 4272905 0 2024-05-13T22:48:37.129 INFO:tasks.workunit.client.1.smithi130.stdout:1/631: rename db/d82/d9f/fb0 to db/fcd 0 2024-05-13T22:48:37.130 INFO:tasks.workunit.client.1.smithi130.stdout:8/605: mknod d1/d2/db/d15/d18/d42/d53/dd0/ce2 0 2024-05-13T22:48:37.131 INFO:tasks.workunit.client.1.smithi130.stdout:8/606: dread - d1/d2/db/d15/d18/f5f zero size 2024-05-13T22:48:37.133 INFO:tasks.workunit.client.1.smithi130.stdout:2/675: write d4/db/dbf/d48/f73 [337752,117690] 0 2024-05-13T22:48:37.135 INFO:tasks.workunit.client.1.smithi130.stdout:9/710: mkdir d1/d7/d23/d28/d73/de6 0 2024-05-13T22:48:37.136 INFO:tasks.workunit.client.1.smithi130.stdout:9/711: write d1/fc9 [5217316,46991] 0 2024-05-13T22:48:37.138 INFO:tasks.workunit.client.1.smithi130.stdout:3/662: mknod d8/dc/d7b/da3/dd3/ce3 0 2024-05-13T22:48:37.141 INFO:tasks.workunit.client.1.smithi130.stdout:6/621: link d4/d15/d1f/f74 d4/d15/d1d/d60/da2/da4/fd6 0 2024-05-13T22:48:37.145 INFO:tasks.workunit.client.1.smithi130.stdout:7/678: rename d7/da/d1d/d5d/da6/d9/d4a/dbd/f30 to d7/da/d1d/d42/d43/da7/fe2 0 2024-05-13T22:48:37.147 INFO:tasks.workunit.client.1.smithi130.stdout:6/622: dwrite d4/d15/d1d/d28/d75/fcf [0,4194304] 0 2024-05-13T22:48:37.148 INFO:tasks.workunit.client.1.smithi130.stdout:6/623: fsync d4/d15/d1f/d64/fd3 0 2024-05-13T22:48:37.158 INFO:tasks.workunit.client.1.smithi130.stdout:0/706: creat d0/d7/dd/d2e/d9e/da7/dc3/ddf/fee x:0 0 0 2024-05-13T22:48:37.160 INFO:tasks.workunit.client.1.smithi130.stdout:5/672: sync 2024-05-13T22:48:37.164 INFO:tasks.workunit.client.1.smithi130.stdout:0/707: dwrite d0/fb7 [0,4194304] 0 2024-05-13T22:48:37.173 INFO:tasks.workunit.client.1.smithi130.stdout:4/639: write d8/dc/d26/f52 [3679331,36032] 0 2024-05-13T22:48:37.179 INFO:tasks.workunit.client.1.smithi130.stdout:2/676: dwrite d4/db/dc/d18/d58/f59 [4194304,4194304] 0 2024-05-13T22:48:37.179 INFO:tasks.workunit.client.1.smithi130.stdout:7/679: creat d7/da/d1d/d5d/da6/d7e/fe3 x:0 0 0 2024-05-13T22:48:37.181 INFO:tasks.workunit.client.1.smithi130.stdout:7/680: truncate d7/da/d1d/f6f 300856 0 2024-05-13T22:48:37.182 INFO:tasks.workunit.client.1.smithi130.stdout:7/681: dread - d7/da/d1d/d42/d43/da7/fb7 zero size 2024-05-13T22:48:37.183 INFO:tasks.workunit.client.1.smithi130.stdout:7/682: write d7/da/d1d/d5d/da6/d9/d66/f81 [3374906,8731] 0 2024-05-13T22:48:37.184 INFO:tasks.workunit.client.1.smithi130.stdout:7/683: write d7/da/d1d/db1/fd1 [2817753,57710] 0 2024-05-13T22:48:37.186 INFO:tasks.workunit.client.1.smithi130.stdout:5/673: rename d6/d2f/f7d to d6/dd/d18/d6e/dc7/fe4 0 2024-05-13T22:48:37.187 INFO:tasks.workunit.client.1.smithi130.stdout:5/674: fdatasync d6/d7/d19/f52 0 2024-05-13T22:48:37.191 INFO:tasks.workunit.client.1.smithi130.stdout:0/708: dread d0/d7/dd/d17/d52/d67/d68/d96/f84 [0,4194304] 0 2024-05-13T22:48:37.192 INFO:tasks.workunit.client.1.smithi130.stdout:5/675: dwrite d6/d7/f5e [0,4194304] 0 2024-05-13T22:48:37.194 INFO:tasks.workunit.client.1.smithi130.stdout:9/712: chown d1/d15/d51/l94 46 1 2024-05-13T22:48:37.198 INFO:tasks.workunit.client.1.smithi130.stdout:5/676: dwrite d6/d1a/d72/fcf [0,4194304] 0 2024-05-13T22:48:37.202 INFO:tasks.workunit.client.1.smithi130.stdout:4/640: unlink d8/dc/d35/d33/d44/d77/d94/db6/faf 0 2024-05-13T22:48:37.203 INFO:tasks.workunit.client.1.smithi130.stdout:8/607: truncate d1/d2/db/d15/d18/f63 1571964 0 2024-05-13T22:48:37.206 INFO:tasks.workunit.client.1.smithi130.stdout:1/632: creat db/d1b/da3/fce x:0 0 0 2024-05-13T22:48:37.209 INFO:tasks.workunit.client.1.smithi130.stdout:1/633: dread db/f17 [0,4194304] 0 2024-05-13T22:48:37.210 INFO:tasks.workunit.client.1.smithi130.stdout:6/624: symlink d4/d15/d1d/d60/da2/ld7 0 2024-05-13T22:48:37.211 INFO:tasks.workunit.client.1.smithi130.stdout:6/625: dread - d4/d15/d1d/d60/f66 zero size 2024-05-13T22:48:37.216 INFO:tasks.workunit.client.1.smithi130.stdout:7/684: mknod d7/da/d1d/d42/ce4 0 2024-05-13T22:48:37.216 INFO:tasks.workunit.client.1.smithi130.stdout:7/685: stat d7/da/d1d/d5d/d70/f8d 0 2024-05-13T22:48:37.217 INFO:tasks.workunit.client.1.smithi130.stdout:7/686: write d7/da/d1d/d5d/da6/d5b/fb3 [2398364,97900] 0 2024-05-13T22:48:37.239 INFO:tasks.workunit.client.1.smithi130.stdout:0/709: write d0/d7/dd/d17/d52/d5b/d5f/f70 [1210615,88496] 0 2024-05-13T22:48:37.256 INFO:tasks.workunit.client.1.smithi130.stdout:3/663: read d8/dc/f3e [1057255,88084] 0 2024-05-13T22:48:37.262 INFO:tasks.workunit.client.1.smithi130.stdout:4/641: symlink d8/lc7 0 2024-05-13T22:48:37.264 INFO:tasks.workunit.client.1.smithi130.stdout:8/608: link d1/d2/db/d15/d18/fb0 d1/d2/db/d15/d16/d54/d58/dab/fe3 0 2024-05-13T22:48:37.268 INFO:tasks.workunit.client.1.smithi130.stdout:1/634: symlink db/d1b/lcf 0 2024-05-13T22:48:37.272 INFO:tasks.workunit.client.1.smithi130.stdout:0/710: creat d0/d7/dd/d17/d1f/d2c/d45/d6b/fef x:0 0 0 2024-05-13T22:48:37.274 INFO:tasks.workunit.client.1.smithi130.stdout:6/626: rename d4/d15/d1d/d28/d4e/cd1 to d4/d15/d1d/d28/d4e/d9e/cd8 0 2024-05-13T22:48:37.276 INFO:tasks.workunit.client.1.smithi130.stdout:2/677: creat d4/db/dc/d18/fed x:0 0 0 2024-05-13T22:48:37.278 INFO:tasks.workunit.client.1.smithi130.stdout:2/678: write d4/db/dc/d18/d2a/f71 [3219567,88949] 0 2024-05-13T22:48:37.280 INFO:tasks.workunit.client.1.smithi130.stdout:3/664: unlink d8/dc/d15/fb1 0 2024-05-13T22:48:37.280 INFO:tasks.workunit.client.1.smithi130.stdout:3/665: fsync d8/dc/f21 0 2024-05-13T22:48:37.283 INFO:tasks.workunit.client.1.smithi130.stdout:5/677: link d6/d1a/f48 d6/d1a/d92/fe5 0 2024-05-13T22:48:37.284 INFO:tasks.workunit.client.1.smithi130.stdout:4/642: creat d8/dc/d35/d33/d44/fc8 x:0 0 0 2024-05-13T22:48:37.284 INFO:tasks.workunit.client.1.smithi130.stdout:4/643: write d8/f12 [2947854,58028] 0 2024-05-13T22:48:37.293 INFO:tasks.workunit.client.1.smithi130.stdout:0/711: creat d0/d7/dd/d2e/d9e/ff0 x:0 0 0 2024-05-13T22:48:37.296 INFO:tasks.workunit.client.1.smithi130.stdout:7/687: dread d7/da/d1d/d42/fb5 [0,4194304] 0 2024-05-13T22:48:37.298 INFO:tasks.workunit.client.1.smithi130.stdout:7/688: truncate d7/da/d1d/d5d/d70/f8c 221362 0 2024-05-13T22:48:37.300 INFO:tasks.workunit.client.1.smithi130.stdout:9/713: getdents d1/d7/d23 0 2024-05-13T22:48:37.300 INFO:tasks.workunit.client.1.smithi130.stdout:0/712: dwrite d0/d7/dd/d17/d52/d5b/d5f/fc9 [0,4194304] 0 2024-05-13T22:48:37.302 INFO:tasks.workunit.client.1.smithi130.stdout:3/666: stat d8/dc/d7b/da3/daf/fb7 0 2024-05-13T22:48:37.303 INFO:tasks.workunit.client.1.smithi130.stdout:0/713: write d0/d7/dd/d17/d52/d5b/d5f/fc9 [4005463,62728] 0 2024-05-13T22:48:37.304 INFO:tasks.workunit.client.1.smithi130.stdout:4/644: fsync d8/dc/d35/d33/d44/d56/f72 0 2024-05-13T22:48:37.305 INFO:tasks.workunit.client.1.smithi130.stdout:3/667: write d8/dc/d19/f8d [267882,60141] 0 2024-05-13T22:48:37.308 INFO:tasks.workunit.client.1.smithi130.stdout:8/609: mkdir d1/d9b/de4 0 2024-05-13T22:48:37.310 INFO:tasks.workunit.client.1.smithi130.stdout:4/645: write d8/dc/d35/d33/d44/d77/d94/db6/f20 [1066440,25518] 0 2024-05-13T22:48:37.310 INFO:tasks.workunit.client.1.smithi130.stdout:1/635: dwrite db/d1b/da3/d2a/d35/f3b [4194304,4194304] 0 2024-05-13T22:48:37.313 INFO:tasks.workunit.client.1.smithi130.stdout:2/679: mknod d4/db/dc/d18/d2a/d30/d88/d99/d9a/cee 0 2024-05-13T22:48:37.319 INFO:tasks.workunit.client.1.smithi130.stdout:3/668: dwrite d8/fbd [0,4194304] 0 2024-05-13T22:48:37.321 INFO:tasks.workunit.client.1.smithi130.stdout:4/646: dwrite d8/dc/d26/f36 [0,4194304] 0 2024-05-13T22:48:37.322 INFO:tasks.workunit.client.1.smithi130.stdout:4/647: read - d8/dc/d35/d33/d44/db8/faa zero size 2024-05-13T22:48:37.323 INFO:tasks.workunit.client.1.smithi130.stdout:9/714: rename d1/d7/d23/c56 to d1/d7/d23/d3d/d43/ce7 0 2024-05-13T22:48:37.327 INFO:tasks.workunit.client.1.smithi130.stdout:6/627: creat d4/d15/d1f/d46/fd9 x:0 0 0 2024-05-13T22:48:37.332 INFO:tasks.workunit.client.1.smithi130.stdout:8/610: mknod d1/d2/db/d15/d7e/dc2/ce5 0 2024-05-13T22:48:37.333 INFO:tasks.workunit.client.1.smithi130.stdout:0/714: dwrite d0/d7/d29/f40 [0,4194304] 0 2024-05-13T22:48:37.336 INFO:tasks.workunit.client.1.smithi130.stdout:2/680: creat d4/db/dc/d18/d2a/d30/d88/d99/fef x:0 0 0 2024-05-13T22:48:37.337 INFO:tasks.workunit.client.1.smithi130.stdout:2/681: write d4/db/dc/f4e [2201925,16259] 0 2024-05-13T22:48:37.342 INFO:tasks.workunit.client.1.smithi130.stdout:4/648: creat d8/dc/d35/d33/d44/d61/da0/fc9 x:0 0 0 2024-05-13T22:48:37.344 INFO:tasks.workunit.client.1.smithi130.stdout:2/682: dwrite d4/db/dbf/d48/d4a/f8c [0,4194304] 0 2024-05-13T22:48:37.347 INFO:tasks.workunit.client.1.smithi130.stdout:3/669: symlink d8/d35/le4 0 2024-05-13T22:48:37.349 INFO:tasks.workunit.client.1.smithi130.stdout:5/678: getdents d6/d7 0 2024-05-13T22:48:37.350 INFO:tasks.workunit.client.1.smithi130.stdout:9/715: mknod d1/d7/ce8 0 2024-05-13T22:48:37.353 INFO:tasks.workunit.client.1.smithi130.stdout:0/715: unlink d0/d7/dd/d2e/l5c 0 2024-05-13T22:48:37.353 INFO:tasks.workunit.client.1.smithi130.stdout:1/636: link db/d1b/da3/d2a/d4c/d5b/c9d db/d82/dc3/cd0 0 2024-05-13T22:48:37.355 INFO:tasks.workunit.client.1.smithi130.stdout:3/670: dwrite d8/d1a/d1c/faa [0,4194304] 0 2024-05-13T22:48:37.362 INFO:tasks.workunit.client.1.smithi130.stdout:1/637: dread db/d1b/da3/d72/daa/fb3 [0,4194304] 0 2024-05-13T22:48:37.362 INFO:tasks.workunit.client.1.smithi130.stdout:6/628: creat d4/d15/d1f/fda x:0 0 0 2024-05-13T22:48:37.362 INFO:tasks.workunit.client.1.smithi130.stdout:5/679: chown d6/d7/d19/d4d/d8a/fa5 4874 1 2024-05-13T22:48:37.363 INFO:tasks.workunit.client.1.smithi130.stdout:7/689: link d7/da/d1d/d5d/da6/d9/cc2 d7/da/d1d/d5d/da6/d9/ce5 0 2024-05-13T22:48:37.365 INFO:tasks.workunit.client.1.smithi130.stdout:2/683: getdents d4/db/dc/d18/d2a/d30/d88/dc5/dcd 0 2024-05-13T22:48:37.365 INFO:tasks.workunit.client.1.smithi130.stdout:4/649: rename d8/dc/d35/d33/d3f/f82 to d8/dc/d35/d33/d44/d77/d94/db6/dad/fca 0 2024-05-13T22:48:37.368 INFO:tasks.workunit.client.1.smithi130.stdout:9/716: dread d1/d7/d69/d79/f80 [0,4194304] 0 2024-05-13T22:48:37.370 INFO:tasks.workunit.client.1.smithi130.stdout:9/717: stat d1/c50 0 2024-05-13T22:48:37.370 INFO:tasks.workunit.client.1.smithi130.stdout:3/671: creat d8/fe5 x:0 0 0 2024-05-13T22:48:37.371 INFO:tasks.workunit.client.1.smithi130.stdout:0/716: dwrite d0/d7/dd/d17/fae [4194304,4194304] 0 2024-05-13T22:48:37.373 INFO:tasks.workunit.client.1.smithi130.stdout:3/672: write d8/fe5 [71489,111620] 0 2024-05-13T22:48:37.377 INFO:tasks.workunit.client.1.smithi130.stdout:2/684: creat d4/db/dbf/d42/d4b/ff0 x:0 0 0 2024-05-13T22:48:37.378 INFO:tasks.workunit.client.1.smithi130.stdout:5/680: dwrite d6/d7/f43 [0,4194304] 0 2024-05-13T22:48:37.378 INFO:tasks.workunit.client.1.smithi130.stdout:4/650: mknod d8/dc/d35/d33/d44/db8/ccb 0 2024-05-13T22:48:37.378 INFO:tasks.workunit.client.1.smithi130.stdout:6/629: dwrite d4/d15/d1f/d46/fb2 [0,4194304] 0 2024-05-13T22:48:37.379 INFO:tasks.workunit.client.1.smithi130.stdout:4/651: write d8/dc/d35/d33/d44/f54 [695364,36169] 0 2024-05-13T22:48:37.383 INFO:tasks.workunit.client.1.smithi130.stdout:9/718: creat d1/d96/fe9 x:0 0 0 2024-05-13T22:48:37.392 INFO:tasks.workunit.client.1.smithi130.stdout:3/673: creat d8/dc/d19/d27/fe6 x:0 0 0 2024-05-13T22:48:37.402 INFO:tasks.workunit.client.1.smithi130.stdout:6/630: mkdir d4/d15/d1d/d28/d4e/d9e/ddb 0 2024-05-13T22:48:37.405 INFO:tasks.workunit.client.1.smithi130.stdout:4/652: truncate d8/dc/d35/d33/d44/d61/f98 4881838 0 2024-05-13T22:48:37.407 INFO:tasks.workunit.client.1.smithi130.stdout:4/653: read d8/dc/d35/d33/d44/d77/d94/db6/f20 [929778,96637] 0 2024-05-13T22:48:37.408 INFO:tasks.workunit.client.1.smithi130.stdout:0/717: fsync d0/d7/dd/d17/d1f/d2c/d5a/fb9 0 2024-05-13T22:48:37.409 INFO:tasks.workunit.client.1.smithi130.stdout:0/718: chown d0/d7/d29/f74 12912679 1 2024-05-13T22:48:37.412 INFO:tasks.workunit.client.1.smithi130.stdout:5/681: symlink d6/d7/d19/dc1/le6 0 2024-05-13T22:48:37.413 INFO:tasks.workunit.client.1.smithi130.stdout:5/682: dread - d6/d7/d19/d4d/fa3 zero size 2024-05-13T22:48:37.414 INFO:tasks.workunit.client.1.smithi130.stdout:5/683: write d6/d7/f46 [5348784,50851] 0 2024-05-13T22:48:37.414 INFO:tasks.workunit.client.1.smithi130.stdout:7/690: getdents d7/da/d1d/d5d/d70 0 2024-05-13T22:48:37.417 INFO:tasks.workunit.client.1.smithi130.stdout:6/631: mkdir d4/d15/d1d/d28/ddc 0 2024-05-13T22:48:37.418 INFO:tasks.workunit.client.1.smithi130.stdout:6/632: write d4/d15/d1d/d28/d75/d7d/fb0 [986958,100822] 0 2024-05-13T22:48:37.421 INFO:tasks.workunit.client.1.smithi130.stdout:4/654: creat d8/dc/d35/d33/d44/db8/fcc x:0 0 0 2024-05-13T22:48:37.425 INFO:tasks.workunit.client.1.smithi130.stdout:9/719: dread d1/d7/fac [0,4194304] 0 2024-05-13T22:48:37.425 INFO:tasks.workunit.client.1.smithi130.stdout:9/720: truncate d1/f9d 36192 0 2024-05-13T22:48:37.425 INFO:tasks.workunit.client.1.smithi130.stdout:5/684: rmdir d6/d7/d19/d4d 39 2024-05-13T22:48:37.426 INFO:tasks.workunit.client.1.smithi130.stdout:9/721: stat d1/d7/d23/d28/d73/d8d/fa5 0 2024-05-13T22:48:37.434 INFO:tasks.workunit.client.1.smithi130.stdout:4/655: read d8/dc/f38 [251205,11868] 0 2024-05-13T22:48:37.440 INFO:tasks.workunit.client.1.smithi130.stdout:9/722: creat d1/d7/dd2/fea x:0 0 0 2024-05-13T22:48:37.441 INFO:tasks.workunit.client.1.smithi130.stdout:7/691: mkdir d7/da/d1d/de6 0 2024-05-13T22:48:37.441 INFO:tasks.workunit.client.1.smithi130.stdout:9/723: fsync d1/d7/d23/d28/d73/d8d/fa5 0 2024-05-13T22:48:37.445 INFO:tasks.workunit.client.1.smithi130.stdout:4/656: creat d8/dc/d35/d33/d44/d77/fcd x:0 0 0 2024-05-13T22:48:37.445 INFO:tasks.workunit.client.1.smithi130.stdout:6/633: stat d4/d15/d1f/d46/d50/f54 0 2024-05-13T22:48:37.447 INFO:tasks.workunit.client.1.smithi130.stdout:3/674: getdents d8/dc/d19 0 2024-05-13T22:48:37.447 INFO:tasks.workunit.client.1.smithi130.stdout:4/657: fsync d8/dc/d35/d33/d44/fa9 0 2024-05-13T22:48:37.447 INFO:tasks.workunit.client.1.smithi130.stdout:6/634: chown d4/d15/d1f/d46/d50/f54 334 1 2024-05-13T22:48:37.453 INFO:tasks.workunit.client.1.smithi130.stdout:4/658: dwrite f7 [0,4194304] 0 2024-05-13T22:48:37.453 INFO:tasks.workunit.client.1.smithi130.stdout:6/635: dwrite d4/d15/d1f/d46/d52/fc3 [0,4194304] 0 2024-05-13T22:48:37.473 INFO:tasks.workunit.client.1.smithi130.stdout:9/724: mkdir d1/d7/d23/d28/d73/d8d/deb 0 2024-05-13T22:48:37.475 INFO:tasks.workunit.client.1.smithi130.stdout:0/719: link d0/l8b d0/d7/dd/d2e/da8/dcd/lf1 0 2024-05-13T22:48:37.482 INFO:tasks.workunit.client.1.smithi130.stdout:8/611: sync 2024-05-13T22:48:37.484 INFO:tasks.workunit.client.1.smithi130.stdout:8/612: write d1/d2/db/d15/d18/f5f [808282,69707] 0 2024-05-13T22:48:37.485 INFO:tasks.workunit.client.1.smithi130.stdout:4/659: dread d8/dc/d35/f50 [0,4194304] 0 2024-05-13T22:48:37.489 INFO:tasks.workunit.client.1.smithi130.stdout:6/636: mkdir d4/d15/d1d/d60/da2/da4/ddd 0 2024-05-13T22:48:37.496 INFO:tasks.workunit.client.1.smithi130.stdout:0/720: symlink d0/d7/dd/d69/dac/lf2 0 2024-05-13T22:48:37.500 INFO:tasks.workunit.client.1.smithi130.stdout:5/685: getdents d6/d2f 0 2024-05-13T22:48:37.501 INFO:tasks.workunit.client.1.smithi130.stdout:5/686: truncate d6/d1a/d92/f98 190480 0 2024-05-13T22:48:37.502 INFO:tasks.workunit.client.1.smithi130.stdout:5/687: write d6/f73 [4794437,40794] 0 2024-05-13T22:48:37.503 INFO:tasks.workunit.client.1.smithi130.stdout:5/688: write d6/d7/f43 [63679,20355] 0 2024-05-13T22:48:37.506 INFO:tasks.workunit.client.1.smithi130.stdout:8/613: stat d1/d2/db/d15/d18/c8a 0 2024-05-13T22:48:37.506 INFO:tasks.workunit.client.1.smithi130.stdout:4/660: creat d8/dc/d35/d33/d44/d77/d94/db6/d81/fce x:0 0 0 2024-05-13T22:48:37.508 INFO:tasks.workunit.client.1.smithi130.stdout:4/661: fdatasync d8/dc/d35/d33/d44/d77/d94/db6/d81/f9e 0 2024-05-13T22:48:37.514 INFO:tasks.workunit.client.1.smithi130.stdout:0/721: unlink d0/d7/dd/d17/d52/d5b/f6f 0 2024-05-13T22:48:37.514 INFO:tasks.workunit.client.1.smithi130.stdout:0/722: dread - d0/d7/dd/d2e/fbe zero size 2024-05-13T22:48:37.519 INFO:tasks.workunit.client.1.smithi130.stdout:5/689: rename d6/dd/d18/d6e/d82/d56 to d6/dd/d18/d6e/d82/d4b/de7 0 2024-05-13T22:48:37.520 INFO:tasks.workunit.client.1.smithi130.stdout:5/690: fdatasync d6/dd/fcd 0 2024-05-13T22:48:37.521 INFO:tasks.workunit.client.1.smithi130.stdout:5/691: chown d6/d7/d32/f3d 9 1 2024-05-13T22:48:37.521 INFO:tasks.workunit.client.1.smithi130.stdout:8/614: symlink d1/d2/db/d15/d7e/d9f/le6 0 2024-05-13T22:48:37.526 INFO:tasks.workunit.client.1.smithi130.stdout:4/662: creat d8/dc/d35/d33/d44/d56/fcf x:0 0 0 2024-05-13T22:48:37.526 INFO:tasks.workunit.client.1.smithi130.stdout:4/663: stat d8/fa 0 2024-05-13T22:48:37.527 INFO:tasks.workunit.client.1.smithi130.stdout:4/664: write d8/dc/d35/fc1 [511713,55159] 0 2024-05-13T22:48:37.528 INFO:tasks.workunit.client.1.smithi130.stdout:9/725: creat d1/d7/d23/d28/d73/fec x:0 0 0 2024-05-13T22:48:37.531 INFO:tasks.workunit.client.1.smithi130.stdout:5/692: mknod d6/dd/d18/d6e/d76/ce8 0 2024-05-13T22:48:37.533 INFO:tasks.workunit.client.1.smithi130.stdout:0/723: dread d0/d7/dd/d17/d52/d67/d68/d96/f84 [0,4194304] 0 2024-05-13T22:48:37.541 INFO:tasks.workunit.client.1.smithi130.stdout:4/665: unlink f7 0 2024-05-13T22:48:37.545 INFO:tasks.workunit.client.1.smithi130.stdout:5/693: symlink d6/d1a/daf/le9 0 2024-05-13T22:48:37.545 INFO:tasks.workunit.client.1.smithi130.stdout:5/694: chown d6/dd/d18/l10 0 1 2024-05-13T22:48:37.551 INFO:tasks.workunit.client.1.smithi130.stdout:6/637: getdents d4/d15/d1d/d28/d75/d7d 0 2024-05-13T22:48:37.553 INFO:tasks.workunit.client.1.smithi130.stdout:1/638: sync 2024-05-13T22:48:37.555 INFO:tasks.workunit.client.1.smithi130.stdout:1/639: chown db/d4a/f95 29 1 2024-05-13T22:48:37.555 INFO:tasks.workunit.client.1.smithi130.stdout:1/640: dread - db/d4a/f95 zero size 2024-05-13T22:48:37.556 INFO:tasks.workunit.client.1.smithi130.stdout:3/675: dwrite d8/dc/d7b/da3/dd3/f74 [0,4194304] 0 2024-05-13T22:48:37.557 INFO:tasks.workunit.client.1.smithi130.stdout:3/676: chown d8/d1a/d1c/f94 2141 1 2024-05-13T22:48:37.558 INFO:tasks.workunit.client.1.smithi130.stdout:3/677: write d8/dc/d15/f76 [260125,15479] 0 2024-05-13T22:48:37.571 INFO:tasks.workunit.client.1.smithi130.stdout:5/695: creat d6/d7/d19/d4d/d6a/fea x:0 0 0 2024-05-13T22:48:37.572 INFO:tasks.workunit.client.1.smithi130.stdout:8/615: write d1/d2/db/d15/d16/d54/f55 [190326,79139] 0 2024-05-13T22:48:37.574 INFO:tasks.workunit.client.1.smithi130.stdout:9/726: getdents d1/d7/dd2 0 2024-05-13T22:48:37.575 INFO:tasks.workunit.client.1.smithi130.stdout:9/727: read d1/d7/d69/d79/f80 [599725,99888] 0 2024-05-13T22:48:37.575 INFO:tasks.workunit.client.1.smithi130.stdout:8/616: dread - d1/d2/db/d15/d16/d54/d58/fd8 zero size 2024-05-13T22:48:37.577 INFO:tasks.workunit.client.1.smithi130.stdout:1/641: mknod db/d82/cd1 0 2024-05-13T22:48:37.580 INFO:tasks.workunit.client.1.smithi130.stdout:5/696: mknod d6/dd/d18/d6e/d82/d4b/dbc/ceb 0 2024-05-13T22:48:37.582 INFO:tasks.workunit.client.1.smithi130.stdout:5/697: write d6/d7/f46 [4535270,20176] 0 2024-05-13T22:48:37.582 INFO:tasks.workunit.client.1.smithi130.stdout:0/724: dwrite d0/d7/dd/d17/d1f/f7d [0,4194304] 0 2024-05-13T22:48:37.586 INFO:tasks.workunit.client.1.smithi130.stdout:6/638: creat d4/d15/fde x:0 0 0 2024-05-13T22:48:37.586 INFO:tasks.workunit.client.1.smithi130.stdout:1/642: dwrite db/f16 [0,4194304] 0 2024-05-13T22:48:37.586 INFO:tasks.workunit.client.1.smithi130.stdout:1/643: readlink db/d1b/lad 0 2024-05-13T22:48:37.593 INFO:tasks.workunit.client.1.smithi130.stdout:9/728: creat d1/d7/d23/d3d/d43/fed x:0 0 0 2024-05-13T22:48:37.597 INFO:tasks.workunit.client.1.smithi130.stdout:4/666: getdents d8 0 2024-05-13T22:48:37.598 INFO:tasks.workunit.client.1.smithi130.stdout:8/617: symlink d1/d2/db/d15/d18/d1a/le7 0 2024-05-13T22:48:37.602 INFO:tasks.workunit.client.1.smithi130.stdout:3/678: creat d8/dc/d19/d2c/d61/fe7 x:0 0 0 2024-05-13T22:48:37.603 INFO:tasks.workunit.client.1.smithi130.stdout:4/667: dwrite d8/dc/d35/d33/d44/d56/fcf [0,4194304] 0 2024-05-13T22:48:37.609 INFO:tasks.workunit.client.1.smithi130.stdout:6/639: creat d4/d15/d1f/d64/fdf x:0 0 0 2024-05-13T22:48:37.610 INFO:tasks.workunit.client.1.smithi130.stdout:3/679: creat d8/dc/d7b/fe8 x:0 0 0 2024-05-13T22:48:37.612 INFO:tasks.workunit.client.1.smithi130.stdout:4/668: rmdir d8/dc/d26 39 2024-05-13T22:48:37.616 INFO:tasks.workunit.client.1.smithi130.stdout:3/680: rename d8/dc/f22 to d8/dc/d7b/da3/dd3/ddd/fe9 0 2024-05-13T22:48:37.617 INFO:tasks.workunit.client.1.smithi130.stdout:6/640: read d4/d15/d1d/f6f [658229,37900] 0 2024-05-13T22:48:37.618 INFO:tasks.workunit.client.1.smithi130.stdout:0/725: creat d0/d7/dd/d17/d52/ff3 x:0 0 0 2024-05-13T22:48:37.621 INFO:tasks.workunit.client.1.smithi130.stdout:4/669: symlink d8/dc/d35/d33/d44/d56/ld0 0 2024-05-13T22:48:37.623 INFO:tasks.workunit.client.1.smithi130.stdout:0/726: dwrite d0/d7/dd/d2e/da8/fc0 [0,4194304] 0 2024-05-13T22:48:37.625 INFO:tasks.workunit.client.1.smithi130.stdout:0/727: read - d0/d7/dd/d2e/d9e/da7/fd8 zero size 2024-05-13T22:48:37.625 INFO:tasks.workunit.client.1.smithi130.stdout:3/681: symlink d8/dc/d7b/da3/dd3/ddd/lea 0 2024-05-13T22:48:37.625 INFO:tasks.workunit.client.1.smithi130.stdout:6/641: stat d4/d15/d1d/d28/c6a 0 2024-05-13T22:48:37.625 INFO:tasks.workunit.client.1.smithi130.stdout:3/682: stat d8/lb3 0 2024-05-13T22:48:37.630 INFO:tasks.workunit.client.1.smithi130.stdout:5/698: getdents d6/dd/d18/d6e/d82/db0 0 2024-05-13T22:48:37.632 INFO:tasks.workunit.client.1.smithi130.stdout:5/699: readlink d6/d1a/daf/le9 0 2024-05-13T22:48:37.633 INFO:tasks.workunit.client.1.smithi130.stdout:6/642: chown d4/d15/d1f/f84 41976037 1 2024-05-13T22:48:37.633 INFO:tasks.workunit.client.1.smithi130.stdout:0/728: dwrite d0/d7/dd/d17/fae [0,4194304] 0 2024-05-13T22:48:37.637 INFO:tasks.workunit.client.1.smithi130.stdout:3/683: read d8/d1a/d55/f9b [823812,94937] 0 2024-05-13T22:48:37.647 INFO:tasks.workunit.client.1.smithi130.stdout:6/643: rmdir d4/d15 39 2024-05-13T22:48:37.649 INFO:tasks.workunit.client.1.smithi130.stdout:0/729: mkdir d0/d7/dd/d69/da9/d8d/df4 0 2024-05-13T22:48:37.651 INFO:tasks.workunit.client.1.smithi130.stdout:3/684: symlink d8/dc/d7b/da3/dd3/ddd/leb 0 2024-05-13T22:48:37.651 INFO:tasks.workunit.client.1.smithi130.stdout:2/685: sync 2024-05-13T22:48:37.652 INFO:tasks.workunit.client.1.smithi130.stdout:7/692: sync 2024-05-13T22:48:37.653 INFO:tasks.workunit.client.1.smithi130.stdout:2/686: chown d4/db/dbf/d48/l60 8310666 1 2024-05-13T22:48:37.654 INFO:tasks.workunit.client.1.smithi130.stdout:6/644: chown d4/d15/d1d/d60/fcc 12952406 1 2024-05-13T22:48:37.654 INFO:tasks.workunit.client.1.smithi130.stdout:5/700: creat d6/d7/d19/d4d/fec x:0 0 0 2024-05-13T22:48:37.654 INFO:tasks.workunit.client.1.smithi130.stdout:2/687: dread - d4/da8/fc7 zero size 2024-05-13T22:48:37.655 INFO:tasks.workunit.client.1.smithi130.stdout:5/701: readlink d6/d7/d32/l4a 0 2024-05-13T22:48:37.655 INFO:tasks.workunit.client.1.smithi130.stdout:6/645: stat d4/d15/d1d/d28/d75/da8 0 2024-05-13T22:48:37.655 INFO:tasks.workunit.client.1.smithi130.stdout:4/670: write d8/dc/d35/d33/d44/d61/f98 [3182348,41459] 0 2024-05-13T22:48:37.664 INFO:tasks.workunit.client.1.smithi130.stdout:5/702: creat d6/d7/d32/fed x:0 0 0 2024-05-13T22:48:37.666 INFO:tasks.workunit.client.1.smithi130.stdout:6/646: write d4/d15/d1f/d46/d50/f8a [47389,92902] 0 2024-05-13T22:48:37.668 INFO:tasks.workunit.client.1.smithi130.stdout:3/685: dread d8/dc/d19/f30 [0,4194304] 0 2024-05-13T22:48:37.668 INFO:tasks.workunit.client.1.smithi130.stdout:2/688: getdents d4/db/dc/d18/d2a/d30/d88/dc5/dcd 0 2024-05-13T22:48:37.668 INFO:tasks.workunit.client.1.smithi130.stdout:5/703: read - d6/dd/d18/d6e/dc7/fe4 zero size 2024-05-13T22:48:37.669 INFO:tasks.workunit.client.1.smithi130.stdout:6/647: chown d4/d15/d1d/db8 793250311 1 2024-05-13T22:48:37.671 INFO:tasks.workunit.client.1.smithi130.stdout:2/689: write d4/db/dbf/fe8 [412793,33596] 0 2024-05-13T22:48:37.673 INFO:tasks.workunit.client.1.smithi130.stdout:7/693: dwrite d7/da/d1d/d42/fb5 [4194304,4194304] 0 2024-05-13T22:48:37.674 INFO:tasks.workunit.client.1.smithi130.stdout:0/730: dread d0/d7/fb [0,4194304] 0 2024-05-13T22:48:37.674 INFO:tasks.workunit.client.1.smithi130.stdout:6/648: creat d4/d15/d1d/d60/fe0 x:0 0 0 2024-05-13T22:48:37.678 INFO:tasks.workunit.client.1.smithi130.stdout:5/704: dread d6/d7/f5e [0,4194304] 0 2024-05-13T22:48:37.679 INFO:tasks.workunit.client.1.smithi130.stdout:7/694: unlink d7/da/d1d/d5d/da6/d5b/l9d 0 2024-05-13T22:48:37.681 INFO:tasks.workunit.client.1.smithi130.stdout:3/686: rename d8/dc/f12 to d8/dc/d7b/da3/fec 0 2024-05-13T22:48:37.683 INFO:tasks.workunit.client.1.smithi130.stdout:2/690: symlink d4/db/dc/d18/d2a/d30/d88/dc5/dcd/lf1 0 2024-05-13T22:48:37.691 INFO:tasks.workunit.client.1.smithi130.stdout:2/691: mkdir d4/db/d26/d9e/de4/df2 0 2024-05-13T22:48:37.695 INFO:tasks.workunit.client.1.smithi130.stdout:7/695: fsync d7/da/d1d/d42/f58 0 2024-05-13T22:48:37.695 INFO:tasks.workunit.client.1.smithi130.stdout:3/687: symlink d8/d35/led 0 2024-05-13T22:48:37.698 INFO:tasks.workunit.client.1.smithi130.stdout:2/692: creat d4/db/dc/d18/d2a/d30/d88/d99/d9a/ff3 x:0 0 0 2024-05-13T22:48:37.701 INFO:tasks.workunit.client.1.smithi130.stdout:7/696: fdatasync d7/da/d1d/d3d/d46/f4b 0 2024-05-13T22:48:37.704 INFO:tasks.workunit.client.1.smithi130.stdout:6/649: truncate d4/f73 4147535 0 2024-05-13T22:48:37.705 INFO:tasks.workunit.client.1.smithi130.stdout:2/693: rename d4/db/dc/d18/c31 to d4/da8/dc9/dbb/cf4 0 2024-05-13T22:48:37.706 INFO:tasks.workunit.client.1.smithi130.stdout:5/705: dread d6/dd/d18/d6e/d82/d71/f81 [0,4194304] 0 2024-05-13T22:48:37.707 INFO:tasks.workunit.client.1.smithi130.stdout:2/694: fdatasync d4/db/dbf/d42/d4b/fe3 0 2024-05-13T22:48:37.708 INFO:tasks.workunit.client.1.smithi130.stdout:3/688: truncate d8/dc/d7b/da3/fec 1070989 0 2024-05-13T22:48:37.711 INFO:tasks.workunit.client.1.smithi130.stdout:3/689: chown d8/dc/d15/cc3 222 1 2024-05-13T22:48:37.712 INFO:tasks.workunit.client.1.smithi130.stdout:7/697: read d7/da/d1d/d5d/da6/d39/fa8 [2806395,79292] 0 2024-05-13T22:48:37.714 INFO:tasks.workunit.client.1.smithi130.stdout:2/695: dwrite d4/db/dc/d18/d2a/d30/d88/d90/fc4 [0,4194304] 0 2024-05-13T22:48:37.720 INFO:tasks.workunit.client.1.smithi130.stdout:0/731: dread d0/d7/dd/d17/d1f/d2c/d5a/fb9 [0,4194304] 0 2024-05-13T22:48:37.720 INFO:tasks.workunit.client.1.smithi130.stdout:1/644: sync 2024-05-13T22:48:37.721 INFO:tasks.workunit.client.1.smithi130.stdout:7/698: mknod d7/da/d1d/db1/dbe/dd3/ce7 0 2024-05-13T22:48:37.721 INFO:tasks.workunit.client.1.smithi130.stdout:6/650: dread d4/d15/d1f/f3e [0,4194304] 0 2024-05-13T22:48:37.722 INFO:tasks.workunit.client.1.smithi130.stdout:1/645: stat db/d82/lbf 0 2024-05-13T22:48:37.723 INFO:tasks.workunit.client.1.smithi130.stdout:6/651: chown d4/d15/d1d/d28/f30 1117035878 1 2024-05-13T22:48:37.724 INFO:tasks.workunit.client.1.smithi130.stdout:1/646: write db/d1b/da3/d49/d56/fbc [347364,23269] 0 2024-05-13T22:48:37.725 INFO:tasks.workunit.client.1.smithi130.stdout:1/647: write db/d4a/f4b [3363317,481] 0 2024-05-13T22:48:37.727 INFO:tasks.workunit.client.1.smithi130.stdout:1/648: chown db/d1b/da3/d49/d56/d7f/f96 2855246 1 2024-05-13T22:48:37.729 INFO:tasks.workunit.client.1.smithi130.stdout:2/696: mknod d4/db/d26/cf5 0 2024-05-13T22:48:37.729 INFO:tasks.workunit.client.1.smithi130.stdout:5/706: write d6/d7/f45 [4045436,48382] 0 2024-05-13T22:48:37.731 INFO:tasks.workunit.client.1.smithi130.stdout:2/697: rename d4/db/dc/d18 to d4/db/dc/d18/d58/df6 22 2024-05-13T22:48:37.732 INFO:tasks.workunit.client.1.smithi130.stdout:2/698: readlink d4/db/dc/d18/d2a/d66/l7c 0 2024-05-13T22:48:37.734 INFO:tasks.workunit.client.1.smithi130.stdout:7/699: creat d7/da/d1d/d5d/da6/d7e/d8b/fe8 x:0 0 0 2024-05-13T22:48:37.736 INFO:tasks.workunit.client.1.smithi130.stdout:5/707: dwrite d6/d7/d32/fed [0,4194304] 0 2024-05-13T22:48:37.740 INFO:tasks.workunit.client.1.smithi130.stdout:2/699: dwrite d4/db/dbf/fe8 [0,4194304] 0 2024-05-13T22:48:37.740 INFO:tasks.workunit.client.1.smithi130.stdout:0/732: mknod d0/d7/dd/d17/cf5 0 2024-05-13T22:48:37.741 INFO:tasks.workunit.client.1.smithi130.stdout:3/690: link d8/dc/c95 d8/dc/d7b/da3/cee 0 2024-05-13T22:48:37.749 INFO:tasks.workunit.client.1.smithi130.stdout:7/700: mknod d7/da/d1d/d5d/d70/ce9 0 2024-05-13T22:48:37.750 INFO:tasks.workunit.client.1.smithi130.stdout:7/701: truncate d7/da/d1d/f6f 917552 0 2024-05-13T22:48:37.753 INFO:tasks.workunit.client.1.smithi130.stdout:6/652: symlink d4/d15/d1d/d28/d75/le1 0 2024-05-13T22:48:37.757 INFO:tasks.workunit.client.1.smithi130.stdout:0/733: creat d0/d7/dd/d2e/d9e/da7/dc3/ff6 x:0 0 0 2024-05-13T22:48:37.760 INFO:tasks.workunit.client.1.smithi130.stdout:1/649: dwrite db/d1b/da3/d2a/d38/d46/d6d/fbe [0,4194304] 0 2024-05-13T22:48:37.762 INFO:tasks.workunit.client.1.smithi130.stdout:6/653: readlink d4/d15/d1f/l3b 0 2024-05-13T22:48:37.763 INFO:tasks.workunit.client.1.smithi130.stdout:3/691: mkdir d8/dc/d19/d2c/d61/def 0 2024-05-13T22:48:37.764 INFO:tasks.workunit.client.1.smithi130.stdout:9/729: sync 2024-05-13T22:48:37.764 INFO:tasks.workunit.client.1.smithi130.stdout:8/618: sync 2024-05-13T22:48:37.764 INFO:tasks.workunit.client.1.smithi130.stdout:2/700: dread d4/db/dbf/d42/d4b/fb5 [0,4194304] 0 2024-05-13T22:48:37.765 INFO:tasks.workunit.client.1.smithi130.stdout:7/702: write d7/da/d1d/d42/d43/da7/fe2 [7939967,68314] 0 2024-05-13T22:48:37.767 INFO:tasks.workunit.client.1.smithi130.stdout:0/734: mkdir d0/d7/dd/d17/d1f/d2c/d45/d57/df7 0 2024-05-13T22:48:37.775 INFO:tasks.workunit.client.1.smithi130.stdout:8/619: mkdir d1/d2/db/d15/d18/d42/d53/de8 0 2024-05-13T22:48:37.776 INFO:tasks.workunit.client.1.smithi130.stdout:2/701: mknod d4/db/dc/d18/d2a/cf7 0 2024-05-13T22:48:37.777 INFO:tasks.workunit.client.1.smithi130.stdout:7/703: creat d7/da/d1d/db1/fea x:0 0 0 2024-05-13T22:48:37.777 INFO:tasks.workunit.client.1.smithi130.stdout:2/702: dread - d4/db/dc/d18/fca zero size 2024-05-13T22:48:37.778 INFO:tasks.workunit.client.1.smithi130.stdout:1/650: mknod db/d1b/da3/d2a/d38/cd2 0 2024-05-13T22:48:37.779 INFO:tasks.workunit.client.1.smithi130.stdout:0/735: rename d0/d7/d29 to d0/d7/dd/d2e/da8/dcd/df8 0 2024-05-13T22:48:37.782 INFO:tasks.workunit.client.1.smithi130.stdout:5/708: dwrite d6/dd/f3e [0,4194304] 0 2024-05-13T22:48:37.782 INFO:tasks.workunit.client.1.smithi130.stdout:0/736: dread - d0/d7/dd/d2e/d9e/da7/fb8 zero size 2024-05-13T22:48:37.784 INFO:tasks.workunit.client.1.smithi130.stdout:0/737: chown d0/d7/dd/d17/fbb 64932 1 2024-05-13T22:48:37.786 INFO:tasks.workunit.client.1.smithi130.stdout:1/651: creat db/d9c/fd3 x:0 0 0 2024-05-13T22:48:37.786 INFO:tasks.workunit.client.1.smithi130.stdout:7/704: dwrite d7/da/d1d/f6f [0,4194304] 0 2024-05-13T22:48:37.788 INFO:tasks.workunit.client.1.smithi130.stdout:7/705: write d7/da/d1d/f9a [1540799,19929] 0 2024-05-13T22:48:37.788 INFO:tasks.workunit.client.1.smithi130.stdout:3/692: creat d8/ff0 x:0 0 0 2024-05-13T22:48:37.814 INFO:tasks.workunit.client.1.smithi130.stdout:0/738: dread d0/d7/dd/d2e/d9e/da7/dc3/fdd [0,4194304] 0 2024-05-13T22:48:37.815 INFO:tasks.workunit.client.1.smithi130.stdout:1/652: unlink db/d1b/da3/d2a/d38/d46/l77 0 2024-05-13T22:48:37.817 INFO:tasks.workunit.client.1.smithi130.stdout:8/620: truncate d1/d2/db/d15/d7e/d9f/fa0 497834 0 2024-05-13T22:48:37.818 INFO:tasks.workunit.client.1.smithi130.stdout:2/703: link d4/db/f28 d4/db/dbf/d42/d4b/ff8 0 2024-05-13T22:48:37.819 INFO:tasks.workunit.client.1.smithi130.stdout:0/739: truncate d0/d7/f23 923864 0 2024-05-13T22:48:37.819 INFO:tasks.workunit.client.1.smithi130.stdout:1/653: mkdir db/d1b/da3/d2a/d35/dd4 0 2024-05-13T22:48:37.821 INFO:tasks.workunit.client.1.smithi130.stdout:0/740: fsync d0/fb7 0 2024-05-13T22:48:37.821 INFO:tasks.workunit.client.1.smithi130.stdout:9/730: dwrite d1/d7/d23/d28/fab [0,4194304] 0 2024-05-13T22:48:37.822 INFO:tasks.workunit.client.1.smithi130.stdout:8/621: read d1/d2/db/d15/d18/f5f [264669,93605] 0 2024-05-13T22:48:37.825 INFO:tasks.workunit.client.1.smithi130.stdout:6/654: dwrite d4/d15/f44 [0,4194304] 0 2024-05-13T22:48:37.829 INFO:tasks.workunit.client.1.smithi130.stdout:9/731: creat d1/d7/d69/fee x:0 0 0 2024-05-13T22:48:37.830 INFO:tasks.workunit.client.1.smithi130.stdout:9/732: chown d1/d7/d23/d3d/l76 6 1 2024-05-13T22:48:37.832 INFO:tasks.workunit.client.1.smithi130.stdout:0/741: creat d0/d7/dd/d69/da9/ff9 x:0 0 0 2024-05-13T22:48:37.834 INFO:tasks.workunit.client.1.smithi130.stdout:1/654: rename db/d9c/la6 to db/d1b/da3/d72/daa/ld5 0 2024-05-13T22:48:37.839 INFO:tasks.workunit.client.1.smithi130.stdout:0/742: mknod d0/d7/dd/d17/d52/d67/d68/d96/cfa 0 2024-05-13T22:48:37.840 INFO:tasks.workunit.client.1.smithi130.stdout:0/743: chown d0/d7/dd/d17/d52/d5b 0 1 2024-05-13T22:48:37.843 INFO:tasks.workunit.client.1.smithi130.stdout:8/622: rename d1/d2/db/d15/d18/l50 to d1/d2/db/d15/d7e/dc2/le9 0 2024-05-13T22:48:37.843 INFO:tasks.workunit.client.1.smithi130.stdout:3/693: write d8/fa [292658,62078] 0 2024-05-13T22:48:37.848 INFO:tasks.workunit.client.1.smithi130.stdout:0/744: chown d0/d7/dd/d2e/l36 319772 1 2024-05-13T22:48:37.848 INFO:tasks.workunit.client.1.smithi130.stdout:4/671: sync 2024-05-13T22:48:37.849 INFO:tasks.workunit.client.1.smithi130.stdout:4/672: stat d8/dc 0 2024-05-13T22:48:37.850 INFO:tasks.workunit.client.1.smithi130.stdout:0/745: chown d0/d7/dd/d17/d52/d67/d68/d96/f83 4108518 1 2024-05-13T22:48:37.851 INFO:tasks.workunit.client.1.smithi130.stdout:6/655: rename d4/d15/d1d/d28/lab to d4/d15/d1d/db8/le2 0 2024-05-13T22:48:37.851 INFO:tasks.workunit.client.1.smithi130.stdout:8/623: dwrite d1/d2/db/f8 [0,4194304] 0 2024-05-13T22:48:37.853 INFO:tasks.workunit.client.1.smithi130.stdout:7/706: truncate d7/da/d1d/f6f 3150228 0 2024-05-13T22:48:37.855 INFO:tasks.workunit.client.1.smithi130.stdout:1/655: dread db/d1b/da3/f7a [0,4194304] 0 2024-05-13T22:48:37.857 INFO:tasks.workunit.client.1.smithi130.stdout:1/656: chown db/d1b/da3/d2a/d35/f3b 122 1 2024-05-13T22:48:37.858 INFO:tasks.workunit.client.1.smithi130.stdout:1/657: dread - db/d4a/f95 zero size 2024-05-13T22:48:37.858 INFO:tasks.workunit.client.1.smithi130.stdout:4/673: dwrite d8/dc/d84/f9c [0,4194304] 0 2024-05-13T22:48:37.866 INFO:tasks.workunit.client.1.smithi130.stdout:9/733: rename d1/d7/d23/d28/d32/d65/c95 to d1/d7/d23/d28/d73/db2/cef 0 2024-05-13T22:48:37.868 INFO:tasks.workunit.client.1.smithi130.stdout:2/704: truncate d4/db/dbf/d48/d4a/d76/f92 438896 0 2024-05-13T22:48:37.869 INFO:tasks.workunit.client.1.smithi130.stdout:3/694: unlink d8/d35/f48 0 2024-05-13T22:48:37.873 INFO:tasks.workunit.client.1.smithi130.stdout:8/624: creat d1/d9b/fea x:0 0 0 2024-05-13T22:48:37.874 INFO:tasks.workunit.client.1.smithi130.stdout:3/695: dwrite d8/dc/d19/f9e [0,4194304] 0 2024-05-13T22:48:37.875 INFO:tasks.workunit.client.1.smithi130.stdout:7/707: mkdir d7/da/d1d/d5d/da6/d7e/deb 0 2024-05-13T22:48:37.880 INFO:tasks.workunit.client.1.smithi130.stdout:4/674: unlink d8/dc/d35/d33/fbe 0 2024-05-13T22:48:37.881 INFO:tasks.workunit.client.1.smithi130.stdout:5/709: sync 2024-05-13T22:48:37.882 INFO:tasks.workunit.client.1.smithi130.stdout:7/708: dread d7/da/d1d/d5d/da6/d7e/fdc [0,4194304] 0 2024-05-13T22:48:37.886 INFO:tasks.workunit.client.1.smithi130.stdout:0/746: dwrite d0/f42 [0,4194304] 0 2024-05-13T22:48:37.890 INFO:tasks.workunit.client.1.smithi130.stdout:2/705: dread - d4/db/dc/d18/d2a/f67 zero size 2024-05-13T22:48:37.894 INFO:tasks.workunit.client.1.smithi130.stdout:8/625: symlink d1/d2/db/d15/d7e/dc2/leb 0 2024-05-13T22:48:37.895 INFO:tasks.workunit.client.1.smithi130.stdout:8/626: chown d1/d2/db/c2b 21873957 1 2024-05-13T22:48:37.897 INFO:tasks.workunit.client.1.smithi130.stdout:1/658: dread db/d9c/f8a [0,4194304] 0 2024-05-13T22:48:37.902 INFO:tasks.workunit.client.1.smithi130.stdout:7/709: creat d7/da/d1d/d3d/d46/fec x:0 0 0 2024-05-13T22:48:37.903 INFO:tasks.workunit.client.1.smithi130.stdout:0/747: symlink d0/da0/lfb 0 2024-05-13T22:48:37.904 INFO:tasks.workunit.client.1.smithi130.stdout:9/734: symlink d1/d7/lf0 0 2024-05-13T22:48:37.908 INFO:tasks.workunit.client.1.smithi130.stdout:8/627: mknod d1/d2/db/d15/d18/d42/d53/d56/cec 0 2024-05-13T22:48:37.909 INFO:tasks.workunit.client.1.smithi130.stdout:0/748: dwrite d0/d7/dd/d17/d52/d67/d68/d96/fa2 [0,4194304] 0 2024-05-13T22:48:37.911 INFO:tasks.workunit.client.1.smithi130.stdout:8/628: truncate d1/d2/db/d15/d18/d42/d53/fd6 396107 0 2024-05-13T22:48:37.911 INFO:tasks.workunit.client.1.smithi130.stdout:1/659: creat db/d1b/da3/d2a/d38/d46/d6d/fd6 x:0 0 0 2024-05-13T22:48:37.912 INFO:tasks.workunit.client.1.smithi130.stdout:1/660: readlink db/d1b/da3/d2a/l97 0 2024-05-13T22:48:37.913 INFO:tasks.workunit.client.1.smithi130.stdout:1/661: write db/d82/fcc [732148,30211] 0 2024-05-13T22:48:37.914 INFO:tasks.workunit.client.1.smithi130.stdout:1/662: chown db/d4a/f94 7285944 1 2024-05-13T22:48:37.914 INFO:tasks.workunit.client.1.smithi130.stdout:1/663: fdatasync db/d9c/fd3 0 2024-05-13T22:48:37.915 INFO:tasks.workunit.client.1.smithi130.stdout:4/675: rename d8/dc/d26/f2b to d8/dc/d26/fd1 0 2024-05-13T22:48:37.917 INFO:tasks.workunit.client.1.smithi130.stdout:4/676: truncate d8/dc/d35/d33/d44/d77/fcd 783009 0 2024-05-13T22:48:37.918 INFO:tasks.workunit.client.1.smithi130.stdout:6/656: rename d4/d15/d1d/d60/da2/ld7 to d4/le3 0 2024-05-13T22:48:37.925 INFO:tasks.workunit.client.1.smithi130.stdout:9/735: creat d1/d7/d23/d28/d2d/d2f/d63/d6b/ff1 x:0 0 0 2024-05-13T22:48:37.926 INFO:tasks.workunit.client.1.smithi130.stdout:5/710: write d6/dd/d18/d6e/f8b [1093984,123927] 0 2024-05-13T22:48:37.926 INFO:tasks.workunit.client.1.smithi130.stdout:9/736: fdatasync d1/d15/f7b 0 2024-05-13T22:48:37.926 INFO:tasks.workunit.client.1.smithi130.stdout:5/711: readlink d6/d1a/d92/la2 0 2024-05-13T22:48:37.927 INFO:tasks.workunit.client.1.smithi130.stdout:5/712: fsync d6/dd/d18/d6e/d82/d84/fdb 0 2024-05-13T22:48:37.929 INFO:tasks.workunit.client.1.smithi130.stdout:3/696: link d8/d1a/f47 d8/dc/d19/d2c/d61/def/ff1 0 2024-05-13T22:48:37.931 INFO:tasks.workunit.client.1.smithi130.stdout:4/677: creat d8/dc/d35/d33/d44/d56/d8c/fd2 x:0 0 0 2024-05-13T22:48:37.932 INFO:tasks.workunit.client.1.smithi130.stdout:6/657: mkdir d4/d15/d1d/d28/d75/da8/da0/de4 0 2024-05-13T22:48:37.934 INFO:tasks.workunit.client.1.smithi130.stdout:3/697: dwrite d8/dc/f9f [0,4194304] 0 2024-05-13T22:48:37.935 INFO:tasks.workunit.client.1.smithi130.stdout:9/737: write d1/d7/f57 [2281567,100364] 0 2024-05-13T22:48:37.936 INFO:tasks.workunit.client.1.smithi130.stdout:0/749: symlink d0/d7/dd/d17/d1f/d2c/de3/lfc 0 2024-05-13T22:48:37.938 INFO:tasks.workunit.client.1.smithi130.stdout:8/629: symlink d1/d2/db/d15/d18/d42/d53/de8/led 0 2024-05-13T22:48:37.944 INFO:tasks.workunit.client.1.smithi130.stdout:7/710: rmdir d7/da/d1d/d5d/da6/d9/d4a/dbd/dbc 0 2024-05-13T22:48:37.950 INFO:tasks.workunit.client.1.smithi130.stdout:9/738: creat d1/db/ff2 x:0 0 0 2024-05-13T22:48:37.952 INFO:tasks.workunit.client.1.smithi130.stdout:8/630: creat d1/dc/d5d/fee x:0 0 0 2024-05-13T22:48:37.957 INFO:tasks.workunit.client.1.smithi130.stdout:8/631: dwrite d1/d2/db/d15/d18/d42/d53/d56/f61 [0,4194304] 0 2024-05-13T22:48:37.958 INFO:tasks.workunit.client.1.smithi130.stdout:2/706: rename d4/db/dc/d18/d2a to d4/db/dbf/df9 0 2024-05-13T22:48:37.960 INFO:tasks.workunit.client.1.smithi130.stdout:6/658: truncate d4/d15/d1f/d46/d50/f51 1683743 0 2024-05-13T22:48:37.965 INFO:tasks.workunit.client.1.smithi130.stdout:7/711: dread d7/da/d1d/d5d/da6/d9/d66/f31 [0,4194304] 0 2024-05-13T22:48:37.966 INFO:tasks.workunit.client.1.smithi130.stdout:7/712: chown d7/da/d1d/d42/d43/da7 2 1 2024-05-13T22:48:37.967 INFO:tasks.workunit.client.1.smithi130.stdout:0/750: link d0/d7/dd/d17/d1f/d2c/d45/d57/f89 d0/d7/dd/d17/d1f/d2c/de3/ffd 0 2024-05-13T22:48:37.968 INFO:tasks.workunit.client.1.smithi130.stdout:8/632: write d1/d2/db/d15/d18/f4e [375096,19608] 0 2024-05-13T22:48:37.969 INFO:tasks.workunit.client.1.smithi130.stdout:7/713: read d7/da/d1d/d5d/da6/d9/d66/fa5 [478868,105111] 0 2024-05-13T22:48:37.971 INFO:tasks.workunit.client.1.smithi130.stdout:1/664: rename db/d82/fcc to db/d1b/da3/d2a/d4c/dc2/fd7 0 2024-05-13T22:48:37.972 INFO:tasks.workunit.client.1.smithi130.stdout:1/665: write db/d82/d9f/da0/fcb [647643,84799] 0 2024-05-13T22:48:37.976 INFO:tasks.workunit.client.1.smithi130.stdout:2/707: creat d4/db/dbf/df9/d30/d88/dc5/dcd/ffa x:0 0 0 2024-05-13T22:48:37.977 INFO:tasks.workunit.client.1.smithi130.stdout:6/659: mknod d4/d15/d1d/d28/d75/d7d/ce5 0 2024-05-13T22:48:37.979 INFO:tasks.workunit.client.1.smithi130.stdout:4/678: link d8/dc/d35/d33/d44/d77/c87 d8/dc/d35/d33/d44/d77/d94/db6/d81/cd3 0 2024-05-13T22:48:37.980 INFO:tasks.workunit.client.1.smithi130.stdout:3/698: dwrite d8/dc/d19/f20 [0,4194304] 0 2024-05-13T22:48:37.985 INFO:tasks.workunit.client.1.smithi130.stdout:8/633: chown d1/d2/db/d15/d18/d42/d53/d56/l8d 20 1 2024-05-13T22:48:37.987 INFO:tasks.workunit.client.1.smithi130.stdout:8/634: write d1/d2/db/d15/d16/d54/f55 [1372461,23066] 0 2024-05-13T22:48:37.989 INFO:tasks.workunit.client.1.smithi130.stdout:2/708: rmdir d4/db/dbf/d42/d4b 39 2024-05-13T22:48:37.993 INFO:tasks.workunit.client.1.smithi130.stdout:0/751: dread d0/d7/dd/d17/d1f/d2c/d5a/f66 [0,4194304] 0 2024-05-13T22:48:37.994 INFO:tasks.workunit.client.1.smithi130.stdout:6/660: rename d4/l41 to d4/d15/d1d/d60/le6 0 2024-05-13T22:48:37.995 INFO:tasks.workunit.client.1.smithi130.stdout:2/709: dread d4/da8/dc9/f6a [0,4194304] 0 2024-05-13T22:48:38.001 INFO:tasks.workunit.client.1.smithi130.stdout:7/714: mknod d7/da/d1d/d5d/da6/d39/ced 0 2024-05-13T22:48:38.001 INFO:tasks.workunit.client.1.smithi130.stdout:4/679: dread d8/dc/d29/f3a [8388608,4194304] 0 2024-05-13T22:48:38.002 INFO:tasks.workunit.client.1.smithi130.stdout:7/715: write d7/da/d1d/d5d/da6/fcd [296963,99142] 0 2024-05-13T22:48:38.005 INFO:tasks.workunit.client.1.smithi130.stdout:4/680: dread d8/dc/d29/f3a [8388608,4194304] 0 2024-05-13T22:48:38.006 INFO:tasks.workunit.client.1.smithi130.stdout:8/635: creat d1/d2/db/d15/d18/dba/fef x:0 0 0 2024-05-13T22:48:38.007 INFO:tasks.workunit.client.1.smithi130.stdout:8/636: fdatasync d1/d2/db/f41 0 2024-05-13T22:48:38.013 INFO:tasks.workunit.client.1.smithi130.stdout:5/713: sync 2024-05-13T22:48:38.014 INFO:tasks.workunit.client.1.smithi130.stdout:9/739: getdents d1/d7/d23/d28/d73 0 2024-05-13T22:48:38.014 INFO:tasks.workunit.client.1.smithi130.stdout:5/714: write d6/d7/d19/d4d/d6a/fc0 [282408,4628] 0 2024-05-13T22:48:38.021 INFO:tasks.workunit.client.1.smithi130.stdout:3/699: unlink d8/d35/l40 0 2024-05-13T22:48:38.022 INFO:tasks.workunit.client.1.smithi130.stdout:1/666: dwrite db/d1b/da3/d72/d75/f81 [0,4194304] 0 2024-05-13T22:48:38.022 INFO:tasks.workunit.client.1.smithi130.stdout:3/700: chown d8/dc/d19/d83/lce 139 1 2024-05-13T22:48:38.023 INFO:tasks.workunit.client.1.smithi130.stdout:7/716: unlink d7/da/d1d/db1/fd1 0 2024-05-13T22:48:38.026 INFO:tasks.workunit.client.1.smithi130.stdout:7/717: write d7/da/d1d/d3d/d46/f4b [144980,51490] 0 2024-05-13T22:48:38.028 INFO:tasks.workunit.client.1.smithi130.stdout:8/637: symlink d1/d2/db/d15/d16/d54/lf0 0 2024-05-13T22:48:38.031 INFO:tasks.workunit.client.1.smithi130.stdout:1/667: dwrite db/d1b/da3/d2a/d35/f3b [0,4194304] 0 2024-05-13T22:48:38.033 INFO:tasks.workunit.client.1.smithi130.stdout:4/681: dwrite d8/dc/d35/d33/d44/d77/d94/db6/d81/f9b [0,4194304] 0 2024-05-13T22:48:38.034 INFO:tasks.workunit.client.1.smithi130.stdout:6/661: rename d4/d15/l2b to d4/d15/d1d/d60/da2/da4/ddd/le7 0 2024-05-13T22:48:38.037 INFO:tasks.workunit.client.1.smithi130.stdout:9/740: chown d1/d7/d69/d79/lc3 1954799716 1 2024-05-13T22:48:38.038 INFO:tasks.workunit.client.1.smithi130.stdout:6/662: truncate d4/d15/d1f/d64/fd3 455800 0 2024-05-13T22:48:38.038 INFO:tasks.workunit.client.1.smithi130.stdout:5/715: symlink d6/dd/d18/d6e/d82/d4b/dbc/lee 0 2024-05-13T22:48:38.041 INFO:tasks.workunit.client.1.smithi130.stdout:2/710: creat d4/db/dbf/d42/d4b/ffb x:0 0 0 2024-05-13T22:48:38.051 INFO:tasks.workunit.client.1.smithi130.stdout:8/638: dread d1/d2/f8c [0,4194304] 0 2024-05-13T22:48:38.052 INFO:tasks.workunit.client.1.smithi130.stdout:1/668: mknod db/d4a/cd8 0 2024-05-13T22:48:38.052 INFO:tasks.workunit.client.1.smithi130.stdout:1/669: chown db/d82/d9f 205 1 2024-05-13T22:48:38.055 INFO:tasks.workunit.client.1.smithi130.stdout:6/663: fsync d4/d15/d1f/d46/f89 0 2024-05-13T22:48:38.056 INFO:tasks.workunit.client.1.smithi130.stdout:5/716: creat d6/dd/d18/d6e/dc7/fef x:0 0 0 2024-05-13T22:48:38.057 INFO:tasks.workunit.client.1.smithi130.stdout:5/717: chown d6/dd/lab 21238106 1 2024-05-13T22:48:38.058 INFO:tasks.workunit.client.1.smithi130.stdout:8/639: dwrite d1/d2/db/d15/d16/f8f [4194304,4194304] 0 2024-05-13T22:48:38.061 INFO:tasks.workunit.client.1.smithi130.stdout:2/711: rmdir d4/db/d26/d9e/de4 39 2024-05-13T22:48:38.065 INFO:tasks.workunit.client.1.smithi130.stdout:5/718: dwrite d6/d7/d19/d4d/f60 [0,4194304] 0 2024-05-13T22:48:38.067 INFO:tasks.workunit.client.1.smithi130.stdout:4/682: link d8/dc/d35/d33/d44/d77/d94/db6/f59 d8/dc/d80/db0/fd4 0 2024-05-13T22:48:38.068 INFO:tasks.workunit.client.1.smithi130.stdout:4/683: truncate d8/dc/d35/d33/d44/d77/d94/db6/dad/fb5 1041742 0 2024-05-13T22:48:38.075 INFO:tasks.workunit.client.1.smithi130.stdout:0/752: write d0/d7/dd/d17/d1f/d2c/d5a/fc7 [1634331,128275] 0 2024-05-13T22:48:38.081 INFO:tasks.workunit.client.1.smithi130.stdout:1/670: truncate db/d1b/da3/d49/f48 2602162 0 2024-05-13T22:48:38.086 INFO:tasks.workunit.client.1.smithi130.stdout:6/664: mknod d4/d15/d1d/d28/d75/da8/da0/ce8 0 2024-05-13T22:48:38.087 INFO:tasks.workunit.client.1.smithi130.stdout:8/640: rmdir d1/d2/db/d15/d18/d1a 39 2024-05-13T22:48:38.094 INFO:tasks.workunit.client.1.smithi130.stdout:5/719: write d6/d7/d19/d4d/fa3 [989352,74075] 0 2024-05-13T22:48:38.109 INFO:tasks.workunit.client.1.smithi130.stdout:1/671: write db/d1b/da3/d2a/d38/d46/f5f [627155,30300] 0 2024-05-13T22:48:38.112 INFO:tasks.workunit.client.1.smithi130.stdout:0/753: dwrite d0/d7/dd/d2e/d9e/da7/dc3/fdd [0,4194304] 0 2024-05-13T22:48:38.114 INFO:tasks.workunit.client.1.smithi130.stdout:9/741: link d1/d96/fb8 d1/d7/d23/d28/ff3 0 2024-05-13T22:48:38.117 INFO:tasks.workunit.client.1.smithi130.stdout:8/641: mknod d1/d2/db/d15/d18/dba/cf1 0 2024-05-13T22:48:38.119 INFO:tasks.workunit.client.1.smithi130.stdout:3/701: getdents d8 0 2024-05-13T22:48:38.120 INFO:tasks.workunit.client.1.smithi130.stdout:0/754: read d0/fe [8260321,128294] 0 2024-05-13T22:48:38.121 INFO:tasks.workunit.client.1.smithi130.stdout:7/718: getdents d7/da/d1d/db1/dbe/dd3 0 2024-05-13T22:48:38.123 INFO:tasks.workunit.client.1.smithi130.stdout:0/755: truncate d0/d7/dd/d2e/f8c 4893603 0 2024-05-13T22:48:38.123 INFO:tasks.workunit.client.1.smithi130.stdout:2/712: rename d4/db/dbf/d48/d4a/d76/c85 to d4/db/dbf/d42/d4b/cfc 0 2024-05-13T22:48:38.125 INFO:tasks.workunit.client.1.smithi130.stdout:5/720: creat d6/dd/d18/d6e/d82/d71/ff0 x:0 0 0 2024-05-13T22:48:38.126 INFO:tasks.workunit.client.1.smithi130.stdout:3/702: dwrite d8/dc/d19/d2c/f6b [0,4194304] 0 2024-05-13T22:48:38.136 INFO:tasks.workunit.client.1.smithi130.stdout:9/742: symlink d1/d7/dd2/lf4 0 2024-05-13T22:48:38.150 INFO:tasks.workunit.client.1.smithi130.stdout:8/642: dread d1/d2/db/d15/d18/f4b [0,4194304] 0 2024-05-13T22:48:38.151 INFO:tasks.workunit.client.1.smithi130.stdout:4/684: rename d8/dc/d29 to d8/dc/d35/d33/d44/d77/d94/db6/dd5 0 2024-05-13T22:48:38.154 INFO:tasks.workunit.client.1.smithi130.stdout:2/713: mkdir d4/db/dbf/df9/dfd 0 2024-05-13T22:48:38.154 INFO:tasks.workunit.client.1.smithi130.stdout:6/665: dread d4/d15/d1f/f20 [0,4194304] 0 2024-05-13T22:48:38.156 INFO:tasks.workunit.client.1.smithi130.stdout:2/714: chown d4/db/dbf/d48/cd2 30 1 2024-05-13T22:48:38.156 INFO:tasks.workunit.client.1.smithi130.stdout:4/685: read d8/dc/d35/d33/d44/d77/d94/db6/d81/fac [1479253,85003] 0 2024-05-13T22:48:38.157 INFO:tasks.workunit.client.1.smithi130.stdout:6/666: chown d4/d15/d1d/d28/d4e/d9e/l83 27201 1 2024-05-13T22:48:38.157 INFO:tasks.workunit.client.1.smithi130.stdout:5/721: creat d6/d7/d19/d4d/d8a/ff1 x:0 0 0 2024-05-13T22:48:38.157 INFO:tasks.workunit.client.1.smithi130.stdout:8/643: dwrite d1/d2/db/d15/d16/d54/d58/fd8 [0,4194304] 0 2024-05-13T22:48:38.159 INFO:tasks.workunit.client.1.smithi130.stdout:8/644: write d1/d2/db/f46 [4903050,116574] 0 2024-05-13T22:48:38.160 INFO:tasks.workunit.client.1.smithi130.stdout:7/719: write d7/da/d1d/d5d/da6/d9/d66/d65/f91 [562231,30209] 0 2024-05-13T22:48:38.162 INFO:tasks.workunit.client.1.smithi130.stdout:1/672: mkdir db/d82/dc3/dd9 0 2024-05-13T22:48:38.162 INFO:tasks.workunit.client.1.smithi130.stdout:7/720: dread - d7/da/fd0 zero size 2024-05-13T22:48:38.163 INFO:tasks.workunit.client.1.smithi130.stdout:9/743: creat d1/d7/d23/d28/d73/ff5 x:0 0 0 2024-05-13T22:48:38.166 INFO:tasks.workunit.client.1.smithi130.stdout:9/744: readlink d1/d7/d23/d28/l38 0 2024-05-13T22:48:38.166 INFO:tasks.workunit.client.1.smithi130.stdout:5/722: dread d6/d7/d32/fed [0,4194304] 0 2024-05-13T22:48:38.167 INFO:tasks.workunit.client.1.smithi130.stdout:0/756: symlink d0/d7/dd/d17/d1f/d2c/lfe 0 2024-05-13T22:48:38.168 INFO:tasks.workunit.client.1.smithi130.stdout:3/703: creat d8/dc/d19/d2c/d61/def/ff2 x:0 0 0 2024-05-13T22:48:38.171 INFO:tasks.workunit.client.1.smithi130.stdout:1/673: dwrite db/d1b/da3/d72/f79 [0,4194304] 0 2024-05-13T22:48:38.173 INFO:tasks.workunit.client.1.smithi130.stdout:2/715: mknod d4/db/dbf/d48/d4a/dc6/cfe 0 2024-05-13T22:48:38.174 INFO:tasks.workunit.client.1.smithi130.stdout:5/723: dwrite d6/d7/f46 [0,4194304] 0 2024-05-13T22:48:38.178 INFO:tasks.workunit.client.1.smithi130.stdout:4/686: creat d8/dc/d35/d33/d44/d77/d94/db6/dd5/fd6 x:0 0 0 2024-05-13T22:48:38.180 INFO:tasks.workunit.client.1.smithi130.stdout:4/687: read - d8/dc/d26/fc2 zero size 2024-05-13T22:48:38.182 INFO:tasks.workunit.client.1.smithi130.stdout:7/721: creat d7/da/d1d/d3d/db9/fee x:0 0 0 2024-05-13T22:48:38.183 INFO:tasks.workunit.client.1.smithi130.stdout:1/674: dread db/d1b/da3/d2a/f5a [0,4194304] 0 2024-05-13T22:48:38.184 INFO:tasks.workunit.client.1.smithi130.stdout:1/675: write db/d9c/fd3 [565025,1] 0 2024-05-13T22:48:38.186 INFO:tasks.workunit.client.1.smithi130.stdout:1/676: write db/d1b/da3/d2a/d38/d46/d6d/f7b [2742174,112959] 0 2024-05-13T22:48:38.187 INFO:tasks.workunit.client.1.smithi130.stdout:0/757: rmdir d0/d7/dd/d69/da9 39 2024-05-13T22:48:38.190 INFO:tasks.workunit.client.1.smithi130.stdout:6/667: mkdir d4/d15/d1d/de9 0 2024-05-13T22:48:38.195 INFO:tasks.workunit.client.1.smithi130.stdout:8/645: mkdir d1/d2/db/d15/d18/df2 0 2024-05-13T22:48:38.197 INFO:tasks.workunit.client.1.smithi130.stdout:8/646: write d1/d2/db/d15/d16/d54/fb8 [969093,80565] 0 2024-05-13T22:48:38.197 INFO:tasks.workunit.client.1.smithi130.stdout:7/722: unlink d7/da/d1d/d5d/da6/d9/l7f 0 2024-05-13T22:48:38.201 INFO:tasks.workunit.client.1.smithi130.stdout:0/758: fsync d0/d7/dd/d17/d1f/d2c/d45/d57/f89 0 2024-05-13T22:48:38.206 INFO:tasks.workunit.client.1.smithi130.stdout:8/647: dwrite d1/d2/db/d15/d18/dba/fca [0,4194304] 0 2024-05-13T22:48:38.206 INFO:tasks.workunit.client.1.smithi130.stdout:0/759: readlink d0/d7/dd/d17/d1f/d2c/l4d 0 2024-05-13T22:48:38.206 INFO:tasks.workunit.client.1.smithi130.stdout:2/716: fsync d4/db/dbf/d48/d4a/d76/f92 0 2024-05-13T22:48:38.206 INFO:tasks.workunit.client.1.smithi130.stdout:2/717: stat d4/db/dbf/df9/d66/fa9 0 2024-05-13T22:48:38.206 INFO:tasks.workunit.client.1.smithi130.stdout:4/688: mknod d8/dc/d26/dab/cd7 0 2024-05-13T22:48:38.208 INFO:tasks.workunit.client.1.smithi130.stdout:9/745: rename d1/d7/d23/d3d/d43/lbc to d1/d15/lf6 0 2024-05-13T22:48:38.210 INFO:tasks.workunit.client.1.smithi130.stdout:9/746: fdatasync d1/d7/d23/d28/f67 0 2024-05-13T22:48:38.212 INFO:tasks.workunit.client.1.smithi130.stdout:3/704: dwrite d8/d1a/d55/f9b [0,4194304] 0 2024-05-13T22:48:38.214 INFO:tasks.workunit.client.1.smithi130.stdout:9/747: write d1/d15/d51/f6d [40988,12672] 0 2024-05-13T22:48:38.220 INFO:tasks.workunit.client.1.smithi130.stdout:2/718: dwrite d4/db/dbf/df9/d30/fb3 [0,4194304] 0 2024-05-13T22:48:38.220 INFO:tasks.workunit.client.1.smithi130.stdout:8/648: mkdir d1/d9b/df3 0 2024-05-13T22:48:38.220 INFO:tasks.workunit.client.1.smithi130.stdout:5/724: dwrite d6/dd/d18/f2b [0,4194304] 0 2024-05-13T22:48:38.221 INFO:tasks.workunit.client.1.smithi130.stdout:9/748: chown d1/d15/d51/d85/fb0 244106 1 2024-05-13T22:48:38.221 INFO:tasks.workunit.client.1.smithi130.stdout:4/689: creat d8/dc/d80/db0/fd8 x:0 0 0 2024-05-13T22:48:38.225 INFO:tasks.workunit.client.1.smithi130.stdout:5/725: chown f4 4974856 1 2024-05-13T22:48:38.227 INFO:tasks.workunit.client.1.smithi130.stdout:9/749: chown d1/d7/lf0 3105729 1 2024-05-13T22:48:38.227 INFO:tasks.workunit.client.1.smithi130.stdout:1/677: dwrite db/d1b/da3/d72/daa/fb3 [0,4194304] 0 2024-05-13T22:48:38.229 INFO:tasks.workunit.client.1.smithi130.stdout:0/760: link d0/d7/dd/d2e/f4e d0/d7/dd/d17/d1f/d2c/d45/d57/df7/fff 0 2024-05-13T22:48:38.230 INFO:tasks.workunit.client.1.smithi130.stdout:9/750: readlink d1/d15/l24 0 2024-05-13T22:48:38.233 INFO:tasks.workunit.client.1.smithi130.stdout:7/723: creat d7/da/d1d/d5d/da6/d9/fef x:0 0 0 2024-05-13T22:48:38.233 INFO:tasks.workunit.client.1.smithi130.stdout:3/705: dread d8/dc/d15/f58 [0,4194304] 0 2024-05-13T22:48:38.234 INFO:tasks.workunit.client.1.smithi130.stdout:3/706: stat d8/dc/d19/f96 0 2024-05-13T22:48:38.236 INFO:tasks.workunit.client.1.smithi130.stdout:7/724: truncate d7/da/d1d/d5d/da6/d39/f94 2018034 0 2024-05-13T22:48:38.237 INFO:tasks.workunit.client.1.smithi130.stdout:2/719: rename d4/db/dbf/d48 to d4/d56/dff 0 2024-05-13T22:48:38.239 INFO:tasks.workunit.client.1.smithi130.stdout:4/690: readlink d8/dc/d35/d33/d44/d56/lb3 0 2024-05-13T22:48:38.241 INFO:tasks.workunit.client.1.smithi130.stdout:5/726: truncate d6/d2f/f39 7022709 0 2024-05-13T22:48:38.241 INFO:tasks.workunit.client.1.smithi130.stdout:5/727: chown d6/d1a/d72 103432478 1 2024-05-13T22:48:38.243 INFO:tasks.workunit.client.1.smithi130.stdout:9/751: rmdir d1/d7/d23 39 2024-05-13T22:48:38.244 INFO:tasks.workunit.client.1.smithi130.stdout:8/649: mkdir d1/d2/db/d15/d7e/dc2/df4 0 2024-05-13T22:48:38.250 INFO:tasks.workunit.client.1.smithi130.stdout:3/707: creat d8/dc/d19/ff3 x:0 0 0 2024-05-13T22:48:38.257 INFO:tasks.workunit.client.1.smithi130.stdout:3/708: dwrite d8/dc/d7b/da3/fbe [0,4194304] 0 2024-05-13T22:48:38.258 INFO:tasks.workunit.client.1.smithi130.stdout:5/728: creat d6/dd/d18/ff2 x:0 0 0 2024-05-13T22:48:38.261 INFO:tasks.workunit.client.1.smithi130.stdout:1/678: write db/d1b/da3/d2a/d38/d46/d6d/f73 [356400,68379] 0 2024-05-13T22:48:38.264 INFO:tasks.workunit.client.1.smithi130.stdout:3/709: dwrite d8/d1a/f89 [0,4194304] 0 2024-05-13T22:48:38.265 INFO:tasks.workunit.client.1.smithi130.stdout:3/710: chown d8/d1a/d1c/l23 1765 1 2024-05-13T22:48:38.269 INFO:tasks.workunit.client.1.smithi130.stdout:2/720: creat d4/db/dbf/df9/dfd/f100 x:0 0 0 2024-05-13T22:48:38.271 INFO:tasks.workunit.client.1.smithi130.stdout:9/752: dread d1/f1e [0,4194304] 0 2024-05-13T22:48:38.275 INFO:tasks.workunit.client.1.smithi130.stdout:1/679: rmdir db/d4a 39 2024-05-13T22:48:38.276 INFO:tasks.workunit.client.1.smithi130.stdout:8/650: mknod d1/d2/db/d15/d18/d1a/cf5 0 2024-05-13T22:48:38.281 INFO:tasks.workunit.client.1.smithi130.stdout:0/761: link d0/d7/dd/d2e/da8/dcd/df8/f40 d0/d7/f100 0 2024-05-13T22:48:38.284 INFO:tasks.workunit.client.1.smithi130.stdout:3/711: dwrite d8/f2b [4194304,4194304] 0 2024-05-13T22:48:38.284 INFO:tasks.workunit.client.1.smithi130.stdout:7/725: creat d7/da/d1d/d5d/da6/d9/d4a/dbd/ff0 x:0 0 0 2024-05-13T22:48:38.285 INFO:tasks.workunit.client.1.smithi130.stdout:4/691: truncate d8/fa5 4105232 0 2024-05-13T22:48:38.286 INFO:tasks.workunit.client.1.smithi130.stdout:7/726: fdatasync d7/da/d1d/d3d/d46/d92/faf 0 2024-05-13T22:48:38.287 INFO:tasks.workunit.client.1.smithi130.stdout:9/753: creat d1/d15/d9e/ff7 x:0 0 0 2024-05-13T22:48:38.289 INFO:tasks.workunit.client.1.smithi130.stdout:5/729: fdatasync d6/dd/d18/fc 0 2024-05-13T22:48:38.291 INFO:tasks.workunit.client.1.smithi130.stdout:1/680: mkdir db/d1b/da3/d49/dda 0 2024-05-13T22:48:38.293 INFO:tasks.workunit.client.1.smithi130.stdout:0/762: write d0/d7/f16 [664024,39307] 0 2024-05-13T22:48:38.297 INFO:tasks.workunit.client.1.smithi130.stdout:3/712: mknod d8/dc/d15/dac/cf4 0 2024-05-13T22:48:38.298 INFO:tasks.workunit.client.1.smithi130.stdout:6/668: sync 2024-05-13T22:48:38.299 INFO:tasks.workunit.client.1.smithi130.stdout:6/669: chown d4/d15/d1f/d64/l90 31806542 1 2024-05-13T22:48:38.300 INFO:tasks.workunit.client.1.smithi130.stdout:6/670: chown d4/d15/d1f/d46/d50/fa7 40624810 1 2024-05-13T22:48:38.301 INFO:tasks.workunit.client.1.smithi130.stdout:7/727: creat d7/da/d1d/d42/d43/ff1 x:0 0 0 2024-05-13T22:48:38.304 INFO:tasks.workunit.client.1.smithi130.stdout:2/721: link d4/db/dbf/d42/d4b/fdc d4/da8/dc9/f101 0 2024-05-13T22:48:38.310 INFO:tasks.workunit.client.1.smithi130.stdout:0/763: symlink d0/d7/dd/d2e/da8/dcd/l101 0 2024-05-13T22:48:38.312 INFO:tasks.workunit.client.1.smithi130.stdout:3/713: dread d8/dc/d7b/da3/daf/fd2 [0,4194304] 0 2024-05-13T22:48:38.316 INFO:tasks.workunit.client.1.smithi130.stdout:9/754: mkdir d1/d7/d23/df8 0 2024-05-13T22:48:38.317 INFO:tasks.workunit.client.1.smithi130.stdout:3/714: dwrite d8/dc/d15/fcd [0,4194304] 0 2024-05-13T22:48:38.318 INFO:tasks.workunit.client.1.smithi130.stdout:2/722: read - d4/db/d26/d9e/de4/feb zero size 2024-05-13T22:48:38.323 INFO:tasks.workunit.client.1.smithi130.stdout:1/681: rename db/d4a/f94 to db/d82/d9f/fdb 0 2024-05-13T22:48:38.323 INFO:tasks.workunit.client.1.smithi130.stdout:6/671: write d4/d15/d1f/d64/f7f [306481,78313] 0 2024-05-13T22:48:38.327 INFO:tasks.workunit.client.1.smithi130.stdout:8/651: creat d1/d2/db/d15/d18/ff6 x:0 0 0 2024-05-13T22:48:38.328 INFO:tasks.workunit.client.1.smithi130.stdout:0/764: mkdir d0/d7/dd/d17/d52/d5b/d102 0 2024-05-13T22:48:38.334 INFO:tasks.workunit.client.1.smithi130.stdout:7/728: truncate d7/f71 726034 0 2024-05-13T22:48:38.338 INFO:tasks.workunit.client.1.smithi130.stdout:2/723: symlink d4/db/dbf/df9/dfd/l102 0 2024-05-13T22:48:38.344 INFO:tasks.workunit.client.1.smithi130.stdout:8/652: symlink d1/d2/db/d15/d18/d42/d53/lf7 0 2024-05-13T22:48:38.348 INFO:tasks.workunit.client.1.smithi130.stdout:0/765: dread d0/d7/dd/d17/d1f/f7d [0,4194304] 0 2024-05-13T22:48:38.349 INFO:tasks.workunit.client.1.smithi130.stdout:4/692: link d8/dc/d35/d33/d3f/c89 d8/dc/d35/d33/dbf/cd9 0 2024-05-13T22:48:38.353 INFO:tasks.workunit.client.1.smithi130.stdout:3/715: chown d8/dc/d19/d27/f82 134821336 1 2024-05-13T22:48:38.355 INFO:tasks.workunit.client.1.smithi130.stdout:2/724: creat d4/db/dbf/df9/d30/d88/d90/f103 x:0 0 0 2024-05-13T22:48:38.355 INFO:tasks.workunit.client.1.smithi130.stdout:2/725: chown d4/db/dbf/df9/d66/l7c 2 1 2024-05-13T22:48:38.356 INFO:tasks.workunit.client.1.smithi130.stdout:2/726: read - d4/db/dbf/df9/d30/d88/d90/f103 zero size 2024-05-13T22:48:38.357 INFO:tasks.workunit.client.1.smithi130.stdout:2/727: truncate d4/db/dc/d18/fca 324165 0 2024-05-13T22:48:38.359 INFO:tasks.workunit.client.1.smithi130.stdout:8/653: write d1/dc/d5d/fa8 [1700365,22522] 0 2024-05-13T22:48:38.362 INFO:tasks.workunit.client.1.smithi130.stdout:0/766: mkdir d0/d7/dd/d69/dac/d103 0 2024-05-13T22:48:38.363 INFO:tasks.workunit.client.1.smithi130.stdout:1/682: dwrite db/f1a [0,4194304] 0 2024-05-13T22:48:38.364 INFO:tasks.workunit.client.1.smithi130.stdout:4/693: unlink d8/l9 0 2024-05-13T22:48:38.367 INFO:tasks.workunit.client.1.smithi130.stdout:7/729: mkdir d7/da/df2 0 2024-05-13T22:48:38.367 INFO:tasks.workunit.client.1.smithi130.stdout:9/755: creat d1/d15/ff9 x:0 0 0 2024-05-13T22:48:38.368 INFO:tasks.workunit.client.1.smithi130.stdout:9/756: dread - d1/d7/d69/d79/fcb zero size 2024-05-13T22:48:38.369 INFO:tasks.workunit.client.1.smithi130.stdout:9/757: write d1/d7/d23/d28/f97 [4099003,77106] 0 2024-05-13T22:48:38.373 INFO:tasks.workunit.client.1.smithi130.stdout:2/728: unlink d4/db/dbf/df9/d66/f7f 0 2024-05-13T22:48:38.378 INFO:tasks.workunit.client.1.smithi130.stdout:7/730: rmdir d7/da/d1d/d42/d43/da7 39 2024-05-13T22:48:38.379 INFO:tasks.workunit.client.1.smithi130.stdout:0/767: dwrite d0/d7/dd/d17/d1f/d2c/d5a/f66 [0,4194304] 0 2024-05-13T22:48:38.380 INFO:tasks.workunit.client.1.smithi130.stdout:7/731: write d7/da/d1d/d5d/da6/d9/d66/f81 [3671706,64785] 0 2024-05-13T22:48:38.380 INFO:tasks.workunit.client.1.smithi130.stdout:0/768: chown d0/d7/dd/d17/d1f/de7 494267515 1 2024-05-13T22:48:38.382 INFO:tasks.workunit.client.1.smithi130.stdout:6/672: link d4/d15/d1d/d28/d4e/d9e/cd8 d4/d15/d1f/d64/cea 0 2024-05-13T22:48:38.383 INFO:tasks.workunit.client.1.smithi130.stdout:6/673: chown d4/d15/d1d/d28/d75/fcf 31622 1 2024-05-13T22:48:38.384 INFO:tasks.workunit.client.1.smithi130.stdout:6/674: dread - d4/d15/d1f/d46/d50/fb6 zero size 2024-05-13T22:48:38.388 INFO:tasks.workunit.client.1.smithi130.stdout:9/758: mkdir d1/d15/d9e/dfa 0 2024-05-13T22:48:38.389 INFO:tasks.workunit.client.1.smithi130.stdout:8/654: mkdir d1/d2/df8 0 2024-05-13T22:48:38.393 INFO:tasks.workunit.client.1.smithi130.stdout:2/729: mkdir d4/db/dbf/df9/d30/d88/d99/d104 0 2024-05-13T22:48:38.394 INFO:tasks.workunit.client.1.smithi130.stdout:2/730: dread - d4/db/dbf/df9/d66/db9/fc1 zero size 2024-05-13T22:48:38.397 INFO:tasks.workunit.client.1.smithi130.stdout:1/683: creat db/d1b/da3/d2a/d35/dd4/fdc x:0 0 0 2024-05-13T22:48:38.410 INFO:tasks.workunit.client.1.smithi130.stdout:6/675: fsync d4/d15/d1f/f74 0 2024-05-13T22:48:38.410 INFO:tasks.workunit.client.1.smithi130.stdout:1/684: dread db/d1b/da3/d2a/d38/d46/d6d/f7b [0,4194304] 0 2024-05-13T22:48:38.411 INFO:tasks.workunit.client.1.smithi130.stdout:1/685: chown db/d9c/f8a 15689707 1 2024-05-13T22:48:38.412 INFO:tasks.workunit.client.1.smithi130.stdout:1/686: write db/d1b/da3/d72/daa/fb3 [2472421,78391] 0 2024-05-13T22:48:38.413 INFO:tasks.workunit.client.1.smithi130.stdout:1/687: chown db/d1b/da3/d49 80 1 2024-05-13T22:48:38.413 INFO:tasks.workunit.client.1.smithi130.stdout:1/688: readlink db/d1b/lcf 0 2024-05-13T22:48:38.415 INFO:tasks.workunit.client.1.smithi130.stdout:8/655: stat d1/d2/db/d15/d18/d42/d53/c69 0 2024-05-13T22:48:38.417 INFO:tasks.workunit.client.1.smithi130.stdout:1/689: dread db/d1b/da3/d2a/f5a [0,4194304] 0 2024-05-13T22:48:38.427 INFO:tasks.workunit.client.1.smithi130.stdout:7/732: rename d7/da/d1d/d5d/da6/d7e/deb to d7/da/d1d/d5d/da6/d39/da9/df3 0 2024-05-13T22:48:38.427 INFO:tasks.workunit.client.1.smithi130.stdout:5/730: sync 2024-05-13T22:48:38.428 INFO:tasks.workunit.client.1.smithi130.stdout:5/731: readlink d6/d7/l20 0 2024-05-13T22:48:38.430 INFO:tasks.workunit.client.1.smithi130.stdout:5/732: dread d6/d7/d19/d4d/d6a/fc0 [0,4194304] 0 2024-05-13T22:48:38.431 INFO:tasks.workunit.client.1.smithi130.stdout:6/676: mknod d4/d15/d1f/d3c/d70/ceb 0 2024-05-13T22:48:38.438 INFO:tasks.workunit.client.1.smithi130.stdout:8/656: chown d1/d2/db/d15/d7e/f95 7 1 2024-05-13T22:48:38.439 INFO:tasks.workunit.client.1.smithi130.stdout:8/657: chown d1/d2/db/d15/d16/d54/d58/db9 58001 1 2024-05-13T22:48:38.449 INFO:tasks.workunit.client.1.smithi130.stdout:4/694: creat d8/dc/d35/d33/dbf/fda x:0 0 0 2024-05-13T22:48:38.450 INFO:tasks.workunit.client.1.smithi130.stdout:4/695: chown d8/dc/d35/d33/d44/d61/ca4 157802 1 2024-05-13T22:48:38.460 INFO:tasks.workunit.client.1.smithi130.stdout:9/759: creat d1/d7/d23/d28/d32/d45/ffb x:0 0 0 2024-05-13T22:48:38.461 INFO:tasks.workunit.client.1.smithi130.stdout:9/760: stat d1/d15/d9e/ca0 0 2024-05-13T22:48:38.461 INFO:tasks.workunit.client.1.smithi130.stdout:1/690: fdatasync db/d1b/da3/d2a/d35/fc4 0 2024-05-13T22:48:38.465 INFO:tasks.workunit.client.1.smithi130.stdout:2/731: write d4/d56/dff/d4a/f52 [4588217,75830] 0 2024-05-13T22:48:38.465 INFO:tasks.workunit.client.1.smithi130.stdout:3/716: sync 2024-05-13T22:48:38.471 INFO:tasks.workunit.client.1.smithi130.stdout:2/732: dwrite d4/da8/fc7 [0,4194304] 0 2024-05-13T22:48:38.497 INFO:tasks.workunit.client.1.smithi130.stdout:0/769: sync 2024-05-13T22:48:38.497 INFO:tasks.workunit.client.1.smithi130.stdout:0/770: rename d0 to d0/d7/dd/d17/d52/d5b/d104 22 2024-05-13T22:48:38.524 INFO:tasks.workunit.client.1.smithi130.stdout:7/733: truncate d7/f71 1739608 0 2024-05-13T22:48:38.525 INFO:tasks.workunit.client.1.smithi130.stdout:7/734: readlink d7/da/d1d/d5d/l5f 0 2024-05-13T22:48:38.528 INFO:tasks.workunit.client.1.smithi130.stdout:7/735: dread d7/da/d1d/d5d/da6/d9/d66/f31 [0,4194304] 0 2024-05-13T22:48:38.528 INFO:tasks.workunit.client.1.smithi130.stdout:4/696: unlink d8/dc/d26/f30 0 2024-05-13T22:48:38.540 INFO:tasks.workunit.client.1.smithi130.stdout:9/761: creat d1/d7/d23/d3d/d43/ffc x:0 0 0 2024-05-13T22:48:38.541 INFO:tasks.workunit.client.1.smithi130.stdout:9/762: dread d1/f9d [0,4194304] 0 2024-05-13T22:48:38.541 INFO:tasks.workunit.client.1.smithi130.stdout:8/658: link d1/d2/db/d15/d18/ff6 d1/d2/db/d15/d16/d54/d58/db9/ff9 0 2024-05-13T22:48:38.546 INFO:tasks.workunit.client.1.smithi130.stdout:7/736: readlink d7/da/d1d/d42/l53 0 2024-05-13T22:48:38.549 INFO:tasks.workunit.client.1.smithi130.stdout:2/733: dread d4/fa [0,4194304] 0 2024-05-13T22:48:38.550 INFO:tasks.workunit.client.1.smithi130.stdout:2/734: fsync d4/db/dc/f20 0 2024-05-13T22:48:38.551 INFO:tasks.workunit.client.1.smithi130.stdout:2/735: write d4/db/dbf/df9/d30/fd8 [2087535,17179] 0 2024-05-13T22:48:38.556 INFO:tasks.workunit.client.1.smithi130.stdout:2/736: dwrite d4/da8/dc9/fbe [0,4194304] 0 2024-05-13T22:48:38.556 INFO:tasks.workunit.client.1.smithi130.stdout:6/677: write d4/d15/d1d/fcd [609520,94124] 0 2024-05-13T22:48:38.557 INFO:tasks.workunit.client.1.smithi130.stdout:5/733: truncate d6/d7/d19/d4d/f2d 2417248 0 2024-05-13T22:48:38.559 INFO:tasks.workunit.client.1.smithi130.stdout:9/763: truncate d1/d7/d23/d28/d32/d65/fa7 2128437 0 2024-05-13T22:48:38.559 INFO:tasks.workunit.client.1.smithi130.stdout:2/737: truncate d4/db/dbf/d42/d4b/fdc 765654 0 2024-05-13T22:48:38.561 INFO:tasks.workunit.client.1.smithi130.stdout:9/764: write d1/d7/d23/d28/d2d/d2f/fdf [512367,78391] 0 2024-05-13T22:48:38.564 INFO:tasks.workunit.client.1.smithi130.stdout:4/697: fsync d8/dc/d26/f36 0 2024-05-13T22:48:38.567 INFO:tasks.workunit.client.1.smithi130.stdout:0/771: dwrite d0/d7/dd/d17/d1f/d2c/f7a [0,4194304] 0 2024-05-13T22:48:38.567 INFO:tasks.workunit.client.1.smithi130.stdout:1/691: truncate f8 15242 0 2024-05-13T22:48:38.568 INFO:tasks.workunit.client.1.smithi130.stdout:1/692: write db/d9c/fd3 [861749,104495] 0 2024-05-13T22:48:38.569 INFO:tasks.workunit.client.1.smithi130.stdout:3/717: write d8/dc/d7b/da3/fec [1359142,120398] 0 2024-05-13T22:48:38.575 INFO:tasks.workunit.client.1.smithi130.stdout:0/772: dwrite d0/d7/dd/d17/d1f/d2c/fbd [0,4194304] 0 2024-05-13T22:48:38.575 INFO:tasks.workunit.client.1.smithi130.stdout:6/678: mknod d4/d15/d1f/d3c/cec 0 2024-05-13T22:48:38.577 INFO:tasks.workunit.client.1.smithi130.stdout:6/679: chown d4/d15/d1d/d28/d75/d7d/fb0 6 1 2024-05-13T22:48:38.579 INFO:tasks.workunit.client.1.smithi130.stdout:7/737: mkdir d7/da/d1d/de6/df4 0 2024-05-13T22:48:38.589 INFO:tasks.workunit.client.1.smithi130.stdout:4/698: creat d8/dc/d35/d33/d44/d77/fdb x:0 0 0 2024-05-13T22:48:38.596 INFO:tasks.workunit.client.1.smithi130.stdout:3/718: dwrite d8/dc/d7b/da3/dd3/f8c [0,4194304] 0 2024-05-13T22:48:38.598 INFO:tasks.workunit.client.1.smithi130.stdout:7/738: stat d7/c87 0 2024-05-13T22:48:38.599 INFO:tasks.workunit.client.1.smithi130.stdout:9/765: symlink d1/d7/d23/d28/lfd 0 2024-05-13T22:48:38.600 INFO:tasks.workunit.client.1.smithi130.stdout:9/766: write d1/d7/d23/d3d/d43/f82 [537367,70121] 0 2024-05-13T22:48:38.602 INFO:tasks.workunit.client.1.smithi130.stdout:4/699: creat d8/dc/d35/d33/d44/d77/d94/db6/dd5/fdc x:0 0 0 2024-05-13T22:48:38.602 INFO:tasks.workunit.client.1.smithi130.stdout:4/700: chown d8/dc/d80/c23 9 1 2024-05-13T22:48:38.605 INFO:tasks.workunit.client.1.smithi130.stdout:8/659: rename d1/d2/db/d15/d18/d42/d53/c96 to d1/dc/cfa 0 2024-05-13T22:48:38.606 INFO:tasks.workunit.client.1.smithi130.stdout:8/660: fdatasync d1/dc/dc3/fcc 0 2024-05-13T22:48:38.608 INFO:tasks.workunit.client.1.smithi130.stdout:8/661: dread d1/d2/f8c [0,4194304] 0 2024-05-13T22:48:38.609 INFO:tasks.workunit.client.1.smithi130.stdout:0/773: link d0/d7/fb d0/d7/dd/d2e/d4b/f105 0 2024-05-13T22:48:38.614 INFO:tasks.workunit.client.1.smithi130.stdout:7/739: rmdir d7 39 2024-05-13T22:48:38.619 INFO:tasks.workunit.client.1.smithi130.stdout:5/734: getdents d6/dd/d18/d6e/d82/d4b 0 2024-05-13T22:48:38.620 INFO:tasks.workunit.client.1.smithi130.stdout:5/735: chown d6/dd/d79/fe3 182478888 1 2024-05-13T22:48:38.628 INFO:tasks.workunit.client.1.smithi130.stdout:6/680: dwrite d4/d15/f31 [4194304,4194304] 0 2024-05-13T22:48:38.628 INFO:tasks.workunit.client.1.smithi130.stdout:2/738: link d4/d56/dff/c6b d4/c105 0 2024-05-13T22:48:38.639 INFO:tasks.workunit.client.1.smithi130.stdout:4/701: truncate d8/dc/d35/d33/d44/d56/fcf 2424926 0 2024-05-13T22:48:38.644 INFO:tasks.workunit.client.1.smithi130.stdout:6/681: fsync d4/d15/d1d/d60/da2/da4/fd6 0 2024-05-13T22:48:38.645 INFO:tasks.workunit.client.1.smithi130.stdout:7/740: mkdir d7/da/d1d/db1/df5 0 2024-05-13T22:48:38.646 INFO:tasks.workunit.client.1.smithi130.stdout:8/662: dread d1/d2/db/d15/d18/f3e [0,4194304] 0 2024-05-13T22:48:38.649 INFO:tasks.workunit.client.1.smithi130.stdout:1/693: rename db/fac to db/d1b/da3/fdd 0 2024-05-13T22:48:38.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:38 smithi130 ceph-mon[36591]: pgmap v36: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 64 MiB/s rd, 208 MiB/s wr, 391 op/s 2024-05-13T22:48:38.651 INFO:tasks.workunit.client.1.smithi130.stdout:0/774: dwrite d0/d7/dd/d2e/da8/dcd/df8/f37 [0,4194304] 0 2024-05-13T22:48:38.653 INFO:tasks.workunit.client.1.smithi130.stdout:6/682: truncate d4/f2e 5127 0 2024-05-13T22:48:38.655 INFO:tasks.workunit.client.1.smithi130.stdout:7/741: fsync d7/da/d1d/d5d/da6/f78 0 2024-05-13T22:48:38.656 INFO:tasks.workunit.client.1.smithi130.stdout:2/739: dwrite d4/fd [0,4194304] 0 2024-05-13T22:48:38.659 INFO:tasks.workunit.client.1.smithi130.stdout:9/767: link d1/d7/d23/d28/d73/ff5 d1/d7/d23/d28/d73/d8d/deb/ffe 0 2024-05-13T22:48:38.660 INFO:tasks.workunit.client.1.smithi130.stdout:8/663: rmdir d1/d2/db/d15/d16/d54 39 2024-05-13T22:48:38.664 INFO:tasks.workunit.client.1.smithi130.stdout:7/742: dwrite d7/da/d1d/d3d/db9/fee [0,4194304] 0 2024-05-13T22:48:38.666 INFO:tasks.workunit.client.1.smithi130.stdout:7/743: write d7/da/d86/f8a [3860475,1855] 0 2024-05-13T22:48:38.666 INFO:tasks.workunit.client.1.smithi130.stdout:3/719: rename d8/d1a/d1c/faa to d8/dc/d19/d2c/ff5 0 2024-05-13T22:48:38.668 INFO:tasks.workunit.client.1.smithi130.stdout:5/736: rename d6/dd/d18/d6e to d6/dd/d18/d6e/d76/d7f/df3 22 2024-05-13T22:48:38.668 INFO:tasks.workunit.client.1.smithi130.stdout:3/720: write d8/dc/f9f [4077141,698] 0 2024-05-13T22:48:38.673 INFO:tasks.workunit.client.1.smithi130.stdout:6/683: mknod d4/d15/d1d/d28/d4e/d9e/ced 0 2024-05-13T22:48:38.675 INFO:tasks.workunit.client.1.smithi130.stdout:6/684: write d4/d15/d1d/fcd [1073453,130669] 0 2024-05-13T22:48:38.676 INFO:tasks.workunit.client.1.smithi130.stdout:6/685: read - d4/d15/d1f/d46/fd9 zero size 2024-05-13T22:48:38.677 INFO:tasks.workunit.client.1.smithi130.stdout:9/768: mknod d1/d7/d23/d3d/d43/cff 0 2024-05-13T22:48:38.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:38 smithi102 ceph-mon[28599]: pgmap v36: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 64 MiB/s rd, 208 MiB/s wr, 391 op/s 2024-05-13T22:48:38.678 INFO:tasks.workunit.client.1.smithi130.stdout:9/769: chown d1/d7/d23/d28/d32/d45/f53 1619 1 2024-05-13T22:48:38.680 INFO:tasks.workunit.client.1.smithi130.stdout:1/694: dwrite db/d1b/da3/d2a/f2f [0,4194304] 0 2024-05-13T22:48:38.681 INFO:tasks.workunit.client.1.smithi130.stdout:9/770: read d1/d7/d23/d28/d2d/d2f/f6a [170357,91890] 0 2024-05-13T22:48:38.682 INFO:tasks.workunit.client.1.smithi130.stdout:1/695: readlink db/d1b/da3/d2a/d38/d46/d6d/l99 0 2024-05-13T22:48:38.682 INFO:tasks.workunit.client.1.smithi130.stdout:1/696: read - db/d1b/da3/d49/fb8 zero size 2024-05-13T22:48:38.684 INFO:tasks.workunit.client.1.smithi130.stdout:7/744: symlink d7/da/d1d/d5d/da6/d39/da9/lf6 0 2024-05-13T22:48:38.688 INFO:tasks.workunit.client.1.smithi130.stdout:0/775: mkdir d0/d7/dd/d17/d1f/d106 0 2024-05-13T22:48:38.690 INFO:tasks.workunit.client.1.smithi130.stdout:0/776: write d0/d7/dd/d17/d52/d67/d68/d96/fa2 [3651873,101966] 0 2024-05-13T22:48:38.691 INFO:tasks.workunit.client.1.smithi130.stdout:0/777: fdatasync d0/d7/dd/d17/d1f/d2c/d5a/f66 0 2024-05-13T22:48:38.695 INFO:tasks.workunit.client.1.smithi130.stdout:1/697: dread db/d1b/da3/f7a [0,4194304] 0 2024-05-13T22:48:38.698 INFO:tasks.workunit.client.1.smithi130.stdout:6/686: fdatasync d4/d15/d1f/d46/d50/fa7 0 2024-05-13T22:48:38.700 INFO:tasks.workunit.client.1.smithi130.stdout:8/664: dread - d1/d2/db/d15/d16/d54/d58/f94 zero size 2024-05-13T22:48:38.702 INFO:tasks.workunit.client.1.smithi130.stdout:4/702: getdents d8/dc/d84 0 2024-05-13T22:48:38.702 INFO:tasks.workunit.client.1.smithi130.stdout:7/745: unlink d7/c72 0 2024-05-13T22:48:38.703 INFO:tasks.workunit.client.1.smithi130.stdout:3/721: symlink d8/d1a/lf6 0 2024-05-13T22:48:38.703 INFO:tasks.workunit.client.1.smithi130.stdout:7/746: fdatasync d7/da/d1d/d5d/da6/d9/d4a/dbd/ff0 0 2024-05-13T22:48:38.704 INFO:tasks.workunit.client.1.smithi130.stdout:5/737: truncate d6/d7/f46 3381067 0 2024-05-13T22:48:38.709 INFO:tasks.workunit.client.1.smithi130.stdout:1/698: read db/d1b/da3/d2a/d35/f40 [986958,91944] 0 2024-05-13T22:48:38.709 INFO:tasks.workunit.client.1.smithi130.stdout:7/747: dwrite d7/da/d1d/d5d/da6/d7e/d8b/fe8 [0,4194304] 0 2024-05-13T22:48:38.710 INFO:tasks.workunit.client.1.smithi130.stdout:1/699: chown db/d58/fc1 0 1 2024-05-13T22:48:38.713 INFO:tasks.workunit.client.1.smithi130.stdout:6/687: fdatasync d4/d15/d1d/d28/d75/da8/da0/fc0 0 2024-05-13T22:48:38.713 INFO:tasks.workunit.client.1.smithi130.stdout:8/665: dread - d1/d2/f80 zero size 2024-05-13T22:48:38.716 INFO:tasks.workunit.client.1.smithi130.stdout:4/703: creat d8/dc/d35/d33/d44/d56/d8c/fdd x:0 0 0 2024-05-13T22:48:38.717 INFO:tasks.workunit.client.1.smithi130.stdout:3/722: write d8/d35/fb0 [2537053,2913] 0 2024-05-13T22:48:38.719 INFO:tasks.workunit.client.1.smithi130.stdout:1/700: dwrite db/d82/d9f/da0/fcb [0,4194304] 0 2024-05-13T22:48:38.721 INFO:tasks.workunit.client.1.smithi130.stdout:3/723: read d8/dc/d19/d2c/d61/d9c/f9d [3134458,95132] 0 2024-05-13T22:48:38.722 INFO:tasks.workunit.client.1.smithi130.stdout:5/738: rename d6/d7/d32/l4a to d6/d1a/d92/lf4 0 2024-05-13T22:48:38.729 INFO:tasks.workunit.client.1.smithi130.stdout:2/740: getdents d4/db/dbf/df9/dfd 0 2024-05-13T22:48:38.730 INFO:tasks.workunit.client.1.smithi130.stdout:2/741: readlink d4/db/dbf/df9/d30/d88/ld7 0 2024-05-13T22:48:38.735 INFO:tasks.workunit.client.1.smithi130.stdout:7/748: dwrite d7/da/d1d/f6b [0,4194304] 0 2024-05-13T22:48:38.736 INFO:tasks.workunit.client.1.smithi130.stdout:9/771: creat d1/d15/f100 x:0 0 0 2024-05-13T22:48:38.740 INFO:tasks.workunit.client.1.smithi130.stdout:6/688: chown d4/d15/d1f/f55 3 1 2024-05-13T22:48:38.740 INFO:tasks.workunit.client.1.smithi130.stdout:8/666: mkdir d1/d2/db/d15/d16/d54/dfb 0 2024-05-13T22:48:38.742 INFO:tasks.workunit.client.1.smithi130.stdout:7/749: dwrite d7/da/d1d/d5d/da6/d9/fef [0,4194304] 0 2024-05-13T22:48:38.748 INFO:tasks.workunit.client.1.smithi130.stdout:8/667: dwrite d1/d2/db/d15/d16/d54/d58/fd8 [0,4194304] 0 2024-05-13T22:48:38.751 INFO:tasks.workunit.client.1.smithi130.stdout:3/724: stat d8/dc/d19/d2c/d61/def/ff1 0 2024-05-13T22:48:38.754 INFO:tasks.workunit.client.1.smithi130.stdout:0/778: creat d0/f107 x:0 0 0 2024-05-13T22:48:38.757 INFO:tasks.workunit.client.1.smithi130.stdout:2/742: creat d4/db/dbf/df9/d30/d88/d99/d9a/f106 x:0 0 0 2024-05-13T22:48:38.763 INFO:tasks.workunit.client.1.smithi130.stdout:7/750: fdatasync d7/da/d1d/d5d/da6/d9/d66/fa5 0 2024-05-13T22:48:38.765 INFO:tasks.workunit.client.1.smithi130.stdout:6/689: read d4/f9 [1433872,3829] 0 2024-05-13T22:48:38.768 INFO:tasks.workunit.client.1.smithi130.stdout:5/739: symlink d6/d7/d19/d4d/d6a/ddf/lf5 0 2024-05-13T22:48:38.769 INFO:tasks.workunit.client.1.smithi130.stdout:9/772: symlink d1/l101 0 2024-05-13T22:48:38.770 INFO:tasks.workunit.client.1.smithi130.stdout:9/773: chown d1/d15/d51/d85/d92 2 1 2024-05-13T22:48:38.772 INFO:tasks.workunit.client.1.smithi130.stdout:2/743: truncate d4/db/dbf/f35 1164906 0 2024-05-13T22:48:38.772 INFO:tasks.workunit.client.1.smithi130.stdout:2/744: chown d4/f8 0 1 2024-05-13T22:48:38.782 INFO:tasks.workunit.client.1.smithi130.stdout:5/740: mkdir d6/dd/d18/d6e/dc7/df6 0 2024-05-13T22:48:38.782 INFO:tasks.workunit.client.1.smithi130.stdout:9/774: rmdir d1/d7/d23/d28/d32/d45 39 2024-05-13T22:48:38.783 INFO:tasks.workunit.client.1.smithi130.stdout:8/668: dwrite d1/d2/db/d15/d18/ff6 [0,4194304] 0 2024-05-13T22:48:38.785 INFO:tasks.workunit.client.1.smithi130.stdout:8/669: readlink d1/d2/db/d15/l86 0 2024-05-13T22:48:38.786 INFO:tasks.workunit.client.1.smithi130.stdout:7/751: link d7/da/d1d/d3d/d46/fec d7/da/d1d/d5d/da6/d5b/ff7 0 2024-05-13T22:48:38.786 INFO:tasks.workunit.client.1.smithi130.stdout:8/670: read - d1/d2/db/d15/d7e/d9f/fc9 zero size 2024-05-13T22:48:38.787 INFO:tasks.workunit.client.1.smithi130.stdout:8/671: fdatasync d1/dc/f44 0 2024-05-13T22:48:38.787 INFO:tasks.workunit.client.1.smithi130.stdout:7/752: fdatasync d7/da/d1d/d42/d43/fb6 0 2024-05-13T22:48:38.787 INFO:tasks.workunit.client.1.smithi130.stdout:1/701: link db/d1b/da3/d49/d56/d88/cae db/d1b/da3/cde 0 2024-05-13T22:48:38.788 INFO:tasks.workunit.client.1.smithi130.stdout:6/690: mknod d4/d15/d1d/d28/ddc/cee 0 2024-05-13T22:48:38.796 INFO:tasks.workunit.client.1.smithi130.stdout:5/741: mkdir d6/d7/d19/d4d/d8a/df7 0 2024-05-13T22:48:38.797 INFO:tasks.workunit.client.1.smithi130.stdout:9/775: dread d1/d7/d23/d28/d32/f62 [0,4194304] 0 2024-05-13T22:48:38.797 INFO:tasks.workunit.client.1.smithi130.stdout:2/745: symlink d4/db/d26/d9e/l107 0 2024-05-13T22:48:38.798 INFO:tasks.workunit.client.1.smithi130.stdout:2/746: stat d4/db/dbf/d42/d4b/fb5 0 2024-05-13T22:48:38.801 INFO:tasks.workunit.client.1.smithi130.stdout:7/753: symlink d7/da/d1d/d3d/d46/lf8 0 2024-05-13T22:48:38.801 INFO:tasks.workunit.client.1.smithi130.stdout:5/742: dwrite d6/dd/f37 [0,4194304] 0 2024-05-13T22:48:38.803 INFO:tasks.workunit.client.1.smithi130.stdout:1/702: mknod db/d1b/da3/d72/d75/cdf 0 2024-05-13T22:48:38.804 INFO:tasks.workunit.client.1.smithi130.stdout:7/754: write d7/da/d1d/d5d/da6/d9/d4a/dbd/d40/f77 [2851368,20393] 0 2024-05-13T22:48:38.804 INFO:tasks.workunit.client.1.smithi130.stdout:6/691: mkdir d4/d15/d1f/d3c/d70/def 0 2024-05-13T22:48:38.806 INFO:tasks.workunit.client.1.smithi130.stdout:7/755: fsync d7/da/d1d/d5d/da6/d39/f7b 0 2024-05-13T22:48:38.807 INFO:tasks.workunit.client.1.smithi130.stdout:3/725: link d8/d1a/d1c/l51 d8/dc/d19/d83/dba/lf7 0 2024-05-13T22:48:38.809 INFO:tasks.workunit.client.1.smithi130.stdout:0/779: getdents d0/d7/dd/d17/d1f 0 2024-05-13T22:48:38.817 INFO:tasks.workunit.client.1.smithi130.stdout:8/672: mkdir d1/d2/db/d15/d18/dfc 0 2024-05-13T22:48:38.819 INFO:tasks.workunit.client.1.smithi130.stdout:5/743: unlink d6/dd/d18/f36 0 2024-05-13T22:48:38.820 INFO:tasks.workunit.client.1.smithi130.stdout:2/747: dwrite d4/db/dc/d18/f98 [0,4194304] 0 2024-05-13T22:48:38.825 INFO:tasks.workunit.client.1.smithi130.stdout:6/692: mknod d4/d15/d1d/d60/da2/da4/dac/cf0 0 2024-05-13T22:48:38.825 INFO:tasks.workunit.client.1.smithi130.stdout:5/744: dwrite d6/d7/f3b [4194304,4194304] 0 2024-05-13T22:48:38.829 INFO:tasks.workunit.client.1.smithi130.stdout:7/756: rename d7/da/d1d/d42/d43 to d7/da/d1d/d3d/d46/d92/df9 0 2024-05-13T22:48:38.830 INFO:tasks.workunit.client.1.smithi130.stdout:7/757: fdatasync d7/da/d1d/d3d/db9/fee 0 2024-05-13T22:48:38.831 INFO:tasks.workunit.client.1.smithi130.stdout:3/726: mkdir d8/dc/d7b/da3/daf/df8 0 2024-05-13T22:48:38.834 INFO:tasks.workunit.client.1.smithi130.stdout:0/780: creat d0/d7/dd/d2e/d9e/da7/dc3/ddf/f108 x:0 0 0 2024-05-13T22:48:38.836 INFO:tasks.workunit.client.1.smithi130.stdout:0/781: fsync d0/d7/dd/d17/d52/d5b/d5f/fc9 0 2024-05-13T22:48:38.836 INFO:tasks.workunit.client.1.smithi130.stdout:4/704: sync 2024-05-13T22:48:38.840 INFO:tasks.workunit.client.1.smithi130.stdout:9/776: write d1/d7/d23/d3d/f40 [2573105,58644] 0 2024-05-13T22:48:38.841 INFO:tasks.workunit.client.1.smithi130.stdout:0/782: dwrite d0/d7/fca [0,4194304] 0 2024-05-13T22:48:38.842 INFO:tasks.workunit.client.1.smithi130.stdout:9/777: write d1/db/ff2 [458413,9355] 0 2024-05-13T22:48:38.843 INFO:tasks.workunit.client.1.smithi130.stdout:0/783: fsync d0/d7/dd/f41 0 2024-05-13T22:48:38.847 INFO:tasks.workunit.client.1.smithi130.stdout:1/703: write db/d1b/da3/d2a/f4f [828800,110562] 0 2024-05-13T22:48:38.857 INFO:tasks.workunit.client.1.smithi130.stdout:2/748: mkdir d4/da8/dc9/dbb/d108 0 2024-05-13T22:48:38.859 INFO:tasks.workunit.client.1.smithi130.stdout:6/693: rename d4/d16/c6c to d4/d15/d1d/d60/da2/da4/cf1 0 2024-05-13T22:48:38.867 INFO:tasks.workunit.client.1.smithi130.stdout:3/727: symlink d8/dc/d19/d83/dba/dc4/lf9 0 2024-05-13T22:48:38.867 INFO:tasks.workunit.client.1.smithi130.stdout:3/728: fdatasync d8/d1a/f89 0 2024-05-13T22:48:38.868 INFO:tasks.workunit.client.1.smithi130.stdout:3/729: fdatasync d8/f2b 0 2024-05-13T22:48:38.876 INFO:tasks.workunit.client.1.smithi130.stdout:9/778: write d1/d7/d23/d28/d32/f62 [7624693,21834] 0 2024-05-13T22:48:38.877 INFO:tasks.workunit.client.1.smithi130.stdout:0/784: fsync d0/d7/dd/d17/d52/d67/d68/d96/f84 0 2024-05-13T22:48:38.880 INFO:tasks.workunit.client.1.smithi130.stdout:7/758: dwrite d7/da/d1d/d5d/da6/d7e/fdc [0,4194304] 0 2024-05-13T22:48:38.881 INFO:tasks.workunit.client.1.smithi130.stdout:1/704: symlink db/d1b/da3/d49/d56/d88/le0 0 2024-05-13T22:48:38.888 INFO:tasks.workunit.client.1.smithi130.stdout:8/673: link d1/d2/db/d15/d18/dba/fca d1/d2/db/d15/d16/d54/dfb/ffd 0 2024-05-13T22:48:38.901 INFO:tasks.workunit.client.1.smithi130.stdout:4/705: link d8/dc/d35/d33/d44/d77/d94/db6/dd5/fd6 d8/dc/d35/d33/d44/d56/d8c/fde 0 2024-05-13T22:48:38.908 INFO:tasks.workunit.client.1.smithi130.stdout:8/674: dread d1/f33 [0,4194304] 0 2024-05-13T22:48:38.908 INFO:tasks.workunit.client.1.smithi130.stdout:8/675: fsync d1/d2/db/d15/d18/dba/fef 0 2024-05-13T22:48:38.910 INFO:tasks.workunit.client.1.smithi130.stdout:8/676: read - d1/d2/db/d15/d16/d54/d58/fdb zero size 2024-05-13T22:48:38.912 INFO:tasks.workunit.client.1.smithi130.stdout:6/694: dread d4/d16/f2a [0,4194304] 0 2024-05-13T22:48:38.912 INFO:tasks.workunit.client.1.smithi130.stdout:4/706: mknod d8/dc/d35/d33/d44/db8/cdf 0 2024-05-13T22:48:38.918 INFO:tasks.workunit.client.1.smithi130.stdout:5/745: getdents d6/dd/d18/d6e 0 2024-05-13T22:48:38.921 INFO:tasks.workunit.client.1.smithi130.stdout:3/730: rename d8/dc/d19 to d8/d1a/dfa 0 2024-05-13T22:48:38.922 INFO:tasks.workunit.client.1.smithi130.stdout:8/677: mknod d1/d2/db/d15/d16/d54/cfe 0 2024-05-13T22:48:38.926 INFO:tasks.workunit.client.1.smithi130.stdout:9/779: creat d1/d7/d23/d28/d73/de6/f102 x:0 0 0 2024-05-13T22:48:38.929 INFO:tasks.workunit.client.1.smithi130.stdout:0/785: creat d0/f109 x:0 0 0 2024-05-13T22:48:38.930 INFO:tasks.workunit.client.1.smithi130.stdout:2/749: getdents d4/da8 0 2024-05-13T22:48:38.930 INFO:tasks.workunit.client.1.smithi130.stdout:0/786: chown d0/d7/dd/d17/d52 995948 1 2024-05-13T22:48:38.931 INFO:tasks.workunit.client.1.smithi130.stdout:0/787: write d0/d7/dd/d2e/fbe [67999,81106] 0 2024-05-13T22:48:38.933 INFO:tasks.workunit.client.1.smithi130.stdout:6/695: dwrite d4/fb1 [0,4194304] 0 2024-05-13T22:48:38.933 INFO:tasks.workunit.client.1.smithi130.stdout:1/705: rename db/d1b/da3/l43 to db/d1b/da3/d2a/d38/d46/le1 0 2024-05-13T22:48:38.951 INFO:tasks.workunit.client.1.smithi130.stdout:1/706: read - db/d1b/da3/d49/fb8 zero size 2024-05-13T22:48:38.951 INFO:tasks.workunit.client.1.smithi130.stdout:6/696: write d4/d15/d1d/d28/f30 [5011786,22074] 0 2024-05-13T22:48:38.952 INFO:tasks.workunit.client.1.smithi130.stdout:3/731: write d8/d1a/dfa/d2c/f46 [4995767,72991] 0 2024-05-13T22:48:38.952 INFO:tasks.workunit.client.1.smithi130.stdout:0/788: unlink d0/d7/dd/d17/d1f/d2c/fd3 0 2024-05-13T22:48:38.954 INFO:tasks.workunit.client.1.smithi130.stdout:6/697: chown d4/d15/d1d/fcd 106 1 2024-05-13T22:48:38.955 INFO:tasks.workunit.client.1.smithi130.stdout:6/698: chown d4/d15/d1d/d28/d75/da8/f94 3 1 2024-05-13T22:48:38.957 INFO:tasks.workunit.client.1.smithi130.stdout:6/699: readlink d4/d15/d1d/d28/d4e/d9e/l83 0 2024-05-13T22:48:38.959 INFO:tasks.workunit.client.1.smithi130.stdout:1/707: dwrite db/d1b/da3/d2a/d38/fab [0,4194304] 0 2024-05-13T22:48:38.960 INFO:tasks.workunit.client.1.smithi130.stdout:8/678: mkdir d1/d9b/df3/dff 0 2024-05-13T22:48:38.962 INFO:tasks.workunit.client.1.smithi130.stdout:6/700: dread d4/d15/d1f/f20 [4194304,4194304] 0 2024-05-13T22:48:38.963 INFO:tasks.workunit.client.1.smithi130.stdout:4/707: link d8/dc/d80/c32 d8/dc/d26/ce0 0 2024-05-13T22:48:38.966 INFO:tasks.workunit.client.1.smithi130.stdout:9/780: mknod d1/d7/c103 0 2024-05-13T22:48:38.966 INFO:tasks.workunit.client.1.smithi130.stdout:9/781: chown d1/d7/d23 1 1 2024-05-13T22:48:38.967 INFO:tasks.workunit.client.1.smithi130.stdout:9/782: write d1/d7/d23/d28/d2d/d2f/fdf [1084690,83553] 0 2024-05-13T22:48:38.968 INFO:tasks.workunit.client.1.smithi130.stdout:9/783: readlink d1/d7/d23/d28/d32/lb3 0 2024-05-13T22:48:38.982 INFO:tasks.workunit.client.1.smithi130.stdout:1/708: creat db/d82/dc3/fe2 x:0 0 0 2024-05-13T22:48:38.984 INFO:tasks.workunit.client.1.smithi130.stdout:8/679: creat d1/d2/db/d15/d18/d42/d53/f100 x:0 0 0 2024-05-13T22:48:38.987 INFO:tasks.workunit.client.1.smithi130.stdout:6/701: rmdir d4/d15/d1d/daa 39 2024-05-13T22:48:38.989 INFO:tasks.workunit.client.1.smithi130.stdout:4/708: creat d8/dc/d84/fe1 x:0 0 0 2024-05-13T22:48:38.989 INFO:tasks.workunit.client.1.smithi130.stdout:5/746: dread d6/d7/f43 [0,4194304] 0 2024-05-13T22:48:38.989 INFO:tasks.workunit.client.1.smithi130.stdout:5/747: readlink d6/d7/d19/dc1/le6 0 2024-05-13T22:48:38.990 INFO:tasks.workunit.client.1.smithi130.stdout:4/709: write d8/dc/d35/d33/d44/fa3 [793458,30835] 0 2024-05-13T22:48:38.991 INFO:tasks.workunit.client.1.smithi130.stdout:7/759: sync 2024-05-13T22:48:38.992 INFO:tasks.workunit.client.1.smithi130.stdout:7/760: truncate d7/da/d86/fc3 346445 0 2024-05-13T22:48:38.995 INFO:tasks.workunit.client.1.smithi130.stdout:9/784: unlink d1/d7/d23/d28/f67 0 2024-05-13T22:48:39.007 INFO:tasks.workunit.client.1.smithi130.stdout:6/702: truncate d4/d15/d1f/f74 2778164 0 2024-05-13T22:48:39.011 INFO:tasks.workunit.client.1.smithi130.stdout:0/789: write d0/fad [674956,119317] 0 2024-05-13T22:48:39.012 INFO:tasks.workunit.client.1.smithi130.stdout:6/703: dwrite d4/d15/d1f/d46/fb2 [4194304,4194304] 0 2024-05-13T22:48:39.017 INFO:tasks.workunit.client.1.smithi130.stdout:1/709: symlink db/d82/le3 0 2024-05-13T22:48:39.018 INFO:tasks.workunit.client.1.smithi130.stdout:1/710: read db/d1b/da3/d2a/d38/fc5 [396960,64607] 0 2024-05-13T22:48:39.023 INFO:tasks.workunit.client.1.smithi130.stdout:2/750: sync 2024-05-13T22:48:39.023 INFO:tasks.workunit.client.1.smithi130.stdout:2/751: chown d4/da8/dc9/f6a 6411 1 2024-05-13T22:48:39.030 INFO:tasks.workunit.client.1.smithi130.stdout:4/710: write d8/dc/d35/d33/d44/d56/d8c/fde [960144,116100] 0 2024-05-13T22:48:39.035 INFO:tasks.workunit.client.1.smithi130.stdout:6/704: fsync d4/f49 0 2024-05-13T22:48:39.037 INFO:tasks.workunit.client.1.smithi130.stdout:6/705: dread d4/d15/d1f/d64/fd3 [0,4194304] 0 2024-05-13T22:48:39.041 INFO:tasks.workunit.client.1.smithi130.stdout:7/761: mknod d7/cfa 0 2024-05-13T22:48:39.044 INFO:tasks.workunit.client.1.smithi130.stdout:3/732: rename d8/d1a/dfa/fe1 to d8/d1a/ffb 0 2024-05-13T22:48:39.044 INFO:tasks.workunit.client.1.smithi130.stdout:3/733: chown c7 433 1 2024-05-13T22:48:39.045 INFO:tasks.workunit.client.1.smithi130.stdout:3/734: stat d8/dc/f3e 0 2024-05-13T22:48:39.045 INFO:tasks.workunit.client.1.smithi130.stdout:8/680: link d1/d2/db/d15/d18/l2e d1/d2/db/d15/d18/dfc/l101 0 2024-05-13T22:48:39.051 INFO:tasks.workunit.client.1.smithi130.stdout:5/748: link d6/d7/d19/f2c d6/d2f/ff8 0 2024-05-13T22:48:39.052 INFO:tasks.workunit.client.1.smithi130.stdout:8/681: dread d1/d2/db/d15/d16/f1c [0,4194304] 0 2024-05-13T22:48:39.052 INFO:tasks.workunit.client.1.smithi130.stdout:4/711: stat d8/dc/d35/d33/d44/d77/d94/db6/d81/cd3 0 2024-05-13T22:48:39.053 INFO:tasks.workunit.client.1.smithi130.stdout:4/712: truncate d8/dc/d80/db0/fb4 664626 0 2024-05-13T22:48:39.054 INFO:tasks.workunit.client.1.smithi130.stdout:4/713: dread - d8/dc/f9a zero size 2024-05-13T22:48:39.057 INFO:tasks.workunit.client.1.smithi130.stdout:4/714: dread d8/dc/d35/d33/f4f [0,4194304] 0 2024-05-13T22:48:39.059 INFO:tasks.workunit.client.1.smithi130.stdout:2/752: dwrite d4/db/dbf/df9/f67 [0,4194304] 0 2024-05-13T22:48:39.059 INFO:tasks.workunit.client.1.smithi130.stdout:0/790: creat d0/d7/dd/d2e/d9e/da7/de1/f10a x:0 0 0 2024-05-13T22:48:39.063 INFO:tasks.workunit.client.1.smithi130.stdout:9/785: creat d1/d7/d23/d28/d73/f104 x:0 0 0 2024-05-13T22:48:39.064 INFO:tasks.workunit.client.1.smithi130.stdout:7/762: mkdir d7/da/d1d/d5d/da6/d39/da9/dfb 0 2024-05-13T22:48:39.071 INFO:tasks.workunit.client.1.smithi130.stdout:5/749: write d6/d2f/ff8 [313254,115342] 0 2024-05-13T22:48:39.083 INFO:tasks.workunit.client.1.smithi130.stdout:2/753: truncate d4/f9 4986393 0 2024-05-13T22:48:39.086 INFO:tasks.workunit.client.1.smithi130.stdout:7/763: mknod d7/da/d1d/db1/dbe/dd3/cfc 0 2024-05-13T22:48:39.090 INFO:tasks.workunit.client.1.smithi130.stdout:1/711: rename db/l29 to db/d1b/da3/d72/le4 0 2024-05-13T22:48:39.096 INFO:tasks.workunit.client.1.smithi130.stdout:3/735: dwrite d8/d1a/dfa/f7f [4194304,4194304] 0 2024-05-13T22:48:39.096 INFO:tasks.workunit.client.1.smithi130.stdout:5/750: creat d6/d2f/ff9 x:0 0 0 2024-05-13T22:48:39.100 INFO:tasks.workunit.client.1.smithi130.stdout:8/682: truncate d1/d2/db/d15/d18/d42/d53/d56/f61 12938 0 2024-05-13T22:48:39.105 INFO:tasks.workunit.client.1.smithi130.stdout:6/706: rename d4/d15/d1d/d28/d75/d7d/ce5 to d4/d15/d1d/d60/da2/da4/cf2 0 2024-05-13T22:48:39.109 INFO:tasks.workunit.client.1.smithi130.stdout:1/712: mknod db/d82/d9f/ce5 0 2024-05-13T22:48:39.110 INFO:tasks.workunit.client.1.smithi130.stdout:1/713: dread - db/d4a/f95 zero size 2024-05-13T22:48:39.111 INFO:tasks.workunit.client.1.smithi130.stdout:6/707: dwrite d4/d15/fde [0,4194304] 0 2024-05-13T22:48:39.113 INFO:tasks.workunit.client.1.smithi130.stdout:9/786: truncate d1/f1e 6571318 0 2024-05-13T22:48:39.120 INFO:tasks.workunit.client.1.smithi130.stdout:4/715: creat d8/dc/d35/d33/d3f/fe2 x:0 0 0 2024-05-13T22:48:39.126 INFO:tasks.workunit.client.1.smithi130.stdout:8/683: symlink d1/d2/db/d15/d16/d54/d58/db9/l102 0 2024-05-13T22:48:39.127 INFO:tasks.workunit.client.1.smithi130.stdout:7/764: rmdir d7/da/d1d/d3d/d46/d92/df9/da7 39 2024-05-13T22:48:39.128 INFO:tasks.workunit.client.1.smithi130.stdout:7/765: write d7/da/d1d/d3d/d46/d92/faf [3130823,95477] 0 2024-05-13T22:48:39.133 INFO:tasks.workunit.client.1.smithi130.stdout:9/787: symlink d1/db/l105 0 2024-05-13T22:48:39.133 INFO:tasks.workunit.client.1.smithi130.stdout:0/791: sync 2024-05-13T22:48:39.137 INFO:tasks.workunit.client.1.smithi130.stdout:4/716: readlink d8/l2e 0 2024-05-13T22:48:39.139 INFO:tasks.workunit.client.1.smithi130.stdout:2/754: creat d4/d56/dff/d4a/f109 x:0 0 0 2024-05-13T22:48:39.140 INFO:tasks.workunit.client.1.smithi130.stdout:2/755: chown d4/db/dbf/df9/d30/d88/dc5/dcd 26207587 1 2024-05-13T22:48:39.142 INFO:tasks.workunit.client.1.smithi130.stdout:5/751: dread d6/dd/d18/d6e/f8b [0,4194304] 0 2024-05-13T22:48:39.145 INFO:tasks.workunit.client.1.smithi130.stdout:7/766: creat d7/da/d1d/d5d/da6/d9/d66/ffd x:0 0 0 2024-05-13T22:48:39.147 INFO:tasks.workunit.client.1.smithi130.stdout:6/708: mknod d4/d15/cf3 0 2024-05-13T22:48:39.148 INFO:tasks.workunit.client.1.smithi130.stdout:6/709: read d4/fb1 [2290623,92993] 0 2024-05-13T22:48:39.149 INFO:tasks.workunit.client.1.smithi130.stdout:0/792: fsync d0/d7/dd/d69/da9/ff9 0 2024-05-13T22:48:39.151 INFO:tasks.workunit.client.1.smithi130.stdout:7/767: dwrite d7/da/d1d/d5d/da6/d9/d66/d65/faa [0,4194304] 0 2024-05-13T22:48:39.154 INFO:tasks.workunit.client.1.smithi130.stdout:3/736: creat d8/d35/ffc x:0 0 0 2024-05-13T22:48:39.156 INFO:tasks.workunit.client.1.smithi130.stdout:2/756: symlink d4/db/dbf/df9/d30/d88/dc5/dcd/l10a 0 2024-05-13T22:48:39.159 INFO:tasks.workunit.client.1.smithi130.stdout:5/752: write f4 [936128,114249] 0 2024-05-13T22:48:39.160 INFO:tasks.workunit.client.1.smithi130.stdout:2/757: stat d4/db/dc/f3b 0 2024-05-13T22:48:39.161 INFO:tasks.workunit.client.1.smithi130.stdout:6/710: dwrite d4/d15/d1f/d64/fdf [0,4194304] 0 2024-05-13T22:48:39.162 INFO:tasks.workunit.client.1.smithi130.stdout:2/758: chown d4/db/dbf/d42 417362 1 2024-05-13T22:48:39.163 INFO:tasks.workunit.client.1.smithi130.stdout:8/684: dwrite d1/d2/db/d15/d18/dba/fca [0,4194304] 0 2024-05-13T22:48:39.171 INFO:tasks.workunit.client.1.smithi130.stdout:3/737: dwrite d8/dc/d7b/da3/daf/fb8 [0,4194304] 0 2024-05-13T22:48:39.175 INFO:tasks.workunit.client.1.smithi130.stdout:8/685: dwrite d1/d2/fc5 [8388608,4194304] 0 2024-05-13T22:48:39.178 INFO:tasks.workunit.client.1.smithi130.stdout:1/714: getdents db/d58 0 2024-05-13T22:48:39.186 INFO:tasks.workunit.client.1.smithi130.stdout:2/759: read d4/f9 [2541877,79626] 0 2024-05-13T22:48:39.187 INFO:tasks.workunit.client.1.smithi130.stdout:1/715: dwrite db/d1b/da3/d2a/d35/f3b [8388608,4194304] 0 2024-05-13T22:48:39.187 INFO:tasks.workunit.client.1.smithi130.stdout:9/788: link d1/d15/f86 d1/d7/d23/d28/d73/db2/f106 0 2024-05-13T22:48:39.189 INFO:tasks.workunit.client.1.smithi130.stdout:1/716: write db/d1b/da3/f53 [1205573,44020] 0 2024-05-13T22:48:39.190 INFO:tasks.workunit.client.1.smithi130.stdout:4/717: creat d8/dc/d80/fe3 x:0 0 0 2024-05-13T22:48:39.192 INFO:tasks.workunit.client.1.smithi130.stdout:7/768: truncate d7/da/d1d/d5d/da6/f4c 824191 0 2024-05-13T22:48:39.195 INFO:tasks.workunit.client.1.smithi130.stdout:8/686: rename d1/d2/db/d15/d7e/dc2/leb to d1/d2/db/d15/d16/d54/d58/dab/l103 0 2024-05-13T22:48:39.197 INFO:tasks.workunit.client.1.smithi130.stdout:7/769: chown d7/da/d1d/db1 117892 1 2024-05-13T22:48:39.199 INFO:tasks.workunit.client.1.smithi130.stdout:2/760: dread d4/fa [0,4194304] 0 2024-05-13T22:48:39.203 INFO:tasks.workunit.client.1.smithi130.stdout:9/789: dwrite d1/d7/d23/fc2 [0,4194304] 0 2024-05-13T22:48:39.203 INFO:tasks.workunit.client.1.smithi130.stdout:5/753: getdents d6/d7/d19/d4d 0 2024-05-13T22:48:39.204 INFO:tasks.workunit.client.1.smithi130.stdout:0/793: dread d0/d7/dd/d17/d52/f9a [0,4194304] 0 2024-05-13T22:48:39.204 INFO:tasks.workunit.client.1.smithi130.stdout:9/790: write d1/f4f [2745082,85278] 0 2024-05-13T22:48:39.206 INFO:tasks.workunit.client.1.smithi130.stdout:7/770: creat d7/da/d1d/d5d/da6/d9/d66/ffe x:0 0 0 2024-05-13T22:48:39.207 INFO:tasks.workunit.client.1.smithi130.stdout:1/717: creat db/d1b/fe6 x:0 0 0 2024-05-13T22:48:39.207 INFO:tasks.workunit.client.1.smithi130.stdout:0/794: read - d0/d7/dd/d2e/d9e/da7/dc3/ddf/fee zero size 2024-05-13T22:48:39.210 INFO:tasks.workunit.client.1.smithi130.stdout:3/738: dread d8/d1a/d1c/f94 [0,4194304] 0 2024-05-13T22:48:39.217 INFO:tasks.workunit.client.1.smithi130.stdout:5/754: rename d6/dd/d18/d6e/d82/d71/lb1 to d6/d7/d19/d4d/d53/lfa 0 2024-05-13T22:48:39.221 INFO:tasks.workunit.client.1.smithi130.stdout:0/795: write d0/d7/dd/d2e/d4b/f65 [3786905,74295] 0 2024-05-13T22:48:39.226 INFO:tasks.workunit.client.1.smithi130.stdout:4/718: link d8/dc/c1c d8/dc/d35/d33/d44/d56/ce4 0 2024-05-13T22:48:39.226 INFO:tasks.workunit.client.1.smithi130.stdout:4/719: stat d8/dc/d35/d33/d44/d61/l7c 0 2024-05-13T22:48:39.227 INFO:tasks.workunit.client.1.smithi130.stdout:3/739: rename d8/d1a/dfa/d2c/d61 to d8/dc/d15/dfd 0 2024-05-13T22:48:39.227 INFO:tasks.workunit.client.1.smithi130.stdout:5/755: creat d6/dd/d18/d6e/d82/d71/ffb x:0 0 0 2024-05-13T22:48:39.229 INFO:tasks.workunit.client.1.smithi130.stdout:0/796: rmdir d0/d7/dd/d2e/d9e/da7/dc3/ddf 39 2024-05-13T22:48:39.234 INFO:tasks.workunit.client.1.smithi130.stdout:0/797: dwrite d0/d7/dd/d17/d52/d67/d68/d96/fa2 [0,4194304] 0 2024-05-13T22:48:39.242 INFO:tasks.workunit.client.1.smithi130.stdout:5/756: write d6/dd/d18/d6e/f8b [4198002,28573] 0 2024-05-13T22:48:39.244 INFO:tasks.workunit.client.1.smithi130.stdout:7/771: truncate d7/da/d1d/d5d/da6/d5b/fb3 3505426 0 2024-05-13T22:48:39.246 INFO:tasks.workunit.client.1.smithi130.stdout:2/761: link d4/db/l54 d4/db/dbf/d42/l10b 0 2024-05-13T22:48:39.247 INFO:tasks.workunit.client.1.smithi130.stdout:0/798: creat d0/d7/dd/d2e/d4b/f10b x:0 0 0 2024-05-13T22:48:39.248 INFO:tasks.workunit.client.1.smithi130.stdout:0/799: dread - d0/d7/dd/d2e/d9e/da7/dc3/ff6 zero size 2024-05-13T22:48:39.251 INFO:tasks.workunit.client.1.smithi130.stdout:3/740: mknod d8/d1a/dfa/cfe 0 2024-05-13T22:48:39.251 INFO:tasks.workunit.client.1.smithi130.stdout:9/791: link d1/d7/d23/d3d/cc8 d1/d7/d23/d3d/c107 0 2024-05-13T22:48:39.252 INFO:tasks.workunit.client.1.smithi130.stdout:5/757: write d6/d7/d32/f91 [625415,93045] 0 2024-05-13T22:48:39.253 INFO:tasks.workunit.client.1.smithi130.stdout:7/772: dread - d7/f83 zero size 2024-05-13T22:48:39.255 INFO:tasks.workunit.client.1.smithi130.stdout:5/758: write d6/dd/d18/d6e/f8b [1862986,98176] 0 2024-05-13T22:48:39.259 INFO:tasks.workunit.client.1.smithi130.stdout:9/792: dwrite d1/d7/d23/d3d/d43/ffc [0,4194304] 0 2024-05-13T22:48:39.261 INFO:tasks.workunit.client.1.smithi130.stdout:3/741: creat d8/dc/d7b/fff x:0 0 0 2024-05-13T22:48:39.263 INFO:tasks.workunit.client.1.smithi130.stdout:4/720: truncate d8/dc/d35/f8a 2568613 0 2024-05-13T22:48:39.266 INFO:tasks.workunit.client.1.smithi130.stdout:3/742: dwrite d8/dc/d15/f76 [0,4194304] 0 2024-05-13T22:48:39.268 INFO:tasks.workunit.client.1.smithi130.stdout:2/762: write d4/da8/dc9/f6a [645495,27063] 0 2024-05-13T22:48:39.270 INFO:tasks.workunit.client.1.smithi130.stdout:7/773: creat d7/da/d1d/d5d/da6/d9/d66/d65/fff x:0 0 0 2024-05-13T22:48:39.273 INFO:tasks.workunit.client.1.smithi130.stdout:5/759: rename d6/dd/d18/d6e/d82/d71/cae to d6/d7/d32/cfc 0 2024-05-13T22:48:39.275 INFO:tasks.workunit.client.1.smithi130.stdout:3/743: dwrite d8/d1a/d1c/f6c [4194304,4194304] 0 2024-05-13T22:48:39.279 INFO:tasks.workunit.client.1.smithi130.stdout:4/721: fdatasync d8/dc/d35/f50 0 2024-05-13T22:48:39.280 INFO:tasks.workunit.client.1.smithi130.stdout:2/763: dwrite d4/db/dbf/df9/d30/f9c [0,4194304] 0 2024-05-13T22:48:39.281 INFO:tasks.workunit.client.1.smithi130.stdout:5/760: readlink d6/lb9 0 2024-05-13T22:48:39.283 INFO:tasks.workunit.client.1.smithi130.stdout:7/774: dwrite d7/da/d1d/d5d/d70/f85 [0,4194304] 0 2024-05-13T22:48:39.286 INFO:tasks.workunit.client.1.smithi130.stdout:2/764: rmdir d4/db/d26/d9e 39 2024-05-13T22:48:39.286 INFO:tasks.workunit.client.1.smithi130.stdout:0/800: dread d0/d7/dd/d69/da9/d8d/f9d [0,4194304] 0 2024-05-13T22:48:39.286 INFO:tasks.workunit.client.1.smithi130.stdout:4/722: creat d8/dc/d35/d33/d44/d77/d94/fe5 x:0 0 0 2024-05-13T22:48:39.287 INFO:tasks.workunit.client.1.smithi130.stdout:5/761: mknod d6/dd/d18/d6e/d82/d84/cfd 0 2024-05-13T22:48:39.289 INFO:tasks.workunit.client.1.smithi130.stdout:3/744: dwrite d8/fbd [0,4194304] 0 2024-05-13T22:48:39.289 INFO:tasks.workunit.client.1.smithi130.stdout:0/801: chown d0/da0/ccb 7944 1 2024-05-13T22:48:39.292 INFO:tasks.workunit.client.1.smithi130.stdout:3/745: write d8/d1a/f89 [2878577,79078] 0 2024-05-13T22:48:39.292 INFO:tasks.workunit.client.1.smithi130.stdout:2/765: mkdir d4/d56/dff/d4a/dc6/d10c 0 2024-05-13T22:48:39.293 INFO:tasks.workunit.client.1.smithi130.stdout:4/723: read - d8/dc/d35/d33/d44/db8/fcc zero size 2024-05-13T22:48:39.294 INFO:tasks.workunit.client.1.smithi130.stdout:3/746: readlink d8/dc/d15/l38 0 2024-05-13T22:48:39.296 INFO:tasks.workunit.client.1.smithi130.stdout:3/747: dread - d8/dc/d7b/fff zero size 2024-05-13T22:48:39.296 INFO:tasks.workunit.client.1.smithi130.stdout:2/766: readlink d4/db/dbf/df9/l3e 0 2024-05-13T22:48:39.299 INFO:tasks.workunit.client.1.smithi130.stdout:5/762: rename d6/dd/d18/f2b to d6/dd/d18/d6e/d76/ffe 0 2024-05-13T22:48:39.299 INFO:tasks.workunit.client.1.smithi130.stdout:2/767: truncate d4/db/f2e 1872870 0 2024-05-13T22:48:39.301 INFO:tasks.workunit.client.1.smithi130.stdout:9/793: dwrite d1/d7/d23/d28/d2d/f34 [0,4194304] 0 2024-05-13T22:48:39.302 INFO:tasks.workunit.client.1.smithi130.stdout:0/802: dread d0/d7/dd/d2e/da8/dcd/df8/f37 [0,4194304] 0 2024-05-13T22:48:39.303 INFO:tasks.workunit.client.1.smithi130.stdout:0/803: stat d0/d7/dd/d2e/da8/dcd 0 2024-05-13T22:48:39.304 INFO:tasks.workunit.client.1.smithi130.stdout:3/748: read d8/d1a/d55/fe2 [3737352,48428] 0 2024-05-13T22:48:39.316 INFO:tasks.workunit.client.1.smithi130.stdout:9/794: rmdir d1/d7/d23/d3d 39 2024-05-13T22:48:39.328 INFO:tasks.workunit.client.1.smithi130.stdout:4/724: write d8/dc/d26/f55 [1314750,93925] 0 2024-05-13T22:48:39.331 INFO:tasks.workunit.client.1.smithi130.stdout:4/725: truncate d8/dc/d35/d33/d44/f4b 5090455 0 2024-05-13T22:48:39.333 INFO:tasks.workunit.client.1.smithi130.stdout:0/804: dread d0/d7/dd/d2e/d4b/f15 [0,4194304] 0 2024-05-13T22:48:39.335 INFO:tasks.workunit.client.1.smithi130.stdout:3/749: rename d8/dc/d7b/da3/cb5 to d8/c100 0 2024-05-13T22:48:39.335 INFO:tasks.workunit.client.1.smithi130.stdout:2/768: dread d4/db/dc/f3b [0,4194304] 0 2024-05-13T22:48:39.336 INFO:tasks.workunit.client.1.smithi130.stdout:0/805: rmdir d0/d7/dd/d17/d1f/d2c/d45 39 2024-05-13T22:48:39.336 INFO:tasks.workunit.client.1.smithi130.stdout:5/763: dread d6/d1a/d72/fd9 [0,4194304] 0 2024-05-13T22:48:39.337 INFO:tasks.workunit.client.1.smithi130.stdout:2/769: chown d4/d56/dff/d4a/dc6/f6f 0 1 2024-05-13T22:48:39.337 INFO:tasks.workunit.client.1.smithi130.stdout:0/806: write d0/d7/dd/d2e/f8c [3955870,35579] 0 2024-05-13T22:48:39.339 INFO:tasks.workunit.client.1.smithi130.stdout:9/795: mknod d1/d7/d23/d28/d73/d8d/deb/c108 0 2024-05-13T22:48:39.342 INFO:tasks.workunit.client.1.smithi130.stdout:4/726: rename d8/dc/d35/d33/d44/d61/da0/lb2 to d8/dc/d35/d33/d44/d77/d94/db6/d81/le6 0 2024-05-13T22:48:39.345 INFO:tasks.workunit.client.1.smithi130.stdout:5/764: mknod d6/d7/cff 0 2024-05-13T22:48:39.346 INFO:tasks.workunit.client.1.smithi130.stdout:2/770: unlink d4/db/dbf/df9/d30/d88/d99/lec 0 2024-05-13T22:48:39.348 INFO:tasks.workunit.client.1.smithi130.stdout:2/771: truncate d4/db/d26/f8f 673875 0 2024-05-13T22:48:39.349 INFO:tasks.workunit.client.1.smithi130.stdout:2/772: write d4/db/dc/d18/fed [894930,35304] 0 2024-05-13T22:48:39.351 INFO:tasks.workunit.client.1.smithi130.stdout:6/711: sync 2024-05-13T22:48:39.352 INFO:tasks.workunit.client.1.smithi130.stdout:3/750: read - d8/dc/d7b/fe0 zero size 2024-05-13T22:48:39.354 INFO:tasks.workunit.client.1.smithi130.stdout:0/807: dwrite d0/d7/dd/d17/d1f/d2c/d45/d57/f85 [0,4194304] 0 2024-05-13T22:48:39.355 INFO:tasks.workunit.client.1.smithi130.stdout:4/727: dread d8/dc/d84/f9c [4194304,4194304] 0 2024-05-13T22:48:39.355 INFO:tasks.workunit.client.1.smithi130.stdout:5/765: dread d6/d7/d32/f51 [0,4194304] 0 2024-05-13T22:48:39.357 INFO:tasks.workunit.client.1.smithi130.stdout:4/728: chown d8/dc/d35/d33/d3f/c89 470200540 1 2024-05-13T22:48:39.360 INFO:tasks.workunit.client.1.smithi130.stdout:9/796: write d1/d15/f78 [174043,25142] 0 2024-05-13T22:48:39.363 INFO:tasks.workunit.client.1.smithi130.stdout:5/766: creat d6/dd/d18/d6e/d82/d71/f100 x:0 0 0 2024-05-13T22:48:39.363 INFO:tasks.workunit.client.1.smithi130.stdout:5/767: chown d6/d1a/daf 204252683 1 2024-05-13T22:48:39.364 INFO:tasks.workunit.client.1.smithi130.stdout:4/729: creat d8/dc/d80/d85/fe7 x:0 0 0 2024-05-13T22:48:39.367 INFO:tasks.workunit.client.1.smithi130.stdout:9/797: symlink d1/db/l109 0 2024-05-13T22:48:39.368 INFO:tasks.workunit.client.1.smithi130.stdout:9/798: chown d1/d7/d23/d28/d73/db2/cae 643374 1 2024-05-13T22:48:39.369 INFO:tasks.workunit.client.1.smithi130.stdout:3/751: creat d8/d1a/dfa/d83/f101 x:0 0 0 2024-05-13T22:48:39.370 INFO:tasks.workunit.client.1.smithi130.stdout:0/808: link d0/d7/dd/d2e/d9e/da7/de1/f10a d0/d7/dd/d2e/d9e/f10c 0 2024-05-13T22:48:39.372 INFO:tasks.workunit.client.1.smithi130.stdout:5/768: fsync d6/d7/d19/d4d/d6a/fc0 0 2024-05-13T22:48:39.373 INFO:tasks.workunit.client.1.smithi130.stdout:6/712: dread d4/d15/d1d/daa/fd0 [0,4194304] 0 2024-05-13T22:48:39.375 INFO:tasks.workunit.client.1.smithi130.stdout:4/730: creat d8/dc/d35/d33/d44/d77/d94/db6/d81/fe8 x:0 0 0 2024-05-13T22:48:39.376 INFO:tasks.workunit.client.1.smithi130.stdout:9/799: creat d1/d7/d23/d28/d2d/d2f/f10a x:0 0 0 2024-05-13T22:48:39.377 INFO:tasks.workunit.client.1.smithi130.stdout:3/752: unlink d8/d1a/lf6 0 2024-05-13T22:48:39.377 INFO:tasks.workunit.client.1.smithi130.stdout:5/769: dwrite d6/d1a/d72/fd9 [0,4194304] 0 2024-05-13T22:48:39.379 INFO:tasks.workunit.client.1.smithi130.stdout:0/809: mknod d0/d7/dd/d17/d1f/db0/c10d 0 2024-05-13T22:48:39.381 INFO:tasks.workunit.client.1.smithi130.stdout:0/810: dread - d0/d7/dd/d17/d52/d5b/d60/fda zero size 2024-05-13T22:48:39.382 INFO:tasks.workunit.client.1.smithi130.stdout:2/773: getdents d4/db/dbf/df9/d30/d88/d90 0 2024-05-13T22:48:39.384 INFO:tasks.workunit.client.1.smithi130.stdout:8/687: sync 2024-05-13T22:48:39.384 INFO:tasks.workunit.client.1.smithi130.stdout:1/718: sync 2024-05-13T22:48:39.385 INFO:tasks.workunit.client.1.smithi130.stdout:1/719: chown db/d1b/da3/f89 1801187188 1 2024-05-13T22:48:39.385 INFO:tasks.workunit.client.1.smithi130.stdout:8/688: dread - d1/d2/db/d15/d7e/d9f/fc9 zero size 2024-05-13T22:48:39.386 INFO:tasks.workunit.client.1.smithi130.stdout:4/731: readlink d8/dc/l68 0 2024-05-13T22:48:39.390 INFO:tasks.workunit.client.1.smithi130.stdout:9/800: symlink d1/d15/d51/db4/l10b 0 2024-05-13T22:48:39.394 INFO:tasks.workunit.client.1.smithi130.stdout:8/689: dwrite d1/dc/d5d/fee [0,4194304] 0 2024-05-13T22:48:39.394 INFO:tasks.workunit.client.1.smithi130.stdout:8/690: dread - d1/d2/db/d15/d18/dba/fbe zero size 2024-05-13T22:48:39.395 INFO:tasks.workunit.client.1.smithi130.stdout:4/732: creat d8/dc/d35/d33/d44/d77/fe9 x:0 0 0 2024-05-13T22:48:39.398 INFO:tasks.workunit.client.1.smithi130.stdout:6/713: dread d4/d15/d1d/d28/f47 [0,4194304] 0 2024-05-13T22:48:39.400 INFO:tasks.workunit.client.1.smithi130.stdout:0/811: mkdir d0/d7/dd/d17/d52/d10e 0 2024-05-13T22:48:39.403 INFO:tasks.workunit.client.1.smithi130.stdout:5/770: dread d6/d1a/f2a [0,4194304] 0 2024-05-13T22:48:39.403 INFO:tasks.workunit.client.1.smithi130.stdout:1/720: mkdir db/d1b/da3/d2a/de7 0 2024-05-13T22:48:39.408 INFO:tasks.workunit.client.1.smithi130.stdout:5/771: dwrite d6/d7/d19/d4d/fec [0,4194304] 0 2024-05-13T22:48:39.418 INFO:tasks.workunit.client.1.smithi130.stdout:2/774: symlink d4/db/d26/d9e/l10d 0 2024-05-13T22:48:39.420 INFO:tasks.workunit.client.1.smithi130.stdout:1/721: creat db/d1b/da3/d72/fe8 x:0 0 0 2024-05-13T22:48:39.423 INFO:tasks.workunit.client.1.smithi130.stdout:3/753: truncate d8/d1a/dfa/d2c/f87 338860 0 2024-05-13T22:48:39.426 INFO:tasks.workunit.client.1.smithi130.stdout:5/772: mknod d6/dd/d18/d6e/d82/d71/c101 0 2024-05-13T22:48:39.427 INFO:tasks.workunit.client.1.smithi130.stdout:4/733: truncate d8/f1f 714277 0 2024-05-13T22:48:39.428 INFO:tasks.workunit.client.1.smithi130.stdout:8/691: write d1/d2/db/d15/d7e/f95 [727473,112261] 0 2024-05-13T22:48:39.429 INFO:tasks.workunit.client.1.smithi130.stdout:4/734: stat d8/dc/d35/d33/d44/d77/d94/db6/c11 0 2024-05-13T22:48:39.429 INFO:tasks.workunit.client.1.smithi130.stdout:2/775: creat d4/db/dc/d18/d58/f10e x:0 0 0 2024-05-13T22:48:39.430 INFO:tasks.workunit.client.1.smithi130.stdout:1/722: mkdir db/d4a/de9 0 2024-05-13T22:48:39.430 INFO:tasks.workunit.client.1.smithi130.stdout:4/735: fdatasync d8/dc/f9a 0 2024-05-13T22:48:39.432 INFO:tasks.workunit.client.1.smithi130.stdout:5/773: mknod d6/d7/d19/d4d/d6a/c102 0 2024-05-13T22:48:39.433 INFO:tasks.workunit.client.1.smithi130.stdout:6/714: creat d4/ff4 x:0 0 0 2024-05-13T22:48:39.434 INFO:tasks.workunit.client.1.smithi130.stdout:2/776: stat d4/db/dbf/f35 0 2024-05-13T22:48:39.435 INFO:tasks.workunit.client.1.smithi130.stdout:6/715: dread - d4/d15/d1d/daa/fc2 zero size 2024-05-13T22:48:39.435 INFO:tasks.workunit.client.1.smithi130.stdout:1/723: mknod db/d9c/cea 0 2024-05-13T22:48:39.436 INFO:tasks.workunit.client.1.smithi130.stdout:4/736: mkdir d8/dc/d35/d33/d44/d77/d94/db6/dea 0 2024-05-13T22:48:39.437 INFO:tasks.workunit.client.1.smithi130.stdout:3/754: dwrite d8/fe5 [0,4194304] 0 2024-05-13T22:48:39.437 INFO:tasks.workunit.client.1.smithi130.stdout:5/774: symlink d6/d7/d32/l103 0 2024-05-13T22:48:39.438 INFO:tasks.workunit.client.1.smithi130.stdout:8/692: mkdir d1/dc/d5d/dde/d104 0 2024-05-13T22:48:39.443 INFO:tasks.workunit.client.1.smithi130.stdout:5/775: dwrite d6/dd/d18/d6e/d82/d4b/de7/fc6 [0,4194304] 0 2024-05-13T22:48:39.447 INFO:tasks.workunit.client.1.smithi130.stdout:5/776: dread d6/d1a/daf/fdd [0,4194304] 0 2024-05-13T22:48:39.455 INFO:tasks.workunit.client.1.smithi130.stdout:8/693: unlink d1/d2/db/d15/f92 0 2024-05-13T22:48:39.455 INFO:tasks.workunit.client.1.smithi130.stdout:3/755: dwrite d8/f33 [0,4194304] 0 2024-05-13T22:48:39.466 INFO:tasks.workunit.client.1.smithi130.stdout:5/777: symlink d6/dd/d18/d6e/d82/db0/l104 0 2024-05-13T22:48:39.474 INFO:tasks.workunit.client.1.smithi130.stdout:7/775: sync 2024-05-13T22:48:39.480 INFO:tasks.workunit.client.1.smithi130.stdout:2/777: write d4/f9 [5481733,25945] 0 2024-05-13T22:48:39.480 INFO:tasks.workunit.client.1.smithi130.stdout:2/778: chown d4/d56/dff/d4a/d76/f92 61417583 1 2024-05-13T22:48:39.482 INFO:tasks.workunit.client.1.smithi130.stdout:6/716: write d4/d16/f2a [2030976,104228] 0 2024-05-13T22:48:39.509 INFO:tasks.workunit.client.1.smithi130.stdout:3/756: fdatasync d8/d1a/f4f 0 2024-05-13T22:48:39.521 INFO:tasks.workunit.client.1.smithi130.stdout:9/801: sync 2024-05-13T22:48:39.522 INFO:tasks.workunit.client.1.smithi130.stdout:6/717: truncate d4/d15/d1f/d46/d52/d7b/f9d 500389 0 2024-05-13T22:48:39.522 INFO:tasks.workunit.client.1.smithi130.stdout:6/718: stat d4/d15/d1d/d28/d75/da8/da0/de4 0 2024-05-13T22:48:39.532 INFO:tasks.workunit.client.1.smithi130.stdout:1/724: getdents db/d1b/da3 0 2024-05-13T22:48:39.533 INFO:tasks.workunit.client.1.smithi130.stdout:1/725: write db/d1b/da3/d2a/d35/fba [50839,5054] 0 2024-05-13T22:48:39.534 INFO:tasks.workunit.client.1.smithi130.stdout:2/779: symlink d4/d56/dff/d4a/dc6/d10c/l10f 0 2024-05-13T22:48:39.535 INFO:tasks.workunit.client.1.smithi130.stdout:7/776: truncate d7/da/d1d/d5d/da6/d5b/fb3 267009 0 2024-05-13T22:48:39.539 INFO:tasks.workunit.client.1.smithi130.stdout:1/726: dread - db/d1b/da3/d2a/d4c/f65 zero size 2024-05-13T22:48:39.541 INFO:tasks.workunit.client.1.smithi130.stdout:2/780: creat d4/db/dbf/df9/d30/d88/dc5/dcd/f110 x:0 0 0 2024-05-13T22:48:39.544 INFO:tasks.workunit.client.1.smithi130.stdout:6/719: mkdir d4/d15/d1d/df5 0 2024-05-13T22:48:39.545 INFO:tasks.workunit.client.1.smithi130.stdout:1/727: dwrite db/d9c/fd3 [0,4194304] 0 2024-05-13T22:48:39.545 INFO:tasks.workunit.client.1.smithi130.stdout:3/757: truncate d8/d35/f52 530758 0 2024-05-13T22:48:39.551 INFO:tasks.workunit.client.1.smithi130.stdout:1/728: symlink db/d9c/leb 0 2024-05-13T22:48:39.553 INFO:tasks.workunit.client.1.smithi130.stdout:7/777: dread d7/da/d1d/d3d/d46/f4b [0,4194304] 0 2024-05-13T22:48:39.553 INFO:tasks.workunit.client.1.smithi130.stdout:7/778: chown d7/da/d1d/de6/df4 7710570 1 2024-05-13T22:48:39.554 INFO:tasks.workunit.client.1.smithi130.stdout:9/802: rename d1/d15/lf6 to d1/d7/d23/d28/d32/l10c 0 2024-05-13T22:48:39.556 INFO:tasks.workunit.client.1.smithi130.stdout:3/758: mknod d8/dc/d7b/da3/daf/df8/c102 0 2024-05-13T22:48:39.558 INFO:tasks.workunit.client.1.smithi130.stdout:1/729: dwrite db/d1b/da3/d2a/f4f [0,4194304] 0 2024-05-13T22:48:39.559 INFO:tasks.workunit.client.1.smithi130.stdout:1/730: write db/d4a/f95 [898603,81519] 0 2024-05-13T22:48:39.563 INFO:tasks.workunit.client.1.smithi130.stdout:9/803: rename d1/f4 to d1/d15/d9e/dfa/f10d 0 2024-05-13T22:48:39.568 INFO:tasks.workunit.client.1.smithi130.stdout:6/720: dread d4/f12 [0,4194304] 0 2024-05-13T22:48:39.574 INFO:tasks.workunit.client.1.smithi130.stdout:3/759: symlink d8/dc/d15/dcc/l103 0 2024-05-13T22:48:39.576 INFO:tasks.workunit.client.1.smithi130.stdout:0/812: sync 2024-05-13T22:48:39.577 INFO:tasks.workunit.client.1.smithi130.stdout:1/731: mknod db/d82/cec 0 2024-05-13T22:48:39.580 INFO:tasks.workunit.client.1.smithi130.stdout:3/760: chown d8/d1a/dfa/ca9 0 1 2024-05-13T22:48:39.581 INFO:tasks.workunit.client.1.smithi130.stdout:4/737: truncate d8/dc/d35/d33/d44/d77/d94/db6/dad/fca 2455709 0 2024-05-13T22:48:39.584 INFO:tasks.workunit.client.1.smithi130.stdout:6/721: creat d4/d15/d1d/d28/d4e/d9e/ddb/ff6 x:0 0 0 2024-05-13T22:48:39.584 INFO:tasks.workunit.client.1.smithi130.stdout:8/694: write d1/d2/db/d15/d16/f5b [1486372,114217] 0 2024-05-13T22:48:39.586 INFO:tasks.workunit.client.1.smithi130.stdout:8/695: chown d1/d2/db/d15/d18/d42/la9 9581 1 2024-05-13T22:48:39.586 INFO:tasks.workunit.client.1.smithi130.stdout:9/804: mkdir d1/d7/d23/d28/d32/d45/d10e 0 2024-05-13T22:48:39.587 INFO:tasks.workunit.client.1.smithi130.stdout:8/696: chown d1/dc/d5d/l99 376552925 1 2024-05-13T22:48:39.587 INFO:tasks.workunit.client.1.smithi130.stdout:1/732: dread db/d1b/da3/d49/d56/d88/f9a [0,4194304] 0 2024-05-13T22:48:39.588 INFO:tasks.workunit.client.1.smithi130.stdout:5/778: write d6/d7/d32/f51 [1194468,112898] 0 2024-05-13T22:48:39.589 INFO:tasks.workunit.client.1.smithi130.stdout:3/761: mknod d8/d1a/d1c/c104 0 2024-05-13T22:48:39.589 INFO:tasks.workunit.client.1.smithi130.stdout:5/779: chown d6/d7/d19/ca8 12 1 2024-05-13T22:48:39.594 INFO:tasks.workunit.client.1.smithi130.stdout:5/780: dwrite d6/d1a/d72/fcf [0,4194304] 0 2024-05-13T22:48:39.601 INFO:tasks.workunit.client.1.smithi130.stdout:3/762: write d8/fbd [423010,96993] 0 2024-05-13T22:48:39.601 INFO:tasks.workunit.client.1.smithi130.stdout:0/813: dread d0/d7/dd/d2e/d9e/da7/dc3/fdd [0,4194304] 0 2024-05-13T22:48:39.602 INFO:tasks.workunit.client.1.smithi130.stdout:1/733: truncate db/d1b/da3/d2a/d4c/f65 502516 0 2024-05-13T22:48:39.605 INFO:tasks.workunit.client.1.smithi130.stdout:9/805: dwrite d1/d7/d23/d3d/d43/ffc [0,4194304] 0 2024-05-13T22:48:39.611 INFO:tasks.workunit.client.1.smithi130.stdout:2/781: write d4/db/dbf/df9/d30/f47 [1844904,48827] 0 2024-05-13T22:48:39.617 INFO:tasks.workunit.client.1.smithi130.stdout:5/781: stat d6/d1a/d92/lf4 0 2024-05-13T22:48:39.618 INFO:tasks.workunit.client.1.smithi130.stdout:7/779: write d7/da/d1d/d3d/d46/f4f [3465634,114963] 0 2024-05-13T22:48:39.621 INFO:tasks.workunit.client.1.smithi130.stdout:8/697: symlink d1/d2/db/d15/l105 0 2024-05-13T22:48:39.622 INFO:tasks.workunit.client.1.smithi130.stdout:8/698: readlink d1/d2/db/d15/d18/d42/d53/d56/l8d 0 2024-05-13T22:48:39.624 INFO:tasks.workunit.client.1.smithi130.stdout:1/734: symlink db/d9c/led 0 2024-05-13T22:48:39.631 INFO:tasks.workunit.client.1.smithi130.stdout:0/814: mkdir d0/d7/dd/d17/d1f/d2c/de3/d10f 0 2024-05-13T22:48:39.634 INFO:tasks.workunit.client.1.smithi130.stdout:0/815: dread - d0/d7/dd/d2e/d9e/da7/de1/f10a zero size 2024-05-13T22:48:39.634 INFO:tasks.workunit.client.1.smithi130.stdout:9/806: creat d1/d7/d23/d28/d73/d8d/deb/f10f x:0 0 0 2024-05-13T22:48:39.634 INFO:tasks.workunit.client.1.smithi130.stdout:9/807: dread - d1/d15/d9e/ff7 zero size 2024-05-13T22:48:39.634 INFO:tasks.workunit.client.1.smithi130.stdout:4/738: creat d8/dc/d35/d33/feb x:0 0 0 2024-05-13T22:48:39.636 INFO:tasks.workunit.client.1.smithi130.stdout:6/722: rmdir d4/d15/d1d/df5 0 2024-05-13T22:48:39.641 INFO:tasks.workunit.client.1.smithi130.stdout:7/780: rename d7/da/d1d/d5d/da6/d7e/c90 to d7/d8f/c100 0 2024-05-13T22:48:39.641 INFO:tasks.workunit.client.1.smithi130.stdout:7/781: chown d7/da/d1d/d5d/da6/d9 0 1 2024-05-13T22:48:39.644 INFO:tasks.workunit.client.1.smithi130.stdout:8/699: fdatasync d1/d2/db/d15/d16/f88 0 2024-05-13T22:48:39.645 INFO:tasks.workunit.client.1.smithi130.stdout:8/700: readlink d1/d2/db/d15/d18/d1a/l6a 0 2024-05-13T22:48:39.651 INFO:tasks.workunit.client.1.smithi130.stdout:3/763: dwrite d8/dc/d15/dfd/def/ff1 [4194304,4194304] 0 2024-05-13T22:48:39.651 INFO:tasks.workunit.client.1.smithi130.stdout:0/816: creat d0/d7/dd/d69/da9/d8d/f110 x:0 0 0 2024-05-13T22:48:39.653 INFO:tasks.workunit.client.1.smithi130.stdout:9/808: rmdir d1/d7/d23/d3d 39 2024-05-13T22:48:39.654 INFO:tasks.workunit.client.1.smithi130.stdout:4/739: write d8/dc/d35/d33/d44/fa3 [2736782,90348] 0 2024-05-13T22:48:39.656 INFO:tasks.workunit.client.1.smithi130.stdout:4/740: write d8/dc/d26/f55 [2753883,118424] 0 2024-05-13T22:48:39.656 INFO:tasks.workunit.client.1.smithi130.stdout:4/741: chown d8/dc/d35/d33/d3f 3603211 1 2024-05-13T22:48:39.657 INFO:tasks.workunit.client.1.smithi130.stdout:6/723: dread - d4/d15/d1f/d3c/fb5 zero size 2024-05-13T22:48:39.658 INFO:tasks.workunit.client.1.smithi130.stdout:4/742: dread - d8/dc/d35/d33/d44/d56/d8c/fdd zero size 2024-05-13T22:48:39.663 INFO:tasks.workunit.client.1.smithi130.stdout:5/782: dwrite d6/d7/d19/f9a [0,4194304] 0 2024-05-13T22:48:39.665 INFO:tasks.workunit.client.1.smithi130.stdout:8/701: symlink d1/d2/db/d15/d7e/l106 0 2024-05-13T22:48:39.669 INFO:tasks.workunit.client.1.smithi130.stdout:7/782: dread d7/da/d1d/d5d/da6/d9/d4a/fba [0,4194304] 0 2024-05-13T22:48:39.670 INFO:tasks.workunit.client.1.smithi130.stdout:3/764: creat d8/d1a/dfa/d27/f105 x:0 0 0 2024-05-13T22:48:39.676 INFO:tasks.workunit.client.1.smithi130.stdout:6/724: fsync d4/d15/d1d/d28/d75/f80 0 2024-05-13T22:48:39.678 INFO:tasks.workunit.client.1.smithi130.stdout:1/735: dwrite db/d1b/da3/d49/f48 [0,4194304] 0 2024-05-13T22:48:39.682 INFO:tasks.workunit.client.1.smithi130.stdout:8/702: mkdir d1/d2/db/d15/d18/dba/d107 0 2024-05-13T22:48:39.684 INFO:tasks.workunit.client.1.smithi130.stdout:7/783: mkdir d7/da/d1d/db1/d101 0 2024-05-13T22:48:39.685 INFO:tasks.workunit.client.1.smithi130.stdout:9/809: symlink d1/l110 0 2024-05-13T22:48:39.687 INFO:tasks.workunit.client.1.smithi130.stdout:0/817: link d0/d7/dd/d17/d1f/d2c/fbd d0/d7/dd/d17/d52/d67/d68/d96/f111 0 2024-05-13T22:48:39.690 INFO:tasks.workunit.client.1.smithi130.stdout:4/743: rename d8/dc/d35/d33/d44/db8/lb7 to d8/dc/d35/d33/d44/d77/d94/db6/dd5/lec 0 2024-05-13T22:48:39.691 INFO:tasks.workunit.client.1.smithi130.stdout:1/736: mkdir db/d1b/da3/d72/d75/dee 0 2024-05-13T22:48:39.693 INFO:tasks.workunit.client.1.smithi130.stdout:8/703: unlink d1/d2/db/d15/d18/c6f 0 2024-05-13T22:48:39.695 INFO:tasks.workunit.client.1.smithi130.stdout:3/765: mkdir d8/dc/d106 0 2024-05-13T22:48:39.697 INFO:tasks.workunit.client.1.smithi130.stdout:6/725: dread d4/d15/d1f/d46/d50/f67 [0,4194304] 0 2024-05-13T22:48:39.703 INFO:tasks.workunit.client.1.smithi130.stdout:5/783: rename d6/d1a/daf/le9 to d6/dd/l105 0 2024-05-13T22:48:39.704 INFO:tasks.workunit.client.1.smithi130.stdout:4/744: creat d8/dc/d35/d33/d44/d77/d94/db6/fed x:0 0 0 2024-05-13T22:48:39.706 INFO:tasks.workunit.client.1.smithi130.stdout:1/737: unlink db/d1b/da3/d2a/d35/f3b 0 2024-05-13T22:48:39.708 INFO:tasks.workunit.client.1.smithi130.stdout:8/704: dread d1/d2/db/d15/f74 [0,4194304] 0 2024-05-13T22:48:39.709 INFO:tasks.workunit.client.1.smithi130.stdout:1/738: read db/d1b/da3/d2a/d38/d46/d6d/fbe [3040843,105940] 0 2024-05-13T22:48:39.710 INFO:tasks.workunit.client.1.smithi130.stdout:3/766: mknod d8/dc/d7b/c107 0 2024-05-13T22:48:39.713 INFO:tasks.workunit.client.1.smithi130.stdout:6/726: stat d4/d15/d1d/d60/da2/da4/fd6 0 2024-05-13T22:48:39.718 INFO:tasks.workunit.client.1.smithi130.stdout:7/784: dwrite d7/da/d1d/d5d/da6/d9/d4a/fba [4194304,4194304] 0 2024-05-13T22:48:39.724 INFO:tasks.workunit.client.1.smithi130.stdout:1/739: mknod db/d1b/da3/d2a/d38/d46/d6d/cef 0 2024-05-13T22:48:39.726 INFO:tasks.workunit.client.1.smithi130.stdout:1/740: truncate db/f1a 5030776 0 2024-05-13T22:48:39.727 INFO:tasks.workunit.client.1.smithi130.stdout:9/810: rmdir d1/d7/d23/d28/d73/db2/de4 0 2024-05-13T22:48:39.730 INFO:tasks.workunit.client.1.smithi130.stdout:4/745: mkdir d8/dc/d35/d33/dee 0 2024-05-13T22:48:39.733 INFO:tasks.workunit.client.1.smithi130.stdout:1/741: dread db/d1b/da3/d2a/d38/d46/f5f [0,4194304] 0 2024-05-13T22:48:39.733 INFO:tasks.workunit.client.1.smithi130.stdout:7/785: unlink d7/da/d1d/d42/ce4 0 2024-05-13T22:48:39.735 INFO:tasks.workunit.client.1.smithi130.stdout:8/705: dread d1/d2/f3 [0,4194304] 0 2024-05-13T22:48:39.739 INFO:tasks.workunit.client.1.smithi130.stdout:5/784: link d6/fac d6/dd/d79/f106 0 2024-05-13T22:48:39.740 INFO:tasks.workunit.client.1.smithi130.stdout:5/785: truncate d6/dd/f37 4822747 0 2024-05-13T22:48:39.742 INFO:tasks.workunit.client.1.smithi130.stdout:1/742: creat db/d1b/da3/d72/d75/ff0 x:0 0 0 2024-05-13T22:48:39.744 INFO:tasks.workunit.client.1.smithi130.stdout:6/727: rename d4/d16/f39 to d4/d15/d1d/d60/da2/ff7 0 2024-05-13T22:48:39.745 INFO:tasks.workunit.client.1.smithi130.stdout:4/746: mknod d8/dc/d35/cef 0 2024-05-13T22:48:39.746 INFO:tasks.workunit.client.1.smithi130.stdout:3/767: getdents d8/d35 0 2024-05-13T22:48:39.747 INFO:tasks.workunit.client.1.smithi130.stdout:3/768: dread - d8/dc/d15/dfd/def/ff2 zero size 2024-05-13T22:48:39.747 INFO:tasks.workunit.client.1.smithi130.stdout:1/743: creat db/d1b/da3/d2a/d38/d46/ff1 x:0 0 0 2024-05-13T22:48:39.751 INFO:tasks.workunit.client.1.smithi130.stdout:5/786: rename d6/dd/d18/d6e/d82/d84/cd6 to d6/d1a/d72/c107 0 2024-05-13T22:48:39.754 INFO:tasks.workunit.client.1.smithi130.stdout:6/728: mkdir d4/d15/d1d/d60/da2/da4/df8 0 2024-05-13T22:48:39.754 INFO:tasks.workunit.client.1.smithi130.stdout:3/769: creat d8/dc/d7b/da3/daf/df8/f108 x:0 0 0 2024-05-13T22:48:39.756 INFO:tasks.workunit.client.1.smithi130.stdout:1/744: chown db/d1b/da3/d5e/c93 3196581 1 2024-05-13T22:48:39.756 INFO:tasks.workunit.client.1.smithi130.stdout:7/786: link d7/da/d1d/d5d/da6/d9/d66/cd8 d7/da/d1d/d3d/d46/dc7/c102 0 2024-05-13T22:48:39.756 INFO:tasks.workunit.client.1.smithi130.stdout:7/787: readlink d7/da/l84 0 2024-05-13T22:48:39.762 INFO:tasks.workunit.client.1.smithi130.stdout:3/770: write d8/fa [643847,12123] 0 2024-05-13T22:48:39.762 INFO:tasks.workunit.client.1.smithi130.stdout:3/771: chown c7 9848733 1 2024-05-13T22:48:39.763 INFO:tasks.workunit.client.1.smithi130.stdout:6/729: rename d4/d15/d1d/d60/da2/da4/dac/cf0 to d4/d15/d1d/d28/d75/da8/da0/cf9 0 2024-05-13T22:48:39.764 INFO:tasks.workunit.client.1.smithi130.stdout:6/730: chown d4/d15/d1f/d3c/fb5 1 1 2024-05-13T22:48:39.773 INFO:tasks.workunit.client.1.smithi130.stdout:5/787: creat d6/d7/d19/d4d/d8a/df7/f108 x:0 0 0 2024-05-13T22:48:39.777 INFO:tasks.workunit.client.1.smithi130.stdout:4/747: write d8/f8b [2045423,57485] 0 2024-05-13T22:48:39.778 INFO:tasks.workunit.client.1.smithi130.stdout:6/731: creat d4/d15/d1d/d60/da2/da4/ffa x:0 0 0 2024-05-13T22:48:39.779 INFO:tasks.workunit.client.1.smithi130.stdout:1/745: symlink db/d1b/da3/d2a/d38/lf2 0 2024-05-13T22:48:39.780 INFO:tasks.workunit.client.1.smithi130.stdout:1/746: write db/d1b/da3/d49/d56/f7d [5063498,12299] 0 2024-05-13T22:48:39.781 INFO:tasks.workunit.client.1.smithi130.stdout:7/788: creat d7/da/d1d/d5d/da6/d39/da9/dfb/f103 x:0 0 0 2024-05-13T22:48:39.783 INFO:tasks.workunit.client.1.smithi130.stdout:3/772: unlink d8/d1a/dfa/d2c/l3a 0 2024-05-13T22:48:39.784 INFO:tasks.workunit.client.1.smithi130.stdout:4/748: fsync d8/dc/d35/d33/d3f/d57/f93 0 2024-05-13T22:48:39.785 INFO:tasks.workunit.client.1.smithi130.stdout:6/732: mknod d4/d15/d1f/d3c/cfb 0 2024-05-13T22:48:39.787 INFO:tasks.workunit.client.1.smithi130.stdout:5/788: dwrite d6/f1f [4194304,4194304] 0 2024-05-13T22:48:39.787 INFO:tasks.workunit.client.1.smithi130.stdout:5/789: chown d6/dd/l6d 1763076816 1 2024-05-13T22:48:39.801 INFO:tasks.workunit.client.1.smithi130.stdout:2/782: sync 2024-05-13T22:48:39.805 INFO:tasks.workunit.client.1.smithi130.stdout:6/733: dread d4/d16/f2a [0,4194304] 0 2024-05-13T22:48:39.805 INFO:tasks.workunit.client.1.smithi130.stdout:3/773: dread d8/dc/d7b/da3/dd3/f81 [0,4194304] 0 2024-05-13T22:48:39.807 INFO:tasks.workunit.client.1.smithi130.stdout:3/774: write d8/dc/d15/fcd [799848,42451] 0 2024-05-13T22:48:39.812 INFO:tasks.workunit.client.1.smithi130.stdout:1/747: dwrite db/d1b/da3/d2a/d35/f3d [0,4194304] 0 2024-05-13T22:48:39.815 INFO:tasks.workunit.client.1.smithi130.stdout:3/775: symlink d8/dc/d7b/l109 0 2024-05-13T22:48:39.820 INFO:tasks.workunit.client.1.smithi130.stdout:3/776: creat d8/dc/d7b/da3/daf/df8/f10a x:0 0 0 2024-05-13T22:48:39.821 INFO:tasks.workunit.client.1.smithi130.stdout:1/748: mknod db/cf3 0 2024-05-13T22:48:39.825 INFO:tasks.workunit.client.1.smithi130.stdout:3/777: write d8/d1a/dfa/f20 [1819710,92912] 0 2024-05-13T22:48:39.826 INFO:tasks.workunit.client.1.smithi130.stdout:5/790: dread d6/d2f/f39 [0,4194304] 0 2024-05-13T22:48:39.827 INFO:tasks.workunit.client.1.smithi130.stdout:7/789: dwrite d7/da/d1d/d5d/da6/f4c [0,4194304] 0 2024-05-13T22:48:39.837 INFO:tasks.workunit.client.1.smithi130.stdout:4/749: dwrite d8/dc/d35/d33/d44/d77/d94/db6/dad/fca [0,4194304] 0 2024-05-13T22:48:39.838 INFO:tasks.workunit.client.1.smithi130.stdout:6/734: dwrite d4/d15/d1d/d28/f71 [0,4194304] 0 2024-05-13T22:48:39.838 INFO:tasks.workunit.client.1.smithi130.stdout:7/790: write d7/da/fd0 [156850,43298] 0 2024-05-13T22:48:39.838 INFO:tasks.workunit.client.1.smithi130.stdout:5/791: fdatasync d6/dd/d18/d6e/d82/d71/f100 0 2024-05-13T22:48:39.844 INFO:tasks.workunit.client.1.smithi130.stdout:3/778: dwrite d8/d1a/dfa/f9e [4194304,4194304] 0 2024-05-13T22:48:39.845 INFO:tasks.workunit.client.1.smithi130.stdout:3/779: rename d8/dc/d7b to d8/dc/d7b/da3/dd3/d10b 22 2024-05-13T22:48:39.847 INFO:tasks.workunit.client.1.smithi130.stdout:6/735: creat d4/d15/d1d/d28/d75/d7d/ffc x:0 0 0 2024-05-13T22:48:39.847 INFO:tasks.workunit.client.1.smithi130.stdout:1/749: write db/d1b/da3/d2a/f41 [830041,8919] 0 2024-05-13T22:48:39.849 INFO:tasks.workunit.client.1.smithi130.stdout:6/736: chown d4/d15/d1f/faf 4813 1 2024-05-13T22:48:39.849 INFO:tasks.workunit.client.1.smithi130.stdout:4/750: chown d8/dc/d35/d33/d44/d56/ce4 161444649 1 2024-05-13T22:48:39.851 INFO:tasks.workunit.client.1.smithi130.stdout:4/751: fdatasync d8/f8b 0 2024-05-13T22:48:39.851 INFO:tasks.workunit.client.1.smithi130.stdout:5/792: creat d6/dd/d18/d6e/d82/db0/f109 x:0 0 0 2024-05-13T22:48:39.853 INFO:tasks.workunit.client.1.smithi130.stdout:3/780: fdatasync d8/d1a/dfa/f8d 0 2024-05-13T22:48:39.855 INFO:tasks.workunit.client.1.smithi130.stdout:3/781: readlink d8/dc/d7b/da3/dd3/l53 0 2024-05-13T22:48:39.857 INFO:tasks.workunit.client.1.smithi130.stdout:1/750: dwrite db/d1b/da3/f53 [0,4194304] 0 2024-05-13T22:48:39.857 INFO:tasks.workunit.client.1.smithi130.stdout:6/737: mknod d4/d15/d1f/d64/cfd 0 2024-05-13T22:48:39.860 INFO:tasks.workunit.client.1.smithi130.stdout:4/752: rename d8/dc/d35/d33/d3f/fe2 to d8/dc/d84/ff0 0 2024-05-13T22:48:39.860 INFO:tasks.workunit.client.1.smithi130.stdout:4/753: chown d8 26405855 1 2024-05-13T22:48:39.867 INFO:tasks.workunit.client.1.smithi130.stdout:7/791: dwrite d7/da/d1d/d42/f73 [0,4194304] 0 2024-05-13T22:48:39.868 INFO:tasks.workunit.client.1.smithi130.stdout:7/792: fdatasync d7/da/d86/fc3 0 2024-05-13T22:48:39.879 INFO:tasks.workunit.client.1.smithi130.stdout:5/793: link d6/d1a/d92/f96 d6/d7/d19/f10a 0 2024-05-13T22:48:39.882 INFO:tasks.workunit.client.1.smithi130.stdout:3/782: dread d8/d1a/dfa/f7f [0,4194304] 0 2024-05-13T22:48:39.882 INFO:tasks.workunit.client.1.smithi130.stdout:5/794: rmdir d6/d7 39 2024-05-13T22:48:39.883 INFO:tasks.workunit.client.1.smithi130.stdout:5/795: chown d6/dd/d79/c8c 12 1 2024-05-13T22:48:39.885 INFO:tasks.workunit.client.1.smithi130.stdout:4/754: getdents d8/dc/d35/d33/d44/d77/d94/db6/d81 0 2024-05-13T22:48:39.886 INFO:tasks.workunit.client.1.smithi130.stdout:3/783: read d8/dc/d15/f9a [12024,55638] 0 2024-05-13T22:48:39.887 INFO:tasks.workunit.client.1.smithi130.stdout:3/784: fsync d8/dc/f9f 0 2024-05-13T22:48:39.887 INFO:tasks.workunit.client.1.smithi130.stdout:0/818: sync 2024-05-13T22:48:39.891 INFO:tasks.workunit.client.1.smithi130.stdout:0/819: dread d0/d7/dd/d2e/da8/dcd/df8/f37 [0,4194304] 0 2024-05-13T22:48:39.896 INFO:tasks.workunit.client.1.smithi130.stdout:7/793: dread d7/da/d1d/d5d/da6/d9/d66/f81 [0,4194304] 0 2024-05-13T22:48:39.899 INFO:tasks.workunit.client.1.smithi130.stdout:3/785: mknod d8/d35/c10c 0 2024-05-13T22:48:39.901 INFO:tasks.workunit.client.1.smithi130.stdout:4/755: rename d8/dc/d35 to d8/dc/d80/db0/df1 0 2024-05-13T22:48:39.903 INFO:tasks.workunit.client.1.smithi130.stdout:4/756: chown d8/dc/d80/db0/df1/d33/d44/d56/d8c/fd2 67 1 2024-05-13T22:48:39.904 INFO:tasks.workunit.client.1.smithi130.stdout:0/820: dread d0/d7/dd/d17/d1f/d2c/fbd [0,4194304] 0 2024-05-13T22:48:39.905 INFO:tasks.workunit.client.1.smithi130.stdout:0/821: dread - d0/d7/dd/d2e/d9e/da7/dc3/ff6 zero size 2024-05-13T22:48:39.905 INFO:tasks.workunit.client.1.smithi130.stdout:6/738: write d4/f73 [2877335,126736] 0 2024-05-13T22:48:39.907 INFO:tasks.workunit.client.1.smithi130.stdout:7/794: rename d7/cfa to d7/da/d1d/db1/de1/c104 0 2024-05-13T22:48:39.907 INFO:tasks.workunit.client.1.smithi130.stdout:1/751: write db/d1b/da3/d2a/d4c/f65 [650467,55938] 0 2024-05-13T22:48:39.911 INFO:tasks.workunit.client.1.smithi130.stdout:0/822: mkdir d0/d7/dd/d69/da9/d8d/d9f/d112 0 2024-05-13T22:48:39.913 INFO:tasks.workunit.client.1.smithi130.stdout:6/739: truncate d4/d15/d1f/d46/d50/fa7 2211220 0 2024-05-13T22:48:39.913 INFO:tasks.workunit.client.1.smithi130.stdout:0/823: dread - d0/d7/dd/d2e/d9e/f10c zero size 2024-05-13T22:48:39.915 INFO:tasks.workunit.client.1.smithi130.stdout:7/795: dwrite d7/da/d1d/d3d/d46/f4b [4194304,4194304] 0 2024-05-13T22:48:39.915 INFO:tasks.workunit.client.1.smithi130.stdout:1/752: dwrite db/f16 [0,4194304] 0 2024-05-13T22:48:39.915 INFO:tasks.workunit.client.1.smithi130.stdout:4/757: creat d8/dc/d80/db0/df1/d33/dee/ff2 x:0 0 0 2024-05-13T22:48:39.920 INFO:tasks.workunit.client.1.smithi130.stdout:9/811: sync 2024-05-13T22:48:39.920 INFO:tasks.workunit.client.1.smithi130.stdout:8/706: sync 2024-05-13T22:48:39.920 INFO:tasks.workunit.client.1.smithi130.stdout:2/783: sync 2024-05-13T22:48:39.923 INFO:tasks.workunit.client.1.smithi130.stdout:6/740: mkdir d4/d15/d1d/d28/d75/da8/da0/dfe 0 2024-05-13T22:48:39.926 INFO:tasks.workunit.client.1.smithi130.stdout:1/753: write db/d1b/da3/fdd [92965,55719] 0 2024-05-13T22:48:39.927 INFO:tasks.workunit.client.1.smithi130.stdout:4/758: creat d8/dc/d80/db0/df1/d33/dbf/ff3 x:0 0 0 2024-05-13T22:48:39.928 INFO:tasks.workunit.client.1.smithi130.stdout:5/796: dwrite d6/d7/d19/d4d/d6a/f9c [0,4194304] 0 2024-05-13T22:48:39.931 INFO:tasks.workunit.client.1.smithi130.stdout:4/759: read - d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/fed zero size 2024-05-13T22:48:39.931 INFO:tasks.workunit.client.1.smithi130.stdout:5/797: read - d6/d7/d19/d4d/d8a/df7/f108 zero size 2024-05-13T22:48:39.933 INFO:tasks.workunit.client.1.smithi130.stdout:8/707: unlink d1/d2/f8c 0 2024-05-13T22:48:39.934 INFO:tasks.workunit.client.1.smithi130.stdout:1/754: dwrite db/d1b/da3/fdd [0,4194304] 0 2024-05-13T22:48:39.936 INFO:tasks.workunit.client.1.smithi130.stdout:4/760: truncate d8/dc/d80/db0/df1/d33/d3f/d57/fb1 819079 0 2024-05-13T22:48:39.940 INFO:tasks.workunit.client.1.smithi130.stdout:2/784: rmdir d4/db/dbf/df9/d30/d88/d99 39 2024-05-13T22:48:39.941 INFO:tasks.workunit.client.1.smithi130.stdout:1/755: dwrite db/d1b/da3/d2a/d35/f3d [0,4194304] 0 2024-05-13T22:48:39.942 INFO:tasks.workunit.client.1.smithi130.stdout:1/756: chown db/d1b/da3/d2a/c2d 1 1 2024-05-13T22:48:39.943 INFO:tasks.workunit.client.1.smithi130.stdout:7/796: fsync d7/da/d1d/d3d/d46/d92/df9/fb6 0 2024-05-13T22:48:39.951 INFO:tasks.workunit.client.1.smithi130.stdout:0/824: dwrite d0/d7/dd/d2e/f4e [0,4194304] 0 2024-05-13T22:48:39.953 INFO:tasks.workunit.client.1.smithi130.stdout:9/812: mkdir d1/d7/d23/d28/d32/d45/d111 0 2024-05-13T22:48:39.956 INFO:tasks.workunit.client.1.smithi130.stdout:4/761: mknod d8/dc/d80/db0/df1/d33/d44/db8/cf4 0 2024-05-13T22:48:39.960 INFO:tasks.workunit.client.1.smithi130.stdout:3/786: link d8/d1a/dfa/d2c/ff5 d8/d35/f10d 0 2024-05-13T22:48:39.961 INFO:tasks.workunit.client.1.smithi130.stdout:3/787: dread - d8/d1a/dfa/d27/f82 zero size 2024-05-13T22:48:39.961 INFO:tasks.workunit.client.1.smithi130.stdout:5/798: mkdir d6/dd/d18/d6e/d82/d10b 0 2024-05-13T22:48:39.962 INFO:tasks.workunit.client.1.smithi130.stdout:5/799: dread - d6/d2f/ff9 zero size 2024-05-13T22:48:39.966 INFO:tasks.workunit.client.1.smithi130.stdout:0/825: mknod d0/d7/dd/d69/da9/c113 0 2024-05-13T22:48:39.969 INFO:tasks.workunit.client.1.smithi130.stdout:1/757: link db/f1d db/d82/dc3/dd9/ff4 0 2024-05-13T22:48:39.972 INFO:tasks.workunit.client.1.smithi130.stdout:5/800: mknod d6/d7/d19/d4d/d8a/c10c 0 2024-05-13T22:48:39.973 INFO:tasks.workunit.client.1.smithi130.stdout:5/801: write d6/d7/f28 [1895221,98749] 0 2024-05-13T22:48:39.973 INFO:tasks.workunit.client.1.smithi130.stdout:8/708: truncate d1/d2/db/d15/d16/d54/d58/f94 540441 0 2024-05-13T22:48:39.978 INFO:tasks.workunit.client.1.smithi130.stdout:2/785: rename d4/db/dbf/c75 to d4/db/dc/d18/c111 0 2024-05-13T22:48:39.978 INFO:tasks.workunit.client.1.smithi130.stdout:0/826: dread - d0/d7/dd/d17/d52/d5b/d5f/f93 zero size 2024-05-13T22:48:39.981 INFO:tasks.workunit.client.1.smithi130.stdout:4/762: getdents d8/dc/d80/db0/df1/d33/d44/d61/da0/dbd 0 2024-05-13T22:48:39.984 INFO:tasks.workunit.client.1.smithi130.stdout:1/758: dread f8 [0,4194304] 0 2024-05-13T22:48:39.984 INFO:tasks.workunit.client.1.smithi130.stdout:0/827: dwrite d0/d7/dd/d17/d52/ff3 [0,4194304] 0 2024-05-13T22:48:39.985 INFO:tasks.workunit.client.1.smithi130.stdout:6/741: getdents d4/d15/d1d/daa 0 2024-05-13T22:48:39.986 INFO:tasks.workunit.client.1.smithi130.stdout:0/828: chown d0/d7/dd/d17/d1f/d2c/d5a/fc7 13595077 1 2024-05-13T22:48:39.987 INFO:tasks.workunit.client.1.smithi130.stdout:3/788: mknod d8/dc/c10e 0 2024-05-13T22:48:39.987 INFO:tasks.workunit.client.1.smithi130.stdout:0/829: chown d0/d7/dd/d69/da9 13971 1 2024-05-13T22:48:39.988 INFO:tasks.workunit.client.1.smithi130.stdout:3/789: read d8/d1a/dfa/f7f [4770632,84955] 0 2024-05-13T22:48:39.989 INFO:tasks.workunit.client.1.smithi130.stdout:5/802: creat d6/dd/d18/d6e/dc7/f10d x:0 0 0 2024-05-13T22:48:39.989 INFO:tasks.workunit.client.1.smithi130.stdout:3/790: write d8/fe5 [3634783,45361] 0 2024-05-13T22:48:39.995 INFO:tasks.workunit.client.1.smithi130.stdout:7/797: truncate d7/da/f5c 3153504 0 2024-05-13T22:48:39.995 INFO:tasks.workunit.client.1.smithi130.stdout:3/791: dwrite d8/dc/d7b/fe8 [0,4194304] 0 2024-05-13T22:48:39.996 INFO:tasks.workunit.client.1.smithi130.stdout:4/763: mknod d8/dc/d80/db0/df1/d33/d44/db8/cf5 0 2024-05-13T22:48:39.997 INFO:tasks.workunit.client.1.smithi130.stdout:3/792: readlink d8/dc/d15/dfd/lbf 0 2024-05-13T22:48:39.998 INFO:tasks.workunit.client.1.smithi130.stdout:1/759: dread - db/d1b/da3/d2a/f8e zero size 2024-05-13T22:48:39.998 INFO:tasks.workunit.client.1.smithi130.stdout:3/793: dread - d8/d1a/d55/fd7 zero size 2024-05-13T22:48:40.000 INFO:tasks.workunit.client.1.smithi130.stdout:6/742: creat d4/d15/d1f/d64/fff x:0 0 0 2024-05-13T22:48:40.001 INFO:tasks.workunit.client.1.smithi130.stdout:0/830: unlink d0/d7/dd/d17/d52/d5b/d60/fda 0 2024-05-13T22:48:40.002 INFO:tasks.workunit.client.1.smithi130.stdout:4/764: write d8/dc/d80/db0/df1/d33/d44/d56/d8c/fde [1374467,76217] 0 2024-05-13T22:48:40.003 INFO:tasks.workunit.client.1.smithi130.stdout:0/831: chown d0/d7/dd/d17/c35 1 1 2024-05-13T22:48:40.004 INFO:tasks.workunit.client.1.smithi130.stdout:9/813: dwrite d1/d7/f55 [0,4194304] 0 2024-05-13T22:48:40.008 INFO:tasks.workunit.client.1.smithi130.stdout:5/803: read d6/d7/f46 [2555383,36310] 0 2024-05-13T22:48:40.010 INFO:tasks.workunit.client.1.smithi130.stdout:5/804: truncate d6/fac 283209 0 2024-05-13T22:48:40.010 INFO:tasks.workunit.client.1.smithi130.stdout:8/709: write d1/d2/db/d15/d18/f31 [9329718,62309] 0 2024-05-13T22:48:40.055 INFO:tasks.workunit.client.1.smithi130.stdout:3/794: symlink d8/dc/d15/dfd/def/l10f 0 2024-05-13T22:48:40.056 INFO:tasks.workunit.client.1.smithi130.stdout:9/814: truncate d1/d7/d23/d28/d2d/f48 550248 0 2024-05-13T22:48:40.056 INFO:tasks.workunit.client.1.smithi130.stdout:7/798: dread d7/da/d86/f8a [0,4194304] 0 2024-05-13T22:48:40.056 INFO:tasks.workunit.client.1.smithi130.stdout:4/765: write d8/dc/d80/db0/df1/d33/d44/d56/fc5 [470639,26927] 0 2024-05-13T22:48:40.057 INFO:tasks.workunit.client.1.smithi130.stdout:0/832: write d0/f8 [6327365,109685] 0 2024-05-13T22:48:40.057 INFO:tasks.workunit.client.1.smithi130.stdout:8/710: fdatasync d1/dc/f5c 0 2024-05-13T22:48:40.058 INFO:tasks.workunit.client.1.smithi130.stdout:1/760: creat db/d1b/da3/d72/d75/dee/ff5 x:0 0 0 2024-05-13T22:48:40.060 INFO:tasks.workunit.client.1.smithi130.stdout:0/833: read - d0/d7/dd/d69/da9/ff9 zero size 2024-05-13T22:48:40.062 INFO:tasks.workunit.client.1.smithi130.stdout:0/834: readlink d0/d7/dd/l2b 0 2024-05-13T22:48:40.064 INFO:tasks.workunit.client.1.smithi130.stdout:3/795: dwrite d8/d1a/dfa/f9e [0,4194304] 0 2024-05-13T22:48:40.064 INFO:tasks.workunit.client.1.smithi130.stdout:0/835: read d0/d7/dd/d17/d52/ff3 [794205,94821] 0 2024-05-13T22:48:40.066 INFO:tasks.workunit.client.1.smithi130.stdout:0/836: chown d0/d7/dd/d69/da9/d8d/df4 65104352 1 2024-05-13T22:48:40.071 INFO:tasks.workunit.client.1.smithi130.stdout:3/796: dwrite d8/dc/d7b/da3/daf/fb7 [0,4194304] 0 2024-05-13T22:48:40.075 INFO:tasks.workunit.client.1.smithi130.stdout:2/786: sync 2024-05-13T22:48:40.076 INFO:tasks.workunit.client.1.smithi130.stdout:1/761: rename db/d82/d9f to db/d1b/da3/d72/d75/dee/df6 0 2024-05-13T22:48:40.082 INFO:tasks.workunit.client.1.smithi130.stdout:5/805: truncate d6/d2f/fa4 2758371 0 2024-05-13T22:48:40.083 INFO:tasks.workunit.client.1.smithi130.stdout:0/837: fsync d0/d7/dd/d17/d1f/d2c/f50 0 2024-05-13T22:48:40.083 INFO:tasks.workunit.client.1.smithi130.stdout:5/806: write d6/dd/d18/d6e/d82/d4b/de7/fd4 [742389,115093] 0 2024-05-13T22:48:40.087 INFO:tasks.workunit.client.1.smithi130.stdout:3/797: write d8/dc/d7b/da3/dd3/fc6 [1762337,125416] 0 2024-05-13T22:48:40.089 INFO:tasks.workunit.client.1.smithi130.stdout:2/787: stat d4/db/dbf/df9/d30/d88/d99 0 2024-05-13T22:48:40.105 INFO:tasks.workunit.client.1.smithi130.stdout:7/799: link d7/da/d1d/db1/dd4/cd6 d7/da/d1d/d3d/d46/dc7/c105 0 2024-05-13T22:48:40.106 INFO:tasks.workunit.client.1.smithi130.stdout:7/800: chown d7/da/l51 630 1 2024-05-13T22:48:40.110 INFO:tasks.workunit.client.1.smithi130.stdout:8/711: rename d1/d2/db/d15/d16/d54/c7d to d1/d2/db/d15/d18/d42/d53/de8/c108 0 2024-05-13T22:48:40.111 INFO:tasks.workunit.client.1.smithi130.stdout:9/815: truncate d1/d7/d23/d3d/fc6 1882931 0 2024-05-13T22:48:40.113 INFO:tasks.workunit.client.1.smithi130.stdout:2/788: write d4/db/dbf/d42/d4b/fa3 [734172,95348] 0 2024-05-13T22:48:40.116 INFO:tasks.workunit.client.1.smithi130.stdout:7/801: creat d7/da/d1d/d5d/d70/f106 x:0 0 0 2024-05-13T22:48:40.118 INFO:tasks.workunit.client.1.smithi130.stdout:1/762: creat db/ff7 x:0 0 0 2024-05-13T22:48:40.119 INFO:tasks.workunit.client.1.smithi130.stdout:4/766: rename d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/c11 to d8/dc/d80/db0/df1/d33/d44/cf6 0 2024-05-13T22:48:40.120 INFO:tasks.workunit.client.1.smithi130.stdout:6/743: sync 2024-05-13T22:48:40.120 INFO:tasks.workunit.client.1.smithi130.stdout:8/712: stat d1/d2/db/d15/d7e/d9f/fa0 0 2024-05-13T22:48:40.122 INFO:tasks.workunit.client.1.smithi130.stdout:6/744: dread - d4/d15/d1f/d64/fff zero size 2024-05-13T22:48:40.122 INFO:tasks.workunit.client.1.smithi130.stdout:4/767: write d8/dc/d80/db0/df1/d33/d44/fc8 [1018951,95609] 0 2024-05-13T22:48:40.124 INFO:tasks.workunit.client.1.smithi130.stdout:8/713: write d1/d2/db/d15/d16/d54/d58/f98 [572879,21637] 0 2024-05-13T22:48:40.125 INFO:tasks.workunit.client.1.smithi130.stdout:1/763: dwrite db/d1b/da3/d72/daa/fb3 [0,4194304] 0 2024-05-13T22:48:40.126 INFO:tasks.workunit.client.1.smithi130.stdout:5/807: creat d6/f10e x:0 0 0 2024-05-13T22:48:40.129 INFO:tasks.workunit.client.1.smithi130.stdout:0/838: rmdir d0/d7/dd/d17/d1f/d2c/de3/d10f 0 2024-05-13T22:48:40.131 INFO:tasks.workunit.client.1.smithi130.stdout:6/745: dwrite d4/d15/d1f/d46/f89 [0,4194304] 0 2024-05-13T22:48:40.134 INFO:tasks.workunit.client.1.smithi130.stdout:3/798: rename d8/dc/d7b/da3/daf/df8/c102 to d8/d1a/dfa/d27/c110 0 2024-05-13T22:48:40.136 INFO:tasks.workunit.client.1.smithi130.stdout:4/768: mkdir d8/dc/d80/db0/df1/d33/d44/db8/df7 0 2024-05-13T22:48:40.137 INFO:tasks.workunit.client.1.smithi130.stdout:4/769: write d8/dc/d80/db0/df1/d33/d44/d56/fc5 [492114,85247] 0 2024-05-13T22:48:40.139 INFO:tasks.workunit.client.1.smithi130.stdout:8/714: fsync d1/dc/d5d/f87 0 2024-05-13T22:48:40.144 INFO:tasks.workunit.client.1.smithi130.stdout:0/839: creat d0/d7/dd/d2e/da8/dcd/df8/f114 x:0 0 0 2024-05-13T22:48:40.145 INFO:tasks.workunit.client.1.smithi130.stdout:6/746: creat d4/d15/d1d/d28/d4e/f100 x:0 0 0 2024-05-13T22:48:40.154 INFO:tasks.workunit.client.1.smithi130.stdout:7/802: rename d7/da/d1d/d3d to d7/da/d1d/d5d/da6/d5b/d107 0 2024-05-13T22:48:40.154 INFO:tasks.workunit.client.1.smithi130.stdout:1/764: dread db/d4a/f4b [0,4194304] 0 2024-05-13T22:48:40.157 INFO:tasks.workunit.client.1.smithi130.stdout:3/799: creat d8/d1a/d55/f111 x:0 0 0 2024-05-13T22:48:40.160 INFO:tasks.workunit.client.1.smithi130.stdout:1/765: dwrite db/d82/dc3/fe2 [0,4194304] 0 2024-05-13T22:48:40.162 INFO:tasks.workunit.client.1.smithi130.stdout:4/770: rmdir d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5 39 2024-05-13T22:48:40.163 INFO:tasks.workunit.client.1.smithi130.stdout:3/800: dread d8/dc/d7b/da3/dd3/fc6 [0,4194304] 0 2024-05-13T22:48:40.163 INFO:tasks.workunit.client.1.smithi130.stdout:4/771: chown d8/dc/d26/fa1 36884 1 2024-05-13T22:48:40.164 INFO:tasks.workunit.client.1.smithi130.stdout:3/801: chown d8/d1a/dfa/d83/dba/dc4/lf9 120 1 2024-05-13T22:48:40.166 INFO:tasks.workunit.client.1.smithi130.stdout:8/715: creat d1/d9b/df3/f109 x:0 0 0 2024-05-13T22:48:40.167 INFO:tasks.workunit.client.1.smithi130.stdout:8/716: stat d1/d2/db/d15/d16/fb1 0 2024-05-13T22:48:40.172 INFO:tasks.workunit.client.1.smithi130.stdout:8/717: dwrite d1/d2/db/d15/d16/fb1 [0,4194304] 0 2024-05-13T22:48:40.173 INFO:tasks.workunit.client.1.smithi130.stdout:2/789: creat d4/d56/dff/d4a/f112 x:0 0 0 2024-05-13T22:48:40.176 INFO:tasks.workunit.client.1.smithi130.stdout:0/840: rmdir d0/d7/dd/d17/d52 39 2024-05-13T22:48:40.177 INFO:tasks.workunit.client.1.smithi130.stdout:0/841: write d0/d7/dd/d17/d1f/d2c/d5a/fb9 [8958430,49073] 0 2024-05-13T22:48:40.185 INFO:tasks.workunit.client.1.smithi130.stdout:6/747: rename d4/d15/d1d/l1e to d4/d15/d1d/d60/da2/da4/df8/l101 0 2024-05-13T22:48:40.188 INFO:tasks.workunit.client.1.smithi130.stdout:7/803: creat d7/da/d1d/db1/dbe/dd3/f108 x:0 0 0 2024-05-13T22:48:40.192 INFO:tasks.workunit.client.1.smithi130.stdout:1/766: unlink db/d1b/da3/d2a/l97 0 2024-05-13T22:48:40.193 INFO:tasks.workunit.client.1.smithi130.stdout:1/767: chown db/d1b/da3/d2a/d38/cd2 28703 1 2024-05-13T22:48:40.195 INFO:tasks.workunit.client.1.smithi130.stdout:4/772: chown d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/f3a 7674566 1 2024-05-13T22:48:40.196 INFO:tasks.workunit.client.1.smithi130.stdout:1/768: dread db/d1b/da3/fdd [0,4194304] 0 2024-05-13T22:48:40.197 INFO:tasks.workunit.client.1.smithi130.stdout:3/802: creat d8/d1a/dfa/d83/dba/dc4/f112 x:0 0 0 2024-05-13T22:48:40.201 INFO:tasks.workunit.client.1.smithi130.stdout:8/718: mkdir d1/d2/db/d15/d7e/dc2/d10a 0 2024-05-13T22:48:40.206 INFO:tasks.workunit.client.1.smithi130.stdout:9/816: link d1/d7/d23/d3d/c107 d1/c112 0 2024-05-13T22:48:40.211 INFO:tasks.workunit.client.1.smithi130.stdout:9/817: dread d1/f3 [0,4194304] 0 2024-05-13T22:48:40.212 INFO:tasks.workunit.client.1.smithi130.stdout:7/804: rename d7/da/d1d/de6 to d7/da/d1d/d5d/da6/d5b/d107/db9/d109 0 2024-05-13T22:48:40.214 INFO:tasks.workunit.client.1.smithi130.stdout:6/748: write d4/f9 [1823520,82551] 0 2024-05-13T22:48:40.217 INFO:tasks.workunit.client.1.smithi130.stdout:1/769: mknod db/d1b/da3/d72/daa/cf8 0 2024-05-13T22:48:40.218 INFO:tasks.workunit.client.1.smithi130.stdout:8/719: symlink d1/d2/db/d15/d18/d42/d53/dd0/l10b 0 2024-05-13T22:48:40.223 INFO:tasks.workunit.client.1.smithi130.stdout:0/842: dread d0/d7/dd/d17/d52/f9a [0,4194304] 0 2024-05-13T22:48:40.223 INFO:tasks.workunit.client.1.smithi130.stdout:1/770: dwrite db/d1b/da3/d2a/d38/d46/d6d/f73 [0,4194304] 0 2024-05-13T22:48:40.224 INFO:tasks.workunit.client.1.smithi130.stdout:9/818: mkdir d1/d7/d23/d3d/d43/d113 0 2024-05-13T22:48:40.225 INFO:tasks.workunit.client.1.smithi130.stdout:1/771: read db/d1b/da3/d2a/f5a [2270707,4101] 0 2024-05-13T22:48:40.226 INFO:tasks.workunit.client.1.smithi130.stdout:1/772: readlink db/d1b/da3/d2a/l57 0 2024-05-13T22:48:40.227 INFO:tasks.workunit.client.1.smithi130.stdout:4/773: rename d8/dc/d80/db0/df1/d33/d44/d56/d8c/fde to d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/ff8 0 2024-05-13T22:48:40.234 INFO:tasks.workunit.client.1.smithi130.stdout:7/805: dread d7/da/d1d/d5d/da6/f78 [0,4194304] 0 2024-05-13T22:48:40.236 INFO:tasks.workunit.client.1.smithi130.stdout:2/790: getdents d4/db/dbf/df9/d30/d88/dc5 0 2024-05-13T22:48:40.240 INFO:tasks.workunit.client.1.smithi130.stdout:9/819: creat d1/d7/d23/d28/d73/db2/f114 x:0 0 0 2024-05-13T22:48:40.240 INFO:tasks.workunit.client.1.smithi130.stdout:1/773: mkdir db/d1b/da3/d49/d56/d88/df9 0 2024-05-13T22:48:40.240 INFO:tasks.workunit.client.1.smithi130.stdout:4/774: dread d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/fd6 [0,4194304] 0 2024-05-13T22:48:40.241 INFO:tasks.workunit.client.1.smithi130.stdout:6/749: mkdir d4/d15/d1d/d102 0 2024-05-13T22:48:40.242 INFO:tasks.workunit.client.1.smithi130.stdout:7/806: readlink d7/da/d1d/d42/lae 0 2024-05-13T22:48:40.242 INFO:tasks.workunit.client.1.smithi130.stdout:1/774: write db/d1b/da3/f53 [844290,4234] 0 2024-05-13T22:48:40.246 INFO:tasks.workunit.client.1.smithi130.stdout:0/843: dwrite d0/d7/dd/d2e/d9e/da7/dc3/fdd [0,4194304] 0 2024-05-13T22:48:40.246 INFO:tasks.workunit.client.1.smithi130.stdout:6/750: write d4/d15/f31 [5252188,31649] 0 2024-05-13T22:48:40.248 INFO:tasks.workunit.client.1.smithi130.stdout:7/807: fsync d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/fc5 0 2024-05-13T22:48:40.250 INFO:tasks.workunit.client.1.smithi130.stdout:8/720: write d1/d2/f7f [805787,128111] 0 2024-05-13T22:48:40.251 INFO:tasks.workunit.client.1.smithi130.stdout:9/820: dwrite d1/d7/d23/d3d/d43/ffc [0,4194304] 0 2024-05-13T22:48:40.256 INFO:tasks.workunit.client.1.smithi130.stdout:4/775: fsync d8/f1f 0 2024-05-13T22:48:40.258 INFO:tasks.workunit.client.1.smithi130.stdout:2/791: write d4/db/dc/d18/d58/f70 [4884117,58370] 0 2024-05-13T22:48:40.259 INFO:tasks.workunit.client.1.smithi130.stdout:2/792: chown d4/d56/dff/d4a/ca2 10117 1 2024-05-13T22:48:40.263 INFO:tasks.workunit.client.1.smithi130.stdout:1/775: rename f8 to db/d1b/da3/d49/d56/d7f/ffa 0 2024-05-13T22:48:40.265 INFO:tasks.workunit.client.1.smithi130.stdout:9/821: unlink d1/d7/d23/d3d/d43/l89 0 2024-05-13T22:48:40.266 INFO:tasks.workunit.client.1.smithi130.stdout:1/776: truncate db/d1b/da3/d72/d75/dee/ff5 924403 0 2024-05-13T22:48:40.266 INFO:tasks.workunit.client.1.smithi130.stdout:9/822: chown d1/d7/d23/lcd 49258227 1 2024-05-13T22:48:40.266 INFO:tasks.workunit.client.1.smithi130.stdout:5/808: sync 2024-05-13T22:48:40.268 INFO:tasks.workunit.client.1.smithi130.stdout:9/823: read d1/db/ff2 [336158,55573] 0 2024-05-13T22:48:40.268 INFO:tasks.workunit.client.1.smithi130.stdout:5/809: truncate d6/d7/d32/f51 1680634 0 2024-05-13T22:48:40.269 INFO:tasks.workunit.client.1.smithi130.stdout:6/751: mkdir d4/d15/d103 0 2024-05-13T22:48:40.270 INFO:tasks.workunit.client.1.smithi130.stdout:4/776: fdatasync d8/dc/d80/db0/df1/d33/d3f/f6d 0 2024-05-13T22:48:40.271 INFO:tasks.workunit.client.1.smithi130.stdout:2/793: unlink d4/db/dc/d18/fca 0 2024-05-13T22:48:40.276 INFO:tasks.workunit.client.1.smithi130.stdout:1/777: unlink db/c52 0 2024-05-13T22:48:40.276 INFO:tasks.workunit.client.1.smithi130.stdout:1/778: fsync db/d4a/f95 0 2024-05-13T22:48:40.284 INFO:tasks.workunit.client.1.smithi130.stdout:6/752: rename d4/f57 to d4/d15/d1d/d28/f104 0 2024-05-13T22:48:40.284 INFO:tasks.workunit.client.1.smithi130.stdout:7/808: truncate d7/da/d1d/d5d/da6/d7e/d8b/fe8 2925507 0 2024-05-13T22:48:40.286 INFO:tasks.workunit.client.1.smithi130.stdout:7/809: readlink d7/da/d1d/d5d/da6/d5b/d107/d46/l60 0 2024-05-13T22:48:40.287 INFO:tasks.workunit.client.1.smithi130.stdout:6/753: dread d4/d15/d1d/d28/f71 [0,4194304] 0 2024-05-13T22:48:40.288 INFO:tasks.workunit.client.1.smithi130.stdout:6/754: stat d4/d15/f31 0 2024-05-13T22:48:40.288 INFO:tasks.workunit.client.1.smithi130.stdout:3/803: sync 2024-05-13T22:48:40.289 INFO:tasks.workunit.client.1.smithi130.stdout:3/804: write d8/dc/d7b/fcf [1019589,94637] 0 2024-05-13T22:48:40.290 INFO:tasks.workunit.client.1.smithi130.stdout:9/824: write d1/d7/d23/d28/d73/db2/f106 [1153327,22642] 0 2024-05-13T22:48:40.294 INFO:tasks.workunit.client.1.smithi130.stdout:8/721: link d1/d2/db/f45 d1/d2/db/d15/d7e/dcd/f10c 0 2024-05-13T22:48:40.298 INFO:tasks.workunit.client.1.smithi130.stdout:2/794: write d4/db/dc/d18/f23 [2375277,32408] 0 2024-05-13T22:48:40.300 INFO:tasks.workunit.client.1.smithi130.stdout:5/810: creat d6/dd/d18/d6e/dc7/df6/f10f x:0 0 0 2024-05-13T22:48:40.303 INFO:tasks.workunit.client.1.smithi130.stdout:7/810: unlink d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/f4d 0 2024-05-13T22:48:40.304 INFO:tasks.workunit.client.1.smithi130.stdout:4/777: mkdir d8/dc/d80/db0/df1/df9 0 2024-05-13T22:48:40.305 INFO:tasks.workunit.client.1.smithi130.stdout:6/755: unlink d4/d15/d1d/d28/lcb 0 2024-05-13T22:48:40.307 INFO:tasks.workunit.client.1.smithi130.stdout:0/844: getdents d0/d7/dd/d2e/da8 0 2024-05-13T22:48:40.308 INFO:tasks.workunit.client.1.smithi130.stdout:3/805: mkdir d8/dc/d15/dac/d113 0 2024-05-13T22:48:40.309 INFO:tasks.workunit.client.1.smithi130.stdout:3/806: truncate d8/d1a/d1c/fc1 370118 0 2024-05-13T22:48:40.312 INFO:tasks.workunit.client.1.smithi130.stdout:2/795: unlink d4/db/d26/cf5 0 2024-05-13T22:48:40.314 INFO:tasks.workunit.client.1.smithi130.stdout:2/796: chown d4/db/dbf/df9/f2f 1 1 2024-05-13T22:48:40.315 INFO:tasks.workunit.client.1.smithi130.stdout:3/807: dwrite d8/d1a/dfa/d2c/f46 [4194304,4194304] 0 2024-05-13T22:48:40.315 INFO:tasks.workunit.client.1.smithi130.stdout:5/811: creat d6/dd/d18/d6e/d76/f110 x:0 0 0 2024-05-13T22:48:40.317 INFO:tasks.workunit.client.1.smithi130.stdout:8/722: write d1/dc/f5c [622352,104926] 0 2024-05-13T22:48:40.319 INFO:tasks.workunit.client.1.smithi130.stdout:1/779: truncate db/d1b/da3/d2a/d38/fc5 652692 0 2024-05-13T22:48:40.327 INFO:tasks.workunit.client.1.smithi130.stdout:0/845: dread d0/d7/dd/d69/da9/d8d/f9d [0,4194304] 0 2024-05-13T22:48:40.327 INFO:tasks.workunit.client.1.smithi130.stdout:0/846: fdatasync d0/f1e 0 2024-05-13T22:48:40.335 INFO:tasks.workunit.client.1.smithi130.stdout:1/780: creat db/d1b/da3/d2a/d35/ffb x:0 0 0 2024-05-13T22:48:40.339 INFO:tasks.workunit.client.1.smithi130.stdout:6/756: mknod d4/d15/d1f/d46/d52/c105 0 2024-05-13T22:48:40.340 INFO:tasks.workunit.client.1.smithi130.stdout:6/757: truncate d4/d15/d1d/daa/fc2 968335 0 2024-05-13T22:48:40.341 INFO:tasks.workunit.client.1.smithi130.stdout:6/758: truncate d4/d15/d1f/d46/d50/f8a 535285 0 2024-05-13T22:48:40.342 INFO:tasks.workunit.client.1.smithi130.stdout:5/812: mknod d6/c111 0 2024-05-13T22:48:40.342 INFO:tasks.workunit.client.1.smithi130.stdout:3/808: write d8/d1a/d1c/fa0 [437989,64291] 0 2024-05-13T22:48:40.344 INFO:tasks.workunit.client.1.smithi130.stdout:3/809: read d8/dc/d7b/da3/dd3/f8c [3495822,17802] 0 2024-05-13T22:48:40.355 INFO:tasks.workunit.client.1.smithi130.stdout:7/811: mkdir d7/da/d1d/d42/dc0/d10a 0 2024-05-13T22:48:40.359 INFO:tasks.workunit.client.1.smithi130.stdout:2/797: truncate d4/db/dc/d18/d58/f59 6726923 0 2024-05-13T22:48:40.360 INFO:tasks.workunit.client.1.smithi130.stdout:4/778: rename d8/dc/d80/db0/df1/d33/d44/d61/da0 to d8/dfa 0 2024-05-13T22:48:40.366 INFO:tasks.workunit.client.1.smithi130.stdout:3/810: mkdir d8/dc/d7b/da3/dd3/ddd/d114 0 2024-05-13T22:48:40.368 INFO:tasks.workunit.client.1.smithi130.stdout:8/723: creat d1/d2/db/d15/f10d x:0 0 0 2024-05-13T22:48:40.368 INFO:tasks.workunit.client.1.smithi130.stdout:1/781: dwrite db/d1b/da3/d49/d56/f7e [0,4194304] 0 2024-05-13T22:48:40.369 INFO:tasks.workunit.client.1.smithi130.stdout:3/811: truncate d8/dc/d7b/da3/dd3/f81 4737138 0 2024-05-13T22:48:40.370 INFO:tasks.workunit.client.1.smithi130.stdout:3/812: fsync d8/d1a/dfa/d83/dba/dc4/fd9 0 2024-05-13T22:48:40.379 INFO:tasks.workunit.client.1.smithi130.stdout:0/847: rename d0/d7/dd/d2e/f3f to d0/d7/dd/d17/d1f/d106/f115 0 2024-05-13T22:48:40.382 INFO:tasks.workunit.client.1.smithi130.stdout:5/813: symlink d6/dd/d18/d6e/d82/d10b/l112 0 2024-05-13T22:48:40.392 INFO:tasks.workunit.client.1.smithi130.stdout:3/813: dread d8/dc/d7b/da3/fbe [0,4194304] 0 2024-05-13T22:48:40.393 INFO:tasks.workunit.client.1.smithi130.stdout:3/814: fsync d8/d1a/d55/f111 0 2024-05-13T22:48:40.395 INFO:tasks.workunit.client.1.smithi130.stdout:0/848: symlink d0/d7/dd/d17/d52/d5b/d60/l116 0 2024-05-13T22:48:40.397 INFO:tasks.workunit.client.1.smithi130.stdout:5/814: mknod d6/d7/d19/d4d/d6a/ddf/c113 0 2024-05-13T22:48:40.400 INFO:tasks.workunit.client.1.smithi130.stdout:2/798: creat d4/db/dbf/f113 x:0 0 0 2024-05-13T22:48:40.400 INFO:tasks.workunit.client.1.smithi130.stdout:2/799: fdatasync d4/db/d26/f2d 0 2024-05-13T22:48:40.404 INFO:tasks.workunit.client.1.smithi130.stdout:3/815: creat d8/dc/d7b/da3/dd3/f115 x:0 0 0 2024-05-13T22:48:40.405 INFO:tasks.workunit.client.1.smithi130.stdout:3/816: read d8/fa [511371,104921] 0 2024-05-13T22:48:40.405 INFO:tasks.workunit.client.1.smithi130.stdout:3/817: fsync d8/d1a/dfa/d2c/f6b 0 2024-05-13T22:48:40.415 INFO:tasks.workunit.client.1.smithi130.stdout:4/779: rmdir d8/dc/d80/db0/df1/df9 0 2024-05-13T22:48:40.417 INFO:tasks.workunit.client.1.smithi130.stdout:4/780: write d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dad/fb5 [1792072,3319] 0 2024-05-13T22:48:40.418 INFO:tasks.workunit.client.1.smithi130.stdout:4/781: write d8/dc/d80/db0/df1/f50 [2647135,43387] 0 2024-05-13T22:48:40.423 INFO:tasks.workunit.client.1.smithi130.stdout:5/815: unlink d6/d1a/f48 0 2024-05-13T22:48:40.423 INFO:tasks.workunit.client.1.smithi130.stdout:4/782: dwrite d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d81/f9e [4194304,4194304] 0 2024-05-13T22:48:40.424 INFO:tasks.workunit.client.1.smithi130.stdout:2/800: write d4/db/dc/d18/f36 [626142,62609] 0 2024-05-13T22:48:40.425 INFO:tasks.workunit.client.1.smithi130.stdout:2/801: fdatasync d4/db/dbf/df9/d30/fb3 0 2024-05-13T22:48:40.428 INFO:tasks.workunit.client.1.smithi130.stdout:7/812: link d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/l47 d7/da/d1d/d5d/da6/d5b/d107/db9/d109/df4/l10b 0 2024-05-13T22:48:40.431 INFO:tasks.workunit.client.1.smithi130.stdout:6/759: rename d4/d15/d1f/d46/d52/c86 to d4/d15/d1d/d60/da2/c106 0 2024-05-13T22:48:40.432 INFO:tasks.workunit.client.1.smithi130.stdout:2/802: dwrite d4/db/dbf/df9/f44 [4194304,4194304] 0 2024-05-13T22:48:40.436 INFO:tasks.workunit.client.1.smithi130.stdout:0/849: link d0/d7/dd/d69/da9/fbc d0/d7/dd/d17/d52/d5b/d102/f117 0 2024-05-13T22:48:40.437 INFO:tasks.workunit.client.1.smithi130.stdout:0/850: chown d0/d7/dd/d2e/d9e/da7/dc3/fdd 143382 1 2024-05-13T22:48:40.443 INFO:tasks.workunit.client.1.smithi130.stdout:4/783: unlink d8/dc/d80/db0/df1/d33/d44/db8/l9d 0 2024-05-13T22:48:40.448 INFO:tasks.workunit.client.1.smithi130.stdout:1/782: rename db/d1b/da3/d49/f54 to db/d4a/ffc 0 2024-05-13T22:48:40.449 INFO:tasks.workunit.client.1.smithi130.stdout:1/783: chown db/d1b/da3/d49/d56/c6c 80827 1 2024-05-13T22:48:40.449 INFO:tasks.workunit.client.1.smithi130.stdout:9/825: sync 2024-05-13T22:48:40.450 INFO:tasks.workunit.client.1.smithi130.stdout:1/784: dread - db/d1b/da3/d2a/f8e zero size 2024-05-13T22:48:40.451 INFO:tasks.workunit.client.1.smithi130.stdout:6/760: mknod d4/d15/d1f/d3c/d70/c107 0 2024-05-13T22:48:40.455 INFO:tasks.workunit.client.1.smithi130.stdout:6/761: chown d4/d15/d1f/d46/d50/f54 11 1 2024-05-13T22:48:40.455 INFO:tasks.workunit.client.1.smithi130.stdout:2/803: symlink d4/db/dbf/df9/d30/d88/dc5/l114 0 2024-05-13T22:48:40.455 INFO:tasks.workunit.client.1.smithi130.stdout:3/818: write d8/d1a/d1c/fb4 [399303,118967] 0 2024-05-13T22:48:40.455 INFO:tasks.workunit.client.1.smithi130.stdout:2/804: chown d4/db/dbf/df9/d30/d88/d99/d9a/ff3 307799 1 2024-05-13T22:48:40.459 INFO:tasks.workunit.client.1.smithi130.stdout:5/816: link d6/dd/d18/fd7 d6/d1a/daf/f114 0 2024-05-13T22:48:40.459 INFO:tasks.workunit.client.1.smithi130.stdout:7/813: mknod d7/da/d1d/d42/dc0/d10a/c10c 0 2024-05-13T22:48:40.460 INFO:tasks.workunit.client.1.smithi130.stdout:5/817: chown d6/d7/d19/f52 3806 1 2024-05-13T22:48:40.460 INFO:tasks.workunit.client.1.smithi130.stdout:7/814: chown d7/da/d1d/db1/dd4 81 1 2024-05-13T22:48:40.461 INFO:tasks.workunit.client.1.smithi130.stdout:5/818: chown d6/d2f/ff8 1004750851 1 2024-05-13T22:48:40.461 INFO:tasks.workunit.client.1.smithi130.stdout:7/815: chown d7/da/d1d/d5d/da6/d9/c9b 38446 1 2024-05-13T22:48:40.463 INFO:tasks.workunit.client.1.smithi130.stdout:9/826: unlink d1/d7/d23/d28/fab 0 2024-05-13T22:48:40.468 INFO:tasks.workunit.client.1.smithi130.stdout:6/762: chown d4/d15/d1d/d28/d75/da8/cc7 24250901 1 2024-05-13T22:48:40.471 INFO:tasks.workunit.client.1.smithi130.stdout:8/724: sync 2024-05-13T22:48:40.473 INFO:tasks.workunit.client.1.smithi130.stdout:8/725: write d1/d2/db/d15/d18/d42/d53/dd0/fd3 [558694,14658] 0 2024-05-13T22:48:40.474 INFO:tasks.workunit.client.1.smithi130.stdout:8/726: chown d1/d2/db/d15/d7e/dc2/d10a 14457686 1 2024-05-13T22:48:40.512 INFO:tasks.workunit.client.1.smithi130.stdout:4/784: creat d8/dfa/dbd/ffb x:0 0 0 2024-05-13T22:48:40.514 INFO:tasks.workunit.client.1.smithi130.stdout:4/785: stat d8/dc/d80/db0/df1/d33/d44/db8/df7 0 2024-05-13T22:48:40.515 INFO:tasks.workunit.client.1.smithi130.stdout:2/805: rename d4/db/dbf/df9/f2f to d4/db/dbf/df9/d30/d88/d99/d9a/f115 0 2024-05-13T22:48:40.516 INFO:tasks.workunit.client.1.smithi130.stdout:5/819: mkdir d6/dd/d18/d6e/dc7/d115 0 2024-05-13T22:48:40.517 INFO:tasks.workunit.client.1.smithi130.stdout:7/816: symlink d7/da/d1d/d5d/da6/d9/d66/d65/l10d 0 2024-05-13T22:48:40.518 INFO:tasks.workunit.client.1.smithi130.stdout:1/785: dwrite db/f1d [0,4194304] 0 2024-05-13T22:48:40.519 INFO:tasks.workunit.client.1.smithi130.stdout:7/817: chown d7/da/d1d/d5d/da6/d39/f7c 21 1 2024-05-13T22:48:40.520 INFO:tasks.workunit.client.1.smithi130.stdout:9/827: unlink d1/d7/d23/d28/d73/d8d/deb/ffe 0 2024-05-13T22:48:40.522 INFO:tasks.workunit.client.1.smithi130.stdout:6/763: rmdir d4/d15/d1f/d64 39 2024-05-13T22:48:40.525 INFO:tasks.workunit.client.1.smithi130.stdout:1/786: dwrite db/f1d [0,4194304] 0 2024-05-13T22:48:40.526 INFO:tasks.workunit.client.1.smithi130.stdout:1/787: chown db/d1b/fe6 2699158 1 2024-05-13T22:48:40.527 INFO:tasks.workunit.client.1.smithi130.stdout:8/727: mkdir d1/d2/db/d15/d18/d42/d53/da4/d10e 0 2024-05-13T22:48:40.531 INFO:tasks.workunit.client.1.smithi130.stdout:5/820: symlink d6/dd/d18/d6e/d82/d71/l116 0 2024-05-13T22:48:40.537 INFO:tasks.workunit.client.1.smithi130.stdout:6/764: write d4/d15/d1f/d64/fff [599614,88622] 0 2024-05-13T22:48:40.538 INFO:tasks.workunit.client.1.smithi130.stdout:3/819: rmdir d8/dc/d106 0 2024-05-13T22:48:40.540 INFO:tasks.workunit.client.1.smithi130.stdout:8/728: fsync d1/d2/db/d15/f74 0 2024-05-13T22:48:40.541 INFO:tasks.workunit.client.1.smithi130.stdout:5/821: mknod d6/dd/d18/d6e/d82/db0/c117 0 2024-05-13T22:48:40.544 INFO:tasks.workunit.client.1.smithi130.stdout:9/828: mknod d1/d7/d23/d28/c115 0 2024-05-13T22:48:40.547 INFO:tasks.workunit.client.1.smithi130.stdout:3/820: rename d8/fe5 to d8/dc/d15/f116 0 2024-05-13T22:48:40.548 INFO:tasks.workunit.client.1.smithi130.stdout:1/788: mknod db/d1b/da3/d49/d56/da4/cfd 0 2024-05-13T22:48:40.549 INFO:tasks.workunit.client.1.smithi130.stdout:1/789: truncate db/d1b/da3/d49/fb8 751647 0 2024-05-13T22:48:40.550 INFO:tasks.workunit.client.1.smithi130.stdout:8/729: fsync d1/d2/db/d15/d16/d54/d58/db9/fce 0 2024-05-13T22:48:40.551 INFO:tasks.workunit.client.1.smithi130.stdout:1/790: fsync db/d1b/da3/f89 0 2024-05-13T22:48:40.551 INFO:tasks.workunit.client.1.smithi130.stdout:9/829: read d1/d7/d23/d28/d2d/f46 [251648,92843] 0 2024-05-13T22:48:40.551 INFO:tasks.workunit.client.1.smithi130.stdout:6/765: fdatasync d4/d15/d1f/d46/d50/f51 0 2024-05-13T22:48:40.552 INFO:tasks.workunit.client.1.smithi130.stdout:8/730: write d1/dc/f44 [3716722,50538] 0 2024-05-13T22:48:40.552 INFO:tasks.workunit.client.1.smithi130.stdout:7/818: dread d7/da/d1d/d5d/da6/d39/f7b [0,4194304] 0 2024-05-13T22:48:40.554 INFO:tasks.workunit.client.1.smithi130.stdout:8/731: fsync d1/d2/db/f46 0 2024-05-13T22:48:40.561 INFO:tasks.workunit.client.1.smithi130.stdout:8/732: read d1/d2/db/d15/d18/f60 [1209934,52163] 0 2024-05-13T22:48:40.561 INFO:tasks.workunit.client.1.smithi130.stdout:7/819: truncate d7/da/d1d/d5d/da6/d39/da9/fb0 60118 0 2024-05-13T22:48:40.562 INFO:tasks.workunit.client.1.smithi130.stdout:7/820: dread - d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/ff1 zero size 2024-05-13T22:48:40.562 INFO:tasks.workunit.client.1.smithi130.stdout:6/766: mkdir d4/d108 0 2024-05-13T22:48:40.563 INFO:tasks.workunit.client.1.smithi130.stdout:9/830: mknod d1/d7/d23/d28/d73/de6/c116 0 2024-05-13T22:48:40.566 INFO:tasks.workunit.client.1.smithi130.stdout:6/767: mknod d4/d15/d1d/d60/da2/da4/ddd/c109 0 2024-05-13T22:48:40.568 INFO:tasks.workunit.client.1.smithi130.stdout:1/791: dread db/d1b/da3/d2a/d38/d46/d6d/f9e [0,4194304] 0 2024-05-13T22:48:40.569 INFO:tasks.workunit.client.1.smithi130.stdout:7/821: dwrite d7/da/d1d/d5d/f89 [0,4194304] 0 2024-05-13T22:48:40.570 INFO:tasks.workunit.client.1.smithi130.stdout:7/822: stat d7/da/d1d/d5d/f89 0 2024-05-13T22:48:40.575 INFO:tasks.workunit.client.1.smithi130.stdout:0/851: truncate d0/d7/dd/d17/d1f/d2c/d45/d57/df7/fff 3446545 0 2024-05-13T22:48:40.578 INFO:tasks.workunit.client.1.smithi130.stdout:0/852: dread d0/d7/dd/d69/fc8 [0,4194304] 0 2024-05-13T22:48:40.579 INFO:tasks.workunit.client.1.smithi130.stdout:7/823: dwrite d7/da/f62 [0,4194304] 0 2024-05-13T22:48:40.579 INFO:tasks.workunit.client.1.smithi130.stdout:1/792: dread db/d1b/da3/d2a/d35/f40 [0,4194304] 0 2024-05-13T22:48:40.581 INFO:tasks.workunit.client.1.smithi130.stdout:2/806: write d4/db/dbf/df9/d30/d88/faa [915868,92990] 0 2024-05-13T22:48:40.586 INFO:tasks.workunit.client.1.smithi130.stdout:1/793: mknod db/d1b/da3/d72/cfe 0 2024-05-13T22:48:40.586 INFO:tasks.workunit.client.1.smithi130.stdout:1/794: read - db/d1b/da3/fce zero size 2024-05-13T22:48:40.589 INFO:tasks.workunit.client.1.smithi130.stdout:0/853: symlink d0/d7/dd/d17/d1f/d106/l118 0 2024-05-13T22:48:40.593 INFO:tasks.workunit.client.1.smithi130.stdout:0/854: read d0/d7/dd/d2e/da8/fc0 [256682,107458] 0 2024-05-13T22:48:40.595 INFO:tasks.workunit.client.1.smithi130.stdout:4/786: truncate d8/dc/d80/db0/df1/d33/d44/d56/fc5 414076 0 2024-05-13T22:48:40.606 INFO:tasks.workunit.client.1.smithi130.stdout:6/768: dwrite d4/d15/d1f/d46/d50/f51 [0,4194304] 0 2024-05-13T22:48:40.607 INFO:tasks.workunit.client.1.smithi130.stdout:0/855: creat d0/d7/dd/d17/d1f/f119 x:0 0 0 2024-05-13T22:48:40.607 INFO:tasks.workunit.client.1.smithi130.stdout:4/787: creat d8/dfa/dbd/ffc x:0 0 0 2024-05-13T22:48:40.608 INFO:tasks.workunit.client.1.smithi130.stdout:2/807: dread d4/db/f28 [0,4194304] 0 2024-05-13T22:48:40.609 INFO:tasks.workunit.client.1.smithi130.stdout:4/788: chown d8/dc/d84/ff0 561 1 2024-05-13T22:48:40.611 INFO:tasks.workunit.client.1.smithi130.stdout:4/789: chown d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d81/fce 3454 1 2024-05-13T22:48:40.617 INFO:tasks.workunit.client.1.smithi130.stdout:4/790: dwrite d8/dc/d26/f52 [0,4194304] 0 2024-05-13T22:48:40.623 INFO:tasks.workunit.client.1.smithi130.stdout:6/769: symlink d4/d15/d1d/d28/d75/d7d/l10a 0 2024-05-13T22:48:40.624 INFO:tasks.workunit.client.1.smithi130.stdout:6/770: chown d4/d15/d1d/d60/da2 6162 1 2024-05-13T22:48:40.629 INFO:tasks.workunit.client.1.smithi130.stdout:9/831: dwrite d1/d7/d23/d28/d73/ff5 [0,4194304] 0 2024-05-13T22:48:40.630 INFO:tasks.workunit.client.1.smithi130.stdout:9/832: readlink d1/db/l13 0 2024-05-13T22:48:40.634 INFO:tasks.workunit.client.1.smithi130.stdout:7/824: dwrite d7/f83 [0,4194304] 0 2024-05-13T22:48:40.637 INFO:tasks.workunit.client.1.smithi130.stdout:2/808: dread d4/db/dc/f15 [0,4194304] 0 2024-05-13T22:48:40.637 INFO:tasks.workunit.client.1.smithi130.stdout:3/821: rmdir d8/dc/d7b/da3 39 2024-05-13T22:48:40.639 INFO:tasks.workunit.client.1.smithi130.stdout:6/771: stat d4/d15/d1d/d28/f104 0 2024-05-13T22:48:40.646 INFO:tasks.workunit.client.1.smithi130.stdout:9/833: symlink d1/d7/d23/d28/d73/db2/l117 0 2024-05-13T22:48:40.650 INFO:tasks.workunit.client.1.smithi130.stdout:6/772: dread d4/d15/d1d/d28/f47 [0,4194304] 0 2024-05-13T22:48:40.650 INFO:tasks.workunit.client.1.smithi130.stdout:3/822: chown d8/d1a/dfa/d83/dba/dc4/lf9 15521 1 2024-05-13T22:48:40.652 INFO:tasks.workunit.client.1.smithi130.stdout:3/823: write d8/d1a/d1c/f5d [3411810,55625] 0 2024-05-13T22:48:40.653 INFO:tasks.workunit.client.1.smithi130.stdout:3/824: fsync d8/dc/f21 0 2024-05-13T22:48:40.654 INFO:tasks.workunit.client.1.smithi130.stdout:1/795: write db/d1b/da3/d49/d56/d7f/ffa [204202,99227] 0 2024-05-13T22:48:40.658 INFO:tasks.workunit.client.1.smithi130.stdout:1/796: truncate db/d1b/da3/d49/fb8 1545547 0 2024-05-13T22:48:40.661 INFO:tasks.workunit.client.1.smithi130.stdout:4/791: dread d8/dc/d80/db0/df1/f8a [0,4194304] 0 2024-05-13T22:48:40.663 INFO:tasks.workunit.client.1.smithi130.stdout:1/797: dwrite db/d1b/da3/f53 [4194304,4194304] 0 2024-05-13T22:48:40.664 INFO:tasks.workunit.client.1.smithi130.stdout:3/825: dread d8/dc/d15/f66 [0,4194304] 0 2024-05-13T22:48:40.666 INFO:tasks.workunit.client.1.smithi130.stdout:6/773: creat d4/d15/d1d/d60/da2/f10b x:0 0 0 2024-05-13T22:48:40.669 INFO:tasks.workunit.client.1.smithi130.stdout:9/834: getdents d1/d7/d23/d28/d73 0 2024-05-13T22:48:40.669 INFO:tasks.workunit.client.1.smithi130.stdout:9/835: read - d1/d7/d23/d28/d73/db2/f114 zero size 2024-05-13T22:48:40.674 INFO:tasks.workunit.client.1.smithi130.stdout:1/798: mknod db/d1b/da3/d2a/d38/d46/d6d/cff 0 2024-05-13T22:48:40.674 INFO:tasks.workunit.client.1.smithi130.stdout:2/809: dwrite d4/fa [0,4194304] 0 2024-05-13T22:48:40.676 INFO:tasks.workunit.client.1.smithi130.stdout:7/825: truncate d7/da/d1d/d5d/da6/d5b/f6d 899296 0 2024-05-13T22:48:40.678 INFO:tasks.workunit.client.1.smithi130.stdout:7/826: write d7/da/d1d/f9a [1460617,112491] 0 2024-05-13T22:48:40.681 INFO:tasks.workunit.client.1.smithi130.stdout:9/836: mknod d1/d7/d23/d28/d73/d8d/c118 0 2024-05-13T22:48:40.682 INFO:tasks.workunit.client.1.smithi130.stdout:2/810: creat d4/db/dbf/df9/d30/d88/dc5/f116 x:0 0 0 2024-05-13T22:48:40.684 INFO:tasks.workunit.client.1.smithi130.stdout:2/811: chown d4/db/dbf/d42/d4b/f4f 3 1 2024-05-13T22:48:40.685 INFO:tasks.workunit.client.1.smithi130.stdout:5/822: sync 2024-05-13T22:48:40.685 INFO:tasks.workunit.client.1.smithi130.stdout:6/774: truncate d4/d15/d1f/d46/d52/d7b/f9d 698434 0 2024-05-13T22:48:40.689 INFO:tasks.workunit.client.1.smithi130.stdout:9/837: rmdir d1/d7/d23/d3d/d43 39 2024-05-13T22:48:40.692 INFO:tasks.workunit.client.1.smithi130.stdout:2/812: creat d4/da8/dc9/dbb/f117 x:0 0 0 2024-05-13T22:48:40.693 INFO:tasks.workunit.client.1.smithi130.stdout:2/813: readlink d4/db/dbf/df9/d66/l7c 0 2024-05-13T22:48:40.694 INFO:tasks.workunit.client.1.smithi130.stdout:5/823: symlink d6/dd/d18/d6e/d82/d4b/l118 0 2024-05-13T22:48:40.695 INFO:tasks.workunit.client.1.smithi130.stdout:6/775: creat d4/d15/d1d/d28/d4e/d9e/f10c x:0 0 0 2024-05-13T22:48:40.697 INFO:tasks.workunit.client.1.smithi130.stdout:6/776: chown d4/d15/d1d/d60/da2 1575638 1 2024-05-13T22:48:40.697 INFO:tasks.workunit.client.1.smithi130.stdout:3/826: dread d8/dc/d7b/da3/dd3/f74 [0,4194304] 0 2024-05-13T22:48:40.698 INFO:tasks.workunit.client.1.smithi130.stdout:1/799: getdents db/d1b/da3/d72/d75/dee 0 2024-05-13T22:48:40.702 INFO:tasks.workunit.client.1.smithi130.stdout:3/827: dwrite d8/dc/d15/dfd/fe7 [0,4194304] 0 2024-05-13T22:48:40.711 INFO:tasks.workunit.client.1.smithi130.stdout:2/814: symlink d4/db/dbf/df9/d30/d88/l118 0 2024-05-13T22:48:40.718 INFO:tasks.workunit.client.1.smithi130.stdout:3/828: rename d8/d1a/d1c/f94 to d8/dc/d15/dac/f117 0 2024-05-13T22:48:40.719 INFO:tasks.workunit.client.1.smithi130.stdout:7/827: write d7/da/d1d/d5d/da6/d5b/fb3 [607554,77147] 0 2024-05-13T22:48:40.720 INFO:tasks.workunit.client.1.smithi130.stdout:7/828: dread - d7/da/d1d/d5d/da6/d9/d66/d65/fff zero size 2024-05-13T22:48:40.721 INFO:tasks.workunit.client.1.smithi130.stdout:1/800: write db/f6a [26305,24425] 0 2024-05-13T22:48:40.724 INFO:tasks.workunit.client.1.smithi130.stdout:2/815: unlink d4/db/dc/d18/l6c 0 2024-05-13T22:48:40.724 INFO:tasks.workunit.client.1.smithi130.stdout:2/816: stat d4/db/dc/ld3 0 2024-05-13T22:48:40.725 INFO:tasks.workunit.client.1.smithi130.stdout:2/817: write d4/db/dbf/df9/d30/fd8 [3541774,55951] 0 2024-05-13T22:48:40.728 INFO:tasks.workunit.client.1.smithi130.stdout:3/829: creat d8/d1a/dfa/d83/dba/f118 x:0 0 0 2024-05-13T22:48:40.730 INFO:tasks.workunit.client.1.smithi130.stdout:7/829: creat d7/da/d1d/db1/dbe/dd3/f10e x:0 0 0 2024-05-13T22:48:40.733 INFO:tasks.workunit.client.1.smithi130.stdout:9/838: getdents d1/d7/d23/d28/d2d 0 2024-05-13T22:48:40.737 INFO:tasks.workunit.client.1.smithi130.stdout:5/824: getdents d6/d1a/d72 0 2024-05-13T22:48:40.740 INFO:tasks.workunit.client.1.smithi130.stdout:6/777: getdents d4/d15/d1f/d3c/d70 0 2024-05-13T22:48:40.740 INFO:tasks.workunit.client.1.smithi130.stdout:2/818: fsync d4/db/dbf/d42/d4b/fb5 0 2024-05-13T22:48:40.741 INFO:tasks.workunit.client.1.smithi130.stdout:2/819: write d4/d56/dff/d4a/f112 [943139,59693] 0 2024-05-13T22:48:40.741 INFO:tasks.workunit.client.1.smithi130.stdout:3/830: fsync d8/dc/d15/f9a 0 2024-05-13T22:48:40.747 INFO:tasks.workunit.client.1.smithi130.stdout:3/831: dwrite d8/d1a/dfa/ff3 [0,4194304] 0 2024-05-13T22:48:40.749 INFO:tasks.workunit.client.1.smithi130.stdout:9/839: mknod d1/d7/d23/d28/d73/d8d/c119 0 2024-05-13T22:48:40.749 INFO:tasks.workunit.client.1.smithi130.stdout:7/830: write d7/da/d1d/d5d/da6/d9/d4a/dbd/f6c [1459338,263] 0 2024-05-13T22:48:40.754 INFO:tasks.workunit.client.1.smithi130.stdout:1/801: truncate db/d1b/da3/d72/f79 356651 0 2024-05-13T22:48:40.756 INFO:tasks.workunit.client.1.smithi130.stdout:2/820: symlink d4/db/dbf/df9/l119 0 2024-05-13T22:48:40.757 INFO:tasks.workunit.client.1.smithi130.stdout:9/840: dwrite d1/d15/d51/fd7 [0,4194304] 0 2024-05-13T22:48:40.757 INFO:tasks.workunit.client.1.smithi130.stdout:7/831: dwrite d7/da/d1d/d5d/d70/f85 [0,4194304] 0 2024-05-13T22:48:40.759 INFO:tasks.workunit.client.1.smithi130.stdout:7/832: fdatasync d7/da/d1d/d42/f73 0 2024-05-13T22:48:40.760 INFO:tasks.workunit.client.1.smithi130.stdout:6/778: symlink d4/d15/d1f/d46/l10d 0 2024-05-13T22:48:40.764 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:40 smithi102 ceph-mon[28599]: pgmap v37: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 92 MiB/s rd, 276 MiB/s wr, 560 op/s 2024-05-13T22:48:40.769 INFO:tasks.workunit.client.1.smithi130.stdout:2/821: dwrite d4/d56/dff/d4a/d76/fdd [0,4194304] 0 2024-05-13T22:48:40.770 INFO:tasks.workunit.client.1.smithi130.stdout:3/832: symlink d8/dc/d7b/l119 0 2024-05-13T22:48:40.771 INFO:tasks.workunit.client.1.smithi130.stdout:1/802: mkdir db/d1b/da3/d49/d56/d88/df9/d100 0 2024-05-13T22:48:40.772 INFO:tasks.workunit.client.1.smithi130.stdout:9/841: fsync d1/fc9 0 2024-05-13T22:48:40.773 INFO:tasks.workunit.client.1.smithi130.stdout:3/833: truncate d8/d1a/d55/fd7 234712 0 2024-05-13T22:48:40.783 INFO:tasks.workunit.client.1.smithi130.stdout:1/803: creat db/d1b/da3/d72/daa/f101 x:0 0 0 2024-05-13T22:48:40.786 INFO:tasks.workunit.client.1.smithi130.stdout:3/834: write d8/dc/d7b/da3/dd3/f74 [3564050,68615] 0 2024-05-13T22:48:40.787 INFO:tasks.workunit.client.1.smithi130.stdout:8/733: sync 2024-05-13T22:48:40.788 INFO:tasks.workunit.client.1.smithi130.stdout:7/833: link d7/da/d1d/d5d/d70/f8c d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/f10f 0 2024-05-13T22:48:40.789 INFO:tasks.workunit.client.1.smithi130.stdout:1/804: mknod db/d1b/da3/d49/d56/d88/c102 0 2024-05-13T22:48:40.790 INFO:tasks.workunit.client.1.smithi130.stdout:7/834: fdatasync d7/da/d1d/d5d/da6/d9/d66/ffe 0 2024-05-13T22:48:40.791 INFO:tasks.workunit.client.1.smithi130.stdout:1/805: write db/d1b/da3/d2a/d35/fba [405530,69637] 0 2024-05-13T22:48:40.791 INFO:tasks.workunit.client.1.smithi130.stdout:9/842: rename d1/d7/d23/d3d/d43/f82 to d1/d7/dd2/f11a 0 2024-05-13T22:48:40.792 INFO:tasks.workunit.client.1.smithi130.stdout:3/835: unlink d8/d1a/d55/l57 0 2024-05-13T22:48:40.793 INFO:tasks.workunit.client.1.smithi130.stdout:8/734: mkdir d1/d2/db/d15/d18/d42/d53/de8/d10f 0 2024-05-13T22:48:40.793 INFO:tasks.workunit.client.1.smithi130.stdout:7/835: write d7/da/d1d/d5d/da6/d5b/d107/d46/d92/faf [4727055,42800] 0 2024-05-13T22:48:40.797 INFO:tasks.workunit.client.1.smithi130.stdout:6/779: dread d4/d15/d1d/d28/f104 [0,4194304] 0 2024-05-13T22:48:40.803 INFO:tasks.workunit.client.1.smithi130.stdout:2/822: write d4/d56/dff/d4a/dc6/f6f [61100,57714] 0 2024-05-13T22:48:40.805 INFO:tasks.workunit.client.1.smithi130.stdout:8/735: mkdir d1/d2/db/d15/d16/d54/dfb/d110 0 2024-05-13T22:48:40.806 INFO:tasks.workunit.client.1.smithi130.stdout:7/836: dwrite d7/da/d1d/d5d/da6/d39/f79 [0,4194304] 0 2024-05-13T22:48:40.810 INFO:tasks.workunit.client.1.smithi130.stdout:1/806: dwrite db/d1b/da3/d2a/d4c/dc2/fd7 [0,4194304] 0 2024-05-13T22:48:40.811 INFO:tasks.workunit.client.1.smithi130.stdout:3/836: dwrite d8/d1a/d55/fe2 [0,4194304] 0 2024-05-13T22:48:40.812 INFO:tasks.workunit.client.1.smithi130.stdout:1/807: readlink db/d1b/lcf 0 2024-05-13T22:48:40.813 INFO:tasks.workunit.client.1.smithi130.stdout:9/843: dread d1/f4f [0,4194304] 0 2024-05-13T22:48:40.815 INFO:tasks.workunit.client.1.smithi130.stdout:2/823: creat d4/db/dbf/df9/d30/d88/dc5/f11a x:0 0 0 2024-05-13T22:48:40.817 INFO:tasks.workunit.client.1.smithi130.stdout:7/837: fdatasync d7/da/d1d/d5d/da6/d39/fa8 0 2024-05-13T22:48:40.820 INFO:tasks.workunit.client.1.smithi130.stdout:9/844: rmdir d1/d7/dd2 39 2024-05-13T22:48:40.821 INFO:tasks.workunit.client.1.smithi130.stdout:8/736: dwrite d1/d2/db/f8 [4194304,4194304] 0 2024-05-13T22:48:40.821 INFO:tasks.workunit.client.1.smithi130.stdout:2/824: truncate d4/db/dbf/d42/d4b/fdc 1704226 0 2024-05-13T22:48:40.822 INFO:tasks.workunit.client.1.smithi130.stdout:8/737: dread - d1/d2/db/d15/d18/dba/fef zero size 2024-05-13T22:48:40.823 INFO:tasks.workunit.client.1.smithi130.stdout:8/738: read - d1/d2/db/d15/fc0 zero size 2024-05-13T22:48:40.825 INFO:tasks.workunit.client.1.smithi130.stdout:1/808: creat db/d1b/da3/d72/d75/dee/df6/f103 x:0 0 0 2024-05-13T22:48:40.827 INFO:tasks.workunit.client.1.smithi130.stdout:3/837: symlink d8/dc/d15/dac/d113/l11a 0 2024-05-13T22:48:40.829 INFO:tasks.workunit.client.1.smithi130.stdout:3/838: truncate d8/d1a/dfa/d27/f84 4942433 0 2024-05-13T22:48:40.830 INFO:tasks.workunit.client.1.smithi130.stdout:3/839: fsync d8/d1a/dfa/d27/f97 0 2024-05-13T22:48:40.831 INFO:tasks.workunit.client.1.smithi130.stdout:3/840: chown d8/dc/d7b/da3/dd3/ddd/la8 6 1 2024-05-13T22:48:40.837 INFO:tasks.workunit.client.1.smithi130.stdout:2/825: dread d4/db/dbf/df9/d30/f47 [0,4194304] 0 2024-05-13T22:48:40.837 INFO:tasks.workunit.client.1.smithi130.stdout:9/845: mkdir d1/d7/d23/d3d/d43/d113/d11b 0 2024-05-13T22:48:40.838 INFO:tasks.workunit.client.1.smithi130.stdout:6/780: dread d4/d15/d1d/d28/f34 [0,4194304] 0 2024-05-13T22:48:40.839 INFO:tasks.workunit.client.1.smithi130.stdout:3/841: symlink d8/dc/d15/dfd/def/l11b 0 2024-05-13T22:48:40.839 INFO:tasks.workunit.client.1.smithi130.stdout:1/809: mknod db/d4a/de9/c104 0 2024-05-13T22:48:40.840 INFO:tasks.workunit.client.1.smithi130.stdout:8/739: truncate d1/d2/db/d15/d18/d42/d53/d56/f61 436997 0 2024-05-13T22:48:40.843 INFO:tasks.workunit.client.1.smithi130.stdout:1/810: mknod db/d1b/da3/d72/d75/dee/c105 0 2024-05-13T22:48:40.845 INFO:tasks.workunit.client.1.smithi130.stdout:8/740: unlink d1/d2/db/d15/d18/d42/c6c 0 2024-05-13T22:48:40.847 INFO:tasks.workunit.client.1.smithi130.stdout:9/846: mkdir d1/d7/d23/d28/d32/d45/d111/d11c 0 2024-05-13T22:48:40.849 INFO:tasks.workunit.client.1.smithi130.stdout:6/781: mkdir d4/d15/d1f/d46/d52/d10e 0 2024-05-13T22:48:40.850 INFO:tasks.workunit.client.1.smithi130.stdout:6/782: dread - d4/ff4 zero size 2024-05-13T22:48:40.850 INFO:tasks.workunit.client.1.smithi130.stdout:6/783: readlink d4/d15/d1d/daa/l78 0 2024-05-13T22:48:40.852 INFO:tasks.workunit.client.1.smithi130.stdout:8/741: mkdir d1/d2/db/d15/d18/dba/d111 0 2024-05-13T22:48:40.853 INFO:tasks.workunit.client.1.smithi130.stdout:9/847: fdatasync d1/d15/f7d 0 2024-05-13T22:48:40.856 INFO:tasks.workunit.client.1.smithi130.stdout:6/784: dwrite d4/d15/d1f/d46/fb2 [0,4194304] 0 2024-05-13T22:48:40.857 INFO:tasks.workunit.client.1.smithi130.stdout:8/742: dread d1/d2/db/f8 [4194304,4194304] 0 2024-05-13T22:48:40.864 INFO:tasks.workunit.client.1.smithi130.stdout:0/856: sync 2024-05-13T22:48:40.864 INFO:tasks.workunit.client.1.smithi130.stdout:5/825: sync 2024-05-13T22:48:40.865 INFO:tasks.workunit.client.1.smithi130.stdout:4/792: sync 2024-05-13T22:48:40.871 INFO:tasks.workunit.client.1.smithi130.stdout:6/785: fdatasync d4/d15/d1f/f3e 0 2024-05-13T22:48:40.872 INFO:tasks.workunit.client.1.smithi130.stdout:3/842: fsync f3 0 2024-05-13T22:48:40.875 INFO:tasks.workunit.client.1.smithi130.stdout:9/848: dread d1/d7/f55 [0,4194304] 0 2024-05-13T22:48:40.876 INFO:tasks.workunit.client.1.smithi130.stdout:5/826: dread - d6/d7/d19/fc4 zero size 2024-05-13T22:48:40.880 INFO:tasks.workunit.client.1.smithi130.stdout:8/743: mkdir d1/d2/db/d15/d18/d42/d53/de8/d10f/d112 0 2024-05-13T22:48:40.881 INFO:tasks.workunit.client.1.smithi130.stdout:5/827: read d6/d7/d19/d4d/d8a/fa5 [748266,117495] 0 2024-05-13T22:48:40.883 INFO:tasks.workunit.client.1.smithi130.stdout:6/786: mknod d4/d16/c10f 0 2024-05-13T22:48:40.884 INFO:tasks.workunit.client.1.smithi130.stdout:3/843: creat d8/d1a/dfa/d83/dba/f11c x:0 0 0 2024-05-13T22:48:40.885 INFO:tasks.workunit.client.1.smithi130.stdout:3/844: truncate d8/d1a/dfa/d83/f101 1033287 0 2024-05-13T22:48:40.886 INFO:tasks.workunit.client.1.smithi130.stdout:3/845: fsync d8/d1a/d1c/f25 0 2024-05-13T22:48:40.886 INFO:tasks.workunit.client.1.smithi130.stdout:3/846: stat d8/d1a/dfa/d27/f105 0 2024-05-13T22:48:40.887 INFO:tasks.workunit.client.1.smithi130.stdout:9/849: mknod d1/d7/d23/d61/c11d 0 2024-05-13T22:48:40.893 INFO:tasks.workunit.client.1.smithi130.stdout:9/850: mkdir d1/db/d11e 0 2024-05-13T22:48:40.893 INFO:tasks.workunit.client.1.smithi130.stdout:9/851: chown d1/db/c22 7824 1 2024-05-13T22:48:40.894 INFO:tasks.workunit.client.1.smithi130.stdout:8/744: creat d1/d2/db/d15/d18/dba/d107/f113 x:0 0 0 2024-05-13T22:48:40.894 INFO:tasks.workunit.client.1.smithi130.stdout:9/852: readlink d1/d7/d23/lcd 0 2024-05-13T22:48:40.895 INFO:tasks.workunit.client.1.smithi130.stdout:9/853: chown d1/d7/d23/d28/f64 408031 1 2024-05-13T22:48:40.895 INFO:tasks.workunit.client.1.smithi130.stdout:9/854: readlink d1/d7/d23/lcd 0 2024-05-13T22:48:40.896 INFO:tasks.workunit.client.1.smithi130.stdout:5/828: mkdir d6/d1a/d119 0 2024-05-13T22:48:40.897 INFO:tasks.workunit.client.1.smithi130.stdout:9/855: write d1/d7/d23/d3d/d43/ffc [1336046,19942] 0 2024-05-13T22:48:40.897 INFO:tasks.workunit.client.1.smithi130.stdout:0/857: link d0/c1 d0/d7/dd/d69/dac/c11a 0 2024-05-13T22:48:40.898 INFO:tasks.workunit.client.1.smithi130.stdout:0/858: write d0/f27 [2688708,125948] 0 2024-05-13T22:48:40.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:40 smithi130 ceph-mon[36591]: pgmap v37: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 92 MiB/s rd, 276 MiB/s wr, 560 op/s 2024-05-13T22:48:40.901 INFO:tasks.workunit.client.1.smithi130.stdout:4/793: getdents d8/dc/d26/dab 0 2024-05-13T22:48:40.902 INFO:tasks.workunit.client.1.smithi130.stdout:4/794: stat d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/c4c 0 2024-05-13T22:48:40.904 INFO:tasks.workunit.client.1.smithi130.stdout:8/745: rename d1/d2/db/d15/d18/d42/d53/f100 to d1/d2/db/d15/d18/dba/d111/f114 0 2024-05-13T22:48:40.910 INFO:tasks.workunit.client.1.smithi130.stdout:5/829: symlink d6/dd/d18/d6e/d82/d4b/l11a 0 2024-05-13T22:48:40.910 INFO:tasks.workunit.client.1.smithi130.stdout:5/830: chown d6/dd/d18/cd0 2 1 2024-05-13T22:48:40.915 INFO:tasks.workunit.client.1.smithi130.stdout:4/795: truncate d8/dc/d80/db0/df1/d33/d44/d56/f7f 8758278 0 2024-05-13T22:48:40.917 INFO:tasks.workunit.client.1.smithi130.stdout:8/746: truncate d1/d2/db/d15/d18/f3e 495219 0 2024-05-13T22:48:40.917 INFO:tasks.workunit.client.1.smithi130.stdout:6/787: link d4/d15/d1d/d60/da2/da4/ddd/c109 d4/c110 0 2024-05-13T22:48:40.920 INFO:tasks.workunit.client.1.smithi130.stdout:6/788: dread d4/d15/d1d/d28/f34 [0,4194304] 0 2024-05-13T22:48:40.929 INFO:tasks.workunit.client.1.smithi130.stdout:5/831: mknod d6/d7/d95/c11b 0 2024-05-13T22:48:40.930 INFO:tasks.workunit.client.1.smithi130.stdout:5/832: readlink d6/d7/d19/d4d/d8a/la6 0 2024-05-13T22:48:40.930 INFO:tasks.workunit.client.1.smithi130.stdout:6/789: rmdir d4/d15/d1d/d28/d75/da8 39 2024-05-13T22:48:40.933 INFO:tasks.workunit.client.1.smithi130.stdout:3/847: getdents d8/d35 0 2024-05-13T22:48:40.936 INFO:tasks.workunit.client.1.smithi130.stdout:0/859: link d0/d7/dd/f41 d0/d7/dd/d2e/f11b 0 2024-05-13T22:48:40.937 INFO:tasks.workunit.client.1.smithi130.stdout:7/838: sync 2024-05-13T22:48:40.941 INFO:tasks.workunit.client.1.smithi130.stdout:9/856: getdents d1/d7/d23/d28/d2d/d2f 0 2024-05-13T22:48:40.945 INFO:tasks.workunit.client.1.smithi130.stdout:8/747: dwrite d1/d2/db/d15/d18/d42/d53/d56/f9a [0,4194304] 0 2024-05-13T22:48:40.949 INFO:tasks.workunit.client.1.smithi130.stdout:0/860: rename d0/d7/dd/d2e/da8/dcd/df8 to d0/d7/dd/d2e/da8/d11c 0 2024-05-13T22:48:40.951 INFO:tasks.workunit.client.1.smithi130.stdout:7/839: chown d7/da/d1d/d5d/da6/d9/d4a/dbd/dc9/lcf 77257964 1 2024-05-13T22:48:40.952 INFO:tasks.workunit.client.1.smithi130.stdout:5/833: write d6/d7/d19/fc4 [825253,116889] 0 2024-05-13T22:48:40.956 INFO:tasks.workunit.client.1.smithi130.stdout:4/796: dwrite d8/dc/d80/db0/df1/d33/d44/d56/fcf [0,4194304] 0 2024-05-13T22:48:40.959 INFO:tasks.workunit.client.1.smithi130.stdout:3/848: unlink d8/d1a/dfa/f30 0 2024-05-13T22:48:40.992 INFO:tasks.workunit.client.1.smithi130.stdout:2/826: sync 2024-05-13T22:48:40.992 INFO:tasks.workunit.client.1.smithi130.stdout:1/811: sync 2024-05-13T22:48:40.993 INFO:tasks.workunit.client.1.smithi130.stdout:1/812: chown db/d1b/da3/d2a/d35/ffb 5549951 1 2024-05-13T22:48:40.998 INFO:tasks.workunit.client.1.smithi130.stdout:2/827: dwrite d4/db/dbf/d42/d4b/ff0 [0,4194304] 0 2024-05-13T22:48:40.999 INFO:tasks.workunit.client.1.smithi130.stdout:2/828: readlink d4/db/d26/d9e/lc8 0 2024-05-13T22:48:41.023 INFO:tasks.workunit.client.1.smithi130.stdout:9/857: sync 2024-05-13T22:48:41.059 INFO:tasks.workunit.client.1.smithi130.stdout:8/748: write d1/d2/db/d15/d16/f88 [127977,33318] 0 2024-05-13T22:48:41.062 INFO:tasks.workunit.client.1.smithi130.stdout:7/840: creat d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/f110 x:0 0 0 2024-05-13T22:48:41.063 INFO:tasks.workunit.client.1.smithi130.stdout:7/841: write d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/ff1 [196199,73490] 0 2024-05-13T22:48:41.064 INFO:tasks.workunit.client.1.smithi130.stdout:6/790: mkdir d4/d15/d1d/d28/d75/da8/da0/dfe/d111 0 2024-05-13T22:48:41.067 INFO:tasks.workunit.client.1.smithi130.stdout:4/797: rmdir d8/dc/d80/db0/df1/d33/dee 39 2024-05-13T22:48:41.067 INFO:tasks.workunit.client.1.smithi130.stdout:5/834: dread d6/dd/f33 [0,4194304] 0 2024-05-13T22:48:41.069 INFO:tasks.workunit.client.1.smithi130.stdout:5/835: write d6/dd/f3e [3654979,28021] 0 2024-05-13T22:48:41.071 INFO:tasks.workunit.client.1.smithi130.stdout:7/842: dwrite d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/f110 [0,4194304] 0 2024-05-13T22:48:41.072 INFO:tasks.workunit.client.1.smithi130.stdout:5/836: write d6/dd/fb5 [4778740,130208] 0 2024-05-13T22:48:41.079 INFO:tasks.workunit.client.1.smithi130.stdout:4/798: dwrite d8/dc/d80/db0/df1/d33/d44/db8/fcc [0,4194304] 0 2024-05-13T22:48:41.081 INFO:tasks.workunit.client.1.smithi130.stdout:4/799: write d8/dc/d80/db0/df1/d33/d44/d77/fcd [1493308,60694] 0 2024-05-13T22:48:41.081 INFO:tasks.workunit.client.1.smithi130.stdout:4/800: read - d8/dfa/fc9 zero size 2024-05-13T22:48:41.083 INFO:tasks.workunit.client.1.smithi130.stdout:8/749: creat d1/d9b/f115 x:0 0 0 2024-05-13T22:48:41.096 INFO:tasks.workunit.client.1.smithi130.stdout:1/813: unlink db/d1b/da3/d2a/d38/d46/lbd 0 2024-05-13T22:48:41.100 INFO:tasks.workunit.client.1.smithi130.stdout:7/843: mkdir d7/da/d1d/d5d/da6/d5b/d107/db9/d111 0 2024-05-13T22:48:41.102 INFO:tasks.workunit.client.1.smithi130.stdout:7/844: stat d7/da/d1d/d5d/da6/f1c 0 2024-05-13T22:48:41.104 INFO:tasks.workunit.client.1.smithi130.stdout:5/837: dwrite d6/d2f/ff8 [4194304,4194304] 0 2024-05-13T22:48:41.108 INFO:tasks.workunit.client.1.smithi130.stdout:8/750: rmdir d1/d2/db/d15/d16/d54/d58 39 2024-05-13T22:48:41.118 INFO:tasks.workunit.client.1.smithi130.stdout:7/845: truncate d7/da/d1d/d5d/da6/f19 4638350 0 2024-05-13T22:48:41.123 INFO:tasks.workunit.client.1.smithi130.stdout:5/838: chown d6/d1a/d92/lf4 1096910430 1 2024-05-13T22:48:41.132 INFO:tasks.workunit.client.1.smithi130.stdout:8/751: getdents d1/d2/db/d15/d7e/dcd 0 2024-05-13T22:48:41.133 INFO:tasks.workunit.client.1.smithi130.stdout:8/752: chown d1/d2/db/f45 45448791 1 2024-05-13T22:48:41.135 INFO:tasks.workunit.client.1.smithi130.stdout:8/753: stat d1/d2/db/d15/d16/d54/lf0 0 2024-05-13T22:48:41.139 INFO:tasks.workunit.client.1.smithi130.stdout:8/754: dread d1/d2/db/f8 [4194304,4194304] 0 2024-05-13T22:48:41.139 INFO:tasks.workunit.client.1.smithi130.stdout:8/755: fdatasync d1/dc/f44 0 2024-05-13T22:48:41.141 INFO:tasks.workunit.client.1.smithi130.stdout:5/839: dread d6/d7/f25 [0,4194304] 0 2024-05-13T22:48:41.143 INFO:tasks.workunit.client.1.smithi130.stdout:8/756: dread d1/dc/d5d/fa8 [0,4194304] 0 2024-05-13T22:48:41.145 INFO:tasks.workunit.client.1.smithi130.stdout:3/849: rmdir d8/dc/d7b/da3/daf 39 2024-05-13T22:48:41.147 INFO:tasks.workunit.client.1.smithi130.stdout:2/829: creat d4/da8/f11b x:0 0 0 2024-05-13T22:48:41.147 INFO:tasks.workunit.client.1.smithi130.stdout:5/840: symlink d6/d7/d19/d4d/d6a/l11c 0 2024-05-13T22:48:41.150 INFO:tasks.workunit.client.1.smithi130.stdout:8/757: creat d1/d2/db/d15/d7e/d9f/f116 x:0 0 0 2024-05-13T22:48:41.156 INFO:tasks.workunit.client.1.smithi130.stdout:8/758: mkdir d1/d2/db/d15/d7e/dc2/dcf/dda/d117 0 2024-05-13T22:48:41.159 INFO:tasks.workunit.client.1.smithi130.stdout:8/759: creat d1/d2/db/d15/d7e/dc2/dcf/dda/f118 x:0 0 0 2024-05-13T22:48:41.175 INFO:tasks.workunit.client.1.smithi130.stdout:0/861: write d0/d7/dd/f41 [3220629,4366] 0 2024-05-13T22:48:41.181 INFO:tasks.workunit.client.1.smithi130.stdout:0/862: dwrite d0/d7/dd/d2e/d9e/da7/fd8 [0,4194304] 0 2024-05-13T22:48:41.182 INFO:tasks.workunit.client.1.smithi130.stdout:0/863: chown d0/da0/ccb 50389 1 2024-05-13T22:48:41.186 INFO:tasks.workunit.client.1.smithi130.stdout:0/864: unlink d0/d7/dd/d69/dac/cb5 0 2024-05-13T22:48:41.189 INFO:tasks.workunit.client.1.smithi130.stdout:0/865: unlink d0/d7/dd/lf 0 2024-05-13T22:48:41.191 INFO:tasks.workunit.client.1.smithi130.stdout:0/866: write d0/d7/dd/d69/da9/ff9 [569127,19983] 0 2024-05-13T22:48:41.195 INFO:tasks.workunit.client.1.smithi130.stdout:6/791: write d4/d15/d1d/d28/f47 [4354275,112033] 0 2024-05-13T22:48:41.198 INFO:tasks.workunit.client.1.smithi130.stdout:6/792: stat d4/d15/d1d/d28/d75/da8/da0/cf9 0 2024-05-13T22:48:41.206 INFO:tasks.workunit.client.1.smithi130.stdout:9/858: dwrite d1/db/fc [0,4194304] 0 2024-05-13T22:48:41.207 INFO:tasks.workunit.client.1.smithi130.stdout:9/859: chown d1/d7/d69/d79/lc3 18668 1 2024-05-13T22:48:41.213 INFO:tasks.workunit.client.1.smithi130.stdout:9/860: dread d1/d7/d23/d28/d2d/f48 [0,4194304] 0 2024-05-13T22:48:41.213 INFO:tasks.workunit.client.1.smithi130.stdout:6/793: dread d4/d15/d1d/fa5 [0,4194304] 0 2024-05-13T22:48:41.214 INFO:tasks.workunit.client.1.smithi130.stdout:9/861: fdatasync d1/d15/d51/f6d 0 2024-05-13T22:48:41.216 INFO:tasks.workunit.client.1.smithi130.stdout:6/794: rmdir d4/d15/d1f/d3c 39 2024-05-13T22:48:41.220 INFO:tasks.workunit.client.1.smithi130.stdout:2/830: write d4/f19 [3068594,95871] 0 2024-05-13T22:48:41.222 INFO:tasks.workunit.client.1.smithi130.stdout:2/831: write d4/fa [1011961,112496] 0 2024-05-13T22:48:41.226 INFO:tasks.workunit.client.1.smithi130.stdout:1/814: rename db/l50 to db/d1b/da3/d2a/de7/l106 0 2024-05-13T22:48:41.227 INFO:tasks.workunit.client.1.smithi130.stdout:2/832: symlink d4/db/dc/d18/l11c 0 2024-05-13T22:48:41.227 INFO:tasks.workunit.client.1.smithi130.stdout:1/815: dread db/f6a [0,4194304] 0 2024-05-13T22:48:41.228 INFO:tasks.workunit.client.1.smithi130.stdout:2/833: chown d4/db/dbf/d42/f53 65 1 2024-05-13T22:48:41.228 INFO:tasks.workunit.client.1.smithi130.stdout:1/816: readlink db/d4a/lb5 0 2024-05-13T22:48:41.228 INFO:tasks.workunit.client.1.smithi130.stdout:7/846: creat d7/da/f112 x:0 0 0 2024-05-13T22:48:41.231 INFO:tasks.workunit.client.1.smithi130.stdout:4/801: rename d8/dc/d80/db0/df1/d33/d44/db8/faa to d8/dbb/ffd 0 2024-05-13T22:48:41.233 INFO:tasks.workunit.client.1.smithi130.stdout:2/834: symlink d4/da8/dc9/l11d 0 2024-05-13T22:48:41.235 INFO:tasks.workunit.client.1.smithi130.stdout:1/817: mkdir db/d1b/da3/d49/d56/d7f/d107 0 2024-05-13T22:48:41.235 INFO:tasks.workunit.client.1.smithi130.stdout:1/818: write db/d1b/da3/f89 [1247858,37162] 0 2024-05-13T22:48:41.237 INFO:tasks.workunit.client.1.smithi130.stdout:7/847: creat d7/da/d1d/db1/dbe/dd3/f113 x:0 0 0 2024-05-13T22:48:41.239 INFO:tasks.workunit.client.1.smithi130.stdout:5/841: rename d6/dd/d18/d6e/d76 to d6/dd/d18/d6e/d82/db0/d11d 0 2024-05-13T22:48:41.241 INFO:tasks.workunit.client.1.smithi130.stdout:4/802: creat d8/dc/d80/db0/df1/d33/d44/d56/d8c/ffe x:0 0 0 2024-05-13T22:48:41.242 INFO:tasks.workunit.client.1.smithi130.stdout:4/803: chown d8/dc/d26/fc2 65612856 1 2024-05-13T22:48:41.243 INFO:tasks.workunit.client.1.smithi130.stdout:2/835: mknod d4/db/dbf/df9/dfd/c11e 0 2024-05-13T22:48:41.244 INFO:tasks.workunit.client.1.smithi130.stdout:1/819: symlink db/d1b/da3/d2a/d35/dd4/l108 0 2024-05-13T22:48:41.252 INFO:tasks.workunit.client.1.smithi130.stdout:2/836: creat d4/da8/dc9/f11f x:0 0 0 2024-05-13T22:48:41.252 INFO:tasks.workunit.client.1.smithi130.stdout:0/867: rmdir d0/d7/dd 39 2024-05-13T22:48:41.255 INFO:tasks.workunit.client.1.smithi130.stdout:7/848: mkdir d7/da/d1d/d42/d114 0 2024-05-13T22:48:41.260 INFO:tasks.workunit.client.1.smithi130.stdout:8/760: rename d1/d2/db/d15/d18/c85 to d1/d2/db/c119 0 2024-05-13T22:48:41.263 INFO:tasks.workunit.client.1.smithi130.stdout:4/804: truncate d8/f1f 579790 0 2024-05-13T22:48:41.263 INFO:tasks.workunit.client.1.smithi130.stdout:2/837: dread d4/f9 [0,4194304] 0 2024-05-13T22:48:41.265 INFO:tasks.workunit.client.1.smithi130.stdout:3/850: sync 2024-05-13T22:48:41.265 INFO:tasks.workunit.client.1.smithi130.stdout:4/805: write d8/dc/d26/f55 [4275124,74037] 0 2024-05-13T22:48:41.266 INFO:tasks.workunit.client.1.smithi130.stdout:4/806: chown d8/dc/d80/fe3 480516299 1 2024-05-13T22:48:41.267 INFO:tasks.workunit.client.1.smithi130.stdout:4/807: stat d8/dc/d80/db0/df1/d33/dbf/cd9 0 2024-05-13T22:48:41.268 INFO:tasks.workunit.client.1.smithi130.stdout:7/849: mkdir d7/da/d1d/d5d/da6/d39/da9/d115 0 2024-05-13T22:48:41.268 INFO:tasks.workunit.client.1.smithi130.stdout:6/795: write d4/d15/d1f/d64/fd3 [1429751,16543] 0 2024-05-13T22:48:41.269 INFO:tasks.workunit.client.1.smithi130.stdout:4/808: fdatasync d8/dc/d80/db0/fd8 0 2024-05-13T22:48:41.272 INFO:tasks.workunit.client.1.smithi130.stdout:5/842: creat d6/d1a/d92/f11e x:0 0 0 2024-05-13T22:48:41.273 INFO:tasks.workunit.client.1.smithi130.stdout:2/838: creat d4/da8/f120 x:0 0 0 2024-05-13T22:48:41.276 INFO:tasks.workunit.client.1.smithi130.stdout:0/868: creat d0/d7/dd/d17/d52/d5b/d5f/f11d x:0 0 0 2024-05-13T22:48:41.276 INFO:tasks.workunit.client.1.smithi130.stdout:6/796: dread - d4/d15/d1f/faf zero size 2024-05-13T22:48:41.276 INFO:tasks.workunit.client.1.smithi130.stdout:3/851: dread d8/d1a/f4f [0,4194304] 0 2024-05-13T22:48:41.277 INFO:tasks.workunit.client.1.smithi130.stdout:0/869: chown d0/d7/dd/d69/da9/d8d/d9f/la4 32614 1 2024-05-13T22:48:41.279 INFO:tasks.workunit.client.1.smithi130.stdout:7/850: rmdir d7/da/d1d/d5d/da6/d9/d4a/dbd/d88 39 2024-05-13T22:48:41.286 INFO:tasks.workunit.client.1.smithi130.stdout:6/797: creat d4/d15/d1f/d46/d52/d7b/f112 x:0 0 0 2024-05-13T22:48:41.287 INFO:tasks.workunit.client.1.smithi130.stdout:7/851: dread d7/da/d1d/d5d/da6/d39/f7c [0,4194304] 0 2024-05-13T22:48:41.288 INFO:tasks.workunit.client.1.smithi130.stdout:7/852: fdatasync d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/fb6 0 2024-05-13T22:48:41.289 INFO:tasks.workunit.client.1.smithi130.stdout:7/853: fsync d7/da/d1d/d5d/d70/f106 0 2024-05-13T22:48:41.289 INFO:tasks.workunit.client.1.smithi130.stdout:7/854: readlink d7/da/l29 0 2024-05-13T22:48:41.292 INFO:tasks.workunit.client.1.smithi130.stdout:5/843: symlink d6/dd/d18/d6e/dc7/d115/l11f 0 2024-05-13T22:48:41.294 INFO:tasks.workunit.client.1.smithi130.stdout:2/839: mkdir d4/db/dbf/d121 0 2024-05-13T22:48:41.298 INFO:tasks.workunit.client.1.smithi130.stdout:1/820: truncate db/d1b/da3/d72/d75/dee/ff5 418808 0 2024-05-13T22:48:41.299 INFO:tasks.workunit.client.1.smithi130.stdout:1/821: stat db/d1b/da3/d49/c59 0 2024-05-13T22:48:41.299 INFO:tasks.workunit.client.1.smithi130.stdout:9/862: sync 2024-05-13T22:48:41.300 INFO:tasks.workunit.client.1.smithi130.stdout:9/863: chown d1/d7/d23/d28/d2d/d2f/d63/d6b/fcf 1 1 2024-05-13T22:48:41.305 INFO:tasks.workunit.client.1.smithi130.stdout:5/844: creat d6/d7/d19/d4d/d53/f120 x:0 0 0 2024-05-13T22:48:41.306 INFO:tasks.workunit.client.1.smithi130.stdout:0/870: fdatasync d0/d7/dd/d17/d1f/d106/f115 0 2024-05-13T22:48:41.309 INFO:tasks.workunit.client.1.smithi130.stdout:5/845: chown d6/d7/d19/f52 29816 1 2024-05-13T22:48:41.309 INFO:tasks.workunit.client.1.smithi130.stdout:4/809: dwrite d8/dc/d26/fa1 [0,4194304] 0 2024-05-13T22:48:41.311 INFO:tasks.workunit.client.1.smithi130.stdout:3/852: dwrite d8/d1a/dfa/d2c/ff5 [4194304,4194304] 0 2024-05-13T22:48:41.312 INFO:tasks.workunit.client.1.smithi130.stdout:1/822: rmdir db/d1b/da3/d72 39 2024-05-13T22:48:41.316 INFO:tasks.workunit.client.1.smithi130.stdout:4/810: stat d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d81/fce 0 2024-05-13T22:48:41.316 INFO:tasks.workunit.client.1.smithi130.stdout:8/761: dwrite d1/d2/db/d15/d16/d54/d58/f94 [0,4194304] 0 2024-05-13T22:48:41.318 INFO:tasks.workunit.client.1.smithi130.stdout:6/798: creat d4/d15/d1d/d60/da2/f113 x:0 0 0 2024-05-13T22:48:41.328 INFO:tasks.workunit.client.1.smithi130.stdout:7/855: dwrite d7/da/d1d/d5d/da6/d9/d66/f81 [0,4194304] 0 2024-05-13T22:48:41.329 INFO:tasks.workunit.client.1.smithi130.stdout:2/840: rename d4/d56/dff/f73 to d4/f122 0 2024-05-13T22:48:41.332 INFO:tasks.workunit.client.1.smithi130.stdout:4/811: fdatasync d8/dc/d80/db0/df1/d33/d44/d56/f72 0 2024-05-13T22:48:41.333 INFO:tasks.workunit.client.1.smithi130.stdout:8/762: mknod d1/d2/db/d15/d7e/dc2/dcf/c11a 0 2024-05-13T22:48:41.339 INFO:tasks.workunit.client.1.smithi130.stdout:6/799: dwrite d4/d16/f2a [0,4194304] 0 2024-05-13T22:48:41.342 INFO:tasks.workunit.client.1.smithi130.stdout:0/871: dwrite d0/d7/dd/d17/d1f/f7d [0,4194304] 0 2024-05-13T22:48:41.343 INFO:tasks.workunit.client.1.smithi130.stdout:3/853: unlink d8/d1a/d1c/f6e 0 2024-05-13T22:48:41.345 INFO:tasks.workunit.client.1.smithi130.stdout:7/856: mknod d7/da/d1d/db1/de1/c116 0 2024-05-13T22:48:41.346 INFO:tasks.workunit.client.1.smithi130.stdout:2/841: mknod d4/d56/dff/d4a/d76/c123 0 2024-05-13T22:48:41.353 INFO:tasks.workunit.client.1.smithi130.stdout:5/846: write d6/dd/d79/f106 [387855,49893] 0 2024-05-13T22:48:41.358 INFO:tasks.workunit.client.1.smithi130.stdout:9/864: getdents d1/d96 0 2024-05-13T22:48:41.362 INFO:tasks.workunit.client.1.smithi130.stdout:1/823: creat db/d1b/da3/d72/d75/dee/df6/da0/f109 x:0 0 0 2024-05-13T22:48:41.365 INFO:tasks.workunit.client.1.smithi130.stdout:0/872: write d0/d7/dd/d2e/da8/d11c/f37 [1477032,34277] 0 2024-05-13T22:48:41.371 INFO:tasks.workunit.client.1.smithi130.stdout:1/824: chown db/d1b/da3/d5e/c93 1 1 2024-05-13T22:48:41.374 INFO:tasks.workunit.client.1.smithi130.stdout:9/865: rename d1/d7/dd2 to d1/d15/d51/db4/d11f 0 2024-05-13T22:48:41.378 INFO:tasks.workunit.client.1.smithi130.stdout:8/763: write d1/d2/db/d15/d18/f5f [564429,87049] 0 2024-05-13T22:48:41.379 INFO:tasks.workunit.client.1.smithi130.stdout:8/764: dread - d1/dc/dc3/fcc zero size 2024-05-13T22:48:41.380 INFO:tasks.workunit.client.1.smithi130.stdout:9/866: creat d1/d15/d51/d85/d92/f120 x:0 0 0 2024-05-13T22:48:41.381 INFO:tasks.workunit.client.1.smithi130.stdout:3/854: dread d8/fa1 [0,4194304] 0 2024-05-13T22:48:41.382 INFO:tasks.workunit.client.1.smithi130.stdout:2/842: dread d4/db/dbf/df9/f7d [0,4194304] 0 2024-05-13T22:48:41.382 INFO:tasks.workunit.client.1.smithi130.stdout:0/873: rmdir d0/d7/dd/d69/da9/d8d/d9f/d112 0 2024-05-13T22:48:41.383 INFO:tasks.workunit.client.1.smithi130.stdout:9/867: fsync d1/d7/d23/d28/d2d/d2f/d63/f9f 0 2024-05-13T22:48:41.383 INFO:tasks.workunit.client.1.smithi130.stdout:6/800: write d4/d15/d1d/d28/d75/fa6 [2006079,126048] 0 2024-05-13T22:48:41.384 INFO:tasks.workunit.client.1.smithi130.stdout:3/855: chown d8/d1a/dfa/d83/dba 4385 1 2024-05-13T22:48:41.384 INFO:tasks.workunit.client.1.smithi130.stdout:4/812: truncate d8/dc/d80/db0/df1/f50 1213055 0 2024-05-13T22:48:41.385 INFO:tasks.workunit.client.1.smithi130.stdout:1/825: symlink db/d1b/da3/d49/d56/l10a 0 2024-05-13T22:48:41.388 INFO:tasks.workunit.client.1.smithi130.stdout:7/857: link d7/da/d1d/db1/dd4/lde d7/da/d1d/d5d/da6/d9/d4a/l117 0 2024-05-13T22:48:41.388 INFO:tasks.workunit.client.1.smithi130.stdout:9/868: readlink d1/d7/d23/d28/d2d/d2f/l60 0 2024-05-13T22:48:41.389 INFO:tasks.workunit.client.1.smithi130.stdout:0/874: write d0/d7/dd/d2e/d9e/da7/fb8 [927114,27070] 0 2024-05-13T22:48:41.391 INFO:tasks.workunit.client.1.smithi130.stdout:8/765: dread d1/d2/db/d15/d18/d42/d53/d56/f9a [0,4194304] 0 2024-05-13T22:48:41.392 INFO:tasks.workunit.client.1.smithi130.stdout:5/847: dwrite d6/d2f/f59 [0,4194304] 0 2024-05-13T22:48:41.393 INFO:tasks.workunit.client.1.smithi130.stdout:9/869: write d1/d7/d23/d3d/d43/ffc [2809734,123360] 0 2024-05-13T22:48:41.393 INFO:tasks.workunit.client.1.smithi130.stdout:7/858: dread d7/da/fd0 [0,4194304] 0 2024-05-13T22:48:41.394 INFO:tasks.workunit.client.1.smithi130.stdout:0/875: write d0/d7/dd/d17/fae [8151336,69687] 0 2024-05-13T22:48:41.396 INFO:tasks.workunit.client.1.smithi130.stdout:7/859: fsync d7/da/f112 0 2024-05-13T22:48:41.397 INFO:tasks.workunit.client.1.smithi130.stdout:2/843: rmdir d4/db/dbf/df9/d66 39 2024-05-13T22:48:41.406 INFO:tasks.workunit.client.1.smithi130.stdout:3/856: truncate d8/d35/f52 134732 0 2024-05-13T22:48:41.410 INFO:tasks.workunit.client.1.smithi130.stdout:5/848: creat d6/dd/d18/d6e/d82/db0/d11d/f121 x:0 0 0 2024-05-13T22:48:41.410 INFO:tasks.workunit.client.1.smithi130.stdout:5/849: chown d6/dd/fcd 122913933 1 2024-05-13T22:48:41.419 INFO:tasks.workunit.client.1.smithi130.stdout:0/876: rename d0/d7/dd/d2e/da8/d11c/f37 to d0/d7/dd/d17/d52/d67/d68/f11e 0 2024-05-13T22:48:41.422 INFO:tasks.workunit.client.1.smithi130.stdout:2/844: mknod d4/db/dbf/df9/d30/d88/d99/c124 0 2024-05-13T22:48:41.422 INFO:tasks.workunit.client.1.smithi130.stdout:6/801: mkdir d4/d15/d114 0 2024-05-13T22:48:41.427 INFO:tasks.workunit.client.1.smithi130.stdout:4/813: symlink d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/lff 0 2024-05-13T22:48:41.428 INFO:tasks.workunit.client.1.smithi130.stdout:3/857: mkdir d8/d1a/dfa/d83/d11d 0 2024-05-13T22:48:41.433 INFO:tasks.workunit.client.1.smithi130.stdout:5/850: rmdir d6/dd/d18/d6e/dc7/d115 39 2024-05-13T22:48:41.440 INFO:tasks.workunit.client.1.smithi130.stdout:2/845: chown d4/d56/dff/d4a/dc6/ca6 211757918 1 2024-05-13T22:48:41.441 INFO:tasks.workunit.client.1.smithi130.stdout:1/826: write db/d1b/da3/d2a/d38/d46/d6d/fbe [4933546,80818] 0 2024-05-13T22:48:41.444 INFO:tasks.workunit.client.1.smithi130.stdout:1/827: dread db/d1b/da3/d49/d56/d7f/ffa [0,4194304] 0 2024-05-13T22:48:41.446 INFO:tasks.workunit.client.1.smithi130.stdout:2/846: dwrite d4/db/dbf/d42/d4b/ff0 [0,4194304] 0 2024-05-13T22:48:41.453 INFO:tasks.workunit.client.1.smithi130.stdout:8/766: link d1/d2/db/d15/d18/d42/d53/d56/l8d d1/d2/db/d15/d18/d42/d53/de8/d10f/d112/l11b 0 2024-05-13T22:48:41.454 INFO:tasks.workunit.client.1.smithi130.stdout:0/877: rmdir d0/d7/dd/d17/d1f/d2c/d45/d6b 39 2024-05-13T22:48:41.459 INFO:tasks.workunit.client.1.smithi130.stdout:8/767: dwrite d1/d2/db/d15/d7e/f95 [0,4194304] 0 2024-05-13T22:48:41.464 INFO:tasks.workunit.client.1.smithi130.stdout:9/870: getdents d1/d7/d69 0 2024-05-13T22:48:41.464 INFO:tasks.workunit.client.1.smithi130.stdout:7/860: getdents d7/da/d1d/d5d/da6/d39/da9/dfb 0 2024-05-13T22:48:41.465 INFO:tasks.workunit.client.1.smithi130.stdout:9/871: stat d1/d7/cde 0 2024-05-13T22:48:41.466 INFO:tasks.workunit.client.1.smithi130.stdout:9/872: stat d1/d7/d23/d3d/d43 0 2024-05-13T22:48:41.473 INFO:tasks.workunit.client.1.smithi130.stdout:6/802: write d4/f12 [1730763,15633] 0 2024-05-13T22:48:41.474 INFO:tasks.workunit.client.1.smithi130.stdout:6/803: stat d4/d15/d1f/d46/d50/f54 0 2024-05-13T22:48:41.481 INFO:tasks.workunit.client.1.smithi130.stdout:4/814: dwrite d8/dc/d26/fd1 [4194304,4194304] 0 2024-05-13T22:48:41.483 INFO:tasks.workunit.client.1.smithi130.stdout:0/878: dread d0/d7/dd/d17/d52/f9a [0,4194304] 0 2024-05-13T22:48:41.504 INFO:tasks.workunit.client.1.smithi130.stdout:5/851: dwrite d6/f7c [0,4194304] 0 2024-05-13T22:48:41.509 INFO:tasks.workunit.client.1.smithi130.stdout:5/852: chown d6/dd/d18/d6e/d82/d84/c89 634923722 1 2024-05-13T22:48:41.509 INFO:tasks.workunit.client.1.smithi130.stdout:5/853: chown d6/d7 23 1 2024-05-13T22:48:41.516 INFO:tasks.workunit.client.1.smithi130.stdout:6/804: dread d4/d15/d1d/d28/d75/da8/da0/fc0 [0,4194304] 0 2024-05-13T22:48:41.533 INFO:tasks.workunit.client.1.smithi130.stdout:9/873: creat d1/d15/d9e/f121 x:0 0 0 2024-05-13T22:48:41.535 INFO:tasks.workunit.client.1.smithi130.stdout:8/768: read d1/d2/db/d15/d16/d54/fb8 [38279,122763] 0 2024-05-13T22:48:41.541 INFO:tasks.workunit.client.1.smithi130.stdout:4/815: dwrite d8/dbb/ffd [0,4194304] 0 2024-05-13T22:48:41.543 INFO:tasks.workunit.client.1.smithi130.stdout:9/874: mknod d1/d7/d23/d28/c122 0 2024-05-13T22:48:41.544 INFO:tasks.workunit.client.1.smithi130.stdout:7/861: getdents d7/da/d1d/d5d/da6/d9/d66/d65 0 2024-05-13T22:48:41.547 INFO:tasks.workunit.client.1.smithi130.stdout:7/862: write d7/da/d1d/d5d/da6/f1c [1501963,44951] 0 2024-05-13T22:48:41.550 INFO:tasks.workunit.client.1.smithi130.stdout:9/875: creat d1/d15/d9e/dfa/f123 x:0 0 0 2024-05-13T22:48:41.554 INFO:tasks.workunit.client.1.smithi130.stdout:4/816: dread d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dad/fb5 [0,4194304] 0 2024-05-13T22:48:41.557 INFO:tasks.workunit.client.1.smithi130.stdout:9/876: fsync d1/d7/d69/d79/f9a 0 2024-05-13T22:48:41.560 INFO:tasks.workunit.client.1.smithi130.stdout:4/817: creat d8/dc/d80/db0/df1/d33/d44/d61/f100 x:0 0 0 2024-05-13T22:48:41.565 INFO:tasks.workunit.client.1.smithi130.stdout:9/877: dwrite d1/d7/d23/d28/d73/db2/f106 [0,4194304] 0 2024-05-13T22:48:41.566 INFO:tasks.workunit.client.1.smithi130.stdout:9/878: fdatasync d1/d7/d23/d28/d73/db2/f114 0 2024-05-13T22:48:41.567 INFO:tasks.workunit.client.1.smithi130.stdout:4/818: symlink d8/l101 0 2024-05-13T22:48:41.571 INFO:tasks.workunit.client.1.smithi130.stdout:4/819: mkdir d8/dc/d80/db0/df1/d33/d44/d61/d102 0 2024-05-13T22:48:41.574 INFO:tasks.workunit.client.1.smithi130.stdout:3/858: sync 2024-05-13T22:48:41.579 INFO:tasks.workunit.client.1.smithi130.stdout:4/820: symlink d8/dc/d80/db0/df1/d33/d44/d61/l103 0 2024-05-13T22:48:41.582 INFO:tasks.workunit.client.1.smithi130.stdout:3/859: read d8/dc/d7b/da3/daf/fb7 [40173,93275] 0 2024-05-13T22:48:41.583 INFO:tasks.workunit.client.1.smithi130.stdout:3/860: dread - d8/dc/d7b/da3/dd3/f115 zero size 2024-05-13T22:48:41.583 INFO:tasks.workunit.client.1.smithi130.stdout:6/805: dread d4/d15/d1f/d46/d50/f67 [0,4194304] 0 2024-05-13T22:48:41.584 INFO:tasks.workunit.client.1.smithi130.stdout:3/861: truncate d8/d1a/dfa/d27/fe6 912988 0 2024-05-13T22:48:41.585 INFO:tasks.workunit.client.1.smithi130.stdout:3/862: chown d8/dc/d15/dfd/lbf 2 1 2024-05-13T22:48:41.588 INFO:tasks.workunit.client.1.smithi130.stdout:6/806: readlink d4/d15/d1d/d60/le6 0 2024-05-13T22:48:41.589 INFO:tasks.workunit.client.1.smithi130.stdout:4/821: unlink d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d81/le6 0 2024-05-13T22:48:41.600 INFO:tasks.workunit.client.1.smithi130.stdout:2/847: truncate d4/db/dc/d18/f23 2045795 0 2024-05-13T22:48:41.601 INFO:tasks.workunit.client.1.smithi130.stdout:2/848: write d4/db/dbf/df9/d30/d88/dc5/dcd/ffa [818134,107586] 0 2024-05-13T22:48:41.602 INFO:tasks.workunit.client.1.smithi130.stdout:6/807: truncate d4/d15/d1d/d60/da2/da4/dac/fc5 1140808 0 2024-05-13T22:48:41.606 INFO:tasks.workunit.client.1.smithi130.stdout:6/808: symlink d4/d15/d1d/d28/l115 0 2024-05-13T22:48:41.607 INFO:tasks.workunit.client.1.smithi130.stdout:3/863: getdents d8/dc/d7b/da3/daf/df8 0 2024-05-13T22:48:41.608 INFO:tasks.workunit.client.1.smithi130.stdout:3/864: stat d8/d1a/d55/f9b 0 2024-05-13T22:48:41.608 INFO:tasks.workunit.client.1.smithi130.stdout:7/863: truncate d7/da/d86/f8a 877468 0 2024-05-13T22:48:41.609 INFO:tasks.workunit.client.1.smithi130.stdout:2/849: symlink d4/db/dbf/l125 0 2024-05-13T22:48:41.610 INFO:tasks.workunit.client.1.smithi130.stdout:8/769: dwrite d1/d2/db/d15/d16/d54/d58/dab/fe3 [0,4194304] 0 2024-05-13T22:48:41.611 INFO:tasks.workunit.client.1.smithi130.stdout:6/809: mknod d4/d15/d1d/d60/da2/da4/ddd/c116 0 2024-05-13T22:48:41.615 INFO:tasks.workunit.client.1.smithi130.stdout:3/865: fdatasync d8/dc/d15/f9a 0 2024-05-13T22:48:41.619 INFO:tasks.workunit.client.1.smithi130.stdout:9/879: dwrite d1/d7/d23/d28/d2d/d2f/d63/fca [0,4194304] 0 2024-05-13T22:48:41.630 INFO:tasks.workunit.client.1.smithi130.stdout:2/850: symlink d4/db/dbf/d121/l126 0 2024-05-13T22:48:41.630 INFO:tasks.workunit.client.1.smithi130.stdout:4/822: dwrite d8/dc/f38 [0,4194304] 0 2024-05-13T22:48:41.634 INFO:tasks.workunit.client.1.smithi130.stdout:3/866: creat d8/dc/d7b/da3/daf/f11e x:0 0 0 2024-05-13T22:48:41.640 INFO:tasks.workunit.client.1.smithi130.stdout:9/880: unlink d1/db/f35 0 2024-05-13T22:48:41.642 INFO:tasks.workunit.client.1.smithi130.stdout:7/864: truncate d7/da/d1d/d5d/da6/d5b/d107/d46/f4f 1349804 0 2024-05-13T22:48:41.643 INFO:tasks.workunit.client.1.smithi130.stdout:8/770: mknod d1/d2/db/d15/d18/dfc/c11c 0 2024-05-13T22:48:41.647 INFO:tasks.workunit.client.1.smithi130.stdout:3/867: fsync d8/dc/d7b/f99 0 2024-05-13T22:48:41.648 INFO:tasks.workunit.client.1.smithi130.stdout:1/828: sync 2024-05-13T22:48:41.650 INFO:tasks.workunit.client.1.smithi130.stdout:4/823: rename d8/dc/d80/db0/df1/d33/d44/d77/c87 to d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dad/c104 0 2024-05-13T22:48:41.651 INFO:tasks.workunit.client.1.smithi130.stdout:1/829: read db/d1b/da3/d2a/d38/d46/d6d/fbe [3482180,73981] 0 2024-05-13T22:48:41.651 INFO:tasks.workunit.client.1.smithi130.stdout:4/824: fsync d8/dfa/dbd/ffb 0 2024-05-13T22:48:41.653 INFO:tasks.workunit.client.1.smithi130.stdout:6/810: dread d4/d15/d1d/d28/f5e [0,4194304] 0 2024-05-13T22:48:41.654 INFO:tasks.workunit.client.1.smithi130.stdout:8/771: creat d1/d2/db/d15/d18/d42/d53/dd0/f11d x:0 0 0 2024-05-13T22:48:41.655 INFO:tasks.workunit.client.1.smithi130.stdout:2/851: mkdir d4/db/d26/d9e/de4/df2/d127 0 2024-05-13T22:48:41.661 INFO:tasks.workunit.client.1.smithi130.stdout:8/772: dwrite d1/d2/db/d15/d18/d42/d53/dd0/f11d [0,4194304] 0 2024-05-13T22:48:41.667 INFO:tasks.workunit.client.1.smithi130.stdout:5/854: sync 2024-05-13T22:48:41.667 INFO:tasks.workunit.client.1.smithi130.stdout:0/879: sync 2024-05-13T22:48:41.667 INFO:tasks.workunit.client.1.smithi130.stdout:6/811: mknod d4/d15/d1d/d60/da2/da4/c117 0 2024-05-13T22:48:41.668 INFO:tasks.workunit.client.1.smithi130.stdout:4/825: dwrite d8/f7b [0,4194304] 0 2024-05-13T22:48:41.668 INFO:tasks.workunit.client.1.smithi130.stdout:2/852: truncate d4/f5f 4329165 0 2024-05-13T22:48:41.674 INFO:tasks.workunit.client.1.smithi130.stdout:9/881: dread d1/d15/d9e/dfa/f10d [0,4194304] 0 2024-05-13T22:48:41.675 INFO:tasks.workunit.client.1.smithi130.stdout:7/865: dread d7/da/d1d/d5d/da6/d5b/d107/d46/d92/fad [0,4194304] 0 2024-05-13T22:48:41.678 INFO:tasks.workunit.client.1.smithi130.stdout:8/773: fdatasync d1/f33 0 2024-05-13T22:48:41.680 INFO:tasks.workunit.client.1.smithi130.stdout:1/830: getdents db/d1b/da3/d49/d56/d88/df9/d100 0 2024-05-13T22:48:41.680 INFO:tasks.workunit.client.1.smithi130.stdout:4/826: dwrite d8/dc/d26/f55 [0,4194304] 0 2024-05-13T22:48:41.686 INFO:tasks.workunit.client.1.smithi130.stdout:5/855: mkdir d6/dd/d18/d6e/d82/d4b/d122 0 2024-05-13T22:48:41.686 INFO:tasks.workunit.client.1.smithi130.stdout:5/856: fdatasync d6/d1a/d72/fd9 0 2024-05-13T22:48:41.687 INFO:tasks.workunit.client.1.smithi130.stdout:0/880: creat d0/d7/dd/d17/d52/d67/d68/f11f x:0 0 0 2024-05-13T22:48:41.696 INFO:tasks.workunit.client.1.smithi130.stdout:8/774: creat d1/d2/db/d15/d18/dba/d111/f11e x:0 0 0 2024-05-13T22:48:41.696 INFO:tasks.workunit.client.1.smithi130.stdout:7/866: dwrite d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/f110 [0,4194304] 0 2024-05-13T22:48:41.703 INFO:tasks.workunit.client.1.smithi130.stdout:0/881: mknod d0/d7/dd/d2e/da8/dcd/c120 0 2024-05-13T22:48:41.705 INFO:tasks.workunit.client.1.smithi130.stdout:0/882: readlink d0/d7/dd/d2e/l36 0 2024-05-13T22:48:41.705 INFO:tasks.workunit.client.1.smithi130.stdout:3/868: dwrite d8/f4b [0,4194304] 0 2024-05-13T22:48:41.706 INFO:tasks.workunit.client.1.smithi130.stdout:3/869: chown d8/dc/d15/dfd/def/l10f 171 1 2024-05-13T22:48:41.707 INFO:tasks.workunit.client.1.smithi130.stdout:8/775: mknod d1/d2/db/d15/d7e/d9f/c11f 0 2024-05-13T22:48:41.707 INFO:tasks.workunit.client.1.smithi130.stdout:3/870: chown d8/dc/d7b/l86 11 1 2024-05-13T22:48:41.708 INFO:tasks.workunit.client.1.smithi130.stdout:7/867: fsync d7/da/d1d/d5d/da6/d5b/ff7 0 2024-05-13T22:48:41.711 INFO:tasks.workunit.client.1.smithi130.stdout:4/827: link d8/dc/d80/fe3 d8/dc/d26/dab/f105 0 2024-05-13T22:48:41.714 INFO:tasks.workunit.client.1.smithi130.stdout:5/857: fsync d6/d1a/d92/f9f 0 2024-05-13T22:48:41.714 INFO:tasks.workunit.client.1.smithi130.stdout:5/858: chown d6/d7/d32 629 1 2024-05-13T22:48:41.716 INFO:tasks.workunit.client.1.smithi130.stdout:1/831: dread db/d1b/da3/d49/f48 [0,4194304] 0 2024-05-13T22:48:41.716 INFO:tasks.workunit.client.1.smithi130.stdout:5/859: chown d6/dd/d18/d6e/d82/d71/l116 15895 1 2024-05-13T22:48:41.716 INFO:tasks.workunit.client.1.smithi130.stdout:9/882: creat d1/d7/d23/d28/d32/f124 x:0 0 0 2024-05-13T22:48:41.717 INFO:tasks.workunit.client.1.smithi130.stdout:5/860: write d6/d7/f28 [3188890,70332] 0 2024-05-13T22:48:41.718 INFO:tasks.workunit.client.1.smithi130.stdout:9/883: write d1/d15/d51/db4/d11f/fe1 [420621,122682] 0 2024-05-13T22:48:41.719 INFO:tasks.workunit.client.1.smithi130.stdout:1/832: truncate db/d1b/da3/d72/d75/dee/df6/f103 930934 0 2024-05-13T22:48:41.719 INFO:tasks.workunit.client.1.smithi130.stdout:5/861: truncate d6/d7/d19/fd8 931549 0 2024-05-13T22:48:41.720 INFO:tasks.workunit.client.1.smithi130.stdout:3/871: chown d8/dc/d15/ca5 1621 1 2024-05-13T22:48:41.728 INFO:tasks.workunit.client.1.smithi130.stdout:7/868: rmdir d7 39 2024-05-13T22:48:41.732 INFO:tasks.workunit.client.1.smithi130.stdout:6/812: getdents d4/d15/d1d/d60 0 2024-05-13T22:48:41.732 INFO:tasks.workunit.client.1.smithi130.stdout:8/776: write d1/d2/db/d15/d18/dba/fd1 [913345,92709] 0 2024-05-13T22:48:41.734 INFO:tasks.workunit.client.1.smithi130.stdout:0/883: rmdir d0/d7/dd/d2e/d4b 39 2024-05-13T22:48:41.737 INFO:tasks.workunit.client.1.smithi130.stdout:9/884: unlink d1/d7/d23/d3d/d43/ffc 0 2024-05-13T22:48:41.738 INFO:tasks.workunit.client.1.smithi130.stdout:5/862: symlink d6/d1a/daf/l123 0 2024-05-13T22:48:41.738 INFO:tasks.workunit.client.1.smithi130.stdout:5/863: readlink d6/d7/d19/d4d/d8a/la6 0 2024-05-13T22:48:41.743 INFO:tasks.workunit.client.1.smithi130.stdout:7/869: write d7/da/d1d/d5d/da6/d5b/f6d [5064655,108839] 0 2024-05-13T22:48:41.746 INFO:tasks.workunit.client.1.smithi130.stdout:4/828: creat d8/dc/d80/db0/df1/d33/dee/f106 x:0 0 0 2024-05-13T22:48:41.747 INFO:tasks.workunit.client.1.smithi130.stdout:8/777: mkdir d1/d2/db/d15/d16/d54/d58/d120 0 2024-05-13T22:48:41.750 INFO:tasks.workunit.client.1.smithi130.stdout:6/813: rename d4/d15/d1d/f38 to d4/d15/d1d/d28/d4e/d9e/ddb/f118 0 2024-05-13T22:48:41.750 INFO:tasks.workunit.client.1.smithi130.stdout:3/872: dwrite d8/dc/d7b/da3/daf/fb7 [0,4194304] 0 2024-05-13T22:48:41.751 INFO:tasks.workunit.client.1.smithi130.stdout:0/884: creat d0/d7/dd/d2e/d9e/f121 x:0 0 0 2024-05-13T22:48:41.751 INFO:tasks.workunit.client.1.smithi130.stdout:9/885: dread d1/d7/d23/d28/d2d/d2f/d63/d6b/f84 [0,4194304] 0 2024-05-13T22:48:41.751 INFO:tasks.workunit.client.1.smithi130.stdout:6/814: fsync d4/d15/d1d/d60/fcc 0 2024-05-13T22:48:41.756 INFO:tasks.workunit.client.1.smithi130.stdout:5/864: mkdir d6/dd/d18/d6e/d82/d4b/dbc/d124 0 2024-05-13T22:48:41.768 INFO:tasks.workunit.client.1.smithi130.stdout:6/815: dread - d4/d15/d1d/d28/d75/da8/f94 zero size 2024-05-13T22:48:41.772 INFO:tasks.workunit.client.1.smithi130.stdout:9/886: truncate d1/d7/d23/d28/f64 109217 0 2024-05-13T22:48:41.773 INFO:tasks.workunit.client.1.smithi130.stdout:9/887: readlink d1/l14 0 2024-05-13T22:48:41.773 INFO:tasks.workunit.client.1.smithi130.stdout:7/870: getdents d7/da/d1d/db1/df5 0 2024-05-13T22:48:41.775 INFO:tasks.workunit.client.1.smithi130.stdout:6/816: dwrite d4/d15/d1f/d64/f81 [0,4194304] 0 2024-05-13T22:48:41.776 INFO:tasks.workunit.client.1.smithi130.stdout:1/833: getdents db 0 2024-05-13T22:48:41.777 INFO:tasks.workunit.client.1.smithi130.stdout:8/778: dwrite d1/f33 [4194304,4194304] 0 2024-05-13T22:48:41.778 INFO:tasks.workunit.client.1.smithi130.stdout:3/873: symlink d8/dc/d7b/l11f 0 2024-05-13T22:48:41.778 INFO:tasks.workunit.client.1.smithi130.stdout:5/865: write d6/d7/d19/d4d/d6a/fc0 [1055106,34126] 0 2024-05-13T22:48:41.781 INFO:tasks.workunit.client.1.smithi130.stdout:0/885: mknod d0/d7/dd/d17/d1f/d2c/d45/d57/df7/c122 0 2024-05-13T22:48:41.783 INFO:tasks.workunit.client.1.smithi130.stdout:2/853: sync 2024-05-13T22:48:41.783 INFO:tasks.workunit.client.1.smithi130.stdout:0/886: stat d0/d7/dd/d69/fc8 0 2024-05-13T22:48:41.784 INFO:tasks.workunit.client.1.smithi130.stdout:4/829: creat d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f107 x:0 0 0 2024-05-13T22:48:41.786 INFO:tasks.workunit.client.1.smithi130.stdout:5/866: dwrite d6/d1a/daf/fe1 [0,4194304] 0 2024-05-13T22:48:41.786 INFO:tasks.workunit.client.1.smithi130.stdout:9/888: symlink d1/d7/d69/l125 0 2024-05-13T22:48:41.787 INFO:tasks.workunit.client.1.smithi130.stdout:7/871: readlink d7/da/d1d/d5d/da6/d39/l3f 0 2024-05-13T22:48:41.789 INFO:tasks.workunit.client.1.smithi130.stdout:6/817: readlink d4/d15/d1f/d3c/d70/lba 0 2024-05-13T22:48:41.790 INFO:tasks.workunit.client.1.smithi130.stdout:1/834: stat db/d1b/da3/c7c 0 2024-05-13T22:48:41.790 INFO:tasks.workunit.client.1.smithi130.stdout:1/835: chown ca 221853 1 2024-05-13T22:48:41.791 INFO:tasks.workunit.client.1.smithi130.stdout:6/818: chown d4/d15/d1d/daa/fd0 47 1 2024-05-13T22:48:41.793 INFO:tasks.workunit.client.1.smithi130.stdout:8/779: creat d1/d2/db/d15/d7e/f121 x:0 0 0 2024-05-13T22:48:41.796 INFO:tasks.workunit.client.1.smithi130.stdout:6/819: write d4/d15/d1d/d28/d75/d7d/ffc [188744,16387] 0 2024-05-13T22:48:41.801 INFO:tasks.workunit.client.1.smithi130.stdout:1/836: rename db/c15 to db/d1b/da3/d2a/d4c/c10b 0 2024-05-13T22:48:41.802 INFO:tasks.workunit.client.1.smithi130.stdout:8/780: symlink d1/d2/db/d15/d7e/d9f/l122 0 2024-05-13T22:48:41.803 INFO:tasks.workunit.client.1.smithi130.stdout:6/820: creat d4/d15/d1d/db8/f119 x:0 0 0 2024-05-13T22:48:41.804 INFO:tasks.workunit.client.1.smithi130.stdout:5/867: mknod d6/dd/d18/d6e/d82/d4b/dbc/d124/c125 0 2024-05-13T22:48:41.807 INFO:tasks.workunit.client.1.smithi130.stdout:3/874: creat d8/dc/f120 x:0 0 0 2024-05-13T22:48:41.808 INFO:tasks.workunit.client.1.smithi130.stdout:4/830: creat d8/dc/d80/db0/df1/f108 x:0 0 0 2024-05-13T22:48:41.809 INFO:tasks.workunit.client.1.smithi130.stdout:6/821: mknod d4/d15/d1d/d28/d75/da8/c11a 0 2024-05-13T22:48:41.810 INFO:tasks.workunit.client.1.smithi130.stdout:9/889: link d1/d7/d23/d28/d73/d8d/f9c d1/d7/d69/f126 0 2024-05-13T22:48:41.811 INFO:tasks.workunit.client.1.smithi130.stdout:5/868: symlink d6/d7/d19/d4d/d8a/df7/l126 0 2024-05-13T22:48:41.812 INFO:tasks.workunit.client.1.smithi130.stdout:8/781: dread d1/d2/db/d15/d18/d42/d53/d56/f61 [0,4194304] 0 2024-05-13T22:48:41.814 INFO:tasks.workunit.client.1.smithi130.stdout:3/875: read d8/fbd [2749127,29902] 0 2024-05-13T22:48:41.817 INFO:tasks.workunit.client.1.smithi130.stdout:7/872: dread d7/da/d1d/d5d/da6/d9/d4a/dbd/f6c [0,4194304] 0 2024-05-13T22:48:41.818 INFO:tasks.workunit.client.1.smithi130.stdout:2/854: write d4/db/d26/d9e/fad [1323122,71616] 0 2024-05-13T22:48:41.818 INFO:tasks.workunit.client.1.smithi130.stdout:9/890: rmdir d1/d7/d23/d28/d2d/d2f/d63 39 2024-05-13T22:48:41.818 INFO:tasks.workunit.client.1.smithi130.stdout:3/876: dread d8/d1a/dfa/d83/f101 [0,4194304] 0 2024-05-13T22:48:41.822 INFO:tasks.workunit.client.1.smithi130.stdout:3/877: fdatasync d8/dc/d7b/da3/daf/fb8 0 2024-05-13T22:48:41.823 INFO:tasks.workunit.client.1.smithi130.stdout:8/782: dread d1/d2/db/d15/d18/f60 [0,4194304] 0 2024-05-13T22:48:41.824 INFO:tasks.workunit.client.1.smithi130.stdout:3/878: chown d8/d1a/dfa/d27/fe6 3881 1 2024-05-13T22:48:41.825 INFO:tasks.workunit.client.1.smithi130.stdout:8/783: chown d1/d2/db/lb3 31661361 1 2024-05-13T22:48:41.826 INFO:tasks.workunit.client.1.smithi130.stdout:7/873: dwrite d7/da/d1d/d5d/d70/f106 [0,4194304] 0 2024-05-13T22:48:41.828 INFO:tasks.workunit.client.1.smithi130.stdout:0/887: dread d0/d7/dd/d2e/d4b/f65 [0,4194304] 0 2024-05-13T22:48:41.828 INFO:tasks.workunit.client.1.smithi130.stdout:5/869: rename d6/d7/d19/d4d/d6a/f9c to d6/dd/d18/d6e/d82/d4b/de7/f127 0 2024-05-13T22:48:41.828 INFO:tasks.workunit.client.1.smithi130.stdout:7/874: chown d7/da/d1d/d5d/da6/d5b/d107 4845 1 2024-05-13T22:48:41.830 INFO:tasks.workunit.client.1.smithi130.stdout:4/831: link d8/dc/d80/db0/df1/d33/d44/fa3 d8/dc/d80/db0/df1/d33/d3f/d57/dc0/f109 0 2024-05-13T22:48:41.838 INFO:tasks.workunit.client.1.smithi130.stdout:6/822: write d4/d15/d1f/d46/d50/f54 [1831203,86208] 0 2024-05-13T22:48:41.849 INFO:tasks.workunit.client.1.smithi130.stdout:2/855: write d4/db/dc/d18/f1d [1496238,11847] 0 2024-05-13T22:48:41.850 INFO:tasks.workunit.client.1.smithi130.stdout:0/888: symlink d0/d7/dd/d17/d1f/d2c/d45/d57/l123 0 2024-05-13T22:48:41.850 INFO:tasks.workunit.client.1.smithi130.stdout:2/856: chown d4/d56/dff/l60 5991 1 2024-05-13T22:48:41.855 INFO:tasks.workunit.client.1.smithi130.stdout:6/823: unlink d4/d15/d1f/l3b 0 2024-05-13T22:48:41.855 INFO:tasks.workunit.client.1.smithi130.stdout:9/891: truncate d1/d7/d23/d28/d32/d65/fa7 1625285 0 2024-05-13T22:48:41.856 INFO:tasks.workunit.client.1.smithi130.stdout:2/857: dwrite d4/db/dc/f79 [0,4194304] 0 2024-05-13T22:48:41.856 INFO:tasks.workunit.client.1.smithi130.stdout:8/784: dwrite d1/d2/db/d15/d18/d1a/f9d [0,4194304] 0 2024-05-13T22:48:41.860 INFO:tasks.workunit.client.1.smithi130.stdout:3/879: truncate d8/dc/d7b/f90 2519733 0 2024-05-13T22:48:41.863 INFO:tasks.workunit.client.1.smithi130.stdout:5/870: fsync d6/d1a/d92/fe5 0 2024-05-13T22:48:41.865 INFO:tasks.workunit.client.1.smithi130.stdout:5/871: fsync d6/dd/d18/d6e/d82/d71/f81 0 2024-05-13T22:48:41.865 INFO:tasks.workunit.client.1.smithi130.stdout:9/892: dwrite d1/d7/d23/d28/d73/db2/f106 [0,4194304] 0 2024-05-13T22:48:41.870 INFO:tasks.workunit.client.1.smithi130.stdout:7/875: mkdir d7/da/d1d/d5d/da6/d9/d118 0 2024-05-13T22:48:41.875 INFO:tasks.workunit.client.1.smithi130.stdout:0/889: mkdir d0/d7/dd/d2e/da8/dcd/d124 0 2024-05-13T22:48:41.879 INFO:tasks.workunit.client.1.smithi130.stdout:8/785: chown d1/d2/db/d15/d18/dba/cbd 130765296 1 2024-05-13T22:48:41.885 INFO:tasks.workunit.client.1.smithi130.stdout:2/858: rename d4/da8/dc9/faf to d4/db/dbf/df9/d30/d88/d99/f128 0 2024-05-13T22:48:41.886 INFO:tasks.workunit.client.1.smithi130.stdout:2/859: chown d4/db/dbf/d42/l10b 41296 1 2024-05-13T22:48:41.886 INFO:tasks.workunit.client.1.smithi130.stdout:2/860: chown d4/d56/dff/c6b 44520607 1 2024-05-13T22:48:41.887 INFO:tasks.workunit.client.1.smithi130.stdout:3/880: chown d8/fa 54 1 2024-05-13T22:48:41.889 INFO:tasks.workunit.client.1.smithi130.stdout:5/872: dread d6/d7/f3b [0,4194304] 0 2024-05-13T22:48:41.896 INFO:tasks.workunit.client.1.smithi130.stdout:6/824: getdents d4/d15/d114 0 2024-05-13T22:48:41.897 INFO:tasks.workunit.client.1.smithi130.stdout:0/890: rmdir d0/d7/dd/d17/d1f/d2c/d45 39 2024-05-13T22:48:41.902 INFO:tasks.workunit.client.1.smithi130.stdout:9/893: rename d1/d7/d23/d3d/cc8 to d1/d7/d23/d28/d32/d65/c127 0 2024-05-13T22:48:41.907 INFO:tasks.workunit.client.1.smithi130.stdout:2/861: unlink d4/d56/dff/d4a/f82 0 2024-05-13T22:48:41.911 INFO:tasks.workunit.client.1.smithi130.stdout:5/873: mknod d6/dd/d18/d6e/dc7/df6/c128 0 2024-05-13T22:48:41.915 INFO:tasks.workunit.client.1.smithi130.stdout:1/837: link db/d1b/da3/d49/d56/d88/f9a db/d82/f10c 0 2024-05-13T22:48:41.915 INFO:tasks.workunit.client.1.smithi130.stdout:6/825: mkdir d4/d16/d11b 0 2024-05-13T22:48:41.917 INFO:tasks.workunit.client.1.smithi130.stdout:0/891: dread d0/d7/dd/d17/d1f/d2c/d5a/f66 [0,4194304] 0 2024-05-13T22:48:41.917 INFO:tasks.workunit.client.1.smithi130.stdout:4/832: sync 2024-05-13T22:48:41.917 INFO:tasks.workunit.client.1.smithi130.stdout:8/786: link d1/d2/db/d15/d18/fb0 d1/d2/db/d15/d7e/dcd/f123 0 2024-05-13T22:48:41.922 INFO:tasks.workunit.client.1.smithi130.stdout:6/826: dwrite d4/d15/d1d/d28/f47 [0,4194304] 0 2024-05-13T22:48:41.922 INFO:tasks.workunit.client.1.smithi130.stdout:6/827: chown d4/c5 34 1 2024-05-13T22:48:41.923 INFO:tasks.workunit.client.1.smithi130.stdout:5/874: creat d6/d1a/d72/f129 x:0 0 0 2024-05-13T22:48:41.931 INFO:tasks.workunit.client.1.smithi130.stdout:3/881: link d8/dc/d15/dcc/l103 d8/d1a/dfa/d27/l121 0 2024-05-13T22:48:41.932 INFO:tasks.workunit.client.1.smithi130.stdout:3/882: dread - d8/dc/d7b/da3/daf/f11e zero size 2024-05-13T22:48:41.934 INFO:tasks.workunit.client.1.smithi130.stdout:4/833: mknod d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/c10a 0 2024-05-13T22:48:41.936 INFO:tasks.workunit.client.1.smithi130.stdout:6/828: chown d4/d15/d1d/d28/d4e/f99 1519005 1 2024-05-13T22:48:41.940 INFO:tasks.workunit.client.1.smithi130.stdout:5/875: fsync d6/dd/d18/d6e/d82/d4b/dbc/fe2 0 2024-05-13T22:48:41.940 INFO:tasks.workunit.client.1.smithi130.stdout:9/894: write d1/d7/d23/d28/d32/d45/f9b [5202046,33099] 0 2024-05-13T22:48:41.941 INFO:tasks.workunit.client.1.smithi130.stdout:8/787: dread d1/d2/db/d15/d18/f4e [0,4194304] 0 2024-05-13T22:48:41.942 INFO:tasks.workunit.client.1.smithi130.stdout:9/895: dread - d1/d7/d23/d28/d2d/f74 zero size 2024-05-13T22:48:41.944 INFO:tasks.workunit.client.1.smithi130.stdout:7/876: rename d7/da/d1d/c61 to d7/da/d1d/d5d/da6/d9/d4a/dbd/c119 0 2024-05-13T22:48:41.947 INFO:tasks.workunit.client.1.smithi130.stdout:0/892: symlink d0/d7/dd/d17/d52/d10e/l125 0 2024-05-13T22:48:41.949 INFO:tasks.workunit.client.1.smithi130.stdout:3/883: creat d8/d1a/dfa/d2c/f122 x:0 0 0 2024-05-13T22:48:41.950 INFO:tasks.workunit.client.1.smithi130.stdout:9/896: dread d1/d7/d23/d28/d2d/f46 [0,4194304] 0 2024-05-13T22:48:41.952 INFO:tasks.workunit.client.1.smithi130.stdout:6/829: readlink d4/d15/d1d/db8/le2 0 2024-05-13T22:48:41.952 INFO:tasks.workunit.client.1.smithi130.stdout:2/862: link d4/db/dc/d18/d58/fcf d4/db/dbf/d121/f129 0 2024-05-13T22:48:41.957 INFO:tasks.workunit.client.1.smithi130.stdout:1/838: link db/d1b/da3/d49/d56/lc9 db/d1b/da3/d2a/d38/d46/d6d/l10d 0 2024-05-13T22:48:41.958 INFO:tasks.workunit.client.1.smithi130.stdout:7/877: creat d7/da/d1d/d5d/da6/d5b/d107/db9/d109/f11a x:0 0 0 2024-05-13T22:48:41.961 INFO:tasks.workunit.client.1.smithi130.stdout:1/839: dread db/d1b/da3/d49/d56/d88/fc7 [0,4194304] 0 2024-05-13T22:48:41.961 INFO:tasks.workunit.client.1.smithi130.stdout:0/893: creat d0/d7/dd/d2e/da8/f126 x:0 0 0 2024-05-13T22:48:41.962 INFO:tasks.workunit.client.1.smithi130.stdout:3/884: creat d8/d1a/dfa/d27/f123 x:0 0 0 2024-05-13T22:48:41.965 INFO:tasks.workunit.client.1.smithi130.stdout:4/834: truncate d8/fa 2297912 0 2024-05-13T22:48:41.967 INFO:tasks.workunit.client.1.smithi130.stdout:5/876: dwrite d6/d2f/f39 [0,4194304] 0 2024-05-13T22:48:41.968 INFO:tasks.workunit.client.1.smithi130.stdout:9/897: mkdir d1/d7/d23/d28/d73/d128 0 2024-05-13T22:48:41.973 INFO:tasks.workunit.client.1.smithi130.stdout:8/788: mknod d1/d9b/df3/dff/c124 0 2024-05-13T22:48:41.975 INFO:tasks.workunit.client.1.smithi130.stdout:7/878: chown d7/da/d1d/d5d/da6/d5b/d107/d46/l60 133 1 2024-05-13T22:48:41.977 INFO:tasks.workunit.client.1.smithi130.stdout:3/885: dread - d8/d1a/dfa/d2c/fd8 zero size 2024-05-13T22:48:41.979 INFO:tasks.workunit.client.1.smithi130.stdout:9/898: fdatasync d1/d15/f37 0 2024-05-13T22:48:41.981 INFO:tasks.workunit.client.1.smithi130.stdout:2/863: rename d4/db/d26/d9e/lc8 to d4/db/dbf/df9/d66/db9/l12a 0 2024-05-13T22:48:41.983 INFO:tasks.workunit.client.1.smithi130.stdout:0/894: symlink d0/d7/dd/d17/l127 0 2024-05-13T22:48:41.985 INFO:tasks.workunit.client.1.smithi130.stdout:7/879: unlink d7/da/d1d/d5d/da6/d5b/d107/db9/fee 0 2024-05-13T22:48:41.987 INFO:tasks.workunit.client.1.smithi130.stdout:7/880: stat d7/da/d1d/db1/dbe/dd3/cfc 0 2024-05-13T22:48:41.987 INFO:tasks.workunit.client.1.smithi130.stdout:4/835: dread d8/dc/f28 [0,4194304] 0 2024-05-13T22:48:41.987 INFO:tasks.workunit.client.1.smithi130.stdout:4/836: fdatasync d8/dc/d80/db0/df1/d33/feb 0 2024-05-13T22:48:41.989 INFO:tasks.workunit.client.1.smithi130.stdout:5/877: mknod d6/dd/d18/d6e/dc7/d115/c12a 0 2024-05-13T22:48:41.992 INFO:tasks.workunit.client.1.smithi130.stdout:1/840: rename db/d1b/da3/d72/d75/dee/df6/da0/fcb to db/d9c/f10e 0 2024-05-13T22:48:41.994 INFO:tasks.workunit.client.1.smithi130.stdout:6/830: link d4/fc4 d4/d15/d1d/d28/d75/da8/da0/de4/f11c 0 2024-05-13T22:48:41.994 INFO:tasks.workunit.client.1.smithi130.stdout:6/831: chown d4/d108 350 1 2024-05-13T22:48:41.995 INFO:tasks.workunit.client.1.smithi130.stdout:6/832: chown d4/d15/d1f/d46/fd9 3671308 1 2024-05-13T22:48:41.995 INFO:tasks.workunit.client.1.smithi130.stdout:8/789: mkdir d1/d2/db/d15/d16/d54/d125 0 2024-05-13T22:48:41.997 INFO:tasks.workunit.client.1.smithi130.stdout:3/886: dread d8/d1a/dfa/ff3 [0,4194304] 0 2024-05-13T22:48:42.001 INFO:tasks.workunit.client.1.smithi130.stdout:5/878: creat d6/dd/d18/d6e/d82/d10b/f12b x:0 0 0 2024-05-13T22:48:42.002 INFO:tasks.workunit.client.1.smithi130.stdout:9/899: symlink d1/d7/d23/d28/d32/d45/d10e/l129 0 2024-05-13T22:48:42.005 INFO:tasks.workunit.client.1.smithi130.stdout:7/881: dread d7/f71 [0,4194304] 0 2024-05-13T22:48:42.007 INFO:tasks.workunit.client.1.smithi130.stdout:7/882: truncate d7/da/d1d/d5d/da6/f1c 5416047 0 2024-05-13T22:48:42.007 INFO:tasks.workunit.client.1.smithi130.stdout:2/864: rename d4/f7 to d4/da8/f12b 0 2024-05-13T22:48:42.008 INFO:tasks.workunit.client.1.smithi130.stdout:2/865: stat d4/da8/dc9/f11f 0 2024-05-13T22:48:42.008 INFO:tasks.workunit.client.1.smithi130.stdout:6/833: creat d4/d15/d1f/d3c/f11d x:0 0 0 2024-05-13T22:48:42.009 INFO:tasks.workunit.client.1.smithi130.stdout:6/834: readlink d4/d15/d1d/d28/d75/ld4 0 2024-05-13T22:48:42.010 INFO:tasks.workunit.client.1.smithi130.stdout:8/790: symlink d1/d2/db/d15/d7e/dc2/dcf/l126 0 2024-05-13T22:48:42.011 INFO:tasks.workunit.client.1.smithi130.stdout:3/887: dread d8/d1a/dfa/d83/f101 [0,4194304] 0 2024-05-13T22:48:42.012 INFO:tasks.workunit.client.1.smithi130.stdout:3/888: write d8/d1a/d55/f111 [220506,126537] 0 2024-05-13T22:48:42.014 INFO:tasks.workunit.client.1.smithi130.stdout:5/879: mknod d6/d1a/daf/c12c 0 2024-05-13T22:48:42.018 INFO:tasks.workunit.client.1.smithi130.stdout:1/841: symlink db/d82/l10f 0 2024-05-13T22:48:42.021 INFO:tasks.workunit.client.1.smithi130.stdout:7/883: truncate d7/da/d1d/d5d/da6/d9/d66/f7a 426639 0 2024-05-13T22:48:42.026 INFO:tasks.workunit.client.1.smithi130.stdout:0/895: read d0/d7/dd/d17/d1f/d2c/d45/d6b/f7c [942400,48100] 0 2024-05-13T22:48:42.029 INFO:tasks.workunit.client.1.smithi130.stdout:3/889: mknod d8/dc/d7b/da3/dd3/ddd/c124 0 2024-05-13T22:48:42.031 INFO:tasks.workunit.client.1.smithi130.stdout:2/866: write d4/db/dbf/d121/f129 [1031074,88515] 0 2024-05-13T22:48:42.032 INFO:tasks.workunit.client.1.smithi130.stdout:5/880: symlink d6/dd/d18/d6e/d82/d4b/dbc/l12d 0 2024-05-13T22:48:42.033 INFO:tasks.workunit.client.1.smithi130.stdout:2/867: stat d4/db/d26/d9e/fad 0 2024-05-13T22:48:42.033 INFO:tasks.workunit.client.1.smithi130.stdout:3/890: dread d8/d1a/d1c/f25 [0,4194304] 0 2024-05-13T22:48:42.035 INFO:tasks.workunit.client.1.smithi130.stdout:6/835: write d4/d15/d1f/f74 [728713,110854] 0 2024-05-13T22:48:42.036 INFO:tasks.workunit.client.1.smithi130.stdout:6/836: write d4/d15/d1f/d64/f81 [170135,88105] 0 2024-05-13T22:48:42.038 INFO:tasks.workunit.client.1.smithi130.stdout:4/837: getdents d8/dc/d80/db0/df1/d33/d3f/d57/dc0 0 2024-05-13T22:48:42.041 INFO:tasks.workunit.client.1.smithi130.stdout:5/881: creat d6/dd/d79/f12e x:0 0 0 2024-05-13T22:48:42.041 INFO:tasks.workunit.client.1.smithi130.stdout:9/900: dwrite d1/d7/d69/f126 [0,4194304] 0 2024-05-13T22:48:42.042 INFO:tasks.workunit.client.1.smithi130.stdout:5/882: fsync d6/d7/f22 0 2024-05-13T22:48:42.042 INFO:tasks.workunit.client.1.smithi130.stdout:2/868: unlink d4/db/dbf/l81 0 2024-05-13T22:48:42.042 INFO:tasks.workunit.client.1.smithi130.stdout:9/901: write d1/d7/d23/fe3 [758881,31588] 0 2024-05-13T22:48:42.044 INFO:tasks.workunit.client.1.smithi130.stdout:7/884: creat d7/da/d1d/d5d/da6/d9/d118/f11b x:0 0 0 2024-05-13T22:48:42.044 INFO:tasks.workunit.client.1.smithi130.stdout:9/902: chown d1/d7/d69/l8c 161 1 2024-05-13T22:48:42.048 INFO:tasks.workunit.client.1.smithi130.stdout:8/791: rename d1/d2/db/d15/d18/d42/cbc to d1/dc/c127 0 2024-05-13T22:48:42.049 INFO:tasks.workunit.client.1.smithi130.stdout:0/896: dwrite d0/d7/dd/d69/da9/d8d/f9d [0,4194304] 0 2024-05-13T22:48:42.052 INFO:tasks.workunit.client.1.smithi130.stdout:2/869: dwrite d4/db/dbf/df9/d30/d88/dc5/f11a [0,4194304] 0 2024-05-13T22:48:42.053 INFO:tasks.workunit.client.1.smithi130.stdout:2/870: chown d4/db/dbf/l46 3871 1 2024-05-13T22:48:42.058 INFO:tasks.workunit.client.1.smithi130.stdout:5/883: mkdir d6/dd/d18/d6e/d82/d4b/de7/d12f 0 2024-05-13T22:48:42.061 INFO:tasks.workunit.client.1.smithi130.stdout:5/884: dread - d6/f10e zero size 2024-05-13T22:48:42.063 INFO:tasks.workunit.client.1.smithi130.stdout:2/871: dwrite d4/db/dc/d18/d58/f70 [4194304,4194304] 0 2024-05-13T22:48:42.064 INFO:tasks.workunit.client.1.smithi130.stdout:4/838: read d8/f12 [1334629,80516] 0 2024-05-13T22:48:42.065 INFO:tasks.workunit.client.1.smithi130.stdout:2/872: fsync d4/d56/dff/d4a/f112 0 2024-05-13T22:48:42.065 INFO:tasks.workunit.client.1.smithi130.stdout:2/873: fsync d4/f5 0 2024-05-13T22:48:42.066 INFO:tasks.workunit.client.1.smithi130.stdout:3/891: rename d8/d35/ld0 to d8/d35/ddb/l125 0 2024-05-13T22:48:42.067 INFO:tasks.workunit.client.1.smithi130.stdout:2/874: chown d4/db/dc/le6 4088 1 2024-05-13T22:48:42.069 INFO:tasks.workunit.client.1.smithi130.stdout:8/792: fsync d1/d2/db/d15/d18/d42/d53/d56/f61 0 2024-05-13T22:48:42.070 INFO:tasks.workunit.client.1.smithi130.stdout:6/837: mknod d4/d15/d1d/d102/c11e 0 2024-05-13T22:48:42.071 INFO:tasks.workunit.client.1.smithi130.stdout:8/793: write d1/d2/db/d15/d18/dba/fef [96259,53457] 0 2024-05-13T22:48:42.073 INFO:tasks.workunit.client.1.smithi130.stdout:2/875: dread d4/db/f2c [4194304,4194304] 0 2024-05-13T22:48:42.074 INFO:tasks.workunit.client.1.smithi130.stdout:5/885: mknod d6/dd/d18/c130 0 2024-05-13T22:48:42.078 INFO:tasks.workunit.client.1.smithi130.stdout:4/839: mkdir d8/dc/d80/db0/d10b 0 2024-05-13T22:48:42.081 INFO:tasks.workunit.client.1.smithi130.stdout:7/885: dread d7/da/d1d/d5d/da6/f19 [0,4194304] 0 2024-05-13T22:48:42.082 INFO:tasks.workunit.client.1.smithi130.stdout:0/897: rename d0/fb7 to d0/d7/dd/d2e/d9e/da7/de1/f128 0 2024-05-13T22:48:42.084 INFO:tasks.workunit.client.1.smithi130.stdout:6/838: readlink d4/d15/lb9 0 2024-05-13T22:48:42.086 INFO:tasks.workunit.client.1.smithi130.stdout:1/842: getdents db/d4a/de9 0 2024-05-13T22:48:42.087 INFO:tasks.workunit.client.1.smithi130.stdout:9/903: mknod d1/d7/d23/d28/d73/c12a 0 2024-05-13T22:48:42.088 INFO:tasks.workunit.client.1.smithi130.stdout:8/794: creat d1/d2/db/d15/d7e/dc2/dcf/f128 x:0 0 0 2024-05-13T22:48:42.090 INFO:tasks.workunit.client.1.smithi130.stdout:2/876: unlink d4/db/dc/d18/l11c 0 2024-05-13T22:48:42.094 INFO:tasks.workunit.client.1.smithi130.stdout:7/886: mknod d7/da/d86/c11c 0 2024-05-13T22:48:42.095 INFO:tasks.workunit.client.1.smithi130.stdout:7/887: chown d7/da/d1d/db1/dbe 8355281 1 2024-05-13T22:48:42.095 INFO:tasks.workunit.client.1.smithi130.stdout:4/840: dwrite d8/f8b [4194304,4194304] 0 2024-05-13T22:48:42.097 INFO:tasks.workunit.client.1.smithi130.stdout:4/841: fsync d8/dc/d80/db0/df1/d33/d3f/f6d 0 2024-05-13T22:48:42.097 INFO:tasks.workunit.client.1.smithi130.stdout:0/898: fsync d0/d7/dd/d17/d52/ff3 0 2024-05-13T22:48:42.100 INFO:tasks.workunit.client.1.smithi130.stdout:6/839: write d4/d15/d1d/d28/f71 [3357586,45991] 0 2024-05-13T22:48:42.101 INFO:tasks.workunit.client.1.smithi130.stdout:1/843: dread db/d1b/da3/d49/d56/d88/fc7 [0,4194304] 0 2024-05-13T22:48:42.104 INFO:tasks.workunit.client.1.smithi130.stdout:2/877: symlink d4/d56/l12c 0 2024-05-13T22:48:42.105 INFO:tasks.workunit.client.1.smithi130.stdout:2/878: fdatasync d4/db/dbf/df9/f44 0 2024-05-13T22:48:42.105 INFO:tasks.workunit.client.1.smithi130.stdout:3/892: mkdir d8/d35/d126 0 2024-05-13T22:48:42.106 INFO:tasks.workunit.client.1.smithi130.stdout:2/879: read - d4/da8/dc9/dbb/f117 zero size 2024-05-13T22:48:42.107 INFO:tasks.workunit.client.1.smithi130.stdout:3/893: write d8/dc/d7b/da3/fec [858197,94288] 0 2024-05-13T22:48:42.107 INFO:tasks.workunit.client.1.smithi130.stdout:2/880: chown d4/d56/dff/d4a/dc6/d10c/l10f 0 1 2024-05-13T22:48:42.108 INFO:tasks.workunit.client.1.smithi130.stdout:3/894: fsync d8/d1a/dfa/d27/fe6 0 2024-05-13T22:48:42.113 INFO:tasks.workunit.client.1.smithi130.stdout:4/842: symlink d8/dbb/l10c 0 2024-05-13T22:48:42.115 INFO:tasks.workunit.client.1.smithi130.stdout:1/844: truncate db/d9c/f8a 3207892 0 2024-05-13T22:48:42.119 INFO:tasks.workunit.client.1.smithi130.stdout:2/881: truncate d4/db/dc/f37 4204652 0 2024-05-13T22:48:42.119 INFO:tasks.workunit.client.1.smithi130.stdout:9/904: dread d1/d7/d23/d28/d2d/d2f/d63/f9f [0,4194304] 0 2024-05-13T22:48:42.120 INFO:tasks.workunit.client.1.smithi130.stdout:2/882: fdatasync d4/f1e 0 2024-05-13T22:48:42.121 INFO:tasks.workunit.client.1.smithi130.stdout:9/905: write d1/d7/d23/fe3 [86671,112368] 0 2024-05-13T22:48:42.127 INFO:tasks.workunit.client.1.smithi130.stdout:8/795: rmdir d1/d2/db/d15/d7e/dc2/df4 0 2024-05-13T22:48:42.132 INFO:tasks.workunit.client.1.smithi130.stdout:4/843: rename d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/fed to d8/dc/d80/db0/df1/d33/dee/f10d 0 2024-05-13T22:48:42.133 INFO:tasks.workunit.client.1.smithi130.stdout:7/888: dwrite d7/da/d1d/d42/f74 [0,4194304] 0 2024-05-13T22:48:42.139 INFO:tasks.workunit.client.1.smithi130.stdout:3/895: mknod d8/dc/d7b/da3/dd3/ddd/d114/c127 0 2024-05-13T22:48:42.139 INFO:tasks.workunit.client.1.smithi130.stdout:4/844: dwrite d8/dc/d80/db0/df1/d33/d3f/f6d [0,4194304] 0 2024-05-13T22:48:42.141 INFO:tasks.workunit.client.1.smithi130.stdout:0/899: creat d0/f129 x:0 0 0 2024-05-13T22:48:42.143 INFO:tasks.workunit.client.1.smithi130.stdout:1/845: symlink db/l110 0 2024-05-13T22:48:42.146 INFO:tasks.workunit.client.1.smithi130.stdout:7/889: unlink d7/f71 0 2024-05-13T22:48:42.152 INFO:tasks.workunit.client.1.smithi130.stdout:7/890: dwrite d7/da/d1d/d5d/da6/d9/d4a/dbd/d40/f77 [4194304,4194304] 0 2024-05-13T22:48:42.152 INFO:tasks.workunit.client.1.smithi130.stdout:4/845: truncate d8/dc/f28 2083570 0 2024-05-13T22:48:42.157 INFO:tasks.workunit.client.1.smithi130.stdout:1/846: dread - db/d1b/da3/d2a/d38/d46/d6d/fd6 zero size 2024-05-13T22:48:42.158 INFO:tasks.workunit.client.1.smithi130.stdout:2/883: link d4/db/dbf/df9/le9 d4/da8/l12d 0 2024-05-13T22:48:42.159 INFO:tasks.workunit.client.1.smithi130.stdout:4/846: read d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/f3a [3787065,35525] 0 2024-05-13T22:48:42.159 INFO:tasks.workunit.client.1.smithi130.stdout:9/906: dwrite d1/d7/d23/d28/d2d/d2f/f93 [0,4194304] 0 2024-05-13T22:48:42.160 INFO:tasks.workunit.client.1.smithi130.stdout:1/847: write db/d1b/da3/d2a/d4c/f65 [748880,118444] 0 2024-05-13T22:48:42.162 INFO:tasks.workunit.client.1.smithi130.stdout:9/907: dread - d1/d15/d9e/dfa/f123 zero size 2024-05-13T22:48:42.163 INFO:tasks.workunit.client.1.smithi130.stdout:1/848: chown db/d1b/da3/d49/d56/d88/c8d 27559962 1 2024-05-13T22:48:42.165 INFO:tasks.workunit.client.1.smithi130.stdout:3/896: truncate d8/dc/f9f 3920336 0 2024-05-13T22:48:42.170 INFO:tasks.workunit.client.1.smithi130.stdout:4/847: dwrite d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/fdc [0,4194304] 0 2024-05-13T22:48:42.174 INFO:tasks.workunit.client.1.smithi130.stdout:8/796: dwrite d1/f10 [0,4194304] 0 2024-05-13T22:48:42.180 INFO:tasks.workunit.client.1.smithi130.stdout:7/891: mkdir d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/d11d 0 2024-05-13T22:48:42.181 INFO:tasks.workunit.client.1.smithi130.stdout:2/884: mkdir d4/db/d26/d12e 0 2024-05-13T22:48:42.183 INFO:tasks.workunit.client.1.smithi130.stdout:7/892: readlink d7/da/d1d/d5d/da6/d9/d66/l3b 0 2024-05-13T22:48:42.186 INFO:tasks.workunit.client.1.smithi130.stdout:1/849: write db/d58/fb1 [600589,50553] 0 2024-05-13T22:48:42.190 INFO:tasks.workunit.client.1.smithi130.stdout:2/885: symlink d4/db/dbf/df9/d66/db9/l12f 0 2024-05-13T22:48:42.191 INFO:tasks.workunit.client.1.smithi130.stdout:8/797: getdents d1/d2/db/d15/d16/d54/dfb/d110 0 2024-05-13T22:48:42.194 INFO:tasks.workunit.client.1.smithi130.stdout:1/850: unlink db/d1b/lcf 0 2024-05-13T22:48:42.196 INFO:tasks.workunit.client.1.smithi130.stdout:7/893: mknod d7/da/d1d/d5d/da6/d5b/d107/db9/d111/c11e 0 2024-05-13T22:48:42.200 INFO:tasks.workunit.client.1.smithi130.stdout:1/851: write db/f1a [5252952,34040] 0 2024-05-13T22:48:42.201 INFO:tasks.workunit.client.1.smithi130.stdout:3/897: getdents d8/dc/d15/dac 0 2024-05-13T22:48:42.202 INFO:tasks.workunit.client.1.smithi130.stdout:3/898: chown d8/dc/d7b/da3/daf/df8/f108 16427868 1 2024-05-13T22:48:42.204 INFO:tasks.workunit.client.1.smithi130.stdout:9/908: dread d1/f1e [0,4194304] 0 2024-05-13T22:48:42.206 INFO:tasks.workunit.client.1.smithi130.stdout:4/848: dread d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f3c [0,4194304] 0 2024-05-13T22:48:42.207 INFO:tasks.workunit.client.1.smithi130.stdout:4/849: dread - d8/dc/d80/db0/df1/d33/dbf/fda zero size 2024-05-13T22:48:42.212 INFO:tasks.workunit.client.1.smithi130.stdout:2/886: creat d4/f130 x:0 0 0 2024-05-13T22:48:42.214 INFO:tasks.workunit.client.1.smithi130.stdout:8/798: dread d1/d2/db/d15/d18/f4c [0,4194304] 0 2024-05-13T22:48:42.215 INFO:tasks.workunit.client.1.smithi130.stdout:1/852: dwrite db/ff7 [0,4194304] 0 2024-05-13T22:48:42.217 INFO:tasks.workunit.client.1.smithi130.stdout:4/850: rename d8/dc/d80/db0/fb4 to d8/dfa/dbd/f10e 0 2024-05-13T22:48:42.217 INFO:tasks.workunit.client.1.smithi130.stdout:2/887: truncate d4/db/dbf/df9/d30/d88/d99/d9a/f106 797562 0 2024-05-13T22:48:42.218 INFO:tasks.workunit.client.1.smithi130.stdout:5/886: sync 2024-05-13T22:48:42.221 INFO:tasks.workunit.client.1.smithi130.stdout:7/894: link d7/da/d1d/d5d/da6/d9/d66/l57 d7/d8f/l11f 0 2024-05-13T22:48:42.226 INFO:tasks.workunit.client.1.smithi130.stdout:9/909: dwrite d1/f9d [0,4194304] 0 2024-05-13T22:48:42.227 INFO:tasks.workunit.client.1.smithi130.stdout:4/851: creat d8/dc/d80/d85/f10f x:0 0 0 2024-05-13T22:48:42.227 INFO:tasks.workunit.client.1.smithi130.stdout:1/853: dwrite db/d1b/da3/d49/d56/f7d [4194304,4194304] 0 2024-05-13T22:48:42.233 INFO:tasks.workunit.client.1.smithi130.stdout:7/895: symlink d7/da/d1d/db1/dbe/dd3/l120 0 2024-05-13T22:48:42.237 INFO:tasks.workunit.client.1.smithi130.stdout:3/899: getdents d8/d1a/dfa 0 2024-05-13T22:48:42.238 INFO:tasks.workunit.client.1.smithi130.stdout:2/888: mkdir d4/db/d26/d9e/d131 0 2024-05-13T22:48:42.241 INFO:tasks.workunit.client.1.smithi130.stdout:9/910: unlink d1/db/l105 0 2024-05-13T22:48:42.241 INFO:tasks.workunit.client.1.smithi130.stdout:8/799: write d1/d2/db/d15/d18/f60 [686399,4291] 0 2024-05-13T22:48:42.242 INFO:tasks.workunit.client.1.smithi130.stdout:1/854: dread - db/d1b/da3/fce zero size 2024-05-13T22:48:42.246 INFO:tasks.workunit.client.1.smithi130.stdout:2/889: dread d4/db/dc/d18/f36 [0,4194304] 0 2024-05-13T22:48:42.249 INFO:tasks.workunit.client.1.smithi130.stdout:9/911: unlink d1/d7/d69/l8c 0 2024-05-13T22:48:42.250 INFO:tasks.workunit.client.1.smithi130.stdout:9/912: write d1/d15/f7b [3504645,101783] 0 2024-05-13T22:48:42.250 INFO:tasks.workunit.client.1.smithi130.stdout:0/900: sync 2024-05-13T22:48:42.250 INFO:tasks.workunit.client.1.smithi130.stdout:6/840: sync 2024-05-13T22:48:42.251 INFO:tasks.workunit.client.1.smithi130.stdout:8/800: fsync d1/d2/db/d15/d18/d1a/f90 0 2024-05-13T22:48:42.253 INFO:tasks.workunit.client.1.smithi130.stdout:1/855: creat db/d1b/da3/d2a/d4c/dc2/f111 x:0 0 0 2024-05-13T22:48:42.253 INFO:tasks.workunit.client.1.smithi130.stdout:8/801: stat d1/d2/db/d15/d16/l35 0 2024-05-13T22:48:42.260 INFO:tasks.workunit.client.1.smithi130.stdout:7/896: truncate d7/da/d1d/f6f 2805200 0 2024-05-13T22:48:42.262 INFO:tasks.workunit.client.1.smithi130.stdout:2/890: dread d4/db/dbf/df9/d30/f9c [4194304,4194304] 0 2024-05-13T22:48:42.265 INFO:tasks.workunit.client.1.smithi130.stdout:8/802: dread d1/d2/f39 [0,4194304] 0 2024-05-13T22:48:42.271 INFO:tasks.workunit.client.1.smithi130.stdout:9/913: unlink d1/d7/f55 0 2024-05-13T22:48:42.274 INFO:tasks.workunit.client.1.smithi130.stdout:0/901: fsync d0/d7/dd/d17/d52/d67/d68/d96/f84 0 2024-05-13T22:48:42.275 INFO:tasks.workunit.client.1.smithi130.stdout:5/887: getdents d6/dd/d18/d6e/dc7/df6 0 2024-05-13T22:48:42.280 INFO:tasks.workunit.client.1.smithi130.stdout:6/841: truncate d4/d15/d1d/d28/d4e/f99 3206629 0 2024-05-13T22:48:42.281 INFO:tasks.workunit.client.1.smithi130.stdout:5/888: dread d6/dd/d18/d6e/d82/d71/f81 [0,4194304] 0 2024-05-13T22:48:42.284 INFO:tasks.workunit.client.1.smithi130.stdout:4/852: link d8/dc/d80/c32 d8/dc/d80/db0/df1/d33/d44/d61/c110 0 2024-05-13T22:48:42.284 INFO:tasks.workunit.client.1.smithi130.stdout:9/914: dread d1/d7/d23/d28/d2d/d2f/d63/f77 [0,4194304] 0 2024-05-13T22:48:42.286 INFO:tasks.workunit.client.1.smithi130.stdout:7/897: mknod d7/da/d1d/d5d/da6/d9/d4a/dbd/dc9/c121 0 2024-05-13T22:48:42.287 INFO:tasks.workunit.client.1.smithi130.stdout:7/898: stat d7/da/d1d/d5d/f89 0 2024-05-13T22:48:42.298 INFO:tasks.workunit.client.1.smithi130.stdout:0/902: rmdir d0/d7/dd/d2e/d4b 39 2024-05-13T22:48:42.304 INFO:tasks.workunit.client.1.smithi130.stdout:6/842: rename d4/fb1 to d4/d15/d1d/d28/d4e/f11f 0 2024-05-13T22:48:42.305 INFO:tasks.workunit.client.1.smithi130.stdout:5/889: read - d6/dd/fb6 zero size 2024-05-13T22:48:42.306 INFO:tasks.workunit.client.1.smithi130.stdout:5/890: write d6/d7/f22 [3585569,3430] 0 2024-05-13T22:48:42.308 INFO:tasks.workunit.client.1.smithi130.stdout:4/853: mknod d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/c111 0 2024-05-13T22:48:42.309 INFO:tasks.workunit.client.1.smithi130.stdout:9/915: truncate d1/d7/d23/d3d/d43/f75 544513 0 2024-05-13T22:48:42.309 INFO:tasks.workunit.client.1.smithi130.stdout:8/803: dwrite d1/d2/db/d15/d16/d54/fb8 [0,4194304] 0 2024-05-13T22:48:42.310 INFO:tasks.workunit.client.1.smithi130.stdout:9/916: chown d1/d7/d23/d28/ff3 0 1 2024-05-13T22:48:42.311 INFO:tasks.workunit.client.1.smithi130.stdout:3/900: link d8/l3f d8/d1a/dfa/d83/dba/l128 0 2024-05-13T22:48:42.311 INFO:tasks.workunit.client.1.smithi130.stdout:9/917: write d1/d7/d69/f126 [895631,81381] 0 2024-05-13T22:48:42.312 INFO:tasks.workunit.client.1.smithi130.stdout:2/891: mkdir d4/db/d26/d9e/d132 0 2024-05-13T22:48:42.314 INFO:tasks.workunit.client.1.smithi130.stdout:2/892: read d4/db/f28 [6233059,95083] 0 2024-05-13T22:48:42.314 INFO:tasks.workunit.client.1.smithi130.stdout:2/893: chown d4/da8 1042518912 1 2024-05-13T22:48:42.315 INFO:tasks.workunit.client.1.smithi130.stdout:9/918: dread d1/d7/d23/fe3 [0,4194304] 0 2024-05-13T22:48:42.315 INFO:tasks.workunit.client.1.smithi130.stdout:2/894: chown d4/d56/dff/d4a/d76/c84 6 1 2024-05-13T22:48:42.325 INFO:tasks.workunit.client.1.smithi130.stdout:4/854: unlink d8/dc/d80/db0/df1/d33/d44/l5d 0 2024-05-13T22:48:42.326 INFO:tasks.workunit.client.1.smithi130.stdout:4/855: readlink d8/dc/d80/db0/df1/d33/d44/d77/l8f 0 2024-05-13T22:48:42.327 INFO:tasks.workunit.client.1.smithi130.stdout:7/899: symlink d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/l122 0 2024-05-13T22:48:42.328 INFO:tasks.workunit.client.1.smithi130.stdout:7/900: readlink d7/da/d1d/lbf 0 2024-05-13T22:48:42.331 INFO:tasks.workunit.client.1.smithi130.stdout:3/901: truncate d8/d1a/d1c/f49 1818987 0 2024-05-13T22:48:42.333 INFO:tasks.workunit.client.1.smithi130.stdout:4/856: dwrite d8/dc/d80/db0/df1/d33/dbf/ff3 [0,4194304] 0 2024-05-13T22:48:42.335 INFO:tasks.workunit.client.1.smithi130.stdout:4/857: readlink d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/l5c 0 2024-05-13T22:48:42.336 INFO:tasks.workunit.client.1.smithi130.stdout:2/895: rename d4/db/d26/d9e/fad to d4/db/dbf/df9/d30/d88/dc5/f133 0 2024-05-13T22:48:42.337 INFO:tasks.workunit.client.1.smithi130.stdout:2/896: chown d4/db/dbf/df9/l2b 0 1 2024-05-13T22:48:42.338 INFO:tasks.workunit.client.1.smithi130.stdout:1/856: getdents db/d1b/da3 0 2024-05-13T22:48:42.344 INFO:tasks.workunit.client.1.smithi130.stdout:5/891: mknod d6/d7/d19/c131 0 2024-05-13T22:48:42.359 INFO:tasks.workunit.client.1.smithi130.stdout:0/903: write d0/d7/dd/d17/d1f/d2c/d45/d57/df7/fff [4076694,9456] 0 2024-05-13T22:48:42.362 INFO:tasks.workunit.client.1.smithi130.stdout:6/843: truncate d4/d15/d1d/d28/d75/fcf 3759349 0 2024-05-13T22:48:42.362 INFO:tasks.workunit.client.1.smithi130.stdout:9/919: dwrite d1/d7/d23/d28/d2d/d2f/d63/f68 [0,4194304] 0 2024-05-13T22:48:42.363 INFO:tasks.workunit.client.1.smithi130.stdout:9/920: chown d1/d7/d69/d79/fa8 25319418 1 2024-05-13T22:48:42.364 INFO:tasks.workunit.client.1.smithi130.stdout:4/858: chown d8/dc/d80/db0/df1/d33/d44/d56/fc5 0 1 2024-05-13T22:48:42.370 INFO:tasks.workunit.client.1.smithi130.stdout:5/892: dread d6/d1a/d92/f96 [0,4194304] 0 2024-05-13T22:48:42.370 INFO:tasks.workunit.client.1.smithi130.stdout:5/893: chown d6/d1a/d72 0 1 2024-05-13T22:48:42.373 INFO:tasks.workunit.client.1.smithi130.stdout:7/901: symlink d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/d11d/l123 0 2024-05-13T22:48:42.374 INFO:tasks.workunit.client.1.smithi130.stdout:5/894: dread d6/dd/d79/f106 [0,4194304] 0 2024-05-13T22:48:42.379 INFO:tasks.workunit.client.1.smithi130.stdout:2/897: dwrite d4/da8/dc9/f86 [0,4194304] 0 2024-05-13T22:48:42.382 INFO:tasks.workunit.client.1.smithi130.stdout:6/844: mknod d4/d15/d1d/d28/d75/da8/da0/dfe/c120 0 2024-05-13T22:48:42.383 INFO:tasks.workunit.client.1.smithi130.stdout:3/902: write d8/f34 [832306,113550] 0 2024-05-13T22:48:42.385 INFO:tasks.workunit.client.1.smithi130.stdout:9/921: read d1/d7/d23/d28/f64 [2528,84056] 0 2024-05-13T22:48:42.387 INFO:tasks.workunit.client.1.smithi130.stdout:2/898: dread d4/da8/dc9/fb4 [0,4194304] 0 2024-05-13T22:48:42.388 INFO:tasks.workunit.client.1.smithi130.stdout:4/859: mkdir d8/dc/d80/db0/df1/d33/d44/d56/d112 0 2024-05-13T22:48:42.391 INFO:tasks.workunit.client.1.smithi130.stdout:4/860: dread - d8/dc/d80/db0/df1/d33/d3f/d57/f93 zero size 2024-05-13T22:48:42.391 INFO:tasks.workunit.client.1.smithi130.stdout:6/845: dwrite d4/d15/d1f/d46/fb2 [8388608,4194304] 0 2024-05-13T22:48:42.391 INFO:tasks.workunit.client.1.smithi130.stdout:8/804: getdents d1/d9b 0 2024-05-13T22:48:42.400 INFO:tasks.workunit.client.1.smithi130.stdout:0/904: dread - d0/d7/dd/d2e/d4b/f10b zero size 2024-05-13T22:48:42.404 INFO:tasks.workunit.client.1.smithi130.stdout:5/895: write d6/d7/f46 [2977620,76734] 0 2024-05-13T22:48:42.406 INFO:tasks.workunit.client.1.smithi130.stdout:0/905: dwrite d0/d7/dd/d17/d1f/f119 [0,4194304] 0 2024-05-13T22:48:42.407 INFO:tasks.workunit.client.1.smithi130.stdout:9/922: mknod d1/d7/d23/d3d/d43/d113/c12b 0 2024-05-13T22:48:42.407 INFO:tasks.workunit.client.1.smithi130.stdout:0/906: write d0/d7/dd/d2e/da8/f126 [501957,59034] 0 2024-05-13T22:48:42.408 INFO:tasks.workunit.client.1.smithi130.stdout:9/923: chown d1/d7/d23/d3d/d43/cff 7 1 2024-05-13T22:48:42.422 INFO:tasks.workunit.client.1.smithi130.stdout:2/899: dread d4/d56/dff/d4a/d76/fdd [0,4194304] 0 2024-05-13T22:48:42.423 INFO:tasks.workunit.client.1.smithi130.stdout:2/900: truncate d4/db/dbf/df9/d30/d88/d99/d9a/ff3 140936 0 2024-05-13T22:48:42.425 INFO:tasks.workunit.client.1.smithi130.stdout:2/901: read d4/f8 [2787163,87911] 0 2024-05-13T22:48:42.428 INFO:tasks.workunit.client.1.smithi130.stdout:2/902: dread d4/db/dbf/d42/d4b/fa3 [0,4194304] 0 2024-05-13T22:48:42.434 INFO:tasks.workunit.client.1.smithi130.stdout:0/907: dread d0/d7/dd/d2e/f11b [0,4194304] 0 2024-05-13T22:48:42.439 INFO:tasks.workunit.client.1.smithi130.stdout:4/861: dwrite d8/dc/d80/db0/df1/f8a [0,4194304] 0 2024-05-13T22:48:42.442 INFO:tasks.workunit.client.1.smithi130.stdout:8/805: creat d1/d2/df8/f129 x:0 0 0 2024-05-13T22:48:42.445 INFO:tasks.workunit.client.1.smithi130.stdout:4/862: dwrite d8/dc/d80/db0/df1/d33/d44/d56/fae [0,4194304] 0 2024-05-13T22:48:42.446 INFO:tasks.workunit.client.1.smithi130.stdout:4/863: truncate d8/dc/d84/ff0 257163 0 2024-05-13T22:48:42.447 INFO:tasks.workunit.client.1.smithi130.stdout:4/864: truncate d8/dc/f9a 328159 0 2024-05-13T22:48:42.452 INFO:tasks.workunit.client.1.smithi130.stdout:6/846: symlink d4/d15/d1f/d46/d52/d10e/l121 0 2024-05-13T22:48:42.453 INFO:tasks.workunit.client.1.smithi130.stdout:6/847: chown d4/d15/d1f/d3c/f11d 0 1 2024-05-13T22:48:42.456 INFO:tasks.workunit.client.1.smithi130.stdout:7/902: link d7/da/d1d/d5d/da6/d5b/ff7 d7/da/d1d/d5d/da6/d7e/f124 0 2024-05-13T22:48:42.456 INFO:tasks.workunit.client.1.smithi130.stdout:7/903: write d7/f6a [3823113,11445] 0 2024-05-13T22:48:42.458 INFO:tasks.workunit.client.1.smithi130.stdout:7/904: chown d7/da/d1d/d5d/d70/f85 67981 1 2024-05-13T22:48:42.463 INFO:tasks.workunit.client.1.smithi130.stdout:3/903: rename d8/dc/d15/dfd/d9c/f9d to d8/dc/f129 0 2024-05-13T22:48:42.471 INFO:tasks.workunit.client.1.smithi130.stdout:0/908: mknod d0/d7/dd/d17/d52/d67/c12a 0 2024-05-13T22:48:42.472 INFO:tasks.workunit.client.1.smithi130.stdout:1/857: getdents db/d9c 0 2024-05-13T22:48:42.473 INFO:tasks.workunit.client.1.smithi130.stdout:0/909: write d0/d7/dd/d69/da9/d8d/f110 [767693,1048] 0 2024-05-13T22:48:42.473 INFO:tasks.workunit.client.1.smithi130.stdout:1/858: readlink db/d1b/da3/d2a/d4c/la2 0 2024-05-13T22:48:42.489 INFO:tasks.workunit.client.1.smithi130.stdout:2/903: write d4/db/dbf/d42/d4b/f4f [914835,13432] 0 2024-05-13T22:48:42.517 INFO:tasks.workunit.client.1.smithi130.stdout:5/896: sync 2024-05-13T22:48:42.523 INFO:tasks.workunit.client.1.smithi130.stdout:7/905: dwrite d7/da/d1d/d42/f58 [0,4194304] 0 2024-05-13T22:48:42.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.548+0000 7f990af5a700 1 -- 172.21.15.102:0/3195455929 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9904072b80 msgr2=0x7f9904109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:42.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.548+0000 7f990af5a700 1 --2- 172.21.15.102:0/3195455929 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9904072b80 0x7f9904109da0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009910 tx=0x7f98f8009c20 comp rx=0 tx=0).stop 2024-05-13T22:48:42.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.549+0000 7f990af5a700 1 -- 172.21.15.102:0/3195455929 shutdown_connections 2024-05-13T22:48:42.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.549+0000 7f990af5a700 1 --2- 172.21.15.102:0/3195455929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990410a370 0x7f990410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:42.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.549+0000 7f990af5a700 1 --2- 172.21.15.102:0/3195455929 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9904072b80 0x7f9904109da0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:42.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.549+0000 7f990af5a700 1 -- 172.21.15.102:0/3195455929 >> 172.21.15.102:0/3195455929 conn(0x7f990406d9f0 msgr2=0x7f990406fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:42.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.549+0000 7f990af5a700 1 -- 172.21.15.102:0/3195455929 shutdown_connections 2024-05-13T22:48:42.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.549+0000 7f990af5a700 1 -- 172.21.15.102:0/3195455929 wait complete. 2024-05-13T22:48:42.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.550+0000 7f990af5a700 1 Processor -- start 2024-05-13T22:48:42.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.550+0000 7f990af5a700 1 -- start start 2024-05-13T22:48:42.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.551+0000 7f990af5a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9904072b80 0x7f99041a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:42.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.551+0000 7f990af5a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990410a370 0x7f99041a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:42.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.551+0000 7f990af5a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99041a1790 con 0x7f990410a370 2024-05-13T22:48:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.551+0000 7f9908cf6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9904072b80 0x7f99041a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.551+0000 7f9908cf6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9904072b80 0x7f99041a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.102:33928/0 (socket says 172.21.15.102:33928) 2024-05-13T22:48:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.551+0000 7f9908cf6700 1 -- 172.21.15.102:0/3655852693 learned_addr learned my addr 172.21.15.102:0/3655852693 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:48:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.552+0000 7f990af5a700 1 -- 172.21.15.102:0/3655852693 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99041a61a0 con 0x7f9904072b80 2024-05-13T22:48:42.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.552+0000 7f9908cf6700 1 -- 172.21.15.102:0/3655852693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990410a370 msgr2=0x7f99041a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:42.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.552+0000 7f9908cf6700 1 --2- 172.21.15.102:0/3655852693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990410a370 0x7f99041a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:42.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.552+0000 7f9908cf6700 1 -- 172.21.15.102:0/3655852693 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f80095f0 con 0x7f9904072b80 2024-05-13T22:48:42.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.553+0000 7f9908cf6700 1 --2- 172.21.15.102:0/3655852693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9904072b80 0x7f99041a0c30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f98f80048c0 tx=0x7f98f80049a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:42.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.554+0000 7f9901ffb700 1 -- 172.21.15.102:0/3655852693 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98f801d070 con 0x7f9904072b80 2024-05-13T22:48:42.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.554+0000 7f9901ffb700 1 -- 172.21.15.102:0/3655852693 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f98f8004f50 con 0x7f9904072b80 2024-05-13T22:48:42.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.554+0000 7f990af5a700 1 -- 172.21.15.102:0/3655852693 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99041a6340 con 0x7f9904072b80 2024-05-13T22:48:42.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.555+0000 7f990af5a700 1 -- 172.21.15.102:0/3655852693 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99041a6880 con 0x7f9904072b80 2024-05-13T22:48:42.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.555+0000 7f9901ffb700 1 -- 172.21.15.102:0/3655852693 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98f800f7f0 con 0x7f9904072b80 2024-05-13T22:48:42.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.556+0000 7f990af5a700 1 -- 172.21.15.102:0/3655852693 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990419afa0 con 0x7f9904072b80 2024-05-13T22:48:42.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.558+0000 7f9901ffb700 1 -- 172.21.15.102:0/3655852693 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 24) v1 ==== 91288+0+0 (secure 0 0 0) 0x7f98f800fa30 con 0x7f9904072b80 2024-05-13T22:48:42.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.559+0000 7f9901ffb700 1 --2- 172.21.15.102:0/3655852693 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f98ec06d440 0x7f98ec06f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:42.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.559+0000 7f9903fff700 1 --2- 172.21.15.102:0/3655852693 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f98ec06d440 0x7f98ec06f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:42.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.560+0000 7f9901ffb700 1 -- 172.21.15.102:0/3655852693 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f98f808ebc0 con 0x7f9904072b80 2024-05-13T22:48:42.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.560+0000 7f9903fff700 1 --2- 172.21.15.102:0/3655852693 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f98ec06d440 0x7f98ec06f900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00ab90 tx=0x7f98fc009250 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:42.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.565+0000 7f9901ffb700 1 -- 172.21.15.102:0/3655852693 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f98f8059030 con 0x7f9904072b80 2024-05-13T22:48:42.563 INFO:tasks.workunit.client.1.smithi130.stdout:9/924: mkdir d1/d7/d23/d28/d73/d128/d12c 0 2024-05-13T22:48:42.563 INFO:tasks.workunit.client.1.smithi130.stdout:3/904: creat d8/d1a/dfa/d83/dba/f12a x:0 0 0 2024-05-13T22:48:42.577 INFO:tasks.workunit.client.1.smithi130.stdout:6/848: symlink d4/d15/d1f/d3c/d70/def/l122 0 2024-05-13T22:48:42.577 INFO:tasks.workunit.client.1.smithi130.stdout:6/849: chown d4/c5 1295144 1 2024-05-13T22:48:42.578 INFO:tasks.workunit.client.1.smithi130.stdout:6/850: chown d4/d15/d1d/d28/d75/le1 78891 1 2024-05-13T22:48:42.584 INFO:tasks.workunit.client.1.smithi130.stdout:0/910: dread d0/d7/dd/d2e/d4b/f105 [0,4194304] 0 2024-05-13T22:48:42.585 INFO:tasks.workunit.client.1.smithi130.stdout:0/911: stat d0/d7/dd/d17/d52/d5b/d5f 0 2024-05-13T22:48:42.590 INFO:tasks.workunit.client.1.smithi130.stdout:8/806: rename d1/d2/db/d15/d18/d42/la9 to d1/d2/db/d15/d18/d42/l12a 0 2024-05-13T22:48:42.591 INFO:tasks.workunit.client.1.smithi130.stdout:1/859: write db/d1b/da3/d2a/d38/d46/d6d/f73 [5067140,93458] 0 2024-05-13T22:48:42.594 INFO:tasks.workunit.client.1.smithi130.stdout:6/851: rmdir d4/d15/d1d/d28/d4e/d9e 39 2024-05-13T22:48:42.598 INFO:tasks.workunit.client.1.smithi130.stdout:2/904: write d4/db/dbf/d42/f89 [1299725,14509] 0 2024-05-13T22:48:42.599 INFO:tasks.workunit.client.1.smithi130.stdout:6/852: dwrite d4/d15/d1f/d46/d50/f54 [0,4194304] 0 2024-05-13T22:48:42.600 INFO:tasks.workunit.client.1.smithi130.stdout:6/853: dread - d4/d15/d1d/d60/da2/f113 zero size 2024-05-13T22:48:42.601 INFO:tasks.workunit.client.1.smithi130.stdout:7/906: mkdir d7/da/d1d/d5d/da6/d5b/d107/d46/d125 0 2024-05-13T22:48:42.602 INFO:tasks.workunit.client.1.smithi130.stdout:6/854: write d4/d15/d1f/d46/d50/f54 [3292630,55708] 0 2024-05-13T22:48:42.603 INFO:tasks.workunit.client.1.smithi130.stdout:7/907: write d7/da/d1d/d5d/da6/fcd [531845,89332] 0 2024-05-13T22:48:42.604 INFO:tasks.workunit.client.1.smithi130.stdout:9/925: creat d1/d7/d23/d28/d32/d45/ddb/f12d x:0 0 0 2024-05-13T22:48:42.605 INFO:tasks.workunit.client.1.smithi130.stdout:4/865: getdents d8/dc/d84 0 2024-05-13T22:48:42.606 INFO:tasks.workunit.client.1.smithi130.stdout:7/908: read - d7/da/d1d/db1/dbe/dd3/f10e zero size 2024-05-13T22:48:42.606 INFO:tasks.workunit.client.1.smithi130.stdout:7/909: dread - d7/da/f112 zero size 2024-05-13T22:48:42.607 INFO:tasks.workunit.client.1.smithi130.stdout:4/866: read d8/dc/d84/ff0 [4709,122724] 0 2024-05-13T22:48:42.614 INFO:tasks.workunit.client.1.smithi130.stdout:2/905: write d4/db/dbf/df9/d30/fd8 [6456133,59048] 0 2024-05-13T22:48:42.619 INFO:tasks.workunit.client.1.smithi130.stdout:0/912: unlink d0/d7/dd/d2e/d4b/f15 0 2024-05-13T22:48:42.621 INFO:tasks.workunit.client.1.smithi130.stdout:6/855: creat d4/d15/d1f/d46/d52/d10e/f123 x:0 0 0 2024-05-13T22:48:42.624 INFO:tasks.workunit.client.1.smithi130.stdout:8/807: unlink d1/d2/db/d15/d16/d54/lac 0 2024-05-13T22:48:42.626 INFO:tasks.workunit.client.1.smithi130.stdout:4/867: creat d8/dc/d80/db0/df1/d33/d44/db8/f113 x:0 0 0 2024-05-13T22:48:42.635 INFO:tasks.workunit.client.1.smithi130.stdout:7/910: mknod d7/da/c126 0 2024-05-13T22:48:42.641 INFO:tasks.workunit.client.1.smithi130.stdout:4/868: truncate d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f24 5278435 0 2024-05-13T22:48:42.642 INFO:tasks.workunit.client.1.smithi130.stdout:5/897: link d6/dd/d18/d6e/d82/db0/d11d/lc8 d6/d1a/d92/l132 0 2024-05-13T22:48:42.647 INFO:tasks.workunit.client.1.smithi130.stdout:4/869: dwrite d8/dc/f38 [0,4194304] 0 2024-05-13T22:48:42.649 INFO:tasks.workunit.client.1.smithi130.stdout:5/898: dwrite d6/dd/d18/d6e/d82/d10b/f12b [0,4194304] 0 2024-05-13T22:48:42.659 INFO:tasks.workunit.client.1.smithi130.stdout:1/860: truncate db/f1d 1514298 0 2024-05-13T22:48:42.661 INFO:tasks.workunit.client.1.smithi130.stdout:4/870: dread d8/dc/d80/db0/df1/d33/f4f [0,4194304] 0 2024-05-13T22:48:42.661 INFO:tasks.workunit.client.1.smithi130.stdout:8/808: truncate d1/d2/db/d15/d16/d54/d58/dab/fe3 593724 0 2024-05-13T22:48:42.667 INFO:tasks.workunit.client.1.smithi130.stdout:7/911: rename d7/da/d1d/d5d/da6/d7e/d8b to d7/da/d1d/d5d/da6/d9/d4a/dbd/d40/d127 0 2024-05-13T22:48:42.668 INFO:tasks.workunit.client.1.smithi130.stdout:0/913: truncate d0/d7/dd/d17/d52/d5b/d5f/f70 1155252 0 2024-05-13T22:48:42.670 INFO:tasks.workunit.client.1.smithi130.stdout:6/856: truncate d4/d15/d1d/fcd 1977433 0 2024-05-13T22:48:42.673 INFO:tasks.workunit.client.1.smithi130.stdout:2/906: link d4/db/dc/d18/f1d d4/d56/dff/f134 0 2024-05-13T22:48:42.675 INFO:tasks.workunit.client.1.smithi130.stdout:5/899: creat d6/d7/d19/d4d/d8a/df7/f133 x:0 0 0 2024-05-13T22:48:42.677 INFO:tasks.workunit.client.1.smithi130.stdout:5/900: write d6/dd/d18/d6e/d82/d71/f100 [324673,106314] 0 2024-05-13T22:48:42.678 INFO:tasks.workunit.client.1.smithi130.stdout:3/905: sync 2024-05-13T22:48:42.684 INFO:tasks.workunit.client.1.smithi130.stdout:3/906: dwrite d8/d1a/f47 [0,4194304] 0 2024-05-13T22:48:42.685 INFO:tasks.workunit.client.1.smithi130.stdout:3/907: write d8/dc/d7b/fe0 [907524,24320] 0 2024-05-13T22:48:42.688 INFO:tasks.workunit.client.1.smithi130.stdout:1/861: rename db/d58/fc1 to db/d1b/da3/d49/d56/d88/f112 0 2024-05-13T22:48:42.697 INFO:tasks.workunit.client.1.smithi130.stdout:0/914: unlink d0/d7/dd/d17/fbb 0 2024-05-13T22:48:42.703 INFO:tasks.workunit.client.1.smithi130.stdout:0/915: dwrite d0/f8 [4194304,4194304] 0 2024-05-13T22:48:42.704 INFO:tasks.workunit.client.1.smithi130.stdout:1/862: dread db/d1b/da3/f89 [0,4194304] 0 2024-05-13T22:48:42.704 INFO:tasks.workunit.client.1.smithi130.stdout:5/901: symlink d6/d7/l134 0 2024-05-13T22:48:42.713 INFO:tasks.workunit.client.1.smithi130.stdout:3/908: unlink d8/dc/d7b/da3/dd3/ddd/d114/c127 0 2024-05-13T22:48:42.717 INFO:tasks.workunit.client.1.smithi130.stdout:6/857: rename d4/d15/d1d/d28/d75/d7d/ffc to d4/d15/d1f/d46/d52/d10e/f124 0 2024-05-13T22:48:42.723 INFO:tasks.workunit.client.1.smithi130.stdout:2/907: mkdir d4/da8/dc9/dbb/d108/d135 0 2024-05-13T22:48:42.725 INFO:tasks.workunit.client.1.smithi130.stdout:5/902: creat d6/d7/d19/d4d/d6a/f135 x:0 0 0 2024-05-13T22:48:42.726 INFO:tasks.workunit.client.1.smithi130.stdout:9/926: sync 2024-05-13T22:48:42.728 INFO:tasks.workunit.client.1.smithi130.stdout:4/871: creat d8/dc/d80/db0/df1/d33/f114 x:0 0 0 2024-05-13T22:48:42.728 INFO:tasks.workunit.client.1.smithi130.stdout:4/872: dread - d8/dfa/dbd/ffc zero size 2024-05-13T22:48:42.729 INFO:tasks.workunit.client.1.smithi130.stdout:8/809: creat d1/d2/db/d15/d16/f12b x:0 0 0 2024-05-13T22:48:42.737 INFO:tasks.workunit.client.1.smithi130.stdout:7/912: write d7/da/d1d/f6f [3793396,119210] 0 2024-05-13T22:48:42.737 INFO:tasks.workunit.client.1.smithi130.stdout:6/858: fsync d4/d15/d1d/d28/d75/d7d/fc1 0 2024-05-13T22:48:42.739 INFO:tasks.workunit.client.1.smithi130.stdout:6/859: chown d4/d15/d1f/d46/d50/c6e 1 1 2024-05-13T22:48:42.748 INFO:tasks.workunit.client.1.smithi130.stdout:5/903: truncate d6/dd/f33 2965785 0 2024-05-13T22:48:42.750 INFO:tasks.workunit.client.1.smithi130.stdout:9/927: write d1/d15/f1d [944214,73084] 0 2024-05-13T22:48:42.753 INFO:tasks.workunit.client.1.smithi130.stdout:2/908: write d4/d56/f61 [273608,76891] 0 2024-05-13T22:48:42.754 INFO:tasks.workunit.client.1.smithi130.stdout:2/909: chown d4/db/dbf/df9/d30/d88/d99/lc3 465 1 2024-05-13T22:48:42.757 INFO:tasks.workunit.client.1.smithi130.stdout:1/863: truncate db/d1b/da3/d2a/f2f 1634301 0 2024-05-13T22:48:42.758 INFO:tasks.workunit.client.1.smithi130.stdout:3/909: creat d8/d1a/dfa/d83/d11d/f12b x:0 0 0 2024-05-13T22:48:42.759 INFO:tasks.workunit.client.1.smithi130.stdout:7/913: mknod d7/da/d1d/d5d/da6/d9/d4a/dbd/dc9/c128 0 2024-05-13T22:48:42.760 INFO:tasks.workunit.client.1.smithi130.stdout:6/860: rmdir d4/d15/d1d/d28/d75/da8 39 2024-05-13T22:48:42.761 INFO:tasks.workunit.client.1.smithi130.stdout:4/873: dwrite d8/dc/d80/db0/df1/d33/dee/f10d [0,4194304] 0 2024-05-13T22:48:42.763 INFO:tasks.workunit.client.1.smithi130.stdout:5/904: readlink d6/d1a/d92/la2 0 2024-05-13T22:48:42.765 INFO:tasks.workunit.client.1.smithi130.stdout:5/905: write d6/d2f/f59 [723353,7958] 0 2024-05-13T22:48:42.766 INFO:tasks.workunit.client.1.smithi130.stdout:9/928: fdatasync d1/d7/d23/d3d/f5c 0 2024-05-13T22:48:42.767 INFO:tasks.workunit.client.1.smithi130.stdout:9/929: readlink d1/d7/d23/d28/d32/l3e 0 2024-05-13T22:48:42.772 INFO:tasks.workunit.client.1.smithi130.stdout:9/930: dread d1/d15/f78 [0,4194304] 0 2024-05-13T22:48:42.774 INFO:tasks.workunit.client.1.smithi130.stdout:1/864: rename db/d1b/da3/d5e to db/d1b/da3/d2a/d4c/d5b/d113 0 2024-05-13T22:48:42.775 INFO:tasks.workunit.client.1.smithi130.stdout:3/910: creat d8/d1a/dfa/d83/dba/f12c x:0 0 0 2024-05-13T22:48:42.776 INFO:tasks.workunit.client.1.smithi130.stdout:6/861: truncate d4/f49 708703 0 2024-05-13T22:48:42.776 INFO:tasks.workunit.client.1.smithi130.stdout:7/914: chown d7/da/d1d/d5d/da6/d9/c24 70607 1 2024-05-13T22:48:42.780 INFO:tasks.workunit.client.1.smithi130.stdout:0/916: link d0/d7/dd/d17/d52/ca5 d0/d7/dd/d17/d1f/db0/c12b 0 2024-05-13T22:48:42.785 INFO:tasks.workunit.client.1.smithi130.stdout:7/915: dwrite d7/da/d1d/d5d/da6/d5b/d107/db9/d109/f11a [0,4194304] 0 2024-05-13T22:48:42.785 INFO:tasks.workunit.client.1.smithi130.stdout:5/906: unlink d6/dd/d18/cd0 0 2024-05-13T22:48:42.786 INFO:tasks.workunit.client.1.smithi130.stdout:5/907: fsync d6/d2f/ff8 0 2024-05-13T22:48:42.787 INFO:tasks.workunit.client.1.smithi130.stdout:4/874: dread d8/dc/d80/db0/fd4 [0,4194304] 0 2024-05-13T22:48:42.791 INFO:tasks.workunit.client.1.smithi130.stdout:5/908: write d6/dd/f37 [368218,123441] 0 2024-05-13T22:48:42.795 INFO:tasks.workunit.client.1.smithi130.stdout:0/917: dwrite d0/d7/dd/d17/d52/d5b/d5f/f11d [0,4194304] 0 2024-05-13T22:48:42.797 INFO:tasks.workunit.client.1.smithi130.stdout:2/910: dread d4/db/dbf/d42/d4b/f83 [0,4194304] 0 2024-05-13T22:48:42.799 INFO:tasks.workunit.client.1.smithi130.stdout:8/810: dwrite d1/d2/db/d15/d18/fb0 [0,4194304] 0 2024-05-13T22:48:42.802 INFO:tasks.workunit.client.1.smithi130.stdout:8/811: chown d1/d2/cc7 12 1 2024-05-13T22:48:42.807 INFO:tasks.workunit.client.1.smithi130.stdout:7/916: fsync d7/da/d1d/d5d/da6/d39/f7c 0 2024-05-13T22:48:42.820 INFO:tasks.workunit.client.1.smithi130.stdout:5/909: mkdir d6/dd/d18/d6e/dc7/d115/d136 0 2024-05-13T22:48:42.831 INFO:tasks.workunit.client.1.smithi130.stdout:3/911: dread d8/d35/f10d [0,4194304] 0 2024-05-13T22:48:42.832 INFO:tasks.workunit.client.1.smithi130.stdout:5/910: mkdir d6/d7/d19/d4d/d137 0 2024-05-13T22:48:42.832 INFO:tasks.workunit.client.1.smithi130.stdout:0/918: mkdir d0/d7/dd/d69/da9/d8d/df4/d12c 0 2024-05-13T22:48:42.835 INFO:tasks.workunit.client.1.smithi130.stdout:4/875: rmdir d8/dc/d80/db0/d10b 0 2024-05-13T22:48:42.836 INFO:tasks.workunit.client.1.smithi130.stdout:6/862: getdents d4/d15/d1d/d28/d75/d7d 0 2024-05-13T22:48:42.837 INFO:tasks.workunit.client.1.smithi130.stdout:6/863: stat d4/d15/d1f/d46/c4f 0 2024-05-13T22:48:42.837 INFO:tasks.workunit.client.1.smithi130.stdout:3/912: creat d8/d1a/dfa/d2c/f12d x:0 0 0 2024-05-13T22:48:42.842 INFO:tasks.workunit.client.1.smithi130.stdout:0/919: creat d0/d7/dd/d17/d52/d5b/d5f/f12d x:0 0 0 2024-05-13T22:48:42.843 INFO:tasks.workunit.client.1.smithi130.stdout:0/920: read d0/d7/fb [1175994,35137] 0 2024-05-13T22:48:42.847 INFO:tasks.workunit.client.1.smithi130.stdout:6/864: mkdir d4/d15/d1d/d60/da2/da4/dac/d125 0 2024-05-13T22:48:42.847 INFO:tasks.workunit.client.1.smithi130.stdout:6/865: stat d4/d15/d1d/d60/f66 0 2024-05-13T22:48:42.849 INFO:tasks.workunit.client.1.smithi130.stdout:2/911: getdents d4/db/dc/d18 0 2024-05-13T22:48:42.850 INFO:tasks.workunit.client.1.smithi130.stdout:8/812: getdents d1/d9b/df3 0 2024-05-13T22:48:42.853 INFO:tasks.workunit.client.1.smithi130.stdout:4/876: dread d8/f1b [0,4194304] 0 2024-05-13T22:48:42.855 INFO:tasks.workunit.client.1.smithi130.stdout:4/877: stat d8/dc/d80/db0/df1/d33/d44/d56/lb3 0 2024-05-13T22:48:42.860 INFO:tasks.workunit.client.1.smithi130.stdout:2/912: creat d4/db/dbf/df9/d66/f136 x:0 0 0 2024-05-13T22:48:42.864 INFO:tasks.workunit.client.1.smithi130.stdout:8/813: chown d1/d2/db/d15/d18/d42/d53/da4/ce1 926603 1 2024-05-13T22:48:42.865 INFO:tasks.workunit.client.1.smithi130.stdout:5/911: mkdir d6/d7/d19/d4d/d138 0 2024-05-13T22:48:42.865 INFO:tasks.workunit.client.1.smithi130.stdout:5/912: chown d6/d2f 0 1 2024-05-13T22:48:42.868 INFO:tasks.workunit.client.1.smithi130.stdout:0/921: dwrite d0/d7/dd/d2e/d9e/da7/fde [0,4194304] 0 2024-05-13T22:48:42.870 INFO:tasks.workunit.client.1.smithi130.stdout:0/922: read - d0/d7/dd/d2e/d9e/f10c zero size 2024-05-13T22:48:42.870 INFO:tasks.workunit.client.1.smithi130.stdout:3/913: creat d8/dc/d7b/da3/f12e x:0 0 0 2024-05-13T22:48:42.871 INFO:tasks.workunit.client.1.smithi130.stdout:4/878: mknod d8/dc/d80/db0/df1/d33/d3f/d57/c115 0 2024-05-13T22:48:42.874 INFO:tasks.workunit.client.1.smithi130.stdout:6/866: dwrite d4/d15/d1d/d60/da2/da4/fc6 [0,4194304] 0 2024-05-13T22:48:42.874 INFO:tasks.workunit.client.1.smithi130.stdout:5/913: mkdir d6/d7/d95/d139 0 2024-05-13T22:48:42.876 INFO:tasks.workunit.client.1.smithi130.stdout:8/814: rename d1/d2/db/d15/d16/d54/d58/dab to d1/d2/db/d15/d18/dba/d107/d12c 0 2024-05-13T22:48:42.887 INFO:tasks.workunit.client.1.smithi130.stdout:2/913: truncate d4/d56/dff/f134 3620066 0 2024-05-13T22:48:42.888 INFO:tasks.workunit.client.1.smithi130.stdout:2/914: stat d4/db/dc/d18/d58/f10e 0 2024-05-13T22:48:42.893 INFO:tasks.workunit.client.1.smithi130.stdout:5/914: symlink d6/d7/d32/l13a 0 2024-05-13T22:48:42.894 INFO:tasks.workunit.client.1.smithi130.stdout:5/915: fsync d6/d1a/f2a 0 2024-05-13T22:48:42.898 INFO:tasks.workunit.client.1.smithi130.stdout:3/914: dwrite f2 [0,4194304] 0 2024-05-13T22:48:42.899 INFO:tasks.workunit.client.1.smithi130.stdout:2/915: creat d4/db/dbf/df9/d30/d88/d99/f137 x:0 0 0 2024-05-13T22:48:42.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:42 smithi130 ceph-mon[36591]: pgmap v38: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 68 MiB/s rd, 178 MiB/s wr, 388 op/s 2024-05-13T22:48:42.901 INFO:tasks.workunit.client.1.smithi130.stdout:3/915: write d8/d1a/d1c/fa0 [149869,94267] 0 2024-05-13T22:48:42.904 INFO:tasks.workunit.client.1.smithi130.stdout:5/916: dwrite d6/d2f/ff9 [0,4194304] 0 2024-05-13T22:48:42.905 INFO:tasks.workunit.client.1.smithi130.stdout:0/923: write d0/d7/f23 [1039383,32408] 0 2024-05-13T22:48:42.905 INFO:tasks.workunit.client.1.smithi130.stdout:6/867: creat d4/d15/d1d/d28/d75/da8/f126 x:0 0 0 2024-05-13T22:48:42.909 INFO:tasks.workunit.client.1.smithi130.stdout:4/879: dread d8/dc/d80/db0/df1/f50 [0,4194304] 0 2024-05-13T22:48:42.912 INFO:tasks.workunit.client.1.smithi130.stdout:4/880: dread - d8/dfa/dbd/ffc zero size 2024-05-13T22:48:42.913 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:42 smithi102 ceph-mon[28599]: pgmap v38: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 68 MiB/s rd, 178 MiB/s wr, 388 op/s 2024-05-13T22:48:42.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.935+0000 7f990af5a700 1 -- 172.21.15.102:0/3655852693 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f99040611d0 con 0x7f98ec06d440 2024-05-13T22:48:42.918 INFO:tasks.workunit.client.1.smithi130.stdout:6/868: unlink d4/f73 0 2024-05-13T22:48:42.918 INFO:tasks.workunit.client.1.smithi130.stdout:8/815: link d1/dc/d5d/l99 d1/d2/db/d15/d18/d42/d53/dd0/l12d 0 2024-05-13T22:48:42.918 INFO:tasks.workunit.client.1.smithi130.stdout:2/916: rename d4/db/dbf/df9/d30/d88/dc5/ld0 to d4/db/d26/d9e/d131/l138 0 2024-05-13T22:48:42.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.938+0000 7f9901ffb700 1 -- 172.21.15.102:0/3655852693 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f99040611d0 con 0x7f98ec06d440 2024-05-13T22:48:42.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.941+0000 7f990af5a700 1 -- 172.21.15.102:0/3655852693 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f98ec06d440 msgr2=0x7f98ec06f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:42.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.941+0000 7f990af5a700 1 --2- 172.21.15.102:0/3655852693 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f98ec06d440 0x7f98ec06f900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00ab90 tx=0x7f98fc009250 comp rx=0 tx=0).stop 2024-05-13T22:48:42.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.941+0000 7f990af5a700 1 -- 172.21.15.102:0/3655852693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9904072b80 msgr2=0x7f99041a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:42.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.941+0000 7f990af5a700 1 --2- 172.21.15.102:0/3655852693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9904072b80 0x7f99041a0c30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f98f80048c0 tx=0x7f98f80049a0 comp rx=0 tx=0).stop 2024-05-13T22:48:42.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.942+0000 7f990af5a700 1 -- 172.21.15.102:0/3655852693 shutdown_connections 2024-05-13T22:48:42.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.942+0000 7f990af5a700 1 --2- 172.21.15.102:0/3655852693 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f98ec06d440 0x7f98ec06f900 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:42.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.942+0000 7f990af5a700 1 --2- 172.21.15.102:0/3655852693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9904072b80 0x7f99041a0c30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:42.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.942+0000 7f990af5a700 1 --2- 172.21.15.102:0/3655852693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990410a370 0x7f99041a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:42.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.942+0000 7f990af5a700 1 -- 172.21.15.102:0/3655852693 >> 172.21.15.102:0/3655852693 conn(0x7f990406d9f0 msgr2=0x7f990406fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:42.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.942+0000 7f990af5a700 1 -- 172.21.15.102:0/3655852693 shutdown_connections 2024-05-13T22:48:42.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:42.942+0000 7f990af5a700 1 -- 172.21.15.102:0/3655852693 wait complete. 2024-05-13T22:48:42.921 INFO:tasks.workunit.client.1.smithi130.stdout:4/881: unlink d8/dc/d80/db0/df1/d33/dee/ff2 0 2024-05-13T22:48:42.924 INFO:tasks.workunit.client.1.smithi130.stdout:8/816: stat d1/d2/db/d15/d16/d54/f55 0 2024-05-13T22:48:42.925 INFO:tasks.workunit.client.1.smithi130.stdout:5/917: creat d6/f13b x:0 0 0 2024-05-13T22:48:42.929 INFO:tasks.workunit.client.1.smithi130.stdout:2/917: unlink d4/db/dbf/df9/d30/d88/dc5/dcd/ffa 0 2024-05-13T22:48:42.930 INFO:tasks.workunit.client.1.smithi130.stdout:2/918: readlink d4/db/dbf/df9/d66/l7c 0 2024-05-13T22:48:42.931 INFO:tasks.workunit.client.1.smithi130.stdout:2/919: chown d4/db/dbf/df9/d30/d88/d99/d9a/fb6 164 1 2024-05-13T22:48:42.932 INFO:tasks.workunit.client.1.smithi130.stdout:3/916: dread d8/dc/d7b/da3/fbe [0,4194304] 0 2024-05-13T22:48:42.934 INFO:tasks.workunit.client.1.smithi130.stdout:0/924: dread d0/d7/dd/d2e/d9e/da7/de1/f128 [0,4194304] 0 2024-05-13T22:48:42.935 INFO:tasks.workunit.client.1.smithi130.stdout:4/882: rename d8/dc/d80/db0/df1/d33/d44/f54 to d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dad/f116 0 2024-05-13T22:48:42.935 INFO:tasks.workunit.client.1.smithi130.stdout:0/925: chown d0/d7/dd/d69/da9/d8d/d9f 40 1 2024-05-13T22:48:42.939 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:48:42.942 INFO:tasks.workunit.client.1.smithi130.stdout:3/917: dread d8/f34 [0,4194304] 0 2024-05-13T22:48:42.946 INFO:tasks.workunit.client.1.smithi130.stdout:2/920: creat d4/db/dbf/df9/d66/f139 x:0 0 0 2024-05-13T22:48:42.952 INFO:tasks.workunit.client.1.smithi130.stdout:6/869: write d4/d15/d1d/fcd [1210870,14502] 0 2024-05-13T22:48:42.955 INFO:tasks.workunit.client.1.smithi130.stdout:1/865: sync 2024-05-13T22:48:42.955 INFO:tasks.workunit.client.1.smithi130.stdout:9/931: sync 2024-05-13T22:48:42.955 INFO:tasks.workunit.client.1.smithi130.stdout:9/932: readlink d1/d7/d23/d28/l2e 0 2024-05-13T22:48:42.957 INFO:tasks.workunit.client.1.smithi130.stdout:5/918: write d6/dd/d18/fce [861348,122901] 0 2024-05-13T22:48:42.963 INFO:tasks.workunit.client.1.smithi130.stdout:8/817: unlink d1/d2/db/d15/d18/d42/d53/dd0/l12d 0 2024-05-13T22:48:42.964 INFO:tasks.workunit.client.1.smithi130.stdout:8/818: write d1/dc/d5d/fee [4483040,123724] 0 2024-05-13T22:48:42.966 INFO:tasks.workunit.client.1.smithi130.stdout:2/921: mknod d4/d56/dff/d4a/dc6/c13a 0 2024-05-13T22:48:42.976 INFO:tasks.workunit.client.1.smithi130.stdout:4/883: rmdir d8/da8 39 2024-05-13T22:48:42.979 INFO:tasks.workunit.client.1.smithi130.stdout:8/819: read d1/d2/db/d15/d18/f63 [438913,21171] 0 2024-05-13T22:48:42.980 INFO:tasks.workunit.client.1.smithi130.stdout:8/820: stat d1/d2/db/d15/d16/d54/d58 0 2024-05-13T22:48:42.980 INFO:tasks.workunit.client.1.smithi130.stdout:2/922: mknod d4/db/dbf/df9/d30/d88/d99/d9a/c13b 0 2024-05-13T22:48:42.989 INFO:tasks.workunit.client.1.smithi130.stdout:5/919: write d6/d1a/daf/f114 [138692,59861] 0 2024-05-13T22:48:42.991 INFO:tasks.workunit.client.1.smithi130.stdout:0/926: rename d0/d7/dd/d17/d1f/d2c/d45/cb3 to d0/d7/dd/d2e/d9e/da7/dc3/c12e 0 2024-05-13T22:48:42.992 INFO:tasks.workunit.client.1.smithi130.stdout:4/884: unlink d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/ff8 0 2024-05-13T22:48:42.996 INFO:tasks.workunit.client.1.smithi130.stdout:8/821: fsync d1/d2/db/d15/d18/f4e 0 2024-05-13T22:48:42.998 INFO:tasks.workunit.client.1.smithi130.stdout:2/923: mknod d4/db/d26/d9e/de4/c13c 0 2024-05-13T22:48:43.001 INFO:tasks.workunit.client.1.smithi130.stdout:6/870: rmdir d4/d15/d103 0 2024-05-13T22:48:43.004 INFO:tasks.workunit.client.1.smithi130.stdout:5/920: read d6/dd/d18/d6e/d82/d4b/f99 [104578,107828] 0 2024-05-13T22:48:43.006 INFO:tasks.workunit.client.1.smithi130.stdout:3/918: rename d8/d1a/d1c to d8/d1a/dfa/d12f 0 2024-05-13T22:48:43.041 INFO:tasks.workunit.client.1.smithi130.stdout:8/822: rmdir d1/d2/db/d15/d16/d54/d58/db9 39 2024-05-13T22:48:43.041 INFO:tasks.workunit.client.1.smithi130.stdout:2/924: unlink d4/d56/dff/c6b 0 2024-05-13T22:48:43.041 INFO:tasks.workunit.client.1.smithi130.stdout:1/866: mknod db/d1b/da3/d2a/d38/c114 0 2024-05-13T22:48:43.042 INFO:tasks.workunit.client.1.smithi130.stdout:3/919: rmdir d8/dc/d7b/da3/dd3 39 2024-05-13T22:48:43.042 INFO:tasks.workunit.client.1.smithi130.stdout:9/933: link d1/d7/d23/d28/d73/d8d/c90 d1/d7/d23/d28/d73/db2/c12e 0 2024-05-13T22:48:43.043 INFO:tasks.workunit.client.1.smithi130.stdout:5/921: write d6/d7/d19/d4d/d6a/f135 [276078,102942] 0 2024-05-13T22:48:43.044 INFO:tasks.workunit.client.1.smithi130.stdout:3/920: fsync d8/d1a/d55/f9b 0 2024-05-13T22:48:43.045 INFO:tasks.workunit.client.1.smithi130.stdout:5/922: stat d6/dd/d18/f15 0 2024-05-13T22:48:43.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.090+0000 7f76d4f08700 1 -- 172.21.15.102:0/3855393369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76d0100d00 msgr2=0x7f76d0101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:43.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.090+0000 7f76d4f08700 1 --2- 172.21.15.102:0/3855393369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76d0100d00 0x7f76d0101180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009910 tx=0x7f76c4009c20 comp rx=0 tx=0).stop 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.090+0000 7f76d4f08700 1 -- 172.21.15.102:0/3855393369 shutdown_connections 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.090+0000 7f76d4f08700 1 --2- 172.21.15.102:0/3855393369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76d0100d00 0x7f76d0101180 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.090+0000 7f76d4f08700 1 --2- 172.21.15.102:0/3855393369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76d00fffe0 0x7f76d0100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.090+0000 7f76d4f08700 1 -- 172.21.15.102:0/3855393369 >> 172.21.15.102:0/3855393369 conn(0x7f76d00fa7c0 msgr2=0x7f76d00fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.090+0000 7f76d4f08700 1 -- 172.21.15.102:0/3855393369 shutdown_connections 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.090+0000 7f76d4f08700 1 -- 172.21.15.102:0/3855393369 wait complete. 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.092+0000 7f76d4f08700 1 Processor -- start 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.092+0000 7f76d4f08700 1 -- start start 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.092+0000 7f76d4f08700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76d00fffe0 0x7f76d01963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.092+0000 7f76d4f08700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76d0100d00 0x7f76d0196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.092+0000 7f76d4f08700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d0196f20 con 0x7f76d0100d00 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.092+0000 7f76d4f08700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d0197060 con 0x7f76d00fffe0 2024-05-13T22:48:43.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.092+0000 7f76ce59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76d00fffe0 0x7f76d01963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.092+0000 7f76ce59c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76d00fffe0 0x7f76d01963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.102:33946/0 (socket says 172.21.15.102:33946) 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.092+0000 7f76ce59c700 1 -- 172.21.15.102:0/1084530069 learned_addr learned my addr 172.21.15.102:0/1084530069 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.093+0000 7f76cdd9b700 1 --2- 172.21.15.102:0/1084530069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76d0100d00 0x7f76d0196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.093+0000 7f76ce59c700 1 -- 172.21.15.102:0/1084530069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76d0100d00 msgr2=0x7f76d0196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.093+0000 7f76ce59c700 1 --2- 172.21.15.102:0/1084530069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76d0100d00 0x7f76d0196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.093+0000 7f76ce59c700 1 -- 172.21.15.102:0/1084530069 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40095f0 con 0x7f76d00fffe0 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.093+0000 7f76ce59c700 1 --2- 172.21.15.102:0/1084530069 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76d00fffe0 0x7f76d01963c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f76c000e8d0 tx=0x7f76c000ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.093+0000 7f76bf7fe700 1 -- 172.21.15.102:0/1084530069 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c0009770 con 0x7f76d00fffe0 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.093+0000 7f76d4f08700 1 -- 172.21.15.102:0/1084530069 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d019bb10 con 0x7f76d00fffe0 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.093+0000 7f76bf7fe700 1 -- 172.21.15.102:0/1084530069 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f76c000c810 con 0x7f76d00fffe0 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.093+0000 7f76bf7fe700 1 -- 172.21.15.102:0/1084530069 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76c0003da0 con 0x7f76d00fffe0 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.094+0000 7f76d4f08700 1 -- 172.21.15.102:0/1084530069 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d019c060 con 0x7f76d00fffe0 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.095+0000 7f76bf7fe700 1 -- 172.21.15.102:0/1084530069 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 24) v1 ==== 91288+0+0 (secure 0 0 0) 0x7f76c0021440 con 0x7f76d00fffe0 2024-05-13T22:48:43.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.095+0000 7f76d4f08700 1 -- 172.21.15.102:0/1084530069 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d0190490 con 0x7f76d00fffe0 2024-05-13T22:48:43.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.096+0000 7f76bf7fe700 1 --2- 172.21.15.102:0/1084530069 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f76b806d2f0 0x7f76b806f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:43.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.096+0000 7f76bf7fe700 1 -- 172.21.15.102:0/1084530069 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f76c0014070 con 0x7f76d00fffe0 2024-05-13T22:48:43.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.096+0000 7f76cdd9b700 1 --2- 172.21.15.102:0/1084530069 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f76b806d2f0 0x7f76b806f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:43.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.097+0000 7f76cdd9b700 1 --2- 172.21.15.102:0/1084530069 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f76b806d2f0 0x7f76b806f7b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f76c400b6f0 tx=0x7f76c4005940 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:43.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.100+0000 7f76bf7fe700 1 -- 172.21.15.102:0/1084530069 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f76c005a2e0 con 0x7f76d00fffe0 2024-05-13T22:48:43.212 INFO:tasks.workunit.client.1.smithi130.stdout:2/925: unlink d4/d56/dff/cd2 0 2024-05-13T22:48:43.212 INFO:tasks.workunit.client.1.smithi130.stdout:4/885: dwrite d8/dc/d80/db0/df1/d33/d3f/d57/f93 [0,4194304] 0 2024-05-13T22:48:43.212 INFO:tasks.workunit.client.1.smithi130.stdout:9/934: mknod d1/d7/d69/c12f 0 2024-05-13T22:48:43.212 INFO:tasks.workunit.client.1.smithi130.stdout:9/935: write d1/d7/d23/d28/d2d/d2f/fdf [1741306,56407] 0 2024-05-13T22:48:43.212 INFO:tasks.workunit.client.1.smithi130.stdout:6/871: rename d4/d108 to d4/d15/d1f/d46/d52/d127 0 2024-05-13T22:48:43.212 INFO:tasks.workunit.client.1.smithi130.stdout:2/926: symlink d4/db/d26/d9e/de4/l13d 0 2024-05-13T22:48:43.212 INFO:tasks.workunit.client.1.smithi130.stdout:4/886: fsync d8/dc/d80/db0/df1/d33/d44/d56/f72 0 2024-05-13T22:48:43.212 INFO:tasks.workunit.client.1.smithi130.stdout:4/887: dread d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f3c [0,4194304] 0 2024-05-13T22:48:43.212 INFO:tasks.workunit.client.1.smithi130.stdout:7/917: sync 2024-05-13T22:48:43.212 INFO:tasks.workunit.client.1.smithi130.stdout:1/867: write db/d1b/da3/d2a/d38/d46/f71 [285433,89584] 0 2024-05-13T22:48:43.212 INFO:tasks.workunit.client.1.smithi130.stdout:6/872: mkdir d4/d15/d1d/d28/ddc/d128 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:3/921: truncate d8/dc/f9f 72015 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:3/922: dread - d8/dc/d7b/da3/f12e zero size 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:1/868: dwrite db/d1b/da3/d2a/d4c/dc2/fd7 [0,4194304] 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:4/888: creat d8/dc/d84/f117 x:0 0 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:7/918: creat d7/d8f/f129 x:0 0 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:3/923: stat d8/d35/fb0 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:1/869: truncate db/d1b/da3/d2a/d38/d46/d6d/fbe 850284 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:4/889: mknod d8/dc/d84/c118 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:9/936: creat d1/f130 x:0 0 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:4/890: write d8/dc/d84/ff0 [588302,83614] 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:9/937: mkdir d1/d15/d51/d85/d131 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:2/927: dread d4/db/dbf/d42/d4b/ff8 [4194304,4194304] 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:8/823: rename d1/d2/db/d15/d18/f3e to d1/d2/f12e 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:3/924: stat d8/d1a/dfa/d2c/fd8 0 2024-05-13T22:48:43.213 INFO:tasks.workunit.client.1.smithi130.stdout:1/870: write db/d1b/da3/d2a/d38/d46/f5f [794778,38247] 0 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:7/919: write d7/da/d1d/d5d/da6/d9/d4a/dbd/d40/d127/fe8 [3135984,110919] 0 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:7/920: read d7/da/d1d/d5d/da6/d7e/fdc [1525731,44063] 0 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:0/927: sync 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:3/925: stat d8/d1a/dfa/f7f 0 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:7/921: dwrite d7/da/d1d/fcb [0,4194304] 0 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:4/891: unlink d8/dc/d80/db0/df1/d33/d44/d56/f7f 0 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:1/871: stat db/d82/dc3/fe2 0 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:3/926: chown d8/dc/d15/la7 125 1 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:6/873: rename d4/d15/d1d/d60/da2/da4/dac/fc5 to d4/d15/d1d/d28/d75/d7d/f129 0 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:3/927: readlink d8/dc/d15/l38 0 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:1/872: dread - db/d1b/da3/d2a/d35/dd4/fdc zero size 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:9/938: dwrite d1/d7/d23/d3d/d43/fb1 [0,4194304] 0 2024-05-13T22:48:43.214 INFO:tasks.workunit.client.1.smithi130.stdout:6/874: chown d4/d15/d1d/d28/d75/d7d/lad 15 1 2024-05-13T22:48:43.215 INFO:tasks.workunit.client.1.smithi130.stdout:7/922: dwrite d7/da/d1d/d5d/da6/d39/f94 [0,4194304] 0 2024-05-13T22:48:43.215 INFO:tasks.workunit.client.1.smithi130.stdout:4/892: creat d8/dc/d80/db0/df1/d33/d44/db8/df7/f119 x:0 0 0 2024-05-13T22:48:43.215 INFO:tasks.workunit.client.1.smithi130.stdout:2/928: dwrite d4/da8/dc9/f101 [0,4194304] 0 2024-05-13T22:48:43.215 INFO:tasks.workunit.client.1.smithi130.stdout:7/923: fsync d7/da/d1d/d5d/da6/d5b/d107/d46/d92/fad 0 2024-05-13T22:48:43.215 INFO:tasks.workunit.client.1.smithi130.stdout:8/824: link d1/d2/db/d15/d18/d42/l12a d1/d2/db/d15/d18/d1a/l12f 0 2024-05-13T22:48:43.215 INFO:tasks.workunit.client.1.smithi130.stdout:7/924: read d7/da/d1d/d5d/d70/f85 [851689,122192] 0 2024-05-13T22:48:43.215 INFO:tasks.workunit.client.1.smithi130.stdout:4/893: mkdir d8/dc/d80/db0/df1/d33/d44/db8/df7/d11a 0 2024-05-13T22:48:43.215 INFO:tasks.workunit.client.1.smithi130.stdout:4/894: write d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/fdc [2435604,114289] 0 2024-05-13T22:48:43.216 INFO:tasks.workunit.client.1.smithi130.stdout:4/895: dwrite d8/dc/d80/d85/f10f [0,4194304] 0 2024-05-13T22:48:43.216 INFO:tasks.workunit.client.1.smithi130.stdout:1/873: mknod db/d1b/da3/d72/d75/dee/df6/da0/c115 0 2024-05-13T22:48:43.216 INFO:tasks.workunit.client.1.smithi130.stdout:1/874: truncate db/d1b/da3/d2a/d38/d46/f5f 1050782 0 2024-05-13T22:48:43.224 INFO:tasks.workunit.client.1.smithi130.stdout:7/925: dread d7/da/d1d/d5d/d70/f85 [0,4194304] 0 2024-05-13T22:48:43.227 INFO:tasks.workunit.client.1.smithi130.stdout:8/825: stat d1/dc/d5d/l99 0 2024-05-13T22:48:43.230 INFO:tasks.workunit.client.1.smithi130.stdout:9/939: dwrite d1/f11 [0,4194304] 0 2024-05-13T22:48:43.231 INFO:tasks.workunit.client.1.smithi130.stdout:1/875: dread db/d1b/da3/d2a/d35/fc4 [0,4194304] 0 2024-05-13T22:48:43.232 INFO:tasks.workunit.client.1.smithi130.stdout:3/928: rename d8/d1a/dfa/d12f/lab to d8/d1a/dfa/d83/dba/dc4/l130 0 2024-05-13T22:48:43.233 INFO:tasks.workunit.client.1.smithi130.stdout:7/926: dwrite d7/da/f62 [0,4194304] 0 2024-05-13T22:48:43.235 INFO:tasks.workunit.client.1.smithi130.stdout:7/927: truncate d7/da/d1d/db1/dbe/dd3/f108 712655 0 2024-05-13T22:48:43.245 INFO:tasks.workunit.client.1.smithi130.stdout:4/896: creat d8/dc/d80/db0/df1/d33/d3f/d57/f11b x:0 0 0 2024-05-13T22:48:43.246 INFO:tasks.workunit.client.1.smithi130.stdout:2/929: symlink d4/l13e 0 2024-05-13T22:48:43.248 INFO:tasks.workunit.client.1.smithi130.stdout:2/930: dread - d4/db/dbf/df9/d66/f136 zero size 2024-05-13T22:48:43.248 INFO:tasks.workunit.client.1.smithi130.stdout:6/875: link d4/d15/d1d/d28/d75/ld4 d4/d15/d1f/d46/d50/l12a 0 2024-05-13T22:48:43.258 INFO:tasks.workunit.client.1.smithi130.stdout:9/940: dread - d1/d7/d69/fe5 zero size 2024-05-13T22:48:43.259 INFO:tasks.workunit.client.1.smithi130.stdout:9/941: stat d1/d7/d23/d28/l2e 0 2024-05-13T22:48:43.262 INFO:tasks.workunit.client.1.smithi130.stdout:3/929: creat d8/d1a/d55/f131 x:0 0 0 2024-05-13T22:48:43.265 INFO:tasks.workunit.client.1.smithi130.stdout:7/928: mkdir d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/d12a 0 2024-05-13T22:48:43.268 INFO:tasks.workunit.client.1.smithi130.stdout:4/897: truncate d8/dc/d80/db0/df1/d33/d44/d77/d94/f97 354248 0 2024-05-13T22:48:43.269 INFO:tasks.workunit.client.1.smithi130.stdout:4/898: write d8/dc/d80/db0/df1/d33/d3f/d57/f93 [4130,104612] 0 2024-05-13T22:48:43.270 INFO:tasks.workunit.client.1.smithi130.stdout:4/899: chown d8/dc/d80/db0/df1/d33/d44/d77/fe9 5665 1 2024-05-13T22:48:43.275 INFO:tasks.workunit.client.1.smithi130.stdout:9/942: rmdir d1/d15 39 2024-05-13T22:48:43.278 INFO:tasks.workunit.client.1.smithi130.stdout:3/930: mkdir d8/dc/d7b/da3/daf/d132 0 2024-05-13T22:48:43.279 INFO:tasks.workunit.client.1.smithi130.stdout:3/931: write d8/dc/d7b/fcf [2135322,35897] 0 2024-05-13T22:48:43.280 INFO:tasks.workunit.client.1.smithi130.stdout:5/923: sync 2024-05-13T22:48:43.280 INFO:tasks.workunit.client.1.smithi130.stdout:3/932: read d8/dc/d7b/fe0 [463992,55964] 0 2024-05-13T22:48:43.281 INFO:tasks.workunit.client.1.smithi130.stdout:3/933: write d8/d1a/dfa/d83/dba/f118 [998363,42238] 0 2024-05-13T22:48:43.284 INFO:tasks.workunit.client.1.smithi130.stdout:2/931: mkdir d4/db/d26/d13f 0 2024-05-13T22:48:43.290 INFO:tasks.workunit.client.1.smithi130.stdout:6/876: mkdir d4/d15/d1f/d46/d12b 0 2024-05-13T22:48:43.291 INFO:tasks.workunit.client.1.smithi130.stdout:6/877: write d4/d15/d1d/d60/fcc [1243475,107078] 0 2024-05-13T22:48:43.292 INFO:tasks.workunit.client.1.smithi130.stdout:4/900: mknod d8/dc/d26/c11c 0 2024-05-13T22:48:43.293 INFO:tasks.workunit.client.1.smithi130.stdout:4/901: write d8/dc/d26/fc2 [641093,112209] 0 2024-05-13T22:48:43.294 INFO:tasks.workunit.client.1.smithi130.stdout:4/902: chown d8/dc/d80/db0/df1/d33/d44/d77/fcd 396419172 1 2024-05-13T22:48:43.296 INFO:tasks.workunit.client.1.smithi130.stdout:9/943: dread - d1/d15/d9e/dfa/f123 zero size 2024-05-13T22:48:43.298 INFO:tasks.workunit.client.1.smithi130.stdout:1/876: write db/d4a/ffc [3473975,107636] 0 2024-05-13T22:48:43.302 INFO:tasks.workunit.client.1.smithi130.stdout:8/826: getdents d1/d2/db/d15/d18/dba/d107 0 2024-05-13T22:48:43.303 INFO:tasks.workunit.client.1.smithi130.stdout:1/877: dwrite db/d1b/da3/d2a/d35/f3d [0,4194304] 0 2024-05-13T22:48:43.305 INFO:tasks.workunit.client.1.smithi130.stdout:0/928: sync 2024-05-13T22:48:43.307 INFO:tasks.workunit.client.1.smithi130.stdout:4/903: fsync d8/dc/d80/db0/df1/f8d 0 2024-05-13T22:48:43.309 INFO:tasks.workunit.client.1.smithi130.stdout:9/944: write d1/d7/d23/d28/d73/f104 [761638,122726] 0 2024-05-13T22:48:43.317 INFO:tasks.workunit.client.1.smithi130.stdout:5/924: mkdir d6/d7/d19/d4d/d8a/df7/d13c 0 2024-05-13T22:48:43.321 INFO:tasks.workunit.client.1.smithi130.stdout:3/934: symlink d8/dc/d15/dfd/l133 0 2024-05-13T22:48:43.322 INFO:tasks.workunit.client.1.smithi130.stdout:7/929: creat d7/da/d1d/d5d/f12b x:0 0 0 2024-05-13T22:48:43.326 INFO:tasks.workunit.client.1.smithi130.stdout:2/932: creat d4/db/d26/d13f/f140 x:0 0 0 2024-05-13T22:48:43.326 INFO:tasks.workunit.client.1.smithi130.stdout:8/827: creat d1/d2/db/d15/d7e/dc2/dcf/dda/f130 x:0 0 0 2024-05-13T22:48:43.329 INFO:tasks.workunit.client.1.smithi130.stdout:1/878: write db/d1b/da3/fce [734006,54244] 0 2024-05-13T22:48:43.329 INFO:tasks.workunit.client.1.smithi130.stdout:6/878: creat d4/d15/d1d/d28/d4e/d9e/f12c x:0 0 0 2024-05-13T22:48:43.331 INFO:tasks.workunit.client.1.smithi130.stdout:1/879: write db/d1b/da3/d2a/d35/dd4/fdc [507773,107257] 0 2024-05-13T22:48:43.331 INFO:tasks.workunit.client.1.smithi130.stdout:4/904: mknod d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dad/c11d 0 2024-05-13T22:48:43.333 INFO:tasks.workunit.client.1.smithi130.stdout:1/880: chown db/d1b/da3/d2a/d38/c114 402 1 2024-05-13T22:48:43.333 INFO:tasks.workunit.client.1.smithi130.stdout:4/905: truncate d8/dc/f9a 1043952 0 2024-05-13T22:48:43.334 INFO:tasks.workunit.client.1.smithi130.stdout:4/906: dread - d8/dc/d26/dab/f105 zero size 2024-05-13T22:48:43.334 INFO:tasks.workunit.client.1.smithi130.stdout:3/935: mkdir d8/dc/d7b/da3/daf/d134 0 2024-05-13T22:48:43.335 INFO:tasks.workunit.client.1.smithi130.stdout:7/930: readlink d7/da/d1d/d5d/da6/d5b/d107/d46/lf8 0 2024-05-13T22:48:43.338 INFO:tasks.workunit.client.1.smithi130.stdout:7/931: dread - d7/da/d1d/d5d/da6/d9/d66/ffd zero size 2024-05-13T22:48:43.338 INFO:tasks.workunit.client.1.smithi130.stdout:0/929: mkdir d0/d7/d12f 0 2024-05-13T22:48:43.339 INFO:tasks.workunit.client.1.smithi130.stdout:6/879: symlink d4/d15/d1d/d28/ddc/l12d 0 2024-05-13T22:48:43.341 INFO:tasks.workunit.client.1.smithi130.stdout:3/936: dwrite d8/d35/fb0 [4194304,4194304] 0 2024-05-13T22:48:43.342 INFO:tasks.workunit.client.1.smithi130.stdout:3/937: stat d8/d1a/d55/f111 0 2024-05-13T22:48:43.342 INFO:tasks.workunit.client.1.smithi130.stdout:3/938: stat d8/d1a/c1b 0 2024-05-13T22:48:43.343 INFO:tasks.workunit.client.1.smithi130.stdout:1/881: stat db/d1b/da3/d2a/d38/d46/l47 0 2024-05-13T22:48:43.346 INFO:tasks.workunit.client.1.smithi130.stdout:5/925: write d6/d7/f25 [2896082,44699] 0 2024-05-13T22:48:43.348 INFO:tasks.workunit.client.1.smithi130.stdout:4/907: symlink d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dad/l11e 0 2024-05-13T22:48:43.353 INFO:tasks.workunit.client.1.smithi130.stdout:2/933: rename d4/c105 to d4/db/d26/d12e/c141 0 2024-05-13T22:48:43.354 INFO:tasks.workunit.client.1.smithi130.stdout:2/934: dread - d4/db/dbf/df9/d66/db9/fc1 zero size 2024-05-13T22:48:43.355 INFO:tasks.workunit.client.1.smithi130.stdout:0/930: fsync d0/d7/dd/d2e/fe5 0 2024-05-13T22:48:43.356 INFO:tasks.workunit.client.1.smithi130.stdout:0/931: chown d0/d7/dd/d2e/fe5 24 1 2024-05-13T22:48:43.356 INFO:tasks.workunit.client.1.smithi130.stdout:7/932: creat d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/f12c x:0 0 0 2024-05-13T22:48:43.359 INFO:tasks.workunit.client.1.smithi130.stdout:9/945: creat d1/d7/d23/d28/d32/f132 x:0 0 0 2024-05-13T22:48:43.360 INFO:tasks.workunit.client.1.smithi130.stdout:6/880: mknod d4/d15/d1d/d60/da2/da4/c12e 0 2024-05-13T22:48:43.361 INFO:tasks.workunit.client.1.smithi130.stdout:0/932: dwrite d0/d7/dd/d17/fae [0,4194304] 0 2024-05-13T22:48:43.362 INFO:tasks.workunit.client.1.smithi130.stdout:0/933: chown d0/d7/dd/d17/d52 5 1 2024-05-13T22:48:43.363 INFO:tasks.workunit.client.1.smithi130.stdout:1/882: write db/d4a/f70 [4116304,66188] 0 2024-05-13T22:48:43.363 INFO:tasks.workunit.client.1.smithi130.stdout:0/934: fsync d0/d7/dd/d17/d1f/d2c/f7a 0 2024-05-13T22:48:43.364 INFO:tasks.workunit.client.1.smithi130.stdout:0/935: dread - d0/d7/dd/d2e/d9e/da7/dc3/ff6 zero size 2024-05-13T22:48:43.367 INFO:tasks.workunit.client.1.smithi130.stdout:5/926: creat d6/d7/d19/d4d/d8a/df7/f13d x:0 0 0 2024-05-13T22:48:43.368 INFO:tasks.workunit.client.1.smithi130.stdout:4/908: rename d8 to d8/dc/d80/db0/df1/d33/d11f 22 2024-05-13T22:48:43.370 INFO:tasks.workunit.client.1.smithi130.stdout:4/909: fsync d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d81/fce 0 2024-05-13T22:48:43.376 INFO:tasks.workunit.client.1.smithi130.stdout:3/939: mknod d8/dc/d7b/da3/daf/d134/c135 0 2024-05-13T22:48:43.380 INFO:tasks.workunit.client.1.smithi130.stdout:8/828: getdents d1/d2/db/d15/d18 0 2024-05-13T22:48:43.382 INFO:tasks.workunit.client.1.smithi130.stdout:0/936: rename d0/d7/dd/d2e/d9e/f121 to d0/d7/dd/d2e/da8/dcd/f130 0 2024-05-13T22:48:43.382 INFO:tasks.workunit.client.1.smithi130.stdout:8/829: write d1/d2/db/d15/d16/d54/d58/fdb [573496,14485] 0 2024-05-13T22:48:43.383 INFO:tasks.workunit.client.1.smithi130.stdout:3/940: dwrite f2 [0,4194304] 0 2024-05-13T22:48:43.384 INFO:tasks.workunit.client.1.smithi130.stdout:8/830: chown d1/d2/db/d15/d7e/l106 27 1 2024-05-13T22:48:43.387 INFO:tasks.workunit.client.1.smithi130.stdout:4/910: mknod d8/dc/d80/db0/df1/d33/d44/d56/c120 0 2024-05-13T22:48:43.391 INFO:tasks.workunit.client.1.smithi130.stdout:4/911: dread d8/dc/d80/db0/df1/f8a [0,4194304] 0 2024-05-13T22:48:43.394 INFO:tasks.workunit.client.1.smithi130.stdout:1/883: unlink db/f17 0 2024-05-13T22:48:43.398 INFO:tasks.workunit.client.1.smithi130.stdout:9/946: dwrite d1/d15/d51/d85/fb0 [0,4194304] 0 2024-05-13T22:48:43.400 INFO:tasks.workunit.client.1.smithi130.stdout:9/947: chown d1/d7/d23/d28/d2d/f46 92 1 2024-05-13T22:48:43.401 INFO:tasks.workunit.client.1.smithi130.stdout:0/937: chown d0/d7/dd/d2e/da8/dcd/lf1 677 1 2024-05-13T22:48:43.405 INFO:tasks.workunit.client.1.smithi130.stdout:5/927: dwrite d6/d1a/f7a [0,4194304] 0 2024-05-13T22:48:43.405 INFO:tasks.workunit.client.1.smithi130.stdout:4/912: unlink d8/dc/d26/fd1 0 2024-05-13T22:48:43.407 INFO:tasks.workunit.client.1.smithi130.stdout:0/938: write d0/d7/dd/d17/d1f/d2c/d45/d57/df7/fff [4183173,109425] 0 2024-05-13T22:48:43.409 INFO:tasks.workunit.client.1.smithi130.stdout:2/935: creat d4/d56/dff/d4a/f142 x:0 0 0 2024-05-13T22:48:43.412 INFO:tasks.workunit.client.1.smithi130.stdout:8/831: mknod d1/d2/db/d15/d18/d42/d53/da4/d10e/c131 0 2024-05-13T22:48:43.413 INFO:tasks.workunit.client.1.smithi130.stdout:8/832: chown d1/d2/db/d15/d18/d42/f62 19569 1 2024-05-13T22:48:43.422 INFO:tasks.workunit.client.1.smithi130.stdout:6/881: getdents d4/d15/d1d/d28/d4e/d9e/ddb 0 2024-05-13T22:48:43.426 INFO:tasks.workunit.client.1.smithi130.stdout:3/941: dread d8/d1a/d55/f9b [0,4194304] 0 2024-05-13T22:48:43.426 INFO:tasks.workunit.client.1.smithi130.stdout:3/942: read d8/dc/d7b/da3/daf/fb7 [176492,47975] 0 2024-05-13T22:48:43.431 INFO:tasks.workunit.client.1.smithi130.stdout:2/936: dread d4/db/dbf/df9/d30/f47 [0,4194304] 0 2024-05-13T22:48:43.437 INFO:tasks.workunit.client.1.smithi130.stdout:2/937: dread d4/db/dbf/f35 [0,4194304] 0 2024-05-13T22:48:43.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.459+0000 7f76d4f08700 1 -- 172.21.15.102:0/1084530069 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f76d00611d0 con 0x7f76b806d2f0 2024-05-13T22:48:43.438 INFO:tasks.workunit.client.1.smithi130.stdout:1/884: unlink db/cb9 0 2024-05-13T22:48:43.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.461+0000 7f76bf7fe700 1 -- 172.21.15.102:0/1084530069 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f76d00611d0 con 0x7f76b806d2f0 2024-05-13T22:48:43.440 INFO:tasks.workunit.client.1.smithi130.stdout:5/928: rmdir d6/d7/d19/d4d/d6a 39 2024-05-13T22:48:43.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.464+0000 7f76d4f08700 1 -- 172.21.15.102:0/1084530069 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f76b806d2f0 msgr2=0x7f76b806f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:43.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.464+0000 7f76d4f08700 1 --2- 172.21.15.102:0/1084530069 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f76b806d2f0 0x7f76b806f7b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f76c400b6f0 tx=0x7f76c4005940 comp rx=0 tx=0).stop 2024-05-13T22:48:43.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.464+0000 7f76d4f08700 1 -- 172.21.15.102:0/1084530069 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76d00fffe0 msgr2=0x7f76d01963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:43.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.464+0000 7f76d4f08700 1 --2- 172.21.15.102:0/1084530069 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76d00fffe0 0x7f76d01963c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f76c000e8d0 tx=0x7f76c000ebe0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.464+0000 7f76d4f08700 1 -- 172.21.15.102:0/1084530069 shutdown_connections 2024-05-13T22:48:43.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.464+0000 7f76d4f08700 1 --2- 172.21.15.102:0/1084530069 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f76b806d2f0 0x7f76b806f7b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.442 INFO:tasks.workunit.client.1.smithi130.stdout:4/913: mkdir d8/dc/d84/d121 0 2024-05-13T22:48:43.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.464+0000 7f76d4f08700 1 --2- 172.21.15.102:0/1084530069 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76d00fffe0 0x7f76d01963c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.464+0000 7f76d4f08700 1 --2- 172.21.15.102:0/1084530069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76d0100d00 0x7f76d0196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.464+0000 7f76d4f08700 1 -- 172.21.15.102:0/1084530069 >> 172.21.15.102:0/1084530069 conn(0x7f76d00fa7c0 msgr2=0x7f76d00fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:43.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.464+0000 7f76d4f08700 1 -- 172.21.15.102:0/1084530069 shutdown_connections 2024-05-13T22:48:43.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.464+0000 7f76d4f08700 1 -- 172.21.15.102:0/1084530069 wait complete. 2024-05-13T22:48:43.443 INFO:tasks.workunit.client.1.smithi130.stdout:4/914: stat d8/dc/d80/db0/df1/d33/d3f 0 2024-05-13T22:48:43.449 INFO:tasks.workunit.client.1.smithi130.stdout:9/948: mkdir d1/d7/d23/d133 0 2024-05-13T22:48:43.450 INFO:tasks.workunit.client.1.smithi130.stdout:9/949: write d1/d15/d51/db4/d11f/fe1 [1493413,11065] 0 2024-05-13T22:48:43.452 INFO:tasks.workunit.client.1.smithi130.stdout:6/882: truncate d4/d15/d1f/d46/f58 790328 0 2024-05-13T22:48:43.454 INFO:tasks.workunit.client.1.smithi130.stdout:3/943: mkdir d8/dc/d15/dfd/d9c/d136 0 2024-05-13T22:48:43.456 INFO:tasks.workunit.client.1.smithi130.stdout:2/938: symlink d4/d56/dff/d4a/d76/l143 0 2024-05-13T22:48:43.457 INFO:tasks.workunit.client.1.smithi130.stdout:2/939: stat d4/db/dbf/df9/d30/d88/dc5/f133 0 2024-05-13T22:48:43.457 INFO:tasks.workunit.client.1.smithi130.stdout:2/940: dread - d4/db/dbf/df9/f50 zero size 2024-05-13T22:48:43.458 INFO:tasks.workunit.client.1.smithi130.stdout:1/885: unlink db/d1b/da3/d2a/d4c/la2 0 2024-05-13T22:48:43.459 INFO:tasks.workunit.client.1.smithi130.stdout:1/886: write db/d1b/da3/d2a/d38/d46/d6d/f73 [5788175,76253] 0 2024-05-13T22:48:43.460 INFO:tasks.workunit.client.1.smithi130.stdout:5/929: mkdir d6/d7/d95/d13e 0 2024-05-13T22:48:43.461 INFO:tasks.workunit.client.1.smithi130.stdout:8/833: write d1/d2/db/d15/f22 [759558,97216] 0 2024-05-13T22:48:43.462 INFO:tasks.workunit.client.1.smithi130.stdout:7/933: sync 2024-05-13T22:48:43.464 INFO:tasks.workunit.client.1.smithi130.stdout:0/939: mknod d0/d7/dd/d2e/d4b/c131 0 2024-05-13T22:48:43.465 INFO:tasks.workunit.client.1.smithi130.stdout:0/940: stat d0/d7/dd/d2e/da8/d11c/f74 0 2024-05-13T22:48:43.468 INFO:tasks.workunit.client.1.smithi130.stdout:9/950: truncate d1/d7/d69/da2/fe2 956199 0 2024-05-13T22:48:43.468 INFO:tasks.workunit.client.1.smithi130.stdout:9/951: dread - d1/d7/d23/d28/d73/db2/f114 zero size 2024-05-13T22:48:43.470 INFO:tasks.workunit.client.1.smithi130.stdout:6/883: symlink d4/d15/d1d/daa/l12f 0 2024-05-13T22:48:43.480 INFO:tasks.workunit.client.1.smithi130.stdout:5/930: fdatasync d6/d7/d19/d4d/fa3 0 2024-05-13T22:48:43.486 INFO:tasks.workunit.client.1.smithi130.stdout:8/834: creat d1/d9b/df3/f132 x:0 0 0 2024-05-13T22:48:43.490 INFO:tasks.workunit.client.1.smithi130.stdout:1/887: dread db/d1b/da3/f7a [0,4194304] 0 2024-05-13T22:48:43.507 INFO:tasks.workunit.client.1.smithi130.stdout:2/941: write d4/d56/dff/f9d [78247,41634] 0 2024-05-13T22:48:43.565 INFO:tasks.workunit.client.1.smithi130.stdout:4/915: creat d8/dc/d80/db0/df1/d33/f122 x:0 0 0 2024-05-13T22:48:43.566 INFO:tasks.workunit.client.1.smithi130.stdout:5/931: creat d6/d7/d95/f13f x:0 0 0 2024-05-13T22:48:43.568 INFO:tasks.workunit.client.1.smithi130.stdout:7/934: mkdir d7/da/d1d/d5d/da6/d9/d12d 0 2024-05-13T22:48:43.569 INFO:tasks.workunit.client.1.smithi130.stdout:7/935: chown d7/da/d1d/d5d/da6/d5b/d107/db9/d109/df4 578 1 2024-05-13T22:48:43.570 INFO:tasks.workunit.client.1.smithi130.stdout:7/936: write d7/da/d1d/f6b [3289362,85888] 0 2024-05-13T22:48:43.570 INFO:tasks.workunit.client.1.smithi130.stdout:7/937: chown d7/da/d1d/d5d/da6/d39/f7b 42806 1 2024-05-13T22:48:43.571 INFO:tasks.workunit.client.1.smithi130.stdout:9/952: mknod d1/d15/c134 0 2024-05-13T22:48:43.576 INFO:tasks.workunit.client.1.smithi130.stdout:2/942: symlink d4/d56/dff/d4a/dc6/l144 0 2024-05-13T22:48:43.579 INFO:tasks.workunit.client.1.smithi130.stdout:6/884: rename d4/d15/d1d/d28/d4e/d9e/cd8 to d4/d15/d1d/d28/ddc/c130 0 2024-05-13T22:48:43.580 INFO:tasks.workunit.client.1.smithi130.stdout:0/941: sync 2024-05-13T22:48:43.585 INFO:tasks.workunit.client.1.smithi130.stdout:9/953: read d1/d7/d23/d3d/d43/f75 [357046,93672] 0 2024-05-13T22:48:43.589 INFO:tasks.workunit.client.1.smithi130.stdout:3/944: link d8/dc/d15/la7 d8/dc/d15/dfd/l137 0 2024-05-13T22:48:43.590 INFO:tasks.workunit.client.1.smithi130.stdout:6/885: dread d4/d15/d1f/d64/fdf [0,4194304] 0 2024-05-13T22:48:43.591 INFO:tasks.workunit.client.1.smithi130.stdout:3/945: truncate d8/d1a/dfa/d12f/fa0 1396159 0 2024-05-13T22:48:43.593 INFO:tasks.workunit.client.1.smithi130.stdout:2/943: creat d4/db/dbf/df9/dfd/f145 x:0 0 0 2024-05-13T22:48:43.593 INFO:tasks.workunit.client.1.smithi130.stdout:1/888: dread db/d1b/da3/d49/d56/fbc [0,4194304] 0 2024-05-13T22:48:43.594 INFO:tasks.workunit.client.1.smithi130.stdout:2/944: chown d4/db/dbf/df9/d30/d88/d99/d9a/c13b 456 1 2024-05-13T22:48:43.598 INFO:tasks.workunit.client.1.smithi130.stdout:1/889: dwrite db/d58/fb1 [0,4194304] 0 2024-05-13T22:48:43.599 INFO:tasks.workunit.client.1.smithi130.stdout:0/942: mknod d0/d7/dd/d17/d52/d67/d68/d96/c132 0 2024-05-13T22:48:43.601 INFO:tasks.workunit.client.1.smithi130.stdout:8/835: link d1/d2/db/d15/f74 d1/d2/db/d15/d16/d54/d58/db9/f133 0 2024-05-13T22:48:43.604 INFO:tasks.workunit.client.1.smithi130.stdout:8/836: write d1/d2/db/d15/d16/fb1 [2913715,128919] 0 2024-05-13T22:48:43.604 INFO:tasks.workunit.client.1.smithi130.stdout:6/886: creat d4/d15/d1d/daa/f131 x:0 0 0 2024-05-13T22:48:43.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.626+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/3916302951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ff8103f70 msgr2=0x7f7ff8106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:43.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.626+0000 7f7ffe4d4700 1 --2- 172.21.15.102:0/3916302951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ff8103f70 0x7f7ff8106360 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009910 tx=0x7f7fec009c20 comp rx=0 tx=0).stop 2024-05-13T22:48:43.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.626+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/3916302951 shutdown_connections 2024-05-13T22:48:43.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.626+0000 7f7ffe4d4700 1 --2- 172.21.15.102:0/3916302951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ff8103f70 0x7f7ff8106360 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.626+0000 7f7ffe4d4700 1 --2- 172.21.15.102:0/3916302951 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ff8101640 0x7f7ff8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.626+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/3916302951 >> 172.21.15.102:0/3916302951 conn(0x7f7ff80fb030 msgr2=0x7f7ff80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:43.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.626+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/3916302951 shutdown_connections 2024-05-13T22:48:43.605 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.627+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/3916302951 wait complete. 2024-05-13T22:48:43.606 INFO:tasks.workunit.client.1.smithi130.stdout:3/946: mknod d8/d1a/d55/c138 0 2024-05-13T22:48:43.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.628+0000 7f7ffe4d4700 1 Processor -- start 2024-05-13T22:48:43.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.628+0000 7f7ffe4d4700 1 -- start start 2024-05-13T22:48:43.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.629+0000 7f7ffe4d4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ff8101640 0x7f7ff81963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:43.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.629+0000 7f7ffe4d4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ff8103f70 0x7f7ff81968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:43.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.629+0000 7f7ffe4d4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8196f00 con 0x7f7ff8101640 2024-05-13T22:48:43.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.629+0000 7f7ffe4d4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8197040 con 0x7f7ff8103f70 2024-05-13T22:48:43.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.629+0000 7f7ff7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ff8101640 0x7f7ff81963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:43.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.629+0000 7f7ff7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ff8101640 0x7f7ff81963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42182/0 (socket says 172.21.15.102:42182) 2024-05-13T22:48:43.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.629+0000 7f7ff7fff700 1 -- 172.21.15.102:0/1179226491 learned_addr learned my addr 172.21.15.102:0/1179226491 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:48:43.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.629+0000 7f7ff7fff700 1 -- 172.21.15.102:0/1179226491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ff8103f70 msgr2=0x7f7ff81968e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:48:43.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.629+0000 7f7ff7fff700 1 --2- 172.21.15.102:0/1179226491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ff8103f70 0x7f7ff81968e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.629+0000 7f7ff7fff700 1 -- 172.21.15.102:0/1179226491 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec0095f0 con 0x7f7ff8101640 2024-05-13T22:48:43.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.630+0000 7f7ff7fff700 1 --2- 172.21.15.102:0/1179226491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ff8101640 0x7f7ff81963a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800b4b0 tx=0x7f7fe800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:43.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.630+0000 7f7ff57fa700 1 -- 172.21.15.102:0/1179226491 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fe8011800 con 0x7f7ff8101640 2024-05-13T22:48:43.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.630+0000 7f7ff57fa700 1 -- 172.21.15.102:0/1179226491 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7fe8011e10 con 0x7f7ff8101640 2024-05-13T22:48:43.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.630+0000 7f7ff57fa700 1 -- 172.21.15.102:0/1179226491 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fe800f2e0 con 0x7f7ff8101640 2024-05-13T22:48:43.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.630+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/1179226491 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff80ff490 con 0x7f7ff8101640 2024-05-13T22:48:43.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.630+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/1179226491 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff80ffa60 con 0x7f7ff8101640 2024-05-13T22:48:43.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.631+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/1179226491 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff8109920 con 0x7f7ff8101640 2024-05-13T22:48:43.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.632+0000 7f7ff57fa700 1 -- 172.21.15.102:0/1179226491 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) v1 ==== 91288+0+0 (secure 0 0 0) 0x7f7fe8010380 con 0x7f7ff8101640 2024-05-13T22:48:43.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.633+0000 7f7ff57fa700 1 --2- 172.21.15.102:0/1179226491 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f7fe006d2f0 0x7f7fe006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:43.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.633+0000 7f7ff57fa700 1 -- 172.21.15.102:0/1179226491 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7fe808c700 con 0x7f7ff8101640 2024-05-13T22:48:43.612 INFO:tasks.workunit.client.1.smithi130.stdout:8/837: dwrite d1/dc/dc3/fcc [0,4194304] 0 2024-05-13T22:48:43.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.633+0000 7f7ff77fe700 1 --2- 172.21.15.102:0/1179226491 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f7fe006d2f0 0x7f7fe006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:43.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.634+0000 7f7ff77fe700 1 --2- 172.21.15.102:0/1179226491 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f7fe006d2f0 0x7f7fe006f7b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7fec000c00 tx=0x7f7fec005fb0 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:43.613 INFO:tasks.workunit.client.1.smithi130.stdout:1/890: mknod db/d1b/da3/d2a/d4c/d5b/c116 0 2024-05-13T22:48:43.613 INFO:tasks.workunit.client.1.smithi130.stdout:1/891: write db/d4a/f95 [538364,18305] 0 2024-05-13T22:48:43.615 INFO:tasks.workunit.client.1.smithi130.stdout:1/892: dread - db/d1b/da3/d49/f64 zero size 2024-05-13T22:48:43.618 INFO:tasks.workunit.client.1.smithi130.stdout:3/947: write d8/d1a/dfa/d2c/ff5 [7534533,114607] 0 2024-05-13T22:48:43.619 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.641+0000 7f7ff57fa700 1 -- 172.21.15.102:0/1179226491 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f7fe8059b00 con 0x7f7ff8101640 2024-05-13T22:48:43.621 INFO:tasks.workunit.client.1.smithi130.stdout:2/945: symlink d4/db/dbf/df9/d30/l146 0 2024-05-13T22:48:43.624 INFO:tasks.workunit.client.1.smithi130.stdout:4/916: rename d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/fdc to d8/dc/d80/db0/df1/f123 0 2024-05-13T22:48:43.630 INFO:tasks.workunit.client.1.smithi130.stdout:1/893: write db/d4a/f70 [4233015,104887] 0 2024-05-13T22:48:43.634 INFO:tasks.workunit.client.1.smithi130.stdout:3/948: dread d8/dc/d15/f58 [0,4194304] 0 2024-05-13T22:48:43.636 INFO:tasks.workunit.client.1.smithi130.stdout:5/932: rename d6/d7/d19/d4d/d6a/faa to d6/d7/d19/d4d/d8a/df7/d13c/f140 0 2024-05-13T22:48:43.638 INFO:tasks.workunit.client.1.smithi130.stdout:4/917: creat d8/dc/d80/db0/df1/d33/d3f/d57/dc0/f124 x:0 0 0 2024-05-13T22:48:43.646 INFO:tasks.workunit.client.1.smithi130.stdout:3/949: truncate d8/dc/f3e 140927 0 2024-05-13T22:48:43.652 INFO:tasks.workunit.client.1.smithi130.stdout:0/943: link d0/d7/dd/d2e/d9e/da7/dc3/c12e d0/d7/dd/d2e/da8/dcd/d124/c133 0 2024-05-13T22:48:43.654 INFO:tasks.workunit.client.1.smithi130.stdout:6/887: getdents d4/d15/d1d/d28 0 2024-05-13T22:48:43.655 INFO:tasks.workunit.client.1.smithi130.stdout:7/938: rmdir d7/da/d1d/d5d/da6/d9 39 2024-05-13T22:48:43.656 INFO:tasks.workunit.client.1.smithi130.stdout:7/939: stat d7/da/l84 0 2024-05-13T22:48:43.658 INFO:tasks.workunit.client.1.smithi130.stdout:1/894: fsync db/d1b/da3/d72/d75/dee/ff5 0 2024-05-13T22:48:43.664 INFO:tasks.workunit.client.1.smithi130.stdout:0/944: dread d0/d7/dd/d17/d1f/d2c/fbd [0,4194304] 0 2024-05-13T22:48:43.672 INFO:tasks.workunit.client.1.smithi130.stdout:7/940: unlink d7/da/d1d/d5d/da6/d5b/d107/c48 0 2024-05-13T22:48:43.679 INFO:tasks.workunit.client.1.smithi130.stdout:9/954: rename d1/d7/d23/d28/d2d/d2f/d63/d6b/ccc to d1/d7/d23/d28/d2d/d2f/d63/c135 0 2024-05-13T22:48:43.683 INFO:tasks.workunit.client.1.smithi130.stdout:0/945: mkdir d0/d7/dd/d17/d52/d10e/d134 0 2024-05-13T22:48:43.684 INFO:tasks.workunit.client.1.smithi130.stdout:4/918: link d8/dc/d80/db0/df1/d33/d44/d61/f98 d8/dc/d80/d85/f125 0 2024-05-13T22:48:43.689 INFO:tasks.workunit.client.1.smithi130.stdout:7/941: write d7/da/d1d/d5d/da6/d9/d4a/dbd/d40/f77 [9254286,86240] 0 2024-05-13T22:48:43.691 INFO:tasks.workunit.client.1.smithi130.stdout:8/838: dwrite d1/d9b/fb5 [0,4194304] 0 2024-05-13T22:48:43.696 INFO:tasks.workunit.client.1.smithi130.stdout:5/933: getdents d6/dd/d18/d6e/d82/db0/d11d 0 2024-05-13T22:48:43.696 INFO:tasks.workunit.client.1.smithi130.stdout:9/955: dread d1/d7/d23/d28/d32/d65/fb9 [0,4194304] 0 2024-05-13T22:48:43.699 INFO:tasks.workunit.client.1.smithi130.stdout:7/942: dwrite d7/da/d1d/d5d/da6/d39/da9/dfb/f103 [0,4194304] 0 2024-05-13T22:48:43.700 INFO:tasks.workunit.client.1.smithi130.stdout:0/946: mknod d0/d7/dd/d2e/d9e/da7/de1/c135 0 2024-05-13T22:48:43.701 INFO:tasks.workunit.client.1.smithi130.stdout:7/943: chown d7/da/d1d/d5d/da6/d39/l3f 1 1 2024-05-13T22:48:43.712 INFO:tasks.workunit.client.1.smithi130.stdout:6/888: dwrite d4/d15/d1f/f84 [0,4194304] 0 2024-05-13T22:48:43.713 INFO:tasks.workunit.client.1.smithi130.stdout:2/946: rename d4/c5b to d4/d56/dff/d4a/dc6/c147 0 2024-05-13T22:48:43.714 INFO:tasks.workunit.client.1.smithi130.stdout:3/950: write d8/dc/f37 [3886933,10138] 0 2024-05-13T22:48:43.715 INFO:tasks.workunit.client.1.smithi130.stdout:3/951: chown d8/dc/d15/dac/cf4 3 1 2024-05-13T22:48:43.725 INFO:tasks.workunit.client.1.smithi130.stdout:4/919: write d8/dc/d80/db0/df1/d33/d44/d56/f72 [606105,48293] 0 2024-05-13T22:48:43.728 INFO:tasks.workunit.client.1.smithi130.stdout:7/944: mknod d7/da/d1d/d5d/da6/d39/da9/df3/c12e 0 2024-05-13T22:48:43.730 INFO:tasks.workunit.client.1.smithi130.stdout:8/839: write d1/d2/f6 [3053216,35768] 0 2024-05-13T22:48:43.734 INFO:tasks.workunit.client.1.smithi130.stdout:1/895: rename db/d1b/da3/d2a/d35/fc4 to db/d1b/da3/d49/dda/f117 0 2024-05-13T22:48:43.734 INFO:tasks.workunit.client.1.smithi130.stdout:9/956: write d1/d7/d23/d28/d73/d8d/fa5 [1042027,68777] 0 2024-05-13T22:48:43.735 INFO:tasks.workunit.client.1.smithi130.stdout:1/896: stat db/d1b/da3/d2a/d4c/dc2 0 2024-05-13T22:48:43.735 INFO:tasks.workunit.client.1.smithi130.stdout:1/897: dread - db/d1b/da3/d72/daa/f101 zero size 2024-05-13T22:48:43.737 INFO:tasks.workunit.client.1.smithi130.stdout:2/947: unlink d4/db/dbf/df9/d30/d88/dc5/f11a 0 2024-05-13T22:48:43.738 INFO:tasks.workunit.client.1.smithi130.stdout:3/952: rmdir d8/dc/d15/dfd/def 39 2024-05-13T22:48:43.738 INFO:tasks.workunit.client.1.smithi130.stdout:3/953: chown d8/d1a/f4f 49724 1 2024-05-13T22:48:43.739 INFO:tasks.workunit.client.1.smithi130.stdout:3/954: write d8/d1a/dfa/d12f/f5d [1428198,46568] 0 2024-05-13T22:48:43.741 INFO:tasks.workunit.client.1.smithi130.stdout:0/947: symlink d0/d7/dd/d2e/da8/dcd/d124/l136 0 2024-05-13T22:48:43.745 INFO:tasks.workunit.client.1.smithi130.stdout:3/955: dwrite d8/d1a/dfa/d83/dba/f118 [0,4194304] 0 2024-05-13T22:48:43.746 INFO:tasks.workunit.client.1.smithi130.stdout:6/889: mkdir d4/d15/d1f/d46/d132 0 2024-05-13T22:48:43.749 INFO:tasks.workunit.client.1.smithi130.stdout:7/945: write d7/da/d1d/d5d/da6/d9/d66/d65/f9e [427702,34208] 0 2024-05-13T22:48:43.755 INFO:tasks.workunit.client.1.smithi130.stdout:1/898: mkdir db/d1b/da3/d72/d75/dee/df6/da0/d118 0 2024-05-13T22:48:43.755 INFO:tasks.workunit.client.1.smithi130.stdout:9/957: dwrite d1/d7/d23/d28/d2d/d2f/d63/f77 [0,4194304] 0 2024-05-13T22:48:43.756 INFO:tasks.workunit.client.1.smithi130.stdout:2/948: mkdir d4/db/dbf/df9/d30/d88/d99/d9a/d148 0 2024-05-13T22:48:43.760 INFO:tasks.workunit.client.1.smithi130.stdout:8/840: symlink d1/d2/db/d15/d18/d42/l134 0 2024-05-13T22:48:43.763 INFO:tasks.workunit.client.1.smithi130.stdout:9/958: mkdir d1/d7/d23/d28/d73/db2/d136 0 2024-05-13T22:48:43.766 INFO:tasks.workunit.client.1.smithi130.stdout:4/920: link d8/dc/d26/dab/f105 d8/dc/d26/f126 0 2024-05-13T22:48:43.769 INFO:tasks.workunit.client.1.smithi130.stdout:6/890: symlink d4/d15/d1d/d28/d75/da8/da0/dfe/d111/l133 0 2024-05-13T22:48:43.769 INFO:tasks.workunit.client.1.smithi130.stdout:6/891: write d4/f9 [142373,68790] 0 2024-05-13T22:48:43.770 INFO:tasks.workunit.client.1.smithi130.stdout:5/934: rename d6/dd/f13 to d6/dd/d18/d6e/dc7/f141 0 2024-05-13T22:48:43.770 INFO:tasks.workunit.client.1.smithi130.stdout:6/892: stat d4/d15/d1f/d46/d52 0 2024-05-13T22:48:43.771 INFO:tasks.workunit.client.1.smithi130.stdout:5/935: readlink d6/dd/d18/d6e/la7 0 2024-05-13T22:48:43.774 INFO:tasks.workunit.client.1.smithi130.stdout:9/959: rmdir d1/d7/d23/d28/d2d/d2f/d63 39 2024-05-13T22:48:43.776 INFO:tasks.workunit.client.1.smithi130.stdout:4/921: stat d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f24 0 2024-05-13T22:48:43.779 INFO:tasks.workunit.client.1.smithi130.stdout:1/899: dread db/d82/dc3/dd9/ff4 [0,4194304] 0 2024-05-13T22:48:43.780 INFO:tasks.workunit.client.1.smithi130.stdout:6/893: mknod d4/d15/d1d/d28/d75/da8/da0/c134 0 2024-05-13T22:48:43.780 INFO:tasks.workunit.client.1.smithi130.stdout:1/900: dread - db/d1b/da3/d72/d75/dee/df6/fdb zero size 2024-05-13T22:48:43.781 INFO:tasks.workunit.client.1.smithi130.stdout:5/936: rename d6/d1a/daf/l123 to d6/dd/d18/l142 0 2024-05-13T22:48:43.783 INFO:tasks.workunit.client.1.smithi130.stdout:9/960: write d1/d15/d51/db4/d11f/f11a [316337,12501] 0 2024-05-13T22:48:43.785 INFO:tasks.workunit.client.1.smithi130.stdout:9/961: chown d1/d15/d51/db4/cb7 222 1 2024-05-13T22:48:43.785 INFO:tasks.workunit.client.1.smithi130.stdout:3/956: dread d8/f2b [0,4194304] 0 2024-05-13T22:48:43.787 INFO:tasks.workunit.client.1.smithi130.stdout:8/841: creat d1/d2/f135 x:0 0 0 2024-05-13T22:48:43.788 INFO:tasks.workunit.client.1.smithi130.stdout:6/894: readlink d4/d15/d1f/d3c/lca 0 2024-05-13T22:48:43.789 INFO:tasks.workunit.client.1.smithi130.stdout:1/901: creat db/d1b/da3/d2a/d38/d46/f119 x:0 0 0 2024-05-13T22:48:43.789 INFO:tasks.workunit.client.1.smithi130.stdout:9/962: dread d1/d7/d23/d28/d73/f104 [0,4194304] 0 2024-05-13T22:48:43.790 INFO:tasks.workunit.client.1.smithi130.stdout:4/922: mkdir d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d127 0 2024-05-13T22:48:43.791 INFO:tasks.workunit.client.1.smithi130.stdout:7/946: dread d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/da7/fe2 [0,4194304] 0 2024-05-13T22:48:43.791 INFO:tasks.workunit.client.1.smithi130.stdout:5/937: rename d6/dd/d18/d6e/d82/l90 to d6/dd/d18/d6e/d82/db0/l143 0 2024-05-13T22:48:43.798 INFO:tasks.workunit.client.1.smithi130.stdout:1/902: rename db/d82/dc3 to db/d1b/da3/d2a/d4c/dc2/d11a 0 2024-05-13T22:48:43.799 INFO:tasks.workunit.client.1.smithi130.stdout:6/895: dread d4/d15/d1d/d28/f30 [4194304,4194304] 0 2024-05-13T22:48:43.802 INFO:tasks.workunit.client.1.smithi130.stdout:2/949: dwrite d4/db/dbf/d42/d4b/fa3 [0,4194304] 0 2024-05-13T22:48:43.802 INFO:tasks.workunit.client.1.smithi130.stdout:9/963: mkdir d1/d7/d23/d28/d73/db2/d136/d137 0 2024-05-13T22:48:43.803 INFO:tasks.workunit.client.1.smithi130.stdout:0/948: dwrite d0/d7/dd/d2e/f11b [0,4194304] 0 2024-05-13T22:48:43.805 INFO:tasks.workunit.client.1.smithi130.stdout:4/923: symlink d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/l128 0 2024-05-13T22:48:43.808 INFO:tasks.workunit.client.1.smithi130.stdout:3/957: creat d8/dc/f139 x:0 0 0 2024-05-13T22:48:43.810 INFO:tasks.workunit.client.1.smithi130.stdout:6/896: dread d4/d15/d1d/d60/da2/da4/fc6 [0,4194304] 0 2024-05-13T22:48:43.811 INFO:tasks.workunit.client.1.smithi130.stdout:0/949: truncate d0/d7/dd/d17/d52/d5b/d5f/fc9 4452682 0 2024-05-13T22:48:43.812 INFO:tasks.workunit.client.1.smithi130.stdout:9/964: dwrite d1/d15/f1d [0,4194304] 0 2024-05-13T22:48:43.815 INFO:tasks.workunit.client.1.smithi130.stdout:0/950: write d0/d7/dd/d17/d52/d67/d68/f11f [267532,35114] 0 2024-05-13T22:48:43.816 INFO:tasks.workunit.client.1.smithi130.stdout:8/842: write d1/d2/db/d15/d18/d42/d53/fd6 [796488,96608] 0 2024-05-13T22:48:43.823 INFO:tasks.workunit.client.1.smithi130.stdout:2/950: write d4/fd [2516750,130330] 0 2024-05-13T22:48:43.827 INFO:tasks.workunit.client.1.smithi130.stdout:8/843: dwrite d1/d2/db/d15/d18/d42/d53/dd0/f11d [0,4194304] 0 2024-05-13T22:48:43.827 INFO:tasks.workunit.client.1.smithi130.stdout:8/844: stat d1/dc/d5d 0 2024-05-13T22:48:43.828 INFO:tasks.workunit.client.1.smithi130.stdout:4/924: rename d8/dc/d26/f36 to d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d127/f129 0 2024-05-13T22:48:43.829 INFO:tasks.workunit.client.1.smithi130.stdout:9/965: dread d1/d7/d23/d28/d2d/d2f/fdf [0,4194304] 0 2024-05-13T22:48:43.829 INFO:tasks.workunit.client.1.smithi130.stdout:6/897: symlink d4/d15/d1f/d46/d52/d127/l135 0 2024-05-13T22:48:43.833 INFO:tasks.workunit.client.1.smithi130.stdout:1/903: symlink db/d1b/da3/d49/d56/d7f/d107/l11b 0 2024-05-13T22:48:43.838 INFO:tasks.workunit.client.1.smithi130.stdout:3/958: mkdir d8/dc/d7b/da3/dd3/d13a 0 2024-05-13T22:48:43.844 INFO:tasks.workunit.client.1.smithi130.stdout:7/947: rename d7/da/d1d/d5d/da6/d9/d66/f31 to d7/da/d1d/d5d/da6/d5b/d107/d46/dc7/f12f 0 2024-05-13T22:48:43.847 INFO:tasks.workunit.client.1.smithi130.stdout:8/845: mknod d1/d2/db/d15/d7e/dcd/c136 0 2024-05-13T22:48:43.848 INFO:tasks.workunit.client.1.smithi130.stdout:6/898: unlink d4/c9a 0 2024-05-13T22:48:43.855 INFO:tasks.workunit.client.1.smithi130.stdout:6/899: dwrite d4/d15/d1d/d28/d4e/d9e/f10c [0,4194304] 0 2024-05-13T22:48:43.855 INFO:tasks.workunit.client.1.smithi130.stdout:2/951: mknod d4/db/dbf/df9/d30/d88/d99/d9a/d148/c149 0 2024-05-13T22:48:43.855 INFO:tasks.workunit.client.1.smithi130.stdout:0/951: write d0/d7/dd/d69/fc8 [138335,106110] 0 2024-05-13T22:48:43.857 INFO:tasks.workunit.client.1.smithi130.stdout:6/900: chown d4/d15/d1d/d28/d4e/d9e/f12c 0 1 2024-05-13T22:48:43.862 INFO:tasks.workunit.client.1.smithi130.stdout:1/904: fsync db/f6a 0 2024-05-13T22:48:43.868 INFO:tasks.workunit.client.1.smithi130.stdout:3/959: unlink d8/dc/d15/dfd/def/l11b 0 2024-05-13T22:48:43.873 INFO:tasks.workunit.client.1.smithi130.stdout:2/952: dread d4/d56/dff/d4a/d76/f92 [0,4194304] 0 2024-05-13T22:48:43.873 INFO:tasks.workunit.client.1.smithi130.stdout:7/948: mknod d7/da/d1d/db1/d101/c130 0 2024-05-13T22:48:43.875 INFO:tasks.workunit.client.1.smithi130.stdout:0/952: rename d0/d7/dd/d17/d1f/db0/c10d to d0/d7/dd/d17/d52/d67/d68/c137 0 2024-05-13T22:48:43.876 INFO:tasks.workunit.client.1.smithi130.stdout:8/846: mkdir d1/d2/db/d15/d137 0 2024-05-13T22:48:43.879 INFO:tasks.workunit.client.1.smithi130.stdout:4/925: dwrite d8/fa5 [0,4194304] 0 2024-05-13T22:48:43.881 INFO:tasks.workunit.client.1.smithi130.stdout:1/905: mknod db/d1b/da3/d49/d56/d88/c11c 0 2024-05-13T22:48:43.883 INFO:tasks.workunit.client.1.smithi130.stdout:3/960: creat d8/d1a/dfa/d83/dba/f13b x:0 0 0 2024-05-13T22:48:43.885 INFO:tasks.workunit.client.1.smithi130.stdout:3/961: readlink d8/d1a/dfa/d12f/l23 0 2024-05-13T22:48:43.887 INFO:tasks.workunit.client.1.smithi130.stdout:7/949: mknod d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/c131 0 2024-05-13T22:48:43.889 INFO:tasks.workunit.client.1.smithi130.stdout:6/901: dwrite d4/d15/d1d/d28/f104 [4194304,4194304] 0 2024-05-13T22:48:43.896 INFO:tasks.workunit.client.1.smithi130.stdout:4/926: rename d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d81/fe8 to d8/dc/d80/db0/df1/d33/d3f/d57/f12a 0 2024-05-13T22:48:43.898 INFO:tasks.workunit.client.1.smithi130.stdout:8/847: dread d1/d2/db/d15/d18/fb0 [0,4194304] 0 2024-05-13T22:48:43.898 INFO:tasks.workunit.client.1.smithi130.stdout:1/906: dwrite db/d4a/f4b [0,4194304] 0 2024-05-13T22:48:43.900 INFO:tasks.workunit.client.1.smithi130.stdout:1/907: truncate db/d82/f10c 2966631 0 2024-05-13T22:48:43.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:43 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:43.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:43 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:43.900 INFO:tasks.workunit.client.1.smithi130.stdout:1/908: chown db/d1b/da3/d2a/l57 165628 1 2024-05-13T22:48:43.901 INFO:tasks.workunit.client.1.smithi130.stdout:2/953: write d4/f122 [574393,32132] 0 2024-05-13T22:48:43.902 INFO:tasks.workunit.client.1.smithi130.stdout:2/954: readlink d4/d56/dff/d4a/d76/lc2 0 2024-05-13T22:48:43.904 INFO:tasks.workunit.client.1.smithi130.stdout:2/955: fdatasync d4/db/dc/f79 0 2024-05-13T22:48:43.905 INFO:tasks.workunit.client.1.smithi130.stdout:2/956: write d4/db/dbf/df9/f71 [6744569,16989] 0 2024-05-13T22:48:43.909 INFO:tasks.workunit.client.1.smithi130.stdout:7/950: rename d7/da/d1d/d5d/da6/d5b/d107/d46/dc7 to d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/d132 0 2024-05-13T22:48:43.911 INFO:tasks.workunit.client.1.smithi130.stdout:3/962: mkdir d8/d35/ddb/d13c 0 2024-05-13T22:48:43.912 INFO:tasks.workunit.client.1.smithi130.stdout:8/848: mknod d1/d2/db/c138 0 2024-05-13T22:48:43.914 INFO:tasks.workunit.client.1.smithi130.stdout:4/927: mknod d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d81/c12b 0 2024-05-13T22:48:43.914 INFO:tasks.workunit.client.1.smithi130.stdout:0/953: dwrite d0/d7/dd/d2e/da8/fc0 [0,4194304] 0 2024-05-13T22:48:43.916 INFO:tasks.workunit.client.1.smithi130.stdout:0/954: chown d0/d7/dd/d17/d1f/d2c/d45/d57/fb6 196778 1 2024-05-13T22:48:43.918 INFO:tasks.workunit.client.1.smithi130.stdout:0/955: write d0/d7/dd/d17/d52/d5b/d5f/f12d [500779,67829] 0 2024-05-13T22:48:43.924 INFO:tasks.workunit.client.1.smithi130.stdout:8/849: mknod d1/d2/db/d15/d18/dba/d111/c139 0 2024-05-13T22:48:43.926 INFO:tasks.workunit.client.1.smithi130.stdout:3/963: dwrite d8/fbd [0,4194304] 0 2024-05-13T22:48:43.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:43 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:43.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:43 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:43.930 INFO:tasks.workunit.client.1.smithi130.stdout:4/928: creat d8/dc/d80/db0/df1/d33/d44/d61/f12c x:0 0 0 2024-05-13T22:48:43.934 INFO:tasks.workunit.client.1.smithi130.stdout:7/951: write d7/da/d1d/d42/fb5 [1087679,55006] 0 2024-05-13T22:48:43.935 INFO:tasks.workunit.client.1.smithi130.stdout:8/850: dread - d1/d2/fa7 zero size 2024-05-13T22:48:43.936 INFO:tasks.workunit.client.1.smithi130.stdout:6/902: dread d4/d15/d1d/d28/d4e/f11f [0,4194304] 0 2024-05-13T22:48:43.936 INFO:tasks.workunit.client.1.smithi130.stdout:1/909: link db/d1b/da3/d2a/d38/d46/l47 db/d4a/de9/l11d 0 2024-05-13T22:48:43.937 INFO:tasks.workunit.client.1.smithi130.stdout:1/910: chown db/d82/ca7 805 1 2024-05-13T22:48:43.941 INFO:tasks.workunit.client.1.smithi130.stdout:4/929: unlink d8/dc/f38 0 2024-05-13T22:48:43.942 INFO:tasks.workunit.client.1.smithi130.stdout:0/956: mkdir d0/d7/dd/d2e/d9e/da7/dc3/d138 0 2024-05-13T22:48:43.946 INFO:tasks.workunit.client.1.smithi130.stdout:7/952: rename d7/da/d1d/d5d/d70 to d7/d8f/d133 0 2024-05-13T22:48:43.947 INFO:tasks.workunit.client.1.smithi130.stdout:8/851: symlink d1/d2/db/d15/d7e/dc2/dcf/l13a 0 2024-05-13T22:48:43.947 INFO:tasks.workunit.client.1.smithi130.stdout:4/930: dwrite d8/fa5 [0,4194304] 0 2024-05-13T22:48:43.948 INFO:tasks.workunit.client.1.smithi130.stdout:0/957: dread d0/d7/dd/d69/da9/ff9 [0,4194304] 0 2024-05-13T22:48:43.949 INFO:tasks.workunit.client.1.smithi130.stdout:2/957: dread d4/f19 [0,4194304] 0 2024-05-13T22:48:43.957 INFO:tasks.workunit.client.1.smithi130.stdout:7/953: dwrite d7/d8f/d133/f106 [0,4194304] 0 2024-05-13T22:48:43.959 INFO:tasks.workunit.client.1.smithi130.stdout:7/954: chown d7/da/d1d/d5d/da6/d7e/f124 14428 1 2024-05-13T22:48:43.961 INFO:tasks.workunit.client.1.smithi130.stdout:3/964: dread d8/dc/d7b/da3/dd3/fd6 [0,4194304] 0 2024-05-13T22:48:43.961 INFO:tasks.workunit.client.1.smithi130.stdout:6/903: rename d4/d15/fde to d4/d15/d1d/d102/f136 0 2024-05-13T22:48:43.962 INFO:tasks.workunit.client.1.smithi130.stdout:6/904: chown d4/d15/d1d/d28/ddc/d128 241 1 2024-05-13T22:48:43.963 INFO:tasks.workunit.client.1.smithi130.stdout:7/955: read d7/da/d1d/d42/f74 [1554141,106872] 0 2024-05-13T22:48:43.963 INFO:tasks.workunit.client.1.smithi130.stdout:6/905: write d4/d15/d1f/d46/d50/f8a [1219889,102397] 0 2024-05-13T22:48:43.966 INFO:tasks.workunit.client.1.smithi130.stdout:5/938: sync 2024-05-13T22:48:43.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:43.992+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/1179226491 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ff8190100 con 0x7f7fe006d2f0 2024-05-13T22:48:43.975 INFO:tasks.workunit.client.1.smithi130.stdout:0/958: mkdir d0/d7/dd/d69/da9/dd2/d139 0 2024-05-13T22:48:43.977 INFO:tasks.workunit.client.1.smithi130.stdout:1/911: dwrite db/d1b/da3/d49/d56/d7f/ffa [0,4194304] 0 2024-05-13T22:48:43.977 INFO:tasks.workunit.client.1.smithi130.stdout:2/958: dread d4/da8/dc9/fbe [0,4194304] 0 2024-05-13T22:48:43.978 INFO:tasks.workunit.client.1.smithi130.stdout:1/912: fsync db/d1b/da3/d49/fb8 0 2024-05-13T22:48:43.979 INFO:tasks.workunit.client.1.smithi130.stdout:8/852: creat d1/d2/db/d15/d18/f13b x:0 0 0 2024-05-13T22:48:43.986 INFO:tasks.workunit.client.1.smithi130.stdout:3/965: write d8/dc/d15/f116 [562901,4115] 0 2024-05-13T22:48:43.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.009+0000 7f7ff57fa700 1 -- 172.21.15.102:0/1179226491 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7ff8190100 con 0x7f7fe006d2f0 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (7m) 1s ago 8m 25.6M - 0.25.0 c8568f914cd2 c096e9e9a228 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (9m) 1s ago 9m 7834k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (8m) 62s ago 8m 7457k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (9m) 1s ago 9m 6656k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (8m) 62s ago 8m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (7m) 1s ago 8m 87.7M - 9.4.7 954c08fa6188 bca1790c17aa 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (3m) 1s ago 3m 132M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (3m) 1s ago 3m 218M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (3m) 62s ago 3m 13.0M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (3m) 62s ago 3m 16.0M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:48:43.988 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (8s) 1s ago 10m 56.7M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (86s) 62s ago 7m 490M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9a28ef996664 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (10m) 1s ago 10m 48.9M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (7m) 62s ago 7m 41.7M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (9m) 1s ago 9m 16.4M - 1.5.0 0da6a335fe13 af297cf52099 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (7m) 62s ago 7m 15.7M - 1.5.0 0da6a335fe13 cc0f64ee7ecd 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (7m) 1s ago 7m 194M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (6m) 1s ago 6m 207M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (6m) 1s ago 6m 175M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (5m) 62s ago 5m 108M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (5m) 62s ago 5m 101M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (4m) 62s ago 4m 95.0M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:48:43.989 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (22s) 1s ago 8m 44.4M - 2.43.0 a07b618ecd1d 5066a0d73666 2024-05-13T22:48:43.990 INFO:tasks.workunit.client.1.smithi130.stdout:3/966: truncate d8/d1a/dfa/d83/dba/f13b 916370 0 2024-05-13T22:48:43.990 INFO:tasks.workunit.client.1.smithi130.stdout:3/967: readlink d8/dc/d7b/l109 0 2024-05-13T22:48:43.990 INFO:tasks.workunit.client.1.smithi130.stdout:4/931: mkdir d8/da8/d12d 0 2024-05-13T22:48:43.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.013+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/1179226491 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f7fe006d2f0 msgr2=0x7f7fe006f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:43.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.013+0000 7f7ffe4d4700 1 --2- 172.21.15.102:0/1179226491 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f7fe006d2f0 0x7f7fe006f7b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7fec000c00 tx=0x7f7fec005fb0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.013+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/1179226491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ff8101640 msgr2=0x7f7ff81963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:43.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.013+0000 7f7ffe4d4700 1 --2- 172.21.15.102:0/1179226491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ff8101640 0x7f7ff81963a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800b4b0 tx=0x7f7fe800b870 comp rx=0 tx=0).stop 2024-05-13T22:48:43.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.013+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/1179226491 shutdown_connections 2024-05-13T22:48:43.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.013+0000 7f7ffe4d4700 1 --2- 172.21.15.102:0/1179226491 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f7fe006d2f0 0x7f7fe006f7b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.013+0000 7f7ffe4d4700 1 --2- 172.21.15.102:0/1179226491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ff8101640 0x7f7ff81963a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.013+0000 7f7ffe4d4700 1 --2- 172.21.15.102:0/1179226491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ff8103f70 0x7f7ff81968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:43.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.013+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/1179226491 >> 172.21.15.102:0/1179226491 conn(0x7f7ff80fb030 msgr2=0x7f7ff80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:43.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.013+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/1179226491 shutdown_connections 2024-05-13T22:48:43.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.013+0000 7f7ffe4d4700 1 -- 172.21.15.102:0/1179226491 wait complete. 2024-05-13T22:48:43.992 INFO:tasks.workunit.client.1.smithi130.stdout:6/906: dwrite d4/d15/d1d/d28/d4e/d9e/ddb/f118 [0,4194304] 0 2024-05-13T22:48:43.996 INFO:tasks.workunit.client.1.smithi130.stdout:0/959: fsync d0/d7/dd/d2e/d9e/da7/de1/f10a 0 2024-05-13T22:48:44.002 INFO:tasks.workunit.client.1.smithi130.stdout:2/959: dread d4/d56/f8e [0,4194304] 0 2024-05-13T22:48:44.003 INFO:tasks.workunit.client.1.smithi130.stdout:2/960: stat d4/db/d26/d9e/de4/l13d 0 2024-05-13T22:48:44.004 INFO:tasks.workunit.client.1.smithi130.stdout:1/913: mknod db/d1b/da3/d2a/d4c/d5b/c11e 0 2024-05-13T22:48:44.005 INFO:tasks.workunit.client.1.smithi130.stdout:1/914: chown db/d1b/da3/d72/d75/dee/df6/da0/f109 3944 1 2024-05-13T22:48:44.005 INFO:tasks.workunit.client.1.smithi130.stdout:1/915: chown ca 6 1 2024-05-13T22:48:44.007 INFO:tasks.workunit.client.1.smithi130.stdout:8/853: mkdir d1/dc/d5d/dde/d13c 0 2024-05-13T22:48:44.013 INFO:tasks.workunit.client.1.smithi130.stdout:4/932: creat d8/dbb/f12e x:0 0 0 2024-05-13T22:48:44.014 INFO:tasks.workunit.client.1.smithi130.stdout:6/907: fsync d4/d15/d1d/d28/f8f 0 2024-05-13T22:48:44.016 INFO:tasks.workunit.client.1.smithi130.stdout:6/908: write d4/d15/d1d/d28/d4e/d9e/ddb/ff6 [784840,4838] 0 2024-05-13T22:48:44.018 INFO:tasks.workunit.client.1.smithi130.stdout:2/961: unlink d4/db/dbf/df9/dfd/f145 0 2024-05-13T22:48:44.021 INFO:tasks.workunit.client.1.smithi130.stdout:8/854: mknod d1/d2/db/d15/d7e/dc2/dcf/c13d 0 2024-05-13T22:48:44.022 INFO:tasks.workunit.client.1.smithi130.stdout:8/855: dread - d1/d2/db/d15/d7e/dc2/dcf/f128 zero size 2024-05-13T22:48:44.023 INFO:tasks.workunit.client.1.smithi130.stdout:8/856: dread - d1/d2/db/d15/d7e/d9f/fc9 zero size 2024-05-13T22:48:44.026 INFO:tasks.workunit.client.1.smithi130.stdout:0/960: symlink d0/d7/dd/d2e/d9e/da7/dc3/d138/l13a 0 2024-05-13T22:48:44.030 INFO:tasks.workunit.client.1.smithi130.stdout:1/916: dread db/d1b/da3/d49/d56/d7f/f96 [0,4194304] 0 2024-05-13T22:48:44.030 INFO:tasks.workunit.client.1.smithi130.stdout:2/962: mknod d4/db/d26/d12e/c14a 0 2024-05-13T22:48:44.032 INFO:tasks.workunit.client.1.smithi130.stdout:2/963: write d4/db/dbf/d121/f129 [1221811,109383] 0 2024-05-13T22:48:44.036 INFO:tasks.workunit.client.1.smithi130.stdout:0/961: creat d0/d7/dd/d17/d52/d10e/f13b x:0 0 0 2024-05-13T22:48:44.037 INFO:tasks.workunit.client.1.smithi130.stdout:8/857: dread d1/d2/db/d15/d7e/f95 [0,4194304] 0 2024-05-13T22:48:44.038 INFO:tasks.workunit.client.1.smithi130.stdout:8/858: chown d1/dc/d5d/dde/d104 3007 1 2024-05-13T22:48:44.038 INFO:tasks.workunit.client.1.smithi130.stdout:1/917: rmdir db/d4a 39 2024-05-13T22:48:44.040 INFO:tasks.workunit.client.1.smithi130.stdout:3/968: getdents d8/d1a/dfa/d12f 0 2024-05-13T22:48:44.041 INFO:tasks.workunit.client.1.smithi130.stdout:2/964: unlink d4/db/f63 0 2024-05-13T22:48:44.043 INFO:tasks.workunit.client.1.smithi130.stdout:2/965: readlink d4/d56/dff/d4a/dc6/d10c/l10f 0 2024-05-13T22:48:44.044 INFO:tasks.workunit.client.1.smithi130.stdout:6/909: dwrite d4/d15/d1d/d28/d75/d7d/fc1 [0,4194304] 0 2024-05-13T22:48:44.049 INFO:tasks.workunit.client.1.smithi130.stdout:3/969: symlink d8/dc/d15/dfd/d9c/l13d 0 2024-05-13T22:48:44.059 INFO:tasks.workunit.client.1.smithi130.stdout:8/859: write d1/d2/db/d15/d18/ff6 [4956524,27583] 0 2024-05-13T22:48:44.059 INFO:tasks.workunit.client.1.smithi130.stdout:2/966: mknod d4/d56/dff/d4a/dc6/c14b 0 2024-05-13T22:48:44.061 INFO:tasks.workunit.client.1.smithi130.stdout:1/918: mkdir db/d4a/de9/d11f 0 2024-05-13T22:48:44.062 INFO:tasks.workunit.client.1.smithi130.stdout:6/910: dread - d4/d15/d1d/d60/fe0 zero size 2024-05-13T22:48:44.065 INFO:tasks.workunit.client.1.smithi130.stdout:0/962: truncate d0/f1e 1358750 0 2024-05-13T22:48:44.067 INFO:tasks.workunit.client.1.smithi130.stdout:2/967: dread - d4/db/dbf/df9/d30/d88/d99/d9a/f115 zero size 2024-05-13T22:48:44.068 INFO:tasks.workunit.client.1.smithi130.stdout:3/970: dwrite d8/d1a/dfa/d27/f123 [0,4194304] 0 2024-05-13T22:48:44.069 INFO:tasks.workunit.client.1.smithi130.stdout:3/971: chown d8/dc/d7b/da3/daf 25129 1 2024-05-13T22:48:44.070 INFO:tasks.workunit.client.1.smithi130.stdout:1/919: symlink db/d1b/da3/d72/d75/dee/df6/da0/l120 0 2024-05-13T22:48:44.079 INFO:tasks.workunit.client.1.smithi130.stdout:8/860: creat d1/d2/db/d15/d18/d42/d53/dd0/f13e x:0 0 0 2024-05-13T22:48:44.082 INFO:tasks.workunit.client.1.smithi130.stdout:9/966: sync 2024-05-13T22:48:44.085 INFO:tasks.workunit.client.1.smithi130.stdout:8/861: dread d1/d2/f7f [0,4194304] 0 2024-05-13T22:48:44.086 INFO:tasks.workunit.client.1.smithi130.stdout:3/972: fsync d8/f33 0 2024-05-13T22:48:44.087 INFO:tasks.workunit.client.1.smithi130.stdout:1/920: symlink db/d1b/da3/d72/d75/l121 0 2024-05-13T22:48:44.087 INFO:tasks.workunit.client.1.smithi130.stdout:3/973: truncate d8/dc/d7b/fff 598043 0 2024-05-13T22:48:44.094 INFO:tasks.workunit.client.1.smithi130.stdout:0/963: dwrite d0/d7/dd/d17/d1f/d2c/d5a/fc7 [0,4194304] 0 2024-05-13T22:48:44.095 INFO:tasks.workunit.client.1.smithi130.stdout:0/964: fsync d0/d7/dd/d17/d52/d5b/d5f/f12d 0 2024-05-13T22:48:44.097 INFO:tasks.workunit.client.1.smithi130.stdout:6/911: write d4/d15/d1f/d46/d52/d7b/f9d [198990,72389] 0 2024-05-13T22:48:44.098 INFO:tasks.workunit.client.1.smithi130.stdout:6/912: chown d4/d15/d1f/d3c/cec 186474 1 2024-05-13T22:48:44.106 INFO:tasks.workunit.client.1.smithi130.stdout:1/921: creat db/d1b/da3/d72/d75/dee/df6/f122 x:0 0 0 2024-05-13T22:48:44.107 INFO:tasks.workunit.client.1.smithi130.stdout:1/922: write db/d1b/da3/d2a/d38/d46/d6d/fd6 [607891,19907] 0 2024-05-13T22:48:44.112 INFO:tasks.workunit.client.1.smithi130.stdout:2/968: creat d4/d56/dff/d4a/f14c x:0 0 0 2024-05-13T22:48:44.115 INFO:tasks.workunit.client.1.smithi130.stdout:8/862: write d1/d2/db/d15/d18/d42/f6d [1684523,53146] 0 2024-05-13T22:48:44.118 INFO:tasks.workunit.client.1.smithi130.stdout:9/967: link d1/d15/f16 d1/d7/d23/d28/d32/d45/ddb/f138 0 2024-05-13T22:48:44.120 INFO:tasks.workunit.client.1.smithi130.stdout:9/968: fsync d1/d15/d9e/dfa/f123 0 2024-05-13T22:48:44.121 INFO:tasks.workunit.client.1.smithi130.stdout:3/974: dwrite d8/dc/d15/f66 [0,4194304] 0 2024-05-13T22:48:44.122 INFO:tasks.workunit.client.1.smithi130.stdout:1/923: fdatasync db/d1b/da3/d49/f48 0 2024-05-13T22:48:44.124 INFO:tasks.workunit.client.1.smithi130.stdout:0/965: write d0/d7/dd/d2e/d9e/ff0 [442491,121919] 0 2024-05-13T22:48:44.127 INFO:tasks.workunit.client.1.smithi130.stdout:1/924: write db/d1b/da3/d72/fe8 [997344,10070] 0 2024-05-13T22:48:44.130 INFO:tasks.workunit.client.1.smithi130.stdout:2/969: write d4/db/dc/f15 [5885915,99525] 0 2024-05-13T22:48:44.130 INFO:tasks.workunit.client.1.smithi130.stdout:2/970: stat d4/d56/dff 0 2024-05-13T22:48:44.131 INFO:tasks.workunit.client.1.smithi130.stdout:3/975: chown d8/d1a/dfa/cfe 90 1 2024-05-13T22:48:44.133 INFO:tasks.workunit.client.1.smithi130.stdout:0/966: chown d0/d7/dd/d17/d1f/d2c/d45/d57/f85 54903 1 2024-05-13T22:48:44.141 INFO:tasks.workunit.client.1.smithi130.stdout:9/969: dread - d1/d7/d23/d28/d2d/d2f/d63/d6b/ff1 zero size 2024-05-13T22:48:44.141 INFO:tasks.workunit.client.1.smithi130.stdout:0/967: dwrite d0/d7/dd/d2e/da8/f126 [0,4194304] 0 2024-05-13T22:48:44.142 INFO:tasks.workunit.client.1.smithi130.stdout:2/971: symlink d4/db/dbf/d121/l14d 0 2024-05-13T22:48:44.143 INFO:tasks.workunit.client.1.smithi130.stdout:3/976: symlink d8/dc/d7b/da3/daf/l13e 0 2024-05-13T22:48:44.145 INFO:tasks.workunit.client.1.smithi130.stdout:1/925: dwrite db/d1b/da3/d2a/f8e [0,4194304] 0 2024-05-13T22:48:44.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.168+0000 7f4e048a4700 1 -- 172.21.15.102:0/2423250773 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e00100cb0 msgr2=0x7f4e00101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:44.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.168+0000 7f4dfd59a700 1 -- 172.21.15.102:0/2423250773 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4df40054b0 con 0x7f4e00100cb0 2024-05-13T22:48:44.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.168+0000 7f4e048a4700 1 --2- 172.21.15.102:0/2423250773 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e00100cb0 0x7f4e00101110 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009910 tx=0x7f4df4009c20 comp rx=0 tx=0).stop 2024-05-13T22:48:44.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.169+0000 7f4e048a4700 1 -- 172.21.15.102:0/2423250773 shutdown_connections 2024-05-13T22:48:44.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.169+0000 7f4e048a4700 1 --2- 172.21.15.102:0/2423250773 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e00100cb0 0x7f4e00101110 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:44.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.169+0000 7f4e048a4700 1 --2- 172.21.15.102:0/2423250773 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e000ffab0 0x7f4e000ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:44.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.169+0000 7f4e048a4700 1 -- 172.21.15.102:0/2423250773 >> 172.21.15.102:0/2423250773 conn(0x7f4e000fb030 msgr2=0x7f4e000fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:44.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.169+0000 7f4e048a4700 1 -- 172.21.15.102:0/2423250773 shutdown_connections 2024-05-13T22:48:44.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.169+0000 7f4e048a4700 1 -- 172.21.15.102:0/2423250773 wait complete. 2024-05-13T22:48:44.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.170+0000 7f4e048a4700 1 Processor -- start 2024-05-13T22:48:44.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.170+0000 7f4e048a4700 1 -- start start 2024-05-13T22:48:44.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.170+0000 7f4e048a4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e000ffab0 0x7f4e001984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:44.148 INFO:tasks.workunit.client.1.smithi130.stdout:8/863: rename d1/d2/db/d15/d18/dba/d107/d12c/ce0 to d1/d2/db/d15/d18/d42/d53/c13f 0 2024-05-13T22:48:44.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.170+0000 7f4e048a4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e00100cb0 0x7f4e00198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:44.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.170+0000 7f4e048a4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00199050 con 0x7f4e000ffab0 2024-05-13T22:48:44.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.170+0000 7f4e048a4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00199190 con 0x7f4e00100cb0 2024-05-13T22:48:44.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.171+0000 7f4dfdd9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e00100cb0 0x7f4e00198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:44.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.171+0000 7f4dfdd9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e00100cb0 0x7f4e00198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.102:33974/0 (socket says 172.21.15.102:33974) 2024-05-13T22:48:44.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.171+0000 7f4dfdd9b700 1 -- 172.21.15.102:0/1150997770 learned_addr learned my addr 172.21.15.102:0/1150997770 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:48:44.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.171+0000 7f4dfdd9b700 1 -- 172.21.15.102:0/1150997770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e000ffab0 msgr2=0x7f4e001984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:44.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.171+0000 7f4dfdd9b700 1 --2- 172.21.15.102:0/1150997770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e000ffab0 0x7f4e001984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:44.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.171+0000 7f4dfdd9b700 1 -- 172.21.15.102:0/1150997770 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40095f0 con 0x7f4e00100cb0 2024-05-13T22:48:44.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.172+0000 7f4dfdd9b700 1 --2- 172.21.15.102:0/1150997770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e00100cb0 0x7f4e00198a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4df400bd70 tx=0x7f4df400be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:44.151 INFO:tasks.workunit.client.1.smithi130.stdout:6/913: getdents d4/d15/d1f/d3c 0 2024-05-13T22:48:44.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.172+0000 7f4def7fe700 1 -- 172.21.15.102:0/1150997770 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4df401d070 con 0x7f4e00100cb0 2024-05-13T22:48:44.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.172+0000 7f4def7fe700 1 -- 172.21.15.102:0/1150997770 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4df400f460 con 0x7f4e00100cb0 2024-05-13T22:48:44.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.173+0000 7f4def7fe700 1 -- 172.21.15.102:0/1150997770 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4df4021640 con 0x7f4e00100cb0 2024-05-13T22:48:44.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.173+0000 7f4e048a4700 1 -- 172.21.15.102:0/1150997770 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0019dbe0 con 0x7f4e00100cb0 2024-05-13T22:48:44.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.173+0000 7f4e048a4700 1 -- 172.21.15.102:0/1150997770 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0019e0d0 con 0x7f4e00100cb0 2024-05-13T22:48:44.152 INFO:tasks.workunit.client.1.smithi130.stdout:7/956: sync 2024-05-13T22:48:44.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.176+0000 7f4e048a4700 1 -- 172.21.15.102:0/1150997770 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e00108610 con 0x7f4e00100cb0 2024-05-13T22:48:44.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.177+0000 7f4def7fe700 1 -- 172.21.15.102:0/1150997770 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 24) v1 ==== 91288+0+0 (secure 0 0 0) 0x7f4df400fa00 con 0x7f4e00100cb0 2024-05-13T22:48:44.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.177+0000 7f4def7fe700 1 --2- 172.21.15.102:0/1150997770 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f4de806d370 0x7f4de806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:44.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.178+0000 7f4def7fe700 1 -- 172.21.15.102:0/1150997770 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4df408df90 con 0x7f4e00100cb0 2024-05-13T22:48:44.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.179+0000 7f4dfe59c700 1 --2- 172.21.15.102:0/1150997770 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f4de806d370 0x7f4de806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:44.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.180+0000 7f4dfe59c700 1 --2- 172.21.15.102:0/1150997770 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f4de806d370 0x7f4de806f830 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4e00100b10 tx=0x7f4df0009470 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:44.158 INFO:tasks.workunit.client.1.smithi130.stdout:1/926: rmdir db/d1b/da3/d2a/d4c 39 2024-05-13T22:48:44.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.185+0000 7f4def7fe700 1 -- 172.21.15.102:0/1150997770 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f4df405b260 con 0x7f4e00100cb0 2024-05-13T22:48:44.165 INFO:tasks.workunit.client.1.smithi130.stdout:2/972: dread d4/db/dc/f4e [0,4194304] 0 2024-05-13T22:48:44.166 INFO:tasks.workunit.client.1.smithi130.stdout:7/957: mknod d7/da/d1d/d5d/da6/d39/da9/dfb/c134 0 2024-05-13T22:48:44.169 INFO:tasks.workunit.client.1.smithi130.stdout:6/914: dread d4/d15/d1f/d46/d50/f51 [0,4194304] 0 2024-05-13T22:48:44.170 INFO:tasks.workunit.client.1.smithi130.stdout:1/927: rmdir db/d1b/da3/d49/d56/d88 39 2024-05-13T22:48:44.174 INFO:tasks.workunit.client.1.smithi130.stdout:6/915: dwrite d4/d16/f2a [0,4194304] 0 2024-05-13T22:48:44.178 INFO:tasks.workunit.client.1.smithi130.stdout:0/968: creat d0/d7/dd/d17/d1f/d2c/f13c x:0 0 0 2024-05-13T22:48:44.186 INFO:tasks.workunit.client.1.smithi130.stdout:8/864: truncate d1/d2/db/d15/d16/d54/dfb/ffd 2805854 0 2024-05-13T22:48:44.187 INFO:tasks.workunit.client.1.smithi130.stdout:9/970: link d1/fc9 d1/d7/d23/d28/d32/d65/f139 0 2024-05-13T22:48:44.187 INFO:tasks.workunit.client.1.smithi130.stdout:7/958: chown d7/da/d1d/db1/dd4/cd6 475 1 2024-05-13T22:48:44.187 INFO:tasks.workunit.client.1.smithi130.stdout:9/971: chown d1/d7/f57 13591 1 2024-05-13T22:48:44.189 INFO:tasks.workunit.client.1.smithi130.stdout:1/928: mkdir db/d1b/da3/d2a/d35/dd4/d123 0 2024-05-13T22:48:44.191 INFO:tasks.workunit.client.1.smithi130.stdout:6/916: mknod d4/d15/d1d/d102/c137 0 2024-05-13T22:48:44.192 INFO:tasks.workunit.client.1.smithi130.stdout:6/917: dread - d4/d15/d1d/d60/da2/f113 zero size 2024-05-13T22:48:44.193 INFO:tasks.workunit.client.1.smithi130.stdout:9/972: dwrite d1/d7/d23/d28/d2d/d2f/d63/f68 [0,4194304] 0 2024-05-13T22:48:44.195 INFO:tasks.workunit.client.1.smithi130.stdout:2/973: unlink d4/f3f 0 2024-05-13T22:48:44.195 INFO:tasks.workunit.client.1.smithi130.stdout:3/977: link d8/d1a/dfa/f96 d8/dc/d7b/da3/dd3/f13f 0 2024-05-13T22:48:44.197 INFO:tasks.workunit.client.1.smithi130.stdout:4/933: sync 2024-05-13T22:48:44.197 INFO:tasks.workunit.client.1.smithi130.stdout:5/939: sync 2024-05-13T22:48:44.199 INFO:tasks.workunit.client.1.smithi130.stdout:4/934: dread - d8/dc/d80/db0/df1/d33/feb zero size 2024-05-13T22:48:44.200 INFO:tasks.workunit.client.1.smithi130.stdout:8/865: stat d1/d2/db/d15/d18/d42/l12a 0 2024-05-13T22:48:44.202 INFO:tasks.workunit.client.1.smithi130.stdout:8/866: chown d1/d2/db/d15/d18/d42/l134 8929585 1 2024-05-13T22:48:44.203 INFO:tasks.workunit.client.1.smithi130.stdout:7/959: rename d7/da/d1d/d5d/da6/d9/l16 to d7/da/d1d/d5d/da6/d5b/d107/d46/d92/l135 0 2024-05-13T22:48:44.206 INFO:tasks.workunit.client.1.smithi130.stdout:0/969: write d0/d7/f16 [337978,108195] 0 2024-05-13T22:48:44.210 INFO:tasks.workunit.client.1.smithi130.stdout:3/978: write d8/dc/d15/dac/f117 [3518722,70709] 0 2024-05-13T22:48:44.211 INFO:tasks.workunit.client.1.smithi130.stdout:7/960: dwrite d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/fc5 [0,4194304] 0 2024-05-13T22:48:44.212 INFO:tasks.workunit.client.1.smithi130.stdout:3/979: chown d8/dc/d7b/da3/dd3/f115 22613783 1 2024-05-13T22:48:44.215 INFO:tasks.workunit.client.1.smithi130.stdout:2/974: creat d4/d56/dff/d4a/dc6/d10c/f14e x:0 0 0 2024-05-13T22:48:44.217 INFO:tasks.workunit.client.1.smithi130.stdout:5/940: creat d6/dd/d18/d6e/d82/d4b/de7/f144 x:0 0 0 2024-05-13T22:48:44.218 INFO:tasks.workunit.client.1.smithi130.stdout:2/975: chown d4/d56/dff/d4a/dc6/l144 911 1 2024-05-13T22:48:44.222 INFO:tasks.workunit.client.1.smithi130.stdout:3/980: dwrite d8/dc/d15/dfd/def/ff2 [0,4194304] 0 2024-05-13T22:48:44.222 INFO:tasks.workunit.client.1.smithi130.stdout:9/973: dread d1/d7/d23/d3d/fc6 [0,4194304] 0 2024-05-13T22:48:44.223 INFO:tasks.workunit.client.1.smithi130.stdout:3/981: stat d8/d1a/dfa/d83/dba/dc4/f112 0 2024-05-13T22:48:44.225 INFO:tasks.workunit.client.1.smithi130.stdout:6/918: dread d4/d15/d1f/d46/fb2 [8388608,4194304] 0 2024-05-13T22:48:44.229 INFO:tasks.workunit.client.1.smithi130.stdout:9/974: dwrite d1/d7/d23/d28/d73/d8d/deb/f10f [0,4194304] 0 2024-05-13T22:48:44.243 INFO:tasks.workunit.client.1.smithi130.stdout:2/976: creat d4/db/dbf/d121/f14f x:0 0 0 2024-05-13T22:48:44.250 INFO:tasks.workunit.client.1.smithi130.stdout:5/941: rename d6/d7/d19/f2c to d6/dd/d18/d6e/d82/db0/d11d/f145 0 2024-05-13T22:48:44.251 INFO:tasks.workunit.client.1.smithi130.stdout:8/867: mknod d1/d2/c140 0 2024-05-13T22:48:44.253 INFO:tasks.workunit.client.1.smithi130.stdout:6/919: rmdir d4/d16 39 2024-05-13T22:48:44.255 INFO:tasks.workunit.client.1.smithi130.stdout:3/982: mknod d8/dc/d7b/da3/dd3/c140 0 2024-05-13T22:48:44.260 INFO:tasks.workunit.client.1.smithi130.stdout:1/929: creat db/d1b/da3/d49/d56/d88/df9/d100/f124 x:0 0 0 2024-05-13T22:48:44.264 INFO:tasks.workunit.client.1.smithi130.stdout:4/935: truncate d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f59 5964103 0 2024-05-13T22:48:44.271 INFO:tasks.workunit.client.1.smithi130.stdout:0/970: mkdir d0/d7/dd/d2e/d4b/d13d 0 2024-05-13T22:48:44.271 INFO:tasks.workunit.client.1.smithi130.stdout:7/961: fdatasync d7/da/d1d/d5d/da6/d5b/d107/d46/fec 0 2024-05-13T22:48:44.275 INFO:tasks.workunit.client.1.smithi130.stdout:7/962: write d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/f12c [177254,114571] 0 2024-05-13T22:48:44.276 INFO:tasks.workunit.client.1.smithi130.stdout:7/963: readlink d7/l9f 0 2024-05-13T22:48:44.276 INFO:tasks.workunit.client.1.smithi130.stdout:0/971: dwrite d0/d7/dd/d17/d1f/d2c/d45/d57/df7/fff [0,4194304] 0 2024-05-13T22:48:44.278 INFO:tasks.workunit.client.1.smithi130.stdout:7/964: readlink d7/da/d1d/d5d/l5f 0 2024-05-13T22:48:44.280 INFO:tasks.workunit.client.1.smithi130.stdout:5/942: unlink d6/dd/f37 0 2024-05-13T22:48:44.281 INFO:tasks.workunit.client.1.smithi130.stdout:9/975: dwrite d1/d15/d51/db4/d11f/fea [0,4194304] 0 2024-05-13T22:48:44.282 INFO:tasks.workunit.client.1.smithi130.stdout:7/965: chown d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/ldb 3983 1 2024-05-13T22:48:44.286 INFO:tasks.workunit.client.1.smithi130.stdout:8/868: unlink d1/d2/db/d15/l5e 0 2024-05-13T22:48:44.286 INFO:tasks.workunit.client.1.smithi130.stdout:6/920: dread - d4/d15/d1f/fda zero size 2024-05-13T22:48:44.289 INFO:tasks.workunit.client.1.smithi130.stdout:1/930: dread - db/d1b/da3/d49/d56/d88/f112 zero size 2024-05-13T22:48:44.293 INFO:tasks.workunit.client.1.smithi130.stdout:4/936: rename d8/dc/d80/db0/df1/f50 to d8/dc/d80/db0/df1/d33/d44/d56/d112/f12f 0 2024-05-13T22:48:44.295 INFO:tasks.workunit.client.1.smithi130.stdout:4/937: write d8/dc/d80/db0/df1/d33/dbf/fda [674606,53176] 0 2024-05-13T22:48:44.304 INFO:tasks.workunit.client.1.smithi130.stdout:0/972: creat d0/d7/dd/d69/da9/f13e x:0 0 0 2024-05-13T22:48:44.309 INFO:tasks.workunit.client.1.smithi130.stdout:9/976: rmdir d1/d15/d51/d85/d92 39 2024-05-13T22:48:44.312 INFO:tasks.workunit.client.1.smithi130.stdout:7/966: creat d7/da/d1d/d5d/da6/d9/d4a/dbd/dc9/f136 x:0 0 0 2024-05-13T22:48:44.316 INFO:tasks.workunit.client.1.smithi130.stdout:3/983: creat d8/dc/d15/dfd/d9c/d136/f141 x:0 0 0 2024-05-13T22:48:44.320 INFO:tasks.workunit.client.1.smithi130.stdout:1/931: rename db/d1b/da3/f7a to db/d4a/de9/d11f/f125 0 2024-05-13T22:48:44.321 INFO:tasks.workunit.client.1.smithi130.stdout:2/977: dwrite d4/f5f [0,4194304] 0 2024-05-13T22:48:44.323 INFO:tasks.workunit.client.1.smithi130.stdout:1/932: write db/d1b/da3/d2a/d38/d46/f71 [4562148,88100] 0 2024-05-13T22:48:44.328 INFO:tasks.workunit.client.1.smithi130.stdout:9/977: creat d1/db/f13a x:0 0 0 2024-05-13T22:48:44.332 INFO:tasks.workunit.client.1.smithi130.stdout:7/967: fdatasync d7/d8f/d133/f8d 0 2024-05-13T22:48:44.333 INFO:tasks.workunit.client.1.smithi130.stdout:8/869: symlink d1/d2/db/d15/l141 0 2024-05-13T22:48:44.337 INFO:tasks.workunit.client.1.smithi130.stdout:3/984: mkdir d8/dc/d15/d142 0 2024-05-13T22:48:44.342 INFO:tasks.workunit.client.1.smithi130.stdout:3/985: dwrite d8/d1a/dfa/d83/dba/f11c [0,4194304] 0 2024-05-13T22:48:44.345 INFO:tasks.workunit.client.1.smithi130.stdout:5/943: rename d6/d7/d19/d4d/d53/f120 to d6/d7/d19/d4d/d8a/f146 0 2024-05-13T22:48:44.362 INFO:tasks.workunit.client.1.smithi130.stdout:5/944: mknod d6/dd/d18/d6e/dc7/df6/c147 0 2024-05-13T22:48:44.362 INFO:tasks.workunit.client.1.smithi130.stdout:5/945: chown d6/dd/d18/d6e/d82/d4b/dbc/ceb 224288470 1 2024-05-13T22:48:44.363 INFO:tasks.workunit.client.1.smithi130.stdout:2/978: unlink d4/db/dbf/df9/d30/d88/d99/f128 0 2024-05-13T22:48:44.364 INFO:tasks.workunit.client.1.smithi130.stdout:5/946: dread d6/d7/d19/fd8 [0,4194304] 0 2024-05-13T22:48:44.365 INFO:tasks.workunit.client.1.smithi130.stdout:4/938: creat d8/dc/d80/db0/df1/d33/d3f/f130 x:0 0 0 2024-05-13T22:48:44.366 INFO:tasks.workunit.client.1.smithi130.stdout:4/939: fdatasync d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d127/f129 0 2024-05-13T22:48:44.367 INFO:tasks.workunit.client.1.smithi130.stdout:0/973: getdents d0/d7/dd/d17/d1f/de7 0 2024-05-13T22:48:44.368 INFO:tasks.workunit.client.1.smithi130.stdout:0/974: chown d0/d7/dd/d17/d1f/d2c/f50 47957 1 2024-05-13T22:48:44.368 INFO:tasks.workunit.client.1.smithi130.stdout:0/975: fsync d0/d7/dd/d17/d52/d10e/f13b 0 2024-05-13T22:48:44.369 INFO:tasks.workunit.client.1.smithi130.stdout:7/968: mknod d7/da/d1d/d42/d114/c137 0 2024-05-13T22:48:44.370 INFO:tasks.workunit.client.1.smithi130.stdout:9/978: mknod d1/d7/d23/d28/d73/de6/c13b 0 2024-05-13T22:48:44.379 INFO:tasks.workunit.client.1.smithi130.stdout:3/986: rename d8/fa to d8/dc/d7b/f143 0 2024-05-13T22:48:44.385 INFO:tasks.workunit.client.1.smithi130.stdout:0/976: mkdir d0/d7/dd/d17/d52/d10e/d134/d13f 0 2024-05-13T22:48:44.386 INFO:tasks.workunit.client.1.smithi130.stdout:1/933: truncate db/d1b/da3/fdd 2321806 0 2024-05-13T22:48:44.388 INFO:tasks.workunit.client.1.smithi130.stdout:9/979: creat d1/d7/d23/d28/d32/d45/d111/d11c/f13c x:0 0 0 2024-05-13T22:48:44.389 INFO:tasks.workunit.client.1.smithi130.stdout:9/980: read d1/d15/d51/db4/d11f/fea [3741051,123121] 0 2024-05-13T22:48:44.390 INFO:tasks.workunit.client.1.smithi130.stdout:2/979: rename d4/d56/dff/d4a/d76 to d4/da8/d150 0 2024-05-13T22:48:44.391 INFO:tasks.workunit.client.1.smithi130.stdout:2/980: stat d4/db/d26/d9e/l107 0 2024-05-13T22:48:44.394 INFO:tasks.workunit.client.1.smithi130.stdout:4/940: write d8/dc/d80/d85/f125 [3837331,79584] 0 2024-05-13T22:48:44.398 INFO:tasks.workunit.client.1.smithi130.stdout:2/981: dwrite d4/da8/dc9/dbb/f117 [0,4194304] 0 2024-05-13T22:48:44.399 INFO:tasks.workunit.client.1.smithi130.stdout:2/982: fsync d4/da8/dc9/f6a 0 2024-05-13T22:48:44.404 INFO:tasks.workunit.client.1.smithi130.stdout:5/947: rename d6/d7/d19/f9a to d6/dd/d18/d6e/d82/d71/f148 0 2024-05-13T22:48:44.408 INFO:tasks.workunit.client.1.smithi130.stdout:0/977: write d0/d7/dd/d2e/fe5 [839613,92830] 0 2024-05-13T22:48:44.418 INFO:tasks.workunit.client.1.smithi130.stdout:4/941: dread d8/f1f [0,4194304] 0 2024-05-13T22:48:44.419 INFO:tasks.workunit.client.1.smithi130.stdout:1/934: truncate db/d1b/da3/d2a/d38/d46/d6d/fbe 1766388 0 2024-05-13T22:48:44.423 INFO:tasks.workunit.client.1.smithi130.stdout:6/921: sync 2024-05-13T22:48:44.423 INFO:tasks.workunit.client.1.smithi130.stdout:8/870: sync 2024-05-13T22:48:44.425 INFO:tasks.workunit.client.1.smithi130.stdout:9/981: mknod d1/d7/d23/d28/c13d 0 2024-05-13T22:48:44.429 INFO:tasks.workunit.client.1.smithi130.stdout:6/922: dwrite d4/d15/d1d/d28/d75/da8/f126 [0,4194304] 0 2024-05-13T22:48:44.434 INFO:tasks.workunit.client.1.smithi130.stdout:3/987: mknod d8/dc/d7b/da3/dd3/ddd/d114/c144 0 2024-05-13T22:48:44.435 INFO:tasks.workunit.client.1.smithi130.stdout:5/948: dwrite d6/d1a/f2a [4194304,4194304] 0 2024-05-13T22:48:44.440 INFO:tasks.workunit.client.1.smithi130.stdout:3/988: dwrite d8/dc/f139 [0,4194304] 0 2024-05-13T22:48:44.450 INFO:tasks.workunit.client.1.smithi130.stdout:4/942: rename d8/dc/d80/db0/df1/d33/d44/d61/f100 to d8/dc/d80/db0/df1/d33/d44/d61/f131 0 2024-05-13T22:48:44.451 INFO:tasks.workunit.client.1.smithi130.stdout:4/943: read - d8/dc/d80/db0/df1/d33/d44/d77/fdb zero size 2024-05-13T22:48:44.453 INFO:tasks.workunit.client.1.smithi130.stdout:9/982: fsync d1/d96/fe9 0 2024-05-13T22:48:44.455 INFO:tasks.workunit.client.1.smithi130.stdout:9/983: dread - d1/d7/d23/d28/d32/f124 zero size 2024-05-13T22:48:44.457 INFO:tasks.workunit.client.1.smithi130.stdout:4/944: dwrite d8/dc/f9a [0,4194304] 0 2024-05-13T22:48:44.460 INFO:tasks.workunit.client.1.smithi130.stdout:6/923: mknod d4/d15/d1f/d3c/c138 0 2024-05-13T22:48:44.464 INFO:tasks.workunit.client.1.smithi130.stdout:1/935: write db/f6a [633572,32084] 0 2024-05-13T22:48:44.466 INFO:tasks.workunit.client.1.smithi130.stdout:1/936: write db/d1b/da3/d49/d56/f7d [1767056,79656] 0 2024-05-13T22:48:44.485 INFO:tasks.workunit.client.1.smithi130.stdout:2/983: creat d4/db/d26/f151 x:0 0 0 2024-05-13T22:48:44.486 INFO:tasks.workunit.client.1.smithi130.stdout:8/871: dwrite d1/d2/db/d15/d18/d42/f62 [0,4194304] 0 2024-05-13T22:48:44.491 INFO:tasks.workunit.client.1.smithi130.stdout:7/969: rename d7/da/d1d/db1/dbe/dd3 to d7/d138 0 2024-05-13T22:48:44.492 INFO:tasks.workunit.client.1.smithi130.stdout:7/970: readlink d7/da/d1d/d5d/da6/d9/d66/d65/l10d 0 2024-05-13T22:48:44.493 INFO:tasks.workunit.client.1.smithi130.stdout:9/984: dread d1/d15/d51/f6d [0,4194304] 0 2024-05-13T22:48:44.497 INFO:tasks.workunit.client.1.smithi130.stdout:0/978: creat d0/f140 x:0 0 0 2024-05-13T22:48:44.501 INFO:tasks.workunit.client.1.smithi130.stdout:0/979: dwrite d0/d7/dd/d17/d1f/d2c/d5a/fb9 [4194304,4194304] 0 2024-05-13T22:48:44.503 INFO:tasks.workunit.client.1.smithi130.stdout:5/949: write d6/dd/d18/d6e/d82/d4b/de7/f5d [1158419,88384] 0 2024-05-13T22:48:44.551 INFO:tasks.workunit.client.1.smithi130.stdout:6/924: write d4/d15/d1d/d60/da2/da4/fc6 [2460811,39718] 0 2024-05-13T22:48:44.557 INFO:tasks.workunit.client.1.smithi130.stdout:1/937: sync 2024-05-13T22:48:44.593 INFO:tasks.workunit.client.1.smithi130.stdout:2/984: fsync d4/da8/d150/fdd 0 2024-05-13T22:48:44.596 INFO:tasks.workunit.client.1.smithi130.stdout:8/872: creat d1/d9b/f142 x:0 0 0 2024-05-13T22:48:44.603 INFO:tasks.workunit.client.1.smithi130.stdout:9/985: mknod d1/d7/d23/d28/d2d/d2f/d63/d6b/c13e 0 2024-05-13T22:48:44.609 INFO:tasks.workunit.client.1.smithi130.stdout:0/980: dwrite d0/d7/dd/d2e/d4b/f65 [0,4194304] 0 2024-05-13T22:48:44.609 INFO:tasks.workunit.client.1.smithi130.stdout:5/950: symlink d6/dd/d18/d6e/d82/d71/l149 0 2024-05-13T22:48:44.610 INFO:tasks.workunit.client.1.smithi130.stdout:5/951: chown d6/d1a/d92/f96 0 1 2024-05-13T22:48:44.611 INFO:tasks.workunit.client.1.smithi130.stdout:6/925: rename d4/d15/d1f/d3c/d70/ceb to d4/d15/d1d/d60/da2/da4/dac/d125/c139 0 2024-05-13T22:48:44.613 INFO:tasks.workunit.client.1.smithi130.stdout:6/926: write d4/d15/d1f/d46/d52/d7b/f112 [854349,80318] 0 2024-05-13T22:48:44.615 INFO:tasks.workunit.client.1.smithi130.stdout:1/938: creat db/d1b/da3/d2a/d38/d46/d6d/d6f/f126 x:0 0 0 2024-05-13T22:48:44.616 INFO:tasks.workunit.client.1.smithi130.stdout:0/981: dwrite d0/d7/dd/d69/da9/d8d/f9d [0,4194304] 0 2024-05-13T22:48:44.621 INFO:tasks.workunit.client.1.smithi130.stdout:8/873: truncate d1/d2/db/d15/d18/d42/d53/d56/f61 553459 0 2024-05-13T22:48:44.624 INFO:tasks.workunit.client.1.smithi130.stdout:4/945: link d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f20 d8/dfa/f132 0 2024-05-13T22:48:44.629 INFO:tasks.workunit.client.1.smithi130.stdout:9/986: symlink d1/db/l13f 0 2024-05-13T22:48:44.630 INFO:tasks.workunit.client.1.smithi130.stdout:9/987: readlink d1/d7/d23/d28/d73/db2/l117 0 2024-05-13T22:48:44.631 INFO:tasks.workunit.client.1.smithi130.stdout:9/988: fdatasync d1/f11 0 2024-05-13T22:48:44.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.654+0000 7f4e048a4700 1 -- 172.21.15.102:0/1150997770 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4e00108800 con 0x7f4e00100cb0 2024-05-13T22:48:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.655+0000 7f4def7fe700 1 -- 172.21.15.102:0/1150997770 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7f4df405adf0 con 0x7f4e00100cb0 2024-05-13T22:48:44.633 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 1 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:48:44.634 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:48:44.635 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-05-13T22:48:44.635 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 1 2024-05-13T22:48:44.635 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:48:44.635 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:48:44.636 INFO:tasks.workunit.client.1.smithi130.stdout:5/952: symlink d6/dd/d18/d6e/d82/d4b/l14a 0 2024-05-13T22:48:44.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.659+0000 7f4e048a4700 1 -- 172.21.15.102:0/1150997770 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f4de806d370 msgr2=0x7f4de806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.659+0000 7f4e048a4700 1 --2- 172.21.15.102:0/1150997770 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f4de806d370 0x7f4de806f830 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4e00100b10 tx=0x7f4df0009470 comp rx=0 tx=0).stop 2024-05-13T22:48:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.659+0000 7f4e048a4700 1 -- 172.21.15.102:0/1150997770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e00100cb0 msgr2=0x7f4e00198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:44.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.659+0000 7f4e048a4700 1 --2- 172.21.15.102:0/1150997770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e00100cb0 0x7f4e00198a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4df400bd70 tx=0x7f4df400be50 comp rx=0 tx=0).stop 2024-05-13T22:48:44.638 INFO:tasks.workunit.client.1.smithi130.stdout:5/953: chown d6/dd/d18/d6e/d82/d71/c101 900483028 1 2024-05-13T22:48:44.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.659+0000 7f4e048a4700 1 -- 172.21.15.102:0/1150997770 shutdown_connections 2024-05-13T22:48:44.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.659+0000 7f4e048a4700 1 --2- 172.21.15.102:0/1150997770 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f4de806d370 0x7f4de806f830 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:44.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.659+0000 7f4e048a4700 1 --2- 172.21.15.102:0/1150997770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e000ffab0 0x7f4e001984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:44.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.659+0000 7f4e048a4700 1 --2- 172.21.15.102:0/1150997770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4e00100cb0 0x7f4e00198a30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:44.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.659+0000 7f4e048a4700 1 -- 172.21.15.102:0/1150997770 >> 172.21.15.102:0/1150997770 conn(0x7f4e000fb030 msgr2=0x7f4e00068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:44.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.659+0000 7f4e048a4700 1 -- 172.21.15.102:0/1150997770 shutdown_connections 2024-05-13T22:48:44.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.659+0000 7f4e048a4700 1 -- 172.21.15.102:0/1150997770 wait complete. 2024-05-13T22:48:44.642 INFO:tasks.workunit.client.1.smithi130.stdout:5/954: dwrite d6/d7/f22 [0,4194304] 0 2024-05-13T22:48:44.643 INFO:tasks.workunit.client.1.smithi130.stdout:3/989: truncate d8/d1a/dfa/d2c/ff5 2865473 0 2024-05-13T22:48:44.649 INFO:tasks.workunit.client.1.smithi130.stdout:4/946: rmdir d8/dc/d26/dab 39 2024-05-13T22:48:44.649 INFO:tasks.workunit.client.1.smithi130.stdout:5/955: dread d6/d7/d19/fc4 [0,4194304] 0 2024-05-13T22:48:44.650 INFO:tasks.workunit.client.1.smithi130.stdout:7/971: truncate d7/da/d1d/d5d/da6/d9/d4a/dbd/d40/f77 4191179 0 2024-05-13T22:48:44.651 INFO:tasks.workunit.client.1.smithi130.stdout:7/972: truncate d7/da/f112 764079 0 2024-05-13T22:48:44.653 INFO:tasks.workunit.client.1.smithi130.stdout:9/989: creat d1/d7/d23/d28/d73/de6/f140 x:0 0 0 2024-05-13T22:48:44.658 INFO:tasks.workunit.client.1.smithi130.stdout:6/927: dwrite d4/d15/d1d/d28/d75/da8/da0/fa3 [0,4194304] 0 2024-05-13T22:48:44.660 INFO:tasks.workunit.client.1.smithi130.stdout:1/939: dwrite db/d82/f10c [0,4194304] 0 2024-05-13T22:48:44.661 INFO:tasks.workunit.client.1.smithi130.stdout:0/982: rename d0/d7/dd/d69/dac/c11a to d0/d7/dd/d2e/da8/d11c/c141 0 2024-05-13T22:48:44.662 INFO:tasks.workunit.client.1.smithi130.stdout:1/940: chown db/d1b/da3/d2a/de7 329952 1 2024-05-13T22:48:44.666 INFO:tasks.workunit.client.1.smithi130.stdout:7/973: creat d7/da/d1d/db1/dbe/f139 x:0 0 0 2024-05-13T22:48:44.669 INFO:tasks.workunit.client.1.smithi130.stdout:9/990: chown d1/d7/d23/d3d/c107 130721 1 2024-05-13T22:48:44.671 INFO:tasks.workunit.client.1.smithi130.stdout:7/974: dwrite d7/da/d1d/d42/fb5 [0,4194304] 0 2024-05-13T22:48:44.672 INFO:tasks.workunit.client.1.smithi130.stdout:7/975: dread - d7/da/d1d/d5d/da6/d9/d4a/dbd/dc9/f136 zero size 2024-05-13T22:48:44.680 INFO:tasks.workunit.client.1.smithi130.stdout:2/985: link d4/db/f2e d4/db/dbf/f152 0 2024-05-13T22:48:44.681 INFO:tasks.workunit.client.1.smithi130.stdout:2/986: write d4/db/dc/f20 [10053038,124423] 0 2024-05-13T22:48:44.684 INFO:tasks.workunit.client.1.smithi130.stdout:3/990: creat d8/dc/d7b/da3/dd3/d13a/f145 x:0 0 0 2024-05-13T22:48:44.687 INFO:tasks.workunit.client.1.smithi130.stdout:6/928: write d4/d15/d1f/d46/fb2 [9726122,111404] 0 2024-05-13T22:48:44.688 INFO:tasks.workunit.client.1.smithi130.stdout:0/983: mkdir d0/d7/dd/d17/d1f/db0/d142 0 2024-05-13T22:48:44.690 INFO:tasks.workunit.client.1.smithi130.stdout:4/947: mknod d8/c133 0 2024-05-13T22:48:44.690 INFO:tasks.workunit.client.1.smithi130.stdout:1/941: write db/d9c/f10e [4321959,15077] 0 2024-05-13T22:48:44.691 INFO:tasks.workunit.client.1.smithi130.stdout:6/929: chown d4/d15/d1d/d28/d4e/f56 5489951 1 2024-05-13T22:48:44.692 INFO:tasks.workunit.client.1.smithi130.stdout:0/984: readlink d0/d7/dd/d2e/d9e/da7/dc3/d138/l13a 0 2024-05-13T22:48:44.694 INFO:tasks.workunit.client.1.smithi130.stdout:8/874: dwrite d1/d2/db/d15/d18/f63 [0,4194304] 0 2024-05-13T22:48:44.694 INFO:tasks.workunit.client.1.smithi130.stdout:2/987: mknod d4/db/d26/d12e/c153 0 2024-05-13T22:48:44.695 INFO:tasks.workunit.client.1.smithi130.stdout:1/942: truncate db/d1b/da3/d72/d75/ff0 864376 0 2024-05-13T22:48:44.696 INFO:tasks.workunit.client.1.smithi130.stdout:3/991: creat d8/dc/d15/dac/f146 x:0 0 0 2024-05-13T22:48:44.696 INFO:tasks.workunit.client.1.smithi130.stdout:5/956: creat d6/f14b x:0 0 0 2024-05-13T22:48:44.698 INFO:tasks.workunit.client.1.smithi130.stdout:6/930: write d4/d15/d1d/d102/f136 [179565,111062] 0 2024-05-13T22:48:44.700 INFO:tasks.workunit.client.1.smithi130.stdout:2/988: readlink d4/db/dbf/l7a 0 2024-05-13T22:48:44.707 INFO:tasks.workunit.client.1.smithi130.stdout:8/875: dwrite d1/d2/db/d15/d18/f31 [8388608,4194304] 0 2024-05-13T22:48:44.709 INFO:tasks.workunit.client.1.smithi130.stdout:0/985: rename d0/d7/dd/d69/da9/dd2/d139 to d0/d7/dd/d69/da9/d8d/df4/d143 0 2024-05-13T22:48:44.709 INFO:tasks.workunit.client.1.smithi130.stdout:0/986: chown d0/d7/dd/d17/d1f/db0 122371 1 2024-05-13T22:48:44.710 INFO:tasks.workunit.client.1.smithi130.stdout:0/987: chown d0/d7/dd/d17/d1f/d2c/d45/d6b 148 1 2024-05-13T22:48:44.710 INFO:tasks.workunit.client.1.smithi130.stdout:1/943: read db/d1b/da3/d72/f79 [352324,79845] 0 2024-05-13T22:48:44.711 INFO:tasks.workunit.client.1.smithi130.stdout:4/948: creat d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dea/f134 x:0 0 0 2024-05-13T22:48:44.718 INFO:tasks.workunit.client.1.smithi130.stdout:4/949: dwrite d8/dc/d80/db0/df1/d33/d44/d61/f12c [0,4194304] 0 2024-05-13T22:48:44.721 INFO:tasks.workunit.client.1.smithi130.stdout:7/976: getdents d7/da/d1d/d42/dc0/d10a 0 2024-05-13T22:48:44.724 INFO:tasks.workunit.client.1.smithi130.stdout:3/992: symlink d8/dc/l147 0 2024-05-13T22:48:44.734 INFO:tasks.workunit.client.1.smithi130.stdout:1/944: creat db/d1b/da3/d72/f127 x:0 0 0 2024-05-13T22:48:44.734 INFO:tasks.workunit.client.1.smithi130.stdout:5/957: getdents d6/d7/d19/d4d/d138 0 2024-05-13T22:48:44.735 INFO:tasks.workunit.client.1.smithi130.stdout:5/958: fdatasync d6/d7/f46 0 2024-05-13T22:48:44.737 INFO:tasks.workunit.client.1.smithi130.stdout:5/959: stat d6/d7/f5e 0 2024-05-13T22:48:44.737 INFO:tasks.workunit.client.1.smithi130.stdout:0/988: unlink d0/d7/dd/d17/d52/d67/d68/d96/f84 0 2024-05-13T22:48:44.737 INFO:tasks.workunit.client.1.smithi130.stdout:2/989: dwrite d4/db/dc/d18/f7e [0,4194304] 0 2024-05-13T22:48:44.742 INFO:tasks.workunit.client.1.smithi130.stdout:3/993: mkdir d8/dc/d7b/da3/dd3/d13a/d148 0 2024-05-13T22:48:44.746 INFO:tasks.workunit.client.1.smithi130.stdout:5/960: symlink d6/dd/d18/d6e/d82/d71/l14c 0 2024-05-13T22:48:44.748 INFO:tasks.workunit.client.1.smithi130.stdout:6/931: dread d4/d15/d1f/d46/d50/fa7 [0,4194304] 0 2024-05-13T22:48:44.748 INFO:tasks.workunit.client.1.smithi130.stdout:1/945: dwrite db/d1b/da3/d2a/d38/d46/f119 [0,4194304] 0 2024-05-13T22:48:44.750 INFO:tasks.workunit.client.1.smithi130.stdout:9/991: sync 2024-05-13T22:48:44.751 INFO:tasks.workunit.client.1.smithi130.stdout:8/876: link d1/d2/db/d15/c91 d1/d2/db/d15/d16/d54/d58/d120/c143 0 2024-05-13T22:48:44.752 INFO:tasks.workunit.client.1.smithi130.stdout:2/990: dwrite d4/db/dbf/df9/d30/fb3 [0,4194304] 0 2024-05-13T22:48:44.753 INFO:tasks.workunit.client.1.smithi130.stdout:8/877: chown d1/d2/db 12836810 1 2024-05-13T22:48:44.758 INFO:tasks.workunit.client.1.smithi130.stdout:7/977: write d7/d8f/d133/f8c [323699,19107] 0 2024-05-13T22:48:44.758 INFO:tasks.workunit.client.1.smithi130.stdout:4/950: creat d8/dc/d26/f135 x:0 0 0 2024-05-13T22:48:44.763 INFO:tasks.workunit.client.1.smithi130.stdout:6/932: read d4/d15/d1f/d46/f58 [326039,71398] 0 2024-05-13T22:48:44.767 INFO:tasks.workunit.client.1.smithi130.stdout:8/878: fdatasync d1/d2/db/d15/d7e/f9c 0 2024-05-13T22:48:44.767 INFO:tasks.workunit.client.1.smithi130.stdout:3/994: dread d8/f5e [0,4194304] 0 2024-05-13T22:48:44.768 INFO:tasks.workunit.client.1.smithi130.stdout:8/879: chown d1/d2/db/d15/l105 5 1 2024-05-13T22:48:44.768 INFO:tasks.workunit.client.1.smithi130.stdout:3/995: write d8/d1a/dfa/d12f/f5d [2555998,69810] 0 2024-05-13T22:48:44.770 INFO:tasks.workunit.client.1.smithi130.stdout:8/880: chown d1/d2/db/d15/d18/dba/d111/f114 58792 1 2024-05-13T22:48:44.775 INFO:tasks.workunit.client.1.smithi130.stdout:0/989: dwrite d0/d7/dd/d17/d52/f9a [0,4194304] 0 2024-05-13T22:48:44.777 INFO:tasks.workunit.client.1.smithi130.stdout:4/951: fsync d8/dc/d80/db0/df1/d33/d44/d56/d8c/fd2 0 2024-05-13T22:48:44.788 INFO:tasks.workunit.client.1.smithi130.stdout:3/996: rmdir d8/dc/d15/dac 39 2024-05-13T22:48:44.790 INFO:tasks.workunit.client.1.smithi130.stdout:5/961: rmdir d6/d1a/d119 0 2024-05-13T22:48:44.792 INFO:tasks.workunit.client.1.smithi130.stdout:2/991: dwrite d4/db/dc/d18/f1d [0,4194304] 0 2024-05-13T22:48:44.792 INFO:tasks.workunit.client.1.smithi130.stdout:5/962: write d6/dd/d18/d6e/d82/db0/d11d/f121 [537753,8878] 0 2024-05-13T22:48:44.801 INFO:tasks.workunit.client.1.smithi130.stdout:9/992: truncate d1/d7/d23/d28/d32/d65/fd0 699988 0 2024-05-13T22:48:44.801 INFO:tasks.workunit.client.1.smithi130.stdout:1/946: link db/d1b/da3/d2a/d35/f4d db/d1b/da3/d49/d56/d88/df9/d100/f128 0 2024-05-13T22:48:44.802 INFO:tasks.workunit.client.1.smithi130.stdout:9/993: chown d1/d7/d23/d28/d2d/f74 370 1 2024-05-13T22:48:44.802 INFO:tasks.workunit.client.1.smithi130.stdout:1/947: fsync db/d1b/da3/d72/f127 0 2024-05-13T22:48:44.803 INFO:tasks.workunit.client.1.smithi130.stdout:9/994: write d1/d15/f86 [2934977,92298] 0 2024-05-13T22:48:44.805 INFO:tasks.workunit.client.1.smithi130.stdout:1/948: read db/d1b/da3/d2a/d35/dd4/fdc [531561,126793] 0 2024-05-13T22:48:44.807 INFO:tasks.workunit.client.1.smithi130.stdout:8/881: fdatasync d1/d2/db/d15/d18/d1a/f90 0 2024-05-13T22:48:44.807 INFO:tasks.workunit.client.1.smithi130.stdout:5/963: rmdir d6/d1a 39 2024-05-13T22:48:44.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.831+0000 7f0addb49700 1 -- 172.21.15.102:0/3529315467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ad80ffae0 msgr2=0x7f0ad80fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:44.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.831+0000 7f0addb49700 1 --2- 172.21.15.102:0/3529315467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ad80ffae0 0x7f0ad80fff00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009910 tx=0x7f0ac8009c20 comp rx=0 tx=0).stop 2024-05-13T22:48:44.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.831+0000 7f0addb49700 1 -- 172.21.15.102:0/3529315467 shutdown_connections 2024-05-13T22:48:44.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.831+0000 7f0addb49700 1 --2- 172.21.15.102:0/3529315467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8100ce0 0x7f0ad8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:44.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.831+0000 7f0addb49700 1 --2- 172.21.15.102:0/3529315467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ad80ffae0 0x7f0ad80fff00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:44.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.831+0000 7f0addb49700 1 -- 172.21.15.102:0/3529315467 >> 172.21.15.102:0/3529315467 conn(0x7f0ad80fb080 msgr2=0x7f0ad80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:44.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.831+0000 7f0addb49700 1 -- 172.21.15.102:0/3529315467 shutdown_connections 2024-05-13T22:48:44.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.831+0000 7f0addb49700 1 -- 172.21.15.102:0/3529315467 wait complete. 2024-05-13T22:48:44.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.832+0000 7f0addb49700 1 Processor -- start 2024-05-13T22:48:44.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.832+0000 7f0addb49700 1 -- start start 2024-05-13T22:48:44.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.832+0000 7f0addb49700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ad80ffae0 0x7f0ad8069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:44.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.832+0000 7f0addb49700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8100ce0 0x7f0ad819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:44.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.832+0000 7f0addb49700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad8197010 con 0x7f0ad8100ce0 2024-05-13T22:48:44.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.832+0000 7f0addb49700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad819dae0 con 0x7f0ad80ffae0 2024-05-13T22:48:44.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.833+0000 7f0ad77fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ad80ffae0 0x7f0ad8069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:44.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.833+0000 7f0ad6ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8100ce0 0x7f0ad819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:44.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.833+0000 7f0ad77fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ad80ffae0 0x7f0ad8069890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.102:33994/0 (socket says 172.21.15.102:33994) 2024-05-13T22:48:44.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.833+0000 7f0ad6ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8100ce0 0x7f0ad819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42204/0 (socket says 172.21.15.102:42204) 2024-05-13T22:48:44.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.833+0000 7f0ad77fe700 1 -- 172.21.15.102:0/79064397 learned_addr learned my addr 172.21.15.102:0/79064397 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:48:44.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.833+0000 7f0ad6ffd700 1 -- 172.21.15.102:0/79064397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ad80ffae0 msgr2=0x7f0ad8069890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:44.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.833+0000 7f0ad6ffd700 1 --2- 172.21.15.102:0/79064397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ad80ffae0 0x7f0ad8069890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:44.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.833+0000 7f0ad6ffd700 1 -- 172.21.15.102:0/79064397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac80095f0 con 0x7f0ad8100ce0 2024-05-13T22:48:44.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.834+0000 7f0ad6ffd700 1 --2- 172.21.15.102:0/79064397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8100ce0 0x7f0ad819d5a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00b400 tx=0x7f0acc00b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:44.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.834+0000 7f0ad4ff9700 1 -- 172.21.15.102:0/79064397 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc011800 con 0x7f0ad8100ce0 2024-05-13T22:48:44.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.834+0000 7f0ad4ff9700 1 -- 172.21.15.102:0/79064397 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0acc00be80 con 0x7f0ad8100ce0 2024-05-13T22:48:44.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.834+0000 7f0ad4ff9700 1 -- 172.21.15.102:0/79064397 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc010e10 con 0x7f0ad8100ce0 2024-05-13T22:48:44.813 INFO:tasks.workunit.client.1.smithi130.stdout:0/990: unlink d0/d7/dd/d2e/d9e/da7/dc3/cc5 0 2024-05-13T22:48:44.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.834+0000 7f0addb49700 1 -- 172.21.15.102:0/79064397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad819dd90 con 0x7f0ad8100ce0 2024-05-13T22:48:44.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.834+0000 7f0addb49700 1 -- 172.21.15.102:0/79064397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad819e310 con 0x7f0ad8100ce0 2024-05-13T22:48:44.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.836+0000 7f0addb49700 1 -- 172.21.15.102:0/79064397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad81086c0 con 0x7f0ad8100ce0 2024-05-13T22:48:44.814 INFO:tasks.workunit.client.1.smithi130.stdout:8/882: dwrite d1/d2/db/d15/f10d [0,4194304] 0 2024-05-13T22:48:44.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.837+0000 7f0ad4ff9700 1 -- 172.21.15.102:0/79064397 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 24) v1 ==== 91288+0+0 (secure 0 0 0) 0x7f0acc011960 con 0x7f0ad8100ce0 2024-05-13T22:48:44.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.837+0000 7f0ad4ff9700 1 --2- 172.21.15.102:0/79064397 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f0ac006d2e0 0x7f0ac006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:48:44.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.838+0000 7f0ad4ff9700 1 -- 172.21.15.102:0/79064397 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0acc08cad0 con 0x7f0ad8100ce0 2024-05-13T22:48:44.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.839+0000 7f0ad77fe700 1 --2- 172.21.15.102:0/79064397 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f0ac006d2e0 0x7f0ac006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:48:44.817 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.840+0000 7f0ad77fe700 1 --2- 172.21.15.102:0/79064397 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f0ac006d2e0 0x7f0ac006f7a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8006010 tx=0x7f0ac800b560 comp rx=0 tx=0).ready entity=mgr.24369 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:48:44.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:44.842+0000 7f0ad4ff9700 1 -- 172.21.15.102:0/79064397 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f0acc059f70 con 0x7f0ad8100ce0 2024-05-13T22:48:44.850 INFO:tasks.workunit.client.1.smithi130.stdout:0/991: read - d0/d7/dd/d17/d1f/d2c/f13c zero size 2024-05-13T22:48:44.850 INFO:tasks.workunit.client.1.smithi130.stdout:4/952: rename d8/dc/d80/db0/df1/d33/d44/d56/l6c to d8/dc/d80/db0/df1/d33/dee/l136 0 2024-05-13T22:48:44.850 INFO:tasks.workunit.client.1.smithi130.stdout:3/997: mkdir d8/d149 0 2024-05-13T22:48:44.852 INFO:tasks.workunit.client.1.smithi130.stdout:4/953: chown d8/dfa/dbd/f10e 1 1 2024-05-13T22:48:44.853 INFO:tasks.workunit.client.1.smithi130.stdout:0/992: write d0/d7/dd/d2e/da8/f126 [1320738,109943] 0 2024-05-13T22:48:44.853 INFO:tasks.workunit.client.1.smithi130.stdout:6/933: link d4/d15/d1d/d28/d4e/f99 d4/d15/d1f/d64/f13a 0 2024-05-13T22:48:44.855 INFO:tasks.workunit.client.1.smithi130.stdout:4/954: dread - d8/dc/d80/db0/df1/f108 zero size 2024-05-13T22:48:44.857 INFO:tasks.workunit.client.1.smithi130.stdout:9/995: mknod d1/d7/d23/d3d/c141 0 2024-05-13T22:48:44.858 INFO:tasks.workunit.client.1.smithi130.stdout:9/996: dread - d1/d15/f7d zero size 2024-05-13T22:48:44.860 INFO:tasks.workunit.client.1.smithi130.stdout:2/992: dwrite d4/db/dc/d18/d58/f59 [0,4194304] 0 2024-05-13T22:48:44.860 INFO:tasks.workunit.client.1.smithi130.stdout:8/883: dwrite d1/d2/f80 [0,4194304] 0 2024-05-13T22:48:44.861 INFO:tasks.workunit.client.1.smithi130.stdout:4/955: write d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f107 [307670,40785] 0 2024-05-13T22:48:44.874 INFO:tasks.workunit.client.1.smithi130.stdout:2/993: creat d4/db/dbf/df9/d66/db9/f154 x:0 0 0 2024-05-13T22:48:44.877 INFO:tasks.workunit.client.1.smithi130.stdout:4/956: creat d8/dfa/f137 x:0 0 0 2024-05-13T22:48:44.879 INFO:tasks.workunit.client.1.smithi130.stdout:8/884: rmdir d1/d2/db/d15/d16/d54/d58/db9 39 2024-05-13T22:48:44.884 INFO:tasks.workunit.client.1.smithi130.stdout:7/978: sync 2024-05-13T22:48:44.886 INFO:tasks.workunit.client.1.smithi130.stdout:0/993: symlink d0/d7/dd/d17/d52/l144 0 2024-05-13T22:48:44.886 INFO:tasks.workunit.client.1.smithi130.stdout:7/979: write d7/f83 [3598051,44375] 0 2024-05-13T22:48:44.887 INFO:tasks.workunit.client.1.smithi130.stdout:2/994: write d4/db/f2c [1573078,20811] 0 2024-05-13T22:48:44.890 INFO:tasks.workunit.client.1.smithi130.stdout:2/995: chown d4/d56/dff/d4a/f14c 58980755 1 2024-05-13T22:48:44.891 INFO:tasks.workunit.client.1.smithi130.stdout:9/997: write d1/d7/d23/d3d/f40 [4662867,29637] 0 2024-05-13T22:48:44.892 INFO:tasks.workunit.client.1.smithi130.stdout:5/964: creat d6/d1a/d92/f14d x:0 0 0 2024-05-13T22:48:44.893 INFO:tasks.workunit.client.1.smithi130.stdout:3/998: dread d8/d1a/d55/fe2 [0,4194304] 0 2024-05-13T22:48:44.893 INFO:tasks.workunit.client.1.smithi130.stdout:6/934: dread d4/d15/d1d/d28/f5e [0,4194304] 0 2024-05-13T22:48:44.895 INFO:tasks.workunit.client.1.smithi130.stdout:8/885: truncate d1/d2/db/d15/d18/f5f 1005258 0 2024-05-13T22:48:44.896 INFO:tasks.workunit.client.1.smithi130.stdout:9/998: fsync d1/d7/d23/d28/d32/d45/f9b 0 2024-05-13T22:48:44.898 INFO:tasks.workunit.client.1.smithi130.stdout:7/980: dwrite d7/da/d1d/d5d/da6/f1c [0,4194304] 0 2024-05-13T22:48:44.899 INFO:tasks.workunit.client.1.smithi130.stdout:0/994: truncate d0/d7/dd/d17/d52/f80 786195 0 2024-05-13T22:48:44.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:44 smithi130 ceph-mon[36591]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:44.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:44 smithi130 ceph-mon[36591]: pgmap v39: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 103 MiB/s rd, 251 MiB/s wr, 566 op/s 2024-05-13T22:48:44.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:44 smithi130 ceph-mon[36591]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:44.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:44 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:44.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:44 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:44.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:44 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1150997770' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:48:44.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:44 smithi102 ceph-mon[28599]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:44.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:44 smithi102 ceph-mon[28599]: pgmap v39: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 103 MiB/s rd, 251 MiB/s wr, 566 op/s 2024-05-13T22:48:44.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:44 smithi102 ceph-mon[28599]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:44.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:44 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:44.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:44 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:44.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:44 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1150997770' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:48:44.941 INFO:tasks.workunit.client.1.smithi130.stdout:4/957: write d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d81/fac [43952,44413] 0 2024-05-13T22:48:44.942 INFO:tasks.workunit.client.1.smithi130.stdout:3/999: write d8/d1a/f4f [3416636,16971] 0 2024-05-13T22:48:44.942 INFO:tasks.workunit.client.1.smithi130.stdout:9/999: creat d1/d7/d23/d28/d2d/f142 x:0 0 0 2024-05-13T22:48:44.943 INFO:tasks.workunit.client.1.smithi130.stdout:0/995: write d0/d7/dd/d2e/fe5 [1242595,91650] 0 2024-05-13T22:48:44.944 INFO:tasks.workunit.client.1.smithi130.stdout:4/958: readlink d8/dc/d80/db0/df1/d33/d44/d56/lb3 0 2024-05-13T22:48:44.948 INFO:tasks.workunit.client.1.smithi130.stdout:2/996: dread d4/db/dc/f79 [0,4194304] 0 2024-05-13T22:48:44.949 INFO:tasks.workunit.client.1.smithi130.stdout:2/997: write d4/d56/dff/f134 [670044,48854] 0 2024-05-13T22:48:44.952 INFO:tasks.workunit.client.1.smithi130.stdout:7/981: rename d7/da/d1d/d5d/da6/d5b/d107/d46/f4b to d7/da/d1d/db1/d101/f13a 0 2024-05-13T22:48:44.957 INFO:tasks.workunit.client.1.smithi130.stdout:6/935: link d4/d15/d1f/d46/d52/fc3 d4/d15/d1f/d46/d132/f13b 0 2024-05-13T22:48:44.958 INFO:tasks.workunit.client.1.smithi130.stdout:6/936: fdatasync d4/d15/d1d/d28/f71 0 2024-05-13T22:48:44.958 INFO:tasks.workunit.client.1.smithi130.stdout:8/886: dread d1/d2/db/d15/f22 [0,4194304] 0 2024-05-13T22:48:44.964 INFO:tasks.workunit.client.1.smithi130.stdout:0/996: truncate d0/d7/dd/d2e/da8/d11c/f40 1384840 0 2024-05-13T22:48:44.965 INFO:tasks.workunit.client.1.smithi130.stdout:0/997: read d0/d7/dd/d17/d1f/d2c/d45/d57/f85 [950829,14281] 0 2024-05-13T22:48:44.966 INFO:tasks.workunit.client.1.smithi130.stdout:4/959: unlink d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dd5/fd6 0 2024-05-13T22:48:44.969 INFO:tasks.workunit.client.1.smithi130.stdout:6/937: stat d4/d15/d1f/c98 0 2024-05-13T22:48:44.970 INFO:tasks.workunit.client.1.smithi130.stdout:8/887: truncate d1/d2/db/f45 817766 0 2024-05-13T22:48:44.971 INFO:tasks.workunit.client.1.smithi130.stdout:4/960: dwrite d8/dc/d80/d85/f10f [4194304,4194304] 0 2024-05-13T22:48:44.973 INFO:tasks.workunit.client.1.smithi130.stdout:7/982: symlink d7/da/d1d/l13b 0 2024-05-13T22:48:44.979 INFO:tasks.workunit.client.1.smithi130.stdout:4/961: creat d8/dc/d80/db0/df1/d33/d44/f138 x:0 0 0 2024-05-13T22:48:44.980 INFO:tasks.workunit.client.1.smithi130.stdout:2/998: creat d4/db/dbf/df9/d30/f155 x:0 0 0 2024-05-13T22:48:44.997 INFO:tasks.workunit.client.1.smithi130.stdout:7/983: dread d7/da/d1d/d5d/da6/d39/f79 [0,4194304] 0 2024-05-13T22:48:45.000 INFO:tasks.workunit.client.1.smithi130.stdout:0/998: dread d0/f9c [0,4194304] 0 2024-05-13T22:48:45.001 INFO:tasks.workunit.client.1.smithi130.stdout:7/984: symlink d7/da/d86/l13c 0 2024-05-13T22:48:45.002 INFO:tasks.workunit.client.1.smithi130.stdout:6/938: write d4/d15/d1f/f2d [1205589,56729] 0 2024-05-13T22:48:45.007 INFO:tasks.workunit.client.1.smithi130.stdout:0/999: mkdir d0/d7/dd/d2e/d9e/d145 0 2024-05-13T22:48:45.009 INFO:tasks.workunit.client.1.smithi130.stdout:2/999: write d4/db/dbf/df9/d30/d88/dc5/fd5 [80450,14252] 0 2024-05-13T22:48:45.012 INFO:tasks.workunit.client.1.smithi130.stdout:7/985: rename d7/da/d1d/fcb to d7/da/d1d/d5d/da6/d5b/d107/d46/d125/f13d 0 2024-05-13T22:48:45.013 INFO:tasks.workunit.client.1.smithi130.stdout:4/962: truncate d8/dc/f9a 718045 0 2024-05-13T22:48:45.020 INFO:tasks.workunit.client.1.smithi130.stdout:7/986: rmdir d7/da/d1d/d5d/da6/d5b/d107/db9/d109/df4 39 2024-05-13T22:48:45.026 INFO:tasks.workunit.client.1.smithi130.stdout:6/939: creat d4/d15/d1d/d28/d75/f13c x:0 0 0 2024-05-13T22:48:45.035 INFO:tasks.workunit.client.1.smithi130.stdout:4/963: dread d8/f12 [0,4194304] 0 2024-05-13T22:48:45.036 INFO:tasks.workunit.client.1.smithi130.stdout:4/964: dread - d8/dc/d80/db0/df1/d33/f122 zero size 2024-05-13T22:48:45.043 INFO:tasks.workunit.client.1.smithi130.stdout:6/940: truncate d4/d15/d1d/d60/da2/da4/fc6 523875 0 2024-05-13T22:48:45.047 INFO:tasks.workunit.client.1.smithi130.stdout:4/965: dread d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f3c [0,4194304] 0 2024-05-13T22:48:45.049 INFO:tasks.workunit.client.1.smithi130.stdout:7/987: rename d7/da/d1d/f9a to d7/da/d1d/d42/f13e 0 2024-05-13T22:48:45.050 INFO:tasks.workunit.client.1.smithi130.stdout:6/941: write d4/d16/f2a [5136511,115967] 0 2024-05-13T22:48:45.051 INFO:tasks.workunit.client.1.smithi130.stdout:6/942: chown d4/d15/d1f/d64/f13a 44262257 1 2024-05-13T22:48:45.051 INFO:tasks.workunit.client.1.smithi130.stdout:6/943: chown d4/d15/d1d/d28/l40 20764920 1 2024-05-13T22:48:45.052 INFO:tasks.workunit.client.1.smithi130.stdout:6/944: write d4/d15/d1d/d28/d75/f80 [484923,110565] 0 2024-05-13T22:48:45.054 INFO:tasks.workunit.client.1.smithi130.stdout:4/966: symlink d8/dc/d80/db0/df1/d33/d44/d77/l139 0 2024-05-13T22:48:45.056 INFO:tasks.workunit.client.1.smithi130.stdout:4/967: write d8/dc/d80/db0/df1/d33/d3f/d57/dc0/f124 [405200,113508] 0 2024-05-13T22:48:45.060 INFO:tasks.workunit.client.1.smithi130.stdout:6/945: unlink d4/d15/d1f/d46/d52/d127/l135 0 2024-05-13T22:48:45.060 INFO:tasks.workunit.client.1.smithi130.stdout:1/949: sync 2024-05-13T22:48:45.062 INFO:tasks.workunit.client.1.smithi130.stdout:4/968: unlink d8/dc/d84/f9c 0 2024-05-13T22:48:45.070 INFO:tasks.workunit.client.1.smithi130.stdout:4/969: rename d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f3c to d8/dc/d80/db0/df1/d33/d44/d56/d112/f13a 0 2024-05-13T22:48:45.072 INFO:tasks.workunit.client.1.smithi130.stdout:7/988: creat d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/f13f x:0 0 0 2024-05-13T22:48:45.073 INFO:tasks.workunit.client.1.smithi130.stdout:7/989: stat d7/da/d1d/d5d/da6/d5b/d107/d46/d92/df9/da7/fb7 0 2024-05-13T22:48:45.073 INFO:tasks.workunit.client.1.smithi130.stdout:1/950: dread db/d1b/da3/d49/d56/d7f/f96 [0,4194304] 0 2024-05-13T22:48:45.077 INFO:tasks.workunit.client.1.smithi130.stdout:6/946: dread d4/d15/d1d/d28/d4e/d9e/ddb/f118 [0,4194304] 0 2024-05-13T22:48:45.078 INFO:tasks.workunit.client.1.smithi130.stdout:4/970: mkdir d8/dc/d80/db0/df1/d33/d44/d56/d13b 0 2024-05-13T22:48:45.083 INFO:tasks.workunit.client.1.smithi130.stdout:6/947: mknod d4/d15/d1d/d60/da2/da4/ddd/c13d 0 2024-05-13T22:48:45.086 INFO:tasks.workunit.client.1.smithi130.stdout:4/971: rmdir d8/dc/d80/db0/df1/d33/dee 39 2024-05-13T22:48:45.088 INFO:tasks.workunit.client.1.smithi130.stdout:1/951: symlink db/d1b/da3/d2a/l129 0 2024-05-13T22:48:45.089 INFO:tasks.workunit.client.1.smithi130.stdout:1/952: truncate db/d4a/f95 1831743 0 2024-05-13T22:48:45.089 INFO:tasks.workunit.client.1.smithi130.stdout:1/953: fsync db/d1b/da3/d2a/f41 0 2024-05-13T22:48:45.094 INFO:tasks.workunit.client.1.smithi130.stdout:4/972: rename d8/dc/d80/db0/df1/cef to d8/dc/d80/db0/df1/d33/d3f/d57/dc0/c13c 0 2024-05-13T22:48:45.095 INFO:tasks.workunit.client.1.smithi130.stdout:6/948: dwrite d4/d15/d1f/d46/d50/f67 [0,4194304] 0 2024-05-13T22:48:45.100 INFO:tasks.workunit.client.1.smithi130.stdout:1/954: creat db/d9c/f12a x:0 0 0 2024-05-13T22:48:45.105 INFO:tasks.workunit.client.1.smithi130.stdout:7/990: getdents d7/da/d1d/d5d/da6/d9/d4a/dbd/d40/d127 0 2024-05-13T22:48:45.106 INFO:tasks.workunit.client.1.smithi130.stdout:4/973: rmdir d8/dc/d80/db0/df1/d33/d44/d61 39 2024-05-13T22:48:45.110 INFO:tasks.workunit.client.1.smithi130.stdout:1/955: creat db/d1b/da3/d49/d56/d88/df9/f12b x:0 0 0 2024-05-13T22:48:45.114 INFO:tasks.workunit.client.1.smithi130.stdout:7/991: creat d7/da/d1d/d5d/da6/d9/d4a/dbd/d88/d132/f140 x:0 0 0 2024-05-13T22:48:45.114 INFO:tasks.workunit.client.1.smithi130.stdout:1/956: dread db/d1b/da3/d2a/d38/d46/f119 [0,4194304] 0 2024-05-13T22:48:45.116 INFO:tasks.workunit.client.1.smithi130.stdout:7/992: dread d7/da/d86/fc3 [0,4194304] 0 2024-05-13T22:48:45.116 INFO:tasks.workunit.client.1.smithi130.stdout:4/974: read d8/dc/d80/db0/df1/d33/d44/fa3 [7313332,67929] 0 2024-05-13T22:48:45.118 INFO:tasks.workunit.client.1.smithi130.stdout:6/949: write d4/d15/d1f/d46/d50/fb6 [884215,115352] 0 2024-05-13T22:48:45.122 INFO:tasks.workunit.client.1.smithi130.stdout:1/957: creat db/d1b/da3/d49/d56/da4/f12c x:0 0 0 2024-05-13T22:48:45.124 INFO:tasks.workunit.client.1.smithi130.stdout:4/975: dwrite d8/dc/d80/db0/df1/d33/d3f/d57/dc0/f124 [0,4194304] 0 2024-05-13T22:48:45.127 INFO:tasks.workunit.client.1.smithi130.stdout:6/950: dwrite d4/d15/d1d/d28/f47 [0,4194304] 0 2024-05-13T22:48:45.128 INFO:tasks.workunit.client.1.smithi130.stdout:4/976: truncate d8/dc/d80/db0/df1/d33/d44/d56/d8c/fdd 819622 0 2024-05-13T22:48:45.130 INFO:tasks.workunit.client.1.smithi130.stdout:4/977: chown d8/dc/d80/db0/df1/d33/f4f 31997193 1 2024-05-13T22:48:45.138 INFO:tasks.workunit.client.1.smithi130.stdout:6/951: dread d4/f9 [0,4194304] 0 2024-05-13T22:48:45.140 INFO:tasks.workunit.client.1.smithi130.stdout:4/978: mknod d8/dc/d80/d85/c13d 0 2024-05-13T22:48:45.141 INFO:tasks.workunit.client.1.smithi130.stdout:4/979: chown d8/dc/d80/db0/df1/d33/d44/d77/d94/db6 666 1 2024-05-13T22:48:45.146 INFO:tasks.workunit.client.1.smithi130.stdout:4/980: creat d8/dc/d80/db0/df1/d33/d3f/d57/dc0/f13e x:0 0 0 2024-05-13T22:48:45.149 INFO:tasks.workunit.client.1.smithi130.stdout:6/952: link d4/d15/cf3 d4/d15/d1f/d46/d52/d10e/c13e 0 2024-05-13T22:48:45.150 INFO:tasks.workunit.client.1.smithi130.stdout:4/981: symlink d8/dc/d80/db0/l13f 0 2024-05-13T22:48:45.153 INFO:tasks.workunit.client.1.smithi130.stdout:4/982: mkdir d8/dc/d80/db0/df1/d33/d44/d77/d140 0 2024-05-13T22:48:45.154 INFO:tasks.workunit.client.1.smithi130.stdout:6/953: truncate d4/d15/d1d/f3d 1173506 0 2024-05-13T22:48:45.154 INFO:tasks.workunit.client.1.smithi130.stdout:1/958: write db/d1b/da3/d2a/d35/dd4/fdc [905530,20278] 0 2024-05-13T22:48:45.156 INFO:tasks.workunit.client.1.smithi130.stdout:6/954: truncate d4/d15/d1f/d46/d50/f67 5021688 0 2024-05-13T22:48:45.159 INFO:tasks.workunit.client.1.smithi130.stdout:4/983: mkdir d8/da8/d12d/d141 0 2024-05-13T22:48:45.160 INFO:tasks.workunit.client.1.smithi130.stdout:4/984: chown d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/dad/f116 5 1 2024-05-13T22:48:45.162 INFO:tasks.workunit.client.1.smithi130.stdout:1/959: rename db/d1b/da3/d2a/d35/f40 to db/d1b/da3/d2a/de7/f12d 0 2024-05-13T22:48:45.162 INFO:tasks.workunit.client.1.smithi130.stdout:1/960: readlink db/d1b/da3/d49/d56/l10a 0 2024-05-13T22:48:45.163 INFO:tasks.workunit.client.1.smithi130.stdout:6/955: dread d4/d15/d1f/f3e [0,4194304] 0 2024-05-13T22:48:45.164 INFO:tasks.workunit.client.1.smithi130.stdout:6/956: write d4/d15/d1f/d46/d52/d10e/f123 [188874,31371] 0 2024-05-13T22:48:45.164 INFO:tasks.workunit.client.1.smithi130.stdout:4/985: mknod d8/dc/d84/d121/c142 0 2024-05-13T22:48:45.166 INFO:tasks.workunit.client.1.smithi130.stdout:1/961: unlink db/d1b/da3/d2a/d38/lf2 0 2024-05-13T22:48:45.170 INFO:tasks.workunit.client.1.smithi130.stdout:6/957: stat d4/d15/d1f/d46/d52/c7e 0 2024-05-13T22:48:45.170 INFO:tasks.workunit.client.1.smithi130.stdout:6/958: readlink d4/d15/d1f/d46/lbc 0 2024-05-13T22:48:45.170 INFO:tasks.workunit.client.1.smithi130.stdout:4/986: mkdir d8/dc/d80/db0/df1/d33/d44/d77/d143 0 2024-05-13T22:48:45.172 INFO:tasks.workunit.client.1.smithi130.stdout:1/962: dwrite db/d1b/da3/fce [0,4194304] 0 2024-05-13T22:48:45.173 INFO:tasks.workunit.client.1.smithi130.stdout:4/987: write d8/dc/d80/db0/df1/d33/d44/d56/f72 [1393634,88016] 0 2024-05-13T22:48:45.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.196+0000 7f0addb49700 1 -- 172.21.15.102:0/79064397 --> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ad8105630 con 0x7f0ac006d2e0 2024-05-13T22:48:45.174 INFO:tasks.workunit.client.1.smithi130.stdout:8/888: sync 2024-05-13T22:48:45.174 INFO:tasks.workunit.client.1.smithi130.stdout:5/965: sync 2024-05-13T22:48:45.174 INFO:tasks.workunit.client.1.smithi130.stdout:6/959: rename d4/d15/d1d/d28/la1 to d4/d15/d1f/d3c/l13f 0 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.198+0000 7f0ad4ff9700 1 -- 172.21.15.102:0/79064397 <== mgr.24369 v2:172.21.15.130:6828/3751205200 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f0ad8105630 con 0x7f0ac006d2e0 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr" 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "2/2 daemons upgraded", 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading mgr daemons", 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:48:45.176 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:48:45.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.201+0000 7f0addb49700 1 -- 172.21.15.102:0/79064397 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f0ac006d2e0 msgr2=0x7f0ac006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:45.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.201+0000 7f0addb49700 1 --2- 172.21.15.102:0/79064397 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f0ac006d2e0 0x7f0ac006f7a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8006010 tx=0x7f0ac800b560 comp rx=0 tx=0).stop 2024-05-13T22:48:45.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.201+0000 7f0addb49700 1 -- 172.21.15.102:0/79064397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8100ce0 msgr2=0x7f0ad819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:48:45.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.201+0000 7f0addb49700 1 --2- 172.21.15.102:0/79064397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8100ce0 0x7f0ad819d5a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00b400 tx=0x7f0acc00b710 comp rx=0 tx=0).stop 2024-05-13T22:48:45.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.201+0000 7f0addb49700 1 -- 172.21.15.102:0/79064397 shutdown_connections 2024-05-13T22:48:45.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.201+0000 7f0addb49700 1 --2- 172.21.15.102:0/79064397 >> [v2:172.21.15.130:6828/3751205200,v1:172.21.15.130:6829/3751205200] conn(0x7f0ac006d2e0 0x7f0ac006f7a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:45.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.201+0000 7f0addb49700 1 --2- 172.21.15.102:0/79064397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ad80ffae0 0x7f0ad8069890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:45.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.202+0000 7f0addb49700 1 --2- 172.21.15.102:0/79064397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ad8100ce0 0x7f0ad819d5a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:48:45.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.202+0000 7f0addb49700 1 -- 172.21.15.102:0/79064397 >> 172.21.15.102:0/79064397 conn(0x7f0ad80fb080 msgr2=0x7f0ad8103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:48:45.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.202+0000 7f0addb49700 1 -- 172.21.15.102:0/79064397 shutdown_connections 2024-05-13T22:48:45.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:48:45.202+0000 7f0addb49700 1 -- 172.21.15.102:0/79064397 wait complete. 2024-05-13T22:48:45.181 INFO:tasks.workunit.client.1.smithi130.stdout:8/889: dwrite d1/d2/db/d15/d16/f88 [0,4194304] 0 2024-05-13T22:48:45.187 INFO:tasks.workunit.client.1.smithi130.stdout:6/960: truncate d4/f49 1457575 0 2024-05-13T22:48:45.194 INFO:tasks.workunit.client.1.smithi130.stdout:7/993: sync 2024-05-13T22:48:45.195 INFO:tasks.workunit.client.1.smithi130.stdout:4/988: creat d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f144 x:0 0 0 2024-05-13T22:48:45.197 INFO:tasks.workunit.client.1.smithi130.stdout:4/989: read d8/dc/d80/db0/df1/d33/d44/d56/d8c/fdd [365910,128538] 0 2024-05-13T22:48:45.198 INFO:tasks.workunit.client.1.smithi130.stdout:4/990: write d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/f144 [1014075,107076] 0 2024-05-13T22:48:45.198 INFO:tasks.workunit.client.1.smithi130.stdout:6/961: creat d4/d15/d1d/d28/d4e/d9e/ddb/f140 x:0 0 0 2024-05-13T22:48:45.202 INFO:tasks.workunit.client.1.smithi130.stdout:5/966: rmdir d6/dd/d18/d6e/d82/d4b/de7/d12f 0 2024-05-13T22:48:45.202 INFO:tasks.workunit.client.1.smithi130.stdout:1/963: link db/d1b/da3/d2a/ca9 db/d1b/da3/d2a/d4c/dc2/d11a/c12e 0 2024-05-13T22:48:45.203 INFO:tasks.workunit.client.1.smithi130.stdout:7/994: dread d7/da/d1d/d5d/da6/f1c [4194304,4194304] 0 2024-05-13T22:48:45.204 INFO:tasks.workunit.client.1.smithi130.stdout:1/964: read db/d1b/da3/d72/fe8 [204737,116344] 0 2024-05-13T22:48:45.212 INFO:tasks.workunit.client.1.smithi130.stdout:5/967: creat d6/dd/d79/f14e x:0 0 0 2024-05-13T22:48:45.216 INFO:tasks.workunit.client.1.smithi130.stdout:7/995: rmdir d7/da/d1d/d5d/da6/d39/da9/df3 39 2024-05-13T22:48:45.219 INFO:tasks.workunit.client.1.smithi130.stdout:1/965: creat db/d1b/da3/d49/d56/d7f/f12f x:0 0 0 2024-05-13T22:48:45.235 INFO:tasks.workunit.client.1.smithi130.stdout:8/890: truncate d1/d2/db/d15/d18/ff6 3129326 0 2024-05-13T22:48:45.237 INFO:tasks.workunit.client.1.smithi130.stdout:7/996: creat d7/da/d1d/d5d/da6/d9/d4a/dbd/d40/d127/f141 x:0 0 0 2024-05-13T22:48:45.244 INFO:tasks.workunit.client.1.smithi130.stdout:6/962: truncate d4/d16/f2a 766051 0 2024-05-13T22:48:45.247 INFO:tasks.workunit.client.1.smithi130.stdout:4/991: truncate d8/dc/d80/db0/df1/d33/d44/d77/d94/db6/d81/fac 2915047 0 2024-05-13T22:48:45.249 INFO:tasks.workunit.client.1.smithi130.stdout:5/968: write d6/dd/d79/f106 [681661,116525] 0 2024-05-13T22:48:45.249 INFO:tasks.workunit.client.1.smithi130.stdout:6/963: dread d4/d15/d1d/d28/d75/d7d/fb0 [0,4194304] 0 2024-05-13T22:48:45.250 INFO:tasks.workunit.client.1.smithi130.stdout:7/997: mknod d7/da/d1d/d5d/da6/d5b/d107/db9/d109/c142 0 2024-05-13T22:48:45.250 INFO:tasks.workunit.client.1.smithi130.stdout:1/966: dread db/f16 [0,4194304] 0 2024-05-13T22:48:45.257 INFO:tasks.workunit.client.1.smithi130.stdout:5/969: dwrite d6/dd/d18/d6e/d82/d71/f100 [0,4194304] 0 2024-05-13T22:48:45.259 INFO:tasks.workunit.client.1.smithi130.stdout:8/891: write d1/d2/db/d15/d7e/dcd/f10c [330384,37997] 0 2024-05-13T22:48:45.265 INFO:tasks.workunit.client.1.smithi130.stdout:7/998: symlink d7/da/d1d/d5d/da6/d39/da9/dfb/l143 0 2024-05-13T22:48:45.265 INFO:tasks.workunit.client.1.smithi130.stdout:7/999: fsync d7/d138/f113 0 2024-05-13T22:48:45.266 INFO:tasks.workunit.client.1.smithi130.stdout:1/967: fsync db/d1b/da3/d49/d56/d88/f112 0 2024-05-13T22:48:45.268 INFO:tasks.workunit.client.1.smithi130.stdout:5/970: dread d6/dd/d18/d6e/d82/d4b/de7/fd4 [0,4194304] 0 2024-05-13T22:48:45.268 INFO:tasks.workunit.client.1.smithi130.stdout:4/992: mkdir d8/dc/d80/db0/df1/d33/d44/d61/d145 0 2024-05-13T22:48:45.275 INFO:tasks.workunit.client.1.smithi130.stdout:6/964: dread d4/d15/d1f/f2d [0,4194304] 0 2024-05-13T22:48:45.275 INFO:tasks.workunit.client.1.smithi130.stdout:8/892: dread d1/d2/db/d15/d18/d1a/f90 [0,4194304] 0 2024-05-13T22:48:45.281 INFO:tasks.workunit.client.1.smithi130.stdout:6/965: fdatasync d4/d15/d1f/d3c/fb5 0 2024-05-13T22:48:45.281 INFO:tasks.workunit.client.1.smithi130.stdout:4/993: dwrite d8/dc/d80/db0/df1/d33/dee/f10d [0,4194304] 0 2024-05-13T22:48:45.283 INFO:tasks.workunit.client.1.smithi130.stdout:4/994: truncate d8/dc/d80/db0/df1/f108 975281 0 2024-05-13T22:48:45.284 INFO:tasks.workunit.client.1.smithi130.stdout:4/995: write d8/dbb/f12e [459059,116014] 0 2024-05-13T22:48:45.285 INFO:tasks.workunit.client.1.smithi130.stdout:5/971: rmdir d6/dd/d18/d6e/dc7/d115/d136 0 2024-05-13T22:48:45.288 INFO:tasks.workunit.client.1.smithi130.stdout:1/968: truncate db/d58/fb1 1959029 0 2024-05-13T22:48:45.291 INFO:tasks.workunit.client.1.smithi130.stdout:4/996: mknod d8/dc/d80/db0/df1/d33/dbf/c146 0 2024-05-13T22:48:45.292 INFO:tasks.workunit.client.1.smithi130.stdout:4/997: dread - d8/dc/d80/db0/df1/d33/d44/db8/f113 zero size 2024-05-13T22:48:45.293 INFO:tasks.workunit.client.1.smithi130.stdout:5/972: rename d6/dd/d18/d6e/d82/db0/d11d/d7f/fb2 to d6/d7/d19/d4d/d8a/df7/f14f 0 2024-05-13T22:48:45.293 INFO:tasks.workunit.client.1.smithi130.stdout:4/998: truncate d8/dc/d84/f117 815085 0 2024-05-13T22:48:45.294 INFO:tasks.workunit.client.1.smithi130.stdout:1/969: symlink db/d1b/da3/d72/d75/dee/l130 0 2024-05-13T22:48:45.295 INFO:tasks.workunit.client.1.smithi130.stdout:1/970: readlink db/d1b/da3/d72/d75/dee/l130 0 2024-05-13T22:48:45.297 INFO:tasks.workunit.client.1.smithi130.stdout:5/973: mknod d6/dd/d18/d6e/d82/d4b/de7/c150 0 2024-05-13T22:48:45.299 INFO:tasks.workunit.client.1.smithi130.stdout:8/893: truncate d1/dc/d5d/fee 510096 0 2024-05-13T22:48:45.301 INFO:tasks.workunit.client.1.smithi130.stdout:4/999: unlink d8/dc/d80/db0/df1/d33/d44/db8/df7/f119 0 2024-05-13T22:48:45.305 INFO:tasks.workunit.client.1.smithi130.stdout:6/966: getdents d4/d15/d1d/d60/da2/da4 0 2024-05-13T22:48:45.314 INFO:tasks.workunit.client.1.smithi130.stdout:1/971: dwrite db/d1b/da3/d72/d75/dee/ff5 [0,4194304] 0 2024-05-13T22:48:45.318 INFO:tasks.workunit.client.1.smithi130.stdout:6/967: rename d4/d15/d1f/d46/d52 to d4/d15/d1f/d3c/d70/d141 0 2024-05-13T22:48:45.320 INFO:tasks.workunit.client.1.smithi130.stdout:6/968: dread d4/d15/d1f/d46/f58 [0,4194304] 0 2024-05-13T22:48:45.322 INFO:tasks.workunit.client.1.smithi130.stdout:5/974: creat d6/dd/d18/d6e/dc7/f151 x:0 0 0 2024-05-13T22:48:45.323 INFO:tasks.workunit.client.1.smithi130.stdout:5/975: write d6/d1a/f2a [1947190,123280] 0 2024-05-13T22:48:45.324 INFO:tasks.workunit.client.1.smithi130.stdout:1/972: write db/d4a/f70 [5114219,125425] 0 2024-05-13T22:48:45.325 INFO:tasks.workunit.client.1.smithi130.stdout:1/973: dread - db/d1b/da3/d49/d56/d88/df9/d100/f124 zero size 2024-05-13T22:48:45.328 INFO:tasks.workunit.client.1.smithi130.stdout:6/969: unlink d4/d15/d1f/d3c/d70/d141/d7b/l82 0 2024-05-13T22:48:45.329 INFO:tasks.workunit.client.1.smithi130.stdout:8/894: dread d1/d2/fc5 [8388608,4194304] 0 2024-05-13T22:48:45.330 INFO:tasks.workunit.client.1.smithi130.stdout:6/970: chown d4/d15/d1d/d28/d4e/f99 54061 1 2024-05-13T22:48:45.330 INFO:tasks.workunit.client.1.smithi130.stdout:5/976: rmdir d6/d2f 39 2024-05-13T22:48:45.332 INFO:tasks.workunit.client.1.smithi130.stdout:1/974: dwrite db/d1b/da3/d49/d56/d88/f9a [0,4194304] 0 2024-05-13T22:48:45.333 INFO:tasks.workunit.client.1.smithi130.stdout:1/975: chown db/d1b/da3/d72/d75/dee/df6/da0/l120 805411450 1 2024-05-13T22:48:45.334 INFO:tasks.workunit.client.1.smithi130.stdout:6/971: rmdir d4/d15/d1d/daa 39 2024-05-13T22:48:45.335 INFO:tasks.workunit.client.1.smithi130.stdout:5/977: fsync d6/d7/d19/d4d/d8a/f146 0 2024-05-13T22:48:45.336 INFO:tasks.workunit.client.1.smithi130.stdout:5/978: write d6/d1a/d72/fcf [2099932,30650] 0 2024-05-13T22:48:45.338 INFO:tasks.workunit.client.1.smithi130.stdout:8/895: truncate d1/d2/db/d15/d18/f76 1015419 0 2024-05-13T22:48:45.345 INFO:tasks.workunit.client.1.smithi130.stdout:6/972: fdatasync d4/d15/d1d/d28/d4e/f99 0 2024-05-13T22:48:45.350 INFO:tasks.workunit.client.1.smithi130.stdout:1/976: write db/d1b/da3/d2a/d38/fab [1233042,87859] 0 2024-05-13T22:48:45.353 INFO:tasks.workunit.client.1.smithi130.stdout:6/973: unlink d4/d15/d1f/lc9 0 2024-05-13T22:48:45.353 INFO:tasks.workunit.client.1.smithi130.stdout:1/977: mknod db/d4a/c131 0 2024-05-13T22:48:45.354 INFO:tasks.workunit.client.1.smithi130.stdout:6/974: readlink d4/d15/d1d/daa/l78 0 2024-05-13T22:48:45.355 INFO:tasks.workunit.client.1.smithi130.stdout:1/978: write db/f6a [956695,97554] 0 2024-05-13T22:48:45.356 INFO:tasks.workunit.client.1.smithi130.stdout:1/979: chown db/d1b/da3/d49/d56/l10a 17272 1 2024-05-13T22:48:45.362 INFO:tasks.workunit.client.1.smithi130.stdout:6/975: dwrite d4/d15/d1d/d60/da2/f10b [0,4194304] 0 2024-05-13T22:48:45.363 INFO:tasks.workunit.client.1.smithi130.stdout:5/979: dwrite d6/d1a/fd5 [0,4194304] 0 2024-05-13T22:48:45.373 INFO:tasks.workunit.client.1.smithi130.stdout:5/980: rename d6/dd/d18/d6e/dc7/d115/c12a to d6/d1a/daf/c152 0 2024-05-13T22:48:45.379 INFO:tasks.workunit.client.1.smithi130.stdout:6/976: getdents d4/d15/d1f/d3c/d70/d141/d7b 0 2024-05-13T22:48:45.380 INFO:tasks.workunit.client.1.smithi130.stdout:6/977: creat d4/d15/d1f/d64/f142 x:0 0 0 2024-05-13T22:48:45.386 INFO:tasks.workunit.client.1.smithi130.stdout:1/980: truncate db/d1b/da3/d2a/d4c/dc2/d11a/fe2 2324899 0 2024-05-13T22:48:45.386 INFO:tasks.workunit.client.1.smithi130.stdout:1/981: readlink db/d58/la5 0 2024-05-13T22:48:45.387 INFO:tasks.workunit.client.1.smithi130.stdout:1/982: read db/d9c/f8a [2267798,7714] 0 2024-05-13T22:48:45.388 INFO:tasks.workunit.client.1.smithi130.stdout:1/983: fsync db/d1b/da3/d2a/d38/d46/d6d/fd6 0 2024-05-13T22:48:45.391 INFO:tasks.workunit.client.1.smithi130.stdout:6/978: dread d4/f12 [4194304,4194304] 0 2024-05-13T22:48:45.392 INFO:tasks.workunit.client.1.smithi130.stdout:5/981: dwrite d6/dd/d18/d6e/f8b [0,4194304] 0 2024-05-13T22:48:45.393 INFO:tasks.workunit.client.1.smithi130.stdout:5/982: write d6/d1a/fd5 [951794,89597] 0 2024-05-13T22:48:45.394 INFO:tasks.workunit.client.1.smithi130.stdout:6/979: dread d4/d15/d1d/f6f [0,4194304] 0 2024-05-13T22:48:45.396 INFO:tasks.workunit.client.1.smithi130.stdout:5/983: creat d6/dd/d18/d6e/d82/db0/d11d/f153 x:0 0 0 2024-05-13T22:48:45.399 INFO:tasks.workunit.client.1.smithi130.stdout:6/980: read d4/d15/d1f/f84 [147223,113462] 0 2024-05-13T22:48:45.403 INFO:tasks.workunit.client.1.smithi130.stdout:5/984: rename d6/d1a/daf/c12c to d6/dd/d18/c154 0 2024-05-13T22:48:45.405 INFO:tasks.workunit.client.1.smithi130.stdout:6/981: symlink d4/d15/d1f/d3c/d70/d141/d10e/l143 0 2024-05-13T22:48:45.406 INFO:tasks.workunit.client.1.smithi130.stdout:6/982: readlink d4/d15/d1d/d28/ld2 0 2024-05-13T22:48:45.408 INFO:tasks.workunit.client.1.smithi130.stdout:6/983: mknod d4/d15/d114/c144 0 2024-05-13T22:48:45.409 INFO:tasks.workunit.client.1.smithi130.stdout:6/984: truncate d4/d15/d1f/d64/fff 1188713 0 2024-05-13T22:48:45.412 INFO:tasks.workunit.client.1.smithi130.stdout:5/985: link d6/d2f/cb4 d6/dd/d18/d6e/d82/d4b/d122/c155 0 2024-05-13T22:48:45.412 INFO:tasks.workunit.client.1.smithi130.stdout:6/985: dread d4/f2e [0,4194304] 0 2024-05-13T22:48:45.418 INFO:tasks.workunit.client.1.smithi130.stdout:5/986: creat d6/dd/d18/d6e/d82/d4b/d122/f156 x:0 0 0 2024-05-13T22:48:45.419 INFO:tasks.workunit.client.1.smithi130.stdout:6/986: dread d4/d16/f2a [0,4194304] 0 2024-05-13T22:48:45.420 INFO:tasks.workunit.client.1.smithi130.stdout:5/987: creat d6/dd/d18/d6e/d82/db0/d11d/d7f/f157 x:0 0 0 2024-05-13T22:48:45.420 INFO:tasks.workunit.client.1.smithi130.stdout:6/987: write d4/d15/d1d/d28/d75/f13c [173308,72601] 0 2024-05-13T22:48:45.421 INFO:tasks.workunit.client.1.smithi130.stdout:6/988: write d4/d15/d1f/f74 [828049,51646] 0 2024-05-13T22:48:45.424 INFO:tasks.workunit.client.1.smithi130.stdout:5/988: dread d6/d1a/d92/fe5 [0,4194304] 0 2024-05-13T22:48:45.428 INFO:tasks.workunit.client.1.smithi130.stdout:6/989: link d4/d15/d1f/d64/f81 d4/d15/d1d/f145 0 2024-05-13T22:48:45.429 INFO:tasks.workunit.client.1.smithi130.stdout:5/989: dread d6/dd/d18/d6e/d82/d71/f81 [0,4194304] 0 2024-05-13T22:48:45.435 INFO:tasks.workunit.client.1.smithi130.stdout:5/990: dwrite d6/d7/d19/d4d/d6a/f135 [0,4194304] 0 2024-05-13T22:48:45.438 INFO:tasks.workunit.client.1.smithi130.stdout:5/991: creat d6/dd/d18/d6e/dc7/f158 x:0 0 0 2024-05-13T22:48:45.440 INFO:tasks.workunit.client.1.smithi130.stdout:5/992: truncate d6/d7/d95/f13f 652337 0 2024-05-13T22:48:45.459 INFO:tasks.workunit.client.1.smithi130.stdout:5/993: truncate d6/d7/f28 1554884 0 2024-05-13T22:48:45.474 INFO:tasks.workunit.client.1.smithi130.stdout:8/896: sync 2024-05-13T22:48:45.491 INFO:tasks.workunit.client.1.smithi130.stdout:8/897: mknod d1/d9b/df3/c144 0 2024-05-13T22:48:45.512 INFO:tasks.workunit.client.1.smithi130.stdout:1/984: sync 2024-05-13T22:48:45.547 INFO:tasks.workunit.client.1.smithi130.stdout:1/985: symlink db/d1b/da3/d49/dda/l132 0 2024-05-13T22:48:45.553 INFO:tasks.workunit.client.1.smithi130.stdout:8/898: mkdir d1/d2/db/d15/d18/d145 0 2024-05-13T22:48:45.553 INFO:tasks.workunit.client.1.smithi130.stdout:8/899: stat d1/d2/db/d15/d16/d54/d58/fd8 0 2024-05-13T22:48:45.554 INFO:tasks.workunit.client.1.smithi130.stdout:8/900: write d1/d2/db/d15/d16/d54/d58/f94 [2477125,76395] 0 2024-05-13T22:48:45.560 INFO:tasks.workunit.client.1.smithi130.stdout:8/901: write d1/d2/db/d15/d18/d42/d53/fd6 [784769,3208] 0 2024-05-13T22:48:45.561 INFO:tasks.workunit.client.1.smithi130.stdout:1/986: symlink db/d1b/da3/d2a/d35/dd4/d123/l133 0 2024-05-13T22:48:45.561 INFO:tasks.workunit.client.1.smithi130.stdout:1/987: stat db/d1b/da3/d49/d56/c6c 0 2024-05-13T22:48:45.567 INFO:tasks.workunit.client.1.smithi130.stdout:8/902: rename d1/d2/db/c6e to d1/d2/db/d15/d18/d42/d53/da4/d10e/c146 0 2024-05-13T22:48:45.569 INFO:tasks.workunit.client.1.smithi130.stdout:8/903: chown d1/d2/db/d15/d18/dba/d111/f114 298 1 2024-05-13T22:48:45.574 INFO:tasks.workunit.client.1.smithi130.stdout:8/904: link d1/d2/f135 d1/d2/db/d15/d7e/dc2/dcf/dda/f147 0 2024-05-13T22:48:45.575 INFO:tasks.workunit.client.1.smithi130.stdout:6/990: sync 2024-05-13T22:48:45.575 INFO:tasks.workunit.client.1.smithi130.stdout:1/988: dread db/d1b/da3/d2a/f5a [0,4194304] 0 2024-05-13T22:48:45.576 INFO:tasks.workunit.client.1.smithi130.stdout:6/991: write d4/ff4 [591557,98283] 0 2024-05-13T22:48:45.578 INFO:tasks.workunit.client.1.smithi130.stdout:8/905: write d1/d2/db/d15/d18/d1a/f9d [2490311,107138] 0 2024-05-13T22:48:45.582 INFO:tasks.workunit.client.1.smithi130.stdout:5/994: dwrite d6/d7/d19/d4d/fec [0,4194304] 0 2024-05-13T22:48:45.582 INFO:tasks.workunit.client.1.smithi130.stdout:8/906: chown d1/d2/db/d15/d16/d54/d125 1 1 2024-05-13T22:48:45.584 INFO:tasks.workunit.client.1.smithi130.stdout:6/992: truncate d4/d15/d1d/d60/fe0 908376 0 2024-05-13T22:48:45.587 INFO:tasks.workunit.client.1.smithi130.stdout:5/995: dwrite d6/d1a/d92/f14d [0,4194304] 0 2024-05-13T22:48:45.599 INFO:tasks.workunit.client.1.smithi130.stdout:1/989: dwrite db/d1b/da3/d2a/d38/d46/d6d/f9e [0,4194304] 0 2024-05-13T22:48:45.601 INFO:tasks.workunit.client.1.smithi130.stdout:5/996: creat d6/d7/f159 x:0 0 0 2024-05-13T22:48:45.608 INFO:tasks.workunit.client.1.smithi130.stdout:1/990: mknod db/d1b/da3/d2a/d35/c134 0 2024-05-13T22:48:45.609 INFO:tasks.workunit.client.1.smithi130.stdout:5/997: unlink d6/d7/d19/d4d/d6a/f135 0 2024-05-13T22:48:45.612 INFO:tasks.workunit.client.1.smithi130.stdout:1/991: rmdir db/d1b/da3/d72/daa 39 2024-05-13T22:48:45.612 INFO:tasks.workunit.client.1.smithi130.stdout:1/992: stat db/d1b/da3/d72/cfe 0 2024-05-13T22:48:45.614 INFO:tasks.workunit.client.1.smithi130.stdout:5/998: mknod d6/d7/d19/d4d/d53/c15a 0 2024-05-13T22:48:45.614 INFO:tasks.workunit.client.1.smithi130.stdout:6/993: getdents d4/d15/d1d/d60/da2/da4/dac 0 2024-05-13T22:48:45.618 INFO:tasks.workunit.client.1.smithi130.stdout:5/999: rmdir d6/dd/d18/d6e/d82/d4b/d122 39 2024-05-13T22:48:45.627 INFO:tasks.workunit.client.1.smithi130.stdout:1/993: truncate db/d82/f10c 1553567 0 2024-05-13T22:48:45.630 INFO:tasks.workunit.client.1.smithi130.stdout:1/994: creat db/d1b/da3/d49/d56/d7f/d107/f135 x:0 0 0 2024-05-13T22:48:45.632 INFO:tasks.workunit.client.1.smithi130.stdout:6/994: dwrite d4/d15/d1f/fda [0,4194304] 0 2024-05-13T22:48:45.633 INFO:tasks.workunit.client.1.smithi130.stdout:1/995: truncate db/d1b/da3/d2a/d4c/dc2/f111 991347 0 2024-05-13T22:48:45.644 INFO:tasks.workunit.client.1.smithi130.stdout:8/907: sync 2024-05-13T22:48:45.647 INFO:tasks.workunit.client.1.smithi130.stdout:6/995: dwrite d4/d15/d1f/d46/d50/f79 [0,4194304] 0 2024-05-13T22:48:45.648 INFO:tasks.workunit.client.1.smithi130.stdout:1/996: dwrite db/d1b/da3/fce [4194304,4194304] 0 2024-05-13T22:48:45.651 INFO:tasks.workunit.client.1.smithi130.stdout:8/908: unlink d1/d9b/fb5 0 2024-05-13T22:48:45.658 INFO:tasks.workunit.client.1.smithi130.stdout:6/996: mkdir d4/d15/d1d/d28/d4e/d9e/d146 0 2024-05-13T22:48:45.663 INFO:tasks.workunit.client.1.smithi130.stdout:1/997: mkdir db/d1b/da3/d2a/d136 0 2024-05-13T22:48:45.674 INFO:tasks.workunit.client.1.smithi130.stdout:8/909: getdents d1/d2/db/d15/d16/d54/d58/d120 0 2024-05-13T22:48:45.681 INFO:tasks.workunit.client.1.smithi130.stdout:8/910: unlink d1/d2/f12e 0 2024-05-13T22:48:45.684 INFO:tasks.workunit.client.1.smithi130.stdout:6/997: dwrite d4/d15/d1f/d3c/f96 [0,4194304] 0 2024-05-13T22:48:45.692 INFO:tasks.workunit.client.1.smithi130.stdout:8/911: dwrite d1/d2/db/d15/d7e/d9f/fc9 [0,4194304] 0 2024-05-13T22:48:45.694 INFO:tasks.workunit.client.1.smithi130.stdout:6/998: dwrite d4/d15/d1f/d3c/f11d [0,4194304] 0 2024-05-13T22:48:45.707 INFO:tasks.workunit.client.1.smithi130.stdout:6/999: getdents d4/d15/d1d/daa 0 2024-05-13T22:48:45.714 INFO:tasks.workunit.client.1.smithi130.stdout:8/912: write d1/d2/db/d15/d16/d54/dfb/ffd [3131715,93391] 0 2024-05-13T22:48:45.717 INFO:tasks.workunit.client.1.smithi130.stdout:8/913: creat d1/d2/db/d15/d7e/f148 x:0 0 0 2024-05-13T22:48:45.718 INFO:tasks.workunit.client.1.smithi130.stdout:1/998: sync 2024-05-13T22:48:45.719 INFO:tasks.workunit.client.1.smithi130.stdout:8/914: creat d1/d2/db/d15/d137/f149 x:0 0 0 2024-05-13T22:48:45.722 INFO:tasks.workunit.client.1.smithi130.stdout:8/915: mknod d1/d2/db/d15/d7e/dcd/c14a 0 2024-05-13T22:48:45.725 INFO:tasks.workunit.client.1.smithi130.stdout:8/916: creat d1/d2/df8/f14b x:0 0 0 2024-05-13T22:48:45.728 INFO:tasks.workunit.client.1.smithi130.stdout:1/999: link db/d1b/da3/d49/d56/d88/cae db/d1b/da3/c137 0 2024-05-13T22:48:45.728 INFO:tasks.workunit.client.1.smithi130.stdout:8/917: symlink d1/d2/db/d15/d16/d54/d125/l14c 0 2024-05-13T22:48:45.729 INFO:tasks.workunit.client.1.smithi130.stdout:8/918: truncate d1/d9b/f115 200334 0 2024-05-13T22:48:45.730 INFO:tasks.workunit.client.1.smithi130.stdout:8/919: dread - d1/d2/db/d15/fc0 zero size 2024-05-13T22:48:45.731 INFO:tasks.workunit.client.1.smithi130.stdout:8/920: symlink d1/d2/db/d15/d18/d42/d53/d56/l14d 0 2024-05-13T22:48:45.734 INFO:tasks.workunit.client.1.smithi130.stdout:8/921: truncate d1/dc/f44 1343770 0 2024-05-13T22:48:45.741 INFO:tasks.workunit.client.1.smithi130.stdout:8/922: dwrite d1/d2/db/d15/f22 [0,4194304] 0 2024-05-13T22:48:45.770 INFO:tasks.workunit.client.1.smithi130.stdout:8/923: dread d1/d2/db/d15/d7e/dcd/f123 [0,4194304] 0 2024-05-13T22:48:45.798 INFO:tasks.workunit.client.1.smithi130.stdout:8/924: dwrite d1/d2/db/d15/d7e/d9f/fa0 [0,4194304] 0 2024-05-13T22:48:45.813 INFO:tasks.workunit.client.1.smithi130.stdout:8/925: write d1/d2/fa7 [275268,41972] 0 2024-05-13T22:48:45.817 INFO:tasks.workunit.client.1.smithi130.stdout:8/926: write d1/d2/f39 [246405,130748] 0 2024-05-13T22:48:45.818 INFO:tasks.workunit.client.1.smithi130.stdout:8/927: dread - d1/d9b/f142 zero size 2024-05-13T22:48:45.819 INFO:tasks.workunit.client.1.smithi130.stdout:8/928: dread - d1/d9b/f142 zero size 2024-05-13T22:48:45.820 INFO:tasks.workunit.client.1.smithi130.stdout:8/929: stat d1/d2/db/d15/d18/dba/d107/d12c/caf 0 2024-05-13T22:48:45.825 INFO:tasks.workunit.client.1.smithi130.stdout:8/930: dwrite d1/d2/db/d15/d18/dba/fef [0,4194304] 0 2024-05-13T22:48:45.833 INFO:tasks.workunit.client.1.smithi130.stdout:8/931: dwrite d1/d2/db/d15/d7e/dc2/dcf/dda/f118 [0,4194304] 0 2024-05-13T22:48:45.839 INFO:tasks.workunit.client.1.smithi130.stdout:8/932: unlink d1/dc/cfa 0 2024-05-13T22:48:45.843 INFO:tasks.workunit.client.1.smithi130.stdout:8/933: symlink d1/d2/db/d15/d7e/dc2/dcf/l14e 0 2024-05-13T22:48:45.850 INFO:tasks.workunit.client.1.smithi130.stdout:8/934: getdents d1 0 2024-05-13T22:48:45.856 INFO:tasks.workunit.client.1.smithi130.stdout:8/935: link d1/d9b/f115 d1/d2/db/d15/d16/d54/d58/db9/f14f 0 2024-05-13T22:48:45.858 INFO:tasks.workunit.client.1.smithi130.stdout:8/936: fsync d1/d2/db/d15/d18/dba/d107/d12c/fe3 0 2024-05-13T22:48:45.863 INFO:tasks.workunit.client.1.smithi130.stdout:8/937: rename d1/d2/db/d15/d18/d42/d53/d56/f9a to d1/d2/db/d15/d7e/dc2/dcf/dda/d117/f150 0 2024-05-13T22:48:45.866 INFO:tasks.workunit.client.1.smithi130.stdout:8/938: unlink d1/d2/db/d15/d18/d1a/f9d 0 2024-05-13T22:48:45.872 INFO:tasks.workunit.client.1.smithi130.stdout:8/939: getdents d1/d2/db/d15/d18/d42/d53/dd0 0 2024-05-13T22:48:45.878 INFO:tasks.workunit.client.1.smithi130.stdout:8/940: rename d1/d2/db/d15/d7e/dc2/dcf/l14e to d1/d2/db/d15/d16/d54/d58/l151 0 2024-05-13T22:48:45.881 INFO:tasks.workunit.client.1.smithi130.stdout:8/941: creat d1/d2/db/d15/d18/d42/d53/da4/d10e/f152 x:0 0 0 2024-05-13T22:48:45.883 INFO:tasks.workunit.client.1.smithi130.stdout:8/942: chown d1/d2/db/d15/d7e/dc2/le9 377 1 2024-05-13T22:48:45.886 INFO:tasks.workunit.client.1.smithi130.stdout:8/943: mkdir d1/d2/db/d15/d18/d42/d53/de8/d10f/d153 0 2024-05-13T22:48:45.889 INFO:tasks.workunit.client.1.smithi130.stdout:8/944: mknod d1/d2/db/d15/d18/d42/c154 0 2024-05-13T22:48:45.890 INFO:tasks.workunit.client.1.smithi130.stdout:8/945: chown d1/d2/db/d15/d18/d42/d53/dd0/f11d 773024468 1 2024-05-13T22:48:45.896 INFO:tasks.workunit.client.1.smithi130.stdout:8/946: dread d1/d2/db/d15/d7e/f95 [0,4194304] 0 2024-05-13T22:48:45.898 INFO:tasks.workunit.client.1.smithi130.stdout:8/947: symlink d1/d2/db/d15/d18/d145/l155 0 2024-05-13T22:48:45.900 INFO:tasks.workunit.client.1.smithi130.stdout:8/948: write d1/d2/db/d15/d18/d42/d53/dd0/f13e [1020656,119373] 0 2024-05-13T22:48:45.901 INFO:tasks.workunit.client.1.smithi130.stdout:8/949: rmdir d1/d2/db/d15/d7e/dcd 39 2024-05-13T22:48:45.902 INFO:tasks.workunit.client.1.smithi130.stdout:8/950: chown d1/d2/df8 99408365 1 2024-05-13T22:48:45.905 INFO:tasks.workunit.client.1.smithi130.stdout:8/951: creat d1/dc/d5d/dde/d13c/f156 x:0 0 0 2024-05-13T22:48:45.910 INFO:tasks.workunit.client.1.smithi130.stdout:8/952: dwrite d1/d2/db/d15/d16/fb1 [0,4194304] 0 2024-05-13T22:48:45.912 INFO:tasks.workunit.client.1.smithi130.stdout:8/953: mkdir d1/dc/d5d/dde/d157 0 2024-05-13T22:48:45.915 INFO:tasks.workunit.client.1.smithi130.stdout:8/954: symlink d1/d2/db/d15/d16/d54/dfb/d110/l158 0 2024-05-13T22:48:45.916 INFO:tasks.workunit.client.1.smithi130.stdout:8/955: write d1/d2/db/d15/d7e/d9f/fa0 [3635524,94570] 0 2024-05-13T22:48:45.919 INFO:tasks.workunit.client.1.smithi130.stdout:8/956: getdents d1/d2/db/d15/d18/d42/d53/de8/d10f/d153 0 2024-05-13T22:48:45.924 INFO:tasks.workunit.client.1.smithi130.stdout:8/957: mknod d1/d2/db/d15/d18/d42/d53/da4/c159 0 2024-05-13T22:48:45.925 INFO:tasks.workunit.client.1.smithi130.stdout:8/958: read d1/d2/db/d15/d18/d42/d53/fd6 [396434,104417] 0 2024-05-13T22:48:45.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:45 smithi102 ceph-mon[28599]: from='client.14698 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:45.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:45 smithi102 ceph-mon[28599]: pgmap v40: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 67 MiB/s rd, 148 MiB/s wr, 357 op/s 2024-05-13T22:48:45.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:45 smithi102 ceph-mon[28599]: from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:45.931 INFO:tasks.workunit.client.1.smithi130.stdout:8/959: dwrite d1/d2/db/d15/d7e/d9f/fc9 [0,4194304] 0 2024-05-13T22:48:45.937 INFO:tasks.workunit.client.1.smithi130.stdout:8/960: mknod d1/d2/db/d15/d18/d42/d53/de8/d10f/d153/c15a 0 2024-05-13T22:48:45.956 INFO:tasks.workunit.client.1.smithi130.stdout:8/961: dread d1/d2/db/d15/d16/d54/dfb/ffd [0,4194304] 0 2024-05-13T22:48:45.959 INFO:tasks.workunit.client.1.smithi130.stdout:8/962: unlink d1/d2/db/d15/d18/d42/c154 0 2024-05-13T22:48:45.960 INFO:tasks.workunit.client.1.smithi130.stdout:8/963: dread - d1/d2/db/d15/d18/f13b zero size 2024-05-13T22:48:46.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:45 smithi130 ceph-mon[36591]: from='client.14698 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:46.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:45 smithi130 ceph-mon[36591]: pgmap v40: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 67 MiB/s rd, 148 MiB/s wr, 357 op/s 2024-05-13T22:48:46.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:45 smithi130 ceph-mon[36591]: from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:48:46.152 INFO:tasks.workunit.client.1.smithi130.stdout:8/964: rename d1/d9b/f115 to d1/d2/db/d15/d18/d42/f15b 0 2024-05-13T22:48:46.153 INFO:tasks.workunit.client.1.smithi130.stdout:8/965: chown d1/d2/db/d15/d18/d42/d53/dd0/l10b 14 1 2024-05-13T22:48:46.154 INFO:tasks.workunit.client.1.smithi130.stdout:8/966: fsync d1/d2/df8/f14b 0 2024-05-13T22:48:46.160 INFO:tasks.workunit.client.1.smithi130.stdout:8/967: dwrite d1/d2/db/d15/d7e/dcd/f10c [0,4194304] 0 2024-05-13T22:48:46.162 INFO:tasks.workunit.client.1.smithi130.stdout:8/968: symlink d1/d2/db/d15/d18/d42/d53/da4/l15c 0 2024-05-13T22:48:46.163 INFO:tasks.workunit.client.1.smithi130.stdout:8/969: dread - d1/d2/db/d15/d7e/f121 zero size 2024-05-13T22:48:46.167 INFO:tasks.workunit.client.1.smithi130.stdout:8/970: mkdir d1/d2/db/d15/d16/d54/d58/d120/d15d 0 2024-05-13T22:48:46.168 INFO:tasks.workunit.client.1.smithi130.stdout:8/971: dread - d1/dc/d5d/dde/d13c/f156 zero size 2024-05-13T22:48:46.171 INFO:tasks.workunit.client.1.smithi130.stdout:8/972: mknod d1/d2/db/d15/d18/d1a/c15e 0 2024-05-13T22:48:46.172 INFO:tasks.workunit.client.1.smithi130.stdout:8/973: read d1/d2/db/d15/d7e/dcd/f123 [2751522,60461] 0 2024-05-13T22:48:46.174 INFO:tasks.workunit.client.1.smithi130.stdout:8/974: mkdir d1/dc/d5d/dde/d15f 0 2024-05-13T22:48:46.184 INFO:tasks.workunit.client.1.smithi130.stdout:8/975: write d1/d2/db/d15/d18/d42/d53/fd6 [1822851,34719] 0 2024-05-13T22:48:46.189 INFO:tasks.workunit.client.1.smithi130.stdout:8/976: dread d1/dc/f44 [0,4194304] 0 2024-05-13T22:48:46.200 INFO:tasks.workunit.client.1.smithi130.stdout:8/977: dread d1/d2/db/f8 [4194304,4194304] 0 2024-05-13T22:48:46.205 INFO:tasks.workunit.client.1.smithi130.stdout:8/978: mkdir d1/d2/db/d15/d18/df2/d160 0 2024-05-13T22:48:46.205 INFO:tasks.workunit.client.1.smithi130.stdout:8/979: chown d1/f10 1388 1 2024-05-13T22:48:46.208 INFO:tasks.workunit.client.1.smithi130.stdout:8/980: mkdir d1/d2/db/d15/d7e/dc2/dcf/dda/d117/d161 0 2024-05-13T22:48:46.213 INFO:tasks.workunit.client.1.smithi130.stdout:8/981: dwrite d1/d2/db/d15/d16/f88 [0,4194304] 0 2024-05-13T22:48:46.216 INFO:tasks.workunit.client.1.smithi130.stdout:8/982: chown d1/d2/db/d15/d18/f63 84 1 2024-05-13T22:48:46.221 INFO:tasks.workunit.client.1.smithi130.stdout:8/983: unlink d1/d2/db/d15/d18/d42/d53/de8/c108 0 2024-05-13T22:48:46.224 INFO:tasks.workunit.client.1.smithi130.stdout:8/984: rmdir d1/d2/db/d15/d7e/dc2/dcf/dda 39 2024-05-13T22:48:46.224 INFO:tasks.workunit.client.1.smithi130.stdout:8/985: readlink d1/d2/db/d15/d16/d54/d125/l14c 0 2024-05-13T22:48:46.226 INFO:tasks.workunit.client.1.smithi130.stdout:8/986: read d1/d2/db/d15/d18/d42/d53/dd0/fd3 [251327,106434] 0 2024-05-13T22:48:46.229 INFO:tasks.workunit.client.1.smithi130.stdout:8/987: rename d1/d9b/df3/dff to d1/d2/db/d15/d18/d145/d162 0 2024-05-13T22:48:46.238 INFO:tasks.workunit.client.1.smithi130.stdout:8/988: truncate d1/d2/db/d15/d18/f60 1104576 0 2024-05-13T22:48:46.241 INFO:tasks.workunit.client.1.smithi130.stdout:8/989: mknod d1/d2/db/d15/d18/d42/d53/da4/d10e/c163 0 2024-05-13T22:48:46.243 INFO:tasks.workunit.client.1.smithi130.stdout:8/990: fdatasync d1/d2/db/d15/d18/f4c 0 2024-05-13T22:48:46.258 INFO:tasks.workunit.client.1.smithi130.stdout:8/991: dwrite d1/dc/d5d/fee [0,4194304] 0 2024-05-13T22:48:46.268 INFO:tasks.workunit.client.1.smithi130.stdout:8/992: dwrite d1/d2/db/d15/d18/f31 [0,4194304] 0 2024-05-13T22:48:46.274 INFO:tasks.workunit.client.1.smithi130.stdout:8/993: rename d1/d2/db/d15/d7e/d9f/fa0 to d1/d2/f164 0 2024-05-13T22:48:46.288 INFO:tasks.workunit.client.1.smithi130.stdout:8/994: dwrite d1/d2/db/d15/d16/d54/d58/f98 [0,4194304] 0 2024-05-13T22:48:46.296 INFO:tasks.workunit.client.1.smithi130.stdout:8/995: creat d1/d2/db/d15/f165 x:0 0 0 2024-05-13T22:48:46.297 INFO:tasks.workunit.client.1.smithi130.stdout:8/996: fsync d1/d2/db/d15/d18/d42/d53/fd6 0 2024-05-13T22:48:46.300 INFO:tasks.workunit.client.1.smithi130.stdout:8/997: creat d1/d2/db/d15/d16/f166 x:0 0 0 2024-05-13T22:48:46.301 INFO:tasks.workunit.client.1.smithi130.stdout:8/998: dread - d1/d2/db/d15/d18/f13b zero size 2024-05-13T22:48:46.306 INFO:tasks.workunit.client.1.smithi130.stdout:8/999: creat d1/d2/db/d15/d18/dfc/f167 x:0 0 0 2024-05-13T22:48:46.309 INFO:tasks.workunit.client.1.smithi130.stderr:+ rm -rf -- ./tmp.zP6VXwHdJc 2024-05-13T22:48:47.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:46 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:48:47.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:46 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:48:47.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:47 smithi102 ceph-mon[28599]: Standby manager daemon smithi102.xcijrj restarted 2024-05-13T22:48:47.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:47 smithi102 ceph-mon[28599]: Standby manager daemon smithi102.xcijrj started 2024-05-13T22:48:47.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:47 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi102.xcijrj/crt"}]: dispatch 2024-05-13T22:48:47.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:47 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:48:47.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:47 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi102.xcijrj/key"}]: dispatch 2024-05-13T22:48:47.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:47 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:48:47.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:47 smithi102 ceph-mon[28599]: pgmap v41: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 76 MiB/s rd, 167 MiB/s wr, 389 op/s 2024-05-13T22:48:48.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:47 smithi130 ceph-mon[36591]: Standby manager daemon smithi102.xcijrj restarted 2024-05-13T22:48:48.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:47 smithi130 ceph-mon[36591]: Standby manager daemon smithi102.xcijrj started 2024-05-13T22:48:48.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:47 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi102.xcijrj/crt"}]: dispatch 2024-05-13T22:48:48.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:47 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:48:48.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:47 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi102.xcijrj/key"}]: dispatch 2024-05-13T22:48:48.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:47 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:48:48.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:47 smithi130 ceph-mon[36591]: pgmap v41: 65 pgs: 65 active+clean; 3.5 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 76 MiB/s rd, 167 MiB/s wr, 389 op/s 2024-05-13T22:48:48.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:48 smithi130 ceph-mon[36591]: mgrmap e25: smithi130.uoqoxl(active, since 76s), standbys: smithi102.xcijrj 2024-05-13T22:48:49.103 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:48 smithi102 ceph-mon[28599]: mgrmap e25: smithi130.uoqoxl(active, since 76s), standbys: smithi102.xcijrj 2024-05-13T22:48:50.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:50.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:50.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:48:50.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:48:50.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:50.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:48:50.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mgr fail", "who": "smithi130.uoqoxl"}]: dispatch 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mgr fail", "who": "smithi130.uoqoxl"}]: dispatch 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: osdmap e49: 6 total, 6 up, 6 in 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd='[{"prefix": "mgr fail", "who": "smithi130.uoqoxl"}]': finished 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: mgrmap e26: smithi102.xcijrj(active, starting, since 0.00588225s) 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi102.xcijrj", "id": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:48:50.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:48:50.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:48:50.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:48:50.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:48:50.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:48:50.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-13T22:48:50.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-13T22:48:50.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-13T22:48:50.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: Manager daemon smithi102.xcijrj is now available 2024-05-13T22:48:50.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:48:50.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:48:50.431 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:48:50.431 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/trash_purge_schedule"}]: dispatch 2024-05-13T22:48:50.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:50.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:50.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:48:50.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:48:50.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' 2024-05-13T22:48:50.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:48:50.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24369 172.21.15.130:0/3265249303' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mgr fail", "who": "smithi130.uoqoxl"}]: dispatch 2024-05-13T22:48:50.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "mgr fail", "who": "smithi130.uoqoxl"}]: dispatch 2024-05-13T22:48:50.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: osdmap e49: 6 total, 6 up, 6 in 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24369 ' entity='mgr.smithi130.uoqoxl' cmd='[{"prefix": "mgr fail", "who": "smithi130.uoqoxl"}]': finished 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: mgrmap e26: smithi102.xcijrj(active, starting, since 0.00588225s) 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi102.xcijrj", "id": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:48:50.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:48:50.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:48:50.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:48:50.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-13T22:48:50.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-13T22:48:50.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-13T22:48:50.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: Manager daemon smithi102.xcijrj is now available 2024-05-13T22:48:50.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:48:50.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:48:50.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:48:50.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/trash_purge_schedule"}]: dispatch 2024-05-13T22:48:51.642 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:51 smithi130 ceph-mon[36591]: mgrmap e27: smithi102.xcijrj(active, since 1.00934s) 2024-05-13T22:48:51.642 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:51 smithi130 ceph-mon[36591]: [13/May/2024:22:48:50] ENGINE Bus STARTING 2024-05-13T22:48:51.642 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:51 smithi130 ceph-mon[36591]: [13/May/2024:22:48:50] ENGINE Serving on https://172.21.15.102:7150 2024-05-13T22:48:51.642 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:51 smithi130 ceph-mon[36591]: [13/May/2024:22:48:50] ENGINE Serving on http://172.21.15.102:8765 2024-05-13T22:48:51.642 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:51 smithi130 ceph-mon[36591]: [13/May/2024:22:48:50] ENGINE Bus STARTED 2024-05-13T22:48:51.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:51 smithi102 ceph-mon[28599]: mgrmap e27: smithi102.xcijrj(active, since 1.00934s) 2024-05-13T22:48:51.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:51 smithi102 ceph-mon[28599]: [13/May/2024:22:48:50] ENGINE Bus STARTING 2024-05-13T22:48:51.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:51 smithi102 ceph-mon[28599]: [13/May/2024:22:48:50] ENGINE Serving on https://172.21.15.102:7150 2024-05-13T22:48:51.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:51 smithi102 ceph-mon[28599]: [13/May/2024:22:48:50] ENGINE Serving on http://172.21.15.102:8765 2024-05-13T22:48:51.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:51 smithi102 ceph-mon[28599]: [13/May/2024:22:48:50] ENGINE Bus STARTED 2024-05-13T22:48:52.609 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:52 smithi130 ceph-mon[36591]: pgmap v4: 65 pgs: 65 active+clean; 3.1 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail 2024-05-13T22:48:52.609 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:52 smithi130 ceph-mon[36591]: mgrmap e28: smithi102.xcijrj(active, since 2s) 2024-05-13T22:48:52.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:52 smithi102 ceph-mon[28599]: pgmap v4: 65 pgs: 65 active+clean; 3.1 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail 2024-05-13T22:48:52.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:52 smithi102 ceph-mon[28599]: mgrmap e28: smithi102.xcijrj(active, since 2s) 2024-05-13T22:48:54.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:54 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:48:54.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:54 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:48:54.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:54 smithi130 ceph-mon[36591]: mgrmap e29: smithi102.xcijrj(active, since 4s) 2024-05-13T22:48:54.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:54 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:48:54.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:54 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:48:54.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:54 smithi102 ceph-mon[28599]: mgrmap e29: smithi102.xcijrj(active, since 4s) 2024-05-13T22:48:55.386 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:55 smithi130 ceph-mon[36591]: pgmap v5: 65 pgs: 65 active+clean; 3.1 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail 2024-05-13T22:48:55.386 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:55 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:48:55.386 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:55 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:48:55.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:55 smithi102 ceph-mon[28599]: pgmap v5: 65 pgs: 65 active+clean; 3.1 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail 2024-05-13T22:48:55.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:55 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:48:55.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:55 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:48:56.578 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:56 smithi102 ceph-mon[28599]: pgmap v6: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 42 KiB/s rd, 2.2 KiB/s wr, 53 op/s 2024-05-13T22:48:56.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:56 smithi130 ceph-mon[36591]: pgmap v6: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 42 KiB/s rd, 2.2 KiB/s wr, 53 op/s 2024-05-13T22:48:57.124 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-13T22:48:57.124 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-13T22:48:57.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:57 smithi130 ceph-mon[36591]: Standby manager daemon smithi130.uoqoxl started 2024-05-13T22:48:57.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:57 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/256583460' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/crt"}]: dispatch 2024-05-13T22:48:57.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:57 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/256583460' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:48:57.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:57 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/256583460' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/key"}]: dispatch 2024-05-13T22:48:57.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:57 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/256583460' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:48:57.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:57 smithi102 ceph-mon[28599]: Standby manager daemon smithi130.uoqoxl started 2024-05-13T22:48:57.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:57 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/256583460' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/crt"}]: dispatch 2024-05-13T22:48:57.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:57 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/256583460' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:48:57.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:57 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/256583460' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/key"}]: dispatch 2024-05-13T22:48:57.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:57 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/256583460' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:48:58.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:58 smithi130 ceph-mon[36591]: pgmap v7: 65 pgs: 65 active+clean; 1.9 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 753 KiB/s rd, 834 KiB/s wr, 144 op/s 2024-05-13T22:48:58.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:58 smithi130 ceph-mon[36591]: mgrmap e30: smithi102.xcijrj(active, since 8s), standbys: smithi130.uoqoxl 2024-05-13T22:48:58.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:48:58 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi130.uoqoxl", "id": "smithi130.uoqoxl"}]: dispatch 2024-05-13T22:48:58.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:58 smithi102 ceph-mon[28599]: pgmap v7: 65 pgs: 65 active+clean; 1.9 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 753 KiB/s rd, 834 KiB/s wr, 144 op/s 2024-05-13T22:48:58.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:58 smithi102 ceph-mon[28599]: mgrmap e30: smithi102.xcijrj(active, since 8s), standbys: smithi130.uoqoxl 2024-05-13T22:48:58.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:48:58 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi130.uoqoxl", "id": "smithi130.uoqoxl"}]: dispatch 2024-05-13T22:49:00.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:00 smithi130 ceph-mon[36591]: pgmap v8: 65 pgs: 65 active+clean; 1.9 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 586 KiB/s rd, 648 KiB/s wr, 112 op/s 2024-05-13T22:49:00.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:00 smithi102 ceph-mon[28599]: pgmap v8: 65 pgs: 65 active+clean; 1.9 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 586 KiB/s rd, 648 KiB/s wr, 112 op/s 2024-05-13T22:49:02.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:02 smithi130 ceph-mon[36591]: pgmap v9: 65 pgs: 65 active+clean; 1001 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-05-13T22:49:02.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:02 smithi102 ceph-mon[28599]: pgmap v9: 65 pgs: 65 active+clean; 1001 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-05-13T22:49:04.619 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:04 smithi130 ceph-mon[36591]: pgmap v10: 65 pgs: 65 active+clean; 1001 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 977 KiB/s rd, 998 KiB/s wr, 156 op/s 2024-05-13T22:49:04.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:04 smithi102 ceph-mon[28599]: pgmap v10: 65 pgs: 65 active+clean; 1001 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 977 KiB/s rd, 998 KiB/s wr, 156 op/s 2024-05-13T22:49:05.609 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:05 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:49:05.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:05 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:49:05.972 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-13T22:49:05.972 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-05-13T22:49:06.622 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:06 smithi102 ceph-mon[28599]: pgmap v11: 65 pgs: 65 active+clean; 812 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 213 op/s 2024-05-13T22:49:06.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:06 smithi130 ceph-mon[36591]: pgmap v11: 65 pgs: 65 active+clean; 812 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 213 op/s 2024-05-13T22:49:08.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:08 smithi130 ceph-mon[36591]: pgmap v12: 65 pgs: 65 active+clean; 570 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 257 op/s 2024-05-13T22:49:08.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:08 smithi102 ceph-mon[28599]: pgmap v12: 65 pgs: 65 active+clean; 570 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 257 op/s 2024-05-13T22:49:10.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:10 smithi130 ceph-mon[36591]: pgmap v13: 65 pgs: 65 active+clean; 570 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1017 KiB/s rd, 1.1 MiB/s wr, 195 op/s 2024-05-13T22:49:10.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:10 smithi102 ceph-mon[28599]: pgmap v13: 65 pgs: 65 active+clean; 570 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1017 KiB/s rd, 1.1 MiB/s wr, 195 op/s 2024-05-13T22:49:13.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:12 smithi130 ceph-mon[36591]: pgmap v14: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 304 op/s 2024-05-13T22:49:13.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:12 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:13.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:12 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:13.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:12 smithi102 ceph-mon[28599]: pgmap v14: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 304 op/s 2024-05-13T22:49:13.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:12 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:13.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:12 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:14.440 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:14 smithi102 ceph-mon[28599]: pgmap v15: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 231 op/s 2024-05-13T22:49:14.441 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:14 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:14.441 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:14 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:14.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:14 smithi130 ceph-mon[36591]: pgmap v15: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 231 op/s 2024-05-13T22:49:14.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:14 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:14.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:14 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.400+0000 7f1c37c0d700 1 -- 172.21.15.102:0/2260099889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c30100cb0 msgr2=0x7f1c30101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.400+0000 7f1c37c0d700 1 --2- 172.21.15.102:0/2260099889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c30100cb0 0x7f1c30101110 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009910 tx=0x7f1c20009c20 comp rx=0 tx=0).stop 2024-05-13T22:49:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.400+0000 7f1c37c0d700 1 -- 172.21.15.102:0/2260099889 shutdown_connections 2024-05-13T22:49:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.400+0000 7f1c37c0d700 1 --2- 172.21.15.102:0/2260099889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c30100cb0 0x7f1c30101110 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.400+0000 7f1c37c0d700 1 --2- 172.21.15.102:0/2260099889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c300ffab0 0x7f1c300ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.400+0000 7f1c37c0d700 1 -- 172.21.15.102:0/2260099889 >> 172.21.15.102:0/2260099889 conn(0x7f1c300fb030 msgr2=0x7f1c300fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:15.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.400+0000 7f1c37c0d700 1 -- 172.21.15.102:0/2260099889 shutdown_connections 2024-05-13T22:49:15.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.400+0000 7f1c37c0d700 1 -- 172.21.15.102:0/2260099889 wait complete. 2024-05-13T22:49:15.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.401+0000 7f1c37c0d700 1 Processor -- start 2024-05-13T22:49:15.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.401+0000 7f1c37c0d700 1 -- start start 2024-05-13T22:49:15.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.401+0000 7f1c37c0d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c300ffab0 0x7f1c30198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:15.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.401+0000 7f1c37c0d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c30100cb0 0x7f1c30198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:15.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.401+0000 7f1c37c0d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c30199090 con 0x7f1c30100cb0 2024-05-13T22:49:15.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.401+0000 7f1c37c0d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c301991d0 con 0x7f1c300ffab0 2024-05-13T22:49:15.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.401+0000 7f1c351a8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c30100cb0 0x7f1c30198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:15.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.401+0000 7f1c351a8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c30100cb0 0x7f1c30198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53482/0 (socket says 172.21.15.102:53482) 2024-05-13T22:49:15.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.401+0000 7f1c351a8700 1 -- 172.21.15.102:0/1141530543 learned_addr learned my addr 172.21.15.102:0/1141530543 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:49:15.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.402+0000 7f1c359a9700 1 --2- 172.21.15.102:0/1141530543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c300ffab0 0x7f1c30198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:15.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.402+0000 7f1c359a9700 1 -- 172.21.15.102:0/1141530543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c30100cb0 msgr2=0x7f1c30198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:15.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.402+0000 7f1c359a9700 1 --2- 172.21.15.102:0/1141530543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c30100cb0 0x7f1c30198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:15.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.402+0000 7f1c359a9700 1 -- 172.21.15.102:0/1141530543 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200095f0 con 0x7f1c300ffab0 2024-05-13T22:49:15.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.402+0000 7f1c359a9700 1 --2- 172.21.15.102:0/1141530543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c300ffab0 0x7f1c30198530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00e940 tx=0x7f1c2c00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:15.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.403+0000 7f1c26ffd700 1 -- 172.21.15.102:0/1141530543 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c2c0041f0 con 0x7f1c300ffab0 2024-05-13T22:49:15.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.403+0000 7f1c26ffd700 1 -- 172.21.15.102:0/1141530543 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1c2c004350 con 0x7f1c300ffab0 2024-05-13T22:49:15.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.403+0000 7f1c26ffd700 1 -- 172.21.15.102:0/1141530543 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c2c003c80 con 0x7f1c300ffab0 2024-05-13T22:49:15.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.403+0000 7f1c37c0d700 1 -- 172.21.15.102:0/1141530543 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3019dc80 con 0x7f1c300ffab0 2024-05-13T22:49:15.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.403+0000 7f1c37c0d700 1 -- 172.21.15.102:0/1141530543 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3019e180 con 0x7f1c300ffab0 2024-05-13T22:49:15.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.405+0000 7f1c37c0d700 1 -- 172.21.15.102:0/1141530543 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c30107500 con 0x7f1c300ffab0 2024-05-13T22:49:15.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.406+0000 7f1c26ffd700 1 -- 172.21.15.102:0/1141530543 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 30) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f1c2c00c7c0 con 0x7f1c300ffab0 2024-05-13T22:49:15.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.406+0000 7f1c26ffd700 1 --2- 172.21.15.102:0/1141530543 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1c1c06dfa0 0x7f1c1c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:15.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.406+0000 7f1c26ffd700 1 -- 172.21.15.102:0/1141530543 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1c2c014070 con 0x7f1c300ffab0 2024-05-13T22:49:15.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.407+0000 7f1c351a8700 1 --2- 172.21.15.102:0/1141530543 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1c1c06dfa0 0x7f1c1c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:15.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.407+0000 7f1c351a8700 1 --2- 172.21.15.102:0/1141530543 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1c1c06dfa0 0x7f1c1c070460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006010 tx=0x7f1c20005940 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.411+0000 7f1c26ffd700 1 -- 172.21.15.102:0/1141530543 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f1c2c05b280 con 0x7f1c300ffab0 2024-05-13T22:49:15.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.732+0000 7f1c37c0d700 1 -- 172.21.15.102:0/1141530543 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c30105600 con 0x7f1c1c06dfa0 2024-05-13T22:49:15.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.734+0000 7f1c26ffd700 1 -- 172.21.15.102:0/1141530543 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f1c30105600 con 0x7f1c1c06dfa0 2024-05-13T22:49:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.738+0000 7f1c37c0d700 1 -- 172.21.15.102:0/1141530543 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1c1c06dfa0 msgr2=0x7f1c1c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.738+0000 7f1c37c0d700 1 --2- 172.21.15.102:0/1141530543 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1c1c06dfa0 0x7f1c1c070460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006010 tx=0x7f1c20005940 comp rx=0 tx=0).stop 2024-05-13T22:49:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.738+0000 7f1c37c0d700 1 -- 172.21.15.102:0/1141530543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c300ffab0 msgr2=0x7f1c30198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.738+0000 7f1c37c0d700 1 --2- 172.21.15.102:0/1141530543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c300ffab0 0x7f1c30198530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00e940 tx=0x7f1c2c00ec50 comp rx=0 tx=0).stop 2024-05-13T22:49:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.738+0000 7f1c37c0d700 1 -- 172.21.15.102:0/1141530543 shutdown_connections 2024-05-13T22:49:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.738+0000 7f1c37c0d700 1 --2- 172.21.15.102:0/1141530543 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1c1c06dfa0 0x7f1c1c070460 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.738+0000 7f1c37c0d700 1 --2- 172.21.15.102:0/1141530543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c300ffab0 0x7f1c30198530 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00e940 tx=0x7f1c2c00ec50 comp rx=0 tx=0).stop 2024-05-13T22:49:15.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.738+0000 7f1c37c0d700 1 --2- 172.21.15.102:0/1141530543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c30100cb0 0x7f1c30198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:15.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.738+0000 7f1c37c0d700 1 -- 172.21.15.102:0/1141530543 >> 172.21.15.102:0/1141530543 conn(0x7f1c300fb030 msgr2=0x7f1c30103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:15.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.738+0000 7f1c37c0d700 1 -- 172.21.15.102:0/1141530543 shutdown_connections 2024-05-13T22:49:15.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.738+0000 7f1c37c0d700 1 -- 172.21.15.102:0/1141530543 wait complete. 2024-05-13T22:49:15.728 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:49:15.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.870+0000 7f4c815dd700 1 -- 172.21.15.102:0/422916795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c7c104030 msgr2=0x7f4c7c106420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:15.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.870+0000 7f4c815dd700 1 --2- 172.21.15.102:0/422916795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c7c104030 0x7f4c7c106420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009910 tx=0x7f4c70009c20 comp rx=0 tx=0).stop 2024-05-13T22:49:15.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.870+0000 7f4c815dd700 1 -- 172.21.15.102:0/422916795 shutdown_connections 2024-05-13T22:49:15.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.870+0000 7f4c815dd700 1 --2- 172.21.15.102:0/422916795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c7c104030 0x7f4c7c106420 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:15.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.870+0000 7f4c815dd700 1 --2- 172.21.15.102:0/422916795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c101700 0x7f4c7c103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:15.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.870+0000 7f4c815dd700 1 -- 172.21.15.102:0/422916795 >> 172.21.15.102:0/422916795 conn(0x7f4c7c0fb050 msgr2=0x7f4c7c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:15.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.870+0000 7f4c815dd700 1 -- 172.21.15.102:0/422916795 shutdown_connections 2024-05-13T22:49:15.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.870+0000 7f4c815dd700 1 -- 172.21.15.102:0/422916795 wait complete. 2024-05-13T22:49:15.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.871+0000 7f4c815dd700 1 Processor -- start 2024-05-13T22:49:15.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.871+0000 7f4c815dd700 1 -- start start 2024-05-13T22:49:15.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.871+0000 7f4c815dd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c101700 0x7f4c7c101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:15.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.871+0000 7f4c815dd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c7c104030 0x7f4c7c0ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:15.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.871+0000 7f4c815dd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0ffbf0 con 0x7f4c7c104030 2024-05-13T22:49:15.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.871+0000 7f4c815dd700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0ffd30 con 0x7f4c7c101700 2024-05-13T22:49:15.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.871+0000 7f4c7a7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c7c104030 0x7f4c7c0ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:15.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.871+0000 7f4c7a7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c7c104030 0x7f4c7c0ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53494/0 (socket says 172.21.15.102:53494) 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.871+0000 7f4c7a7fc700 1 -- 172.21.15.102:0/2844979098 learned_addr learned my addr 172.21.15.102:0/2844979098 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.871+0000 7f4c7affd700 1 --2- 172.21.15.102:0/2844979098 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c101700 0x7f4c7c101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.872+0000 7f4c7a7fc700 1 -- 172.21.15.102:0/2844979098 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c101700 msgr2=0x7f4c7c101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.872+0000 7f4c7a7fc700 1 --2- 172.21.15.102:0/2844979098 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c101700 0x7f4c7c101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.872+0000 7f4c7a7fc700 1 -- 172.21.15.102:0/2844979098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c009520 con 0x7f4c7c104030 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.872+0000 7f4c7a7fc700 1 --2- 172.21.15.102:0/2844979098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c7c104030 0x7f4c7c0ff6b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4c700092e0 tx=0x7f4c7000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.872+0000 7f4c63fff700 1 -- 172.21.15.102:0/2844979098 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c7001d070 con 0x7f4c7c104030 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.872+0000 7f4c815dd700 1 -- 172.21.15.102:0/2844979098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c700095f0 con 0x7f4c7c104030 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.872+0000 7f4c63fff700 1 -- 172.21.15.102:0/2844979098 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c70005dc0 con 0x7f4c7c104030 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.872+0000 7f4c815dd700 1 -- 172.21.15.102:0/2844979098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c100310 con 0x7f4c7c104030 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.872+0000 7f4c63fff700 1 -- 172.21.15.102:0/2844979098 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c70017630 con 0x7f4c7c104030 2024-05-13T22:49:15.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.873+0000 7f4c815dd700 1 -- 172.21.15.102:0/2844979098 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c190520 con 0x7f4c7c104030 2024-05-13T22:49:15.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.874+0000 7f4c63fff700 1 -- 172.21.15.102:0/2844979098 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 30) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f4c7000faf0 con 0x7f4c7c104030 2024-05-13T22:49:15.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.874+0000 7f4c63fff700 1 --2- 172.21.15.102:0/2844979098 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4c6406dfc0 0x7f4c64070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:15.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.875+0000 7f4c63fff700 1 -- 172.21.15.102:0/2844979098 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4c7008f1f0 con 0x7f4c7c104030 2024-05-13T22:49:15.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.875+0000 7f4c7affd700 1 --2- 172.21.15.102:0/2844979098 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4c6406dfc0 0x7f4c64070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:15.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.875+0000 7f4c7affd700 1 --2- 172.21.15.102:0/2844979098 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4c6406dfc0 0x7f4c64070480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009df0 tx=0x7f4c6c009260 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:15.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:15.883+0000 7f4c63fff700 1 -- 172.21.15.102:0/2844979098 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f4c7005c390 con 0x7f4c7c104030 2024-05-13T22:49:16.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.209+0000 7f4c815dd700 1 -- 172.21.15.102:0/2844979098 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c7c0611d0 con 0x7f4c6406dfc0 2024-05-13T22:49:16.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.210+0000 7f4c63fff700 1 -- 172.21.15.102:0/2844979098 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f4c7c0611d0 con 0x7f4c6406dfc0 2024-05-13T22:49:16.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.214+0000 7f4c815dd700 1 -- 172.21.15.102:0/2844979098 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4c6406dfc0 msgr2=0x7f4c64070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:16.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.214+0000 7f4c815dd700 1 --2- 172.21.15.102:0/2844979098 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4c6406dfc0 0x7f4c64070480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009df0 tx=0x7f4c6c009260 comp rx=0 tx=0).stop 2024-05-13T22:49:16.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.214+0000 7f4c815dd700 1 -- 172.21.15.102:0/2844979098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c7c104030 msgr2=0x7f4c7c0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:16.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.214+0000 7f4c815dd700 1 --2- 172.21.15.102:0/2844979098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c7c104030 0x7f4c7c0ff6b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4c700092e0 tx=0x7f4c7000f690 comp rx=0 tx=0).stop 2024-05-13T22:49:16.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.214+0000 7f4c815dd700 1 -- 172.21.15.102:0/2844979098 shutdown_connections 2024-05-13T22:49:16.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.214+0000 7f4c815dd700 1 --2- 172.21.15.102:0/2844979098 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4c6406dfc0 0x7f4c64070480 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.214+0000 7f4c815dd700 1 --2- 172.21.15.102:0/2844979098 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c101700 0x7f4c7c101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.214+0000 7f4c815dd700 1 --2- 172.21.15.102:0/2844979098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c7c104030 0x7f4c7c0ff6b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.214+0000 7f4c815dd700 1 -- 172.21.15.102:0/2844979098 >> 172.21.15.102:0/2844979098 conn(0x7f4c7c0fb050 msgr2=0x7f4c7c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:16.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.214+0000 7f4c815dd700 1 -- 172.21.15.102:0/2844979098 shutdown_connections 2024-05-13T22:49:16.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.214+0000 7f4c815dd700 1 -- 172.21.15.102:0/2844979098 wait complete. 2024-05-13T22:49:16.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.338+0000 7f9f9153b700 1 -- 172.21.15.102:0/2794816200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f8c10a370 msgr2=0x7f9f8c10a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:16.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.338+0000 7f9f9153b700 1 --2- 172.21.15.102:0/2794816200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f8c10a370 0x7f9f8c10a7d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009870 tx=0x7f9f7c009b80 comp rx=0 tx=0).stop 2024-05-13T22:49:16.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.339+0000 7f9f9153b700 1 -- 172.21.15.102:0/2794816200 shutdown_connections 2024-05-13T22:49:16.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.339+0000 7f9f9153b700 1 --2- 172.21.15.102:0/2794816200 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f8c10a370 0x7f9f8c10a7d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.339+0000 7f9f9153b700 1 --2- 172.21.15.102:0/2794816200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c072b80 0x7f9f8c109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.317 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.339+0000 7f9f9153b700 1 -- 172.21.15.102:0/2794816200 >> 172.21.15.102:0/2794816200 conn(0x7f9f8c06d9f0 msgr2=0x7f9f8c06fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:16.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.340+0000 7f9f9153b700 1 -- 172.21.15.102:0/2794816200 shutdown_connections 2024-05-13T22:49:16.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.341+0000 7f9f9153b700 1 -- 172.21.15.102:0/2794816200 wait complete. 2024-05-13T22:49:16.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.342+0000 7f9f9153b700 1 Processor -- start 2024-05-13T22:49:16.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.343+0000 7f9f9153b700 1 -- start start 2024-05-13T22:49:16.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.343+0000 7f9f9153b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f8c072b80 0x7f9f8c10ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:16.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.343+0000 7f9f9153b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c10a370 0x7f9f8c10ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:16.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.343+0000 7f9f9153b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c10b510 con 0x7f9f8c10a370 2024-05-13T22:49:16.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.343+0000 7f9f9153b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c10b680 con 0x7f9f8c072b80 2024-05-13T22:49:16.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.343+0000 7f9f8a7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c10a370 0x7f9f8c10ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:16.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.343+0000 7f9f8a7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c10a370 0x7f9f8c10ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53502/0 (socket says 172.21.15.102:53502) 2024-05-13T22:49:16.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.343+0000 7f9f8a7fc700 1 -- 172.21.15.102:0/2169135559 learned_addr learned my addr 172.21.15.102:0/2169135559 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:49:16.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.343+0000 7f9f8affd700 1 --2- 172.21.15.102:0/2169135559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f8c072b80 0x7f9f8c10ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:16.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.344+0000 7f9f8a7fc700 1 -- 172.21.15.102:0/2169135559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f8c072b80 msgr2=0x7f9f8c10ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:16.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.344+0000 7f9f8a7fc700 1 --2- 172.21.15.102:0/2169135559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f8c072b80 0x7f9f8c10ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.344+0000 7f9f8a7fc700 1 -- 172.21.15.102:0/2169135559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c009520 con 0x7f9f8c10a370 2024-05-13T22:49:16.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.344+0000 7f9f8a7fc700 1 --2- 172.21.15.102:0/2169135559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c10a370 0x7f9f8c10ef60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c005c50 tx=0x7f9f7c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:16.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.345+0000 7f9f73fff700 1 -- 172.21.15.102:0/2169135559 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f7c01c070 con 0x7f9f8c10a370 2024-05-13T22:49:16.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.345+0000 7f9f73fff700 1 -- 172.21.15.102:0/2169135559 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9f7c0203f0 con 0x7f9f8c10a370 2024-05-13T22:49:16.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.345+0000 7f9f73fff700 1 -- 172.21.15.102:0/2169135559 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f7c017620 con 0x7f9f8c10a370 2024-05-13T22:49:16.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.346+0000 7f9f9153b700 1 -- 172.21.15.102:0/2169135559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8c10b900 con 0x7f9f8c10a370 2024-05-13T22:49:16.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.347+0000 7f9f9153b700 1 -- 172.21.15.102:0/2169135559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c10be70 con 0x7f9f8c10a370 2024-05-13T22:49:16.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.348+0000 7f9f9153b700 1 -- 172.21.15.102:0/2169135559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8c19af20 con 0x7f9f8c10a370 2024-05-13T22:49:16.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.350+0000 7f9f73fff700 1 -- 172.21.15.102:0/2169135559 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 30) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f9f7c020560 con 0x7f9f8c10a370 2024-05-13T22:49:16.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.350+0000 7f9f73fff700 1 --2- 172.21.15.102:0/2169135559 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f9f7406e040 0x7f9f74070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:16.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.350+0000 7f9f73fff700 1 -- 172.21.15.102:0/2169135559 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9f7c08f470 con 0x7f9f8c10a370 2024-05-13T22:49:16.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.351+0000 7f9f8affd700 1 --2- 172.21.15.102:0/2169135559 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f9f7406e040 0x7f9f74070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:16.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.352+0000 7f9f8affd700 1 --2- 172.21.15.102:0/2169135559 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f9f7406e040 0x7f9f74070500 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9f80005fd0 tx=0x7f9f80009310 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:16.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.359+0000 7f9f73fff700 1 -- 172.21.15.102:0/2169135559 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f9f7c05c6b0 con 0x7f9f8c10a370 2024-05-13T22:49:16.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.658+0000 7f9f9153b700 1 -- 172.21.15.102:0/2169135559 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f8c0611d0 con 0x7f9f7406e040 2024-05-13T22:49:16.637 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:16 smithi102 ceph-mon[28599]: pgmap v16: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-05-13T22:49:16.637 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:16 smithi102 ceph-mon[28599]: from='client.24477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:16.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.672+0000 7f9f73fff700 1 -- 172.21.15.102:0/2169135559 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f9f8c0611d0 con 0x7f9f7406e040 2024-05-13T22:49:16.650 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:49:16.650 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (8m) 4s ago 9m 25.6M - 0.25.0 c8568f914cd2 c096e9e9a228 2024-05-13T22:49:16.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (9m) 4s ago 9m 7856k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:49:16.650 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (8m) 23s ago 8m 7558k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:49:16.650 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (9m) 4s ago 9m 6656k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:49:16.650 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (8m) 23s ago 8m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (7m) 4s ago 9m 88.1M - 9.4.7 954c08fa6188 bca1790c17aa 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (4m) 4s ago 4m 139M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (3m) 4s ago 3m 220M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (3m) 23s ago 3m 13.6M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (3m) 23s ago 3m 16.7M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (40s) 4s ago 10m 475M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (118s) 23s ago 8m 76.1M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9a28ef996664 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (10m) 4s ago 11m 51.6M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (8m) 23s ago 8m 43.3M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (9m) 4s ago 9m 16.3M - 1.5.0 0da6a335fe13 af297cf52099 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (8m) 23s ago 8m 15.7M - 1.5.0 0da6a335fe13 cc0f64ee7ecd 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (7m) 4s ago 7m 197M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:49:16.651 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (7m) 4s ago 7m 214M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:49:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (6m) 4s ago 6m 178M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:49:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (6m) 23s ago 6m 244M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:49:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (5m) 23s ago 5m 225M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:49:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (5m) 23s ago 5m 217M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:49:16.652 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (55s) 4s ago 9m 51.2M - 2.43.0 a07b618ecd1d 5066a0d73666 2024-05-13T22:49:16.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:16 smithi130 ceph-mon[36591]: pgmap v16: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-05-13T22:49:16.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:16 smithi130 ceph-mon[36591]: from='client.24477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:16.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.675+0000 7f9f9153b700 1 -- 172.21.15.102:0/2169135559 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f9f7406e040 msgr2=0x7f9f74070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:16.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.675+0000 7f9f9153b700 1 --2- 172.21.15.102:0/2169135559 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f9f7406e040 0x7f9f74070500 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9f80005fd0 tx=0x7f9f80009310 comp rx=0 tx=0).stop 2024-05-13T22:49:16.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.675+0000 7f9f9153b700 1 -- 172.21.15.102:0/2169135559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c10a370 msgr2=0x7f9f8c10ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:16.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.675+0000 7f9f9153b700 1 --2- 172.21.15.102:0/2169135559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c10a370 0x7f9f8c10ef60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c005c50 tx=0x7f9f7c00f510 comp rx=0 tx=0).stop 2024-05-13T22:49:16.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.675+0000 7f9f9153b700 1 -- 172.21.15.102:0/2169135559 shutdown_connections 2024-05-13T22:49:16.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.675+0000 7f9f9153b700 1 --2- 172.21.15.102:0/2169135559 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f9f7406e040 0x7f9f74070500 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.675+0000 7f9f9153b700 1 --2- 172.21.15.102:0/2169135559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f8c072b80 0x7f9f8c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.675+0000 7f9f9153b700 1 --2- 172.21.15.102:0/2169135559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f8c10a370 0x7f9f8c10ef60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.675+0000 7f9f9153b700 1 -- 172.21.15.102:0/2169135559 >> 172.21.15.102:0/2169135559 conn(0x7f9f8c06d9f0 msgr2=0x7f9f8c06fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:16.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.675+0000 7f9f9153b700 1 -- 172.21.15.102:0/2169135559 shutdown_connections 2024-05-13T22:49:16.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.675+0000 7f9f9153b700 1 -- 172.21.15.102:0/2169135559 wait complete. 2024-05-13T22:49:16.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.794+0000 7f0631f07700 1 -- 172.21.15.102:0/1909154135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f062c104020 msgr2=0x7f062c106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:16.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.794+0000 7f0631f07700 1 --2- 172.21.15.102:0/1909154135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f062c104020 0x7f062c106410 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0620009910 tx=0x7f0620009c20 comp rx=0 tx=0).stop 2024-05-13T22:49:16.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.794+0000 7f0631f07700 1 -- 172.21.15.102:0/1909154135 shutdown_connections 2024-05-13T22:49:16.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.794+0000 7f0631f07700 1 --2- 172.21.15.102:0/1909154135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f062c104020 0x7f062c106410 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.794+0000 7f0631f07700 1 --2- 172.21.15.102:0/1909154135 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f062c1016f0 0x7f062c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.794+0000 7f0631f07700 1 -- 172.21.15.102:0/1909154135 >> 172.21.15.102:0/1909154135 conn(0x7f062c0fb080 msgr2=0x7f062c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.794+0000 7f0631f07700 1 -- 172.21.15.102:0/1909154135 shutdown_connections 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.794+0000 7f0631f07700 1 -- 172.21.15.102:0/1909154135 wait complete. 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.795+0000 7f0631f07700 1 Processor -- start 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.795+0000 7f0631f07700 1 -- start start 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.795+0000 7f0631f07700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f062c1016f0 0x7f062c196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.795+0000 7f0631f07700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f062c104020 0x7f062c196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.795+0000 7f0631f07700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c196e20 con 0x7f062c104020 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.795+0000 7f0631f07700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c196f60 con 0x7f062c1016f0 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.795+0000 7f062affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f062c104020 0x7f062c196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.795+0000 7f062affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f062c104020 0x7f062c196890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53528/0 (socket says 172.21.15.102:53528) 2024-05-13T22:49:16.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.795+0000 7f062affd700 1 -- 172.21.15.102:0/2115263711 learned_addr learned my addr 172.21.15.102:0/2115263711 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:49:16.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.796+0000 7f062b7fe700 1 --2- 172.21.15.102:0/2115263711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f062c1016f0 0x7f062c196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:16.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.796+0000 7f062affd700 1 -- 172.21.15.102:0/2115263711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f062c1016f0 msgr2=0x7f062c196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:16.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.796+0000 7f062affd700 1 --2- 172.21.15.102:0/2115263711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f062c1016f0 0x7f062c196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:16.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.796+0000 7f062affd700 1 -- 172.21.15.102:0/2115263711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06200095f0 con 0x7f062c104020 2024-05-13T22:49:16.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.796+0000 7f062affd700 1 --2- 172.21.15.102:0/2115263711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f062c104020 0x7f062c196890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f062000bd70 tx=0x7f062000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:16.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.796+0000 7f0628ff9700 1 -- 172.21.15.102:0/2115263711 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f062001d070 con 0x7f062c104020 2024-05-13T22:49:16.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.796+0000 7f0631f07700 1 -- 172.21.15.102:0/2115263711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062c19b9c0 con 0x7f062c104020 2024-05-13T22:49:16.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.796+0000 7f0628ff9700 1 -- 172.21.15.102:0/2115263711 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0620022470 con 0x7f062c104020 2024-05-13T22:49:16.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.796+0000 7f0628ff9700 1 -- 172.21.15.102:0/2115263711 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f062000f670 con 0x7f062c104020 2024-05-13T22:49:16.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.796+0000 7f0631f07700 1 -- 172.21.15.102:0/2115263711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c19be80 con 0x7f062c104020 2024-05-13T22:49:16.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.797+0000 7f0631f07700 1 -- 172.21.15.102:0/2115263711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062c190550 con 0x7f062c104020 2024-05-13T22:49:16.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.798+0000 7f0628ff9700 1 -- 172.21.15.102:0/2115263711 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 30) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f0620022a50 con 0x7f062c104020 2024-05-13T22:49:16.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.798+0000 7f0628ff9700 1 --2- 172.21.15.102:0/2115263711 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f061406dfc0 0x7f0614070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:16.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.798+0000 7f0628ff9700 1 -- 172.21.15.102:0/2115263711 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f062008f540 con 0x7f062c104020 2024-05-13T22:49:16.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.798+0000 7f062b7fe700 1 --2- 172.21.15.102:0/2115263711 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f061406dfc0 0x7f0614070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:16.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.799+0000 7f062b7fe700 1 --2- 172.21.15.102:0/2115263711 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f061406dfc0 0x7f0614070480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f061c005ea0 tx=0x7f061c005e30 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:16.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:16.803+0000 7f0628ff9700 1 -- 172.21.15.102:0/2115263711 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f062005c710 con 0x7f062c104020 2024-05-13T22:49:17.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.213+0000 7f0631f07700 1 -- 172.21.15.102:0/2115263711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f062c02cc70 con 0x7f062c104020 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.214+0000 7f0628ff9700 1 -- 172.21.15.102:0/2115263711 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f062005c2a0 con 0x7f062c104020 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:49:17.192 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:49:17.193 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:49:17.193 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:49:17.193 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:49:17.193 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-05-13T22:49:17.193 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:49:17.193 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:49:17.193 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:49:17.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.216+0000 7f0631f07700 1 -- 172.21.15.102:0/2115263711 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f061406dfc0 msgr2=0x7f0614070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:17.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.216+0000 7f0631f07700 1 --2- 172.21.15.102:0/2115263711 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f061406dfc0 0x7f0614070480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f061c005ea0 tx=0x7f061c005e30 comp rx=0 tx=0).stop 2024-05-13T22:49:17.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.216+0000 7f0631f07700 1 -- 172.21.15.102:0/2115263711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f062c104020 msgr2=0x7f062c196890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:17.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.216+0000 7f0631f07700 1 --2- 172.21.15.102:0/2115263711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f062c104020 0x7f062c196890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f062000bd70 tx=0x7f062000be50 comp rx=0 tx=0).stop 2024-05-13T22:49:17.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.216+0000 7f0631f07700 1 -- 172.21.15.102:0/2115263711 shutdown_connections 2024-05-13T22:49:17.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.216+0000 7f0631f07700 1 --2- 172.21.15.102:0/2115263711 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f061406dfc0 0x7f0614070480 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:17.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.216+0000 7f0631f07700 1 --2- 172.21.15.102:0/2115263711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f062c1016f0 0x7f062c196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:17.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.216+0000 7f0631f07700 1 --2- 172.21.15.102:0/2115263711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f062c104020 0x7f062c196890 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:17.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.216+0000 7f0631f07700 1 -- 172.21.15.102:0/2115263711 >> 172.21.15.102:0/2115263711 conn(0x7f062c0fb080 msgr2=0x7f062c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:17.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.217+0000 7f0631f07700 1 -- 172.21.15.102:0/2115263711 shutdown_connections 2024-05-13T22:49:17.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.217+0000 7f0631f07700 1 -- 172.21.15.102:0/2115263711 wait complete. 2024-05-13T22:49:17.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.342+0000 7ff68478e700 1 -- 172.21.15.102:0/2053161130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff67c1016e0 msgr2=0x7ff67c103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:17.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.342+0000 7ff68478e700 1 --2- 172.21.15.102:0/2053161130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff67c1016e0 0x7ff67c103ad0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff678009910 tx=0x7ff678009c20 comp rx=0 tx=0).stop 2024-05-13T22:49:17.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.342+0000 7ff68478e700 1 -- 172.21.15.102:0/2053161130 shutdown_connections 2024-05-13T22:49:17.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.342+0000 7ff68478e700 1 --2- 172.21.15.102:0/2053161130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff67c104010 0x7ff67c106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:17.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.342+0000 7ff68478e700 1 --2- 172.21.15.102:0/2053161130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff67c1016e0 0x7ff67c103ad0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:17.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.342+0000 7ff68478e700 1 -- 172.21.15.102:0/2053161130 >> 172.21.15.102:0/2053161130 conn(0x7ff67c0fb030 msgr2=0x7ff67c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.342+0000 7ff68478e700 1 -- 172.21.15.102:0/2053161130 shutdown_connections 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.342+0000 7ff68478e700 1 -- 172.21.15.102:0/2053161130 wait complete. 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.343+0000 7ff68478e700 1 Processor -- start 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.343+0000 7ff68478e700 1 -- start start 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.343+0000 7ff68478e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff67c1016e0 0x7ff67c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.343+0000 7ff68478e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff67c104010 0x7ff67c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.343+0000 7ff68478e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c196e80 con 0x7ff67c1016e0 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.343+0000 7ff68478e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c196fc0 con 0x7ff67c104010 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.343+0000 7ff68252a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff67c1016e0 0x7ff67c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.343+0000 7ff68252a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff67c1016e0 0x7ff67c196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:53544/0 (socket says 172.21.15.102:53544) 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.343+0000 7ff68252a700 1 -- 172.21.15.102:0/4285398565 learned_addr learned my addr 172.21.15.102:0/4285398565 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:49:17.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.343+0000 7ff681d29700 1 --2- 172.21.15.102:0/4285398565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff67c104010 0x7ff67c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:17.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.344+0000 7ff68252a700 1 -- 172.21.15.102:0/4285398565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff67c104010 msgr2=0x7ff67c196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:17.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.344+0000 7ff68252a700 1 --2- 172.21.15.102:0/4285398565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff67c104010 0x7ff67c196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:17.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.344+0000 7ff68252a700 1 -- 172.21.15.102:0/4285398565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6780095f0 con 0x7ff67c1016e0 2024-05-13T22:49:17.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.344+0000 7ff68252a700 1 --2- 172.21.15.102:0/4285398565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff67c1016e0 0x7ff67c196320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff6780092e0 tx=0x7ff67800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:17.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.344+0000 7ff6737fe700 1 -- 172.21.15.102:0/4285398565 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff67801d070 con 0x7ff67c1016e0 2024-05-13T22:49:17.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.344+0000 7ff68478e700 1 -- 172.21.15.102:0/4285398565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c19ba10 con 0x7ff67c1016e0 2024-05-13T22:49:17.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.344+0000 7ff6737fe700 1 -- 172.21.15.102:0/4285398565 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff678005dc0 con 0x7ff67c1016e0 2024-05-13T22:49:17.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.344+0000 7ff6737fe700 1 -- 172.21.15.102:0/4285398565 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff678017630 con 0x7ff67c1016e0 2024-05-13T22:49:17.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.344+0000 7ff68478e700 1 -- 172.21.15.102:0/4285398565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c19bf00 con 0x7ff67c1016e0 2024-05-13T22:49:17.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.346+0000 7ff68478e700 1 -- 172.21.15.102:0/4285398565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c190530 con 0x7ff67c1016e0 2024-05-13T22:49:17.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.346+0000 7ff6737fe700 1 -- 172.21.15.102:0/4285398565 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 30) v1 ==== 92160+0+0 (secure 0 0 0) 0x7ff6780178a0 con 0x7ff67c1016e0 2024-05-13T22:49:17.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.347+0000 7ff6737fe700 1 --2- 172.21.15.102:0/4285398565 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff66806dfc0 0x7ff668070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:17.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.347+0000 7ff6737fe700 1 -- 172.21.15.102:0/4285398565 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff67808f300 con 0x7ff67c1016e0 2024-05-13T22:49:17.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.347+0000 7ff681d29700 1 --2- 172.21.15.102:0/4285398565 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff66806dfc0 0x7ff668070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:17.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.347+0000 7ff681d29700 1 --2- 172.21.15.102:0/4285398565 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff66806dfc0 0x7ff668070480 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009c60 tx=0x7ff66c009260 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:17.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.353+0000 7ff6737fe700 1 -- 172.21.15.102:0/4285398565 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7ff67805d480 con 0x7ff67c1016e0 2024-05-13T22:49:17.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:17 smithi130 ceph-mon[36591]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:17.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:17 smithi130 ceph-mon[36591]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:17.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:17 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2115263711' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:17.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:17 smithi102 ceph-mon[28599]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:17.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:17 smithi102 ceph-mon[28599]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:17.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:17 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2115263711' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:17.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.728+0000 7ff68478e700 1 -- 172.21.15.102:0/4285398565 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff67c0611d0 con 0x7ff66806dfc0 2024-05-13T22:49:17.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.730+0000 7ff6737fe700 1 -- 172.21.15.102:0/4285398565 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7ff67c0611d0 con 0x7ff66806dfc0 2024-05-13T22:49:17.708 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:49:17.708 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:49:17.708 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:49:17.708 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-13T22:49:17.708 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:49:17.708 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr" 2024-05-13T22:49:17.708 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:49:17.708 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "2/2 daemons upgraded", 2024-05-13T22:49:17.708 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "", 2024-05-13T22:49:17.709 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:49:17.709 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:49:17.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.733+0000 7ff68478e700 1 -- 172.21.15.102:0/4285398565 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff66806dfc0 msgr2=0x7ff668070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:17.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.733+0000 7ff68478e700 1 --2- 172.21.15.102:0/4285398565 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff66806dfc0 0x7ff668070480 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009c60 tx=0x7ff66c009260 comp rx=0 tx=0).stop 2024-05-13T22:49:17.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.733+0000 7ff68478e700 1 -- 172.21.15.102:0/4285398565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff67c1016e0 msgr2=0x7ff67c196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:17.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.733+0000 7ff68478e700 1 --2- 172.21.15.102:0/4285398565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff67c1016e0 0x7ff67c196320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff6780092e0 tx=0x7ff67800f690 comp rx=0 tx=0).stop 2024-05-13T22:49:17.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.734+0000 7ff68478e700 1 -- 172.21.15.102:0/4285398565 shutdown_connections 2024-05-13T22:49:17.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.734+0000 7ff68478e700 1 --2- 172.21.15.102:0/4285398565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff67c1016e0 0x7ff67c196320 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:17.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.734+0000 7ff68478e700 1 --2- 172.21.15.102:0/4285398565 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff66806dfc0 0x7ff668070480 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:17.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.734+0000 7ff68478e700 1 --2- 172.21.15.102:0/4285398565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff67c104010 0x7ff67c196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:17.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.734+0000 7ff68478e700 1 -- 172.21.15.102:0/4285398565 >> 172.21.15.102:0/4285398565 conn(0x7ff67c0fb030 msgr2=0x7ff67c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:17.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.734+0000 7ff68478e700 1 -- 172.21.15.102:0/4285398565 shutdown_connections 2024-05-13T22:49:17.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:17.734+0000 7ff68478e700 1 -- 172.21.15.102:0/4285398565 wait complete. 2024-05-13T22:49:18.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:18 smithi130 ceph-mon[36591]: pgmap v17: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 356 op/s 2024-05-13T22:49:18.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:18 smithi130 ceph-mon[36591]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:18.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:18 smithi102 ceph-mon[28599]: pgmap v17: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 356 op/s 2024-05-13T22:49:18.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:18 smithi102 ceph-mon[28599]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:20.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:20 smithi130 ceph-mon[36591]: pgmap v18: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 290 op/s 2024-05-13T22:49:20.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:20 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:49:20.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:20 smithi102 ceph-mon[28599]: pgmap v18: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 290 op/s 2024-05-13T22:49:20.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:20 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:49:21.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:21 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:21.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:21 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:21.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:21 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:49:21.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:21 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:49:21.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:21 smithi102 ceph-mon[28599]: Updating smithi102:/etc/ceph/ceph.conf 2024-05-13T22:49:21.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:21 smithi102 ceph-mon[28599]: Updating smithi130:/etc/ceph/ceph.conf 2024-05-13T22:49:21.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:21 smithi102 ceph-mon[28599]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:49:21.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:21 smithi102 ceph-mon[28599]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:49:21.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:21 smithi102 ceph-mon[28599]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:49:21.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:21 smithi102 ceph-mon[28599]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:49:21.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:21 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:21.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:21 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:21.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:21 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:49:21.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:21 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:49:21.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:21 smithi130 ceph-mon[36591]: Updating smithi102:/etc/ceph/ceph.conf 2024-05-13T22:49:21.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:21 smithi130 ceph-mon[36591]: Updating smithi130:/etc/ceph/ceph.conf 2024-05-13T22:49:21.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:21 smithi130 ceph-mon[36591]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:49:21.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:21 smithi130 ceph-mon[36591]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:49:21.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:21 smithi130 ceph-mon[36591]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:49:21.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:21 smithi130 ceph-mon[36591]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:49:22.874 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:22 smithi102 ceph-mon[28599]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:49:22.874 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:22 smithi102 ceph-mon[28599]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:49:22.874 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:22 smithi102 ceph-mon[28599]: pgmap v19: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 430 op/s 2024-05-13T22:49:22.874 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:22 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:22.875 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:22 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:22.875 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:22 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:22.875 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:22 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:22.875 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:22 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:22.875 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:22 smithi102 ceph-mon[28599]: Reconfiguring prometheus.smithi102 (dependencies changed)... 2024-05-13T22:49:22.875 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:22 smithi102 ceph-mon[28599]: Reconfiguring daemon prometheus.smithi102 on smithi102 2024-05-13T22:49:22.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:22 smithi130 ceph-mon[36591]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:49:22.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:22 smithi130 ceph-mon[36591]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:49:22.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:22 smithi130 ceph-mon[36591]: pgmap v19: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 430 op/s 2024-05-13T22:49:22.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:22 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:22.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:22 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:22.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:22 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:22.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:22 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:22.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:22 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:22.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:22 smithi130 ceph-mon[36591]: Reconfiguring prometheus.smithi102 (dependencies changed)... 2024-05-13T22:49:22.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:22 smithi130 ceph-mon[36591]: Reconfiguring daemon prometheus.smithi102 on smithi102 2024-05-13T22:49:24.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:24 smithi102 ceph-mon[28599]: pgmap v20: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 321 op/s 2024-05-13T22:49:24.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:24 smithi130 ceph-mon[36591]: pgmap v20: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 321 op/s 2024-05-13T22:49:26.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:26 smithi130 ceph-mon[36591]: pgmap v21: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 368 op/s 2024-05-13T22:49:26.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:26 smithi102 ceph-mon[28599]: pgmap v21: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 368 op/s 2024-05-13T22:49:28.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:28 smithi130 ceph-mon[36591]: pgmap v22: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 425 op/s 2024-05-13T22:49:28.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:28 smithi102 ceph-mon[28599]: pgmap v22: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 425 op/s 2024-05-13T22:49:30.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:30 smithi130 ceph-mon[36591]: pgmap v23: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-05-13T22:49:30.651 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:30 smithi102 ceph-mon[28599]: pgmap v23: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-05-13T22:49:32.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:32 smithi102 ceph-mon[28599]: pgmap v24: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 470 op/s 2024-05-13T22:49:32.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:32 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:32.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:32 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:32.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:32 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:49:32.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:32 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:49:32.894 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:32 smithi130 ceph-mon[36591]: pgmap v24: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 470 op/s 2024-05-13T22:49:32.894 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:32 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:32.894 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:32 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:32.894 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:32 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:49:32.894 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:32 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:49:33.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:33 smithi102 ceph-mon[28599]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:49:33.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:33 smithi130 ceph-mon[36591]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:49:34.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:34 smithi102 ceph-mon[28599]: pgmap v25: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 330 op/s 2024-05-13T22:49:34.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:34 smithi102 ceph-mon[28599]: Upgrade: Updating mgr.smithi130.uoqoxl 2024-05-13T22:49:34.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:34 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi130.uoqoxl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:49:34.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:34 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:49:34.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:34 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:49:34.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:34 smithi102 ceph-mon[28599]: Deploying daemon mgr.smithi130.uoqoxl on smithi130 2024-05-13T22:49:34.715 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:34 smithi130 ceph-mon[36591]: pgmap v25: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 330 op/s 2024-05-13T22:49:34.715 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:34 smithi130 ceph-mon[36591]: Upgrade: Updating mgr.smithi130.uoqoxl 2024-05-13T22:49:34.715 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:34 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi130.uoqoxl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:49:34.715 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:34 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:49:34.715 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:34 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:49:34.715 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:34 smithi130 ceph-mon[36591]: Deploying daemon mgr.smithi130.uoqoxl on smithi130 2024-05-13T22:49:35.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:35 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:49:35.767 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:35 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:49:36.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:36 smithi102 ceph-mon[28599]: pgmap v26: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 366 op/s 2024-05-13T22:49:36.732 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:36 smithi130 ceph-mon[36591]: pgmap v26: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 366 op/s 2024-05-13T22:49:38.643 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:38 smithi130 ceph-mon[36591]: pgmap v27: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 439 op/s 2024-05-13T22:49:38.644 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:38 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:38.644 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:38 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:38.644 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:38 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:49:38.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:38 smithi102 ceph-mon[28599]: pgmap v27: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 439 op/s 2024-05-13T22:49:38.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:38 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:38.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:38 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:38.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:38 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:49:40.150 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:40 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:40.150 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:40 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:40.301 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:40 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:40.301 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:40 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:41.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:41 smithi130 ceph-mon[36591]: pgmap v28: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 331 op/s 2024-05-13T22:49:41.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:41 smithi102 ceph-mon[28599]: pgmap v28: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 331 op/s 2024-05-13T22:49:43.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:42 smithi130 ceph-mon[36591]: pgmap v29: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 606 op/s 2024-05-13T22:49:43.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:42 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:43.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:42 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:43.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:42 smithi102 ceph-mon[28599]: pgmap v29: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 606 op/s 2024-05-13T22:49:43.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:42 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:43.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:42 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:44.345 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:44.345 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:44.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:44.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:45.332 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:45 smithi130 ceph-mon[36591]: pgmap v30: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 432 op/s 2024-05-13T22:49:45.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:45 smithi102 ceph-mon[28599]: pgmap v30: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 432 op/s 2024-05-13T22:49:46.813 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:46 smithi130 ceph-mon[36591]: pgmap v31: 65 pgs: 65 active+clean; 271 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 474 op/s 2024-05-13T22:49:46.814 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:46 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:46.814 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:46 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:46.814 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:46 smithi130 ceph-mon[36591]: Standby manager daemon smithi130.uoqoxl restarted 2024-05-13T22:49:46.814 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:46 smithi130 ceph-mon[36591]: Standby manager daemon smithi130.uoqoxl started 2024-05-13T22:49:46.814 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:46 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/1027885074' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/crt"}]: dispatch 2024-05-13T22:49:46.814 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:46 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/1027885074' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:49:46.814 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:46 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/1027885074' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/key"}]: dispatch 2024-05-13T22:49:46.814 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:46 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/1027885074' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:49:46.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:46 smithi102 ceph-mon[28599]: pgmap v31: 65 pgs: 65 active+clean; 271 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 474 op/s 2024-05-13T22:49:46.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:46 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:46.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:46 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:46.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:46 smithi102 ceph-mon[28599]: Standby manager daemon smithi130.uoqoxl restarted 2024-05-13T22:49:46.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:46 smithi102 ceph-mon[28599]: Standby manager daemon smithi130.uoqoxl started 2024-05-13T22:49:46.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:46 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/1027885074' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/crt"}]: dispatch 2024-05-13T22:49:46.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:46 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/1027885074' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:49:46.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:46 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/1027885074' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/key"}]: dispatch 2024-05-13T22:49:46.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:46 smithi102 ceph-mon[28599]: from='mgr.? 172.21.15.130:0/1027885074' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:49:47.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.891+0000 7f64b7962700 1 -- 172.21.15.102:0/2908671389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64b00ff240 msgr2=0x7f64b00ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:47.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.891+0000 7f64b7962700 1 --2- 172.21.15.102:0/2908671389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64b00ff240 0x7f64b00ff660 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f64a800b0a0 tx=0x7f64a800b3b0 comp rx=0 tx=0).stop 2024-05-13T22:49:47.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.891+0000 7f64b7962700 1 -- 172.21.15.102:0/2908671389 shutdown_connections 2024-05-13T22:49:47.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.891+0000 7f64b7962700 1 --2- 172.21.15.102:0/2908671389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b0100440 0x7f64b01008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:47.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.891+0000 7f64b7962700 1 --2- 172.21.15.102:0/2908671389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64b00ff240 0x7f64b00ff660 secure :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f64a800b0a0 tx=0x7f64a800b3b0 comp rx=0 tx=0).stop 2024-05-13T22:49:47.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.891+0000 7f64b7962700 1 -- 172.21.15.102:0/2908671389 >> 172.21.15.102:0/2908671389 conn(0x7f64b00fa7c0 msgr2=0x7f64b00fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:47.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.893+0000 7f64b7962700 1 -- 172.21.15.102:0/2908671389 shutdown_connections 2024-05-13T22:49:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.893+0000 7f64b7962700 1 -- 172.21.15.102:0/2908671389 wait complete. 2024-05-13T22:49:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.894+0000 7f64b7962700 1 Processor -- start 2024-05-13T22:49:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.894+0000 7f64b7962700 1 -- start start 2024-05-13T22:49:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b7962700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b0100440 0x7f64b0194450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b7962700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64b0194990 0x7f64b0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b7962700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b0194ea0 con 0x7f64b0194990 2024-05-13T22:49:47.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b7962700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b0195010 con 0x7f64b0100440 2024-05-13T22:49:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b56fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b0100440 0x7f64b0194450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b4efd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64b0194990 0x7f64b0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b56fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b0100440 0x7f64b0194450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.102:39658/0 (socket says 172.21.15.102:39658) 2024-05-13T22:49:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b4efd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64b0194990 0x7f64b0199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43588/0 (socket says 172.21.15.102:43588) 2024-05-13T22:49:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b56fe700 1 -- 172.21.15.102:0/3770084876 learned_addr learned my addr 172.21.15.102:0/3770084876 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:49:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b4efd700 1 -- 172.21.15.102:0/3770084876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b0100440 msgr2=0x7f64b0194450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b4efd700 1 --2- 172.21.15.102:0/3770084876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b0100440 0x7f64b0194450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.895+0000 7f64b4efd700 1 -- 172.21.15.102:0/3770084876 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8009d00 con 0x7f64b0194990 2024-05-13T22:49:47.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.896+0000 7f64b4efd700 1 --2- 172.21.15.102:0/3770084876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64b0194990 0x7f64b0199a00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00ea10 tx=0x7f64ac00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:47.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.896+0000 7f64a67fc700 1 -- 172.21.15.102:0/3770084876 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64ac00cca0 con 0x7f64b0194990 2024-05-13T22:49:47.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.896+0000 7f64a67fc700 1 -- 172.21.15.102:0/3770084876 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f64ac00ce00 con 0x7f64b0194990 2024-05-13T22:49:47.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.896+0000 7f64a67fc700 1 -- 172.21.15.102:0/3770084876 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64ac00b7f0 con 0x7f64b0194990 2024-05-13T22:49:47.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.896+0000 7f64b7962700 1 -- 172.21.15.102:0/3770084876 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b0199fa0 con 0x7f64b0194990 2024-05-13T22:49:47.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.896+0000 7f64b7962700 1 -- 172.21.15.102:0/3770084876 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b019a570 con 0x7f64b0194990 2024-05-13T22:49:47.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.899+0000 7f64b7962700 1 -- 172.21.15.102:0/3770084876 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6494005320 con 0x7f64b0194990 2024-05-13T22:49:47.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.899+0000 7f64a67fc700 1 -- 172.21.15.102:0/3770084876 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f64ac016410 con 0x7f64b0194990 2024-05-13T22:49:47.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.900+0000 7f64a67fc700 1 --2- 172.21.15.102:0/3770084876 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f649c06e110 0x7f649c0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:47.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.900+0000 7f64a67fc700 1 -- 172.21.15.102:0/3770084876 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f64ac08dc20 con 0x7f64b0194990 2024-05-13T22:49:47.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.905+0000 7f64b56fe700 1 --2- 172.21.15.102:0/3770084876 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f649c06e110 0x7f649c0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:47.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.906+0000 7f64b56fe700 1 --2- 172.21.15.102:0/3770084876 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f649c06e110 0x7f649c0705d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009fd0 tx=0x7f64a80096e0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:47.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:47.911+0000 7f64a67fc700 1 -- 172.21.15.102:0/3770084876 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f64ac05ada0 con 0x7f64b0194990 2024-05-13T22:49:48.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.257+0000 7f64b7962700 1 -- 172.21.15.102:0/3770084876 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6494000bf0 con 0x7f649c06e110 2024-05-13T22:49:48.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.258+0000 7f64a67fc700 1 -- 172.21.15.102:0/3770084876 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f6494000bf0 con 0x7f649c06e110 2024-05-13T22:49:48.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.261+0000 7f64b7962700 1 -- 172.21.15.102:0/3770084876 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f649c06e110 msgr2=0x7f649c0705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:48.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.261+0000 7f64b7962700 1 --2- 172.21.15.102:0/3770084876 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f649c06e110 0x7f649c0705d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009fd0 tx=0x7f64a80096e0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.261+0000 7f64b7962700 1 -- 172.21.15.102:0/3770084876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64b0194990 msgr2=0x7f64b0199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:48.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.261+0000 7f64b7962700 1 --2- 172.21.15.102:0/3770084876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64b0194990 0x7f64b0199a00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00ea10 tx=0x7f64ac00edd0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.261+0000 7f64b7962700 1 -- 172.21.15.102:0/3770084876 shutdown_connections 2024-05-13T22:49:48.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.261+0000 7f64b7962700 1 --2- 172.21.15.102:0/3770084876 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f649c06e110 0x7f649c0705d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.261+0000 7f64b7962700 1 --2- 172.21.15.102:0/3770084876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b0100440 0x7f64b0194450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.261+0000 7f64b7962700 1 --2- 172.21.15.102:0/3770084876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64b0194990 0x7f64b0199a00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.261+0000 7f64b7962700 1 -- 172.21.15.102:0/3770084876 >> 172.21.15.102:0/3770084876 conn(0x7f64b00fa7c0 msgr2=0x7f64b0103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:48.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.261+0000 7f64b7962700 1 -- 172.21.15.102:0/3770084876 shutdown_connections 2024-05-13T22:49:48.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.261+0000 7f64b7962700 1 -- 172.21.15.102:0/3770084876 wait complete. 2024-05-13T22:49:48.251 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:49:48.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.389+0000 7f24d7e47700 1 -- 172.21.15.102:0/4261209775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24d0103f70 msgr2=0x7f24d0106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:48.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.389+0000 7f24d7e47700 1 --2- 172.21.15.102:0/4261209775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24d0103f70 0x7f24d0106360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009910 tx=0x7f24c0009c20 comp rx=0 tx=0).stop 2024-05-13T22:49:48.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.389+0000 7f24d7e47700 1 -- 172.21.15.102:0/4261209775 shutdown_connections 2024-05-13T22:49:48.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.389+0000 7f24d7e47700 1 --2- 172.21.15.102:0/4261209775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24d0103f70 0x7f24d0106360 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.389+0000 7f24d7e47700 1 --2- 172.21.15.102:0/4261209775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24d0101640 0x7f24d0103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.389+0000 7f24d7e47700 1 -- 172.21.15.102:0/4261209775 >> 172.21.15.102:0/4261209775 conn(0x7f24d00fb030 msgr2=0x7f24d00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:48.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.390+0000 7f24d7e47700 1 -- 172.21.15.102:0/4261209775 shutdown_connections 2024-05-13T22:49:48.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.390+0000 7f24d7e47700 1 -- 172.21.15.102:0/4261209775 wait complete. 2024-05-13T22:49:48.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.390+0000 7f24d7e47700 1 Processor -- start 2024-05-13T22:49:48.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.390+0000 7f24d7e47700 1 -- start start 2024-05-13T22:49:48.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d7e47700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24d0101640 0x7f24d0196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:48.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d7e47700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24d0103f70 0x7f24d0196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:48.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d7e47700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0196e90 con 0x7f24d0101640 2024-05-13T22:49:48.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d7e47700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0196fd0 con 0x7f24d0103f70 2024-05-13T22:49:48.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d5be3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24d0101640 0x7f24d0196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:48.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d5be3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24d0101640 0x7f24d0196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43620/0 (socket says 172.21.15.102:43620) 2024-05-13T22:49:48.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d5be3700 1 -- 172.21.15.102:0/1482328064 learned_addr learned my addr 172.21.15.102:0/1482328064 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:49:48.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d53e2700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24d0103f70 0x7f24d0196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:48.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d5be3700 1 -- 172.21.15.102:0/1482328064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24d0103f70 msgr2=0x7f24d0196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:48.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d5be3700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24d0103f70 0x7f24d0196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d5be3700 1 -- 172.21.15.102:0/1482328064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00095f0 con 0x7f24d0101640 2024-05-13T22:49:48.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d53e2700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24d0103f70 0x7f24d0196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:49:48.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.391+0000 7f24d5be3700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24d0101640 0x7f24d0196330 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00e920 tx=0x7f24cc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:48.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.392+0000 7f24c6ffd700 1 -- 172.21.15.102:0/1482328064 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24cc00c9d0 con 0x7f24d0101640 2024-05-13T22:49:48.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.392+0000 7f24d7e47700 1 -- 172.21.15.102:0/1482328064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d00ff490 con 0x7f24d0101640 2024-05-13T22:49:48.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.392+0000 7f24d7e47700 1 -- 172.21.15.102:0/1482328064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d00ff9e0 con 0x7f24d0101640 2024-05-13T22:49:48.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.392+0000 7f24c6ffd700 1 -- 172.21.15.102:0/1482328064 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f24cc010430 con 0x7f24d0101640 2024-05-13T22:49:48.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.393+0000 7f24c6ffd700 1 -- 172.21.15.102:0/1482328064 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24cc018650 con 0x7f24d0101640 2024-05-13T22:49:48.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.393+0000 7f24d7e47700 1 -- 172.21.15.102:0/1482328064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b4005320 con 0x7f24d0101640 2024-05-13T22:49:48.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.394+0000 7f24c6ffd700 1 -- 172.21.15.102:0/1482328064 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f24cc010a20 con 0x7f24d0101640 2024-05-13T22:49:48.373 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.395+0000 7f24c6ffd700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f24bc06e010 0x7f24bc0704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:48.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.395+0000 7f24d53e2700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f24bc06e010 0x7f24bc0704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:48.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.395+0000 7f24c6ffd700 1 -- 172.21.15.102:0/1482328064 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f24cc014070 con 0x7f24d0101640 2024-05-13T22:49:48.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.396+0000 7f24d53e2700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f24bc06e010 0x7f24bc0704d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009fd0 tx=0x7f24c0005940 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:48.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.400+0000 7f24c6ffd700 1 -- 172.21.15.102:0/1482328064 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f24cc053590 con 0x7f24d0101640 2024-05-13T22:49:48.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:48 smithi130 ceph-mon[36591]: mgrmap e31: smithi102.xcijrj(active, since 57s), standbys: smithi130.uoqoxl 2024-05-13T22:49:48.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:48 smithi130 ceph-mon[36591]: pgmap v32: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 579 op/s 2024-05-13T22:49:48.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:48 smithi102 ceph-mon[28599]: mgrmap e31: smithi102.xcijrj(active, since 57s), standbys: smithi130.uoqoxl 2024-05-13T22:49:48.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:48 smithi102 ceph-mon[28599]: pgmap v32: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 579 op/s 2024-05-13T22:49:48.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.790+0000 7f24d7e47700 1 -- 172.21.15.102:0/1482328064 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24b4000bf0 con 0x7f24bc06e010 2024-05-13T22:49:48.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.791+0000 7f24c6ffd700 1 -- 172.21.15.102:0/1482328064 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f24b4000bf0 con 0x7f24bc06e010 2024-05-13T22:49:48.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.794+0000 7f24d7e47700 1 -- 172.21.15.102:0/1482328064 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f24bc06e010 msgr2=0x7f24bc0704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:48.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.794+0000 7f24d7e47700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f24bc06e010 0x7f24bc0704d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009fd0 tx=0x7f24c0005940 comp rx=0 tx=0).stop 2024-05-13T22:49:48.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.794+0000 7f24d7e47700 1 -- 172.21.15.102:0/1482328064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24d0101640 msgr2=0x7f24d0196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:48.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.794+0000 7f24d7e47700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24d0101640 0x7f24d0196330 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00e920 tx=0x7f24cc00ec30 comp rx=0 tx=0).stop 2024-05-13T22:49:48.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.794+0000 7f24d7e47700 1 -- 172.21.15.102:0/1482328064 shutdown_connections 2024-05-13T22:49:48.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.794+0000 7f24d7e47700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24d0101640 0x7f24d0196330 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.794+0000 7f24d7e47700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f24bc06e010 0x7f24bc0704d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.794+0000 7f24d7e47700 1 --2- 172.21.15.102:0/1482328064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24d0103f70 0x7f24d0196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.794+0000 7f24d7e47700 1 -- 172.21.15.102:0/1482328064 >> 172.21.15.102:0/1482328064 conn(0x7f24d00fb030 msgr2=0x7f24d00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:48.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.795+0000 7f24d7e47700 1 -- 172.21.15.102:0/1482328064 shutdown_connections 2024-05-13T22:49:48.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.795+0000 7f24d7e47700 1 -- 172.21.15.102:0/1482328064 wait complete. 2024-05-13T22:49:48.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.964+0000 7f5da97be700 1 -- 172.21.15.102:0/1228917955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4100cb0 msgr2=0x7f5da4101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:48.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.964+0000 7f5da97be700 1 --2- 172.21.15.102:0/1228917955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4100cb0 0x7f5da4101110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009910 tx=0x7f5d98009c20 comp rx=0 tx=0).stop 2024-05-13T22:49:48.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.964+0000 7f5da97be700 1 -- 172.21.15.102:0/1228917955 shutdown_connections 2024-05-13T22:49:48.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.964+0000 7f5da97be700 1 --2- 172.21.15.102:0/1228917955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4100cb0 0x7f5da4101110 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.964+0000 7f5da97be700 1 --2- 172.21.15.102:0/1228917955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da40ffab0 0x7f5da40ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.964+0000 7f5da97be700 1 -- 172.21.15.102:0/1228917955 >> 172.21.15.102:0/1228917955 conn(0x7f5da40fb030 msgr2=0x7f5da40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:48.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.965+0000 7f5da97be700 1 -- 172.21.15.102:0/1228917955 shutdown_connections 2024-05-13T22:49:48.945 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.965+0000 7f5da97be700 1 -- 172.21.15.102:0/1228917955 wait complete. 2024-05-13T22:49:48.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.966+0000 7f5da97be700 1 Processor -- start 2024-05-13T22:49:48.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.966+0000 7f5da97be700 1 -- start start 2024-05-13T22:49:48.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.966+0000 7f5da97be700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da40ffab0 0x7f5da4103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:48.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.966+0000 7f5da97be700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4100cb0 0x7f5da4102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:48.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.966+0000 7f5da97be700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4199dc0 con 0x7f5da4100cb0 2024-05-13T22:49:48.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.966+0000 7f5da97be700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4199f00 con 0x7f5da40ffab0 2024-05-13T22:49:48.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.966+0000 7f5da2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da40ffab0 0x7f5da4103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:48.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.966+0000 7f5da2ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da40ffab0 0x7f5da4103a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.102:52480/0 (socket says 172.21.15.102:52480) 2024-05-13T22:49:48.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.966+0000 7f5da2ffd700 1 -- 172.21.15.102:0/1681443466 learned_addr learned my addr 172.21.15.102:0/1681443466 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:49:48.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.967+0000 7f5da27fc700 1 --2- 172.21.15.102:0/1681443466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4100cb0 0x7f5da4102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:48.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.967+0000 7f5da2ffd700 1 -- 172.21.15.102:0/1681443466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4100cb0 msgr2=0x7f5da4102090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:48.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.967+0000 7f5da2ffd700 1 --2- 172.21.15.102:0/1681443466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4100cb0 0x7f5da4102090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:48.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.967+0000 7f5da2ffd700 1 -- 172.21.15.102:0/1681443466 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d980095f0 con 0x7f5da40ffab0 2024-05-13T22:49:48.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.967+0000 7f5da2ffd700 1 --2- 172.21.15.102:0/1681443466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da40ffab0 0x7f5da4103a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e8f0 tx=0x7f5d9400ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:48.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.967+0000 7f5d8bfff700 1 -- 172.21.15.102:0/1681443466 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d94009790 con 0x7f5da40ffab0 2024-05-13T22:49:48.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.968+0000 7f5d8bfff700 1 -- 172.21.15.102:0/1681443466 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5d9400c810 con 0x7f5da40ffab0 2024-05-13T22:49:48.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.968+0000 7f5d8bfff700 1 -- 172.21.15.102:0/1681443466 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d94003d20 con 0x7f5da40ffab0 2024-05-13T22:49:48.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.968+0000 7f5da97be700 1 -- 172.21.15.102:0/1681443466 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da41025d0 con 0x7f5da40ffab0 2024-05-13T22:49:48.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.968+0000 7f5da97be700 1 -- 172.21.15.102:0/1681443466 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da4102aa0 con 0x7f5da40ffab0 2024-05-13T22:49:48.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.969+0000 7f5da97be700 1 -- 172.21.15.102:0/1681443466 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da4102dd0 con 0x7f5da40ffab0 2024-05-13T22:49:48.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.969+0000 7f5d8bfff700 1 -- 172.21.15.102:0/1681443466 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f5d940041f0 con 0x7f5da40ffab0 2024-05-13T22:49:48.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.970+0000 7f5d8bfff700 1 --2- 172.21.15.102:0/1681443466 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5d8c06dfc0 0x7f5d8c070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:48.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.970+0000 7f5d8bfff700 1 -- 172.21.15.102:0/1681443466 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5d94014070 con 0x7f5da40ffab0 2024-05-13T22:49:48.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.970+0000 7f5da27fc700 1 --2- 172.21.15.102:0/1681443466 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5d8c06dfc0 0x7f5d8c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:48.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.971+0000 7f5da27fc700 1 --2- 172.21.15.102:0/1681443466 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5d8c06dfc0 0x7f5d8c070480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009fd0 tx=0x7f5d98000bc0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:48.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:48.975+0000 7f5d8bfff700 1 -- 172.21.15.102:0/1681443466 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f5d9405b290 con 0x7f5da40ffab0 2024-05-13T22:49:49.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.281+0000 7f5da97be700 1 -- 172.21.15.102:0/1681443466 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5da4068b40 con 0x7f5d8c06dfc0 2024-05-13T22:49:49.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.299+0000 7f5d8bfff700 1 -- 172.21.15.102:0/1681443466 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5da4068b40 con 0x7f5d8c06dfc0 2024-05-13T22:49:49.278 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:49:49.278 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (8m) 3s ago 10m 25.6M - 0.25.0 c8568f914cd2 c096e9e9a228 2024-05-13T22:49:49.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (10m) 3s ago 10m 7914k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:49:49.279 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (9m) 7s ago 9m 7616k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:49:49.279 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (10m) 3s ago 10m 6660k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:49:49.279 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (9m) 7s ago 9m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:49:49.281 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (8m) 3s ago 9m 89.0M - 9.4.7 954c08fa6188 bca1790c17aa 2024-05-13T22:49:49.282 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (4m) 3s ago 4m 134M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:49:49.282 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (4m) 3s ago 4m 204M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:49:49.282 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (4m) 7s ago 4m 14.1M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:49:49.282 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (4m) 7s ago 4m 17.1M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:49:49.282 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (73s) 3s ago 11m 494M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:49:49.282 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (11s) 7s ago 9m 67.6M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:49:49.282 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (11m) 3s ago 11m 51.6M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:49:49.283 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (8m) 7s ago 8m 44.4M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:49:49.283 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (10m) 3s ago 10m 16.3M - 1.5.0 0da6a335fe13 af297cf52099 2024-05-13T22:49:49.283 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (9m) 7s ago 9m 16.2M - 1.5.0 0da6a335fe13 cc0f64ee7ecd 2024-05-13T22:49:49.283 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (8m) 3s ago 8m 208M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:49:49.283 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (7m) 3s ago 7m 219M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:49:49.283 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (7m) 3s ago 7m 188M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:49:49.284 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (6m) 7s ago 6m 256M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:49:49.284 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (6m) 7s ago 6m 228M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:49:49.284 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (6m) 7s ago 6m 210M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:49:49.284 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (17s) 3s ago 9m 44.7M - 2.43.0 a07b618ecd1d 99aa6bee5d86 2024-05-13T22:49:49.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.301+0000 7f5da97be700 1 -- 172.21.15.102:0/1681443466 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5d8c06dfc0 msgr2=0x7f5d8c070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:49.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.301+0000 7f5da97be700 1 --2- 172.21.15.102:0/1681443466 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5d8c06dfc0 0x7f5d8c070480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009fd0 tx=0x7f5d98000bc0 comp rx=0 tx=0).stop 2024-05-13T22:49:49.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.301+0000 7f5da97be700 1 -- 172.21.15.102:0/1681443466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da40ffab0 msgr2=0x7f5da4103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:49.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.301+0000 7f5da97be700 1 --2- 172.21.15.102:0/1681443466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da40ffab0 0x7f5da4103a40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e8f0 tx=0x7f5d9400ec00 comp rx=0 tx=0).stop 2024-05-13T22:49:49.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.302+0000 7f5da97be700 1 -- 172.21.15.102:0/1681443466 shutdown_connections 2024-05-13T22:49:49.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.302+0000 7f5da97be700 1 --2- 172.21.15.102:0/1681443466 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5d8c06dfc0 0x7f5d8c070480 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:49.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.302+0000 7f5da97be700 1 --2- 172.21.15.102:0/1681443466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5da40ffab0 0x7f5da4103a40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:49.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.302+0000 7f5da97be700 1 --2- 172.21.15.102:0/1681443466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4100cb0 0x7f5da4102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:49.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.302+0000 7f5da97be700 1 -- 172.21.15.102:0/1681443466 >> 172.21.15.102:0/1681443466 conn(0x7f5da40fb030 msgr2=0x7f5da4068430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:49.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.302+0000 7f5da97be700 1 -- 172.21.15.102:0/1681443466 shutdown_connections 2024-05-13T22:49:49.286 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.302+0000 7f5da97be700 1 -- 172.21.15.102:0/1681443466 wait complete. 2024-05-13T22:49:49.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.434+0000 7f475f46b700 1 -- 172.21.15.102:0/3194960610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4758103fd0 msgr2=0x7f47581063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:49.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.434+0000 7f475f46b700 1 --2- 172.21.15.102:0/3194960610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4758103fd0 0x7f47581063c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4748009910 tx=0x7f4748009c20 comp rx=0 tx=0).stop 2024-05-13T22:49:49.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.434+0000 7f475f46b700 1 -- 172.21.15.102:0/3194960610 shutdown_connections 2024-05-13T22:49:49.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.434+0000 7f475f46b700 1 --2- 172.21.15.102:0/3194960610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4758103fd0 0x7f47581063c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:49.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.434+0000 7f475f46b700 1 --2- 172.21.15.102:0/3194960610 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47581016a0 0x7f4758103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:49.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.434+0000 7f475f46b700 1 -- 172.21.15.102:0/3194960610 >> 172.21.15.102:0/3194960610 conn(0x7f47580fb030 msgr2=0x7f47580fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:49.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.435+0000 7f475f46b700 1 -- 172.21.15.102:0/3194960610 shutdown_connections 2024-05-13T22:49:49.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.435+0000 7f475f46b700 1 -- 172.21.15.102:0/3194960610 wait complete. 2024-05-13T22:49:49.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.435+0000 7f475f46b700 1 Processor -- start 2024-05-13T22:49:49.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.435+0000 7f475f46b700 1 -- start start 2024-05-13T22:49:49.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475f46b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47581016a0 0x7f4758196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:49.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475f46b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4758103fd0 0x7f4758196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:49.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475f46b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4758196e90 con 0x7f4758103fd0 2024-05-13T22:49:49.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475f46b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4758196fd0 con 0x7f47581016a0 2024-05-13T22:49:49.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475ca06700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4758103fd0 0x7f4758196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475d207700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47581016a0 0x7f4758196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475ca06700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4758103fd0 0x7f4758196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55616/0 (socket says 172.21.15.102:55616) 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475ca06700 1 -- 172.21.15.102:0/1734804633 learned_addr learned my addr 172.21.15.102:0/1734804633 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475d207700 1 -- 172.21.15.102:0/1734804633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4758103fd0 msgr2=0x7f4758196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475d207700 1 --2- 172.21.15.102:0/1734804633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4758103fd0 0x7f4758196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475d207700 1 -- 172.21.15.102:0/1734804633 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47480095f0 con 0x7f47581016a0 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f475d207700 1 --2- 172.21.15.102:0/1734804633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47581016a0 0x7f4758196330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f475400e970 tx=0x7f475400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.436+0000 7f474e7fc700 1 -- 172.21.15.102:0/1734804633 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f475400c9d0 con 0x7f47581016a0 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.437+0000 7f474e7fc700 1 -- 172.21.15.102:0/1734804633 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4754010430 con 0x7f47581016a0 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.437+0000 7f475f46b700 1 -- 172.21.15.102:0/1734804633 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475819ba80 con 0x7f47581016a0 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.437+0000 7f474e7fc700 1 -- 172.21.15.102:0/1734804633 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4754018650 con 0x7f47581016a0 2024-05-13T22:49:49.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.437+0000 7f475f46b700 1 -- 172.21.15.102:0/1734804633 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475819bfd0 con 0x7f47581016a0 2024-05-13T22:49:49.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.438+0000 7f475f46b700 1 -- 172.21.15.102:0/1734804633 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4758190520 con 0x7f47581016a0 2024-05-13T22:49:49.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.439+0000 7f474e7fc700 1 -- 172.21.15.102:0/1734804633 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f47540105a0 con 0x7f47581016a0 2024-05-13T22:49:49.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.439+0000 7f474e7fc700 1 --2- 172.21.15.102:0/1734804633 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f474406dfc0 0x7f4744070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:49.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.439+0000 7f474e7fc700 1 -- 172.21.15.102:0/1734804633 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4754014070 con 0x7f47581016a0 2024-05-13T22:49:49.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.439+0000 7f475ca06700 1 --2- 172.21.15.102:0/1734804633 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f474406dfc0 0x7f4744070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:49.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.440+0000 7f475ca06700 1 --2- 172.21.15.102:0/1734804633 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f474406dfc0 0x7f4744070480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4748009fd0 tx=0x7f4748005940 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:49.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.445+0000 7f474e7fc700 1 -- 172.21.15.102:0/1734804633 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f4754053590 con 0x7f47581016a0 2024-05-13T22:49:49.565 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: Upgrade: Setting container_image for all mgr 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi102.xcijrj"}]: dispatch 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi102.xcijrj"}]': finished 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi130.uoqoxl"}]: dispatch 2024-05-13T22:49:49.566 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi130.uoqoxl"}]': finished 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: Upgrade: Setting container_image for all rgw 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: Upgrade: Setting container_image for all rbd-mirror 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: Upgrade: Setting container_image for all cephfs-mirror 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: Upgrade: Setting container_image for all iscsi 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: Upgrade: Setting container_image for all nfs 2024-05-13T22:49:49.567 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.568 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.568 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: Upgrade: Setting container_image for all nvmeof 2024-05-13T22:49:49.568 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.568 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:49 smithi102 ceph-mon[28599]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:49.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:49.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:49:49.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:49:49.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:49:49.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: Upgrade: Setting container_image for all mgr 2024-05-13T22:49:49.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi102.xcijrj"}]: dispatch 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi102.xcijrj"}]': finished 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi130.uoqoxl"}]: dispatch 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi130.uoqoxl"}]': finished 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: Upgrade: Setting container_image for all rgw 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: Upgrade: Setting container_image for all rbd-mirror 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: Upgrade: Setting container_image for all cephfs-mirror 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: Upgrade: Setting container_image for all iscsi 2024-05-13T22:49:49.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: Upgrade: Setting container_image for all nfs 2024-05-13T22:49:49.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:49.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: Upgrade: Setting container_image for all nvmeof 2024-05-13T22:49:49.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:49:49.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:49 smithi130 ceph-mon[36591]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:49.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.884+0000 7f475f46b700 1 -- 172.21.15.102:0/1734804633 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f475802cc70 con 0x7f47581016a0 2024-05-13T22:49:49.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.889+0000 7f474e7fc700 1 -- 172.21.15.102:0/1734804633 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f475405b1d0 con 0x7f47581016a0 2024-05-13T22:49:49.867 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:49:49.868 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:49:49.869 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-05-13T22:49:49.869 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:49:49.869 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:49:49.869 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:49:49.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.894+0000 7f475f46b700 1 -- 172.21.15.102:0/1734804633 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f474406dfc0 msgr2=0x7f4744070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:49.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.894+0000 7f475f46b700 1 --2- 172.21.15.102:0/1734804633 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f474406dfc0 0x7f4744070480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4748009fd0 tx=0x7f4748005940 comp rx=0 tx=0).stop 2024-05-13T22:49:49.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.894+0000 7f475f46b700 1 -- 172.21.15.102:0/1734804633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47581016a0 msgr2=0x7f4758196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:49.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.894+0000 7f475f46b700 1 --2- 172.21.15.102:0/1734804633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47581016a0 0x7f4758196330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f475400e970 tx=0x7f475400ec80 comp rx=0 tx=0).stop 2024-05-13T22:49:49.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.894+0000 7f475f46b700 1 -- 172.21.15.102:0/1734804633 shutdown_connections 2024-05-13T22:49:49.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.895+0000 7f475f46b700 1 --2- 172.21.15.102:0/1734804633 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f474406dfc0 0x7f4744070480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:49.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.895+0000 7f475f46b700 1 --2- 172.21.15.102:0/1734804633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47581016a0 0x7f4758196330 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:49.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.895+0000 7f475f46b700 1 --2- 172.21.15.102:0/1734804633 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4758103fd0 0x7f4758196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:49.874 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.895+0000 7f475f46b700 1 -- 172.21.15.102:0/1734804633 >> 172.21.15.102:0/1734804633 conn(0x7f47580fb030 msgr2=0x7f47580fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:49.875 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.897+0000 7f475f46b700 1 -- 172.21.15.102:0/1734804633 shutdown_connections 2024-05-13T22:49:49.876 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:49.897+0000 7f475f46b700 1 -- 172.21.15.102:0/1734804633 wait complete. 2024-05-13T22:49:50.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.060+0000 7f3f6e313700 1 -- 172.21.15.102:0/1794943597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f680ff290 msgr2=0x7f3f680ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:50.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.060+0000 7f3f6e313700 1 --2- 172.21.15.102:0/1794943597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f680ff290 0x7f3f680ff6b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009910 tx=0x7f3f58009c20 comp rx=0 tx=0).stop 2024-05-13T22:49:50.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.060+0000 7f3f6e313700 1 -- 172.21.15.102:0/1794943597 shutdown_connections 2024-05-13T22:49:50.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.060+0000 7f3f6e313700 1 --2- 172.21.15.102:0/1794943597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f68100490 0x7f3f681008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:50.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.060+0000 7f3f6e313700 1 --2- 172.21.15.102:0/1794943597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f680ff290 0x7f3f680ff6b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:50.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.060+0000 7f3f6e313700 1 -- 172.21.15.102:0/1794943597 >> 172.21.15.102:0/1794943597 conn(0x7f3f680fa810 msgr2=0x7f3f680fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:50.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.060+0000 7f3f6e313700 1 -- 172.21.15.102:0/1794943597 shutdown_connections 2024-05-13T22:49:50.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.060+0000 7f3f6e313700 1 -- 172.21.15.102:0/1794943597 wait complete. 2024-05-13T22:49:50.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.061+0000 7f3f6e313700 1 Processor -- start 2024-05-13T22:49:50.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.061+0000 7f3f6e313700 1 -- start start 2024-05-13T22:49:50.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.061+0000 7f3f6e313700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f680ff290 0x7f3f68106230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:50.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.061+0000 7f3f6e313700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f68100490 0x7f3f6819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:50.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.061+0000 7f3f6e313700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6819b720 con 0x7f3f68100490 2024-05-13T22:49:50.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.061+0000 7f3f6e313700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6819b860 con 0x7f3f680ff290 2024-05-13T22:49:50.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.061+0000 7f3f677fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f68100490 0x7f3f6819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:50.041 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.061+0000 7f3f677fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f68100490 0x7f3f6819b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:55622/0 (socket says 172.21.15.102:55622) 2024-05-13T22:49:50.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.061+0000 7f3f677fe700 1 -- 172.21.15.102:0/1382678010 learned_addr learned my addr 172.21.15.102:0/1382678010 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:49:50.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.062+0000 7f3f67fff700 1 --2- 172.21.15.102:0/1382678010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f680ff290 0x7f3f68106230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:50.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.062+0000 7f3f677fe700 1 -- 172.21.15.102:0/1382678010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f680ff290 msgr2=0x7f3f68106230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:50.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.062+0000 7f3f677fe700 1 --2- 172.21.15.102:0/1382678010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f680ff290 0x7f3f68106230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:50.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.062+0000 7f3f677fe700 1 -- 172.21.15.102:0/1382678010 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580095f0 con 0x7f3f68100490 2024-05-13T22:49:50.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.062+0000 7f3f677fe700 1 --2- 172.21.15.102:0/1382678010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f68100490 0x7f3f6819b150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00d8d0 tx=0x7f3f5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:50.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.062+0000 7f3f657fa700 1 -- 172.21.15.102:0/1382678010 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f5c009940 con 0x7f3f68100490 2024-05-13T22:49:50.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.062+0000 7f3f6e313700 1 -- 172.21.15.102:0/1382678010 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6819bb40 con 0x7f3f68100490 2024-05-13T22:49:50.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.062+0000 7f3f6e313700 1 -- 172.21.15.102:0/1382678010 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6819c090 con 0x7f3f68100490 2024-05-13T22:49:50.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.062+0000 7f3f657fa700 1 -- 172.21.15.102:0/1382678010 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3f5c010460 con 0x7f3f68100490 2024-05-13T22:49:50.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.062+0000 7f3f657fa700 1 -- 172.21.15.102:0/1382678010 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f5c01b620 con 0x7f3f68100490 2024-05-13T22:49:50.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.063+0000 7f3f6e313700 1 -- 172.21.15.102:0/1382678010 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f54005320 con 0x7f3f68100490 2024-05-13T22:49:50.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.064+0000 7f3f657fa700 1 -- 172.21.15.102:0/1382678010 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f3f5c01b780 con 0x7f3f68100490 2024-05-13T22:49:50.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.065+0000 7f3f657fa700 1 --2- 172.21.15.102:0/1382678010 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3f5006dfc0 0x7f3f50070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:49:50.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.065+0000 7f3f657fa700 1 -- 172.21.15.102:0/1382678010 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3f5c08e090 con 0x7f3f68100490 2024-05-13T22:49:50.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.065+0000 7f3f67fff700 1 --2- 172.21.15.102:0/1382678010 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3f5006dfc0 0x7f3f50070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:49:50.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.065+0000 7f3f67fff700 1 --2- 172.21.15.102:0/1382678010 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3f5006dfc0 0x7f3f50070480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3f58005090 tx=0x7f3f5801a040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:49:50.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.070+0000 7f3f657fa700 1 -- 172.21.15.102:0/1382678010 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f3f5c05b1b0 con 0x7f3f68100490 2024-05-13T22:49:50.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.351+0000 7f3f6e313700 1 -- 172.21.15.102:0/1382678010 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f54000bf0 con 0x7f3f5006dfc0 2024-05-13T22:49:50.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.352+0000 7f3f657fa700 1 -- 172.21.15.102:0/1382678010 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f3f54000bf0 con 0x7f3f5006dfc0 2024-05-13T22:49:50.331 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:49:50.331 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:49:50.331 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:49:50.331 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-13T22:49:50.332 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:49:50.332 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr" 2024-05-13T22:49:50.332 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:49:50.332 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "2/2 daemons upgraded", 2024-05-13T22:49:50.332 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading node-exporter daemons", 2024-05-13T22:49:50.332 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:49:50.332 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:49:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.355+0000 7f3f6e313700 1 -- 172.21.15.102:0/1382678010 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3f5006dfc0 msgr2=0x7f3f50070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.355+0000 7f3f6e313700 1 --2- 172.21.15.102:0/1382678010 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3f5006dfc0 0x7f3f50070480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3f58005090 tx=0x7f3f5801a040 comp rx=0 tx=0).stop 2024-05-13T22:49:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.355+0000 7f3f6e313700 1 -- 172.21.15.102:0/1382678010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f68100490 msgr2=0x7f3f6819b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:49:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.355+0000 7f3f6e313700 1 --2- 172.21.15.102:0/1382678010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f68100490 0x7f3f6819b150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00d8d0 tx=0x7f3f5c00dc90 comp rx=0 tx=0).stop 2024-05-13T22:49:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.356+0000 7f3f6e313700 1 -- 172.21.15.102:0/1382678010 shutdown_connections 2024-05-13T22:49:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.356+0000 7f3f6e313700 1 --2- 172.21.15.102:0/1382678010 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3f5006dfc0 0x7f3f50070480 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.356+0000 7f3f6e313700 1 --2- 172.21.15.102:0/1382678010 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f680ff290 0x7f3f68106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.356+0000 7f3f6e313700 1 --2- 172.21.15.102:0/1382678010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f68100490 0x7f3f6819b150 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:49:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.356+0000 7f3f6e313700 1 -- 172.21.15.102:0/1382678010 >> 172.21.15.102:0/1382678010 conn(0x7f3f680fa810 msgr2=0x7f3f681036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:49:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.356+0000 7f3f6e313700 1 -- 172.21.15.102:0/1382678010 shutdown_connections 2024-05-13T22:49:50.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:49:50.356+0000 7f3f6e313700 1 -- 172.21.15.102:0/1382678010 wait complete. 2024-05-13T22:49:50.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:50 smithi130 ceph-mon[36591]: from='client.24507 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:50.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:50 smithi130 ceph-mon[36591]: pgmap v33: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 459 op/s 2024-05-13T22:49:50.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:49:50.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:50 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1734804633' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:50.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:50 smithi102 ceph-mon[28599]: from='client.24507 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:50.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:50 smithi102 ceph-mon[28599]: pgmap v33: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 459 op/s 2024-05-13T22:49:50.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:49:50.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:50 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1734804633' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:49:51.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:51 smithi130 ceph-mon[36591]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:51.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:51 smithi130 ceph-mon[36591]: Upgrade: Updating node-exporter.smithi102 (1/2) 2024-05-13T22:49:51.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:51 smithi130 ceph-mon[36591]: Deploying daemon node-exporter.smithi102 on smithi102 2024-05-13T22:49:51.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:51 smithi102 ceph-mon[28599]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:49:51.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:51 smithi102 ceph-mon[28599]: Upgrade: Updating node-exporter.smithi102 (1/2) 2024-05-13T22:49:51.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:51 smithi102 ceph-mon[28599]: Deploying daemon node-exporter.smithi102 on smithi102 2024-05-13T22:49:52.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:52 smithi130 ceph-mon[36591]: pgmap v34: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 618 op/s 2024-05-13T22:49:52.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:52 smithi102 ceph-mon[28599]: pgmap v34: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 618 op/s 2024-05-13T22:49:54.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:54 smithi102 ceph-mon[28599]: pgmap v35: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 342 op/s 2024-05-13T22:49:54.680 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:54 smithi130 ceph-mon[36591]: pgmap v35: 65 pgs: 65 active+clean; 265 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 342 op/s 2024-05-13T22:49:56.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:56 smithi102 ceph-mon[28599]: pgmap v36: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 454 op/s 2024-05-13T22:49:56.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:56 smithi130 ceph-mon[36591]: pgmap v36: 65 pgs: 65 active+clean; 262 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 454 op/s 2024-05-13T22:49:58.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:49:58 smithi102 ceph-mon[28599]: pgmap v37: 65 pgs: 65 active+clean; 256 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 455 op/s 2024-05-13T22:49:58.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:49:58 smithi130 ceph-mon[36591]: pgmap v37: 65 pgs: 65 active+clean; 256 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 455 op/s 2024-05-13T22:50:00.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:00 smithi102 ceph-mon[28599]: pgmap v38: 65 pgs: 65 active+clean; 256 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 920 KiB/s rd, 1.0 MiB/s wr, 313 op/s 2024-05-13T22:50:00.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:00 smithi102 ceph-mon[28599]: overall HEALTH_OK 2024-05-13T22:50:00.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:00 smithi130 ceph-mon[36591]: pgmap v38: 65 pgs: 65 active+clean; 256 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 920 KiB/s rd, 1.0 MiB/s wr, 313 op/s 2024-05-13T22:50:00.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:00 smithi130 ceph-mon[36591]: overall HEALTH_OK 2024-05-13T22:50:02.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:02 smithi102 ceph-mon[28599]: pgmap v39: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 460 op/s 2024-05-13T22:50:02.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:02 smithi130 ceph-mon[36591]: pgmap v39: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 460 op/s 2024-05-13T22:50:04.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:04 smithi130 ceph-mon[36591]: pgmap v40: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 952 KiB/s rd, 993 KiB/s wr, 301 op/s 2024-05-13T22:50:04.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:04 smithi102 ceph-mon[28599]: pgmap v40: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 952 KiB/s rd, 993 KiB/s wr, 301 op/s 2024-05-13T22:50:05.478 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:05 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:50:05.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:05 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:50:06.484 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:06 smithi102 ceph-mon[28599]: pgmap v41: 65 pgs: 65 active+clean; 239 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 393 op/s 2024-05-13T22:50:06.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:06 smithi130 ceph-mon[36591]: pgmap v41: 65 pgs: 65 active+clean; 239 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 393 op/s 2024-05-13T22:50:08.290 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-05-13T22:50:08.290 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-13T22:50:08.623 DEBUG:teuthology.parallel:result is None 2024-05-13T22:50:08.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:08 smithi130 ceph-mon[36591]: pgmap v42: 65 pgs: 65 active+clean; 241 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 942 KiB/s rd, 983 KiB/s wr, 308 op/s 2024-05-13T22:50:08.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:08 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:08.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:08 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:08.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:08 smithi102 ceph-mon[28599]: pgmap v42: 65 pgs: 65 active+clean; 241 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 942 KiB/s rd, 983 KiB/s wr, 308 op/s 2024-05-13T22:50:08.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:08 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:08.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:08 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:10.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:10 smithi102 ceph-mon[28599]: pgmap v43: 65 pgs: 65 active+clean; 241 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 942 KiB/s rd, 982 KiB/s wr, 265 op/s 2024-05-13T22:50:10.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:10 smithi102 ceph-mon[28599]: Upgrade: Updating node-exporter.smithi130 (2/2) 2024-05-13T22:50:10.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:10 smithi102 ceph-mon[28599]: Deploying daemon node-exporter.smithi130 on smithi130 2024-05-13T22:50:11.069 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:10 smithi130 ceph-mon[36591]: pgmap v43: 65 pgs: 65 active+clean; 241 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 942 KiB/s rd, 982 KiB/s wr, 265 op/s 2024-05-13T22:50:11.069 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:10 smithi130 ceph-mon[36591]: Upgrade: Updating node-exporter.smithi130 (2/2) 2024-05-13T22:50:11.070 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:10 smithi130 ceph-mon[36591]: Deploying daemon node-exporter.smithi130 on smithi130 2024-05-13T22:50:12.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:12 smithi102 ceph-mon[28599]: pgmap v44: 65 pgs: 65 active+clean; 230 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 417 op/s 2024-05-13T22:50:12.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:12 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:12.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:12 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:12.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:12 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:50:13.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:12 smithi130 ceph-mon[36591]: pgmap v44: 65 pgs: 65 active+clean; 230 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 417 op/s 2024-05-13T22:50:13.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:12 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:13.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:12 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:13.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:12 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:50:14.480 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-05-13T22:50:14.481 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-05-13T22:50:14.793 DEBUG:teuthology.parallel:result is None 2024-05-13T22:50:14.793 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-13T22:50:14.827 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-13T22:50:14.828 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-05-13T22:50:14.861 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-05-13T22:50:14.862 DEBUG:teuthology.parallel:result is None 2024-05-13T22:50:14.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:14 smithi102 ceph-mon[28599]: pgmap v45: 65 pgs: 65 active+clean; 230 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 967 KiB/s rd, 1001 KiB/s wr, 270 op/s 2024-05-13T22:50:14.963 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:14 smithi130 ceph-mon[36591]: pgmap v45: 65 pgs: 65 active+clean; 230 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 967 KiB/s rd, 1001 KiB/s wr, 270 op/s 2024-05-13T22:50:17.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:17 smithi130 ceph-mon[36591]: pgmap v46: 65 pgs: 65 active+clean; 224 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 381 op/s 2024-05-13T22:50:17.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:17 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:17.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:17 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:17.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:17 smithi102 ceph-mon[28599]: pgmap v46: 65 pgs: 65 active+clean; 224 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 381 op/s 2024-05-13T22:50:17.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:17 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:17.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:17 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:18.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:18 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:18.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:18 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:18.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:18 smithi130 ceph-mon[36591]: pgmap v47: 65 pgs: 65 active+clean; 224 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 307 op/s 2024-05-13T22:50:18.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:18 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:18.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:18 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:18.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:18 smithi102 ceph-mon[28599]: pgmap v47: 65 pgs: 65 active+clean; 224 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 307 op/s 2024-05-13T22:50:20.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.469+0000 7f79c7115700 1 -- 172.21.15.102:0/1507872433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79c0100450 msgr2=0x7f79c01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:20.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.469+0000 7f79c7115700 1 --2- 172.21.15.102:0/1507872433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79c0100450 0x7f79c01008b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009910 tx=0x7f79ac009c20 comp rx=0 tx=0).stop 2024-05-13T22:50:20.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 -- 172.21.15.102:0/1507872433 shutdown_connections 2024-05-13T22:50:20.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 --2- 172.21.15.102:0/1507872433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79c0100450 0x7f79c01008b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:20.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 --2- 172.21.15.102:0/1507872433 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79c00ff250 0x7f79c00ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:20.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 -- 172.21.15.102:0/1507872433 >> 172.21.15.102:0/1507872433 conn(0x7f79c00fa7f0 msgr2=0x7f79c00fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:20.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 -- 172.21.15.102:0/1507872433 shutdown_connections 2024-05-13T22:50:20.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 -- 172.21.15.102:0/1507872433 wait complete. 2024-05-13T22:50:20.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 Processor -- start 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 -- start start 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79c00ff250 0x7f79c01021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79c0100450 0x7f79c019b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c019b670 con 0x7f79c0100450 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.470+0000 7f79c7115700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c019b7b0 con 0x7f79c00ff250 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79bffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79c0100450 0x7f79c019b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79bffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79c0100450 0x7f79c019b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41366/0 (socket says 172.21.15.102:41366) 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79bffff700 1 -- 172.21.15.102:0/594286901 learned_addr learned my addr 172.21.15.102:0/594286901 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79c4eb1700 1 --2- 172.21.15.102:0/594286901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79c00ff250 0x7f79c01021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79bffff700 1 -- 172.21.15.102:0/594286901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79c00ff250 msgr2=0x7f79c01021d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79bffff700 1 --2- 172.21.15.102:0/594286901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79c00ff250 0x7f79c01021d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79bffff700 1 -- 172.21.15.102:0/594286901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac0095f0 con 0x7f79c0100450 2024-05-13T22:50:20.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79bffff700 1 --2- 172.21.15.102:0/594286901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79c0100450 0x7f79c019b0a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00bd70 tx=0x7f79ac00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:20.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79bdffb700 1 -- 172.21.15.102:0/594286901 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79ac01d070 con 0x7f79c0100450 2024-05-13T22:50:20.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79bdffb700 1 -- 172.21.15.102:0/594286901 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f79ac005dc0 con 0x7f79c0100450 2024-05-13T22:50:20.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79c7115700 1 -- 172.21.15.102:0/594286901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c019ba30 con 0x7f79c0100450 2024-05-13T22:50:20.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79bdffb700 1 -- 172.21.15.102:0/594286901 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79ac021410 con 0x7f79c0100450 2024-05-13T22:50:20.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.471+0000 7f79c7115700 1 -- 172.21.15.102:0/594286901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c019bf20 con 0x7f79c0100450 2024-05-13T22:50:20.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.472+0000 7f79c7115700 1 -- 172.21.15.102:0/594286901 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c0107db0 con 0x7f79c0100450 2024-05-13T22:50:20.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.473+0000 7f79bdffb700 1 -- 172.21.15.102:0/594286901 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f79ac00f7d0 con 0x7f79c0100450 2024-05-13T22:50:20.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.474+0000 7f79bdffb700 1 --2- 172.21.15.102:0/594286901 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f79b006dfc0 0x7f79b0070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:20.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.474+0000 7f79bdffb700 1 -- 172.21.15.102:0/594286901 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f79ac08f4c0 con 0x7f79c0100450 2024-05-13T22:50:20.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.474+0000 7f79c4eb1700 1 --2- 172.21.15.102:0/594286901 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f79b006dfc0 0x7f79b0070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:20.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.474+0000 7f79c4eb1700 1 --2- 172.21.15.102:0/594286901 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f79b006dfc0 0x7f79b0070480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f79c01002b0 tx=0x7f79b4007400 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:20.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.478+0000 7f79bdffb700 1 -- 172.21.15.102:0/594286901 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f79ac05c5c0 con 0x7f79c0100450 2024-05-13T22:50:20.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:20 smithi130 ceph-mon[36591]: pgmap v48: 65 pgs: 65 active+clean; 224 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 925 KiB/s rd, 873 KiB/s wr, 280 op/s 2024-05-13T22:50:20.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:20 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:50:20.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:20 smithi102 ceph-mon[28599]: pgmap v48: 65 pgs: 65 active+clean; 224 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 925 KiB/s rd, 873 KiB/s wr, 280 op/s 2024-05-13T22:50:20.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:20 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:50:20.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.701+0000 7f79c7115700 1 -- 172.21.15.102:0/594286901 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79c0104790 con 0x7f79b006dfc0 2024-05-13T22:50:20.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.703+0000 7f79bdffb700 1 -- 172.21.15.102:0/594286901 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f79c0104790 con 0x7f79b006dfc0 2024-05-13T22:50:20.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.705+0000 7f79c7115700 1 -- 172.21.15.102:0/594286901 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f79b006dfc0 msgr2=0x7f79b0070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:20.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.705+0000 7f79c7115700 1 --2- 172.21.15.102:0/594286901 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f79b006dfc0 0x7f79b0070480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f79c01002b0 tx=0x7f79b4007400 comp rx=0 tx=0).stop 2024-05-13T22:50:20.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.705+0000 7f79c7115700 1 -- 172.21.15.102:0/594286901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79c0100450 msgr2=0x7f79c019b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:20.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.705+0000 7f79c7115700 1 --2- 172.21.15.102:0/594286901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79c0100450 0x7f79c019b0a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00bd70 tx=0x7f79ac00bda0 comp rx=0 tx=0).stop 2024-05-13T22:50:20.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.705+0000 7f79c7115700 1 -- 172.21.15.102:0/594286901 shutdown_connections 2024-05-13T22:50:20.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.705+0000 7f79c7115700 1 --2- 172.21.15.102:0/594286901 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f79b006dfc0 0x7f79b0070480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:20.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.705+0000 7f79c7115700 1 --2- 172.21.15.102:0/594286901 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79c00ff250 0x7f79c01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:20.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.705+0000 7f79c7115700 1 --2- 172.21.15.102:0/594286901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79c0100450 0x7f79c019b0a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:20.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.705+0000 7f79c7115700 1 -- 172.21.15.102:0/594286901 >> 172.21.15.102:0/594286901 conn(0x7f79c00fa7f0 msgr2=0x7f79c00fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:20.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.705+0000 7f79c7115700 1 -- 172.21.15.102:0/594286901 shutdown_connections 2024-05-13T22:50:20.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.705+0000 7f79c7115700 1 -- 172.21.15.102:0/594286901 wait complete. 2024-05-13T22:50:20.693 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:50:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.819+0000 7f791af77700 1 -- 172.21.15.102:0/2973232356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79141016a0 msgr2=0x7f7914103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.819+0000 7f791af77700 1 --2- 172.21.15.102:0/2973232356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79141016a0 0x7f7914103a90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7908009910 tx=0x7f7908009c20 comp rx=0 tx=0).stop 2024-05-13T22:50:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.819+0000 7f791af77700 1 -- 172.21.15.102:0/2973232356 shutdown_connections 2024-05-13T22:50:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.819+0000 7f791af77700 1 --2- 172.21.15.102:0/2973232356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7914103fd0 0x7f79141063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.819+0000 7f791af77700 1 --2- 172.21.15.102:0/2973232356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79141016a0 0x7f7914103a90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.819+0000 7f791af77700 1 -- 172.21.15.102:0/2973232356 >> 172.21.15.102:0/2973232356 conn(0x7f79140fb030 msgr2=0x7f79140fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.819+0000 7f791af77700 1 -- 172.21.15.102:0/2973232356 shutdown_connections 2024-05-13T22:50:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.819+0000 7f791af77700 1 -- 172.21.15.102:0/2973232356 wait complete. 2024-05-13T22:50:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.819+0000 7f791af77700 1 Processor -- start 2024-05-13T22:50:20.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.819+0000 7f791af77700 1 -- start start 2024-05-13T22:50:20.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f791af77700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79141016a0 0x7f7914196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f791af77700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7914103fd0 0x7f7914196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f791af77700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7914196e70 con 0x7f7914103fd0 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f791af77700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7914196fb0 con 0x7f79141016a0 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f7913fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7914103fd0 0x7f7914196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f7913fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7914103fd0 0x7f7914196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41380/0 (socket says 172.21.15.102:41380) 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f7913fff700 1 -- 172.21.15.102:0/3779623861 learned_addr learned my addr 172.21.15.102:0/3779623861 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f7918d13700 1 --2- 172.21.15.102:0/3779623861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79141016a0 0x7f7914196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f7913fff700 1 -- 172.21.15.102:0/3779623861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79141016a0 msgr2=0x7f7914196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f7913fff700 1 --2- 172.21.15.102:0/3779623861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79141016a0 0x7f7914196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f7913fff700 1 -- 172.21.15.102:0/3779623861 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79080095f0 con 0x7f7914103fd0 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f7913fff700 1 --2- 172.21.15.102:0/3779623861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7914103fd0 0x7f7914196850 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f790000d8d0 tx=0x7f790000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f7911ffb700 1 -- 172.21.15.102:0/3779623861 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79000098a0 con 0x7f7914103fd0 2024-05-13T22:50:20.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f791af77700 1 -- 172.21.15.102:0/3779623861 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791419ba60 con 0x7f7914103fd0 2024-05-13T22:50:20.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f7911ffb700 1 -- 172.21.15.102:0/3779623861 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7900010460 con 0x7f7914103fd0 2024-05-13T22:50:20.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.820+0000 7f7911ffb700 1 -- 172.21.15.102:0/3779623861 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f790000f5f0 con 0x7f7914103fd0 2024-05-13T22:50:20.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.821+0000 7f791af77700 1 -- 172.21.15.102:0/3779623861 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791419c030 con 0x7f7914103fd0 2024-05-13T22:50:20.800 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.822+0000 7f791af77700 1 -- 172.21.15.102:0/3779623861 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7914190520 con 0x7f7914103fd0 2024-05-13T22:50:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.822+0000 7f7911ffb700 1 -- 172.21.15.102:0/3779623861 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f7900009a00 con 0x7f7914103fd0 2024-05-13T22:50:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.823+0000 7f7911ffb700 1 --2- 172.21.15.102:0/3779623861 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f790406dfc0 0x7f7904070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.823+0000 7f7911ffb700 1 -- 172.21.15.102:0/3779623861 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f790008dd10 con 0x7f7914103fd0 2024-05-13T22:50:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.823+0000 7f7918d13700 1 --2- 172.21.15.102:0/3779623861 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f790406dfc0 0x7f7904070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:20.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.823+0000 7f7918d13700 1 --2- 172.21.15.102:0/3779623861 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f790406dfc0 0x7f7904070480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f790800b5c0 tx=0x7f7908005ab0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:20.805 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:20.827+0000 7f7911ffb700 1 -- 172.21.15.102:0/3779623861 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f790005ae30 con 0x7f7914103fd0 2024-05-13T22:50:21.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.090+0000 7f791af77700 1 -- 172.21.15.102:0/3779623861 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79140611d0 con 0x7f790406dfc0 2024-05-13T22:50:21.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.092+0000 7f7911ffb700 1 -- 172.21.15.102:0/3779623861 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f79140611d0 con 0x7f790406dfc0 2024-05-13T22:50:21.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.095+0000 7f791af77700 1 -- 172.21.15.102:0/3779623861 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f790406dfc0 msgr2=0x7f7904070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.095+0000 7f791af77700 1 --2- 172.21.15.102:0/3779623861 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f790406dfc0 0x7f7904070480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f790800b5c0 tx=0x7f7908005ab0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.095+0000 7f791af77700 1 -- 172.21.15.102:0/3779623861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7914103fd0 msgr2=0x7f7914196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.095+0000 7f791af77700 1 --2- 172.21.15.102:0/3779623861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7914103fd0 0x7f7914196850 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f790000d8d0 tx=0x7f790000dbe0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.095+0000 7f791af77700 1 -- 172.21.15.102:0/3779623861 shutdown_connections 2024-05-13T22:50:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.095+0000 7f791af77700 1 --2- 172.21.15.102:0/3779623861 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f790406dfc0 0x7f7904070480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.095+0000 7f791af77700 1 --2- 172.21.15.102:0/3779623861 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f79141016a0 0x7f7914196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.095+0000 7f791af77700 1 --2- 172.21.15.102:0/3779623861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7914103fd0 0x7f7914196850 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.095+0000 7f791af77700 1 -- 172.21.15.102:0/3779623861 >> 172.21.15.102:0/3779623861 conn(0x7f79140fb030 msgr2=0x7f79140fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.095+0000 7f791af77700 1 -- 172.21.15.102:0/3779623861 shutdown_connections 2024-05-13T22:50:21.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.095+0000 7f791af77700 1 -- 172.21.15.102:0/3779623861 wait complete. 2024-05-13T22:50:21.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.202+0000 7f05b46e2700 1 -- 172.21.15.102:0/2837099067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05ac1016e0 msgr2=0x7f05ac103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.202+0000 7f05b46e2700 1 --2- 172.21.15.102:0/2837099067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05ac1016e0 0x7f05ac103ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009910 tx=0x7f05a8009c20 comp rx=0 tx=0).stop 2024-05-13T22:50:21.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.202+0000 7f05b46e2700 1 -- 172.21.15.102:0/2837099067 shutdown_connections 2024-05-13T22:50:21.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.202+0000 7f05b46e2700 1 --2- 172.21.15.102:0/2837099067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ac104010 0x7f05ac106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.202+0000 7f05b46e2700 1 --2- 172.21.15.102:0/2837099067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05ac1016e0 0x7f05ac103ad0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.202+0000 7f05b46e2700 1 -- 172.21.15.102:0/2837099067 >> 172.21.15.102:0/2837099067 conn(0x7f05ac0fb030 msgr2=0x7f05ac0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:21.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.202+0000 7f05b46e2700 1 -- 172.21.15.102:0/2837099067 shutdown_connections 2024-05-13T22:50:21.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.202+0000 7f05b46e2700 1 -- 172.21.15.102:0/2837099067 wait complete. 2024-05-13T22:50:21.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.203+0000 7f05b46e2700 1 Processor -- start 2024-05-13T22:50:21.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.203+0000 7f05b46e2700 1 -- start start 2024-05-13T22:50:21.182 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.203+0000 7f05b46e2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ac1016e0 0x7f05ac196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.203+0000 7f05b46e2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05ac104010 0x7f05ac1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.203+0000 7f05b46e2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac196ee0 con 0x7f05ac104010 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.203+0000 7f05b46e2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac197020 con 0x7f05ac1016e0 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.203+0000 7f05b1c7d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05ac104010 0x7f05ac1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.203+0000 7f05b1c7d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05ac104010 0x7f05ac1968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41406/0 (socket says 172.21.15.102:41406) 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.203+0000 7f05b1c7d700 1 -- 172.21.15.102:0/3999818042 learned_addr learned my addr 172.21.15.102:0/3999818042 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.203+0000 7f05b247e700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ac1016e0 0x7f05ac196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.204+0000 7f05b1c7d700 1 -- 172.21.15.102:0/3999818042 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ac1016e0 msgr2=0x7f05ac196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.204+0000 7f05b1c7d700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ac1016e0 0x7f05ac196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.204+0000 7f05b1c7d700 1 -- 172.21.15.102:0/3999818042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80095f0 con 0x7f05ac104010 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.204+0000 7f05b247e700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ac1016e0 0x7f05ac196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.204+0000 7f05b1c7d700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05ac104010 0x7f05ac1968c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f059c00b4b0 tx=0x7f059c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.204+0000 7f05a37fe700 1 -- 172.21.15.102:0/3999818042 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f059c011800 con 0x7f05ac104010 2024-05-13T22:50:21.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.204+0000 7f05a37fe700 1 -- 172.21.15.102:0/3999818042 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f059c011e10 con 0x7f05ac104010 2024-05-13T22:50:21.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.204+0000 7f05b46e2700 1 -- 172.21.15.102:0/3999818042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac19bad0 con 0x7f05ac104010 2024-05-13T22:50:21.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.204+0000 7f05a37fe700 1 -- 172.21.15.102:0/3999818042 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f059c00f2e0 con 0x7f05ac104010 2024-05-13T22:50:21.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.204+0000 7f05b46e2700 1 -- 172.21.15.102:0/3999818042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac19c0a0 con 0x7f05ac104010 2024-05-13T22:50:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.205+0000 7f05b46e2700 1 -- 172.21.15.102:0/3999818042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac1905b0 con 0x7f05ac104010 2024-05-13T22:50:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.206+0000 7f05a37fe700 1 -- 172.21.15.102:0/3999818042 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f059c010380 con 0x7f05ac104010 2024-05-13T22:50:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.206+0000 7f05a37fe700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f059806dfb0 0x7f0598070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.207+0000 7f05a37fe700 1 -- 172.21.15.102:0/3999818042 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f059c08d9a0 con 0x7f05ac104010 2024-05-13T22:50:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.207+0000 7f05b247e700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f059806dfb0 0x7f0598070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:21.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.207+0000 7f05b247e700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f059806dfb0 0x7f0598070470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f05a8006010 tx=0x7f05a800b540 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:21.188 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.211+0000 7f05a37fe700 1 -- 172.21.15.102:0/3999818042 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f059c05abd0 con 0x7f05ac104010 2024-05-13T22:50:21.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.426+0000 7f05b46e2700 1 -- 172.21.15.102:0/3999818042 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f05ac0611d0 con 0x7f059806dfb0 2024-05-13T22:50:21.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.442+0000 7f05a37fe700 1 -- 172.21.15.102:0/3999818042 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f05ac0611d0 con 0x7f059806dfb0 2024-05-13T22:50:21.419 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:50:21.419 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (9m) 0s ago 10m 25.6M - 0.25.0 c8568f914cd2 c096e9e9a228 2024-05-13T22:50:21.419 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (10m) 0s ago 10m 7964k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (9m) 5s ago 9m 7662k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (10m) 0s ago 10m 6660k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (9m) 5s ago 9m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (8m) 0s ago 10m 89.0M - 9.4.7 954c08fa6188 bca1790c17aa 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (5m) 0s ago 5m 101M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (5m) 0s ago 5m 168M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (5m) 5s ago 5m 14.3M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (4m) 5s ago 4m 17.5M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (105s) 0s ago 11m 497M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (43s) 5s ago 9m 450M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (12m) 0s ago 12m 53.1M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (9m) 5s ago 9m 46.6M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:50:21.420 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (18s) 0s ago 10m 8040k - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:50:21.421 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (9s) 5s ago 9m 3670k - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:50:21.421 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (8m) 0s ago 8m 220M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:50:21.421 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (8m) 0s ago 8m 236M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:50:21.421 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (7m) 0s ago 7m 200M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:50:21.421 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (7m) 5s ago 7m 288M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:50:21.421 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (7m) 5s ago 6m 244M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:50:21.421 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (6m) 5s ago 6m 224M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:50:21.421 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (49s) 0s ago 10m 63.3M - 2.43.0 a07b618ecd1d 99aa6bee5d86 2024-05-13T22:50:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.445+0000 7f05b46e2700 1 -- 172.21.15.102:0/3999818042 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f059806dfb0 msgr2=0x7f0598070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.445+0000 7f05b46e2700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f059806dfb0 0x7f0598070470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f05a8006010 tx=0x7f05a800b540 comp rx=0 tx=0).stop 2024-05-13T22:50:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.445+0000 7f05b46e2700 1 -- 172.21.15.102:0/3999818042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05ac104010 msgr2=0x7f05ac1968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.445+0000 7f05b46e2700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05ac104010 0x7f05ac1968c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f059c00b4b0 tx=0x7f059c00b870 comp rx=0 tx=0).stop 2024-05-13T22:50:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.445+0000 7f05b46e2700 1 -- 172.21.15.102:0/3999818042 shutdown_connections 2024-05-13T22:50:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.445+0000 7f05b46e2700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f059806dfb0 0x7f0598070470 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.445+0000 7f05b46e2700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05ac1016e0 0x7f05ac196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.445+0000 7f05b46e2700 1 --2- 172.21.15.102:0/3999818042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05ac104010 0x7f05ac1968c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.445+0000 7f05b46e2700 1 -- 172.21.15.102:0/3999818042 >> 172.21.15.102:0/3999818042 conn(0x7f05ac0fb030 msgr2=0x7f05ac0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.445+0000 7f05b46e2700 1 -- 172.21.15.102:0/3999818042 shutdown_connections 2024-05-13T22:50:21.423 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.445+0000 7f05b46e2700 1 -- 172.21.15.102:0/3999818042 wait complete. 2024-05-13T22:50:21.539 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.561+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4063046577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c80ffdf0 msgr2=0x7ff6c8100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.561+0000 7ff6cd24a700 1 --2- 172.21.15.102:0/4063046577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c80ffdf0 0x7ff6c8100270 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009910 tx=0x7ff6bc009c20 comp rx=0 tx=0).stop 2024-05-13T22:50:21.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.561+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4063046577 shutdown_connections 2024-05-13T22:50:21.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.561+0000 7ff6cd24a700 1 --2- 172.21.15.102:0/4063046577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c80ffdf0 0x7ff6c8100270 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.561+0000 7ff6cd24a700 1 --2- 172.21.15.102:0/4063046577 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c80ff490 0x7ff6c80ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.561+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4063046577 >> 172.21.15.102:0/4063046577 conn(0x7ff6c80fb050 msgr2=0x7ff6c80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:21.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.561+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4063046577 shutdown_connections 2024-05-13T22:50:21.540 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.561+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4063046577 wait complete. 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.562+0000 7ff6cd24a700 1 Processor -- start 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6cd24a700 1 -- start start 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6cd24a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c80ff490 0x7ff6c8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6cd24a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c80ffdf0 0x7ff6c8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6cd24a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c8199090 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6cd24a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c81991d0 con 0x7ff6c80ff490 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6c659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c80ffdf0 0x7ff6c8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6c659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c80ffdf0 0x7ff6c8198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41426/0 (socket says 172.21.15.102:41426) 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6c659c700 1 -- 172.21.15.102:0/4048201229 learned_addr learned my addr 172.21.15.102:0/4048201229 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6c6d9d700 1 --2- 172.21.15.102:0/4048201229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c80ff490 0x7ff6c8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6c659c700 1 -- 172.21.15.102:0/4048201229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c80ff490 msgr2=0x7ff6c8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.542 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6c659c700 1 --2- 172.21.15.102:0/4048201229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c80ff490 0x7ff6c8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.563+0000 7ff6c659c700 1 -- 172.21.15.102:0/4048201229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc0095f0 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.564+0000 7ff6c659c700 1 --2- 172.21.15.102:0/4048201229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c80ffdf0 0x7ff6c8198a70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00bd70 tx=0x7ff6bc00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:21.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.564+0000 7ff6affff700 1 -- 172.21.15.102:0/4048201229 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6bc01d070 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.564+0000 7ff6affff700 1 -- 172.21.15.102:0/4048201229 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff6bc005940 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.564+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4048201229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c819dc20 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.564+0000 7ff6affff700 1 -- 172.21.15.102:0/4048201229 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6bc021430 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.543 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.564+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4048201229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c819e110 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.565+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4048201229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c810b910 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.566+0000 7ff6affff700 1 -- 172.21.15.102:0/4048201229 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7ff6bc00f860 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.566+0000 7ff6affff700 1 --2- 172.21.15.102:0/4048201229 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff6b006de80 0x7ff6b0070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:21.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.566+0000 7ff6affff700 1 -- 172.21.15.102:0/4048201229 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff6bc08f090 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.567+0000 7ff6c6d9d700 1 --2- 172.21.15.102:0/4048201229 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff6b006de80 0x7ff6b0070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:21.544 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.567+0000 7ff6c6d9d700 1 --2- 172.21.15.102:0/4048201229 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff6b006de80 0x7ff6b0070340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8005950 tx=0x7ff6b800a300 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:21.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.571+0000 7ff6affff700 1 -- 172.21.15.102:0/4048201229 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7ff6bc05c330 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:21 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:21.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:21 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:21.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:21 smithi102 ceph-mon[28599]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:21.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:21 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:21.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:21 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:21.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:21 smithi102 ceph-mon[28599]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:21.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.871+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4048201229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff6c810bb00 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.871+0000 7ff6affff700 1 -- 172.21.15.102:0/4048201229 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7ff6bc05bec0 con 0x7ff6c80ffdf0 2024-05-13T22:50:21.849 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:50:21.849 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:50:21.849 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:50:21.849 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:50:21.849 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:50:21.849 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:50:21.849 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:50:21.849 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:50:21.849 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:50:21.850 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:50:21.850 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:50:21.850 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:50:21.850 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:50:21.850 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:50:21.850 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-05-13T22:50:21.850 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:50:21.850 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:50:21.850 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:50:21.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.873+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4048201229 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff6b006de80 msgr2=0x7ff6b0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.873+0000 7ff6cd24a700 1 --2- 172.21.15.102:0/4048201229 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff6b006de80 0x7ff6b0070340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8005950 tx=0x7ff6b800a300 comp rx=0 tx=0).stop 2024-05-13T22:50:21.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.873+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4048201229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c80ffdf0 msgr2=0x7ff6c8198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.873+0000 7ff6cd24a700 1 --2- 172.21.15.102:0/4048201229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c80ffdf0 0x7ff6c8198a70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00bd70 tx=0x7ff6bc00bda0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.874+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4048201229 shutdown_connections 2024-05-13T22:50:21.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.874+0000 7ff6cd24a700 1 --2- 172.21.15.102:0/4048201229 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ff6b006de80 0x7ff6b0070340 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.874+0000 7ff6cd24a700 1 --2- 172.21.15.102:0/4048201229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6c80ff490 0x7ff6c8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.874+0000 7ff6cd24a700 1 --2- 172.21.15.102:0/4048201229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6c80ffdf0 0x7ff6c8198a70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.874+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4048201229 >> 172.21.15.102:0/4048201229 conn(0x7ff6c80fb050 msgr2=0x7ff6c8107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:21.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.874+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4048201229 shutdown_connections 2024-05-13T22:50:21.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.874+0000 7ff6cd24a700 1 -- 172.21.15.102:0/4048201229 wait complete. 2024-05-13T22:50:21.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:21 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:21.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:21 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:21.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:21 smithi130 ceph-mon[36591]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:21.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:21 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:21.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:21 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:21.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:21 smithi130 ceph-mon[36591]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:21.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.982+0000 7fa35e5d7700 1 -- 172.21.15.102:0/1372885026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3580ffad0 msgr2=0x7fa3580ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.982+0000 7fa35e5d7700 1 --2- 172.21.15.102:0/1372885026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3580ffad0 0x7fa3580ffef0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa348009910 tx=0x7fa348009c20 comp rx=0 tx=0).stop 2024-05-13T22:50:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.982+0000 7fa35e5d7700 1 -- 172.21.15.102:0/1372885026 shutdown_connections 2024-05-13T22:50:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.982+0000 7fa35e5d7700 1 --2- 172.21.15.102:0/1372885026 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa358100cd0 0x7fa358101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.982+0000 7fa35e5d7700 1 --2- 172.21.15.102:0/1372885026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3580ffad0 0x7fa3580ffef0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.982+0000 7fa35e5d7700 1 -- 172.21.15.102:0/1372885026 >> 172.21.15.102:0/1372885026 conn(0x7fa3580fb050 msgr2=0x7fa3580fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.982+0000 7fa35e5d7700 1 -- 172.21.15.102:0/1372885026 shutdown_connections 2024-05-13T22:50:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.982+0000 7fa35e5d7700 1 -- 172.21.15.102:0/1372885026 wait complete. 2024-05-13T22:50:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.983+0000 7fa35e5d7700 1 Processor -- start 2024-05-13T22:50:21.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.983+0000 7fa35e5d7700 1 -- start start 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.983+0000 7fa35e5d7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3580ffad0 0x7fa358196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.983+0000 7fa35e5d7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa358100cd0 0x7fa3581968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.983+0000 7fa35e5d7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa358196ef0 con 0x7fa358100cd0 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.983+0000 7fa35e5d7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa358197030 con 0x7fa3580ffad0 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.983+0000 7fa3577fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa358100cd0 0x7fa3581968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.983+0000 7fa3577fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa358100cd0 0x7fa3581968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41446/0 (socket says 172.21.15.102:41446) 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.983+0000 7fa3577fe700 1 -- 172.21.15.102:0/2421959528 learned_addr learned my addr 172.21.15.102:0/2421959528 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.984+0000 7fa357fff700 1 --2- 172.21.15.102:0/2421959528 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3580ffad0 0x7fa358196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.984+0000 7fa3577fe700 1 -- 172.21.15.102:0/2421959528 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3580ffad0 msgr2=0x7fa358196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.984+0000 7fa3577fe700 1 --2- 172.21.15.102:0/2421959528 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3580ffad0 0x7fa358196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.984+0000 7fa3577fe700 1 -- 172.21.15.102:0/2421959528 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3480095f0 con 0x7fa358100cd0 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.984+0000 7fa3577fe700 1 --2- 172.21.15.102:0/2421959528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa358100cd0 0x7fa3581968d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00c8a0 tx=0x7fa34c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.984+0000 7fa3557fa700 1 -- 172.21.15.102:0/2421959528 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa34c00cea0 con 0x7fa358100cd0 2024-05-13T22:50:21.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.984+0000 7fa35e5d7700 1 -- 172.21.15.102:0/2421959528 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35819bae0 con 0x7fa358100cd0 2024-05-13T22:50:21.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.984+0000 7fa3557fa700 1 -- 172.21.15.102:0/2421959528 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa34c004830 con 0x7fa358100cd0 2024-05-13T22:50:21.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.984+0000 7fa3557fa700 1 -- 172.21.15.102:0/2421959528 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa34c0055b0 con 0x7fa358100cd0 2024-05-13T22:50:21.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.984+0000 7fa35e5d7700 1 -- 172.21.15.102:0/2421959528 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35819c030 con 0x7fa358100cd0 2024-05-13T22:50:21.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.985+0000 7fa35e5d7700 1 -- 172.21.15.102:0/2421959528 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa358108630 con 0x7fa358100cd0 2024-05-13T22:50:21.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.986+0000 7fa3557fa700 1 -- 172.21.15.102:0/2421959528 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fa34c007ad0 con 0x7fa358100cd0 2024-05-13T22:50:21.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.986+0000 7fa3557fa700 1 --2- 172.21.15.102:0/2421959528 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa34006de80 0x7fa340070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:21.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.987+0000 7fa3557fa700 1 -- 172.21.15.102:0/2421959528 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa34c08dc90 con 0x7fa358100cd0 2024-05-13T22:50:21.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.987+0000 7fa357fff700 1 --2- 172.21.15.102:0/2421959528 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa34006de80 0x7fa340070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:21.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.987+0000 7fa357fff700 1 --2- 172.21.15.102:0/2421959528 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa34006de80 0x7fa340070340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa348006010 tx=0x7fa348005920 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:21.990+0000 7fa3557fa700 1 -- 172.21.15.102:0/2421959528 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fa34c05b0f0 con 0x7fa358100cd0 2024-05-13T22:50:22.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.224+0000 7fa35e5d7700 1 -- 172.21.15.102:0/2421959528 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa358069440 con 0x7fa34006de80 2024-05-13T22:50:22.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.226+0000 7fa3557fa700 1 -- 172.21.15.102:0/2421959528 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fa358069440 con 0x7fa34006de80 2024-05-13T22:50:22.203 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:50:22.204 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:50:22.204 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:50:22.204 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-13T22:50:22.204 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:50:22.204 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr" 2024-05-13T22:50:22.204 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:50:22.204 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "2/2 daemons upgraded", 2024-05-13T22:50:22.204 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading node-exporter daemons", 2024-05-13T22:50:22.204 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:50:22.204 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:50:22.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.228+0000 7fa35e5d7700 1 -- 172.21.15.102:0/2421959528 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa34006de80 msgr2=0x7fa340070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:22.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.228+0000 7fa35e5d7700 1 --2- 172.21.15.102:0/2421959528 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa34006de80 0x7fa340070340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa348006010 tx=0x7fa348005920 comp rx=0 tx=0).stop 2024-05-13T22:50:22.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.228+0000 7fa35e5d7700 1 -- 172.21.15.102:0/2421959528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa358100cd0 msgr2=0x7fa3581968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:22.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.228+0000 7fa35e5d7700 1 --2- 172.21.15.102:0/2421959528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa358100cd0 0x7fa3581968d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00c8a0 tx=0x7fa34c00cc60 comp rx=0 tx=0).stop 2024-05-13T22:50:22.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.228+0000 7fa35e5d7700 1 -- 172.21.15.102:0/2421959528 shutdown_connections 2024-05-13T22:50:22.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.228+0000 7fa35e5d7700 1 --2- 172.21.15.102:0/2421959528 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa34006de80 0x7fa340070340 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:22.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.228+0000 7fa35e5d7700 1 --2- 172.21.15.102:0/2421959528 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3580ffad0 0x7fa358196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:22.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.228+0000 7fa35e5d7700 1 --2- 172.21.15.102:0/2421959528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa358100cd0 0x7fa3581968d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:22.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.228+0000 7fa35e5d7700 1 -- 172.21.15.102:0/2421959528 >> 172.21.15.102:0/2421959528 conn(0x7fa3580fb050 msgr2=0x7fa358068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:22.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.228+0000 7fa35e5d7700 1 -- 172.21.15.102:0/2421959528 shutdown_connections 2024-05-13T22:50:22.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:22.228+0000 7fa35e5d7700 1 -- 172.21.15.102:0/2421959528 wait complete. 2024-05-13T22:50:22.748 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:22 smithi102 ceph-mon[28599]: pgmap v49: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 952 KiB/s rd, 901 KiB/s wr, 308 op/s 2024-05-13T22:50:22.748 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:22 smithi102 ceph-mon[28599]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:22.748 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:22 smithi102 ceph-mon[28599]: mgrmap e32: smithi102.xcijrj(active, since 92s), standbys: smithi130.uoqoxl 2024-05-13T22:50:22.748 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:22 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/4048201229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:22.749 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:22 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:22.749 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:22 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:22.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:22 smithi130 ceph-mon[36591]: pgmap v49: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 952 KiB/s rd, 901 KiB/s wr, 308 op/s 2024-05-13T22:50:22.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:22 smithi130 ceph-mon[36591]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:22.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:22 smithi130 ceph-mon[36591]: mgrmap e32: smithi102.xcijrj(active, since 92s), standbys: smithi130.uoqoxl 2024-05-13T22:50:22.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:22 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/4048201229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:22.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:22 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:22.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:22 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:23.842 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:23 smithi102 ceph-mon[28599]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:23.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:23 smithi130 ceph-mon[36591]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:24.802 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:24 smithi130 ceph-mon[36591]: pgmap v50: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 444 KiB/s rd, 407 KiB/s wr, 156 op/s 2024-05-13T22:50:24.887 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:24 smithi102 ceph-mon[28599]: pgmap v50: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 444 KiB/s rd, 407 KiB/s wr, 156 op/s 2024-05-13T22:50:26.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: pgmap v51: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 444 KiB/s rd, 407 KiB/s wr, 156 op/s 2024-05-13T22:50:26.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:50:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:50:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:50:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:26.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:26.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:26.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:26.929 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:27.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: pgmap v51: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 444 KiB/s rd, 407 KiB/s wr, 156 op/s 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:27.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:28.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:28 smithi102 ceph-mon[28599]: pgmap v52: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 68 KiB/s rd, 70 KiB/s wr, 46 op/s 2024-05-13T22:50:28.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:28 smithi102 ceph-mon[28599]: Upgrade: Updating prometheus.smithi102 2024-05-13T22:50:28.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:28 smithi102 ceph-mon[28599]: Deploying daemon prometheus.smithi102 on smithi102 2024-05-13T22:50:29.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:28 smithi130 ceph-mon[36591]: pgmap v52: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 68 KiB/s rd, 70 KiB/s wr, 46 op/s 2024-05-13T22:50:29.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:28 smithi130 ceph-mon[36591]: Upgrade: Updating prometheus.smithi102 2024-05-13T22:50:29.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:28 smithi130 ceph-mon[36591]: Deploying daemon prometheus.smithi102 on smithi102 2024-05-13T22:50:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:30 smithi102 ceph-mon[28599]: pgmap v53: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 28 KiB/s wr, 29 op/s 2024-05-13T22:50:31.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:30 smithi130 ceph-mon[36591]: pgmap v53: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 28 KiB/s wr, 29 op/s 2024-05-13T22:50:32.896 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:32 smithi102 ceph-mon[28599]: pgmap v54: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 28 KiB/s wr, 29 op/s 2024-05-13T22:50:33.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:32 smithi130 ceph-mon[36591]: pgmap v54: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 28 KiB/s wr, 29 op/s 2024-05-13T22:50:34.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:34 smithi102 ceph-mon[28599]: pgmap v55: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-13T22:50:34.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:34 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:34.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:34 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:34.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:34 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:50:34.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:34 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:34.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:34 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:34.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:34 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:50:34.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:34 smithi130 ceph-mon[36591]: pgmap v55: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-13T22:50:34.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:34 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:34.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:34 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:34.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:34 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:50:34.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:34 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:34.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:34 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:34.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:34 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:50:36.574 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:36 smithi102 ceph-mon[28599]: pgmap v56: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-13T22:50:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:36 smithi130 ceph-mon[36591]: pgmap v56: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-13T22:50:38.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:38 smithi102 ceph-mon[28599]: pgmap v57: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 2 op/s 2024-05-13T22:50:38.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:38 smithi130 ceph-mon[36591]: pgmap v57: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 2 op/s 2024-05-13T22:50:39.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:39 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:39.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:39 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:39.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:39 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:39.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:39 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:40.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:40 smithi102 ceph-mon[28599]: pgmap v58: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:50:40.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:40 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:40.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:40 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:41.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:40 smithi130 ceph-mon[36591]: pgmap v58: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:50:41.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:40 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:41.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:40 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:42.870 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:42 smithi102 ceph-mon[28599]: pgmap v59: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:50:43.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:42 smithi130 ceph-mon[36591]: pgmap v59: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:50:44.610 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:44.611 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:44.611 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:50:44.611 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:50:44.611 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:44.611 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:50:44.611 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:50:44.611 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:50:44.611 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.611 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.611 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.612 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.612 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.612 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.612 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.612 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.612 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.612 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:44 smithi130 ceph-mon[36591]: pgmap v60: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:50:44.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:44.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:44.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:44.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:44 smithi102 ceph-mon[28599]: pgmap v60: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:50:45.538 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:45 smithi102 ceph-mon[28599]: Upgrade: Updating alertmanager.smithi102 2024-05-13T22:50:45.538 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:45 smithi102 ceph-mon[28599]: Deploying daemon alertmanager.smithi102 on smithi102 2024-05-13T22:50:45.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:45 smithi130 ceph-mon[36591]: Upgrade: Updating alertmanager.smithi102 2024-05-13T22:50:45.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:45 smithi130 ceph-mon[36591]: Deploying daemon alertmanager.smithi102 on smithi102 2024-05-13T22:50:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:46 smithi130 ceph-mon[36591]: pgmap v61: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:50:46.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:46 smithi102 ceph-mon[28599]: pgmap v61: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:50:48.606 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:48 smithi102 ceph-mon[28599]: pgmap v62: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T22:50:48.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:48 smithi130 ceph-mon[36591]: pgmap v62: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T22:50:50.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:50 smithi102 ceph-mon[28599]: pgmap v63: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T22:50:50.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:50 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:50:50.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:50 smithi130 ceph-mon[36591]: pgmap v63: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T22:50:50.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:50 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:50:52.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.338+0000 7f6461e45700 1 -- 172.21.15.102:0/804236616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f645c103fa0 msgr2=0x7f645c106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:52.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.338+0000 7f6461e45700 1 --2- 172.21.15.102:0/804236616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f645c103fa0 0x7f645c106390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6450009910 tx=0x7f6450009c20 comp rx=0 tx=0).stop 2024-05-13T22:50:52.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.338+0000 7f6461e45700 1 -- 172.21.15.102:0/804236616 shutdown_connections 2024-05-13T22:50:52.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.338+0000 7f6461e45700 1 --2- 172.21.15.102:0/804236616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f645c103fa0 0x7f645c106390 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.338+0000 7f6461e45700 1 --2- 172.21.15.102:0/804236616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f645c101670 0x7f645c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.338+0000 7f6461e45700 1 -- 172.21.15.102:0/804236616 >> 172.21.15.102:0/804236616 conn(0x7f645c0fb080 msgr2=0x7f645c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:52.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.338+0000 7f6461e45700 1 -- 172.21.15.102:0/804236616 shutdown_connections 2024-05-13T22:50:52.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.338+0000 7f6461e45700 1 -- 172.21.15.102:0/804236616 wait complete. 2024-05-13T22:50:52.320 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:52 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:52.320 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:52 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:52.321 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:52 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.338+0000 7f6461e45700 1 Processor -- start 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f6461e45700 1 -- start start 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f6461e45700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f645c101670 0x7f645c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f6461e45700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f645c103fa0 0x7f645c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f6461e45700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c1990a0 con 0x7f645c101670 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f6461e45700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c1991e0 con 0x7f645c103fa0 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f645b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f645c101670 0x7f645c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f645b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f645c101670 0x7f645c198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38748/0 (socket says 172.21.15.102:38748) 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f645b7fe700 1 -- 172.21.15.102:0/1188149698 learned_addr learned my addr 172.21.15.102:0/1188149698 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f645affd700 1 --2- 172.21.15.102:0/1188149698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f645c103fa0 0x7f645c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f645b7fe700 1 -- 172.21.15.102:0/1188149698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f645c103fa0 msgr2=0x7f645c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:52.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f645b7fe700 1 --2- 172.21.15.102:0/1188149698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f645c103fa0 0x7f645c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f645b7fe700 1 -- 172.21.15.102:0/1188149698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64500095f0 con 0x7f645c101670 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f645b7fe700 1 --2- 172.21.15.102:0/1188149698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f645c101670 0x7f645c198540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c8a0 tx=0x7f644c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f6458ff9700 1 -- 172.21.15.102:0/1188149698 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f644c00cea0 con 0x7f645c101670 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.339+0000 7f6458ff9700 1 -- 172.21.15.102:0/1188149698 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f644c004830 con 0x7f645c101670 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.340+0000 7f6461e45700 1 -- 172.21.15.102:0/1188149698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c19dc90 con 0x7f645c101670 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.340+0000 7f6458ff9700 1 -- 172.21.15.102:0/1188149698 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f644c0055b0 con 0x7f645c101670 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.340+0000 7f6461e45700 1 -- 172.21.15.102:0/1188149698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c19e190 con 0x7f645c101670 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.341+0000 7f6461e45700 1 -- 172.21.15.102:0/1188149698 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c0fcc50 con 0x7f645c101670 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.341+0000 7f6458ff9700 1 -- 172.21.15.102:0/1188149698 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f644c007ad0 con 0x7f645c101670 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.341+0000 7f6458ff9700 1 --2- 172.21.15.102:0/1188149698 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f644406de80 0x7f6444070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.342+0000 7f6458ff9700 1 -- 172.21.15.102:0/1188149698 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f644c08d920 con 0x7f645c101670 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.342+0000 7f645affd700 1 --2- 172.21.15.102:0/1188149698 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f644406de80 0x7f6444070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:52.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.342+0000 7f645affd700 1 --2- 172.21.15.102:0/1188149698 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f644406de80 0x7f6444070340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f64500059d0 tx=0x7f6450005920 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:52.325 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.346+0000 7f6458ff9700 1 -- 172.21.15.102:0/1188149698 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f644c05ac60 con 0x7f645c101670 2024-05-13T22:50:52.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:52 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:52.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:52 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:52.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:52 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:50:52.556 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.578+0000 7f6461e45700 1 -- 172.21.15.102:0/1188149698 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f645c0611d0 con 0x7f644406de80 2024-05-13T22:50:52.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.580+0000 7f6458ff9700 1 -- 172.21.15.102:0/1188149698 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f645c0611d0 con 0x7f644406de80 2024-05-13T22:50:52.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.583+0000 7f6461e45700 1 -- 172.21.15.102:0/1188149698 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f644406de80 msgr2=0x7f6444070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:52.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.583+0000 7f6461e45700 1 --2- 172.21.15.102:0/1188149698 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f644406de80 0x7f6444070340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f64500059d0 tx=0x7f6450005920 comp rx=0 tx=0).stop 2024-05-13T22:50:52.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.583+0000 7f6461e45700 1 -- 172.21.15.102:0/1188149698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f645c101670 msgr2=0x7f645c198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:52.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.583+0000 7f6461e45700 1 --2- 172.21.15.102:0/1188149698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f645c101670 0x7f645c198540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c8a0 tx=0x7f644c00cc60 comp rx=0 tx=0).stop 2024-05-13T22:50:52.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.583+0000 7f6461e45700 1 -- 172.21.15.102:0/1188149698 shutdown_connections 2024-05-13T22:50:52.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.583+0000 7f6461e45700 1 --2- 172.21.15.102:0/1188149698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f645c101670 0x7f645c198540 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.584+0000 7f6461e45700 1 --2- 172.21.15.102:0/1188149698 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f644406de80 0x7f6444070340 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.584+0000 7f6461e45700 1 --2- 172.21.15.102:0/1188149698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f645c103fa0 0x7f645c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.584+0000 7f6461e45700 1 -- 172.21.15.102:0/1188149698 >> 172.21.15.102:0/1188149698 conn(0x7f645c0fb080 msgr2=0x7f645c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:52.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.584+0000 7f6461e45700 1 -- 172.21.15.102:0/1188149698 shutdown_connections 2024-05-13T22:50:52.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.584+0000 7f6461e45700 1 -- 172.21.15.102:0/1188149698 wait complete. 2024-05-13T22:50:52.578 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:50:52.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.703+0000 7fc26c02c700 1 -- 172.21.15.102:0/659503784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2641021c0 msgr2=0x7fc2641025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:52.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.703+0000 7fc26c02c700 1 --2- 172.21.15.102:0/659503784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2641021c0 0x7fc2641025e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc260009910 tx=0x7fc260009c20 comp rx=0 tx=0).stop 2024-05-13T22:50:52.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.703+0000 7fc26c02c700 1 -- 172.21.15.102:0/659503784 shutdown_connections 2024-05-13T22:50:52.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.703+0000 7fc26c02c700 1 --2- 172.21.15.102:0/659503784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2640fec40 0x7fc2640ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.703+0000 7fc26c02c700 1 --2- 172.21.15.102:0/659503784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2641021c0 0x7fc2641025e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.703+0000 7fc26c02c700 1 -- 172.21.15.102:0/659503784 >> 172.21.15.102:0/659503784 conn(0x7fc2640fa7c0 msgr2=0x7fc2640fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:52.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.703+0000 7fc26c02c700 1 -- 172.21.15.102:0/659503784 shutdown_connections 2024-05-13T22:50:52.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.703+0000 7fc26c02c700 1 -- 172.21.15.102:0/659503784 wait complete. 2024-05-13T22:50:52.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc26c02c700 1 Processor -- start 2024-05-13T22:50:52.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc26c02c700 1 -- start start 2024-05-13T22:50:52.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc26c02c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2640fec40 0x7fc264196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:52.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc26c02c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2641021c0 0x7fc2641968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:52.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc26c02c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc264196e60 con 0x7fc2640fec40 2024-05-13T22:50:52.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc26c02c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc264196fa0 con 0x7fc2641021c0 2024-05-13T22:50:52.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc269dc8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2640fec40 0x7fc264196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:52.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc269dc8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2640fec40 0x7fc264196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38776/0 (socket says 172.21.15.102:38776) 2024-05-13T22:50:52.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc269dc8700 1 -- 172.21.15.102:0/902237322 learned_addr learned my addr 172.21.15.102:0/902237322 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:50:52.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc2695c7700 1 --2- 172.21.15.102:0/902237322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2641021c0 0x7fc2641968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:52.683 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc269dc8700 1 -- 172.21.15.102:0/902237322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2641021c0 msgr2=0x7fc2641968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:52.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc269dc8700 1 --2- 172.21.15.102:0/902237322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2641021c0 0x7fc2641968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.704+0000 7fc269dc8700 1 -- 172.21.15.102:0/902237322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2600095f0 con 0x7fc2640fec40 2024-05-13T22:50:52.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.705+0000 7fc269dc8700 1 --2- 172.21.15.102:0/902237322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2640fec40 0x7fc264196390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc26000bda0 tx=0x7fc26000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:52.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.705+0000 7fc25affd700 1 -- 172.21.15.102:0/902237322 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc26001d070 con 0x7fc2640fec40 2024-05-13T22:50:52.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.705+0000 7fc26c02c700 1 -- 172.21.15.102:0/902237322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26419ba00 con 0x7fc2640fec40 2024-05-13T22:50:52.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.705+0000 7fc25affd700 1 -- 172.21.15.102:0/902237322 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc260022470 con 0x7fc2640fec40 2024-05-13T22:50:52.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.706+0000 7fc25affd700 1 -- 172.21.15.102:0/902237322 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc26000f670 con 0x7fc2640fec40 2024-05-13T22:50:52.684 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.706+0000 7fc26c02c700 1 -- 172.21.15.102:0/902237322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26419bec0 con 0x7fc2640fec40 2024-05-13T22:50:52.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.707+0000 7fc26c02c700 1 -- 172.21.15.102:0/902237322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc264109740 con 0x7fc2640fec40 2024-05-13T22:50:52.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.707+0000 7fc25affd700 1 -- 172.21.15.102:0/902237322 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fc260004af0 con 0x7fc2640fec40 2024-05-13T22:50:52.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.707+0000 7fc25affd700 1 --2- 172.21.15.102:0/902237322 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc25006de80 0x7fc250070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:52.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.708+0000 7fc25affd700 1 -- 172.21.15.102:0/902237322 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc26008f250 con 0x7fc2640fec40 2024-05-13T22:50:52.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.708+0000 7fc2695c7700 1 --2- 172.21.15.102:0/902237322 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc25006de80 0x7fc250070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:52.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.708+0000 7fc2695c7700 1 --2- 172.21.15.102:0/902237322 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc25006de80 0x7fc250070340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc264068380 tx=0x7fc25400a3b0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:52.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.716+0000 7fc25affd700 1 -- 172.21.15.102:0/902237322 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fc26005c640 con 0x7fc2640fec40 2024-05-13T22:50:52.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.956+0000 7fc26c02c700 1 -- 172.21.15.102:0/902237322 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2641068b0 con 0x7fc25006de80 2024-05-13T22:50:52.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.958+0000 7fc25affd700 1 -- 172.21.15.102:0/902237322 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fc2641068b0 con 0x7fc25006de80 2024-05-13T22:50:52.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.960+0000 7fc26c02c700 1 -- 172.21.15.102:0/902237322 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc25006de80 msgr2=0x7fc250070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:52.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.960+0000 7fc26c02c700 1 --2- 172.21.15.102:0/902237322 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc25006de80 0x7fc250070340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc264068380 tx=0x7fc25400a3b0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.960+0000 7fc26c02c700 1 -- 172.21.15.102:0/902237322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2640fec40 msgr2=0x7fc264196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:52.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.960+0000 7fc26c02c700 1 --2- 172.21.15.102:0/902237322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2640fec40 0x7fc264196390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc26000bda0 tx=0x7fc26000bdd0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.960+0000 7fc26c02c700 1 -- 172.21.15.102:0/902237322 shutdown_connections 2024-05-13T22:50:52.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.960+0000 7fc26c02c700 1 --2- 172.21.15.102:0/902237322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2640fec40 0x7fc264196390 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.960+0000 7fc26c02c700 1 --2- 172.21.15.102:0/902237322 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc25006de80 0x7fc250070340 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.960+0000 7fc26c02c700 1 --2- 172.21.15.102:0/902237322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc2641021c0 0x7fc2641968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:52.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.960+0000 7fc26c02c700 1 -- 172.21.15.102:0/902237322 >> 172.21.15.102:0/902237322 conn(0x7fc2640fa7c0 msgr2=0x7fc264105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:52.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.960+0000 7fc26c02c700 1 -- 172.21.15.102:0/902237322 shutdown_connections 2024-05-13T22:50:52.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:52.960+0000 7fc26c02c700 1 -- 172.21.15.102:0/902237322 wait complete. 2024-05-13T22:50:53.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.065+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/152655130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dc4100bc0 msgr2=0x7f3dc4100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:53.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.065+0000 7f3dcb5d8700 1 --2- 172.21.15.102:0/152655130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dc4100bc0 0x7f3dc4100fe0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009910 tx=0x7f3dc0009c20 comp rx=0 tx=0).stop 2024-05-13T22:50:53.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.065+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/152655130 shutdown_connections 2024-05-13T22:50:53.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.065+0000 7f3dcb5d8700 1 --2- 172.21.15.102:0/152655130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dc4101dc0 0x7f3dc4102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.065+0000 7f3dcb5d8700 1 --2- 172.21.15.102:0/152655130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dc4100bc0 0x7f3dc4100fe0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.065+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/152655130 >> 172.21.15.102:0/152655130 conn(0x7f3dc40fc140 msgr2=0x7f3dc40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:53.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.065+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/152655130 shutdown_connections 2024-05-13T22:50:53.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.066+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/152655130 wait complete. 2024-05-13T22:50:53.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.066+0000 7f3dcb5d8700 1 Processor -- start 2024-05-13T22:50:53.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.066+0000 7f3dcb5d8700 1 -- start start 2024-05-13T22:50:53.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.066+0000 7f3dcb5d8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dc4100bc0 0x7f3dc41963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.066+0000 7f3dcb5d8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dc4101dc0 0x7f3dc4196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.066+0000 7f3dcb5d8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc4196f50 con 0x7f3dc4101dc0 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.066+0000 7f3dcb5d8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc4197090 con 0x7f3dc4100bc0 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.066+0000 7f3dc8b73700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dc4101dc0 0x7f3dc4196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.066+0000 7f3dc8b73700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dc4101dc0 0x7f3dc4196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38790/0 (socket says 172.21.15.102:38790) 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.066+0000 7f3dc8b73700 1 -- 172.21.15.102:0/4102441251 learned_addr learned my addr 172.21.15.102:0/4102441251 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.067+0000 7f3dc9374700 1 --2- 172.21.15.102:0/4102441251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dc4100bc0 0x7f3dc41963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.067+0000 7f3dc8b73700 1 -- 172.21.15.102:0/4102441251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dc4100bc0 msgr2=0x7f3dc41963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.067+0000 7f3dc8b73700 1 --2- 172.21.15.102:0/4102441251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dc4100bc0 0x7f3dc41963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.067+0000 7f3dc8b73700 1 -- 172.21.15.102:0/4102441251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc00095f0 con 0x7f3dc4101dc0 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.067+0000 7f3dc8b73700 1 --2- 172.21.15.102:0/4102441251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dc4101dc0 0x7f3dc4196930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3db400b4b0 tx=0x7f3db400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.067+0000 7f3dba7fc700 1 -- 172.21.15.102:0/4102441251 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3db4011800 con 0x7f3dc4101dc0 2024-05-13T22:50:53.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.067+0000 7f3dba7fc700 1 -- 172.21.15.102:0/4102441251 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3db4011e10 con 0x7f3dc4101dc0 2024-05-13T22:50:53.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.067+0000 7f3dba7fc700 1 -- 172.21.15.102:0/4102441251 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3db400f2e0 con 0x7f3dc4101dc0 2024-05-13T22:50:53.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.067+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/4102441251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc419bb40 con 0x7f3dc4101dc0 2024-05-13T22:50:53.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.067+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/4102441251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc419c110 con 0x7f3dc4101dc0 2024-05-13T22:50:53.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.068+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/4102441251 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc41097a0 con 0x7f3dc4101dc0 2024-05-13T22:50:53.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.069+0000 7f3dba7fc700 1 -- 172.21.15.102:0/4102441251 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f3db4010380 con 0x7f3dc4101dc0 2024-05-13T22:50:53.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.069+0000 7f3dba7fc700 1 --2- 172.21.15.102:0/4102441251 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3db006de80 0x7f3db0070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:53.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.069+0000 7f3dba7fc700 1 -- 172.21.15.102:0/4102441251 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3db408d750 con 0x7f3dc4101dc0 2024-05-13T22:50:53.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.069+0000 7f3dc9374700 1 --2- 172.21.15.102:0/4102441251 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3db006de80 0x7f3db0070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:53.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.070+0000 7f3dc9374700 1 --2- 172.21.15.102:0/4102441251 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3db006de80 0x7f3db0070340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0006010 tx=0x7f3dc000b540 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:53.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.074+0000 7f3dba7fc700 1 -- 172.21.15.102:0/4102441251 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f3db405aa70 con 0x7f3dc4101dc0 2024-05-13T22:50:53.287 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.309+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/4102441251 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3dc4106710 con 0x7f3db006de80 2024-05-13T22:50:53.287 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:53 smithi102 ceph-mon[28599]: pgmap v64: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:50:53.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.320+0000 7f3dba7fc700 1 -- 172.21.15.102:0/4102441251 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3dc4106710 con 0x7f3db006de80 2024-05-13T22:50:53.297 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:50:53.297 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 starting - - - - 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (11m) 14s ago 11m 7990k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (10m) 37s ago 10m 7662k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (11m) 14s ago 11m 6660k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (10m) 37s ago 10m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (9m) 14s ago 10m 89.0M - 9.4.7 954c08fa6188 bca1790c17aa 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (5m) 14s ago 5m 88.3M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (5m) 14s ago 5m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (5m) 37s ago 5m 14.3M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (5m) 37s ago 5m 17.5M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (2m) 14s ago 12m 500M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (75s) 37s ago 10m 450M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (12m) 14s ago 12m 54.2M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (10m) 37s ago 9m 46.6M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:50:53.298 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (50s) 14s ago 11m 8488k - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:50:53.299 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (41s) 37s ago 10m 3670k - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:50:53.299 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (9m) 14s ago 9m 222M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:50:53.299 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (8m) 14s ago 8m 237M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:50:53.299 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (8m) 14s ago 8m 201M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:50:53.299 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (7m) 37s ago 7m 288M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:50:53.299 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (7m) 37s ago 7m 244M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:50:53.299 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (7m) 37s ago 7m 224M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:50:53.299 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (20s) 14s ago 10m 47.3M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:50:53.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.322+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/4102441251 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3db006de80 msgr2=0x7f3db0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:53.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.322+0000 7f3dcb5d8700 1 --2- 172.21.15.102:0/4102441251 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3db006de80 0x7f3db0070340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0006010 tx=0x7f3dc000b540 comp rx=0 tx=0).stop 2024-05-13T22:50:53.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.322+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/4102441251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dc4101dc0 msgr2=0x7f3dc4196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:53.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.322+0000 7f3dcb5d8700 1 --2- 172.21.15.102:0/4102441251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dc4101dc0 0x7f3dc4196930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3db400b4b0 tx=0x7f3db400b7c0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.322+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/4102441251 shutdown_connections 2024-05-13T22:50:53.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.322+0000 7f3dcb5d8700 1 --2- 172.21.15.102:0/4102441251 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f3db006de80 0x7f3db0070340 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.322+0000 7f3dcb5d8700 1 --2- 172.21.15.102:0/4102441251 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dc4100bc0 0x7f3dc41963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.322+0000 7f3dcb5d8700 1 --2- 172.21.15.102:0/4102441251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dc4101dc0 0x7f3dc4196930 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.322+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/4102441251 >> 172.21.15.102:0/4102441251 conn(0x7f3dc40fc140 msgr2=0x7f3dc4104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:53.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.322+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/4102441251 shutdown_connections 2024-05-13T22:50:53.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.322+0000 7f3dcb5d8700 1 -- 172.21.15.102:0/4102441251 wait complete. 2024-05-13T22:50:53.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:53 smithi130 ceph-mon[36591]: pgmap v64: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:50:53.404 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.427+0000 7f22f099b700 1 -- 172.21.15.102:0/392558316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec101eb0 msgr2=0x7f22ec102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:53.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.427+0000 7f22f099b700 1 --2- 172.21.15.102:0/392558316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec101eb0 0x7f22ec102310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009910 tx=0x7f22d4009c20 comp rx=0 tx=0).stop 2024-05-13T22:50:53.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.427+0000 7f22f099b700 1 -- 172.21.15.102:0/392558316 shutdown_connections 2024-05-13T22:50:53.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.427+0000 7f22f099b700 1 --2- 172.21.15.102:0/392558316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec101eb0 0x7f22ec102310 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.427+0000 7f22f099b700 1 --2- 172.21.15.102:0/392558316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ec0feed0 0x7f22ec0ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.427+0000 7f22f099b700 1 -- 172.21.15.102:0/392558316 >> 172.21.15.102:0/392558316 conn(0x7f22ec0fa810 msgr2=0x7f22ec0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:53.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.427+0000 7f22f099b700 1 -- 172.21.15.102:0/392558316 shutdown_connections 2024-05-13T22:50:53.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.427+0000 7f22f099b700 1 -- 172.21.15.102:0/392558316 wait complete. 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22f099b700 1 Processor -- start 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22f099b700 1 -- start start 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22f099b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec0feed0 0x7f22ec1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22f099b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ec101eb0 0x7f22ec196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22f099b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec196f40 con 0x7f22ec0feed0 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22f099b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec197080 con 0x7f22ec101eb0 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22ea59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec0feed0 0x7f22ec1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22ea59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec0feed0 0x7f22ec1963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38808/0 (socket says 172.21.15.102:38808) 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22ea59c700 1 -- 172.21.15.102:0/2712368823 learned_addr learned my addr 172.21.15.102:0/2712368823 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22ea59c700 1 -- 172.21.15.102:0/2712368823 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ec101eb0 msgr2=0x7f22ec196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22ea59c700 1 --2- 172.21.15.102:0/2712368823 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ec101eb0 0x7f22ec196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.428+0000 7f22ea59c700 1 -- 172.21.15.102:0/2712368823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d40095f0 con 0x7f22ec0feed0 2024-05-13T22:50:53.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.429+0000 7f22ea59c700 1 --2- 172.21.15.102:0/2712368823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec0feed0 0x7f22ec1963e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f22dc007e90 tx=0x7f22dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:53.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.429+0000 7f22e37fe700 1 -- 172.21.15.102:0/2712368823 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22dc017070 con 0x7f22ec0feed0 2024-05-13T22:50:53.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.429+0000 7f22f099b700 1 -- 172.21.15.102:0/2712368823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec19bb30 con 0x7f22ec0feed0 2024-05-13T22:50:53.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.429+0000 7f22e37fe700 1 -- 172.21.15.102:0/2712368823 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f22dc00f040 con 0x7f22ec0feed0 2024-05-13T22:50:53.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.429+0000 7f22e37fe700 1 -- 172.21.15.102:0/2712368823 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22dc004880 con 0x7f22ec0feed0 2024-05-13T22:50:53.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.429+0000 7f22f099b700 1 -- 172.21.15.102:0/2712368823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec19c080 con 0x7f22ec0feed0 2024-05-13T22:50:53.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.430+0000 7f22f099b700 1 -- 172.21.15.102:0/2712368823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec109770 con 0x7f22ec0feed0 2024-05-13T22:50:53.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.430+0000 7f22e37fe700 1 -- 172.21.15.102:0/2712368823 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f22dc0040b0 con 0x7f22ec0feed0 2024-05-13T22:50:53.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.431+0000 7f22e37fe700 1 --2- 172.21.15.102:0/2712368823 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f22d806de30 0x7f22d80702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:53.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.431+0000 7f22e37fe700 1 -- 172.21.15.102:0/2712368823 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f22dc08d660 con 0x7f22ec0feed0 2024-05-13T22:50:53.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.431+0000 7f22e9d9b700 1 --2- 172.21.15.102:0/2712368823 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f22d806de30 0x7f22d80702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:53.409 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.431+0000 7f22e9d9b700 1 --2- 172.21.15.102:0/2712368823 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f22d806de30 0x7f22d80702f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22d400b5c0 tx=0x7f22d4005960 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:53.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.435+0000 7f22e37fe700 1 -- 172.21.15.102:0/2712368823 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f22dc05a9a0 con 0x7f22ec0feed0 2024-05-13T22:50:53.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.716+0000 7f22f099b700 1 -- 172.21.15.102:0/2712368823 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f22ec19cbe0 con 0x7f22ec0feed0 2024-05-13T22:50:53.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.716+0000 7f22e37fe700 1 -- 172.21.15.102:0/2712368823 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f22dc05a530 con 0x7f22ec0feed0 2024-05-13T22:50:53.694 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:50:53.694 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:50:53.694 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:50:53.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:50:53.694 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:50:53.694 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:50:53.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:50:53.694 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:50:53.694 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:50:53.694 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:50:53.695 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:50:53.695 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:50:53.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:50:53.695 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:50:53.695 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-05-13T22:50:53.695 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:50:53.695 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:50:53.695 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:50:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.719+0000 7f22f099b700 1 -- 172.21.15.102:0/2712368823 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f22d806de30 msgr2=0x7f22d80702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.719+0000 7f22f099b700 1 --2- 172.21.15.102:0/2712368823 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f22d806de30 0x7f22d80702f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22d400b5c0 tx=0x7f22d4005960 comp rx=0 tx=0).stop 2024-05-13T22:50:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.719+0000 7f22f099b700 1 -- 172.21.15.102:0/2712368823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec0feed0 msgr2=0x7f22ec1963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.719+0000 7f22f099b700 1 --2- 172.21.15.102:0/2712368823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec0feed0 0x7f22ec1963e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f22dc007e90 tx=0x7f22dc00d3b0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.719+0000 7f22f099b700 1 -- 172.21.15.102:0/2712368823 shutdown_connections 2024-05-13T22:50:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.719+0000 7f22f099b700 1 --2- 172.21.15.102:0/2712368823 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22ec0feed0 0x7f22ec1963e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.719+0000 7f22f099b700 1 --2- 172.21.15.102:0/2712368823 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f22d806de30 0x7f22d80702f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.719+0000 7f22f099b700 1 --2- 172.21.15.102:0/2712368823 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22ec101eb0 0x7f22ec196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.719+0000 7f22f099b700 1 -- 172.21.15.102:0/2712368823 >> 172.21.15.102:0/2712368823 conn(0x7f22ec0fa810 msgr2=0x7f22ec1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.719+0000 7f22f099b700 1 -- 172.21.15.102:0/2712368823 shutdown_connections 2024-05-13T22:50:53.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.719+0000 7f22f099b700 1 -- 172.21.15.102:0/2712368823 wait complete. 2024-05-13T22:50:53.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.828+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/3588334057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c781016d0 msgr2=0x7f5c78103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:53.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.828+0000 7f5c7d7b0700 1 --2- 172.21.15.102:0/3588334057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c781016d0 0x7f5c78103ac0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009910 tx=0x7f5c68009c20 comp rx=0 tx=0).stop 2024-05-13T22:50:53.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.828+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/3588334057 shutdown_connections 2024-05-13T22:50:53.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.828+0000 7f5c7d7b0700 1 --2- 172.21.15.102:0/3588334057 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c78104000 0x7f5c781063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.828+0000 7f5c7d7b0700 1 --2- 172.21.15.102:0/3588334057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c781016d0 0x7f5c78103ac0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.828+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/3588334057 >> 172.21.15.102:0/3588334057 conn(0x7f5c780fb060 msgr2=0x7f5c780fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:53.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.828+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/3588334057 shutdown_connections 2024-05-13T22:50:53.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.828+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/3588334057 wait complete. 2024-05-13T22:50:53.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.828+0000 7f5c7d7b0700 1 Processor -- start 2024-05-13T22:50:53.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c7d7b0700 1 -- start start 2024-05-13T22:50:53.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c7d7b0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c781016d0 0x7f5c781963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c7d7b0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c78104000 0x7f5c78196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c7d7b0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78196f30 con 0x7f5c781016d0 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c7d7b0700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78197070 con 0x7f5c78104000 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c76ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c781016d0 0x7f5c781963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c76ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c781016d0 0x7f5c781963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38828/0 (socket says 172.21.15.102:38828) 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c76ffd700 1 -- 172.21.15.102:0/4185050263 learned_addr learned my addr 172.21.15.102:0/4185050263 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c6e7fc700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c78104000 0x7f5c78196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c76ffd700 1 -- 172.21.15.102:0/4185050263 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c78104000 msgr2=0x7f5c78196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c76ffd700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c78104000 0x7f5c78196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c76ffd700 1 -- 172.21.15.102:0/4185050263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c680095f0 con 0x7f5c781016d0 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c6e7fc700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c78104000 0x7f5c78196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.829+0000 7f5c76ffd700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c781016d0 0x7f5c781963d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800bd70 tx=0x7f5c6800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:53.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.830+0000 7f5c74ff9700 1 -- 172.21.15.102:0/4185050263 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6801d070 con 0x7f5c781016d0 2024-05-13T22:50:53.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.830+0000 7f5c74ff9700 1 -- 172.21.15.102:0/4185050263 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5c68022470 con 0x7f5c781016d0 2024-05-13T22:50:53.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.830+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/4185050263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7819bac0 con 0x7f5c781016d0 2024-05-13T22:50:53.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.830+0000 7f5c74ff9700 1 -- 172.21.15.102:0/4185050263 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6800f670 con 0x7f5c781016d0 2024-05-13T22:50:53.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.830+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/4185050263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7819c030 con 0x7f5c781016d0 2024-05-13T22:50:53.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.831+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/4185050263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c781905b0 con 0x7f5c781016d0 2024-05-13T22:50:53.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.831+0000 7f5c74ff9700 1 -- 172.21.15.102:0/4185050263 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f5c68022a50 con 0x7f5c781016d0 2024-05-13T22:50:53.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.832+0000 7f5c74ff9700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5c6406de80 0x7f5c64070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:50:53.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.832+0000 7f5c74ff9700 1 -- 172.21.15.102:0/4185050263 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5c6808f2e0 con 0x7f5c781016d0 2024-05-13T22:50:53.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.832+0000 7f5c6e7fc700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5c6406de80 0x7f5c64070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:50:53.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.832+0000 7f5c6e7fc700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5c6406de80 0x7f5c64070340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c60007900 tx=0x7f5c60008040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:50:53.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:53.836+0000 7f5c74ff9700 1 -- 172.21.15.102:0/4185050263 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f5c6805c6d0 con 0x7f5c781016d0 2024-05-13T22:50:54.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.066+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/4185050263 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c780611d0 con 0x7f5c6406de80 2024-05-13T22:50:54.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.067+0000 7f5c74ff9700 1 -- 172.21.15.102:0/4185050263 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f5c780611d0 con 0x7f5c6406de80 2024-05-13T22:50:54.044 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:50:54.044 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:50:54.045 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:50:54.045 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-13T22:50:54.045 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:50:54.045 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr" 2024-05-13T22:50:54.045 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:50:54.045 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "2/2 daemons upgraded", 2024-05-13T22:50:54.045 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading alertmanager daemons", 2024-05-13T22:50:54.045 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:50:54.045 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:50:54.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.069+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/4185050263 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5c6406de80 msgr2=0x7f5c64070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:54.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.069+0000 7f5c7d7b0700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5c6406de80 0x7f5c64070340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c60007900 tx=0x7f5c60008040 comp rx=0 tx=0).stop 2024-05-13T22:50:54.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.069+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/4185050263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c781016d0 msgr2=0x7f5c781963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:50:54.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.069+0000 7f5c7d7b0700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c781016d0 0x7f5c781963d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800bd70 tx=0x7f5c6800be50 comp rx=0 tx=0).stop 2024-05-13T22:50:54.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.069+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/4185050263 shutdown_connections 2024-05-13T22:50:54.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.069+0000 7f5c7d7b0700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c781016d0 0x7f5c781963d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:54.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.069+0000 7f5c7d7b0700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f5c6406de80 0x7f5c64070340 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:54.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.069+0000 7f5c7d7b0700 1 --2- 172.21.15.102:0/4185050263 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c78104000 0x7f5c78196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:50:54.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.069+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/4185050263 >> 172.21.15.102:0/4185050263 conn(0x7f5c780fb060 msgr2=0x7f5c780fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:50:54.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.069+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/4185050263 shutdown_connections 2024-05-13T22:50:54.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:50:54.069+0000 7f5c7d7b0700 1 -- 172.21.15.102:0/4185050263 wait complete. 2024-05-13T22:50:54.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:54 smithi102 ceph-mon[28599]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:54.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:54 smithi102 ceph-mon[28599]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:54.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:54 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/2712368823' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:54.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:54 smithi130 ceph-mon[36591]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:54.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:54 smithi130 ceph-mon[36591]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:54.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:54 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/2712368823' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:50:55.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:55 smithi102 ceph-mon[28599]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:55.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:55 smithi102 ceph-mon[28599]: pgmap v65: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T22:50:55.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:55 smithi102 ceph-mon[28599]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:55.567 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:55 smithi130 ceph-mon[36591]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:55.567 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:55 smithi130 ceph-mon[36591]: pgmap v65: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T22:50:55.567 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:55 smithi130 ceph-mon[36591]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:50:56.498 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:56 smithi130 ceph-mon[36591]: pgmap v66: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:50:56.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:56 smithi102 ceph-mon[28599]: pgmap v66: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:50:57.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:57 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:57.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:57 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:58.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:57 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:58.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:57 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:59.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:58 smithi130 ceph-mon[36591]: pgmap v67: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:50:59.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:58 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:59.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:50:58 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:59.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:58 smithi102 ceph-mon[28599]: pgmap v67: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:50:59.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:58 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:50:59.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:50:58 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:01.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:00 smithi130 ceph-mon[36591]: pgmap v68: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:01.170 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:00 smithi102 ceph-mon[28599]: pgmap v68: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:02.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: pgmap v69: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:02.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:02 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: pgmap v69: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:02.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:02.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:02.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:51:02.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:51:02.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:02.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:02.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:02 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:04.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:04 smithi130 ceph-mon[36591]: Upgrade: Updating grafana.smithi102 2024-05-13T22:51:04.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:04 smithi130 ceph-mon[36591]: Deploying daemon grafana.smithi102 on smithi102 2024-05-13T22:51:04.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:04 smithi130 ceph-mon[36591]: pgmap v70: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:04.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:04 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:51:04.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:04 smithi102 ceph-mon[28599]: Upgrade: Updating grafana.smithi102 2024-05-13T22:51:04.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:04 smithi102 ceph-mon[28599]: Deploying daemon grafana.smithi102 on smithi102 2024-05-13T22:51:04.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:04 smithi102 ceph-mon[28599]: pgmap v70: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:04.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:04 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:51:06.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:06 smithi130 ceph-mon[36591]: pgmap v71: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:06.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:06 smithi102 ceph-mon[28599]: pgmap v71: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:08.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:08 smithi102 ceph-mon[28599]: pgmap v72: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:08.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:08 smithi130 ceph-mon[36591]: pgmap v72: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:10.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:10 smithi102 ceph-mon[28599]: pgmap v73: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:10.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:10 smithi130 ceph-mon[36591]: pgmap v73: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:12.649 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:12 smithi102 ceph-mon[28599]: pgmap v74: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:12.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:12 smithi130 ceph-mon[36591]: pgmap v74: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:14.531 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:14 smithi102 ceph-mon[28599]: pgmap v75: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:14.887 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:14 smithi130 ceph-mon[36591]: pgmap v75: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:15 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:15 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:15 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:51:16.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:15 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:16.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:15 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:16.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:15 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:51:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:16 smithi130 ceph-mon[36591]: pgmap v76: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:17.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:16 smithi102 ceph-mon[28599]: pgmap v76: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:19.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:18 smithi130 ceph-mon[36591]: pgmap v77: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:19.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:18 smithi102 ceph-mon[28599]: pgmap v77: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:20.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:19 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:51:20.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:19 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:51:21.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:20 smithi130 ceph-mon[36591]: pgmap v78: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:21.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:20 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:21.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:20 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:21.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:20 smithi102 ceph-mon[28599]: pgmap v78: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:21.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:20 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:21.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:20 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:22.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:22 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:22.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:22 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:22.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:22 smithi130 ceph-mon[36591]: pgmap v79: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:22.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:22 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:22.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:22 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:22.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:22 smithi102 ceph-mon[28599]: pgmap v79: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:24.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.179+0000 7fc839b60700 1 -- 172.21.15.102:0/319481710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834101640 msgr2=0x7fc834103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:24.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.179+0000 7fc839b60700 1 --2- 172.21.15.102:0/319481710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834101640 0x7fc834103a30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc824009910 tx=0x7fc824009c20 comp rx=0 tx=0).stop 2024-05-13T22:51:24.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.179+0000 7fc839b60700 1 -- 172.21.15.102:0/319481710 shutdown_connections 2024-05-13T22:51:24.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.179+0000 7fc839b60700 1 --2- 172.21.15.102:0/319481710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc834103f70 0x7fc834106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.179+0000 7fc839b60700 1 --2- 172.21.15.102:0/319481710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834101640 0x7fc834103a30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.179+0000 7fc839b60700 1 -- 172.21.15.102:0/319481710 >> 172.21.15.102:0/319481710 conn(0x7fc8340fb030 msgr2=0x7fc8340fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:24.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.179+0000 7fc839b60700 1 -- 172.21.15.102:0/319481710 shutdown_connections 2024-05-13T22:51:24.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.179+0000 7fc839b60700 1 -- 172.21.15.102:0/319481710 wait complete. 2024-05-13T22:51:24.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.180+0000 7fc839b60700 1 Processor -- start 2024-05-13T22:51:24.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.180+0000 7fc839b60700 1 -- start start 2024-05-13T22:51:24.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.180+0000 7fc839b60700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc834101640 0x7fc834196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:24.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.180+0000 7fc839b60700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834103f70 0x7fc8341968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:24.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.180+0000 7fc839b60700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc834196ee0 con 0x7fc834103f70 2024-05-13T22:51:24.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.180+0000 7fc839b60700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc834197020 con 0x7fc834101640 2024-05-13T22:51:24.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.180+0000 7fc832ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834103f70 0x7fc8341968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:24.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc832ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834103f70 0x7fc8341968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56788/0 (socket says 172.21.15.102:56788) 2024-05-13T22:51:24.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc832ffd700 1 -- 172.21.15.102:0/1426712659 learned_addr learned my addr 172.21.15.102:0/1426712659 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:51:24.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc8337fe700 1 --2- 172.21.15.102:0/1426712659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc834101640 0x7fc834196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:24.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc832ffd700 1 -- 172.21.15.102:0/1426712659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc834101640 msgr2=0x7fc834196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:24.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc832ffd700 1 --2- 172.21.15.102:0/1426712659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc834101640 0x7fc834196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc832ffd700 1 -- 172.21.15.102:0/1426712659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240095f0 con 0x7fc834103f70 2024-05-13T22:51:24.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc832ffd700 1 --2- 172.21.15.102:0/1426712659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834103f70 0x7fc8341968c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc82800b4b0 tx=0x7fc82800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:24.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc830ff9700 1 -- 172.21.15.102:0/1426712659 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc828011800 con 0x7fc834103f70 2024-05-13T22:51:24.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc830ff9700 1 -- 172.21.15.102:0/1426712659 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc828011e10 con 0x7fc834103f70 2024-05-13T22:51:24.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc839b60700 1 -- 172.21.15.102:0/1426712659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83419bad0 con 0x7fc834103f70 2024-05-13T22:51:24.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc830ff9700 1 -- 172.21.15.102:0/1426712659 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc82800f2e0 con 0x7fc834103f70 2024-05-13T22:51:24.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.181+0000 7fc839b60700 1 -- 172.21.15.102:0/1426712659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83419c0a0 con 0x7fc834103f70 2024-05-13T22:51:24.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.182+0000 7fc839b60700 1 -- 172.21.15.102:0/1426712659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8341905a0 con 0x7fc834103f70 2024-05-13T22:51:24.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.183+0000 7fc830ff9700 1 -- 172.21.15.102:0/1426712659 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fc828010380 con 0x7fc834103f70 2024-05-13T22:51:24.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.183+0000 7fc830ff9700 1 --2- 172.21.15.102:0/1426712659 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc81c06de80 0x7fc81c070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:24.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.183+0000 7fc830ff9700 1 -- 172.21.15.102:0/1426712659 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc82808d790 con 0x7fc834103f70 2024-05-13T22:51:24.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.183+0000 7fc8337fe700 1 --2- 172.21.15.102:0/1426712659 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc81c06de80 0x7fc81c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:24.169 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.184+0000 7fc8337fe700 1 --2- 172.21.15.102:0/1426712659 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc81c06de80 0x7fc81c070340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc82400b5c0 tx=0x7fc8240052a0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:24.170 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.188+0000 7fc830ff9700 1 -- 172.21.15.102:0/1426712659 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fc82805aad0 con 0x7fc834103f70 2024-05-13T22:51:24.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.407+0000 7fc839b60700 1 -- 172.21.15.102:0/1426712659 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8340611d0 con 0x7fc81c06de80 2024-05-13T22:51:24.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.409+0000 7fc830ff9700 1 -- 172.21.15.102:0/1426712659 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fc8340611d0 con 0x7fc81c06de80 2024-05-13T22:51:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.411+0000 7fc839b60700 1 -- 172.21.15.102:0/1426712659 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc81c06de80 msgr2=0x7fc81c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.411+0000 7fc839b60700 1 --2- 172.21.15.102:0/1426712659 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc81c06de80 0x7fc81c070340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc82400b5c0 tx=0x7fc8240052a0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.411+0000 7fc839b60700 1 -- 172.21.15.102:0/1426712659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834103f70 msgr2=0x7fc8341968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.411+0000 7fc839b60700 1 --2- 172.21.15.102:0/1426712659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834103f70 0x7fc8341968c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc82800b4b0 tx=0x7fc82800b7c0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.411+0000 7fc839b60700 1 -- 172.21.15.102:0/1426712659 shutdown_connections 2024-05-13T22:51:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.411+0000 7fc839b60700 1 --2- 172.21.15.102:0/1426712659 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc81c06de80 0x7fc81c070340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.411+0000 7fc839b60700 1 --2- 172.21.15.102:0/1426712659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc834101640 0x7fc834196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.411+0000 7fc839b60700 1 --2- 172.21.15.102:0/1426712659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc834103f70 0x7fc8341968c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.411+0000 7fc839b60700 1 -- 172.21.15.102:0/1426712659 >> 172.21.15.102:0/1426712659 conn(0x7fc8340fb030 msgr2=0x7fc8340fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:24.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.411+0000 7fc839b60700 1 -- 172.21.15.102:0/1426712659 shutdown_connections 2024-05-13T22:51:24.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.411+0000 7fc839b60700 1 -- 172.21.15.102:0/1426712659 wait complete. 2024-05-13T22:51:24.400 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:51:24.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.523+0000 7f51c4279700 1 -- 172.21.15.102:0/3877594103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51bc100cc0 msgr2=0x7f51bc101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:24.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.523+0000 7f51c4279700 1 --2- 172.21.15.102:0/3877594103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51bc100cc0 0x7f51bc101120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009910 tx=0x7f51b0009c20 comp rx=0 tx=0).stop 2024-05-13T22:51:24.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.523+0000 7f51c4279700 1 -- 172.21.15.102:0/3877594103 shutdown_connections 2024-05-13T22:51:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.523+0000 7f51c4279700 1 --2- 172.21.15.102:0/3877594103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51bc100cc0 0x7f51bc101120 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.523+0000 7f51c4279700 1 --2- 172.21.15.102:0/3877594103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51bc0ffac0 0x7f51bc0ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.523+0000 7f51c4279700 1 -- 172.21.15.102:0/3877594103 >> 172.21.15.102:0/3877594103 conn(0x7f51bc0fb060 msgr2=0x7f51bc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.523+0000 7f51c4279700 1 -- 172.21.15.102:0/3877594103 shutdown_connections 2024-05-13T22:51:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.523+0000 7f51c4279700 1 -- 172.21.15.102:0/3877594103 wait complete. 2024-05-13T22:51:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.524+0000 7f51c4279700 1 Processor -- start 2024-05-13T22:51:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.524+0000 7f51c4279700 1 -- start start 2024-05-13T22:51:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.524+0000 7f51c4279700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51bc0ffac0 0x7f51bc195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:24.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.524+0000 7f51c4279700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51bc100cc0 0x7f51bc1957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.524+0000 7f51c4279700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc195dd0 con 0x7f51bc100cc0 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.524+0000 7f51c4279700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc195f10 con 0x7f51bc0ffac0 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.524+0000 7f51c1814700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51bc100cc0 0x7f51bc1957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.524+0000 7f51c1814700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51bc100cc0 0x7f51bc1957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56800/0 (socket says 172.21.15.102:56800) 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.524+0000 7f51c1814700 1 -- 172.21.15.102:0/439796124 learned_addr learned my addr 172.21.15.102:0/439796124 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.524+0000 7f51c2015700 1 --2- 172.21.15.102:0/439796124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51bc0ffac0 0x7f51bc195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.525+0000 7f51c1814700 1 -- 172.21.15.102:0/439796124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51bc0ffac0 msgr2=0x7f51bc195270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.525+0000 7f51c1814700 1 --2- 172.21.15.102:0/439796124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51bc0ffac0 0x7f51bc195270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.525+0000 7f51c1814700 1 -- 172.21.15.102:0/439796124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b00095f0 con 0x7f51bc100cc0 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.525+0000 7f51c1814700 1 --2- 172.21.15.102:0/439796124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51bc100cc0 0x7f51bc1957b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f51b0006010 tx=0x7f51b000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.525+0000 7f51bb7fe700 1 -- 172.21.15.102:0/439796124 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51b001d070 con 0x7f51bc100cc0 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.525+0000 7f51c4279700 1 -- 172.21.15.102:0/439796124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc19a960 con 0x7f51bc100cc0 2024-05-13T22:51:24.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.525+0000 7f51bb7fe700 1 -- 172.21.15.102:0/439796124 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51b000f460 con 0x7f51bc100cc0 2024-05-13T22:51:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.525+0000 7f51bb7fe700 1 -- 172.21.15.102:0/439796124 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51b0021640 con 0x7f51bc100cc0 2024-05-13T22:51:24.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.525+0000 7f51c4279700 1 -- 172.21.15.102:0/439796124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc19ae50 con 0x7f51bc100cc0 2024-05-13T22:51:24.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.526+0000 7f51c4279700 1 -- 172.21.15.102:0/439796124 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc108620 con 0x7f51bc100cc0 2024-05-13T22:51:24.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.526+0000 7f51bb7fe700 1 -- 172.21.15.102:0/439796124 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f51b000fab0 con 0x7f51bc100cc0 2024-05-13T22:51:24.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.527+0000 7f51bb7fe700 1 --2- 172.21.15.102:0/439796124 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f51a406de80 0x7f51a4070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:24.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.527+0000 7f51bb7fe700 1 -- 172.21.15.102:0/439796124 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f51b008eee0 con 0x7f51bc100cc0 2024-05-13T22:51:24.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.527+0000 7f51c2015700 1 --2- 172.21.15.102:0/439796124 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f51a406de80 0x7f51a4070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:24.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.527+0000 7f51c2015700 1 --2- 172.21.15.102:0/439796124 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f51a406de80 0x7f51a4070340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009550 tx=0x7f51ac0091b0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:24.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.531+0000 7f51bb7fe700 1 -- 172.21.15.102:0/439796124 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f51b005c220 con 0x7f51bc100cc0 2024-05-13T22:51:24.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:24 smithi102 ceph-mon[28599]: pgmap v80: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:24.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.764+0000 7f51c4279700 1 -- 172.21.15.102:0/439796124 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f51bc105610 con 0x7f51a406de80 2024-05-13T22:51:24.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.765+0000 7f51bb7fe700 1 -- 172.21.15.102:0/439796124 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f51bc105610 con 0x7f51a406de80 2024-05-13T22:51:24.746 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.767+0000 7f51c4279700 1 -- 172.21.15.102:0/439796124 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f51a406de80 msgr2=0x7f51a4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.767+0000 7f51c4279700 1 --2- 172.21.15.102:0/439796124 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f51a406de80 0x7f51a4070340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009550 tx=0x7f51ac0091b0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.767+0000 7f51c4279700 1 -- 172.21.15.102:0/439796124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51bc100cc0 msgr2=0x7f51bc1957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.767+0000 7f51c4279700 1 --2- 172.21.15.102:0/439796124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51bc100cc0 0x7f51bc1957b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f51b0006010 tx=0x7f51b000bd70 comp rx=0 tx=0).stop 2024-05-13T22:51:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.768+0000 7f51c4279700 1 -- 172.21.15.102:0/439796124 shutdown_connections 2024-05-13T22:51:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.768+0000 7f51c4279700 1 --2- 172.21.15.102:0/439796124 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f51a406de80 0x7f51a4070340 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.768+0000 7f51c4279700 1 --2- 172.21.15.102:0/439796124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51bc0ffac0 0x7f51bc195270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.768+0000 7f51c4279700 1 --2- 172.21.15.102:0/439796124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51bc100cc0 0x7f51bc1957b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.768+0000 7f51c4279700 1 -- 172.21.15.102:0/439796124 >> 172.21.15.102:0/439796124 conn(0x7f51bc0fb060 msgr2=0x7f51bc103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.768+0000 7f51c4279700 1 -- 172.21.15.102:0/439796124 shutdown_connections 2024-05-13T22:51:24.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.768+0000 7f51c4279700 1 -- 172.21.15.102:0/439796124 wait complete. 2024-05-13T22:51:24.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.875+0000 7f983920c700 1 -- 172.21.15.102:0/2991360410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9834101660 msgr2=0x7f9834103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.875+0000 7f983920c700 1 --2- 172.21.15.102:0/2991360410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9834101660 0x7f9834103a50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9824009910 tx=0x7f9824009c20 comp rx=0 tx=0).stop 2024-05-13T22:51:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.875+0000 7f983920c700 1 -- 172.21.15.102:0/2991360410 shutdown_connections 2024-05-13T22:51:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.875+0000 7f983920c700 1 --2- 172.21.15.102:0/2991360410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9834103f90 0x7f9834106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.875+0000 7f983920c700 1 --2- 172.21.15.102:0/2991360410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9834101660 0x7f9834103a50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.875+0000 7f983920c700 1 -- 172.21.15.102:0/2991360410 >> 172.21.15.102:0/2991360410 conn(0x7f98340fb050 msgr2=0x7f98340fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.875+0000 7f983920c700 1 -- 172.21.15.102:0/2991360410 shutdown_connections 2024-05-13T22:51:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.875+0000 7f983920c700 1 -- 172.21.15.102:0/2991360410 wait complete. 2024-05-13T22:51:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.875+0000 7f983920c700 1 Processor -- start 2024-05-13T22:51:24.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f983920c700 1 -- start start 2024-05-13T22:51:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f983920c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9834101660 0x7f9834196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f983920c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9834103f90 0x7f9834196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f983920c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9834196eb0 con 0x7f9834103f90 2024-05-13T22:51:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f983920c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9834196ff0 con 0x7f9834101660 2024-05-13T22:51:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f982a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9834103f90 0x7f9834196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f982a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9834103f90 0x7f9834196890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56820/0 (socket says 172.21.15.102:56820) 2024-05-13T22:51:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f982a59c700 1 -- 172.21.15.102:0/1859473379 learned_addr learned my addr 172.21.15.102:0/1859473379 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:51:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f9832d9d700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9834101660 0x7f9834196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f982a59c700 1 -- 172.21.15.102:0/1859473379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9834101660 msgr2=0x7f9834196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:24.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f982a59c700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9834101660 0x7f9834196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:24.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f982a59c700 1 -- 172.21.15.102:0/1859473379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98240095f0 con 0x7f9834103f90 2024-05-13T22:51:24.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f9832d9d700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9834101660 0x7f9834196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:51:24.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.876+0000 7f982a59c700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9834103f90 0x7f9834196890 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f981c00e920 tx=0x7f981c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:24.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.877+0000 7f9830d99700 1 -- 172.21.15.102:0/1859473379 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f981c009790 con 0x7f9834103f90 2024-05-13T22:51:24.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.877+0000 7f983920c700 1 -- 172.21.15.102:0/1859473379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983419baa0 con 0x7f9834103f90 2024-05-13T22:51:24.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.877+0000 7f9830d99700 1 -- 172.21.15.102:0/1859473379 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f981c00cba0 con 0x7f9834103f90 2024-05-13T22:51:24.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.877+0000 7f983920c700 1 -- 172.21.15.102:0/1859473379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983419bff0 con 0x7f9834103f90 2024-05-13T22:51:24.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.877+0000 7f9830d99700 1 -- 172.21.15.102:0/1859473379 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f981c018630 con 0x7f9834103f90 2024-05-13T22:51:24.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.878+0000 7f983920c700 1 -- 172.21.15.102:0/1859473379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9834190540 con 0x7f9834103f90 2024-05-13T22:51:24.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.878+0000 7f9830d99700 1 -- 172.21.15.102:0/1859473379 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f981c010430 con 0x7f9834103f90 2024-05-13T22:51:24.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.879+0000 7f9830d99700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f982006de80 0x7f9820070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:24.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.879+0000 7f9830d99700 1 -- 172.21.15.102:0/1859473379 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f981c014070 con 0x7f9834103f90 2024-05-13T22:51:24.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.879+0000 7f9832d9d700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f982006de80 0x7f9820070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:24.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.879+0000 7f9832d9d700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f982006de80 0x7f9820070340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9824009fd0 tx=0x7f9824005940 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:24.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:24.883+0000 7f9830d99700 1 -- 172.21.15.102:0/1859473379 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f981c053590 con 0x7f9834103f90 2024-05-13T22:51:24.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:24 smithi130 ceph-mon[36591]: pgmap v80: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:25.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.095+0000 7f983920c700 1 -- 172.21.15.102:0/1859473379 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f98340611d0 con 0x7f982006de80 2024-05-13T22:51:25.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.105+0000 7f9830d99700 1 -- 172.21.15.102:0/1859473379 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f98340611d0 con 0x7f982006de80 2024-05-13T22:51:25.084 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:51:25.084 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (34s) 4s ago 11m 16.5M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:51:25.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (11m) 4s ago 11m 8044k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:51:25.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (10m) 69s ago 10m 7662k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:51:25.084 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (11m) 4s ago 11m 6660k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:51:25.084 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (10m) 69s ago 10m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:51:25.084 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (10s) 4s ago 11m 53.0M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:51:25.084 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (6m) 4s ago 6m 88.2M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (6m) 4s ago 6m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (6m) 69s ago 6m 14.3M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (6m) 69s ago 6m 17.5M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (2m) 4s ago 13m 508M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (107s) 69s ago 10m 450M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (13m) 4s ago 13m 55.3M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (10m) 69s ago 10m 46.6M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (82s) 4s ago 11m 16.0M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (73s) 69s ago 10m 3670k - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (9m) 4s ago 9m 224M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (9m) 4s ago 9m 238M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (8m) 4s ago 8m 202M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:51:25.085 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (8m) 69s ago 8m 288M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:51:25.086 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (8m) 69s ago 8m 244M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:51:25.086 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (7m) 69s ago 7m 224M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:51:25.086 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (52s) 4s ago 11m 54.0M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:51:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.107+0000 7f983920c700 1 -- 172.21.15.102:0/1859473379 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f982006de80 msgr2=0x7f9820070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.107+0000 7f983920c700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f982006de80 0x7f9820070340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9824009fd0 tx=0x7f9824005940 comp rx=0 tx=0).stop 2024-05-13T22:51:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.107+0000 7f983920c700 1 -- 172.21.15.102:0/1859473379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9834103f90 msgr2=0x7f9834196890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.107+0000 7f983920c700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9834103f90 0x7f9834196890 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f981c00e920 tx=0x7f981c00ece0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.108+0000 7f983920c700 1 -- 172.21.15.102:0/1859473379 shutdown_connections 2024-05-13T22:51:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.108+0000 7f983920c700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f982006de80 0x7f9820070340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.108+0000 7f983920c700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9834101660 0x7f9834196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.108+0000 7f983920c700 1 --2- 172.21.15.102:0/1859473379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9834103f90 0x7f9834196890 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.108+0000 7f983920c700 1 -- 172.21.15.102:0/1859473379 >> 172.21.15.102:0/1859473379 conn(0x7f98340fb050 msgr2=0x7f98340fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:25.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.108+0000 7f983920c700 1 -- 172.21.15.102:0/1859473379 shutdown_connections 2024-05-13T22:51:25.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.108+0000 7f983920c700 1 -- 172.21.15.102:0/1859473379 wait complete. 2024-05-13T22:51:25.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.214+0000 7f485cccc700 1 -- 172.21.15.102:0/3889793601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48580ffab0 msgr2=0x7f48580ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:25.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.214+0000 7f485cccc700 1 --2- 172.21.15.102:0/3889793601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48580ffab0 0x7f48580ffed0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4848009910 tx=0x7f4848009c20 comp rx=0 tx=0).stop 2024-05-13T22:51:25.194 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.214+0000 7f485cccc700 1 -- 172.21.15.102:0/3889793601 shutdown_connections 2024-05-13T22:51:25.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.214+0000 7f485cccc700 1 --2- 172.21.15.102:0/3889793601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4858100cb0 0x7f4858101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.214+0000 7f485cccc700 1 --2- 172.21.15.102:0/3889793601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48580ffab0 0x7f48580ffed0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.214+0000 7f485cccc700 1 -- 172.21.15.102:0/3889793601 >> 172.21.15.102:0/3889793601 conn(0x7f48580fb030 msgr2=0x7f48580fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:25.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.214+0000 7f485cccc700 1 -- 172.21.15.102:0/3889793601 shutdown_connections 2024-05-13T22:51:25.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.214+0000 7f485cccc700 1 -- 172.21.15.102:0/3889793601 wait complete. 2024-05-13T22:51:25.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f485cccc700 1 Processor -- start 2024-05-13T22:51:25.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f485cccc700 1 -- start start 2024-05-13T22:51:25.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f485cccc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48580ffab0 0x7f4858195240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:25.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f485cccc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4858100cb0 0x7f4858195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:25.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f485cccc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4858195da0 con 0x7f4858100cb0 2024-05-13T22:51:25.195 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f485cccc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4858195ee0 con 0x7f48580ffab0 2024-05-13T22:51:25.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f4855d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4858100cb0 0x7f4858195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:25.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f4855d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4858100cb0 0x7f4858195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56844/0 (socket says 172.21.15.102:56844) 2024-05-13T22:51:25.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f4855d9b700 1 -- 172.21.15.102:0/286393224 learned_addr learned my addr 172.21.15.102:0/286393224 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:51:25.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f485659c700 1 --2- 172.21.15.102:0/286393224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48580ffab0 0x7f4858195240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:25.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f4855d9b700 1 -- 172.21.15.102:0/286393224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48580ffab0 msgr2=0x7f4858195240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:25.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f4855d9b700 1 --2- 172.21.15.102:0/286393224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48580ffab0 0x7f4858195240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f4855d9b700 1 -- 172.21.15.102:0/286393224 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48480095f0 con 0x7f4858100cb0 2024-05-13T22:51:25.196 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.215+0000 7f4855d9b700 1 --2- 172.21.15.102:0/286393224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4858100cb0 0x7f4858195780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f484c00d8d0 tx=0x7f484c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:25.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.216+0000 7f48477fe700 1 -- 172.21.15.102:0/286393224 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f484c0098a0 con 0x7f4858100cb0 2024-05-13T22:51:25.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.216+0000 7f48477fe700 1 -- 172.21.15.102:0/286393224 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f484c010460 con 0x7f4858100cb0 2024-05-13T22:51:25.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.216+0000 7f485cccc700 1 -- 172.21.15.102:0/286393224 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485819a990 con 0x7f4858100cb0 2024-05-13T22:51:25.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.216+0000 7f48477fe700 1 -- 172.21.15.102:0/286393224 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f484c00f5f0 con 0x7f4858100cb0 2024-05-13T22:51:25.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.216+0000 7f485cccc700 1 -- 172.21.15.102:0/286393224 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485819aee0 con 0x7f4858100cb0 2024-05-13T22:51:25.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.217+0000 7f485cccc700 1 -- 172.21.15.102:0/286393224 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4858108610 con 0x7f4858100cb0 2024-05-13T22:51:25.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.217+0000 7f48477fe700 1 -- 172.21.15.102:0/286393224 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f484c009a00 con 0x7f4858100cb0 2024-05-13T22:51:25.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.218+0000 7f48477fe700 1 --2- 172.21.15.102:0/286393224 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f484006de80 0x7f4840070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:25.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.218+0000 7f485659c700 1 --2- 172.21.15.102:0/286393224 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f484006de80 0x7f4840070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:25.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.218+0000 7f48477fe700 1 -- 172.21.15.102:0/286393224 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f484c08dab0 con 0x7f4858100cb0 2024-05-13T22:51:25.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.218+0000 7f485659c700 1 --2- 172.21.15.102:0/286393224 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f484006de80 0x7f4840070340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4848006010 tx=0x7f484800b540 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:25.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.222+0000 7f48477fe700 1 -- 172.21.15.102:0/286393224 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f484c05adf0 con 0x7f4858100cb0 2024-05-13T22:51:25.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.521+0000 7f485cccc700 1 -- 172.21.15.102:0/286393224 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f48581087d0 con 0x7f4858100cb0 2024-05-13T22:51:25.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.522+0000 7f48477fe700 1 -- 172.21.15.102:0/286393224 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f484c05a980 con 0x7f4858100cb0 2024-05-13T22:51:25.501 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:51:25.501 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:51:25.501 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:51:25.501 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:51:25.502 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:51:25.502 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:51:25.502 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:51:25.502 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:51:25.502 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:51:25.502 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:51:25.503 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:51:25.503 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:51:25.503 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:51:25.503 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:51:25.503 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-05-13T22:51:25.503 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:51:25.504 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:51:25.504 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:51:25.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.524+0000 7f485cccc700 1 -- 172.21.15.102:0/286393224 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f484006de80 msgr2=0x7f4840070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:25.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.524+0000 7f485cccc700 1 --2- 172.21.15.102:0/286393224 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f484006de80 0x7f4840070340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4848006010 tx=0x7f484800b540 comp rx=0 tx=0).stop 2024-05-13T22:51:25.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.524+0000 7f485cccc700 1 -- 172.21.15.102:0/286393224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4858100cb0 msgr2=0x7f4858195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:25.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.524+0000 7f485cccc700 1 --2- 172.21.15.102:0/286393224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4858100cb0 0x7f4858195780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f484c00d8d0 tx=0x7f484c00dbe0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.524+0000 7f485cccc700 1 -- 172.21.15.102:0/286393224 shutdown_connections 2024-05-13T22:51:25.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.524+0000 7f485cccc700 1 --2- 172.21.15.102:0/286393224 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f484006de80 0x7f4840070340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.524+0000 7f485cccc700 1 --2- 172.21.15.102:0/286393224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48580ffab0 0x7f4858195240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.524+0000 7f485cccc700 1 --2- 172.21.15.102:0/286393224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4858100cb0 0x7f4858195780 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.524+0000 7f485cccc700 1 -- 172.21.15.102:0/286393224 >> 172.21.15.102:0/286393224 conn(0x7f48580fb030 msgr2=0x7f4858103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:25.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.524+0000 7f485cccc700 1 -- 172.21.15.102:0/286393224 shutdown_connections 2024-05-13T22:51:25.506 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.524+0000 7f485cccc700 1 -- 172.21.15.102:0/286393224 wait complete. 2024-05-13T22:51:25.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.650+0000 7fc348d54700 1 -- 172.21.15.102:0/3393295975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc344100be0 msgr2=0x7fc344101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:25.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.650+0000 7fc348d54700 1 --2- 172.21.15.102:0/3393295975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc344100be0 0x7fc344101000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc334009910 tx=0x7fc334009c20 comp rx=0 tx=0).stop 2024-05-13T22:51:25.629 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:25 smithi102 ceph-mon[28599]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:51:25.629 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:25 smithi102 ceph-mon[28599]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:51:25.630 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:25 smithi102 ceph-mon[28599]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:51:25.630 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:25 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:25.630 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:25 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:25.630 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:25 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:51:25.630 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:25 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:51:25.630 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:25 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:25.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.651+0000 7fc348d54700 1 -- 172.21.15.102:0/3393295975 shutdown_connections 2024-05-13T22:51:25.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.651+0000 7fc348d54700 1 --2- 172.21.15.102:0/3393295975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc344101de0 0x7fc344102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.651+0000 7fc348d54700 1 --2- 172.21.15.102:0/3393295975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc344100be0 0x7fc344101000 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.651+0000 7fc348d54700 1 -- 172.21.15.102:0/3393295975 >> 172.21.15.102:0/3393295975 conn(0x7fc3440fc140 msgr2=0x7fc3440fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:25.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.651+0000 7fc348d54700 1 -- 172.21.15.102:0/3393295975 shutdown_connections 2024-05-13T22:51:25.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.651+0000 7fc348d54700 1 -- 172.21.15.102:0/3393295975 wait complete. 2024-05-13T22:51:25.632 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc348d54700 1 Processor -- start 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc348d54700 1 -- start start 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc348d54700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc344100be0 0x7fc344196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc348d54700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc344101de0 0x7fc3441968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc348d54700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344196ef0 con 0x7fc344101de0 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc348d54700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344197030 con 0x7fc344100be0 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc341d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc344101de0 0x7fc3441968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc341d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc344101de0 0x7fc3441968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:56858/0 (socket says 172.21.15.102:56858) 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc341d9b700 1 -- 172.21.15.102:0/2633176705 learned_addr learned my addr 172.21.15.102:0/2633176705 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc34259c700 1 --2- 172.21.15.102:0/2633176705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc344100be0 0x7fc344196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc341d9b700 1 -- 172.21.15.102:0/2633176705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc344100be0 msgr2=0x7fc344196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc341d9b700 1 --2- 172.21.15.102:0/2633176705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc344100be0 0x7fc344196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.652+0000 7fc341d9b700 1 -- 172.21.15.102:0/2633176705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3340095f0 con 0x7fc344101de0 2024-05-13T22:51:25.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.653+0000 7fc341d9b700 1 --2- 172.21.15.102:0/2633176705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc344101de0 0x7fc3441968d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc33800d8d0 tx=0x7fc33800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:25.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.653+0000 7fc3337fe700 1 -- 172.21.15.102:0/2633176705 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3380098a0 con 0x7fc344101de0 2024-05-13T22:51:25.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.653+0000 7fc348d54700 1 -- 172.21.15.102:0/2633176705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34419bae0 con 0x7fc344101de0 2024-05-13T22:51:25.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.653+0000 7fc3337fe700 1 -- 172.21.15.102:0/2633176705 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc338010460 con 0x7fc344101de0 2024-05-13T22:51:25.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.653+0000 7fc3337fe700 1 -- 172.21.15.102:0/2633176705 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc33800f5f0 con 0x7fc344101de0 2024-05-13T22:51:25.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.653+0000 7fc348d54700 1 -- 172.21.15.102:0/2633176705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34419c0b0 con 0x7fc344101de0 2024-05-13T22:51:25.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.654+0000 7fc348d54700 1 -- 172.21.15.102:0/2633176705 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc344109740 con 0x7fc344101de0 2024-05-13T22:51:25.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.654+0000 7fc3337fe700 1 -- 172.21.15.102:0/2633176705 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fc338009a00 con 0x7fc344101de0 2024-05-13T22:51:25.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.655+0000 7fc3337fe700 1 --2- 172.21.15.102:0/2633176705 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc32c06de80 0x7fc32c070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:25.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.655+0000 7fc3337fe700 1 -- 172.21.15.102:0/2633176705 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc33808da70 con 0x7fc344101de0 2024-05-13T22:51:25.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.655+0000 7fc34259c700 1 --2- 172.21.15.102:0/2633176705 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc32c06de80 0x7fc32c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:25.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.655+0000 7fc34259c700 1 --2- 172.21.15.102:0/2633176705 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc32c06de80 0x7fc32c070340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc334006010 tx=0x7fc33400b540 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:25.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.660+0000 7fc3337fe700 1 -- 172.21.15.102:0/2633176705 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fc33805adb0 con 0x7fc344101de0 2024-05-13T22:51:25.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.882+0000 7fc348d54700 1 -- 172.21.15.102:0/2633176705 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc344106730 con 0x7fc32c06de80 2024-05-13T22:51:25.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.883+0000 7fc3337fe700 1 -- 172.21.15.102:0/2633176705 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fc344106730 con 0x7fc32c06de80 2024-05-13T22:51:25.862 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:51:25.862 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": null, 2024-05-13T22:51:25.862 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": false, 2024-05-13T22:51:25.863 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "", 2024-05-13T22:51:25.863 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [], 2024-05-13T22:51:25.863 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": null, 2024-05-13T22:51:25.863 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "", 2024-05-13T22:51:25.863 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:51:25.863 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:51:25.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.885+0000 7fc348d54700 1 -- 172.21.15.102:0/2633176705 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc32c06de80 msgr2=0x7fc32c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:25.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.885+0000 7fc348d54700 1 --2- 172.21.15.102:0/2633176705 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc32c06de80 0x7fc32c070340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc334006010 tx=0x7fc33400b540 comp rx=0 tx=0).stop 2024-05-13T22:51:25.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.885+0000 7fc348d54700 1 -- 172.21.15.102:0/2633176705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc344101de0 msgr2=0x7fc3441968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:25.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.885+0000 7fc348d54700 1 --2- 172.21.15.102:0/2633176705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc344101de0 0x7fc3441968d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc33800d8d0 tx=0x7fc33800dbe0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.885+0000 7fc348d54700 1 -- 172.21.15.102:0/2633176705 shutdown_connections 2024-05-13T22:51:25.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.885+0000 7fc348d54700 1 --2- 172.21.15.102:0/2633176705 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fc32c06de80 0x7fc32c070340 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.885+0000 7fc348d54700 1 --2- 172.21.15.102:0/2633176705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc344100be0 0x7fc344196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.885+0000 7fc348d54700 1 --2- 172.21.15.102:0/2633176705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc344101de0 0x7fc3441968d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:25.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.885+0000 7fc348d54700 1 -- 172.21.15.102:0/2633176705 >> 172.21.15.102:0/2633176705 conn(0x7fc3440fc140 msgr2=0x7fc344105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:25.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.886+0000 7fc348d54700 1 -- 172.21.15.102:0/2633176705 shutdown_connections 2024-05-13T22:51:25.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:25.886+0000 7fc348d54700 1 -- 172.21.15.102:0/2633176705 wait complete. 2024-05-13T22:51:25.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:25 smithi130 ceph-mon[36591]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:51:25.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:25 smithi130 ceph-mon[36591]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:51:25.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:25 smithi130 ceph-mon[36591]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:51:25.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:25 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:25.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:25 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:25.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:25 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:51:25.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:25 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:51:25.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:25 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:26.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: pgmap v81: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:26.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/286393224' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-13T22:51:26.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-13T22:51:26.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:51:26.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: Upgrade: Finalizing container_image settings 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-13T22:51:26.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-13T22:51:26.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-13T22:51:26.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-13T22:51:26.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-13T22:51:26.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-13T22:51:26.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-13T22:51:26.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-13T22:51:26.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: Upgrade: Complete! 2024-05-13T22:51:26.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-13T22:51:26.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-13T22:51:26.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:51:26.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:51:26.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:51:26.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:51:26.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:26.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:51:26.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:51:26.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:51:26.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:26 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:26.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: pgmap v81: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:26.967 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/286393224' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.967 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-13T22:51:26.967 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-13T22:51:26.967 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:51:26.967 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.967 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.967 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.967 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.967 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.967 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.967 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: Upgrade: Finalizing container_image settings 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-13T22:51:26.968 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-13T22:51:26.969 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: Upgrade: Complete! 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:51:26.970 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:51:26.971 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:26.971 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:51:26.971 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:51:26.971 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:51:26.971 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:26 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:28.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:28 smithi130 ceph-mon[36591]: pgmap v82: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:28.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:28 smithi102 ceph-mon[28599]: pgmap v82: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:30 smithi130 ceph-mon[36591]: pgmap v83: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:30 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:30 smithi102 ceph-mon[28599]: pgmap v83: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:30 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:51:32.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:32 smithi130 ceph-mon[36591]: pgmap v84: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:32.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:32 smithi102 ceph-mon[28599]: pgmap v84: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:34.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:34 smithi130 ceph-mon[36591]: pgmap v85: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:34.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:34 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:51:34.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:34 smithi102 ceph-mon[28599]: pgmap v85: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:34.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:34 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:51:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:36 smithi130 ceph-mon[36591]: pgmap v86: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:36.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:36 smithi102 ceph-mon[28599]: pgmap v86: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:38.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:38 smithi102 ceph-mon[28599]: pgmap v87: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:38.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:38 smithi130 ceph-mon[36591]: pgmap v87: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:40.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:40 smithi130 ceph-mon[36591]: pgmap v88: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:40.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:40 smithi102 ceph-mon[28599]: pgmap v88: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:42.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:42 smithi102 ceph-mon[28599]: pgmap v89: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:42.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:42 smithi130 ceph-mon[36591]: pgmap v89: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:44.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:44 smithi130 ceph-mon[36591]: pgmap v90: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:44.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:44 smithi102 ceph-mon[28599]: pgmap v90: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:46 smithi130 ceph-mon[36591]: pgmap v91: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:46.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:46 smithi102 ceph-mon[28599]: pgmap v91: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:48.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:48 smithi102 ceph-mon[28599]: pgmap v92: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:48.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:48 smithi130 ceph-mon[36591]: pgmap v92: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:49.792 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:49 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:51:49.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:49 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:51:50.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:50 smithi130 ceph-mon[36591]: pgmap v93: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:50.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:50 smithi102 ceph-mon[28599]: pgmap v93: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:52.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:52 smithi102 ceph-mon[28599]: pgmap v94: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:52.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:52 smithi130 ceph-mon[36591]: pgmap v94: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:54.702 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:54 smithi130 ceph-mon[36591]: pgmap v95: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:54.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:54 smithi102 ceph-mon[28599]: pgmap v95: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:55.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.004+0000 7f83220b2700 1 -- 172.21.15.102:0/1474954124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f831c101f30 msgr2=0x7f831c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:55.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.004+0000 7f83220b2700 1 --2- 172.21.15.102:0/1474954124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f831c101f30 0x7f831c102390 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8310009910 tx=0x7f8310009c20 comp rx=0 tx=0).stop 2024-05-13T22:51:55.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.004+0000 7f83220b2700 1 -- 172.21.15.102:0/1474954124 shutdown_connections 2024-05-13T22:51:55.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.004+0000 7f83220b2700 1 --2- 172.21.15.102:0/1474954124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f831c101f30 0x7f831c102390 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:55.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.004+0000 7f83220b2700 1 --2- 172.21.15.102:0/1474954124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f831c0feeb0 0x7f831c0ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:55.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.004+0000 7f83220b2700 1 -- 172.21.15.102:0/1474954124 >> 172.21.15.102:0/1474954124 conn(0x7f831c0fa7f0 msgr2=0x7f831c0fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:55.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.004+0000 7f83220b2700 1 -- 172.21.15.102:0/1474954124 shutdown_connections 2024-05-13T22:51:55.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.004+0000 7f83220b2700 1 -- 172.21.15.102:0/1474954124 wait complete. 2024-05-13T22:51:55.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f83220b2700 1 Processor -- start 2024-05-13T22:51:55.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f83220b2700 1 -- start start 2024-05-13T22:51:55.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f83220b2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f831c0feeb0 0x7f831c1964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:55.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f83220b2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f831c101f30 0x7f831c1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:55.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f83220b2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c197000 con 0x7f831c101f30 2024-05-13T22:51:55.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f83220b2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c197140 con 0x7f831c0feeb0 2024-05-13T22:51:55.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f831affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f831c101f30 0x7f831c1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f831affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f831c101f30 0x7f831c1969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42802/0 (socket says 172.21.15.102:42802) 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f831affd700 1 -- 172.21.15.102:0/3385942512 learned_addr learned my addr 172.21.15.102:0/3385942512 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f831b7fe700 1 --2- 172.21.15.102:0/3385942512 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f831c0feeb0 0x7f831c1964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f831affd700 1 -- 172.21.15.102:0/3385942512 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f831c0feeb0 msgr2=0x7f831c1964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f831affd700 1 --2- 172.21.15.102:0/3385942512 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f831c0feeb0 0x7f831c1964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.005+0000 7f831affd700 1 -- 172.21.15.102:0/3385942512 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83100095f0 con 0x7f831c101f30 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.006+0000 7f831affd700 1 --2- 172.21.15.102:0/3385942512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f831c101f30 0x7f831c1969e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f831000bd70 tx=0x7f831000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.006+0000 7f8318ff9700 1 -- 172.21.15.102:0/3385942512 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f831001d070 con 0x7f831c101f30 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.006+0000 7f8318ff9700 1 -- 172.21.15.102:0/3385942512 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f831000f460 con 0x7f831c101f30 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.006+0000 7f83220b2700 1 -- 172.21.15.102:0/3385942512 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831c19bb90 con 0x7f831c101f30 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.006+0000 7f8318ff9700 1 -- 172.21.15.102:0/3385942512 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8310021640 con 0x7f831c101f30 2024-05-13T22:51:55.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.006+0000 7f83220b2700 1 -- 172.21.15.102:0/3385942512 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831c19c100 con 0x7f831c101f30 2024-05-13T22:51:55.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.007+0000 7f83220b2700 1 -- 172.21.15.102:0/3385942512 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831c1097d0 con 0x7f831c101f30 2024-05-13T22:51:55.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.008+0000 7f8318ff9700 1 -- 172.21.15.102:0/3385942512 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f831000fab0 con 0x7f831c101f30 2024-05-13T22:51:55.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.008+0000 7f8318ff9700 1 --2- 172.21.15.102:0/3385942512 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f830406de80 0x7f8304070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:55.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.008+0000 7f8318ff9700 1 -- 172.21.15.102:0/3385942512 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f831008f0b0 con 0x7f831c101f30 2024-05-13T22:51:55.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.008+0000 7f831b7fe700 1 --2- 172.21.15.102:0/3385942512 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f830406de80 0x7f8304070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:55.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.009+0000 7f831b7fe700 1 --2- 172.21.15.102:0/3385942512 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f830406de80 0x7f8304070340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f831c068b50 tx=0x7f830c0094f0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:55.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.013+0000 7f8318ff9700 1 -- 172.21.15.102:0/3385942512 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f831005c350 con 0x7f831c101f30 2024-05-13T22:51:56.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.232+0000 7f83220b2700 1 -- 172.21.15.102:0/3385942512 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f831c106880 con 0x7f830406de80 2024-05-13T22:51:56.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.233+0000 7f8318ff9700 1 -- 172.21.15.102:0/3385942512 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f831c106880 con 0x7f830406de80 2024-05-13T22:51:56.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.235+0000 7f83220b2700 1 -- 172.21.15.102:0/3385942512 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f830406de80 msgr2=0x7f8304070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:56.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.235+0000 7f83220b2700 1 --2- 172.21.15.102:0/3385942512 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f830406de80 0x7f8304070340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f831c068b50 tx=0x7f830c0094f0 comp rx=0 tx=0).stop 2024-05-13T22:51:56.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.235+0000 7f83220b2700 1 -- 172.21.15.102:0/3385942512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f831c101f30 msgr2=0x7f831c1969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:56.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.235+0000 7f83220b2700 1 --2- 172.21.15.102:0/3385942512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f831c101f30 0x7f831c1969e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f831000bd70 tx=0x7f831000be50 comp rx=0 tx=0).stop 2024-05-13T22:51:56.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.236+0000 7f83220b2700 1 -- 172.21.15.102:0/3385942512 shutdown_connections 2024-05-13T22:51:56.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.236+0000 7f83220b2700 1 --2- 172.21.15.102:0/3385942512 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f830406de80 0x7f8304070340 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:56.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.236+0000 7f83220b2700 1 --2- 172.21.15.102:0/3385942512 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f831c0feeb0 0x7f831c1964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:56.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.236+0000 7f83220b2700 1 --2- 172.21.15.102:0/3385942512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f831c101f30 0x7f831c1969e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:56.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.236+0000 7f83220b2700 1 -- 172.21.15.102:0/3385942512 >> 172.21.15.102:0/3385942512 conn(0x7f831c0fa7f0 msgr2=0x7f831c105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:56.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.236+0000 7f83220b2700 1 -- 172.21.15.102:0/3385942512 shutdown_connections 2024-05-13T22:51:56.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:56.236+0000 7f83220b2700 1 -- 172.21.15.102:0/3385942512 wait complete. 2024-05-13T22:51:56.536 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:56 smithi102 ceph-mon[28599]: pgmap v96: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:56.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:56 smithi130 ceph-mon[36591]: pgmap v96: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:51:56.885 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-05-13T22:51:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:51:57.482 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:57 smithi102 ceph-mon[28599]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:51:57.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:57 smithi130 ceph-mon[36591]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:51:58.547 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:58 smithi102 ceph-mon[28599]: pgmap v97: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:58.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:58 smithi130 ceph-mon[36591]: pgmap v97: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:51:58.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.797+0000 7f4a46ab7700 1 -- 172.21.15.102:0/647951339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a40105a40 msgr2=0x7f4a40105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:58.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.797+0000 7f4a46ab7700 1 --2- 172.21.15.102:0/647951339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a40105a40 0x7f4a40105e20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00b0a0 tx=0x7f4a3c00b3b0 comp rx=0 tx=0).stop 2024-05-13T22:51:58.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.797+0000 7f4a46ab7700 1 -- 172.21.15.102:0/647951339 shutdown_connections 2024-05-13T22:51:58.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.797+0000 7f4a46ab7700 1 --2- 172.21.15.102:0/647951339 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a400ffbd0 0x7f4a40100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:58.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.797+0000 7f4a46ab7700 1 --2- 172.21.15.102:0/647951339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a40105a40 0x7f4a40105e20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:58.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.797+0000 7f4a46ab7700 1 -- 172.21.15.102:0/647951339 >> 172.21.15.102:0/647951339 conn(0x7f4a400755a0 msgr2=0x7f4a400759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:58.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.797+0000 7f4a46ab7700 1 -- 172.21.15.102:0/647951339 shutdown_connections 2024-05-13T22:51:58.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.797+0000 7f4a46ab7700 1 -- 172.21.15.102:0/647951339 wait complete. 2024-05-13T22:51:58.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.797+0000 7f4a46ab7700 1 Processor -- start 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.797+0000 7f4a46ab7700 1 -- start start 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a46ab7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a400ffbd0 0x7f4a40195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a46ab7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a40105a40 0x7f4a40195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a46ab7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40196260 con 0x7f4a400ffbd0 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a46ab7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40199ff0 con 0x7f4a40105a40 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a44853700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a400ffbd0 0x7f4a40195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a44853700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a400ffbd0 0x7f4a40195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33978/0 (socket says 172.21.15.102:33978) 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a44853700 1 -- 172.21.15.102:0/1012217060 learned_addr learned my addr 172.21.15.102:0/1012217060 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a37fff700 1 --2- 172.21.15.102:0/1012217060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a40105a40 0x7f4a40195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a44853700 1 -- 172.21.15.102:0/1012217060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a40105a40 msgr2=0x7f4a40195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a44853700 1 --2- 172.21.15.102:0/1012217060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a40105a40 0x7f4a40195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a44853700 1 -- 172.21.15.102:0/1012217060 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c009d00 con 0x7f4a400ffbd0 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a44853700 1 --2- 172.21.15.102:0/1012217060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a400ffbd0 0x7f4a40195640 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c000f80 tx=0x7f4a3c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:58.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.798+0000 7f4a35ffb700 1 -- 172.21.15.102:0/1012217060 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a3c00e040 con 0x7f4a400ffbd0 2024-05-13T22:51:58.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.799+0000 7f4a46ab7700 1 -- 172.21.15.102:0/1012217060 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4019a270 con 0x7f4a400ffbd0 2024-05-13T22:51:58.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.799+0000 7f4a35ffb700 1 -- 172.21.15.102:0/1012217060 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a3c003de0 con 0x7f4a400ffbd0 2024-05-13T22:51:58.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.799+0000 7f4a35ffb700 1 -- 172.21.15.102:0/1012217060 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a3c01d7d0 con 0x7f4a400ffbd0 2024-05-13T22:51:58.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.799+0000 7f4a46ab7700 1 -- 172.21.15.102:0/1012217060 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4019a760 con 0x7f4a400ffbd0 2024-05-13T22:51:58.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.800+0000 7f4a46ab7700 1 -- 172.21.15.102:0/1012217060 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a40108040 con 0x7f4a400ffbd0 2024-05-13T22:51:58.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.800+0000 7f4a35ffb700 1 -- 172.21.15.102:0/1012217060 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f4a3c004040 con 0x7f4a400ffbd0 2024-05-13T22:51:58.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.801+0000 7f4a35ffb700 1 --2- 172.21.15.102:0/1012217060 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4a2806de80 0x7f4a28070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:51:58.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.801+0000 7f4a35ffb700 1 -- 172.21.15.102:0/1012217060 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4a3c08f500 con 0x7f4a400ffbd0 2024-05-13T22:51:58.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.801+0000 7f4a37fff700 1 --2- 172.21.15.102:0/1012217060 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4a2806de80 0x7f4a28070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:51:58.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.801+0000 7f4a37fff700 1 --2- 172.21.15.102:0/1012217060 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4a2806de80 0x7f4a28070340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a40196c60 tx=0x7f4a3000a300 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:51:58.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:58.805+0000 7f4a35ffb700 1 -- 172.21.15.102:0/1012217060 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f4a3c058d10 con 0x7f4a400ffbd0 2024-05-13T22:51:59.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.086+0000 7f4a46ab7700 1 -- 172.21.15.102:0/1012217060 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4a4002cd30 con 0x7f4a400ffbd0 2024-05-13T22:51:59.063 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.086+0000 7f4a35ffb700 1 -- 172.21.15.102:0/1012217060 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f4a3c05c3d0 con 0x7f4a400ffbd0 2024-05-13T22:51:59.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.088+0000 7f4a46ab7700 1 -- 172.21.15.102:0/1012217060 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4a2806de80 msgr2=0x7f4a28070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:59.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.088+0000 7f4a46ab7700 1 --2- 172.21.15.102:0/1012217060 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4a2806de80 0x7f4a28070340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4a40196c60 tx=0x7f4a3000a300 comp rx=0 tx=0).stop 2024-05-13T22:51:59.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.088+0000 7f4a46ab7700 1 -- 172.21.15.102:0/1012217060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a400ffbd0 msgr2=0x7f4a40195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:51:59.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.088+0000 7f4a46ab7700 1 --2- 172.21.15.102:0/1012217060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a400ffbd0 0x7f4a40195640 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c000f80 tx=0x7f4a3c00cd20 comp rx=0 tx=0).stop 2024-05-13T22:51:59.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.089+0000 7f4a46ab7700 1 -- 172.21.15.102:0/1012217060 shutdown_connections 2024-05-13T22:51:59.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.089+0000 7f4a46ab7700 1 --2- 172.21.15.102:0/1012217060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a400ffbd0 0x7f4a40195640 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:59.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.089+0000 7f4a46ab7700 1 --2- 172.21.15.102:0/1012217060 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f4a2806de80 0x7f4a28070340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:59.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.089+0000 7f4a46ab7700 1 --2- 172.21.15.102:0/1012217060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a40105a40 0x7f4a40195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:51:59.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.089+0000 7f4a46ab7700 1 -- 172.21.15.102:0/1012217060 >> 172.21.15.102:0/1012217060 conn(0x7f4a400755a0 msgr2=0x7f4a400fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:51:59.066 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.089+0000 7f4a46ab7700 1 -- 172.21.15.102:0/1012217060 shutdown_connections 2024-05-13T22:51:59.067 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:51:59.089+0000 7f4a46ab7700 1 -- 172.21.15.102:0/1012217060 wait complete. 2024-05-13T22:51:59.077 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:51:59.549 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:51:59 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1012217060' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:59.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:51:59 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1012217060' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:51:59.729 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-05-13T22:51:59.940 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:52:00.353 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:00 smithi102 ceph-mon[28599]: pgmap v98: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:00.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:00 smithi130 ceph-mon[36591]: pgmap v98: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:01.775 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.796+0000 7f86b3d52700 1 -- 172.21.15.102:0/336935466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86ac105a40 msgr2=0x7f86ac105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:01.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.796+0000 7f86b3d52700 1 --2- 172.21.15.102:0/336935466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86ac105a40 0x7f86ac105e20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f869c009910 tx=0x7f869c009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:01.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.796+0000 7f86b3d52700 1 -- 172.21.15.102:0/336935466 shutdown_connections 2024-05-13T22:52:01.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.796+0000 7f86b3d52700 1 --2- 172.21.15.102:0/336935466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86ac0ffbd0 0x7f86ac100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:01.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.796+0000 7f86b3d52700 1 --2- 172.21.15.102:0/336935466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86ac105a40 0x7f86ac105e20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:01.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.796+0000 7f86b3d52700 1 -- 172.21.15.102:0/336935466 >> 172.21.15.102:0/336935466 conn(0x7f86ac0755a0 msgr2=0x7f86ac0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:01.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.796+0000 7f86b3d52700 1 -- 172.21.15.102:0/336935466 shutdown_connections 2024-05-13T22:52:01.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.796+0000 7f86b3d52700 1 -- 172.21.15.102:0/336935466 wait complete. 2024-05-13T22:52:01.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b3d52700 1 Processor -- start 2024-05-13T22:52:01.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b3d52700 1 -- start start 2024-05-13T22:52:01.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b3d52700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86ac0ffbd0 0x7f86ac197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:01.776 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b3d52700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86ac105a40 0x7f86ac197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b3d52700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac198320 con 0x7f86ac0ffbd0 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b3d52700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac198490 con 0x7f86ac105a40 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b1aee700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86ac0ffbd0 0x7f86ac197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b1aee700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86ac0ffbd0 0x7f86ac197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33988/0 (socket says 172.21.15.102:33988) 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b1aee700 1 -- 172.21.15.102:0/25305584 learned_addr learned my addr 172.21.15.102:0/25305584 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b1aee700 1 -- 172.21.15.102:0/25305584 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86ac105a40 msgr2=0x7f86ac197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b1aee700 1 --2- 172.21.15.102:0/25305584 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86ac105a40 0x7f86ac197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.797+0000 7f86b1aee700 1 -- 172.21.15.102:0/25305584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0095f0 con 0x7f86ac0ffbd0 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.798+0000 7f86b1aee700 1 --2- 172.21.15.102:0/25305584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86ac0ffbd0 0x7f86ac197700 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f869c006010 tx=0x7f869c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.798+0000 7f86a2ffd700 1 -- 172.21.15.102:0/25305584 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f869c01d070 con 0x7f86ac0ffbd0 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.798+0000 7f86b3d52700 1 -- 172.21.15.102:0/25305584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ac1a0770 con 0x7f86ac0ffbd0 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.798+0000 7f86a2ffd700 1 -- 172.21.15.102:0/25305584 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f869c004fb0 con 0x7f86ac0ffbd0 2024-05-13T22:52:01.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.798+0000 7f86a2ffd700 1 -- 172.21.15.102:0/25305584 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f869c022ea0 con 0x7f86ac0ffbd0 2024-05-13T22:52:01.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.798+0000 7f86b3d52700 1 -- 172.21.15.102:0/25305584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ac1a0be0 con 0x7f86ac0ffbd0 2024-05-13T22:52:01.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.799+0000 7f86b3d52700 1 -- 172.21.15.102:0/25305584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ac108040 con 0x7f86ac0ffbd0 2024-05-13T22:52:01.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.799+0000 7f86a2ffd700 1 -- 172.21.15.102:0/25305584 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f869c022470 con 0x7f86ac0ffbd0 2024-05-13T22:52:01.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.800+0000 7f86a2ffd700 1 --2- 172.21.15.102:0/25305584 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f869806de80 0x7f8698070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:01.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.800+0000 7f86a2ffd700 1 -- 172.21.15.102:0/25305584 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f869c08f150 con 0x7f86ac0ffbd0 2024-05-13T22:52:01.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.800+0000 7f86b12ed700 1 --2- 172.21.15.102:0/25305584 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f869806de80 0x7f8698070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:01.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.801+0000 7f86b12ed700 1 --2- 172.21.15.102:0/25305584 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f869806de80 0x7f8698070340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f86ac068cc0 tx=0x7f86a8006d20 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:01.782 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:01.805+0000 7f86a2ffd700 1 -- 172.21.15.102:0/25305584 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f869c058a10 con 0x7f86ac0ffbd0 2024-05-13T22:52:02.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.088+0000 7f86b3d52700 1 -- 172.21.15.102:0/25305584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f86ac02cc70 con 0x7f86ac0ffbd0 2024-05-13T22:52:02.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.088+0000 7f86a2ffd700 1 -- 172.21.15.102:0/25305584 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f869c022720 con 0x7f86ac0ffbd0 2024-05-13T22:52:02.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.090+0000 7f86b3d52700 1 -- 172.21.15.102:0/25305584 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f869806de80 msgr2=0x7f8698070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:02.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.090+0000 7f86b3d52700 1 --2- 172.21.15.102:0/25305584 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f869806de80 0x7f8698070340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f86ac068cc0 tx=0x7f86a8006d20 comp rx=0 tx=0).stop 2024-05-13T22:52:02.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.090+0000 7f86b3d52700 1 -- 172.21.15.102:0/25305584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86ac0ffbd0 msgr2=0x7f86ac197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:02.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.090+0000 7f86b3d52700 1 --2- 172.21.15.102:0/25305584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86ac0ffbd0 0x7f86ac197700 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f869c006010 tx=0x7f869c00be20 comp rx=0 tx=0).stop 2024-05-13T22:52:02.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.090+0000 7f86b3d52700 1 -- 172.21.15.102:0/25305584 shutdown_connections 2024-05-13T22:52:02.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.090+0000 7f86b3d52700 1 --2- 172.21.15.102:0/25305584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f86ac0ffbd0 0x7f86ac197700 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:02.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.090+0000 7f86b3d52700 1 --2- 172.21.15.102:0/25305584 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f869806de80 0x7f8698070340 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:02.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.090+0000 7f86b3d52700 1 --2- 172.21.15.102:0/25305584 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f86ac105a40 0x7f86ac197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:02.068 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.090+0000 7f86b3d52700 1 -- 172.21.15.102:0/25305584 >> 172.21.15.102:0/25305584 conn(0x7f86ac0755a0 msgr2=0x7f86ac0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.090+0000 7f86b3d52700 1 -- 172.21.15.102:0/25305584 shutdown_connections 2024-05-13T22:52:02.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:02.090+0000 7f86b3d52700 1 -- 172.21.15.102:0/25305584 wait complete. 2024-05-13T22:52:02.078 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)" 2024-05-13T22:52:02.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:02 smithi102 ceph-mon[28599]: pgmap v99: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:02.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:02 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/25305584' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:52:02.748 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-05-13T22:52:02.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:02 smithi130 ceph-mon[36591]: pgmap v99: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:02.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:02 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/25305584' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:52:02.955 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:52:04.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.628+0000 7fbb7999c700 1 -- 172.21.15.102:0/997932194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb74102620 msgr2=0x7fbb7410ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:04.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.628+0000 7fbb7999c700 1 --2- 172.21.15.102:0/997932194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb74102620 0x7fbb7410ab10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009910 tx=0x7fbb5c009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:04.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.628+0000 7fbb7999c700 1 -- 172.21.15.102:0/997932194 shutdown_connections 2024-05-13T22:52:04.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.628+0000 7fbb7999c700 1 --2- 172.21.15.102:0/997932194 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb74102620 0x7fbb7410ab10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:04.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.628+0000 7fbb7999c700 1 --2- 172.21.15.102:0/997932194 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb74101d00 0x7fbb741020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:04.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.628+0000 7fbb7999c700 1 -- 172.21.15.102:0/997932194 >> 172.21.15.102:0/997932194 conn(0x7fbb740fb570 msgr2=0x7fbb740fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:04.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.628+0000 7fbb7999c700 1 -- 172.21.15.102:0/997932194 shutdown_connections 2024-05-13T22:52:04.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.628+0000 7fbb7999c700 1 -- 172.21.15.102:0/997932194 wait complete. 2024-05-13T22:52:04.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb7999c700 1 Processor -- start 2024-05-13T22:52:04.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb7999c700 1 -- start start 2024-05-13T22:52:04.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb7999c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb74101d00 0x7fbb7419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:04.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb7999c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb74102620 0x7fbb7419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:04.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb7999c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7419f2c0 con 0x7fbb74101d00 2024-05-13T22:52:04.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb7999c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb74198800 con 0x7fbb74102620 2024-05-13T22:52:04.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb72ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb74101d00 0x7fbb7419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:04.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb72ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb74101d00 0x7fbb7419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34008/0 (socket says 172.21.15.102:34008) 2024-05-13T22:52:04.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb72ffd700 1 -- 172.21.15.102:0/950335542 learned_addr learned my addr 172.21.15.102:0/950335542 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:04.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb72ffd700 1 -- 172.21.15.102:0/950335542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb74102620 msgr2=0x7fbb7419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:04.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb6a7fc700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb74102620 0x7fbb7419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:04.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb72ffd700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb74102620 0x7fbb7419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:04.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.629+0000 7fbb72ffd700 1 -- 172.21.15.102:0/950335542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c0095f0 con 0x7fbb74101d00 2024-05-13T22:52:04.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.630+0000 7fbb6a7fc700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb74102620 0x7fbb7419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:52:04.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.630+0000 7fbb72ffd700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb74101d00 0x7fbb7419e780 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c930 tx=0x7fbb6400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:04.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.630+0000 7fbb7899a700 1 -- 172.21.15.102:0/950335542 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb64007ad0 con 0x7fbb74101d00 2024-05-13T22:52:04.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.630+0000 7fbb7999c700 1 -- 172.21.15.102:0/950335542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb74198ae0 con 0x7fbb74101d00 2024-05-13T22:52:04.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.630+0000 7fbb7899a700 1 -- 172.21.15.102:0/950335542 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb64007c30 con 0x7fbb74101d00 2024-05-13T22:52:04.609 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:04 smithi102 ceph-mon[28599]: pgmap v100: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:04.610 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:04 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:52:04.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.630+0000 7fbb7999c700 1 -- 172.21.15.102:0/950335542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb74199030 con 0x7fbb74101d00 2024-05-13T22:52:04.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.630+0000 7fbb7899a700 1 -- 172.21.15.102:0/950335542 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb64018610 con 0x7fbb74101d00 2024-05-13T22:52:04.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.631+0000 7fbb7999c700 1 -- 172.21.15.102:0/950335542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7410a610 con 0x7fbb74101d00 2024-05-13T22:52:04.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.631+0000 7fbb7899a700 1 -- 172.21.15.102:0/950335542 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fbb6400f450 con 0x7fbb74101d00 2024-05-13T22:52:04.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.632+0000 7fbb7899a700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fbb6006de80 0x7fbb60070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:04.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.632+0000 7fbb7899a700 1 -- 172.21.15.102:0/950335542 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbb6408dde0 con 0x7fbb74101d00 2024-05-13T22:52:04.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.632+0000 7fbb6a7fc700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fbb6006de80 0x7fbb60070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:04.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.632+0000 7fbb6a7fc700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fbb6006de80 0x7fbb60070340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c006010 tx=0x7fbb5c005960 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:04.613 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.636+0000 7fbb7899a700 1 -- 172.21.15.102:0/950335542 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fbb6405b120 con 0x7fbb74101d00 2024-05-13T22:52:04.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:04 smithi130 ceph-mon[36591]: pgmap v100: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:04.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:04 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:52:04.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.925+0000 7fbb7999c700 1 -- 172.21.15.102:0/950335542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbb74199db0 con 0x7fbb74101d00 2024-05-13T22:52:04.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.925+0000 7fbb7899a700 1 -- 172.21.15.102:0/950335542 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fbb6405acb0 con 0x7fbb74101d00 2024-05-13T22:52:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.927+0000 7fbb7999c700 1 -- 172.21.15.102:0/950335542 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fbb6006de80 msgr2=0x7fbb60070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.927+0000 7fbb7999c700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fbb6006de80 0x7fbb60070340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c006010 tx=0x7fbb5c005960 comp rx=0 tx=0).stop 2024-05-13T22:52:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.927+0000 7fbb7999c700 1 -- 172.21.15.102:0/950335542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb74101d00 msgr2=0x7fbb7419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.927+0000 7fbb7999c700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb74101d00 0x7fbb7419e780 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c930 tx=0x7fbb6400ccf0 comp rx=0 tx=0).stop 2024-05-13T22:52:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.928+0000 7fbb7999c700 1 -- 172.21.15.102:0/950335542 shutdown_connections 2024-05-13T22:52:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.928+0000 7fbb7999c700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb74101d00 0x7fbb7419e780 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.928+0000 7fbb7999c700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fbb6006de80 0x7fbb60070340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.928+0000 7fbb7999c700 1 --2- 172.21.15.102:0/950335542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb74102620 0x7fbb7419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:04.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.928+0000 7fbb7999c700 1 -- 172.21.15.102:0/950335542 >> 172.21.15.102:0/950335542 conn(0x7fbb740fb570 msgr2=0x7fbb74107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:04.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.928+0000 7fbb7999c700 1 -- 172.21.15.102:0/950335542 shutdown_connections 2024-05-13T22:52:04.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:04.928+0000 7fbb7999c700 1 -- 172.21.15.102:0/950335542 wait complete. 2024-05-13T22:52:04.916 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:52:05.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:05 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/950335542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:52:05.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:05 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/950335542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:52:05.803 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-05-13T22:52:06.017 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:52:06.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:06 smithi102 ceph-mon[28599]: pgmap v101: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:06.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:06 smithi130 ceph-mon[36591]: pgmap v101: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:07.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.717+0000 7fb56bf63700 1 -- 172.21.15.102:0/2554240028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5640ff530 msgr2=0x7fb5640ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.717+0000 7fb56bf63700 1 --2- 172.21.15.102:0/2554240028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5640ff530 0x7fb5640ff910 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb560009910 tx=0x7fb560009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.717+0000 7fb56bf63700 1 -- 172.21.15.102:0/2554240028 shutdown_connections 2024-05-13T22:52:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.717+0000 7fb56bf63700 1 --2- 172.21.15.102:0/2554240028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5640ffee0 0x7fb564105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.717+0000 7fb56bf63700 1 --2- 172.21.15.102:0/2554240028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5640ff530 0x7fb5640ff910 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.717+0000 7fb56bf63700 1 -- 172.21.15.102:0/2554240028 >> 172.21.15.102:0/2554240028 conn(0x7fb564074df0 msgr2=0x7fb564075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.717+0000 7fb56bf63700 1 -- 172.21.15.102:0/2554240028 shutdown_connections 2024-05-13T22:52:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.717+0000 7fb56bf63700 1 -- 172.21.15.102:0/2554240028 wait complete. 2024-05-13T22:52:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.717+0000 7fb56bf63700 1 Processor -- start 2024-05-13T22:52:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb56bf63700 1 -- start start 2024-05-13T22:52:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb56bf63700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5640ff530 0x7fb56419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:07.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb56bf63700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5640ffee0 0x7fb56419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb56bf63700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56419d1d0 con 0x7fb5640ffee0 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb56bf63700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb564196680 con 0x7fb5640ff530 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb5694fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5640ffee0 0x7fb56419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb5694fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5640ffee0 0x7fb56419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:34022/0 (socket says 172.21.15.102:34022) 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb5694fe700 1 -- 172.21.15.102:0/4265570094 learned_addr learned my addr 172.21.15.102:0/4265570094 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb569cff700 1 --2- 172.21.15.102:0/4265570094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5640ff530 0x7fb56419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb5694fe700 1 -- 172.21.15.102:0/4265570094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5640ff530 msgr2=0x7fb56419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb5694fe700 1 --2- 172.21.15.102:0/4265570094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5640ff530 0x7fb56419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb5694fe700 1 -- 172.21.15.102:0/4265570094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5600095f0 con 0x7fb5640ffee0 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb5694fe700 1 --2- 172.21.15.102:0/4265570094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5640ffee0 0x7fb56419cb40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb55400b480 tx=0x7fb55400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb55affd700 1 -- 172.21.15.102:0/4265570094 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5540118c0 con 0x7fb5640ffee0 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb55affd700 1 -- 172.21.15.102:0/4265570094 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb554010380 con 0x7fb5640ffee0 2024-05-13T22:52:07.697 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.718+0000 7fb56bf63700 1 -- 172.21.15.102:0/4265570094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb564196960 con 0x7fb5640ffee0 2024-05-13T22:52:07.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.719+0000 7fb55affd700 1 -- 172.21.15.102:0/4265570094 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb55400f4f0 con 0x7fb5640ffee0 2024-05-13T22:52:07.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.719+0000 7fb56bf63700 1 -- 172.21.15.102:0/4265570094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb564196eb0 con 0x7fb5640ffee0 2024-05-13T22:52:07.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.719+0000 7fb56bf63700 1 -- 172.21.15.102:0/4265570094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5641091e0 con 0x7fb5640ffee0 2024-05-13T22:52:07.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.720+0000 7fb55affd700 1 -- 172.21.15.102:0/4265570094 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fb5540104f0 con 0x7fb5640ffee0 2024-05-13T22:52:07.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.720+0000 7fb55affd700 1 --2- 172.21.15.102:0/4265570094 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fb55006de80 0x7fb550070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:07.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.720+0000 7fb55affd700 1 -- 172.21.15.102:0/4265570094 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb55408d760 con 0x7fb5640ffee0 2024-05-13T22:52:07.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.721+0000 7fb569cff700 1 --2- 172.21.15.102:0/4265570094 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fb55006de80 0x7fb550070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:07.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.721+0000 7fb569cff700 1 --2- 172.21.15.102:0/4265570094 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fb55006de80 0x7fb550070340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb56000b5c0 tx=0x7fb56001a040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:07.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.725+0000 7fb55affd700 1 -- 172.21.15.102:0/4265570094 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fb55405ab20 con 0x7fb5640ffee0 2024-05-13T22:52:07.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:07.944+0000 7fb56bf63700 1 -- 172.21.15.102:0/4265570094 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5641971f0 con 0x7fb55006de80 2024-05-13T22:52:08.833 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:08 smithi102 ceph-mon[28599]: pgmap v102: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:08.833 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:08 smithi102 ceph-mon[28599]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:08.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:08 smithi130 ceph-mon[36591]: pgmap v102: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:08.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:08 smithi130 ceph-mon[36591]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:09.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.014+0000 7fb55affd700 1 -- 172.21.15.102:0/4265570094 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7fb5641971f0 con 0x7fb55006de80 2024-05-13T22:52:09.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.016+0000 7fb56bf63700 1 -- 172.21.15.102:0/4265570094 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fb55006de80 msgr2=0x7fb550070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:09.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.016+0000 7fb56bf63700 1 --2- 172.21.15.102:0/4265570094 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fb55006de80 0x7fb550070340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb56000b5c0 tx=0x7fb56001a040 comp rx=0 tx=0).stop 2024-05-13T22:52:09.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.016+0000 7fb56bf63700 1 -- 172.21.15.102:0/4265570094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5640ffee0 msgr2=0x7fb56419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:09.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.016+0000 7fb56bf63700 1 --2- 172.21.15.102:0/4265570094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5640ffee0 0x7fb56419cb40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb55400b480 tx=0x7fb55400b840 comp rx=0 tx=0).stop 2024-05-13T22:52:09.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.016+0000 7fb56bf63700 1 -- 172.21.15.102:0/4265570094 shutdown_connections 2024-05-13T22:52:09.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.016+0000 7fb56bf63700 1 --2- 172.21.15.102:0/4265570094 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fb55006de80 0x7fb550070340 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:09.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.016+0000 7fb56bf63700 1 --2- 172.21.15.102:0/4265570094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5640ff530 0x7fb56419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.016+0000 7fb56bf63700 1 --2- 172.21.15.102:0/4265570094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5640ffee0 0x7fb56419cb40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.016+0000 7fb56bf63700 1 -- 172.21.15.102:0/4265570094 >> 172.21.15.102:0/4265570094 conn(0x7fb564074df0 msgr2=0x7fb5640feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.016+0000 7fb56bf63700 1 -- 172.21.15.102:0/4265570094 shutdown_connections 2024-05-13T22:52:09.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:10.016+0000 7fb56bf63700 1 -- 172.21.15.102:0/4265570094 wait complete. 2024-05-13T22:52:10.004 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:52:10.585 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:10 smithi102 ceph-mon[28599]: pgmap v103: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:10.608 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph orch ps' 2024-05-13T22:52:10.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:10 smithi130 ceph-mon[36591]: pgmap v103: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:10.816 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:52:12.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.515+0000 7f29842a5700 1 -- 172.21.15.102:0/3960642266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f297c105a40 msgr2=0x7f297c105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:12.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.515+0000 7f29842a5700 1 --2- 172.21.15.102:0/3960642266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f297c105a40 0x7f297c105e20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2978009910 tx=0x7f2978009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:12.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.515+0000 7f29842a5700 1 -- 172.21.15.102:0/3960642266 shutdown_connections 2024-05-13T22:52:12.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.515+0000 7f29842a5700 1 --2- 172.21.15.102:0/3960642266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f297c0ffbd0 0x7f297c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:12.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.515+0000 7f29842a5700 1 --2- 172.21.15.102:0/3960642266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f297c105a40 0x7f297c105e20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:12.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.515+0000 7f29842a5700 1 -- 172.21.15.102:0/3960642266 >> 172.21.15.102:0/3960642266 conn(0x7f297c0755a0 msgr2=0x7f297c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:12.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.515+0000 7f29842a5700 1 -- 172.21.15.102:0/3960642266 shutdown_connections 2024-05-13T22:52:12.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.515+0000 7f29842a5700 1 -- 172.21.15.102:0/3960642266 wait complete. 2024-05-13T22:52:12.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f29842a5700 1 Processor -- start 2024-05-13T22:52:12.494 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f29842a5700 1 -- start start 2024-05-13T22:52:12.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f29842a5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f297c0ffbd0 0x7f297c197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:12.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f29842a5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f297c105a40 0x7f297c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:12.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f29842a5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c1980f0 con 0x7f297c105a40 2024-05-13T22:52:12.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f29842a5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c198260 con 0x7f297c0ffbd0 2024-05-13T22:52:12.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f2981840700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f297c105a40 0x7f297c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:12.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f2981840700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f297c105a40 0x7f297c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33674/0 (socket says 172.21.15.102:33674) 2024-05-13T22:52:12.496 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f2981840700 1 -- 172.21.15.102:0/404882168 learned_addr learned my addr 172.21.15.102:0/404882168 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:12.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f2981840700 1 -- 172.21.15.102:0/404882168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f297c0ffbd0 msgr2=0x7f297c197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:12.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f2981840700 1 --2- 172.21.15.102:0/404882168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f297c0ffbd0 0x7f297c197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:12.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.516+0000 7f2981840700 1 -- 172.21.15.102:0/404882168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29780095f0 con 0x7f297c105a40 2024-05-13T22:52:12.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.517+0000 7f2981840700 1 --2- 172.21.15.102:0/404882168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f297c105a40 0x7f297c197ad0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f296c00d900 tx=0x7f296c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:12.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.517+0000 7f29737fe700 1 -- 172.21.15.102:0/404882168 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f296c0041f0 con 0x7f297c105a40 2024-05-13T22:52:12.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.517+0000 7f29737fe700 1 -- 172.21.15.102:0/404882168 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f296c004cf0 con 0x7f297c105a40 2024-05-13T22:52:12.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.517+0000 7f29737fe700 1 -- 172.21.15.102:0/404882168 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f296c00b750 con 0x7f297c105a40 2024-05-13T22:52:12.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.517+0000 7f29842a5700 1 -- 172.21.15.102:0/404882168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c1a0600 con 0x7f297c105a40 2024-05-13T22:52:12.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.517+0000 7f29842a5700 1 -- 172.21.15.102:0/404882168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c1a0af0 con 0x7f297c105a40 2024-05-13T22:52:12.497 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.518+0000 7f29842a5700 1 -- 172.21.15.102:0/404882168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c108040 con 0x7f297c105a40 2024-05-13T22:52:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.518+0000 7f29737fe700 1 -- 172.21.15.102:0/404882168 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f296c004350 con 0x7f297c105a40 2024-05-13T22:52:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.519+0000 7f29737fe700 1 --2- 172.21.15.102:0/404882168 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f296806de80 0x7f2968070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.519+0000 7f29737fe700 1 -- 172.21.15.102:0/404882168 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f296c08dce0 con 0x7f297c105a40 2024-05-13T22:52:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.519+0000 7f2982041700 1 --2- 172.21.15.102:0/404882168 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f296806de80 0x7f2968070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:12.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.519+0000 7f2982041700 1 --2- 172.21.15.102:0/404882168 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f296806de80 0x7f2968070340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f297800b5c0 tx=0x7f2978005960 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:12.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.524+0000 7f29737fe700 1 -- 172.21.15.102:0/404882168 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f296c0574f0 con 0x7f297c105a40 2024-05-13T22:52:12.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:12 smithi102 ceph-mon[28599]: pgmap v104: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:12.727 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.748+0000 7f29842a5700 1 -- 172.21.15.102:0/404882168 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f297c02cde0 con 0x7f296806de80 2024-05-13T22:52:12.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.760+0000 7f29737fe700 1 -- 172.21.15.102:0/404882168 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f297c02cde0 con 0x7f296806de80 2024-05-13T22:52:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:52:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (82s) 52s ago 12m 16.5M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:52:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (12m) 52s ago 12m 8044k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:52:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (11m) 116s ago 11m 7662k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:52:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (12m) 52s ago 12m 6660k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:52:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (11m) 116s ago 11m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:52:12.738 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (58s) 52s ago 12m 53.0M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (6m) 52s ago 6m 88.2M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (6m) 52s ago 6m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (6m) 116s ago 6m 14.3M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (6m) 116s ago 6m 17.5M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (3m) 52s ago 13m 508M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (2m) 116s ago 11m 450M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (13m) 52s ago 13m 55.3M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (11m) 116s ago 11m 46.6M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (2m) 52s ago 12m 16.0M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (2m) 116s ago 11m 3670k - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (10m) 52s ago 10m 224M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (10m) 52s ago 10m 238M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (9m) 52s ago 9m 202M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:52:12.739 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (9m) 116s ago 9m 288M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:52:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (8m) 116s ago 8m 244M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:52:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (8m) 116s ago 8m 224M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:52:12.740 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (99s) 52s ago 11m 54.0M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:52:12.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.762+0000 7f29842a5700 1 -- 172.21.15.102:0/404882168 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f296806de80 msgr2=0x7f2968070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:12.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.762+0000 7f29842a5700 1 --2- 172.21.15.102:0/404882168 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f296806de80 0x7f2968070340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f297800b5c0 tx=0x7f2978005960 comp rx=0 tx=0).stop 2024-05-13T22:52:12.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.762+0000 7f29842a5700 1 -- 172.21.15.102:0/404882168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f297c105a40 msgr2=0x7f297c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:12.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.762+0000 7f29842a5700 1 --2- 172.21.15.102:0/404882168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f297c105a40 0x7f297c197ad0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f296c00d900 tx=0x7f296c00dcc0 comp rx=0 tx=0).stop 2024-05-13T22:52:12.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.762+0000 7f29842a5700 1 -- 172.21.15.102:0/404882168 shutdown_connections 2024-05-13T22:52:12.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.762+0000 7f29842a5700 1 --2- 172.21.15.102:0/404882168 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f296806de80 0x7f2968070340 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:12.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.762+0000 7f29842a5700 1 --2- 172.21.15.102:0/404882168 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f297c0ffbd0 0x7f297c197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:12.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.762+0000 7f29842a5700 1 --2- 172.21.15.102:0/404882168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f297c105a40 0x7f297c197ad0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:12.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.762+0000 7f29842a5700 1 -- 172.21.15.102:0/404882168 >> 172.21.15.102:0/404882168 conn(0x7f297c0755a0 msgr2=0x7f297c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:12.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.762+0000 7f29842a5700 1 -- 172.21.15.102:0/404882168 shutdown_connections 2024-05-13T22:52:12.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:12.762+0000 7f29842a5700 1 -- 172.21.15.102:0/404882168 wait complete. 2024-05-13T22:52:12.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:12 smithi130 ceph-mon[36591]: pgmap v104: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:13.482 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:13 smithi102 ceph-mon[28599]: from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:13.519 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-13T22:52:13.520 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:52:13.520 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-05-13T22:52:13.733 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:52:13.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:13 smithi130 ceph-mon[36591]: from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:14.750 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:14 smithi102 ceph-mon[28599]: pgmap v105: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:14.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:14 smithi130 ceph-mon[36591]: pgmap v105: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:15.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.281+0000 7f1e12279700 1 -- 172.21.15.102:0/3789602167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e0c0ffa40 msgr2=0x7f1e0c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:15.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.281+0000 7f1e12279700 1 --2- 172.21.15.102:0/3789602167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e0c0ffa40 0x7f1e0c0ffec0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009910 tx=0x7f1df4009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.281+0000 7f1e12279700 1 -- 172.21.15.102:0/3789602167 shutdown_connections 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.281+0000 7f1e12279700 1 --2- 172.21.15.102:0/3789602167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e0c0ffa40 0x7f1e0c0ffec0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.281+0000 7f1e12279700 1 --2- 172.21.15.102:0/3789602167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e0c105a40 0x7f1e0c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.282+0000 7f1e12279700 1 -- 172.21.15.102:0/3789602167 >> 172.21.15.102:0/3789602167 conn(0x7f1e0c0fb570 msgr2=0x7f1e0c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.282+0000 7f1e12279700 1 -- 172.21.15.102:0/3789602167 shutdown_connections 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.282+0000 7f1e12279700 1 -- 172.21.15.102:0/3789602167 wait complete. 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.282+0000 7f1e12279700 1 Processor -- start 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.282+0000 7f1e12279700 1 -- start start 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.282+0000 7f1e12279700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e0c0ffa40 0x7f1e0c196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.282+0000 7f1e12279700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e0c105a40 0x7f1e0c196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.282+0000 7f1e12279700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c1972e0 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.282+0000 7f1e12279700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c19a010 con 0x7f1e0c105a40 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e0c0ffa40 0x7f1e0c196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e0bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e0c0ffa40 0x7f1e0c196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33686/0 (socket says 172.21.15.102:33686) 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e0bfff700 1 -- 172.21.15.102:0/686694945 learned_addr learned my addr 172.21.15.102:0/686694945 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e03fff700 1 --2- 172.21.15.102:0/686694945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e0c105a40 0x7f1e0c196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e0bfff700 1 -- 172.21.15.102:0/686694945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e0c105a40 msgr2=0x7f1e0c196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e0bfff700 1 --2- 172.21.15.102:0/686694945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e0c105a40 0x7f1e0c196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e0bfff700 1 -- 172.21.15.102:0/686694945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df40095f0 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e0bfff700 1 --2- 172.21.15.102:0/686694945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e0c0ffa40 0x7f1e0c196710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00d8d0 tx=0x7f1dfc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e09ffb700 1 -- 172.21.15.102:0/686694945 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dfc0098a0 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e09ffb700 1 -- 172.21.15.102:0/686694945 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1dfc010460 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e09ffb700 1 -- 172.21.15.102:0/686694945 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dfc00f5f0 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e12279700 1 -- 172.21.15.102:0/686694945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c19a2f0 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.263 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.283+0000 7f1e12279700 1 -- 172.21.15.102:0/686694945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c19a810 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.284+0000 7f1e12279700 1 -- 172.21.15.102:0/686694945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c108fe0 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.285+0000 7f1e09ffb700 1 -- 172.21.15.102:0/686694945 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f1dfc009a00 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.285+0000 7f1e09ffb700 1 --2- 172.21.15.102:0/686694945 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1df806de80 0x7f1df8070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:15.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.285+0000 7f1e09ffb700 1 -- 172.21.15.102:0/686694945 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1dfc08dab0 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.285+0000 7f1e03fff700 1 --2- 172.21.15.102:0/686694945 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1df806de80 0x7f1df8070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:15.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.286+0000 7f1e03fff700 1 --2- 172.21.15.102:0/686694945 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1df806de80 0x7f1df8070340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1df400b5c0 tx=0x7f1df401a040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:15.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.290+0000 7f1e09ffb700 1 -- 172.21.15.102:0/686694945 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f1dfc0572c0 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.472 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.494+0000 7f1e12279700 1 -- 172.21.15.102:0/686694945 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f1e0c04ea90 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.498+0000 7f1e09ffb700 1 -- 172.21.15.102:0/686694945 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1dfc05a980 con 0x7f1e0c0ffa40 2024-05-13T22:52:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.501+0000 7f1e12279700 1 -- 172.21.15.102:0/686694945 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1df806de80 msgr2=0x7f1df8070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.501+0000 7f1e12279700 1 --2- 172.21.15.102:0/686694945 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1df806de80 0x7f1df8070340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1df400b5c0 tx=0x7f1df401a040 comp rx=0 tx=0).stop 2024-05-13T22:52:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.501+0000 7f1e12279700 1 -- 172.21.15.102:0/686694945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e0c0ffa40 msgr2=0x7f1e0c196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.501+0000 7f1e12279700 1 --2- 172.21.15.102:0/686694945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e0c0ffa40 0x7f1e0c196710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00d8d0 tx=0x7f1dfc00dbe0 comp rx=0 tx=0).stop 2024-05-13T22:52:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.501+0000 7f1e12279700 1 -- 172.21.15.102:0/686694945 shutdown_connections 2024-05-13T22:52:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.501+0000 7f1e12279700 1 --2- 172.21.15.102:0/686694945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1e0c0ffa40 0x7f1e0c196710 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.501+0000 7f1e12279700 1 --2- 172.21.15.102:0/686694945 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f1df806de80 0x7f1df8070340 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.501+0000 7f1e12279700 1 --2- 172.21.15.102:0/686694945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1e0c105a40 0x7f1e0c196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.501+0000 7f1e12279700 1 -- 172.21.15.102:0/686694945 >> 172.21.15.102:0/686694945 conn(0x7f1e0c0fb570 msgr2=0x7f1e0c069070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:15.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.501+0000 7f1e12279700 1 -- 172.21.15.102:0/686694945 shutdown_connections 2024-05-13T22:52:15.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:15.501+0000 7f1e12279700 1 -- 172.21.15.102:0/686694945 wait complete. 2024-05-13T22:52:16.289 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-13T22:52:16.290 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:52:16.295 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-13T22:52:16.505 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:52:16.627 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:16 smithi102 ceph-mon[28599]: pgmap v106: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:16.627 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:16 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/686694945' entity='client.admin' 2024-05-13T22:52:16.627 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:16 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:52:16.627 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:16 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:52:16.627 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:16 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:52:16.627 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:16 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:16.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:16 smithi130 ceph-mon[36591]: pgmap v106: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:16.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:16 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/686694945' entity='client.admin' 2024-05-13T22:52:16.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:16 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:52:16.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:16 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:52:16.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:16 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:52:16.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:16 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:18.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.274+0000 7f9417943700 1 -- 172.21.15.102:0/2108801646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9410101770 msgr2=0x7f94101057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:18.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.274+0000 7f9417943700 1 --2- 172.21.15.102:0/2108801646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9410101770 0x7f94101057c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9400009910 tx=0x7f9400009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:18.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.274+0000 7f9417943700 1 -- 172.21.15.102:0/2108801646 shutdown_connections 2024-05-13T22:52:18.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.274+0000 7f9417943700 1 --2- 172.21.15.102:0/2108801646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9410101770 0x7f94101057c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:18.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.274+0000 7f9417943700 1 --2- 172.21.15.102:0/2108801646 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9410100dc0 0x7f94101011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:18.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.274+0000 7f9417943700 1 -- 172.21.15.102:0/2108801646 >> 172.21.15.102:0/2108801646 conn(0x7f94100fc650 msgr2=0x7f94100fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:18.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.274+0000 7f9417943700 1 -- 172.21.15.102:0/2108801646 shutdown_connections 2024-05-13T22:52:18.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.274+0000 7f9417943700 1 -- 172.21.15.102:0/2108801646 wait complete. 2024-05-13T22:52:18.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.274+0000 7f9417943700 1 Processor -- start 2024-05-13T22:52:18.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.274+0000 7f9417943700 1 -- start start 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f9417943700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9410100dc0 0x7f941019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f9417943700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9410101770 0x7f941019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f9417943700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941019d240 con 0x7f9410100dc0 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f9417943700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94101966f0 con 0x7f9410101770 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f94156df700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9410100dc0 0x7f941019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f94156df700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9410100dc0 0x7f941019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33698/0 (socket says 172.21.15.102:33698) 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f94156df700 1 -- 172.21.15.102:0/3998577744 learned_addr learned my addr 172.21.15.102:0/3998577744 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f9414ede700 1 --2- 172.21.15.102:0/3998577744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9410101770 0x7f941019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f94156df700 1 -- 172.21.15.102:0/3998577744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9410101770 msgr2=0x7f941019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f94156df700 1 --2- 172.21.15.102:0/3998577744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9410101770 0x7f941019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f94156df700 1 -- 172.21.15.102:0/3998577744 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94000095f0 con 0x7f9410100dc0 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f94156df700 1 --2- 172.21.15.102:0/3998577744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9410100dc0 0x7f941019c670 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f940c00b4b0 tx=0x7f940c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f94067fc700 1 -- 172.21.15.102:0/3998577744 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f940c011800 con 0x7f9410100dc0 2024-05-13T22:52:18.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f94067fc700 1 -- 172.21.15.102:0/3998577744 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f940c011e20 con 0x7f9410100dc0 2024-05-13T22:52:18.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f9417943700 1 -- 172.21.15.102:0/3998577744 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94101969d0 con 0x7f9410100dc0 2024-05-13T22:52:18.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f94067fc700 1 -- 172.21.15.102:0/3998577744 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f940c00f2e0 con 0x7f9410100dc0 2024-05-13T22:52:18.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.275+0000 7f9417943700 1 -- 172.21.15.102:0/3998577744 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9410196f20 con 0x7f9410100dc0 2024-05-13T22:52:18.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.276+0000 7f9417943700 1 -- 172.21.15.102:0/3998577744 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94101090f0 con 0x7f9410100dc0 2024-05-13T22:52:18.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.277+0000 7f94067fc700 1 -- 172.21.15.102:0/3998577744 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f940c010380 con 0x7f9410100dc0 2024-05-13T22:52:18.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.277+0000 7f94067fc700 1 --2- 172.21.15.102:0/3998577744 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93fc06de80 0x7f93fc070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:18.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.278+0000 7f9414ede700 1 --2- 172.21.15.102:0/3998577744 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93fc06de80 0x7f93fc070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:18.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.278+0000 7f94067fc700 1 -- 172.21.15.102:0/3998577744 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f940c08d710 con 0x7f9410100dc0 2024-05-13T22:52:18.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.278+0000 7f9414ede700 1 --2- 172.21.15.102:0/3998577744 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93fc06de80 0x7f93fc070340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9410197b60 tx=0x7f940001a040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:18.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.282+0000 7f94067fc700 1 -- 172.21.15.102:0/3998577744 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f940c05aa50 con 0x7f9410100dc0 2024-05-13T22:52:18.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.481+0000 7f9417943700 1 -- 172.21.15.102:0/3998577744 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f941004ea90 con 0x7f9410100dc0 2024-05-13T22:52:18.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.481+0000 7f94067fc700 1 -- 172.21.15.102:0/3998577744 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7f940c05a5e0 con 0x7f9410100dc0 2024-05-13T22:52:18.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.484+0000 7f9417943700 1 -- 172.21.15.102:0/3998577744 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93fc06de80 msgr2=0x7f93fc070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:18.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.484+0000 7f9417943700 1 --2- 172.21.15.102:0/3998577744 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93fc06de80 0x7f93fc070340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9410197b60 tx=0x7f940001a040 comp rx=0 tx=0).stop 2024-05-13T22:52:18.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.484+0000 7f9417943700 1 -- 172.21.15.102:0/3998577744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9410100dc0 msgr2=0x7f941019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:18.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.484+0000 7f9417943700 1 --2- 172.21.15.102:0/3998577744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9410100dc0 0x7f941019c670 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f940c00b4b0 tx=0x7f940c00b7c0 comp rx=0 tx=0).stop 2024-05-13T22:52:18.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.484+0000 7f9417943700 1 -- 172.21.15.102:0/3998577744 shutdown_connections 2024-05-13T22:52:18.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.484+0000 7f9417943700 1 --2- 172.21.15.102:0/3998577744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9410100dc0 0x7f941019c670 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:18.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.484+0000 7f9417943700 1 --2- 172.21.15.102:0/3998577744 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93fc06de80 0x7f93fc070340 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:18.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.484+0000 7f9417943700 1 --2- 172.21.15.102:0/3998577744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9410101770 0x7f941019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:18.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.484+0000 7f9417943700 1 -- 172.21.15.102:0/3998577744 >> 172.21.15.102:0/3998577744 conn(0x7f94100fc650 msgr2=0x7f94100fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:18.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.484+0000 7f9417943700 1 -- 172.21.15.102:0/3998577744 shutdown_connections 2024-05-13T22:52:18.463 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:18.484+0000 7f9417943700 1 -- 172.21.15.102:0/3998577744 wait complete. 2024-05-13T22:52:18.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:18 smithi130 ceph-mon[36591]: pgmap v107: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:18.911 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:18 smithi102 ceph-mon[28599]: pgmap v107: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:19.080 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-13T22:52:19.290 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:52:19.664 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:19 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:52:19.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:19 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:52:20.643 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:20 smithi102 ceph-mon[28599]: pgmap v108: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:20.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.863+0000 7fa6be925700 1 -- 172.21.15.102:0/2411660942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6b80730b0 msgr2=0x7fa6b8073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:20.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.863+0000 7fa6be925700 1 --2- 172.21.15.102:0/2411660942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6b80730b0 0x7fa6b8073490 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009910 tx=0x7fa6a8009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:20.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.863+0000 7fa6be925700 1 -- 172.21.15.102:0/2411660942 shutdown_connections 2024-05-13T22:52:20.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.863+0000 7fa6be925700 1 --2- 172.21.15.102:0/2411660942 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b80739d0 0x7fa6b810cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:20.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.863+0000 7fa6be925700 1 --2- 172.21.15.102:0/2411660942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6b80730b0 0x7fa6b8073490 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:20.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.863+0000 7fa6be925700 1 -- 172.21.15.102:0/2411660942 >> 172.21.15.102:0/2411660942 conn(0x7fa6b80fc650 msgr2=0x7fa6b80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:20.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.863+0000 7fa6be925700 1 -- 172.21.15.102:0/2411660942 shutdown_connections 2024-05-13T22:52:20.842 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.863+0000 7fa6be925700 1 -- 172.21.15.102:0/2411660942 wait complete. 2024-05-13T22:52:20.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6be925700 1 Processor -- start 2024-05-13T22:52:20.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6be925700 1 -- start start 2024-05-13T22:52:20.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6be925700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b80730b0 0x7fa6b819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:20.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6be925700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6b80739d0 0x7fa6b819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:20.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6be925700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b819f400 con 0x7fa6b80739d0 2024-05-13T22:52:20.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6be925700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b81988b0 con 0x7fa6b80730b0 2024-05-13T22:52:20.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6b77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6b80739d0 0x7fa6b819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:20.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6b77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6b80739d0 0x7fa6b819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58626/0 (socket says 172.21.15.102:58626) 2024-05-13T22:52:20.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6b77fe700 1 -- 172.21.15.102:0/1227191152 learned_addr learned my addr 172.21.15.102:0/1227191152 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:20.844 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6b7fff700 1 --2- 172.21.15.102:0/1227191152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b80730b0 0x7fa6b819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:20.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6b77fe700 1 -- 172.21.15.102:0/1227191152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b80730b0 msgr2=0x7fa6b819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:20.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6b77fe700 1 --2- 172.21.15.102:0/1227191152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b80730b0 0x7fa6b819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:20.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.864+0000 7fa6b77fe700 1 -- 172.21.15.102:0/1227191152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a80095f0 con 0x7fa6b80739d0 2024-05-13T22:52:20.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.865+0000 7fa6b77fe700 1 --2- 172.21.15.102:0/1227191152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6b80739d0 0x7fa6b819ed70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00b4b0 tx=0x7fa6ac00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:20.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.865+0000 7fa6b57fa700 1 -- 172.21.15.102:0/1227191152 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6ac011800 con 0x7fa6b80739d0 2024-05-13T22:52:20.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.865+0000 7fa6be925700 1 -- 172.21.15.102:0/1227191152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b8198b90 con 0x7fa6b80739d0 2024-05-13T22:52:20.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.865+0000 7fa6b57fa700 1 -- 172.21.15.102:0/1227191152 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6ac011e20 con 0x7fa6b80739d0 2024-05-13T22:52:20.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.865+0000 7fa6b57fa700 1 -- 172.21.15.102:0/1227191152 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6ac00f2e0 con 0x7fa6b80739d0 2024-05-13T22:52:20.845 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.865+0000 7fa6be925700 1 -- 172.21.15.102:0/1227191152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b81990e0 con 0x7fa6b80739d0 2024-05-13T22:52:20.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.866+0000 7fa6be925700 1 -- 172.21.15.102:0/1227191152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b810a5e0 con 0x7fa6b80739d0 2024-05-13T22:52:20.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.866+0000 7fa6b57fa700 1 -- 172.21.15.102:0/1227191152 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fa6ac010380 con 0x7fa6b80739d0 2024-05-13T22:52:20.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.867+0000 7fa6b57fa700 1 --2- 172.21.15.102:0/1227191152 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa6a006de80 0x7fa6a0070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:20.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.867+0000 7fa6b57fa700 1 -- 172.21.15.102:0/1227191152 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa6ac08d7c0 con 0x7fa6b80739d0 2024-05-13T22:52:20.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.867+0000 7fa6b7fff700 1 --2- 172.21.15.102:0/1227191152 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa6a006de80 0x7fa6a0070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:20.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.867+0000 7fa6b7fff700 1 --2- 172.21.15.102:0/1227191152 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa6a006de80 0x7fa6a0070340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800b5c0 tx=0x7fa6a801a040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:20.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:20.871+0000 7fa6b57fa700 1 -- 172.21.15.102:0/1227191152 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fa6ac05ab00 con 0x7fa6b80739d0 2024-05-13T22:52:20.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:20 smithi130 ceph-mon[36591]: pgmap v108: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.076+0000 7fa6be925700 1 -- 172.21.15.102:0/1227191152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fa6b81993b0 con 0x7fa6b80739d0 2024-05-13T22:52:21.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.077+0000 7fa6b57fa700 1 -- 172.21.15.102:0/1227191152 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7fa6ac05a690 con 0x7fa6b80739d0 2024-05-13T22:52:21.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.079+0000 7fa6be925700 1 -- 172.21.15.102:0/1227191152 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa6a006de80 msgr2=0x7fa6a0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:21.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.079+0000 7fa6be925700 1 --2- 172.21.15.102:0/1227191152 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa6a006de80 0x7fa6a0070340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800b5c0 tx=0x7fa6a801a040 comp rx=0 tx=0).stop 2024-05-13T22:52:21.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.079+0000 7fa6be925700 1 -- 172.21.15.102:0/1227191152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6b80739d0 msgr2=0x7fa6b819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:21.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.079+0000 7fa6be925700 1 --2- 172.21.15.102:0/1227191152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6b80739d0 0x7fa6b819ed70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00b4b0 tx=0x7fa6ac00b870 comp rx=0 tx=0).stop 2024-05-13T22:52:21.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.079+0000 7fa6be925700 1 -- 172.21.15.102:0/1227191152 shutdown_connections 2024-05-13T22:52:21.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.079+0000 7fa6be925700 1 --2- 172.21.15.102:0/1227191152 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fa6a006de80 0x7fa6a0070340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.079+0000 7fa6be925700 1 --2- 172.21.15.102:0/1227191152 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6b80730b0 0x7fa6b819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.079+0000 7fa6be925700 1 --2- 172.21.15.102:0/1227191152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6b80739d0 0x7fa6b819ed70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.079+0000 7fa6be925700 1 -- 172.21.15.102:0/1227191152 >> 172.21.15.102:0/1227191152 conn(0x7fa6b80fc650 msgr2=0x7fa6b8107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.079+0000 7fa6be925700 1 -- 172.21.15.102:0/1227191152 shutdown_connections 2024-05-13T22:52:21.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:21.079+0000 7fa6be925700 1 -- 172.21.15.102:0/1227191152 wait complete. 2024-05-13T22:52:21.829 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-13T22:52:22.038 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:52:22.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:22 smithi130 ceph-mon[36591]: pgmap v109: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:22.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:22 smithi102 ceph-mon[28599]: pgmap v109: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:23.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.772+0000 7f4244db1700 1 -- 172.21.15.102:0/3890112207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42400ffbd0 msgr2=0x7f4240100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:23.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.772+0000 7f4244db1700 1 --2- 172.21.15.102:0/3890112207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42400ffbd0 0x7f4240100050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4234009910 tx=0x7f4234009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:23.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.773+0000 7f4244db1700 1 -- 172.21.15.102:0/3890112207 shutdown_connections 2024-05-13T22:52:23.750 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.773+0000 7f4244db1700 1 --2- 172.21.15.102:0/3890112207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42400ffbd0 0x7f4240100050 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:23.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.773+0000 7f4244db1700 1 --2- 172.21.15.102:0/3890112207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4240105a40 0x7f4240105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:23.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.773+0000 7f4244db1700 1 -- 172.21.15.102:0/3890112207 >> 172.21.15.102:0/3890112207 conn(0x7f42400755a0 msgr2=0x7f42400759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:23.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.773+0000 7f4244db1700 1 -- 172.21.15.102:0/3890112207 shutdown_connections 2024-05-13T22:52:23.751 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.773+0000 7f4244db1700 1 -- 172.21.15.102:0/3890112207 wait complete. 2024-05-13T22:52:23.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f4244db1700 1 Processor -- start 2024-05-13T22:52:23.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f4244db1700 1 -- start start 2024-05-13T22:52:23.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f4244db1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f42400ffbd0 0x7f424019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:23.752 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f4244db1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4240105a40 0x7f424019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f4244db1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424019d0d0 con 0x7f4240105a40 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f4244db1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4240196580 con 0x7f42400ffbd0 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f423dd9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4240105a40 0x7f424019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f423dd9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4240105a40 0x7f424019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58648/0 (socket says 172.21.15.102:58648) 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f423dd9b700 1 -- 172.21.15.102:0/2648724048 learned_addr learned my addr 172.21.15.102:0/2648724048 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f423e59c700 1 --2- 172.21.15.102:0/2648724048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f42400ffbd0 0x7f424019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f423dd9b700 1 -- 172.21.15.102:0/2648724048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f42400ffbd0 msgr2=0x7f424019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f423dd9b700 1 --2- 172.21.15.102:0/2648724048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f42400ffbd0 0x7f424019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f423dd9b700 1 -- 172.21.15.102:0/2648724048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42340095f0 con 0x7f4240105a40 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.774+0000 7f423dd9b700 1 --2- 172.21.15.102:0/2648724048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4240105a40 0x7f424019ca40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4234006010 tx=0x7f423400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.775+0000 7f422f7fe700 1 -- 172.21.15.102:0/2648724048 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f423401d070 con 0x7f4240105a40 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.775+0000 7f4244db1700 1 -- 172.21.15.102:0/2648724048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4240196800 con 0x7f4240105a40 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.775+0000 7f422f7fe700 1 -- 172.21.15.102:0/2648724048 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4234005dc0 con 0x7f4240105a40 2024-05-13T22:52:23.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.775+0000 7f422f7fe700 1 -- 172.21.15.102:0/2648724048 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4234017400 con 0x7f4240105a40 2024-05-13T22:52:23.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.775+0000 7f4244db1700 1 -- 172.21.15.102:0/2648724048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4240196cf0 con 0x7f4240105a40 2024-05-13T22:52:23.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.776+0000 7f4244db1700 1 -- 172.21.15.102:0/2648724048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4240108fe0 con 0x7f4240105a40 2024-05-13T22:52:23.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.776+0000 7f422f7fe700 1 -- 172.21.15.102:0/2648724048 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f4234003a60 con 0x7f4240105a40 2024-05-13T22:52:23.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.777+0000 7f422f7fe700 1 --2- 172.21.15.102:0/2648724048 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f422806de80 0x7f4228070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:23.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.777+0000 7f422f7fe700 1 -- 172.21.15.102:0/2648724048 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f423408f030 con 0x7f4240105a40 2024-05-13T22:52:23.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.777+0000 7f423e59c700 1 --2- 172.21.15.102:0/2648724048 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f422806de80 0x7f4228070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:23.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.777+0000 7f423e59c700 1 --2- 172.21.15.102:0/2648724048 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f422806de80 0x7f4228070340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4230005950 tx=0x7f423000b410 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:23.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.781+0000 7f422f7fe700 1 -- 172.21.15.102:0/2648724048 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f423405c370 con 0x7f4240105a40 2024-05-13T22:52:23.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.979+0000 7f4244db1700 1 -- 172.21.15.102:0/2648724048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f424004ea90 con 0x7f4240105a40 2024-05-13T22:52:23.956 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.979+0000 7f422f7fe700 1 -- 172.21.15.102:0/2648724048 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7f423405bf00 con 0x7f4240105a40 2024-05-13T22:52:23.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.981+0000 7f4244db1700 1 -- 172.21.15.102:0/2648724048 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f422806de80 msgr2=0x7f4228070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:23.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.981+0000 7f4244db1700 1 --2- 172.21.15.102:0/2648724048 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f422806de80 0x7f4228070340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4230005950 tx=0x7f423000b410 comp rx=0 tx=0).stop 2024-05-13T22:52:23.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.981+0000 7f4244db1700 1 -- 172.21.15.102:0/2648724048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4240105a40 msgr2=0x7f424019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:23.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.981+0000 7f4244db1700 1 --2- 172.21.15.102:0/2648724048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4240105a40 0x7f424019ca40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4234006010 tx=0x7f423400bd70 comp rx=0 tx=0).stop 2024-05-13T22:52:23.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.981+0000 7f4244db1700 1 -- 172.21.15.102:0/2648724048 shutdown_connections 2024-05-13T22:52:23.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.981+0000 7f4244db1700 1 --2- 172.21.15.102:0/2648724048 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f422806de80 0x7f4228070340 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:23.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.981+0000 7f4244db1700 1 --2- 172.21.15.102:0/2648724048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f42400ffbd0 0x7f424019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:23.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.981+0000 7f4244db1700 1 --2- 172.21.15.102:0/2648724048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4240105a40 0x7f424019ca40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:23.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.981+0000 7f4244db1700 1 -- 172.21.15.102:0/2648724048 >> 172.21.15.102:0/2648724048 conn(0x7f42400755a0 msgr2=0x7f4240068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:23.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.981+0000 7f4244db1700 1 -- 172.21.15.102:0/2648724048 shutdown_connections 2024-05-13T22:52:23.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:23.981+0000 7f4244db1700 1 -- 172.21.15.102:0/2648724048 wait complete. 2024-05-13T22:52:24.563 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:24 smithi102 ceph-mon[28599]: pgmap v110: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:24.586 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-05-13T22:52:24.798 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:52:24.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:24 smithi130 ceph-mon[36591]: pgmap v110: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:26.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.388+0000 7f4646dd5700 1 -- 172.21.15.102:0/3216191308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640101770 msgr2=0x7f46401057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:26.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.388+0000 7f4646dd5700 1 --2- 172.21.15.102:0/3216191308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640101770 0x7f46401057c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f462c009910 tx=0x7f462c009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:26.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.388+0000 7f4646dd5700 1 -- 172.21.15.102:0/3216191308 shutdown_connections 2024-05-13T22:52:26.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.388+0000 7f4646dd5700 1 --2- 172.21.15.102:0/3216191308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640101770 0x7f46401057c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:26.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.388+0000 7f4646dd5700 1 --2- 172.21.15.102:0/3216191308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4640100dc0 0x7f46401011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:26.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.388+0000 7f4646dd5700 1 -- 172.21.15.102:0/3216191308 >> 172.21.15.102:0/3216191308 conn(0x7f46400fc650 msgr2=0x7f46400fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:26.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.389+0000 7f4646dd5700 1 -- 172.21.15.102:0/3216191308 shutdown_connections 2024-05-13T22:52:26.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.389+0000 7f4646dd5700 1 -- 172.21.15.102:0/3216191308 wait complete. 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.389+0000 7f4646dd5700 1 Processor -- start 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.389+0000 7f4646dd5700 1 -- start start 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.389+0000 7f4646dd5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4640100dc0 0x7f464019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.389+0000 7f4646dd5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640101770 0x7f464019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.389+0000 7f4646dd5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464019d2b0 con 0x7f4640101770 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.389+0000 7f4646dd5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4640196760 con 0x7f4640100dc0 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.389+0000 7f463ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640101770 0x7f464019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.389+0000 7f463ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640101770 0x7f464019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58670/0 (socket says 172.21.15.102:58670) 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.389+0000 7f463ffff700 1 -- 172.21.15.102:0/548444530 learned_addr learned my addr 172.21.15.102:0/548444530 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.390+0000 7f4644b71700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4640100dc0 0x7f464019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.390+0000 7f463ffff700 1 -- 172.21.15.102:0/548444530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4640100dc0 msgr2=0x7f464019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:26.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.390+0000 7f463ffff700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4640100dc0 0x7f464019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:26.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.390+0000 7f463ffff700 1 -- 172.21.15.102:0/548444530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c0095f0 con 0x7f4640101770 2024-05-13T22:52:26.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.390+0000 7f4644b71700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4640100dc0 0x7f464019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:52:26.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.390+0000 7f463ffff700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640101770 0x7f464019cc20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f462c0092e0 tx=0x7f462c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:26.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.390+0000 7f463dffb700 1 -- 172.21.15.102:0/548444530 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f462c01d070 con 0x7f4640101770 2024-05-13T22:52:26.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.390+0000 7f463dffb700 1 -- 172.21.15.102:0/548444530 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f462c00f460 con 0x7f4640101770 2024-05-13T22:52:26.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.390+0000 7f4646dd5700 1 -- 172.21.15.102:0/548444530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46401969e0 con 0x7f4640101770 2024-05-13T22:52:26.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.390+0000 7f463dffb700 1 -- 172.21.15.102:0/548444530 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f462c0175f0 con 0x7f4640101770 2024-05-13T22:52:26.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.390+0000 7f4646dd5700 1 -- 172.21.15.102:0/548444530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4640196ed0 con 0x7f4640101770 2024-05-13T22:52:26.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.391+0000 7f4646dd5700 1 -- 172.21.15.102:0/548444530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4640109110 con 0x7f4640101770 2024-05-13T22:52:26.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.392+0000 7f463dffb700 1 -- 172.21.15.102:0/548444530 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f462c00f5d0 con 0x7f4640101770 2024-05-13T22:52:26.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.392+0000 7f463dffb700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f463006de80 0x7f4630070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.392+0000 7f463dffb700 1 -- 172.21.15.102:0/548444530 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f462c08f160 con 0x7f4640101770 2024-05-13T22:52:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.392+0000 7f4644b71700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f463006de80 0x7f4630070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:26.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.393+0000 7f4644b71700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f463006de80 0x7f4630070340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4634007900 tx=0x7f4634008040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:26.374 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.397+0000 7f463dffb700 1 -- 172.21.15.102:0/548444530 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f462c05c4a0 con 0x7f4640101770 2024-05-13T22:52:26.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:26 smithi102 ceph-mon[28599]: pgmap v111: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:26.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.617+0000 7f4646dd5700 1 -- 172.21.15.102:0/548444530 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d", "target": ["mon-mgr", ""]}) v1 -- 0x7f46401971b0 con 0x7f463006de80 2024-05-13T22:52:26.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.622+0000 7f463dffb700 1 -- 172.21.15.102:0/548444530 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f46401971b0 con 0x7f463006de80 2024-05-13T22:52:26.599 INFO:teuthology.orchestra.run.smithi102.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:52:26.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.624+0000 7f4646dd5700 1 -- 172.21.15.102:0/548444530 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f463006de80 msgr2=0x7f4630070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:26.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.624+0000 7f4646dd5700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f463006de80 0x7f4630070340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4634007900 tx=0x7f4634008040 comp rx=0 tx=0).stop 2024-05-13T22:52:26.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.624+0000 7f4646dd5700 1 -- 172.21.15.102:0/548444530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640101770 msgr2=0x7f464019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:26.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.624+0000 7f4646dd5700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640101770 0x7f464019cc20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f462c0092e0 tx=0x7f462c00be20 comp rx=0 tx=0).stop 2024-05-13T22:52:26.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.624+0000 7f4646dd5700 1 -- 172.21.15.102:0/548444530 shutdown_connections 2024-05-13T22:52:26.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.624+0000 7f4646dd5700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f463006de80 0x7f4630070340 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:26.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.624+0000 7f4646dd5700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4640100dc0 0x7f464019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:26.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.624+0000 7f4646dd5700 1 --2- 172.21.15.102:0/548444530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640101770 0x7f464019cc20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:26.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.624+0000 7f4646dd5700 1 -- 172.21.15.102:0/548444530 >> 172.21.15.102:0/548444530 conn(0x7f46400fc650 msgr2=0x7f46400fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:26.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.625+0000 7f4646dd5700 1 -- 172.21.15.102:0/548444530 shutdown_connections 2024-05-13T22:52:26.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:26.625+0000 7f4646dd5700 1 -- 172.21.15.102:0/548444530 wait complete. 2024-05-13T22:52:26.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:26 smithi130 ceph-mon[36591]: pgmap v111: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:27.512 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-13T22:52:27.512 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T22:52:27.513 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-05-13T22:52:27.734 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:52:27.749 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:27 smithi102 ceph-mon[28599]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:27.749 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:27 smithi102 ceph-mon[28599]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:52:27.749 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:27 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:27.749 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:27 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:52:27.750 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:27 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:52:27.750 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:27 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:52:27.750 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:27 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:27.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:27 smithi130 ceph-mon[36591]: from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:27.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:27 smithi130 ceph-mon[36591]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:52:27.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:27 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:27.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:27 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:52:27.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:27 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:52:27.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:27 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:52:27.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:27 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:28.878 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:28 smithi102 ceph-mon[28599]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:52:28.878 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:28 smithi102 ceph-mon[28599]: pgmap v112: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:28.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:28 smithi130 ceph-mon[36591]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c3e55145f4f4b3a500bb66ee08c938a215bc231d 2024-05-13T22:52:28.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:28 smithi130 ceph-mon[36591]: pgmap v112: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:29.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.461+0000 7fd5be169700 1 -- 172.21.15.102:0/3054703608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b81026c0 msgr2=0x7fd5b810abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:29.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.461+0000 7fd5be169700 1 --2- 172.21.15.102:0/3054703608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b81026c0 0x7fd5b810abb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009910 tx=0x7fd5ac009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:29.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.461+0000 7fd5be169700 1 -- 172.21.15.102:0/3054703608 shutdown_connections 2024-05-13T22:52:29.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.461+0000 7fd5be169700 1 --2- 172.21.15.102:0/3054703608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b81026c0 0x7fd5b810abb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:29.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.461+0000 7fd5be169700 1 --2- 172.21.15.102:0/3054703608 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5b8101da0 0x7fd5b8102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:29.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.461+0000 7fd5be169700 1 -- 172.21.15.102:0/3054703608 >> 172.21.15.102:0/3054703608 conn(0x7fd5b80fb550 msgr2=0x7fd5b80fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:29.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.461+0000 7fd5be169700 1 -- 172.21.15.102:0/3054703608 shutdown_connections 2024-05-13T22:52:29.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.461+0000 7fd5be169700 1 -- 172.21.15.102:0/3054703608 wait complete. 2024-05-13T22:52:29.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.462+0000 7fd5be169700 1 Processor -- start 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.462+0000 7fd5be169700 1 -- start start 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.462+0000 7fd5be169700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b8101da0 0x7fd5b819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.462+0000 7fd5be169700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5b81026c0 0x7fd5b819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.462+0000 7fd5be169700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b819d2b0 con 0x7fd5b8101da0 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.462+0000 7fd5be169700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b8196760 con 0x7fd5b81026c0 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5b77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b8101da0 0x7fd5b819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5b77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b8101da0 0x7fd5b819c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41518/0 (socket says 172.21.15.102:41518) 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5b77fe700 1 -- 172.21.15.102:0/3755614303 learned_addr learned my addr 172.21.15.102:0/3755614303 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5b6ffd700 1 --2- 172.21.15.102:0/3755614303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5b81026c0 0x7fd5b819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5b77fe700 1 -- 172.21.15.102:0/3755614303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5b81026c0 msgr2=0x7fd5b819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5b77fe700 1 --2- 172.21.15.102:0/3755614303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5b81026c0 0x7fd5b819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:29.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5b77fe700 1 -- 172.21.15.102:0/3755614303 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac0095f0 con 0x7fd5b8101da0 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5b77fe700 1 --2- 172.21.15.102:0/3755614303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b8101da0 0x7fd5b819c6e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800b4b0 tx=0x7fd5a800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5b4ff9700 1 -- 172.21.15.102:0/3755614303 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a8011800 con 0x7fd5b8101da0 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5b4ff9700 1 -- 172.21.15.102:0/3755614303 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5a8011e20 con 0x7fd5b8101da0 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5be169700 1 -- 172.21.15.102:0/3755614303 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b8196a40 con 0x7fd5b8101da0 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5b4ff9700 1 -- 172.21.15.102:0/3755614303 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a800f2e0 con 0x7fd5b8101da0 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.463+0000 7fd5be169700 1 -- 172.21.15.102:0/3755614303 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b8196f90 con 0x7fd5b8101da0 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.464+0000 7fd5be169700 1 -- 172.21.15.102:0/3755614303 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b81082b0 con 0x7fd5b8101da0 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.465+0000 7fd5b4ff9700 1 -- 172.21.15.102:0/3755614303 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fd5a8011960 con 0x7fd5b8101da0 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.465+0000 7fd5b4ff9700 1 --2- 172.21.15.102:0/3755614303 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fd5a006de80 0x7fd5a0070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.465+0000 7fd5b4ff9700 1 -- 172.21.15.102:0/3755614303 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd5a808d7a0 con 0x7fd5b8101da0 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.465+0000 7fd5b6ffd700 1 --2- 172.21.15.102:0/3755614303 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fd5a006de80 0x7fd5a0070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:29.489 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.465+0000 7fd5b6ffd700 1 --2- 172.21.15.102:0/3755614303 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fd5a006de80 0x7fd5a0070340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8197bd0 tx=0x7fd5ac01a040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:29.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.470+0000 7fd5b4ff9700 1 -- 172.21.15.102:0/3755614303 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fd5a805aae0 con 0x7fd5b8101da0 2024-05-13T22:52:29.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.698+0000 7fd5be169700 1 -- 172.21.15.102:0/3755614303 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5b8197260 con 0x7fd5a006de80 2024-05-13T22:52:29.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.699+0000 7fd5b4ff9700 1 -- 172.21.15.102:0/3755614303 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fd5b8197260 con 0x7fd5a006de80 2024-05-13T22:52:29.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.701+0000 7fd5be169700 1 -- 172.21.15.102:0/3755614303 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fd5a006de80 msgr2=0x7fd5a0070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:29.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.701+0000 7fd5be169700 1 --2- 172.21.15.102:0/3755614303 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fd5a006de80 0x7fd5a0070340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8197bd0 tx=0x7fd5ac01a040 comp rx=0 tx=0).stop 2024-05-13T22:52:29.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.701+0000 7fd5be169700 1 -- 172.21.15.102:0/3755614303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b8101da0 msgr2=0x7fd5b819c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:29.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.701+0000 7fd5be169700 1 --2- 172.21.15.102:0/3755614303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b8101da0 0x7fd5b819c6e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800b4b0 tx=0x7fd5a800b870 comp rx=0 tx=0).stop 2024-05-13T22:52:29.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.701+0000 7fd5be169700 1 -- 172.21.15.102:0/3755614303 shutdown_connections 2024-05-13T22:52:29.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.701+0000 7fd5be169700 1 --2- 172.21.15.102:0/3755614303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5b8101da0 0x7fd5b819c6e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:29.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.701+0000 7fd5be169700 1 --2- 172.21.15.102:0/3755614303 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fd5a006de80 0x7fd5a0070340 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:29.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.701+0000 7fd5be169700 1 --2- 172.21.15.102:0/3755614303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5b81026c0 0x7fd5b819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:29.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.701+0000 7fd5be169700 1 -- 172.21.15.102:0/3755614303 >> 172.21.15.102:0/3755614303 conn(0x7fd5b80fb550 msgr2=0x7fd5b81053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:29.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.702+0000 7fd5be169700 1 -- 172.21.15.102:0/3755614303 shutdown_connections 2024-05-13T22:52:29.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.702+0000 7fd5be169700 1 -- 172.21.15.102:0/3755614303 wait complete. 2024-05-13T22:52:29.689 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:52:29.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.809+0000 7ffb58e25700 1 -- 172.21.15.102:0/2910015268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb540ff510 msgr2=0x7ffb540ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:29.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.809+0000 7ffb58e25700 1 --2- 172.21.15.102:0/2910015268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb540ff510 0x7ffb540ff8f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffb44009910 tx=0x7ffb44009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:29.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.809+0000 7ffb58e25700 1 -- 172.21.15.102:0/2910015268 shutdown_connections 2024-05-13T22:52:29.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.809+0000 7ffb58e25700 1 --2- 172.21.15.102:0/2910015268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb540ffec0 0x7ffb541057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:29.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.809+0000 7ffb58e25700 1 --2- 172.21.15.102:0/2910015268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb540ff510 0x7ffb540ff8f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:29.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.809+0000 7ffb58e25700 1 -- 172.21.15.102:0/2910015268 >> 172.21.15.102:0/2910015268 conn(0x7ffb54074e00 msgr2=0x7ffb54075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:29.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.809+0000 7ffb58e25700 1 -- 172.21.15.102:0/2910015268 shutdown_connections 2024-05-13T22:52:29.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.809+0000 7ffb58e25700 1 -- 172.21.15.102:0/2910015268 wait complete. 2024-05-13T22:52:29.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb58e25700 1 Processor -- start 2024-05-13T22:52:29.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb58e25700 1 -- start start 2024-05-13T22:52:29.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb58e25700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb540ff510 0x7ffb54100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:29.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb58e25700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb540ffec0 0x7ffb541009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:29.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb58e25700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb54100f10 con 0x7ffb540ffec0 2024-05-13T22:52:29.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb58e25700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb54101050 con 0x7ffb540ff510 2024-05-13T22:52:29.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb51d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb540ffec0 0x7ffb541009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:29.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb51d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb540ffec0 0x7ffb541009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41540/0 (socket says 172.21.15.102:41540) 2024-05-13T22:52:29.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb51d9b700 1 -- 172.21.15.102:0/1927270681 learned_addr learned my addr 172.21.15.102:0/1927270681 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:29.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb51d9b700 1 -- 172.21.15.102:0/1927270681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb540ff510 msgr2=0x7ffb54100490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:29.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb5259c700 1 --2- 172.21.15.102:0/1927270681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb540ff510 0x7ffb54100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:29.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb51d9b700 1 --2- 172.21.15.102:0/1927270681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb540ff510 0x7ffb54100490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:29.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb51d9b700 1 -- 172.21.15.102:0/1927270681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb440095f0 con 0x7ffb540ffec0 2024-05-13T22:52:29.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.810+0000 7ffb51d9b700 1 --2- 172.21.15.102:0/1927270681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb540ffec0 0x7ffb541009d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c8f0 tx=0x7ffb4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:29.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.811+0000 7ffb437fe700 1 -- 172.21.15.102:0/1927270681 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb48007a50 con 0x7ffb540ffec0 2024-05-13T22:52:29.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.811+0000 7ffb437fe700 1 -- 172.21.15.102:0/1927270681 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb48007bb0 con 0x7ffb540ffec0 2024-05-13T22:52:29.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.811+0000 7ffb58e25700 1 -- 172.21.15.102:0/1927270681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb541a29b0 con 0x7ffb540ffec0 2024-05-13T22:52:29.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.811+0000 7ffb437fe700 1 -- 172.21.15.102:0/1927270681 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb48018630 con 0x7ffb540ffec0 2024-05-13T22:52:29.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.811+0000 7ffb58e25700 1 -- 172.21.15.102:0/1927270681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb541a2dc0 con 0x7ffb540ffec0 2024-05-13T22:52:29.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.812+0000 7ffb58e25700 1 -- 172.21.15.102:0/1927270681 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb541091e0 con 0x7ffb540ffec0 2024-05-13T22:52:29.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.812+0000 7ffb437fe700 1 -- 172.21.15.102:0/1927270681 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7ffb48007d20 con 0x7ffb540ffec0 2024-05-13T22:52:29.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.813+0000 7ffb437fe700 1 --2- 172.21.15.102:0/1927270681 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ffb3c06de80 0x7ffb3c070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:29.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.813+0000 7ffb5259c700 1 --2- 172.21.15.102:0/1927270681 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ffb3c06de80 0x7ffb3c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:29.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.813+0000 7ffb437fe700 1 -- 172.21.15.102:0/1927270681 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ffb4808dba0 con 0x7ffb540ffec0 2024-05-13T22:52:29.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.813+0000 7ffb5259c700 1 --2- 172.21.15.102:0/1927270681 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ffb3c06de80 0x7ffb3c070340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffb44006010 tx=0x7ffb4400b540 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:29.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:29.818+0000 7ffb437fe700 1 -- 172.21.15.102:0/1927270681 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7ffb480573b0 con 0x7ffb540ffec0 2024-05-13T22:52:30.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.052+0000 7ffb58e25700 1 -- 172.21.15.102:0/1927270681 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb5402cd30 con 0x7ffb3c06de80 2024-05-13T22:52:30.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.053+0000 7ffb437fe700 1 -- 172.21.15.102:0/1927270681 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7ffb5402cd30 con 0x7ffb3c06de80 2024-05-13T22:52:30.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.055+0000 7ffb58e25700 1 -- 172.21.15.102:0/1927270681 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ffb3c06de80 msgr2=0x7ffb3c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.055+0000 7ffb58e25700 1 --2- 172.21.15.102:0/1927270681 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ffb3c06de80 0x7ffb3c070340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffb44006010 tx=0x7ffb4400b540 comp rx=0 tx=0).stop 2024-05-13T22:52:30.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.055+0000 7ffb58e25700 1 -- 172.21.15.102:0/1927270681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb540ffec0 msgr2=0x7ffb541009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.055+0000 7ffb58e25700 1 --2- 172.21.15.102:0/1927270681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb540ffec0 0x7ffb541009d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c8f0 tx=0x7ffb4800ccb0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.055+0000 7ffb58e25700 1 -- 172.21.15.102:0/1927270681 shutdown_connections 2024-05-13T22:52:30.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.055+0000 7ffb58e25700 1 --2- 172.21.15.102:0/1927270681 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7ffb3c06de80 0x7ffb3c070340 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.055+0000 7ffb58e25700 1 --2- 172.21.15.102:0/1927270681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb540ff510 0x7ffb54100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.055+0000 7ffb58e25700 1 --2- 172.21.15.102:0/1927270681 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb540ffec0 0x7ffb541009d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.055+0000 7ffb58e25700 1 -- 172.21.15.102:0/1927270681 >> 172.21.15.102:0/1927270681 conn(0x7ffb54074e00 msgr2=0x7ffb540fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:30.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.056+0000 7ffb58e25700 1 -- 172.21.15.102:0/1927270681 shutdown_connections 2024-05-13T22:52:30.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.056+0000 7ffb58e25700 1 -- 172.21.15.102:0/1927270681 wait complete. 2024-05-13T22:52:30.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.171+0000 7f2bbfdca700 1 -- 172.21.15.102:0/327149547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bb8101770 msgr2=0x7f2bb81057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.171+0000 7f2bbfdca700 1 --2- 172.21.15.102:0/327149547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bb8101770 0x7f2bb81057c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009910 tx=0x7f2ba8009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:30.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.171+0000 7f2bbfdca700 1 -- 172.21.15.102:0/327149547 shutdown_connections 2024-05-13T22:52:30.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.171+0000 7f2bbfdca700 1 --2- 172.21.15.102:0/327149547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bb8101770 0x7f2bb81057c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.171+0000 7f2bbfdca700 1 --2- 172.21.15.102:0/327149547 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bb8100dc0 0x7f2bb81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.171+0000 7f2bbfdca700 1 -- 172.21.15.102:0/327149547 >> 172.21.15.102:0/327149547 conn(0x7f2bb80fc650 msgr2=0x7f2bb80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:30.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.171+0000 7f2bbfdca700 1 -- 172.21.15.102:0/327149547 shutdown_connections 2024-05-13T22:52:30.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.171+0000 7f2bbfdca700 1 -- 172.21.15.102:0/327149547 wait complete. 2024-05-13T22:52:30.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.172+0000 7f2bbfdca700 1 Processor -- start 2024-05-13T22:52:30.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.172+0000 7f2bbfdca700 1 -- start start 2024-05-13T22:52:30.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.172+0000 7f2bbfdca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bb8100dc0 0x7f2bb8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:30.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.172+0000 7f2bbfdca700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bb8101770 0x7f2bb80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:30.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.172+0000 7f2bbfdca700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8073b10 con 0x7f2bb8100dc0 2024-05-13T22:52:30.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.172+0000 7f2bbfdca700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8073c50 con 0x7f2bb8101770 2024-05-13T22:52:30.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.172+0000 7f2bbdb66700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bb8100dc0 0x7f2bb8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.172+0000 7f2bbdb66700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bb8100dc0 0x7f2bb8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41558/0 (socket says 172.21.15.102:41558) 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.172+0000 7f2bbdb66700 1 -- 172.21.15.102:0/3135232598 learned_addr learned my addr 172.21.15.102:0/3135232598 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.172+0000 7f2bbd365700 1 --2- 172.21.15.102:0/3135232598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bb8101770 0x7f2bb80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.173+0000 7f2bbdb66700 1 -- 172.21.15.102:0/3135232598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bb8101770 msgr2=0x7f2bb80735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.173+0000 7f2bbdb66700 1 --2- 172.21.15.102:0/3135232598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bb8101770 0x7f2bb80735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.173+0000 7f2bbdb66700 1 -- 172.21.15.102:0/3135232598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba80095f0 con 0x7f2bb8100dc0 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.173+0000 7f2bbdb66700 1 --2- 172.21.15.102:0/3135232598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bb8100dc0 0x7f2bb8073090 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400e920 tx=0x7f2bb400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.173+0000 7f2baeffd700 1 -- 172.21.15.102:0/3135232598 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb400c9d0 con 0x7f2bb8100dc0 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.173+0000 7f2bbfdca700 1 -- 172.21.15.102:0/3135232598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb81a2930 con 0x7f2bb8100dc0 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.173+0000 7f2bbfdca700 1 -- 172.21.15.102:0/3135232598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb81a2ce0 con 0x7f2bb8100dc0 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.173+0000 7f2baeffd700 1 -- 172.21.15.102:0/3135232598 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bb4010430 con 0x7f2bb8100dc0 2024-05-13T22:52:30.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.173+0000 7f2baeffd700 1 -- 172.21.15.102:0/3135232598 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb4018630 con 0x7f2bb8100dc0 2024-05-13T22:52:30.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.174+0000 7f2bbfdca700 1 -- 172.21.15.102:0/3135232598 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb81091c0 con 0x7f2bb8100dc0 2024-05-13T22:52:30.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.175+0000 7f2baeffd700 1 -- 172.21.15.102:0/3135232598 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f2bb40105a0 con 0x7f2bb8100dc0 2024-05-13T22:52:30.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.176+0000 7f2baeffd700 1 --2- 172.21.15.102:0/3135232598 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f2ba406de80 0x7f2ba4070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:30.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.176+0000 7f2baeffd700 1 -- 172.21.15.102:0/3135232598 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2bb4014070 con 0x7f2bb8100dc0 2024-05-13T22:52:30.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.176+0000 7f2bbd365700 1 --2- 172.21.15.102:0/3135232598 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f2ba406de80 0x7f2ba4070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:30.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.176+0000 7f2bbd365700 1 --2- 172.21.15.102:0/3135232598 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f2ba406de80 0x7f2ba4070340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80059d0 tx=0x7f2ba8005940 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:30.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.184+0000 7f2baeffd700 1 -- 172.21.15.102:0/3135232598 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f2bb4053590 con 0x7f2bb8100dc0 2024-05-13T22:52:30.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.405+0000 7f2bbfdca700 1 -- 172.21.15.102:0/3135232598 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bb802cce0 con 0x7f2ba406de80 2024-05-13T22:52:30.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.416+0000 7f2baeffd700 1 -- 172.21.15.102:0/3135232598 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f2bb802cce0 con 0x7f2ba406de80 2024-05-13T22:52:30.393 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (99s) 70s ago 12m 16.5M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (13m) 70s ago 12m 8044k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (11m) 2m ago 11m 7662k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (12m) 70s ago 12m 6660k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (11m) 2m ago 11m 6665k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (75s) 70s ago 12m 53.0M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (7m) 70s ago 7m 88.2M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (7m) 70s ago 7m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (7m) 2m ago 7m 14.3M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (7m) 2m ago 7m 17.5M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (3m) 70s ago 14m 508M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (2m) 2m ago 11m 450M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:52:30.394 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (14m) 70s ago 14m 55.3M 2048M 18.2.1 5be31c24972a 73e113d0d1b3 2024-05-13T22:52:30.395 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (11m) 2m ago 11m 46.6M 2048M 18.2.1 5be31c24972a bc7d892fcb0c 2024-05-13T22:52:30.395 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (2m) 70s ago 12m 16.0M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:52:30.395 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (2m) 2m ago 11m 3670k - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:52:30.395 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (10m) 70s ago 10m 224M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:52:30.395 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (10m) 70s ago 10m 238M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:52:30.395 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (9m) 70s ago 9m 202M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:52:30.395 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (9m) 2m ago 9m 288M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:52:30.395 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (9m) 2m ago 9m 244M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:52:30.395 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (8m) 2m ago 8m 224M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:52:30.395 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (117s) 70s ago 12m 54.0M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:52:30.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.418+0000 7f2bbfdca700 1 -- 172.21.15.102:0/3135232598 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f2ba406de80 msgr2=0x7f2ba4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.418+0000 7f2bbfdca700 1 --2- 172.21.15.102:0/3135232598 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f2ba406de80 0x7f2ba4070340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80059d0 tx=0x7f2ba8005940 comp rx=0 tx=0).stop 2024-05-13T22:52:30.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.418+0000 7f2bbfdca700 1 -- 172.21.15.102:0/3135232598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bb8100dc0 msgr2=0x7f2bb8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.418+0000 7f2bbfdca700 1 --2- 172.21.15.102:0/3135232598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bb8100dc0 0x7f2bb8073090 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400e920 tx=0x7f2bb400ec30 comp rx=0 tx=0).stop 2024-05-13T22:52:30.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.418+0000 7f2bbfdca700 1 -- 172.21.15.102:0/3135232598 shutdown_connections 2024-05-13T22:52:30.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.418+0000 7f2bbfdca700 1 --2- 172.21.15.102:0/3135232598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bb8100dc0 0x7f2bb8073090 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.418+0000 7f2bbfdca700 1 --2- 172.21.15.102:0/3135232598 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f2ba406de80 0x7f2ba4070340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.418+0000 7f2bbfdca700 1 --2- 172.21.15.102:0/3135232598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2bb8101770 0x7f2bb80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.418+0000 7f2bbfdca700 1 -- 172.21.15.102:0/3135232598 >> 172.21.15.102:0/3135232598 conn(0x7f2bb80fc650 msgr2=0x7f2bb80fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:30.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.418+0000 7f2bbfdca700 1 -- 172.21.15.102:0/3135232598 shutdown_connections 2024-05-13T22:52:30.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.418+0000 7f2bbfdca700 1 -- 172.21.15.102:0/3135232598 wait complete. 2024-05-13T22:52:30.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.529+0000 7f97a21b1700 1 -- 172.21.15.102:0/2652469325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f979c0ffa50 msgr2=0x7f979c0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.529+0000 7f97a21b1700 1 --2- 172.21.15.102:0/2652469325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f979c0ffa50 0x7f979c0ffed0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9790009910 tx=0x7f9790009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:30.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.529+0000 7f97a21b1700 1 -- 172.21.15.102:0/2652469325 shutdown_connections 2024-05-13T22:52:30.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.529+0000 7f97a21b1700 1 --2- 172.21.15.102:0/2652469325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f979c0ffa50 0x7f979c0ffed0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.529+0000 7f97a21b1700 1 --2- 172.21.15.102:0/2652469325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f979c105a50 0x7f979c105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.529+0000 7f97a21b1700 1 -- 172.21.15.102:0/2652469325 >> 172.21.15.102:0/2652469325 conn(0x7f979c0fb540 msgr2=0x7f979c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:30.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.529+0000 7f97a21b1700 1 -- 172.21.15.102:0/2652469325 shutdown_connections 2024-05-13T22:52:30.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.529+0000 7f97a21b1700 1 -- 172.21.15.102:0/2652469325 wait complete. 2024-05-13T22:52:30.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.529+0000 7f97a21b1700 1 Processor -- start 2024-05-13T22:52:30.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f97a21b1700 1 -- start start 2024-05-13T22:52:30.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f97a21b1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f979c0ffa50 0x7f979c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f97a21b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f979c105a50 0x7f979c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f97a21b1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c19d1e0 con 0x7f979c105a50 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f97a21b1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c196690 con 0x7f979c0ffa50 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f979affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f979c105a50 0x7f979c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f979affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f979c105a50 0x7f979c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41580/0 (socket says 172.21.15.102:41580) 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f979affd700 1 -- 172.21.15.102:0/1422173917 learned_addr learned my addr 172.21.15.102:0/1422173917 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f979b7fe700 1 --2- 172.21.15.102:0/1422173917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f979c0ffa50 0x7f979c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f979affd700 1 -- 172.21.15.102:0/1422173917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f979c0ffa50 msgr2=0x7f979c19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f979affd700 1 --2- 172.21.15.102:0/1422173917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f979c0ffa50 0x7f979c19c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f979affd700 1 -- 172.21.15.102:0/1422173917 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97900095f0 con 0x7f979c105a50 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.530+0000 7f979affd700 1 --2- 172.21.15.102:0/1422173917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f979c105a50 0x7f979c19cb50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9790006010 tx=0x7f979000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.531+0000 7f9798ff9700 1 -- 172.21.15.102:0/1422173917 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979001d070 con 0x7f979c105a50 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.531+0000 7f9798ff9700 1 -- 172.21.15.102:0/1422173917 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9790004fb0 con 0x7f979c105a50 2024-05-13T22:52:30.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.531+0000 7f9798ff9700 1 -- 172.21.15.102:0/1422173917 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9790022ea0 con 0x7f979c105a50 2024-05-13T22:52:30.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.531+0000 7f97a21b1700 1 -- 172.21.15.102:0/1422173917 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c196910 con 0x7f979c105a50 2024-05-13T22:52:30.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.531+0000 7f97a21b1700 1 -- 172.21.15.102:0/1422173917 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c196e00 con 0x7f979c105a50 2024-05-13T22:52:30.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.532+0000 7f97a21b1700 1 -- 172.21.15.102:0/1422173917 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c109080 con 0x7f979c105a50 2024-05-13T22:52:30.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.532+0000 7f9798ff9700 1 -- 172.21.15.102:0/1422173917 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f9790022470 con 0x7f979c105a50 2024-05-13T22:52:30.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.533+0000 7f9798ff9700 1 --2- 172.21.15.102:0/1422173917 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f978406de80 0x7f9784070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:30.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.533+0000 7f9798ff9700 1 -- 172.21.15.102:0/1422173917 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f979008f080 con 0x7f979c105a50 2024-05-13T22:52:30.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.533+0000 7f979b7fe700 1 --2- 172.21.15.102:0/1422173917 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f978406de80 0x7f9784070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:30.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.533+0000 7f979b7fe700 1 --2- 172.21.15.102:0/1422173917 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f978406de80 0x7f9784070340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f978c006fd0 tx=0x7f978c008040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:30.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.537+0000 7f9798ff9700 1 -- 172.21.15.102:0/1422173917 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f979005c4f0 con 0x7f979c105a50 2024-05-13T22:52:30.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.842+0000 7f97a21b1700 1 -- 172.21.15.102:0/1422173917 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f979c04ea90 con 0x7f979c105a50 2024-05-13T22:52:30.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.843+0000 7f9798ff9700 1 -- 172.21.15.102:0/1422173917 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f9790022720 con 0x7f979c105a50 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:52:30.821 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-05-13T22:52:30.822 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:52:30.822 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:52:30.822 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:52:30.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.846+0000 7f97a21b1700 1 -- 172.21.15.102:0/1422173917 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f978406de80 msgr2=0x7f9784070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.846+0000 7f97a21b1700 1 --2- 172.21.15.102:0/1422173917 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f978406de80 0x7f9784070340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f978c006fd0 tx=0x7f978c008040 comp rx=0 tx=0).stop 2024-05-13T22:52:30.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.846+0000 7f97a21b1700 1 -- 172.21.15.102:0/1422173917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f979c105a50 msgr2=0x7f979c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.846+0000 7f97a21b1700 1 --2- 172.21.15.102:0/1422173917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f979c105a50 0x7f979c19cb50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9790006010 tx=0x7f979000be20 comp rx=0 tx=0).stop 2024-05-13T22:52:30.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.846+0000 7f97a21b1700 1 -- 172.21.15.102:0/1422173917 shutdown_connections 2024-05-13T22:52:30.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.846+0000 7f97a21b1700 1 --2- 172.21.15.102:0/1422173917 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f978406de80 0x7f9784070340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.846+0000 7f97a21b1700 1 --2- 172.21.15.102:0/1422173917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f979c0ffa50 0x7f979c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.846+0000 7f97a21b1700 1 --2- 172.21.15.102:0/1422173917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f979c105a50 0x7f979c19cb50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.846+0000 7f97a21b1700 1 -- 172.21.15.102:0/1422173917 >> 172.21.15.102:0/1422173917 conn(0x7f979c0fb540 msgr2=0x7f979c069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:30.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.846+0000 7f97a21b1700 1 -- 172.21.15.102:0/1422173917 shutdown_connections 2024-05-13T22:52:30.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.846+0000 7f97a21b1700 1 -- 172.21.15.102:0/1422173917 wait complete. 2024-05-13T22:52:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:30 smithi130 ceph-mon[36591]: pgmap v113: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:30 smithi130 ceph-mon[36591]: from='client.14882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:30 smithi130 ceph-mon[36591]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:30.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:30 smithi102 ceph-mon[28599]: pgmap v113: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:30 smithi102 ceph-mon[28599]: from='client.14882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:30 smithi102 ceph-mon[28599]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:30.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.956+0000 7fcd6a13d700 1 -- 172.21.15.102:0/1110191516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd64069020 msgr2=0x7fcd64105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.956+0000 7fcd6a13d700 1 --2- 172.21.15.102:0/1110191516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd64069020 0x7fcd64105790 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009910 tx=0x7fcd58009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:30.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.956+0000 7fcd6a13d700 1 -- 172.21.15.102:0/1110191516 shutdown_connections 2024-05-13T22:52:30.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.956+0000 7fcd6a13d700 1 --2- 172.21.15.102:0/1110191516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd64069020 0x7fcd64105790 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.956+0000 7fcd6a13d700 1 --2- 172.21.15.102:0/1110191516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd64068670 0x7fcd64068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.956+0000 7fcd6a13d700 1 -- 172.21.15.102:0/1110191516 >> 172.21.15.102:0/1110191516 conn(0x7fcd640755a0 msgr2=0x7fcd640759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:30.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.956+0000 7fcd6a13d700 1 -- 172.21.15.102:0/1110191516 shutdown_connections 2024-05-13T22:52:30.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.956+0000 7fcd6a13d700 1 -- 172.21.15.102:0/1110191516 wait complete. 2024-05-13T22:52:30.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.957+0000 7fcd6a13d700 1 Processor -- start 2024-05-13T22:52:30.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.957+0000 7fcd6a13d700 1 -- start start 2024-05-13T22:52:30.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.957+0000 7fcd6a13d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd64068670 0x7fcd640ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:30.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.957+0000 7fcd6a13d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd64069020 0x7fcd641001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:30.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.957+0000 7fcd6a13d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd64100720 con 0x7fcd64069020 2024-05-13T22:52:30.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.957+0000 7fcd6a13d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd64100860 con 0x7fcd64068670 2024-05-13T22:52:30.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.957+0000 7fcd62ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd64069020 0x7fcd641001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.957+0000 7fcd62ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd64069020 0x7fcd641001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41602/0 (socket says 172.21.15.102:41602) 2024-05-13T22:52:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.957+0000 7fcd62ffd700 1 -- 172.21.15.102:0/3364887575 learned_addr learned my addr 172.21.15.102:0/3364887575 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.958+0000 7fcd637fe700 1 --2- 172.21.15.102:0/3364887575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd64068670 0x7fcd640ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.958+0000 7fcd62ffd700 1 -- 172.21.15.102:0/3364887575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd64068670 msgr2=0x7fcd640ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.958+0000 7fcd62ffd700 1 --2- 172.21.15.102:0/3364887575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd64068670 0x7fcd640ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.958+0000 7fcd62ffd700 1 -- 172.21.15.102:0/3364887575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580095f0 con 0x7fcd64069020 2024-05-13T22:52:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.958+0000 7fcd62ffd700 1 --2- 172.21.15.102:0/3364887575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd64069020 0x7fcd641001e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800bda0 tx=0x7fcd5800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:30.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.958+0000 7fcd60ff9700 1 -- 172.21.15.102:0/3364887575 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd5801d070 con 0x7fcd64069020 2024-05-13T22:52:30.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.958+0000 7fcd6a13d700 1 -- 172.21.15.102:0/3364887575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd64100ae0 con 0x7fcd64069020 2024-05-13T22:52:30.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.958+0000 7fcd60ff9700 1 -- 172.21.15.102:0/3364887575 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd58004fb0 con 0x7fcd64069020 2024-05-13T22:52:30.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.958+0000 7fcd60ff9700 1 -- 172.21.15.102:0/3364887575 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd5800f480 con 0x7fcd64069020 2024-05-13T22:52:30.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.958+0000 7fcd6a13d700 1 -- 172.21.15.102:0/3364887575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd641a2bb0 con 0x7fcd64069020 2024-05-13T22:52:30.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.959+0000 7fcd6a13d700 1 -- 172.21.15.102:0/3364887575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd641090a0 con 0x7fcd64069020 2024-05-13T22:52:30.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.960+0000 7fcd60ff9700 1 -- 172.21.15.102:0/3364887575 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fcd58022870 con 0x7fcd64069020 2024-05-13T22:52:30.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.961+0000 7fcd60ff9700 1 --2- 172.21.15.102:0/3364887575 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fcd4c06de80 0x7fcd4c070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:30.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.961+0000 7fcd60ff9700 1 -- 172.21.15.102:0/3364887575 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcd5808f620 con 0x7fcd64069020 2024-05-13T22:52:30.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.961+0000 7fcd637fe700 1 --2- 172.21.15.102:0/3364887575 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fcd4c06de80 0x7fcd4c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:30.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.961+0000 7fcd637fe700 1 --2- 172.21.15.102:0/3364887575 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fcd4c06de80 0x7fcd4c070340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400ba60 tx=0x7fcd54005d50 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:30.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:30.965+0000 7fcd60ff9700 1 -- 172.21.15.102:0/3364887575 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7fcd58058f60 con 0x7fcd64069020 2024-05-13T22:52:31.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.227+0000 7fcd6a13d700 1 -- 172.21.15.102:0/3364887575 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcd6404ea90 con 0x7fcd64069020 2024-05-13T22:52:31.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.228+0000 7fcd60ff9700 1 -- 172.21.15.102:0/3364887575 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1856 (secure 0 0 0) 0x7fcd5805c620 con 0x7fcd64069020 2024-05-13T22:52:31.205 INFO:teuthology.orchestra.run.smithi102.stdout:e17 2024-05-13T22:52:31.205 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:52:31.205 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:52:31.205 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 17 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:52:31.206 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 47 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=14498} 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:52:31.207 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:52:31.208 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:52:31.208 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:52:31.208 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:52:31.208 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:52:31.208 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:52:31.208 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:52:31.208 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:52:31.208 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:52:31.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.231+0000 7fcd6a13d700 1 -- 172.21.15.102:0/3364887575 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fcd4c06de80 msgr2=0x7fcd4c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:31.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.231+0000 7fcd6a13d700 1 --2- 172.21.15.102:0/3364887575 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fcd4c06de80 0x7fcd4c070340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400ba60 tx=0x7fcd54005d50 comp rx=0 tx=0).stop 2024-05-13T22:52:31.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.231+0000 7fcd6a13d700 1 -- 172.21.15.102:0/3364887575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd64069020 msgr2=0x7fcd641001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:31.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.231+0000 7fcd6a13d700 1 --2- 172.21.15.102:0/3364887575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd64069020 0x7fcd641001e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800bda0 tx=0x7fcd5800be80 comp rx=0 tx=0).stop 2024-05-13T22:52:31.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.231+0000 7fcd6a13d700 1 -- 172.21.15.102:0/3364887575 shutdown_connections 2024-05-13T22:52:31.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.231+0000 7fcd6a13d700 1 --2- 172.21.15.102:0/3364887575 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7fcd4c06de80 0x7fcd4c070340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.231+0000 7fcd6a13d700 1 --2- 172.21.15.102:0/3364887575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd64068670 0x7fcd640ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.231+0000 7fcd6a13d700 1 --2- 172.21.15.102:0/3364887575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd64069020 0x7fcd641001e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.231+0000 7fcd6a13d700 1 -- 172.21.15.102:0/3364887575 >> 172.21.15.102:0/3364887575 conn(0x7fcd640755a0 msgr2=0x7fcd64105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:31.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.231+0000 7fcd6a13d700 1 -- 172.21.15.102:0/3364887575 shutdown_connections 2024-05-13T22:52:31.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.231+0000 7fcd6a13d700 1 -- 172.21.15.102:0/3364887575 wait complete. 2024-05-13T22:52:31.210 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:52:31.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.349+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2033310573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93d8101d00 msgr2=0x7f93d81020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:31.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.349+0000 7f93dfc1e700 1 --2- 172.21.15.102:0/2033310573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93d8101d00 0x7f93d81020e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009910 tx=0x7f93c8009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:31.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.349+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2033310573 shutdown_connections 2024-05-13T22:52:31.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.349+0000 7f93dfc1e700 1 --2- 172.21.15.102:0/2033310573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93d8102620 0x7f93d810ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.349+0000 7f93dfc1e700 1 --2- 172.21.15.102:0/2033310573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93d8101d00 0x7f93d81020e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.349+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2033310573 >> 172.21.15.102:0/2033310573 conn(0x7f93d80fb570 msgr2=0x7f93d80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:31.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.350+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2033310573 shutdown_connections 2024-05-13T22:52:31.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.350+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2033310573 wait complete. 2024-05-13T22:52:31.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.350+0000 7f93dfc1e700 1 Processor -- start 2024-05-13T22:52:31.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.350+0000 7f93dfc1e700 1 -- start start 2024-05-13T22:52:31.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.350+0000 7f93dfc1e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93d8101d00 0x7f93d819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:31.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.350+0000 7f93dfc1e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93d8102620 0x7f93d819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:31.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.350+0000 7f93dfc1e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d819d070 con 0x7f93d8101d00 2024-05-13T22:52:31.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.350+0000 7f93dfc1e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d81965b0 con 0x7f93d8102620 2024-05-13T22:52:31.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93dd9ba700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93d8101d00 0x7f93d819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:31.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93dd9ba700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93d8101d00 0x7f93d819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41614/0 (socket says 172.21.15.102:41614) 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93dd9ba700 1 -- 172.21.15.102:0/2840206221 learned_addr learned my addr 172.21.15.102:0/2840206221 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93dd1b9700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93d8102620 0x7f93d819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93dd9ba700 1 -- 172.21.15.102:0/2840206221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93d8102620 msgr2=0x7f93d819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93dd9ba700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93d8102620 0x7f93d819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93dd9ba700 1 -- 172.21.15.102:0/2840206221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c80095f0 con 0x7f93d8101d00 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93dd1b9700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93d8102620 0x7f93d819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93dd9ba700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93d8101d00 0x7f93d819c530 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f93c8006010 tx=0x7f93c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93ceffd700 1 -- 172.21.15.102:0/2840206221 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93c801d070 con 0x7f93d8101d00 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93ceffd700 1 -- 172.21.15.102:0/2840206221 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93c8004fb0 con 0x7f93d8101d00 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2840206221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d8196830 con 0x7f93d8101d00 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93ceffd700 1 -- 172.21.15.102:0/2840206221 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93c8022ea0 con 0x7f93d8101d00 2024-05-13T22:52:31.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.351+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2840206221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d8196d20 con 0x7f93d8101d00 2024-05-13T22:52:31.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.352+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2840206221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d8108210 con 0x7f93d8101d00 2024-05-13T22:52:31.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.353+0000 7f93ceffd700 1 -- 172.21.15.102:0/2840206221 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f93c8022470 con 0x7f93d8101d00 2024-05-13T22:52:31.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.353+0000 7f93ceffd700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93c406de80 0x7f93c4070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:31.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.353+0000 7f93ceffd700 1 -- 172.21.15.102:0/2840206221 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f93c808f0a0 con 0x7f93d8101d00 2024-05-13T22:52:31.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.354+0000 7f93dd1b9700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93c406de80 0x7f93c4070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:31.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.354+0000 7f93dd1b9700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93c406de80 0x7f93c4070340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f93d4005fd0 tx=0x7f93d4005e30 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:31.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.361+0000 7f93ceffd700 1 -- 172.21.15.102:0/2840206221 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f93c805c630 con 0x7f93d8101d00 2024-05-13T22:52:31.560 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.582+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2840206221 --> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93d8197050 con 0x7f93c406de80 2024-05-13T22:52:31.561 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.584+0000 7f93ceffd700 1 -- 172.21.15.102:0/2840206221 <== mgr.24431 v2:172.21.15.102:6800/2015310067 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f93d8197050 con 0x7f93c406de80 2024-05-13T22:52:31.561 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:52:31.561 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:52:31.561 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:52:31.561 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:52:31.561 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:52:31.561 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr" 2024-05-13T22:52:31.561 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:52:31.561 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "2/23 daemons upgraded", 2024-05-13T22:52:31.561 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading mon daemons", 2024-05-13T22:52:31.562 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:52:31.562 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:52:31.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.586+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2840206221 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93c406de80 msgr2=0x7f93c4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:31.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.586+0000 7f93dfc1e700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93c406de80 0x7f93c4070340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f93d4005fd0 tx=0x7f93d4005e30 comp rx=0 tx=0).stop 2024-05-13T22:52:31.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.586+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2840206221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93d8101d00 msgr2=0x7f93d819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:31.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.586+0000 7f93dfc1e700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93d8101d00 0x7f93d819c530 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f93c8006010 tx=0x7f93c800be20 comp rx=0 tx=0).stop 2024-05-13T22:52:31.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.586+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2840206221 shutdown_connections 2024-05-13T22:52:31.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.586+0000 7f93dfc1e700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93d8101d00 0x7f93d819c530 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.586+0000 7f93dfc1e700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f93c406de80 0x7f93c4070340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.586+0000 7f93dfc1e700 1 --2- 172.21.15.102:0/2840206221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93d8102620 0x7f93d819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.586+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2840206221 >> 172.21.15.102:0/2840206221 conn(0x7f93d80fb570 msgr2=0x7f93d8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:31.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.586+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2840206221 shutdown_connections 2024-05-13T22:52:31.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.586+0000 7f93dfc1e700 1 -- 172.21.15.102:0/2840206221 wait complete. 2024-05-13T22:52:31.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.694+0000 7f8520e20700 1 -- 172.21.15.102:0/2112826392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f851c0ffa50 msgr2=0x7f851c0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:31.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.694+0000 7f8520e20700 1 --2- 172.21.15.102:0/2112826392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f851c0ffa50 0x7f851c0ffed0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f850c009910 tx=0x7f850c009c20 comp rx=0 tx=0).stop 2024-05-13T22:52:31.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.694+0000 7f8520e20700 1 -- 172.21.15.102:0/2112826392 shutdown_connections 2024-05-13T22:52:31.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.694+0000 7f8520e20700 1 --2- 172.21.15.102:0/2112826392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f851c0ffa50 0x7f851c0ffed0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.694+0000 7f8520e20700 1 --2- 172.21.15.102:0/2112826392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f851c105a50 0x7f851c105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.694+0000 7f8520e20700 1 -- 172.21.15.102:0/2112826392 >> 172.21.15.102:0/2112826392 conn(0x7f851c0fb540 msgr2=0x7f851c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:31.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.694+0000 7f8520e20700 1 -- 172.21.15.102:0/2112826392 shutdown_connections 2024-05-13T22:52:31.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.694+0000 7f8520e20700 1 -- 172.21.15.102:0/2112826392 wait complete. 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.695+0000 7f8520e20700 1 Processor -- start 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.695+0000 7f8520e20700 1 -- start start 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.695+0000 7f8520e20700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f851c0ffa50 0x7f851c197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.695+0000 7f8520e20700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f851c105a50 0x7f851c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.695+0000 7f8520e20700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c198170 con 0x7f851c105a50 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.695+0000 7f8520e20700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c1982b0 con 0x7f851c0ffa50 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.695+0000 7f8519d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f851c105a50 0x7f851c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f8519d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f851c105a50 0x7f851c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41632/0 (socket says 172.21.15.102:41632) 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f8519d9b700 1 -- 172.21.15.102:0/1719613623 learned_addr learned my addr 172.21.15.102:0/1719613623 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f851a59c700 1 --2- 172.21.15.102:0/1719613623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f851c0ffa50 0x7f851c197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f8519d9b700 1 -- 172.21.15.102:0/1719613623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f851c0ffa50 msgr2=0x7f851c197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f8519d9b700 1 --2- 172.21.15.102:0/1719613623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f851c0ffa50 0x7f851c197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f8519d9b700 1 -- 172.21.15.102:0/1719613623 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c0095f0 con 0x7f851c105a50 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f8519d9b700 1 --2- 172.21.15.102:0/1719613623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f851c105a50 0x7f851c197b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f850c006010 tx=0x7f850c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f850b7fe700 1 -- 172.21.15.102:0/1719613623 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f850c01d070 con 0x7f851c105a50 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f8520e20700 1 -- 172.21.15.102:0/1719613623 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c1a0680 con 0x7f851c105a50 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f850b7fe700 1 -- 172.21.15.102:0/1719613623 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f850c005dc0 con 0x7f851c105a50 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f850b7fe700 1 -- 172.21.15.102:0/1719613623 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f850c017400 con 0x7f851c105a50 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.696+0000 7f8520e20700 1 -- 172.21.15.102:0/1719613623 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c1a0a90 con 0x7f851c105a50 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.697+0000 7f8520e20700 1 -- 172.21.15.102:0/1719613623 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c107fe0 con 0x7f851c105a50 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.698+0000 7f850b7fe700 1 -- 172.21.15.102:0/1719613623 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f850c003a60 con 0x7f851c105a50 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.699+0000 7f850b7fe700 1 --2- 172.21.15.102:0/1719613623 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f850406de80 0x7f8504070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.699+0000 7f850b7fe700 1 -- 172.21.15.102:0/1719613623 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f850c08f040 con 0x7f851c105a50 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.699+0000 7f851a59c700 1 --2- 172.21.15.102:0/1719613623 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f850406de80 0x7f8504070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:52:31.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.699+0000 7f851a59c700 1 --2- 172.21.15.102:0/1719613623 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f850406de80 0x7f8504070340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8510005950 tx=0x7f851000b410 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:52:31.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.704+0000 7f850b7fe700 1 -- 172.21.15.102:0/1719613623 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185723 (secure 0 0 0) 0x7f850c058850 con 0x7f851c105a50 2024-05-13T22:52:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:31 smithi102 ceph-mon[28599]: from='client.14890 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:31 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:31 smithi102 ceph-mon[28599]: Upgrade: Target is version 19.0.0-2565-gc3e55145 (squid) 2024-05-13T22:52:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:31 smithi102 ceph-mon[28599]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, digests ['quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64'] 2024-05-13T22:52:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:31 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:52:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:31 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:52:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:31 smithi102 ceph-mon[28599]: Upgrade: Setting container_image for all mgr 2024-05-13T22:52:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:31 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:31 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-13T22:52:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:31 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1422173917' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:52:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:31 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/3364887575' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:52:31.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.997+0000 7f8520e20700 1 -- 172.21.15.102:0/1719613623 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f851c1079c0 con 0x7f851c105a50 2024-05-13T22:52:31.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:31.998+0000 7f850b7fe700 1 -- 172.21.15.102:0/1719613623 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f850c05bf10 con 0x7f851c105a50 2024-05-13T22:52:31.975 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:52:31.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:32.000+0000 7f8520e20700 1 -- 172.21.15.102:0/1719613623 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f850406de80 msgr2=0x7f8504070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:31.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:32.000+0000 7f8520e20700 1 --2- 172.21.15.102:0/1719613623 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f850406de80 0x7f8504070340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8510005950 tx=0x7f851000b410 comp rx=0 tx=0).stop 2024-05-13T22:52:31.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:32.000+0000 7f8520e20700 1 -- 172.21.15.102:0/1719613623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f851c105a50 msgr2=0x7f851c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:52:31.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:32.000+0000 7f8520e20700 1 --2- 172.21.15.102:0/1719613623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f851c105a50 0x7f851c197b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f850c006010 tx=0x7f850c00bd70 comp rx=0 tx=0).stop 2024-05-13T22:52:31.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:32.000+0000 7f8520e20700 1 -- 172.21.15.102:0/1719613623 shutdown_connections 2024-05-13T22:52:31.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:32.000+0000 7f8520e20700 1 --2- 172.21.15.102:0/1719613623 >> [v2:172.21.15.102:6800/2015310067,v1:172.21.15.102:6801/2015310067] conn(0x7f850406de80 0x7f8504070340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:32.000+0000 7f8520e20700 1 --2- 172.21.15.102:0/1719613623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f851c0ffa50 0x7f851c197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:32.000+0000 7f8520e20700 1 --2- 172.21.15.102:0/1719613623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f851c105a50 0x7f851c197b50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:52:31.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:32.000+0000 7f8520e20700 1 -- 172.21.15.102:0/1719613623 >> 172.21.15.102:0/1719613623 conn(0x7f851c0fb540 msgr2=0x7f851c0fc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:52:31.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:32.000+0000 7f8520e20700 1 -- 172.21.15.102:0/1719613623 shutdown_connections 2024-05-13T22:52:31.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:52:32.000+0000 7f8520e20700 1 -- 172.21.15.102:0/1719613623 wait complete. 2024-05-13T22:52:32.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:31 smithi130 ceph-mon[36591]: from='client.14890 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:32.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:31 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:32.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:31 smithi130 ceph-mon[36591]: Upgrade: Target is version 19.0.0-2565-gc3e55145 (squid) 2024-05-13T22:52:32.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:31 smithi130 ceph-mon[36591]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, digests ['quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64'] 2024-05-13T22:52:32.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:31 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:52:32.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:31 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:52:32.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:31 smithi130 ceph-mon[36591]: Upgrade: Setting container_image for all mgr 2024-05-13T22:52:32.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:31 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:32.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:31 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-13T22:52:32.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:31 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1422173917' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:52:32.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:31 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/3364887575' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:52:33.045 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:32 smithi102 ceph-mon[28599]: pgmap v114: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:33.045 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:32 smithi102 ceph-mon[28599]: from='client.14902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:33.045 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:32 smithi102 ceph-mon[28599]: from='client.? 172.21.15.102:0/1719613623' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:52:33.045 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:32 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:33.045 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:32 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:52:33.045 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:32 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:52:33.046 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:32 smithi102 ceph-mon[28599]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:52:33.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:32 smithi130 ceph-mon[36591]: pgmap v114: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:33.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:32 smithi130 ceph-mon[36591]: from='client.14902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:52:33.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:32 smithi130 ceph-mon[36591]: from='client.? 172.21.15.102:0/1719613623' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:52:33.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:32 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:33.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:32 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:52:33.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:32 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:52:33.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:32 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:52:34.012 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:33 smithi102 ceph-mon[28599]: Upgrade: Updating mon.smithi102 2024-05-13T22:52:34.012 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:33 smithi102 ceph-mon[28599]: Deploying daemon mon.smithi102 on smithi102 2024-05-13T22:52:34.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:33 smithi130 ceph-mon[36591]: Upgrade: Updating mon.smithi102 2024-05-13T22:52:34.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:33 smithi130 ceph-mon[36591]: Deploying daemon mon.smithi102 on smithi102 2024-05-13T22:52:34.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:34 smithi102 systemd[1]: Stopping Ceph mon.smithi102 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:52:34.818 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:34 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102[28595]: 2024-05-13T22:52:34.506+0000 7efcbfcd9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi102 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T22:52:34.818 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:34 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102[28595]: 2024-05-13T22:52:34.506+0000 7efcbfcd9700 -1 mon.smithi102@0(leader) e2 *** Got Signal Terminated *** 2024-05-13T22:52:34.818 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:34 smithi102 podman[111523]: 2024-05-13 22:52:34.588041128 +0000 UTC m=+0.212341506 container died 73e113d0d1b38bc6b4bcdf7197137859284183a31e8c14b26f4f658b6b07b284 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=HEAD) 2024-05-13T22:52:35.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:34 smithi102 podman[111523]: 2024-05-13 22:52:34.93997659 +0000 UTC m=+0.564276972 container cleanup 73e113d0d1b38bc6b4bcdf7197137859284183a31e8c14b26f4f658b6b07b284 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=HEAD) 2024-05-13T22:52:35.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:34 smithi102 bash[111523]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102 2024-05-13T22:52:35.562 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:35 smithi102 podman[111546]: 2024-05-13 22:52:35.207940759 +0000 UTC m=+0.619373132 container remove 73e113d0d1b38bc6b4bcdf7197137859284183a31e8c14b26f4f658b6b07b284 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=HEAD, maintainer=Guillaume Abrioux ) 2024-05-13T22:52:35.562 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:35 smithi102 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi102.service: Deactivated successfully. 2024-05-13T22:52:35.562 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:35 smithi102 systemd[1]: Stopped Ceph mon.smithi102 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:52:35.562 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:35 smithi102 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi102.service: Consumed 14.181s CPU time. 2024-05-13T22:52:35.913 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:35 smithi102 systemd[1]: Starting Ceph mon.smithi102 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:52:35.913 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:35 smithi102 podman[111675]: 2024-05-13 22:52:35.706164254 +0000 UTC m=+0.017869338 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:52:35.913 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:35 smithi102 podman[111675]: 2024-05-13 22:52:35.809635348 +0000 UTC m=+0.121340439 container create 0b15816f3f6bcd70bdc4a8681809b5598c8f715b1f4c5333e3bebdd3e1b25435 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T22:52:36.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 podman[111675]: 2024-05-13 22:52:36.045562826 +0000 UTC m=+0.357267904 container init 0b15816f3f6bcd70bdc4a8681809b5598c8f715b1f4c5333e3bebdd3e1b25435 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-c3e5514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-13T22:52:36.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 podman[111675]: 2024-05-13 22:52:36.04909945 +0000 UTC m=+0.360804519 container start 0b15816f3f6bcd70bdc4a8681809b5598c8f715b1f4c5333e3bebdd3e1b25435 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-c3e5514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: set uid:gid to 167:167 (ceph:ceph) 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev), process ceph-mon, pid 2 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: pidfile_write: ignore empty --pid-file 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: load: jerasure load: lrc 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: RocksDB version: 7.9.2 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Git sha 0 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Compile date 2024-05-13 16:38:45 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: DB SUMMARY 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: DB Session ID: EC6BJL2KB6XRZGD1R9SU 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: CURRENT file: CURRENT 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: IDENTITY file: IDENTITY 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1509 Bytes 2024-05-13T22:52:36.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi102/store.db dir, Total Num: 1, files: 000032.sst 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi102/store.db: 000030.log size: 3810682 ; 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.error_if_exists: 0 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.create_if_missing: 0 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.paranoid_checks: 1 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.env: 0x562da2fa1c60 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.fs: PosixFileSystem 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.info_log: 0x562da3c29980 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_file_opening_threads: 16 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.statistics: (nil) 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.use_fsync: 0 2024-05-13T22:52:36.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_log_file_size: 0 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.keep_log_file_num: 1000 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.recycle_log_file_num: 0 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.allow_fallocate: 1 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.allow_mmap_reads: 0 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.allow_mmap_writes: 0 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.use_direct_reads: 0 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.create_missing_column_families: 0 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.db_log_dir: 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.wal_dir: 2024-05-13T22:52:36.180 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.advise_random_on_open: 1 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.db_write_buffer_size: 0 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.write_buffer_manager: 0x562da3bfc1e0 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.rate_limiter: (nil) 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-13T22:52:36.181 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.wal_recovery_mode: 2 2024-05-13T22:52:36.182 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.enable_thread_tracking: 0 2024-05-13T22:52:36.182 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.enable_pipelined_write: 0 2024-05-13T22:52:36.182 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.unordered_write: 0 2024-05-13T22:52:36.182 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-13T22:52:36.182 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-13T22:52:36.182 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-13T22:52:36.182 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-13T22:52:36.182 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.row_cache: None 2024-05-13T22:52:36.182 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.wal_filter: None 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.allow_ingest_behind: 0 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.two_write_queues: 0 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.manual_wal_flush: 0 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.wal_compression: 0 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.atomic_flush: 0 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.log_readahead_size: 0 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-13T22:52:36.184 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.best_efforts_recovery: 0 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.allow_data_in_errors: 0 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.db_host_id: __hostname__ 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_background_jobs: 2 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_background_compactions: -1 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_subcompactions: 1 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_total_wal_size: 0 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-13T22:52:36.185 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_open_files: -1 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bytes_per_sync: 0 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_readahead_size: 0 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_background_flushes: -1 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Compression algorithms supported: 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: kZSTD supported: 0 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: kXpressCompression supported: 0 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: kLZ4HCCompression supported: 1 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: kZlibCompression supported: 1 2024-05-13T22:52:36.186 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: kSnappyCompression supported: 1 2024-05-13T22:52:36.187 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: kLZ4Compression supported: 1 2024-05-13T22:52:36.187 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: kBZip2Compression supported: 0 2024-05-13T22:52:36.187 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-13T22:52:36.187 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-13T22:52:36.187 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi102/store.db/MANIFEST-000015 2024-05-13T22:52:36.187 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-13T22:52:36.189 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-13T22:52:36.189 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.merge_operator: 2024-05-13T22:52:36.189 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_filter: None 2024-05-13T22:52:36.189 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_filter_factory: None 2024-05-13T22:52:36.189 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.sst_partitioner_factory: None 2024-05-13T22:52:36.189 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-13T22:52:36.189 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-13T22:52:36.189 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562da3c29a80) 2024-05-13T22:52:36.190 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: cache_index_and_filter_blocks: 1 2024-05-13T22:52:36.190 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-13T22:52:36.190 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-13T22:52:36.190 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: pin_top_level_index_and_filter: 1 2024-05-13T22:52:36.190 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: index_type: 0 2024-05-13T22:52:36.190 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: data_block_index_type: 0 2024-05-13T22:52:36.190 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: index_shortening: 1 2024-05-13T22:52:36.190 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-13T22:52:36.190 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: checksum: 4 2024-05-13T22:52:36.190 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: no_block_cache: 0 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_cache: 0x562da3c2d090 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_cache_name: BinnedLRUCache 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_cache_options: 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: capacity : 536870912 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: num_shard_bits : 4 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: strict_capacity_limit : 0 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: high_pri_pool_ratio: 0.000 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_cache_compressed: (nil) 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: persistent_cache: (nil) 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_size: 4096 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_size_deviation: 10 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_restart_interval: 16 2024-05-13T22:52:36.191 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: index_block_restart_interval: 1 2024-05-13T22:52:36.192 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: metadata_block_size: 4096 2024-05-13T22:52:36.192 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: partition_filters: 0 2024-05-13T22:52:36.192 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: use_delta_encoding: 1 2024-05-13T22:52:36.192 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: filter_policy: bloomfilter 2024-05-13T22:52:36.192 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: whole_key_filtering: 1 2024-05-13T22:52:36.192 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: verify_compression: 0 2024-05-13T22:52:36.192 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: read_amp_bytes_per_bit: 0 2024-05-13T22:52:36.192 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: format_version: 5 2024-05-13T22:52:36.192 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: enable_index_compression: 1 2024-05-13T22:52:36.192 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: block_align: 0 2024-05-13T22:52:36.192 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: max_auto_readahead_size: 262144 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: prepopulate_block_cache: 0 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: initial_auto_readahead_size: 8192 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: num_file_reads_for_auto_readahead: 2 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.write_buffer_size: 33554432 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_write_buffer_number: 2 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compression: NoCompression 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bottommost_compression: Disabled 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.prefix_extractor: nullptr 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.num_levels: 7 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-13T22:52:36.193 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compression_opts.level: 32767 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compression_opts.strategy: 0 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-13T22:52:36.194 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compression_opts.enabled: false 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.target_file_size_base: 67108864 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-13T22:52:36.195 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.arena_block_size: 1048576 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.disable_auto_compactions: 0 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-13T22:52:36.196 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.table_properties_collectors: 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.inplace_update_support: 0 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.bloom_locality: 0 2024-05-13T22:52:36.197 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.max_successive_merges: 0 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.paranoid_file_checks: 0 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.force_consistency_checks: 1 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.report_bg_io_stats: 0 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.ttl: 2592000 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.enable_blob_files: false 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.min_blob_size: 0 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.blob_file_size: 268435456 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-13T22:52:36.198 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.blob_file_starting_level: 0 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi102/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12117, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bf00ac29-5921-412f-8cd0-088117acaf84 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715640756078933, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715640756090095, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 3314384, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12118, "largest_seqno": 15416, "table_properties": {"data_size": 3301702, "index_size": 8013, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3653, "raw_key_size": 31806, "raw_average_key_size": 21, "raw_value_size": 3273075, "raw_average_value_size": 2252, "num_data_blocks": 364, "num_entries": 1453, "num_filter_entries": 1453, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715640756, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bf00ac29-5921-412f-8cd0-088117acaf84", "db_session_id": "EC6BJL2KB6XRZGD1R9SU", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715640756090200, "job": 1, "event": "recovery_finished"} 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-05-13T22:52:36.199 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi102/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562da3d1e000 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: DB pointer 0x562da3d08000 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: ** DB Stats ** 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-13T22:52:36.200 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: ** Compaction Stats [default] ** 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: L0 1/0 3.16 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 630.8 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: L6 1/0 5.80 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Sum 2/0 8.97 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 630.8 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 630.8 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: ** Compaction Stats [default] ** 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 630.8 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-13T22:52:36.201 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Cumulative compaction: 0.00 GB write, 176.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Interval compaction: 0.00 GB write, 176.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Block cache BinnedLRUCache@0x562da3c2d090#2 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-13T22:52:36.202 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: starting mon.smithi102 rank 0 at public addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] at bind addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi102 fsid 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: mon.smithi102@-1(???) e2 preinit fsid 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: mon.smithi102@-1(???).mds e17 new map 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: mon.smithi102@-1(???).mds e17 print_map 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: e17 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: legacy client fscid: 1 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Filesystem 'cephfs' (1) 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: fs_name cephfs 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: epoch 17 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:52:36.203 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: tableserver 0 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: root 0 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: session_timeout 60 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: session_autoclose 300 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: max_file_size 1099511627776 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: max_xattr_size 65536 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: required_client_features {} 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: last_failure 0 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: last_failure_osd_epoch 47 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: max_mds 1 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: in 0 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: up {0=14498} 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: failed 2024-05-13T22:52:36.204 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: damaged 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: stopped 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: data_pools [3] 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: metadata_pool 2 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: inline_data disabled 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: balancer 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: bal_rank_mask -1 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: standby_count_wanted 1 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: qdb_cluster leader: 0 members: 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: [mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: [mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: Standby daemons: 2024-05-13T22:52:36.205 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: 2024-05-13T22:52:36.206 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: [mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:52:36.206 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout: [mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:52:36.206 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: mon.smithi102@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-13T22:52:36.206 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: mon.smithi102@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-13T22:52:36.206 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: mon.smithi102@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-13T22:52:36.206 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: mon.smithi102@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-13T22:52:36.206 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 ceph-mon[111694]: mon.smithi102@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-05-13T22:52:36.206 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 bash[111675]: 0b15816f3f6bcd70bdc4a8681809b5598c8f715b1f4c5333e3bebdd3e1b25435 2024-05-13T22:52:36.206 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:36 smithi102 systemd[1]: Started Ceph mon.smithi102 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:52:39.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:52:39.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: pgmap v116: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: pgmap v117: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: mon.smithi102 calling monitor election 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: mon.smithi102 is new leader, mons smithi102,smithi130 in quorum (ranks 0,1) 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: monmap epoch 2 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: fsid 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: last_changed 2024-05-13T22:40:53.326926+0000 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: created 2024-05-13T22:38:11.166577+0000 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: min_mon_release 18 (reef) 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: election_strategy: 1 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: 0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.smithi102 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.smithi130 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:52:39.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: osdmap e49: 6 total, 6 up, 6 in 2024-05-13T22:52:39.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: mgrmap e32: smithi102.xcijrj(active, since 3m), standbys: smithi130.uoqoxl 2024-05-13T22:52:39.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: overall HEALTH_OK 2024-05-13T22:52:39.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: from='mgr.24431 ' entity='' 2024-05-13T22:52:39.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:39 smithi130 ceph-mon[36591]: mgrmap e33: smithi102.xcijrj(active, since 3m), standbys: smithi130.uoqoxl 2024-05-13T22:52:39.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:52:39.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: from='mgr.24431 172.21.15.102:0/1167449457' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:52:39.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: pgmap v116: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:52:39.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: pgmap v117: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:52:39.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: mon.smithi102 calling monitor election 2024-05-13T22:52:39.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: mon.smithi102 is new leader, mons smithi102,smithi130 in quorum (ranks 0,1) 2024-05-13T22:52:39.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: monmap epoch 2 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: fsid 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: last_changed 2024-05-13T22:40:53.326926+0000 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: created 2024-05-13T22:38:11.166577+0000 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: min_mon_release 18 (reef) 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: election_strategy: 1 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: 0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.smithi102 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.smithi130 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: osdmap e49: 6 total, 6 up, 6 in 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: mgrmap e32: smithi102.xcijrj(active, since 3m), standbys: smithi130.uoqoxl 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: overall HEALTH_OK 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: from='mgr.24431 ' entity='' 2024-05-13T22:52:39.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:39 smithi102 ceph-mon[111694]: mgrmap e33: smithi102.xcijrj(active, since 3m), standbys: smithi130.uoqoxl 2024-05-13T22:52:46.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: Active manager daemon smithi102.xcijrj restarted 2024-05-13T22:52:46.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: Activating manager daemon smithi102.xcijrj 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: osdmap e50: 6 total, 6 up, 6 in 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: mgrmap e34: smithi102.xcijrj(active, starting, since 0.0059511s), standbys: smithi130.uoqoxl 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi102.xcijrj", "id": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi130.uoqoxl", "id": "smithi130.uoqoxl"}]: dispatch 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:52:46.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: Manager daemon smithi102.xcijrj is now available 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: Standby manager daemon smithi130.uoqoxl restarted 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: Standby manager daemon smithi130.uoqoxl started 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/2003429921' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/crt"}]: dispatch 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/2003429921' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/2003429921' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/key"}]: dispatch 2024-05-13T22:52:46.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:46 smithi130 ceph-mon[36591]: from='mgr.? 172.21.15.130:0/2003429921' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:52:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: Active manager daemon smithi102.xcijrj restarted 2024-05-13T22:52:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: Activating manager daemon smithi102.xcijrj 2024-05-13T22:52:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: osdmap e50: 6 total, 6 up, 6 in 2024-05-13T22:52:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: mgrmap e34: smithi102.xcijrj(active, starting, since 0.0059511s), standbys: smithi130.uoqoxl 2024-05-13T22:52:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:52:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:52:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi102.xcijrj", "id": "smithi102.xcijrj"}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr metadata", "who": "smithi130.uoqoxl", "id": "smithi130.uoqoxl"}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-13T22:52:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-13T22:52:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-13T22:52:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: Manager daemon smithi102.xcijrj is now available 2024-05-13T22:52:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: Standby manager daemon smithi130.uoqoxl restarted 2024-05-13T22:52:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: Standby manager daemon smithi130.uoqoxl started 2024-05-13T22:52:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.? 172.21.15.130:0/2003429921' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/crt"}]: dispatch 2024-05-13T22:52:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.? 172.21.15.130:0/2003429921' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-13T22:52:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.? 172.21.15.130:0/2003429921' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi130.uoqoxl/key"}]: dispatch 2024-05-13T22:52:46.429 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:46 smithi102 ceph-mon[111694]: from='mgr.? 172.21.15.130:0/2003429921' entity='mgr.smithi130.uoqoxl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-13T22:52:47.260 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:52:47.260 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:52:47.260 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:52:47.260 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:47 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:52:47.260 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/trash_purge_schedule"}]: dispatch 2024-05-13T22:52:47.260 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:47 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/trash_purge_schedule"}]: dispatch 2024-05-13T22:52:47.260 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:47 smithi102 ceph-mon[111694]: mgrmap e35: smithi102.xcijrj(active, since 1.00978s), standbys: smithi130.uoqoxl 2024-05-13T22:52:47.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:47 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:52:47.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:47 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:52:47.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:47 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:52:47.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:47 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/mirror_snapshot_schedule"}]: dispatch 2024-05-13T22:52:47.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:47 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/trash_purge_schedule"}]: dispatch 2024-05-13T22:52:47.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:47 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi102.xcijrj/trash_purge_schedule"}]: dispatch 2024-05-13T22:52:47.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:47 smithi130 ceph-mon[36591]: mgrmap e35: smithi102.xcijrj(active, since 1.00978s), standbys: smithi130.uoqoxl 2024-05-13T22:52:48.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:48 smithi130 ceph-mon[36591]: [13/May/2024:22:52:46] ENGINE Bus STARTING 2024-05-13T22:52:48.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:48 smithi130 ceph-mon[36591]: [13/May/2024:22:52:46] ENGINE Serving on http://172.21.15.102:8765 2024-05-13T22:52:48.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:48 smithi130 ceph-mon[36591]: [13/May/2024:22:52:46] ENGINE Serving on https://172.21.15.102:7150 2024-05-13T22:52:48.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:48 smithi130 ceph-mon[36591]: [13/May/2024:22:52:46] ENGINE Bus STARTED 2024-05-13T22:52:48.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:48 smithi130 ceph-mon[36591]: pgmap v3: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-13T22:52:48.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:48 smithi102 ceph-mon[111694]: [13/May/2024:22:52:46] ENGINE Bus STARTING 2024-05-13T22:52:48.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:48 smithi102 ceph-mon[111694]: [13/May/2024:22:52:46] ENGINE Serving on http://172.21.15.102:8765 2024-05-13T22:52:48.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:48 smithi102 ceph-mon[111694]: [13/May/2024:22:52:46] ENGINE Serving on https://172.21.15.102:7150 2024-05-13T22:52:48.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:48 smithi102 ceph-mon[111694]: [13/May/2024:22:52:46] ENGINE Bus STARTED 2024-05-13T22:52:48.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:48 smithi102 ceph-mon[111694]: pgmap v3: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-13T22:52:49.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:49 smithi130 ceph-mon[36591]: pgmap v4: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-13T22:52:49.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:49 smithi130 ceph-mon[36591]: mgrmap e36: smithi102.xcijrj(active, since 2s), standbys: smithi130.uoqoxl 2024-05-13T22:52:49.419 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:49 smithi102 ceph-mon[111694]: pgmap v4: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-13T22:52:49.420 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:49 smithi102 ceph-mon[111694]: mgrmap e36: smithi102.xcijrj(active, since 2s), standbys: smithi130.uoqoxl 2024-05-13T22:52:50.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:50 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:50.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:50 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:50.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:50 smithi102 ceph-mon[111694]: pgmap v5: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-13T22:52:50.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:50 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:50.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:50 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:50.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:50 smithi130 ceph-mon[36591]: pgmap v5: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-13T22:52:51.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:51 smithi130 ceph-mon[36591]: mgrmap e37: smithi102.xcijrj(active, since 4s), standbys: smithi130.uoqoxl 2024-05-13T22:52:51.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:51 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:51.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:51 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:51.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:51 smithi102 ceph-mon[111694]: mgrmap e37: smithi102.xcijrj(active, since 4s), standbys: smithi130.uoqoxl 2024-05-13T22:52:51.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:51 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:51.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:51 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:52.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:52 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:52.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:52 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:52.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:52 smithi102 ceph-mon[111694]: pgmap v6: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 0 B/s wr, 25 op/s 2024-05-13T22:52:52.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:52 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:52.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:52 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:52.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:52 smithi130 ceph-mon[36591]: pgmap v6: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 0 B/s wr, 25 op/s 2024-05-13T22:52:53.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:53 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:53.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:53 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:53.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:53 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:53.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:53 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:54.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:54 smithi130 ceph-mon[36591]: pgmap v7: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-05-13T22:52:54.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:54 smithi102 ceph-mon[111694]: pgmap v7: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-05-13T22:52:57.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:57 smithi130 ceph-mon[36591]: pgmap v8: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-13T22:52:57.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:57 smithi130 ceph-mon[36591]: Detected new or changed devices on smithi102 2024-05-13T22:52:57.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:57 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:57.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:57 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:57.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:57 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:52:57.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:57 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:52:57.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:57 smithi130 ceph-mon[36591]: Updating smithi102:/etc/ceph/ceph.conf 2024-05-13T22:52:57.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:57 smithi130 ceph-mon[36591]: Updating smithi130:/etc/ceph/ceph.conf 2024-05-13T22:52:57.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:57 smithi102 ceph-mon[111694]: pgmap v8: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-05-13T22:52:57.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:57 smithi102 ceph-mon[111694]: Detected new or changed devices on smithi102 2024-05-13T22:52:57.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:57 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:57.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:57 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:57.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:57 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:52:57.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:57 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:52:57.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:57 smithi102 ceph-mon[111694]: Updating smithi102:/etc/ceph/ceph.conf 2024-05-13T22:52:57.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:57 smithi102 ceph-mon[111694]: Updating smithi130:/etc/ceph/ceph.conf 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:58.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:52:58.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:52:58.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:58 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-13T22:52:58.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:52:58.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-13T22:52:58.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:52:58.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-05-13T22:52:58.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: Updating smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:52:58.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: Updating smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-13T22:52:58.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:58.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:58.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:58.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:58.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:58.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:52:58.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:52:58.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:58 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-13T22:52:59.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:59 smithi130 ceph-mon[36591]: pgmap v9: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-13T22:52:59.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:59 smithi130 ceph-mon[36591]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:59.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:59 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:52:59.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:59 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:52:59.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:52:59 smithi130 ceph-mon[36591]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:52:59.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:59 smithi102 ceph-mon[111694]: pgmap v9: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-13T22:52:59.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:59 smithi102 ceph-mon[111694]: from='mgr.34104 ' entity='mgr.smithi102.xcijrj' 2024-05-13T22:52:59.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:52:59.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:52:59.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:52:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:00.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:00 smithi102 ceph-mon[111694]: Upgrade: Updating mon.smithi130 2024-05-13T22:53:00.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:00 smithi102 ceph-mon[111694]: Deploying daemon mon.smithi130 on smithi130 2024-05-13T22:53:00.439 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:00 smithi130 ceph-mon[36591]: Upgrade: Updating mon.smithi130 2024-05-13T22:53:00.439 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:00 smithi130 ceph-mon[36591]: Deploying daemon mon.smithi130 on smithi130 2024-05-13T22:53:00.994 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:00 smithi130 systemd[1]: Stopping Ceph mon.smithi130 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:53:01.337 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:01 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130[36573]: 2024-05-13T22:53:01.017+0000 7f00e95e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi130 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T22:53:01.337 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:01 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130[36573]: 2024-05-13T22:53:01.017+0000 7f00e95e0700 -1 mon.smithi130@1(peon) e2 *** Got Signal Terminated *** 2024-05-13T22:53:01.337 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:01 smithi130 podman[84106]: 2024-05-13 22:53:01.069225773 +0000 UTC m=+0.182386497 container died bc7d892fcb0c45c6d841ea9d3d63a8ea5808af907ea9daa9f657f16146c53bd4 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-05-13T22:53:01.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:01 smithi130 podman[84106]: 2024-05-13 22:53:01.459207113 +0000 UTC m=+0.572367832 container cleanup bc7d892fcb0c45c6d841ea9d3d63a8ea5808af907ea9daa9f657f16146c53bd4 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1) 2024-05-13T22:53:01.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:01 smithi130 bash[84106]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130 2024-05-13T22:53:01.914 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:01 smithi130 podman[84118]: 2024-05-13 22:53:01.685745727 +0000 UTC m=+0.617156827 container remove bc7d892fcb0c45c6d841ea9d3d63a8ea5808af907ea9daa9f657f16146c53bd4 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-05-13T22:53:02.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.115+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1238635195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c0ffbd0 msgr2=0x7fcc4c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:02.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.115+0000 7fcc50a9e700 1 --2- 172.21.15.102:0/1238635195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c0ffbd0 0x7fcc4c100050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009910 tx=0x7fcc40009c20 comp rx=0 tx=0).stop 2024-05-13T22:53:02.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.115+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1238635195 shutdown_connections 2024-05-13T22:53:02.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.115+0000 7fcc50a9e700 1 --2- 172.21.15.102:0/1238635195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c0ffbd0 0x7fcc4c100050 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.115+0000 7fcc50a9e700 1 --2- 172.21.15.102:0/1238635195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc4c105a40 0x7fcc4c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.115+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1238635195 >> 172.21.15.102:0/1238635195 conn(0x7fcc4c0755a0 msgr2=0x7fcc4c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.115+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1238635195 shutdown_connections 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.115+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1238635195 wait complete. 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc50a9e700 1 Processor -- start 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc50a9e700 1 -- start start 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc50a9e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c0ffbd0 0x7fcc4c106990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc50a9e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc4c105a40 0x7fcc4c106ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc50a9e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c107410 con 0x7fcc4c0ffbd0 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc50a9e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c107550 con 0x7fcc4c105a40 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc49d9b700 1 -- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc4c105a40 msgr2=0x7fcc4c106ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.130:3300/0 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc49d9b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc4c105a40 0x7fcc4c106ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc4a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c0ffbd0 0x7fcc4c106990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:02.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc4a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c0ffbd0 0x7fcc4c106990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48652/0 (socket says 172.21.15.102:48652) 2024-05-13T22:53:02.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc4a59c700 1 -- 172.21.15.102:0/1717202893 learned_addr learned my addr 172.21.15.102:0/1717202893 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:02.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc4a59c700 1 -- 172.21.15.102:0/1717202893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc4c105a40 msgr2=0x7fcc4c106ed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:53:02.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc4a59c700 1 --2- 172.21.15.102:0/1717202893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc4c105a40 0x7fcc4c106ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.116+0000 7fcc4a59c700 1 -- 172.21.15.102:0/1717202893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc400095f0 con 0x7fcc4c0ffbd0 2024-05-13T22:53:02.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.117+0000 7fcc4a59c700 1 --2- 172.21.15.102:0/1717202893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c0ffbd0 0x7fcc4c106990 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00b700 tx=0x7fcc3c00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:02.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.117+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1717202893 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc3c011800 con 0x7fcc4c0ffbd0 2024-05-13T22:53:02.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.117+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1717202893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4c1a29d0 con 0x7fcc4c0ffbd0 2024-05-13T22:53:02.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.117+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1717202893 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc3c011e20 con 0x7fcc4c0ffbd0 2024-05-13T22:53:02.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.117+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1717202893 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc3c00f2e0 con 0x7fcc4c0ffbd0 2024-05-13T22:53:02.097 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.117+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1717202893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c1a2de0 con 0x7fcc4c0ffbd0 2024-05-13T22:53:02.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.118+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1717202893 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c04ea90 con 0x7fcc4c0ffbd0 2024-05-13T22:53:02.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.118+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1717202893 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fcc3c010380 con 0x7fcc4c0ffbd0 2024-05-13T22:53:02.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.119+0000 7fcc3b7fe700 1 --2- 172.21.15.102:0/1717202893 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc3406e280 0x7fcc34070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:02.098 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.119+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1717202893 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fcc3c08dee0 con 0x7fcc4c0ffbd0 2024-05-13T22:53:02.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.119+0000 7fcc49d9b700 1 --2- 172.21.15.102:0/1717202893 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc3406e280 0x7fcc34070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:02.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.119+0000 7fcc49d9b700 1 --2- 172.21.15.102:0/1717202893 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc3406e280 0x7fcc34070740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000b540 tx=0x7fcc40009fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:02.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.124+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1717202893 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcc3c057160 con 0x7fcc4c0ffbd0 2024-05-13T22:53:02.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.345+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1717202893 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc4c068be0 con 0x7fcc3406e280 2024-05-13T22:53:02.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.347+0000 7fcc3b7fe700 1 -- 172.21.15.102:0/1717202893 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fcc4c068be0 con 0x7fcc3406e280 2024-05-13T22:53:02.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.349+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1717202893 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc3406e280 msgr2=0x7fcc34070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:02.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.349+0000 7fcc50a9e700 1 --2- 172.21.15.102:0/1717202893 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc3406e280 0x7fcc34070740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000b540 tx=0x7fcc40009fd0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.349+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1717202893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c0ffbd0 msgr2=0x7fcc4c106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:02.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.349+0000 7fcc50a9e700 1 --2- 172.21.15.102:0/1717202893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c0ffbd0 0x7fcc4c106990 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00b700 tx=0x7fcc3c00bac0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.349+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1717202893 shutdown_connections 2024-05-13T22:53:02.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.349+0000 7fcc50a9e700 1 --2- 172.21.15.102:0/1717202893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc4c0ffbd0 0x7fcc4c106990 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.349+0000 7fcc50a9e700 1 --2- 172.21.15.102:0/1717202893 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc3406e280 0x7fcc34070740 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.349+0000 7fcc50a9e700 1 --2- 172.21.15.102:0/1717202893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc4c105a40 0x7fcc4c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.349+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1717202893 >> 172.21.15.102:0/1717202893 conn(0x7fcc4c0755a0 msgr2=0x7fcc4c0684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:02.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.349+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1717202893 shutdown_connections 2024-05-13T22:53:02.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.349+0000 7fcc50a9e700 1 -- 172.21.15.102:0/1717202893 wait complete. 2024-05-13T22:53:02.335 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:01 smithi130 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi130.service: Deactivated successfully. 2024-05-13T22:53:02.335 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:01 smithi130 systemd[1]: Stopped Ceph mon.smithi130 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:53:02.335 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:01 smithi130 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi130.service: Consumed 7.889s CPU time. 2024-05-13T22:53:02.335 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 systemd[1]: Starting Ceph mon.smithi130 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:53:02.337 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:53:02.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.454+0000 7fef137b3700 1 -- 172.21.15.102:0/2878820354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef0c101db0 msgr2=0x7fef0c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:02.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.454+0000 7fef137b3700 1 --2- 172.21.15.102:0/2878820354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef0c101db0 0x7fef0c102190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fef08009910 tx=0x7fef08009c20 comp rx=0 tx=0).stop 2024-05-13T22:53:02.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.454+0000 7fef137b3700 1 -- 172.21.15.102:0/2878820354 shutdown_connections 2024-05-13T22:53:02.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.454+0000 7fef137b3700 1 --2- 172.21.15.102:0/2878820354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef0c1026d0 0x7fef0c10abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.454+0000 7fef137b3700 1 --2- 172.21.15.102:0/2878820354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef0c101db0 0x7fef0c102190 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.435 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.454+0000 7fef137b3700 1 -- 172.21.15.102:0/2878820354 >> 172.21.15.102:0/2878820354 conn(0x7fef0c0fb560 msgr2=0x7fef0c0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:02.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.455+0000 7fef137b3700 1 -- 172.21.15.102:0/2878820354 shutdown_connections 2024-05-13T22:53:02.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.455+0000 7fef137b3700 1 -- 172.21.15.102:0/2878820354 wait complete. 2024-05-13T22:53:02.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.455+0000 7fef137b3700 1 Processor -- start 2024-05-13T22:53:02.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.455+0000 7fef137b3700 1 -- start start 2024-05-13T22:53:02.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.455+0000 7fef137b3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef0c101db0 0x7fef0c0ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:02.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.455+0000 7fef137b3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef0c1026d0 0x7fef0c0fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:02.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.455+0000 7fef137b3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c100490 con 0x7fef0c101db0 2024-05-13T22:53:02.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.455+0000 7fef137b3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c1005d0 con 0x7fef0c1026d0 2024-05-13T22:53:02.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.455+0000 7fef1154f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef0c101db0 0x7fef0c0ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:02.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.455+0000 7fef1154f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef0c101db0 0x7fef0c0ffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48676/0 (socket says 172.21.15.102:48676) 2024-05-13T22:53:02.436 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.455+0000 7fef1154f700 1 -- 172.21.15.102:0/2273166731 learned_addr learned my addr 172.21.15.102:0/2273166731 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.456+0000 7fef10d4e700 1 -- 172.21.15.102:0/2273166731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef0c1026d0 msgr2=0x7fef0c0fff50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.130:3300/0 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.456+0000 7fef10d4e700 1 --2- 172.21.15.102:0/2273166731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef0c1026d0 0x7fef0c0fff50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.456+0000 7fef1154f700 1 -- 172.21.15.102:0/2273166731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef0c1026d0 msgr2=0x7fef0c0fff50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.456+0000 7fef1154f700 1 --2- 172.21.15.102:0/2273166731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef0c1026d0 0x7fef0c0fff50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.456+0000 7fef1154f700 1 -- 172.21.15.102:0/2273166731 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef080095f0 con 0x7fef0c101db0 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.456+0000 7fef1154f700 1 --2- 172.21.15.102:0/2273166731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef0c101db0 0x7fef0c0ffa10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fef08005b40 tx=0x7fef0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.456+0000 7fef027fc700 1 -- 172.21.15.102:0/2273166731 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef0801d070 con 0x7fef0c101db0 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.456+0000 7fef137b3700 1 -- 172.21.15.102:0/2273166731 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0c100850 con 0x7fef0c101db0 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.456+0000 7fef027fc700 1 -- 172.21.15.102:0/2273166731 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef080048e0 con 0x7fef0c101db0 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.456+0000 7fef027fc700 1 -- 172.21.15.102:0/2273166731 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef08003b80 con 0x7fef0c101db0 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.456+0000 7fef137b3700 1 -- 172.21.15.102:0/2273166731 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c1a2ce0 con 0x7fef0c101db0 2024-05-13T22:53:02.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.457+0000 7fef137b3700 1 -- 172.21.15.102:0/2273166731 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0c108290 con 0x7fef0c101db0 2024-05-13T22:53:02.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.457+0000 7fef027fc700 1 -- 172.21.15.102:0/2273166731 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fef08004a50 con 0x7fef0c101db0 2024-05-13T22:53:02.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.458+0000 7fef027fc700 1 --2- 172.21.15.102:0/2273166731 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7feef806e280 0x7feef8070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:02.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.458+0000 7fef027fc700 1 -- 172.21.15.102:0/2273166731 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fef0808f940 con 0x7fef0c101db0 2024-05-13T22:53:02.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.458+0000 7fef10d4e700 1 --2- 172.21.15.102:0/2273166731 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7feef806e280 0x7feef8070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:02.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.458+0000 7fef10d4e700 1 --2- 172.21.15.102:0/2273166731 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7feef806e280 0x7feef8070740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fef0c101200 tx=0x7feefc006df0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:02.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.463+0000 7fef027fc700 1 -- 172.21.15.102:0/2273166731 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fef08058ab0 con 0x7fef0c101db0 2024-05-13T22:53:02.582 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 podman[84246]: 2024-05-13 22:53:02.261295209 +0000 UTC m=+0.017237421 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:53:02.583 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 podman[84246]: 2024-05-13 22:53:02.373718782 +0000 UTC m=+0.129660967 container create 2c1e5923909a1b2fa2ffd8d1fa02f8f5dbef15cb23246814ff80e64c4e3af358 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-13T22:53:02.583 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 podman[84246]: 2024-05-13 22:53:02.57253457 +0000 UTC m=+0.328476764 container init 2c1e5923909a1b2fa2ffd8d1fa02f8f5dbef15cb23246814ff80e64c4e3af358 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-13T22:53:02.583 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 podman[84246]: 2024-05-13 22:53:02.575628966 +0000 UTC m=+0.331571164 container start 2c1e5923909a1b2fa2ffd8d1fa02f8f5dbef15cb23246814ff80e64c4e3af358 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-c3e5514, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-13T22:53:02.583 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: set uid:gid to 167:167 (ceph:ceph) 2024-05-13T22:53:02.583 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev), process ceph-mon, pid 2 2024-05-13T22:53:02.583 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: pidfile_write: ignore empty --pid-file 2024-05-13T22:53:02.583 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: load: jerasure load: lrc 2024-05-13T22:53:02.583 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: RocksDB version: 7.9.2 2024-05-13T22:53:02.583 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Git sha 0 2024-05-13T22:53:02.583 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Compile date 2024-05-13 16:38:45 2024-05-13T22:53:02.584 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: DB SUMMARY 2024-05-13T22:53:02.584 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: DB Session ID: CCJGRF28FK0KEPHC2I87 2024-05-13T22:53:02.584 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: CURRENT file: CURRENT 2024-05-13T22:53:02.584 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: IDENTITY file: IDENTITY 2024-05-13T22:53:02.584 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1403 Bytes 2024-05-13T22:53:02.584 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi130/store.db dir, Total Num: 1, files: 000027.sst 2024-05-13T22:53:02.584 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi130/store.db: 000025.log size: 8345951 ; 2024-05-13T22:53:02.584 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.error_if_exists: 0 2024-05-13T22:53:02.585 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.create_if_missing: 0 2024-05-13T22:53:02.585 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.paranoid_checks: 1 2024-05-13T22:53:02.585 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-13T22:53:02.586 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-13T22:53:02.586 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-13T22:53:02.586 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.env: 0x55b61ee67c60 2024-05-13T22:53:02.586 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.fs: PosixFileSystem 2024-05-13T22:53:02.586 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.info_log: 0x55b61fa19980 2024-05-13T22:53:02.586 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_file_opening_threads: 16 2024-05-13T22:53:02.586 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.statistics: (nil) 2024-05-13T22:53:02.586 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.use_fsync: 0 2024-05-13T22:53:02.586 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_log_file_size: 0 2024-05-13T22:53:02.586 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-13T22:53:02.586 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.keep_log_file_num: 1000 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.recycle_log_file_num: 0 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.allow_fallocate: 1 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.allow_mmap_reads: 0 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.allow_mmap_writes: 0 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.use_direct_reads: 0 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.create_missing_column_families: 0 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.db_log_dir: 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.wal_dir: 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-13T22:53:02.587 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.advise_random_on_open: 1 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.db_write_buffer_size: 0 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.write_buffer_manager: 0x55b61f9ec1e0 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.rate_limiter: (nil) 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.wal_recovery_mode: 2 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.enable_thread_tracking: 0 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.enable_pipelined_write: 0 2024-05-13T22:53:02.588 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.unordered_write: 0 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.row_cache: None 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.wal_filter: None 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.allow_ingest_behind: 0 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.two_write_queues: 0 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.manual_wal_flush: 0 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.wal_compression: 0 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.atomic_flush: 0 2024-05-13T22:53:02.589 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.log_readahead_size: 0 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.best_efforts_recovery: 0 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.allow_data_in_errors: 0 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.db_host_id: __hostname__ 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_background_jobs: 2 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_background_compactions: -1 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_subcompactions: 1 2024-05-13T22:53:02.590 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_total_wal_size: 0 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_open_files: -1 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bytes_per_sync: 0 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-13T22:53:02.591 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_readahead_size: 0 2024-05-13T22:53:02.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_background_flushes: -1 2024-05-13T22:53:02.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Compression algorithms supported: 2024-05-13T22:53:02.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-13T22:53:02.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: kZSTD supported: 0 2024-05-13T22:53:02.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: kXpressCompression supported: 0 2024-05-13T22:53:02.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: kLZ4HCCompression supported: 1 2024-05-13T22:53:02.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: kZlibCompression supported: 1 2024-05-13T22:53:02.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: kSnappyCompression supported: 1 2024-05-13T22:53:02.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: kLZ4Compression supported: 1 2024-05-13T22:53:02.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: kBZip2Compression supported: 0 2024-05-13T22:53:02.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi130/store.db/MANIFEST-000010 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.merge_operator: 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_filter: None 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_filter_factory: None 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.sst_partitioner_factory: None 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b61fa19a80) 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: cache_index_and_filter_blocks: 1 2024-05-13T22:53:02.593 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: pin_top_level_index_and_filter: 1 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: index_type: 0 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: data_block_index_type: 0 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: index_shortening: 1 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: checksum: 4 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: no_block_cache: 0 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: block_cache: 0x55b61fa1d090 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: block_cache_name: BinnedLRUCache 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: block_cache_options: 2024-05-13T22:53:02.594 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: capacity : 536870912 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: num_shard_bits : 4 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: strict_capacity_limit : 0 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: high_pri_pool_ratio: 0.000 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: block_cache_compressed: (nil) 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: persistent_cache: (nil) 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: block_size: 4096 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: block_size_deviation: 10 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: block_restart_interval: 16 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: index_block_restart_interval: 1 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: metadata_block_size: 4096 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: partition_filters: 0 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: use_delta_encoding: 1 2024-05-13T22:53:02.595 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: filter_policy: bloomfilter 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: whole_key_filtering: 1 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: verify_compression: 0 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: read_amp_bytes_per_bit: 0 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: format_version: 5 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: enable_index_compression: 1 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: block_align: 0 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: max_auto_readahead_size: 262144 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: prepopulate_block_cache: 0 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: initial_auto_readahead_size: 8192 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: num_file_reads_for_auto_readahead: 2 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.write_buffer_size: 33554432 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_write_buffer_number: 2 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compression: NoCompression 2024-05-13T22:53:02.596 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bottommost_compression: Disabled 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.prefix_extractor: nullptr 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.num_levels: 7 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-13T22:53:02.597 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compression_opts.level: 32767 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compression_opts.strategy: 0 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compression_opts.enabled: false 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-13T22:53:02.598 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.target_file_size_base: 67108864 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-13T22:53:02.599 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.arena_block_size: 1048576 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.disable_auto_compactions: 0 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-13T22:53:02.600 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-13T22:53:02.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.table_properties_collectors: 2024-05-13T22:53:02.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.inplace_update_support: 0 2024-05-13T22:53:02.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-13T22:53:02.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-13T22:53:02.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-13T22:53:02.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-13T22:53:02.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.bloom_locality: 0 2024-05-13T22:53:02.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.max_successive_merges: 0 2024-05-13T22:53:02.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-13T22:53:02.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.paranoid_file_checks: 0 2024-05-13T22:53:02.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.force_consistency_checks: 1 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.report_bg_io_stats: 0 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.ttl: 2592000 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.enable_blob_files: false 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.min_blob_size: 0 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.blob_file_size: 268435456 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-13T22:53:02.602 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.blob_file_starting_level: 0 2024-05-13T22:53:02.603 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-13T22:53:02.603 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-13T22:53:02.603 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi130/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12151, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-05-13T22:53:02.603 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-05-13T22:53:02.603 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e5080f94-1e7b-4a3b-b6bd-ed7f0cee5f09 2024-05-13T22:53:02.603 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715640782605340, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-05-13T22:53:02.603 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-05-13T22:53:02.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.680+0000 7fef137b3700 1 -- 172.21.15.102:0/2273166731 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef0c02cce0 con 0x7feef806e280 2024-05-13T22:53:02.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.681+0000 7fef027fc700 1 -- 172.21.15.102:0/2273166731 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fef0c02cce0 con 0x7feef806e280 2024-05-13T22:53:02.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.684+0000 7fef137b3700 1 -- 172.21.15.102:0/2273166731 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7feef806e280 msgr2=0x7feef8070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:02.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.684+0000 7fef137b3700 1 --2- 172.21.15.102:0/2273166731 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7feef806e280 0x7feef8070740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fef0c101200 tx=0x7feefc006df0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.684+0000 7fef137b3700 1 -- 172.21.15.102:0/2273166731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef0c101db0 msgr2=0x7fef0c0ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:02.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.684+0000 7fef137b3700 1 --2- 172.21.15.102:0/2273166731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef0c101db0 0x7fef0c0ffa10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fef08005b40 tx=0x7fef0800be20 comp rx=0 tx=0).stop 2024-05-13T22:53:02.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.684+0000 7fef137b3700 1 -- 172.21.15.102:0/2273166731 shutdown_connections 2024-05-13T22:53:02.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.684+0000 7fef137b3700 1 --2- 172.21.15.102:0/2273166731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef0c101db0 0x7fef0c0ffa10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.684+0000 7fef137b3700 1 --2- 172.21.15.102:0/2273166731 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7feef806e280 0x7feef8070740 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.684+0000 7fef137b3700 1 --2- 172.21.15.102:0/2273166731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef0c1026d0 0x7fef0c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.684+0000 7fef137b3700 1 -- 172.21.15.102:0/2273166731 >> 172.21.15.102:0/2273166731 conn(0x7fef0c0fb560 msgr2=0x7fef0c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:02.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.684+0000 7fef137b3700 1 -- 172.21.15.102:0/2273166731 shutdown_connections 2024-05-13T22:53:02.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.684+0000 7fef137b3700 1 -- 172.21.15.102:0/2273166731 wait complete. 2024-05-13T22:53:02.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.790+0000 7f4f4dc08700 1 -- 172.21.15.102:0/2799215571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f48100dc0 msgr2=0x7f4f481011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:02.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.790+0000 7f4f4dc08700 1 --2- 172.21.15.102:0/2799215571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f48100dc0 0x7f4f481011a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009910 tx=0x7f4f30009c20 comp rx=0 tx=0).stop 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.790+0000 7f4f4dc08700 1 -- 172.21.15.102:0/2799215571 shutdown_connections 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.790+0000 7f4f4dc08700 1 --2- 172.21.15.102:0/2799215571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f48101770 0x7f4f481057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.790+0000 7f4f4dc08700 1 --2- 172.21.15.102:0/2799215571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f48100dc0 0x7f4f481011a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.790+0000 7f4f4dc08700 1 -- 172.21.15.102:0/2799215571 >> 172.21.15.102:0/2799215571 conn(0x7f4f480fc650 msgr2=0x7f4f480fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.790+0000 7f4f4dc08700 1 -- 172.21.15.102:0/2799215571 shutdown_connections 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.790+0000 7f4f4dc08700 1 -- 172.21.15.102:0/2799215571 wait complete. 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.791+0000 7f4f4dc08700 1 Processor -- start 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.791+0000 7f4f4dc08700 1 -- start start 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.791+0000 7f4f4dc08700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f48100dc0 0x7f4f4819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.791+0000 7f4f4dc08700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f48101770 0x7f4f4819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.791+0000 7f4f4dc08700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4819d210 con 0x7f4f48100dc0 2024-05-13T22:53:02.771 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.791+0000 7f4f4dc08700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f481966c0 con 0x7f4f48101770 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.791+0000 7f4f477fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f48100dc0 0x7f4f4819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.791+0000 7f4f477fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f48100dc0 0x7f4f4819c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48688/0 (socket says 172.21.15.102:48688) 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.791+0000 7f4f477fe700 1 -- 172.21.15.102:0/181954319 learned_addr learned my addr 172.21.15.102:0/181954319 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.791+0000 7f4f46ffd700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f48101770 0x7f4f4819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f46ffd700 1 -- 172.21.15.102:0/181954319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f48101770 msgr2=0x7f4f4819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f46ffd700 1 -- 172.21.15.102:0/181954319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f48101770 msgr2=0x7f4f4819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f46ffd700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f48101770 0x7f4f4819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f477fe700 1 -- 172.21.15.102:0/181954319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f48101770 msgr2=0x7f4f4819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f46ffd700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f48101770 0x7f4f4819cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f477fe700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f48101770 0x7f4f4819cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f477fe700 1 -- 172.21.15.102:0/181954319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f300095f0 con 0x7f4f48100dc0 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f477fe700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f48100dc0 0x7f4f4819c640 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4f30006010 tx=0x7f4f3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:02.772 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f44ff9700 1 -- 172.21.15.102:0/181954319 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f3001d070 con 0x7f4f48100dc0 2024-05-13T22:53:02.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f4dc08700 1 -- 172.21.15.102:0/181954319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f48196940 con 0x7f4f48100dc0 2024-05-13T22:53:02.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f44ff9700 1 -- 172.21.15.102:0/181954319 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f30004fb0 con 0x7f4f48100dc0 2024-05-13T22:53:02.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f44ff9700 1 -- 172.21.15.102:0/181954319 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f30022ea0 con 0x7f4f48100dc0 2024-05-13T22:53:02.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.792+0000 7f4f4dc08700 1 -- 172.21.15.102:0/181954319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f48196e30 con 0x7f4f48100dc0 2024-05-13T22:53:02.773 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.793+0000 7f4f4dc08700 1 -- 172.21.15.102:0/181954319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f48109160 con 0x7f4f48100dc0 2024-05-13T22:53:02.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.794+0000 7f4f44ff9700 1 -- 172.21.15.102:0/181954319 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f4f30022470 con 0x7f4f48100dc0 2024-05-13T22:53:02.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.794+0000 7f4f44ff9700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4f3406e340 0x7f4f34070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:02.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.795+0000 7f4f44ff9700 1 -- 172.21.15.102:0/181954319 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4f3008f880 con 0x7f4f48100dc0 2024-05-13T22:53:02.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.795+0000 7f4f46ffd700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4f3406e340 0x7f4f34070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:02.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.795+0000 7f4f46ffd700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4f3406e340 0x7f4f34070800 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4f38006fd0 tx=0x7f4f38008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:02.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:02.800+0000 7f4f44ff9700 1 -- 172.21.15.102:0/181954319 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4f3005c5d0 con 0x7f4f48100dc0 2024-05-13T22:53:02.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715640782626458, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4957011, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12156, "largest_seqno": 16299, "table_properties": {"data_size": 4942347, "index_size": 9483, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4165, "raw_key_size": 36684, "raw_average_key_size": 22, "raw_value_size": 4909308, "raw_average_value_size": 2955, "num_data_blocks": 429, "num_entries": 1661, "num_filter_entries": 1661, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715640782, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e5080f94-1e7b-4a3b-b6bd-ed7f0cee5f09", "db_session_id": "CCJGRF28FK0KEPHC2I87", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715640782626565, "job": 1, "event": "recovery_finished"} 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi130/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b61fb0e000 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: DB pointer 0x55b61faf8000 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: ** DB Stats ** 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-13T22:53:02.901 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: ** Compaction Stats [default] ** 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: L0 1/0 4.73 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 540.1 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: L6 1/0 5.80 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Sum 2/0 10.53 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 540.1 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 540.1 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: ** Compaction Stats [default] ** 2024-05-13T22:53:02.902 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 540.1 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Cumulative compaction: 0.00 GB write, 186.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Interval compaction: 0.00 GB write, 186.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-13T22:53:02.903 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Block cache BinnedLRUCache@0x55b61fa1d090#2 capacity: 512.00 MB usage: 14.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,10.08 KB,0.00192225%) Misc(1,0.00 KB,0%) 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: starting mon.smithi130 rank 1 at public addrs [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] at bind addrs [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi130 fsid 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: mon.smithi130@-1(???) e2 preinit fsid 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: mon.smithi130@-1(???).mds e17 new map 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: mon.smithi130@-1(???).mds e17 print_map 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: e17 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: legacy client fscid: 1 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Filesystem 'cephfs' (1) 2024-05-13T22:53:02.904 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: fs_name cephfs 2024-05-13T22:53:02.905 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: epoch 17 2024-05-13T22:53:02.905 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:53:02.905 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:53:02.905 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:53:02.905 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: tableserver 0 2024-05-13T22:53:02.905 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: root 0 2024-05-13T22:53:02.905 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: session_timeout 60 2024-05-13T22:53:02.905 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: session_autoclose 300 2024-05-13T22:53:02.905 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: max_file_size 1099511627776 2024-05-13T22:53:02.905 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: max_xattr_size 65536 2024-05-13T22:53:02.905 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: required_client_features {} 2024-05-13T22:53:02.906 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: last_failure 0 2024-05-13T22:53:02.906 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: last_failure_osd_epoch 47 2024-05-13T22:53:02.906 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:53:02.906 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: max_mds 1 2024-05-13T22:53:02.906 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: in 0 2024-05-13T22:53:02.906 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: up {0=14498} 2024-05-13T22:53:02.906 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: failed 2024-05-13T22:53:02.906 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: damaged 2024-05-13T22:53:02.906 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: stopped 2024-05-13T22:53:02.906 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: data_pools [3] 2024-05-13T22:53:02.906 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: metadata_pool 2 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: inline_data disabled 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: balancer 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: bal_rank_mask -1 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: standby_count_wanted 1 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: qdb_cluster leader: 0 members: 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: [mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: [mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: Standby daemons: 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: [mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout: [mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:02.907 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: mon.smithi130@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-13T22:53:02.908 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: mon.smithi130@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-13T22:53:02.908 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: mon.smithi130@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-13T22:53:02.908 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: mon.smithi130@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-05-13T22:53:02.908 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 ceph-mon[84265]: mon.smithi130@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-05-13T22:53:02.908 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 bash[84246]: 2c1e5923909a1b2fa2ffd8d1fa02f8f5dbef15cb23246814ff80e64c4e3af358 2024-05-13T22:53:02.908 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:02 smithi130 systemd[1]: Started Ceph mon.smithi130 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:53:02.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.005+0000 7f4f4dc08700 1 -- 172.21.15.102:0/181954319 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f4802cce0 con 0x7f4f3406e340 2024-05-13T22:53:02.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.016+0000 7f4f44ff9700 1 -- 172.21.15.102:0/181954319 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f4f4802cce0 con 0x7f4f3406e340 2024-05-13T22:53:02.994 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:53:02.994 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (2m) 11s ago 13m 26.7M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:53:02.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (13m) 11s ago 13m 8162k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:53:02.994 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (12m) 13s ago 12m 7872k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:53:02.994 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (13m) 11s ago 13m 6660k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:53:02.994 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (12m) 13s ago 12m 6669k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:53:02.994 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (108s) 11s ago 12m 75.0M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:53:02.994 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (7m) 11s ago 7m 88.4M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:53:02.994 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (7m) 11s ago 7m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:53:02.994 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (7m) 13s ago 7m 15.5M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:53:02.994 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (7m) 13s ago 7m 18.5M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:53:02.995 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (4m) 11s ago 14m 475M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:53:02.995 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (3m) 13s ago 12m 449M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:53:02.995 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (27s) 11s ago 14m 31.7M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:53:02.995 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 starting - - - 2048M 2024-05-13T22:53:02.995 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (3m) 11s ago 13m 16.0M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:53:02.995 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (2m) 13s ago 12m 16.6M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:53:02.995 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (11m) 11s ago 11m 224M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:53:02.995 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (10m) 11s ago 10m 239M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:53:02.995 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (10m) 11s ago 10m 203M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:53:02.995 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (10m) 13s ago 10m 291M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:53:02.995 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (9m) 13s ago 9m 246M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:53:02.996 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (9m) 13s ago 9m 227M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:53:02.996 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (2m) 11s ago 12m 67.2M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:53:02.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.018+0000 7f4f4dc08700 1 -- 172.21.15.102:0/181954319 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4f3406e340 msgr2=0x7f4f34070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:02.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.018+0000 7f4f4dc08700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4f3406e340 0x7f4f34070800 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4f38006fd0 tx=0x7f4f38008040 comp rx=0 tx=0).stop 2024-05-13T22:53:02.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.018+0000 7f4f4dc08700 1 -- 172.21.15.102:0/181954319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f48100dc0 msgr2=0x7f4f4819c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:02.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.018+0000 7f4f4dc08700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f48100dc0 0x7f4f4819c640 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4f30006010 tx=0x7f4f3000be20 comp rx=0 tx=0).stop 2024-05-13T22:53:02.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.018+0000 7f4f4dc08700 1 -- 172.21.15.102:0/181954319 shutdown_connections 2024-05-13T22:53:02.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.018+0000 7f4f4dc08700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f48100dc0 0x7f4f4819c640 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.018+0000 7f4f4dc08700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4f3406e340 0x7f4f34070800 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.018+0000 7f4f4dc08700 1 --2- 172.21.15.102:0/181954319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f48101770 0x7f4f4819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:02.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.018+0000 7f4f4dc08700 1 -- 172.21.15.102:0/181954319 >> 172.21.15.102:0/181954319 conn(0x7f4f480fc650 msgr2=0x7f4f480fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:02.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.018+0000 7f4f4dc08700 1 -- 172.21.15.102:0/181954319 shutdown_connections 2024-05-13T22:53:02.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.018+0000 7f4f4dc08700 1 -- 172.21.15.102:0/181954319 wait complete. 2024-05-13T22:53:03.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.121+0000 7f8f698c4700 1 -- 172.21.15.102:0/1656909667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f64101da0 msgr2=0x7f8f64102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:03.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.121+0000 7f8f698c4700 1 --2- 172.21.15.102:0/1656909667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f64101da0 0x7f8f64102180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00b0a0 tx=0x7f8f5c00b3b0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.121+0000 7f8f698c4700 1 -- 172.21.15.102:0/1656909667 shutdown_connections 2024-05-13T22:53:03.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.121+0000 7f8f698c4700 1 --2- 172.21.15.102:0/1656909667 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f641026c0 0x7f8f6410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.121+0000 7f8f698c4700 1 --2- 172.21.15.102:0/1656909667 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f64101da0 0x7f8f64102180 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.121+0000 7f8f698c4700 1 -- 172.21.15.102:0/1656909667 >> 172.21.15.102:0/1656909667 conn(0x7f8f640fb550 msgr2=0x7f8f640fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:03.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.121+0000 7f8f698c4700 1 -- 172.21.15.102:0/1656909667 shutdown_connections 2024-05-13T22:53:03.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.121+0000 7f8f698c4700 1 -- 172.21.15.102:0/1656909667 wait complete. 2024-05-13T22:53:03.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.122+0000 7f8f698c4700 1 Processor -- start 2024-05-13T22:53:03.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.122+0000 7f8f698c4700 1 -- start start 2024-05-13T22:53:03.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.122+0000 7f8f698c4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f64101da0 0x7f8f640ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:03.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.122+0000 7f8f698c4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f641026c0 0x7f8f640fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.122+0000 7f8f698c4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64100480 con 0x7f8f641026c0 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.122+0000 7f8f698c4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f641005c0 con 0x7f8f64101da0 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.122+0000 7f8f627fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f641026c0 0x7f8f640fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.122+0000 7f8f627fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f641026c0 0x7f8f640fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48710/0 (socket says 172.21.15.102:48710) 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.122+0000 7f8f627fc700 1 -- 172.21.15.102:0/436209134 learned_addr learned my addr 172.21.15.102:0/436209134 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.123+0000 7f8f627fc700 1 -- 172.21.15.102:0/436209134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f64101da0 msgr2=0x7f8f640ffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.123+0000 7f8f62ffd700 1 --2- 172.21.15.102:0/436209134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f64101da0 0x7f8f640ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.123+0000 7f8f627fc700 1 --2- 172.21.15.102:0/436209134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f64101da0 0x7f8f640ffa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.123+0000 7f8f627fc700 1 -- 172.21.15.102:0/436209134 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c009d00 con 0x7f8f641026c0 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.123+0000 7f8f627fc700 1 --2- 172.21.15.102:0/436209134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f641026c0 0x7f8f640fff40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00d900 tx=0x7f8f4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.123+0000 7f8f688c2700 1 -- 172.21.15.102:0/436209134 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f4c009900 con 0x7f8f641026c0 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.123+0000 7f8f688c2700 1 -- 172.21.15.102:0/436209134 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f4c010460 con 0x7f8f641026c0 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.123+0000 7f8f698c4700 1 -- 172.21.15.102:0/436209134 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f641a28f0 con 0x7f8f641026c0 2024-05-13T22:53:03.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.123+0000 7f8f688c2700 1 -- 172.21.15.102:0/436209134 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f4c00f5f0 con 0x7f8f641026c0 2024-05-13T22:53:03.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.123+0000 7f8f698c4700 1 -- 172.21.15.102:0/436209134 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f641a2d00 con 0x7f8f641026c0 2024-05-13T22:53:03.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.124+0000 7f8f698c4700 1 -- 172.21.15.102:0/436209134 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f641082b0 con 0x7f8f641026c0 2024-05-13T22:53:03.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.125+0000 7f8f688c2700 1 -- 172.21.15.102:0/436209134 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f8f4c0105d0 con 0x7f8f641026c0 2024-05-13T22:53:03.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.125+0000 7f8f688c2700 1 --2- 172.21.15.102:0/436209134 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f5006e340 0x7f8f50070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:03.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.125+0000 7f8f688c2700 1 -- 172.21.15.102:0/436209134 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8f4c08e140 con 0x7f8f641026c0 2024-05-13T22:53:03.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.125+0000 7f8f62ffd700 1 --2- 172.21.15.102:0/436209134 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f5006e340 0x7f8f50070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:03.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.126+0000 7f8f62ffd700 1 --2- 172.21.15.102:0/436209134 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f5006e340 0x7f8f50070800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00b830 tx=0x7f8f5c006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:03.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.130+0000 7f8f688c2700 1 -- 172.21.15.102:0/436209134 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8f4c057330 con 0x7f8f641026c0 2024-05-13T22:53:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.438+0000 7f8f698c4700 1 -- 172.21.15.102:0/436209134 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8f6404ea90 con 0x7f8f641026c0 2024-05-13T22:53:03.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.438+0000 7f8f688c2700 1 -- 172.21.15.102:0/436209134 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+779 (secure 0 0 0) 0x7f8f4c05a9f0 con 0x7f8f641026c0 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 1 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:53:03.417 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:53:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:53:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:53:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 11, 2024-05-13T22:53:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 3 2024-05-13T22:53:03.418 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:53:03.418 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:53:03.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.440+0000 7f8f698c4700 1 -- 172.21.15.102:0/436209134 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f5006e340 msgr2=0x7f8f50070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:03.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.440+0000 7f8f698c4700 1 --2- 172.21.15.102:0/436209134 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f5006e340 0x7f8f50070800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00b830 tx=0x7f8f5c006000 comp rx=0 tx=0).stop 2024-05-13T22:53:03.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.440+0000 7f8f698c4700 1 -- 172.21.15.102:0/436209134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f641026c0 msgr2=0x7f8f640fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:03.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.440+0000 7f8f698c4700 1 --2- 172.21.15.102:0/436209134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f641026c0 0x7f8f640fff40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00d900 tx=0x7f8f4c00dcc0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.440+0000 7f8f698c4700 1 -- 172.21.15.102:0/436209134 shutdown_connections 2024-05-13T22:53:03.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.440+0000 7f8f698c4700 1 --2- 172.21.15.102:0/436209134 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f5006e340 0x7f8f50070800 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.441+0000 7f8f698c4700 1 --2- 172.21.15.102:0/436209134 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f64101da0 0x7f8f640ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.441+0000 7f8f698c4700 1 --2- 172.21.15.102:0/436209134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f641026c0 0x7f8f640fff40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.441+0000 7f8f698c4700 1 -- 172.21.15.102:0/436209134 >> 172.21.15.102:0/436209134 conn(0x7f8f640fb550 msgr2=0x7f8f641053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:03.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.441+0000 7f8f698c4700 1 -- 172.21.15.102:0/436209134 shutdown_connections 2024-05-13T22:53:03.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.441+0000 7f8f698c4700 1 -- 172.21.15.102:0/436209134 wait complete. 2024-05-13T22:53:03.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.546+0000 7f719a81d700 1 -- 172.21.15.102:0/3939603725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71940ffbd0 msgr2=0x7f7194100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:03.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.546+0000 7f719a81d700 1 --2- 172.21.15.102:0/3939603725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71940ffbd0 0x7f7194100050 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f717c009910 tx=0x7f717c009c20 comp rx=0 tx=0).stop 2024-05-13T22:53:03.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.546+0000 7f719a81d700 1 -- 172.21.15.102:0/3939603725 shutdown_connections 2024-05-13T22:53:03.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.546+0000 7f719a81d700 1 --2- 172.21.15.102:0/3939603725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71940ffbd0 0x7f7194100050 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.546+0000 7f719a81d700 1 --2- 172.21.15.102:0/3939603725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7194105a40 0x7f7194105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.546+0000 7f719a81d700 1 -- 172.21.15.102:0/3939603725 >> 172.21.15.102:0/3939603725 conn(0x7f71940755a0 msgr2=0x7f71940759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:03.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.546+0000 7f719a81d700 1 -- 172.21.15.102:0/3939603725 shutdown_connections 2024-05-13T22:53:03.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.546+0000 7f719a81d700 1 -- 172.21.15.102:0/3939603725 wait complete. 2024-05-13T22:53:03.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.547+0000 7f719a81d700 1 Processor -- start 2024-05-13T22:53:03.526 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.547+0000 7f719a81d700 1 -- start start 2024-05-13T22:53:03.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.547+0000 7f719a81d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71940ffbd0 0x7f7194197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:03.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.547+0000 7f719a81d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7194105a40 0x7f7194197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:03.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.547+0000 7f719a81d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71941980c0 con 0x7f71940ffbd0 2024-05-13T22:53:03.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.547+0000 7f719a81d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7194198200 con 0x7f7194105a40 2024-05-13T22:53:03.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.547+0000 7f7193fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71940ffbd0 0x7f7194197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:03.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.547+0000 7f7193fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71940ffbd0 0x7f7194197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48726/0 (socket says 172.21.15.102:48726) 2024-05-13T22:53:03.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.547+0000 7f7193fff700 1 -- 172.21.15.102:0/3784497907 learned_addr learned my addr 172.21.15.102:0/3784497907 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:03.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.547+0000 7f71937fe700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7194105a40 0x7f7194197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f71937fe700 1 -- 172.21.15.102:0/3784497907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7194105a40 msgr2=0x7f7194197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f71937fe700 1 -- 172.21.15.102:0/3784497907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7194105a40 msgr2=0x7f7194197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f71937fe700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7194105a40 0x7f7194197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f71937fe700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7194105a40 0x7f7194197aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f7193fff700 1 -- 172.21.15.102:0/3784497907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7194105a40 msgr2=0x7f7194197aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f7193fff700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7194105a40 0x7f7194197aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f7193fff700 1 -- 172.21.15.102:0/3784497907 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0095f0 con 0x7f71940ffbd0 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f7193fff700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71940ffbd0 0x7f7194197560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f718400d8d0 tx=0x7f718400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f71917fa700 1 -- 172.21.15.102:0/3784497907 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7184004a00 con 0x7f71940ffbd0 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f71917fa700 1 -- 172.21.15.102:0/3784497907 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71840054e0 con 0x7f71940ffbd0 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f719a81d700 1 -- 172.21.15.102:0/3784497907 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71941a05d0 con 0x7f71940ffbd0 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f71917fa700 1 -- 172.21.15.102:0/3784497907 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7184009d60 con 0x7f71940ffbd0 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.548+0000 7f719a81d700 1 -- 172.21.15.102:0/3784497907 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71941a0a90 con 0x7f71940ffbd0 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.549+0000 7f719a81d700 1 -- 172.21.15.102:0/3784497907 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7194107fd0 con 0x7f71940ffbd0 2024-05-13T22:53:03.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.550+0000 7f71917fa700 1 -- 172.21.15.102:0/3784497907 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f7184004f50 con 0x7f71940ffbd0 2024-05-13T22:53:03.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.550+0000 7f71917fa700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f718006e340 0x7f7180070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:03.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.550+0000 7f71917fa700 1 -- 172.21.15.102:0/3784497907 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f718408df90 con 0x7f71940ffbd0 2024-05-13T22:53:03.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.550+0000 7f71937fe700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f718006e340 0x7f7180070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:03.530 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.551+0000 7f71937fe700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f718006e340 0x7f7180070800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7194068cc0 tx=0x7f717c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:03.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.555+0000 7f71917fa700 1 -- 172.21.15.102:0/3784497907 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7184057100 con 0x7f71940ffbd0 2024-05-13T22:53:03.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.795+0000 7f719a81d700 1 -- 172.21.15.102:0/3784497907 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f719402cca0 con 0x7f71940ffbd0 2024-05-13T22:53:03.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.796+0000 7f71917fa700 1 -- 172.21.15.102:0/3784497907 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1909 (secure 0 0 0) 0x7f718401f790 con 0x7f71940ffbd0 2024-05-13T22:53:03.774 INFO:teuthology.orchestra.run.smithi102.stdout:e17 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 17 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:53:03.775 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 47 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=14498} 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:53:03.776 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 0 members: 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:53:03.777 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:03.778 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:03.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.798+0000 7f719a81d700 1 -- 172.21.15.102:0/3784497907 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f718006e340 msgr2=0x7f7180070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:03.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.798+0000 7f719a81d700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f718006e340 0x7f7180070800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7194068cc0 tx=0x7f717c01a040 comp rx=0 tx=0).stop 2024-05-13T22:53:03.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.798+0000 7f719a81d700 1 -- 172.21.15.102:0/3784497907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71940ffbd0 msgr2=0x7f7194197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:03.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.798+0000 7f719a81d700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71940ffbd0 0x7f7194197560 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f718400d8d0 tx=0x7f718400dc90 comp rx=0 tx=0).stop 2024-05-13T22:53:03.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.798+0000 7f719a81d700 1 -- 172.21.15.102:0/3784497907 shutdown_connections 2024-05-13T22:53:03.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.798+0000 7f719a81d700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71940ffbd0 0x7f7194197560 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.798+0000 7f719a81d700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f718006e340 0x7f7180070800 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.798+0000 7f719a81d700 1 --2- 172.21.15.102:0/3784497907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7194105a40 0x7f7194197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.798+0000 7f719a81d700 1 -- 172.21.15.102:0/3784497907 >> 172.21.15.102:0/3784497907 conn(0x7f71940755a0 msgr2=0x7f71940feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:03.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.799+0000 7f719a81d700 1 -- 172.21.15.102:0/3784497907 shutdown_connections 2024-05-13T22:53:03.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.799+0000 7f719a81d700 1 -- 172.21.15.102:0/3784497907 wait complete. 2024-05-13T22:53:03.779 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:53:03.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.903+0000 7fbe1be4e700 1 -- 172.21.15.102:0/50960344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe14073aa0 msgr2=0x7fbe1410ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:03.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.903+0000 7fbe1be4e700 1 --2- 172.21.15.102:0/50960344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe14073aa0 0x7fbe1410ce80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009940 tx=0x7fbe04009c50 comp rx=0 tx=0).stop 2024-05-13T22:53:03.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.904+0000 7fbe1be4e700 1 -- 172.21.15.102:0/50960344 shutdown_connections 2024-05-13T22:53:03.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.904+0000 7fbe1be4e700 1 --2- 172.21.15.102:0/50960344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe14073aa0 0x7fbe1410ce80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.904+0000 7fbe1be4e700 1 --2- 172.21.15.102:0/50960344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe14073180 0x7fbe14073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.904+0000 7fbe1be4e700 1 -- 172.21.15.102:0/50960344 >> 172.21.15.102:0/50960344 conn(0x7fbe140fc650 msgr2=0x7fbe140fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:03.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.904+0000 7fbe1be4e700 1 -- 172.21.15.102:0/50960344 shutdown_connections 2024-05-13T22:53:03.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.904+0000 7fbe1be4e700 1 -- 172.21.15.102:0/50960344 wait complete. 2024-05-13T22:53:03.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.904+0000 7fbe1be4e700 1 Processor -- start 2024-05-13T22:53:03.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.904+0000 7fbe1be4e700 1 -- start start 2024-05-13T22:53:03.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe1be4e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe14073180 0x7fbe1419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:03.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe1be4e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe14073aa0 0x7fbe1419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:03.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe1be4e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1419f460 con 0x7fbe14073aa0 2024-05-13T22:53:03.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe1be4e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe14198910 con 0x7fbe14073180 2024-05-13T22:53:03.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe193e9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe14073aa0 0x7fbe1419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:03.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe193e9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe14073aa0 0x7fbe1419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48756/0 (socket says 172.21.15.102:48756) 2024-05-13T22:53:03.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe193e9700 1 -- 172.21.15.102:0/696687806 learned_addr learned my addr 172.21.15.102:0/696687806 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe19bea700 1 --2- 172.21.15.102:0/696687806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe14073180 0x7fbe1419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe193e9700 1 -- 172.21.15.102:0/696687806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe14073180 msgr2=0x7fbe1419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe193e9700 1 --2- 172.21.15.102:0/696687806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe14073180 0x7fbe1419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe193e9700 1 -- 172.21.15.102:0/696687806 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe040095f0 con 0x7fbe14073aa0 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe193e9700 1 --2- 172.21.15.102:0/696687806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe14073aa0 0x7fbe1419edd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe040055f0 tx=0x7fbe040056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe0affd700 1 -- 172.21.15.102:0/696687806 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe0401d070 con 0x7fbe14073aa0 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe0affd700 1 -- 172.21.15.102:0/696687806 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe0400f460 con 0x7fbe14073aa0 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe1be4e700 1 -- 172.21.15.102:0/696687806 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe14198b90 con 0x7fbe14073aa0 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe0affd700 1 -- 172.21.15.102:0/696687806 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe04021620 con 0x7fbe14073aa0 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.905+0000 7fbe1be4e700 1 -- 172.21.15.102:0/696687806 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe14199080 con 0x7fbe14073aa0 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.906+0000 7fbe1be4e700 1 -- 172.21.15.102:0/696687806 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1410a580 con 0x7fbe14073aa0 2024-05-13T22:53:03.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.907+0000 7fbe0affd700 1 -- 172.21.15.102:0/696687806 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fbe0400fa60 con 0x7fbe14073aa0 2024-05-13T22:53:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.907+0000 7fbe0affd700 1 --2- 172.21.15.102:0/696687806 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbe0006e340 0x7fbe00070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.907+0000 7fbe0affd700 1 -- 172.21.15.102:0/696687806 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbe0408f6d0 con 0x7fbe14073aa0 2024-05-13T22:53:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.907+0000 7fbe19bea700 1 --2- 172.21.15.102:0/696687806 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbe0006e340 0x7fbe00070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:03.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.908+0000 7fbe19bea700 1 --2- 172.21.15.102:0/696687806 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbe0006e340 0x7fbe00070800 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbe10006fd0 tx=0x7fbe10008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:03.890 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:03.912+0000 7fbe0affd700 1 -- 172.21.15.102:0/696687806 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbe0405c370 con 0x7fbe14073aa0 2024-05-13T22:53:04.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.131+0000 7fbe1be4e700 1 -- 172.21.15.102:0/696687806 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe1402cce0 con 0x7fbe0006e340 2024-05-13T22:53:04.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.132+0000 7fbe0affd700 1 -- 172.21.15.102:0/696687806 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fbe1402cce0 con 0x7fbe0006e340 2024-05-13T22:53:04.110 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:53:04.111 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:53:04.111 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:53:04.111 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:53:04.111 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:53:04.111 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr" 2024-05-13T22:53:04.111 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:53:04.111 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "3/23 daemons upgraded", 2024-05-13T22:53:04.111 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading mon daemons", 2024-05-13T22:53:04.112 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:53:04.112 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:53:04.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.135+0000 7fbe1be4e700 1 -- 172.21.15.102:0/696687806 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbe0006e340 msgr2=0x7fbe00070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:04.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.135+0000 7fbe1be4e700 1 --2- 172.21.15.102:0/696687806 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbe0006e340 0x7fbe00070800 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbe10006fd0 tx=0x7fbe10008040 comp rx=0 tx=0).stop 2024-05-13T22:53:04.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.135+0000 7fbe1be4e700 1 -- 172.21.15.102:0/696687806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe14073aa0 msgr2=0x7fbe1419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:04.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.135+0000 7fbe1be4e700 1 --2- 172.21.15.102:0/696687806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe14073aa0 0x7fbe1419edd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe040055f0 tx=0x7fbe040056d0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.136+0000 7fbe1be4e700 1 -- 172.21.15.102:0/696687806 shutdown_connections 2024-05-13T22:53:04.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.136+0000 7fbe1be4e700 1 --2- 172.21.15.102:0/696687806 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbe0006e340 0x7fbe00070800 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.136+0000 7fbe1be4e700 1 --2- 172.21.15.102:0/696687806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe14073180 0x7fbe1419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.136+0000 7fbe1be4e700 1 --2- 172.21.15.102:0/696687806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe14073aa0 0x7fbe1419edd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.136+0000 7fbe1be4e700 1 -- 172.21.15.102:0/696687806 >> 172.21.15.102:0/696687806 conn(0x7fbe140fc650 msgr2=0x7fbe141076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:04.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.136+0000 7fbe1be4e700 1 -- 172.21.15.102:0/696687806 shutdown_connections 2024-05-13T22:53:04.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.136+0000 7fbe1be4e700 1 -- 172.21.15.102:0/696687806 wait complete. 2024-05-13T22:53:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.250+0000 7fedcc18f700 1 -- 172.21.15.102:0/795235920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedc4101d60 msgr2=0x7fedc4102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.250+0000 7fedcc18f700 1 --2- 172.21.15.102:0/795235920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedc4101d60 0x7fedc4102140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fedb400b0d0 tx=0x7fedb400b3e0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.250+0000 7fedcc18f700 1 -- 172.21.15.102:0/795235920 shutdown_connections 2024-05-13T22:53:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.250+0000 7fedcc18f700 1 --2- 172.21.15.102:0/795235920 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedc4102680 0x7fedc410ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.250+0000 7fedcc18f700 1 --2- 172.21.15.102:0/795235920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedc4101d60 0x7fedc4102140 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.250+0000 7fedcc18f700 1 -- 172.21.15.102:0/795235920 >> 172.21.15.102:0/795235920 conn(0x7fedc40fb550 msgr2=0x7fedc40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.250+0000 7fedcc18f700 1 -- 172.21.15.102:0/795235920 shutdown_connections 2024-05-13T22:53:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.250+0000 7fedcc18f700 1 -- 172.21.15.102:0/795235920 wait complete. 2024-05-13T22:53:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedcc18f700 1 Processor -- start 2024-05-13T22:53:04.229 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedcc18f700 1 -- start start 2024-05-13T22:53:04.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedcc18f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedc4101d60 0x7fedc419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:04.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedcc18f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedc4102680 0x7fedc419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:04.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedcc18f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc419d110 con 0x7fedc4102680 2024-05-13T22:53:04.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedcc18f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc41965c0 con 0x7fedc4101d60 2024-05-13T22:53:04.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedc972a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedc4102680 0x7fedc419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:04.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedc972a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedc4102680 0x7fedc419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:48770/0 (socket says 172.21.15.102:48770) 2024-05-13T22:53:04.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedc972a700 1 -- 172.21.15.102:0/2761336922 learned_addr learned my addr 172.21.15.102:0/2761336922 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:04.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedc9f2b700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedc4101d60 0x7fedc419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:04.231 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedc9f2b700 1 -- 172.21.15.102:0/2761336922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedc4101d60 msgr2=0x7fedc419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedc9f2b700 1 -- 172.21.15.102:0/2761336922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedc4101d60 msgr2=0x7fedc419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedc9f2b700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedc4101d60 0x7fedc419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedc9f2b700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedc4101d60 0x7fedc419c540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedc972a700 1 -- 172.21.15.102:0/2761336922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedc4101d60 msgr2=0x7fedc419c540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.251+0000 7fedc972a700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedc4101d60 0x7fedc419c540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.252+0000 7fedc972a700 1 -- 172.21.15.102:0/2761336922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb4009d00 con 0x7fedc4102680 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.252+0000 7fedc972a700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedc4102680 0x7fedc419ca80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fedb800d900 tx=0x7fedb800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.252+0000 7fedc2ffd700 1 -- 172.21.15.102:0/2761336922 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedb8004a00 con 0x7fedc4102680 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.252+0000 7fedc2ffd700 1 -- 172.21.15.102:0/2761336922 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedb800b5e0 con 0x7fedc4102680 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.252+0000 7fedc2ffd700 1 -- 172.21.15.102:0/2761336922 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedb8009bc0 con 0x7fedc4102680 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.252+0000 7fedcc18f700 1 -- 172.21.15.102:0/2761336922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc41968a0 con 0x7fedc4102680 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.252+0000 7fedcc18f700 1 -- 172.21.15.102:0/2761336922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc4196df0 con 0x7fedc4102680 2024-05-13T22:53:04.232 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.253+0000 7fedcc18f700 1 -- 172.21.15.102:0/2761336922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc4108270 con 0x7fedc4102680 2024-05-13T22:53:04.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.253+0000 7fedc2ffd700 1 -- 172.21.15.102:0/2761336922 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fedb8005020 con 0x7fedc4102680 2024-05-13T22:53:04.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.254+0000 7fedc2ffd700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedac06e340 0x7fedac070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:04.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.254+0000 7fedc2ffd700 1 -- 172.21.15.102:0/2761336922 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fedb808deb0 con 0x7fedc4102680 2024-05-13T22:53:04.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.254+0000 7fedc9f2b700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedac06e340 0x7fedac070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:04.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.254+0000 7fedc9f2b700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedac06e340 0x7fedac070800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fedb40062a0 tx=0x7fedb40061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:04.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.259+0000 7fedc2ffd700 1 -- 172.21.15.102:0/2761336922 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fedb805ab50 con 0x7fedc4102680 2024-05-13T22:53:04.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:04 smithi130 ceph-mon[84265]: pgmap v10: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-13T22:53:04.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:04 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:53:04.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:04 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:53:04.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:04 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:53:04.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:04 smithi102 ceph-mon[111694]: pgmap v10: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-13T22:53:04.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:04 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:53:04.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:04 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:53:04.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:04 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:53:04.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.539+0000 7fedcc18f700 1 -- 172.21.15.102:0/2761336922 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fedc404ea90 con 0x7fedc4102680 2024-05-13T22:53:04.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.539+0000 7fedc2ffd700 1 -- 172.21.15.102:0/2761336922 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fedb8017090 con 0x7fedc4102680 2024-05-13T22:53:04.517 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:53:04.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.541+0000 7fedcc18f700 1 -- 172.21.15.102:0/2761336922 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedac06e340 msgr2=0x7fedac070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:04.519 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.541+0000 7fedcc18f700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedac06e340 0x7fedac070800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fedb40062a0 tx=0x7fedb40061f0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.541+0000 7fedcc18f700 1 -- 172.21.15.102:0/2761336922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedc4102680 msgr2=0x7fedc419ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.541+0000 7fedcc18f700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedc4102680 0x7fedc419ca80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fedb800d900 tx=0x7fedb800dcc0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.541+0000 7fedcc18f700 1 -- 172.21.15.102:0/2761336922 shutdown_connections 2024-05-13T22:53:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.541+0000 7fedcc18f700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedac06e340 0x7fedac070800 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.541+0000 7fedcc18f700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedc4101d60 0x7fedc419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.541+0000 7fedcc18f700 1 --2- 172.21.15.102:0/2761336922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fedc4102680 0x7fedc419ca80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.541+0000 7fedcc18f700 1 -- 172.21.15.102:0/2761336922 >> 172.21.15.102:0/2761336922 conn(0x7fedc40fb550 msgr2=0x7fedc41053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.541+0000 7fedcc18f700 1 -- 172.21.15.102:0/2761336922 shutdown_connections 2024-05-13T22:53:04.520 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:04.541+0000 7fedcc18f700 1 -- 172.21.15.102:0/2761336922 wait complete. 2024-05-13T22:53:05.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: pgmap v12: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:05.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:53:05.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: overall HEALTH_OK 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: mon.smithi102 calling monitor election 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: mon.smithi130 calling monitor election 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: mon.smithi102 is new leader, mons smithi102,smithi130 in quorum (ranks 0,1) 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: monmap epoch 3 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: fsid 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: last_changed 2024-05-13T22:53:04.134894+0000 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: created 2024-05-13T22:38:11.166577+0000 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: min_mon_release 19 (squid) 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: election_strategy: 1 2024-05-13T22:53:05.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: 0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.smithi102 2024-05-13T22:53:05.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.smithi130 2024-05-13T22:53:05.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:53:05.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: osdmap e50: 6 total, 6 up, 6 in 2024-05-13T22:53:05.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: mgrmap e37: smithi102.xcijrj(active, since 18s), standbys: smithi130.uoqoxl 2024-05-13T22:53:05.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: overall HEALTH_OK 2024-05-13T22:53:05.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:05.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:05.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:53:05.402 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:05 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2761336922' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:53:05.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: pgmap v12: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: overall HEALTH_OK 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: mon.smithi102 calling monitor election 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: mon.smithi130 calling monitor election 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi102"}]: dispatch 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mon metadata", "id": "smithi130"}]: dispatch 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: mon.smithi102 is new leader, mons smithi102,smithi130 in quorum (ranks 0,1) 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: monmap epoch 3 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: fsid 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: last_changed 2024-05-13T22:53:04.134894+0000 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: created 2024-05-13T22:38:11.166577+0000 2024-05-13T22:53:05.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: min_mon_release 19 (squid) 2024-05-13T22:53:05.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: election_strategy: 1 2024-05-13T22:53:05.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: 0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.smithi102 2024-05-13T22:53:05.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.smithi130 2024-05-13T22:53:05.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:53:05.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: osdmap e50: 6 total, 6 up, 6 in 2024-05-13T22:53:05.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: mgrmap e37: smithi102.xcijrj(active, since 18s), standbys: smithi130.uoqoxl 2024-05-13T22:53:05.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: overall HEALTH_OK 2024-05-13T22:53:05.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:05.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:05.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:53:05.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:05 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2761336922' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:53:07.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:07 smithi130 ceph-mon[84265]: pgmap v13: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:07.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:07.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:07.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:07 smithi102 ceph-mon[111694]: pgmap v13: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:07.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:07.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:09.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:09 smithi130 ceph-mon[84265]: pgmap v14: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:09.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:09.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:09 smithi102 ceph-mon[111694]: pgmap v14: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:11.327 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:11 smithi130 ceph-mon[84265]: pgmap v15: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:11.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:11 smithi102 ceph-mon[111694]: pgmap v15: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:12.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: Detected new or changed devices on smithi130 2024-05-13T22:53:12.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:12.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:12.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:12.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:53:12.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:12.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: Reconfiguring mon.smithi102 (monmap changed)... 2024-05-13T22:53:12.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:53:12.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:53:12.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:12.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: Reconfiguring daemon mon.smithi102 on smithi102 2024-05-13T22:53:12.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:12 smithi130 ceph-mon[84265]: pgmap v16: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:12.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: Detected new or changed devices on smithi130 2024-05-13T22:53:12.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:12.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:12.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:12.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:53:12.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:12.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: Reconfiguring mon.smithi102 (monmap changed)... 2024-05-13T22:53:12.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:53:12.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:53:12.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:12.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: Reconfiguring daemon mon.smithi102 on smithi102 2024-05-13T22:53:12.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:12 smithi102 ceph-mon[111694]: pgmap v16: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:14.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:14 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:14.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:14 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:14.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:14 smithi102 ceph-mon[111694]: Reconfiguring mgr.smithi102.xcijrj (monmap changed)... 2024-05-13T22:53:14.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:14 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi102.xcijrj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:53:14.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:14 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:53:14.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:14 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:14.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:14 smithi102 ceph-mon[111694]: Reconfiguring daemon mgr.smithi102.xcijrj on smithi102 2024-05-13T22:53:14.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:14 smithi102 ceph-mon[111694]: pgmap v17: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:14.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:14 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:14.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:14 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:14.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:14 smithi130 ceph-mon[84265]: Reconfiguring mgr.smithi102.xcijrj (monmap changed)... 2024-05-13T22:53:14.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:14 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi102.xcijrj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:53:14.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:14 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:53:14.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:14 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:14.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:14 smithi130 ceph-mon[84265]: Reconfiguring daemon mgr.smithi102.xcijrj on smithi102 2024-05-13T22:53:14.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:14 smithi130 ceph-mon[84265]: pgmap v17: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:15.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:15 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:15.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:15 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:15.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:15 smithi102 ceph-mon[111694]: Reconfiguring ceph-exporter.smithi102 (monmap changed)... 2024-05-13T22:53:15.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:15 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:53:15.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:15 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:53:15.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:15 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-13T22:53:15.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:15 smithi102 ceph-mon[111694]: Unable to update caps for client.ceph-exporter.smithi102 2024-05-13T22:53:15.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:15 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi102"}]: dispatch 2024-05-13T22:53:15.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:15 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:15.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:15 smithi102 ceph-mon[111694]: Reconfiguring daemon ceph-exporter.smithi102 on smithi102 2024-05-13T22:53:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:15 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:15 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:15 smithi130 ceph-mon[84265]: Reconfiguring ceph-exporter.smithi102 (monmap changed)... 2024-05-13T22:53:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:15 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:53:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:15 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:53:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:15 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-13T22:53:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:15 smithi130 ceph-mon[84265]: Unable to update caps for client.ceph-exporter.smithi102 2024-05-13T22:53:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:15 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi102"}]: dispatch 2024-05-13T22:53:16.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:15 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:16.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:15 smithi130 ceph-mon[84265]: Reconfiguring daemon ceph-exporter.smithi102 on smithi102 2024-05-13T22:53:17.150 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:16 smithi102 ceph-mon[111694]: pgmap v18: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:17.150 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:53:17.150 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:17.150 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:17.151 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi102", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:53:17.151 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:16 smithi130 ceph-mon[84265]: pgmap v18: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:53:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:17.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:17.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi102", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:53:17.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:18.089 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:17 smithi102 ceph-mon[111694]: Reconfiguring crash.smithi102 (monmap changed)... 2024-05-13T22:53:18.089 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:17 smithi102 ceph-mon[111694]: Reconfiguring daemon crash.smithi102 on smithi102 2024-05-13T22:53:18.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:17 smithi130 ceph-mon[84265]: Reconfiguring crash.smithi102 (monmap changed)... 2024-05-13T22:53:18.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:17 smithi130 ceph-mon[84265]: Reconfiguring daemon crash.smithi102 on smithi102 2024-05-13T22:53:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:19 smithi130 ceph-mon[84265]: pgmap v19: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:19 smithi130 ceph-mon[84265]: Reconfiguring osd.0 (monmap changed)... 2024-05-13T22:53:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-13T22:53:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:19 smithi130 ceph-mon[84265]: Reconfiguring daemon osd.0 on smithi102 2024-05-13T22:53:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:19 smithi102 ceph-mon[111694]: pgmap v19: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:19.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:19.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:19.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:19 smithi102 ceph-mon[111694]: Reconfiguring osd.0 (monmap changed)... 2024-05-13T22:53:19.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-13T22:53:19.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:19.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:19 smithi102 ceph-mon[111694]: Reconfiguring daemon osd.0 on smithi102 2024-05-13T22:53:20.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:20.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:20.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:20 smithi102 ceph-mon[111694]: Reconfiguring osd.1 (monmap changed)... 2024-05-13T22:53:20.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-13T22:53:20.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:20.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:20 smithi102 ceph-mon[111694]: Reconfiguring daemon osd.1 on smithi102 2024-05-13T22:53:20.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:20 smithi102 ceph-mon[111694]: pgmap v20: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:21.046 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:21.046 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:21.046 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:20 smithi130 ceph-mon[84265]: Reconfiguring osd.1 (monmap changed)... 2024-05-13T22:53:21.046 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-13T22:53:21.046 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:21.046 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:20 smithi130 ceph-mon[84265]: Reconfiguring daemon osd.1 on smithi102 2024-05-13T22:53:21.046 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:20 smithi130 ceph-mon[84265]: pgmap v20: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:22.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:22 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:22.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:22 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:22.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:22 smithi102 ceph-mon[111694]: Reconfiguring osd.2 (monmap changed)... 2024-05-13T22:53:22.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:22 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-13T22:53:22.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:22 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:22.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:22 smithi102 ceph-mon[111694]: Reconfiguring daemon osd.2 on smithi102 2024-05-13T22:53:22.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:22 smithi102 ceph-mon[111694]: pgmap v21: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:22.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:22 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:22.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:22 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:22.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:22 smithi130 ceph-mon[84265]: Reconfiguring osd.2 (monmap changed)... 2024-05-13T22:53:22.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:22 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-13T22:53:22.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:22 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:22.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:22 smithi130 ceph-mon[84265]: Reconfiguring daemon osd.2 on smithi102 2024-05-13T22:53:22.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:22 smithi130 ceph-mon[84265]: pgmap v21: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:24.091 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:24.092 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:24.092 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:24 smithi102 ceph-mon[111694]: Reconfiguring mds.cephfs.smithi102.cgkcwz (monmap changed)... 2024-05-13T22:53:24.092 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.cgkcwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:53:24.092 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:24.092 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:24 smithi102 ceph-mon[111694]: Reconfiguring daemon mds.cephfs.smithi102.cgkcwz on smithi102 2024-05-13T22:53:24.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:24.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:24.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:24 smithi130 ceph-mon[84265]: Reconfiguring mds.cephfs.smithi102.cgkcwz (monmap changed)... 2024-05-13T22:53:24.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.cgkcwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:53:24.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:24.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:24 smithi130 ceph-mon[84265]: Reconfiguring daemon mds.cephfs.smithi102.cgkcwz on smithi102 2024-05-13T22:53:25.355 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:25 smithi102 ceph-mon[111694]: pgmap v22: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:25.356 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:25 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:25.356 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:25 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:25.356 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:25 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.sdylgv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:53:25.356 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:25 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:25.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:25 smithi130 ceph-mon[84265]: pgmap v22: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:25.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:25 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:25.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:25 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:25.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:25 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.sdylgv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:53:25.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:25 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:26.344 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:26 smithi102 ceph-mon[111694]: Reconfiguring mds.cephfs.smithi102.sdylgv (monmap changed)... 2024-05-13T22:53:26.345 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:26 smithi102 ceph-mon[111694]: Reconfiguring daemon mds.cephfs.smithi102.sdylgv on smithi102 2024-05-13T22:53:26.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:26 smithi130 ceph-mon[84265]: Reconfiguring mds.cephfs.smithi102.sdylgv (monmap changed)... 2024-05-13T22:53:26.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:26 smithi130 ceph-mon[84265]: Reconfiguring daemon mds.cephfs.smithi102.sdylgv on smithi102 2024-05-13T22:53:27.262 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:27 smithi130 ceph-mon[84265]: pgmap v23: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:27.263 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:27 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:27.263 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:27 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:27.263 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:27 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:53:27.263 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:27 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:53:27.263 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:27 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-13T22:53:27.263 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:27 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi130"}]: dispatch 2024-05-13T22:53:27.263 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:27 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:27.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:27 smithi102 ceph-mon[111694]: pgmap v23: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:27.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:27 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:27.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:27 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:27.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:27 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:53:27.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:27 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:53:27.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:27 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-13T22:53:27.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:27 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi130"}]: dispatch 2024-05-13T22:53:27.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:27 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:28.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:28 smithi130 ceph-mon[84265]: Reconfiguring ceph-exporter.smithi130 (monmap changed)... 2024-05-13T22:53:28.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:28 smithi130 ceph-mon[84265]: Unable to update caps for client.ceph-exporter.smithi130 2024-05-13T22:53:28.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:28 smithi130 ceph-mon[84265]: Reconfiguring daemon ceph-exporter.smithi130 on smithi130 2024-05-13T22:53:28.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:28 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:28.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:28 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:28.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:28 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi130", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:53:28.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:28 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:28.407 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:28 smithi102 ceph-mon[111694]: Reconfiguring ceph-exporter.smithi130 (monmap changed)... 2024-05-13T22:53:28.407 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:28 smithi102 ceph-mon[111694]: Unable to update caps for client.ceph-exporter.smithi130 2024-05-13T22:53:28.407 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:28 smithi102 ceph-mon[111694]: Reconfiguring daemon ceph-exporter.smithi130 on smithi130 2024-05-13T22:53:28.407 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:28 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:28.407 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:28 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:28.407 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:28 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi130", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:53:28.407 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:28 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:29.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:29 smithi130 ceph-mon[84265]: pgmap v24: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:29.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:29 smithi130 ceph-mon[84265]: Reconfiguring crash.smithi130 (monmap changed)... 2024-05-13T22:53:29.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:29 smithi130 ceph-mon[84265]: Reconfiguring daemon crash.smithi130 on smithi130 2024-05-13T22:53:29.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:29 smithi102 ceph-mon[111694]: pgmap v24: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:29.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:29 smithi102 ceph-mon[111694]: Reconfiguring crash.smithi130 (monmap changed)... 2024-05-13T22:53:29.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:29 smithi102 ceph-mon[111694]: Reconfiguring daemon crash.smithi130 on smithi130 2024-05-13T22:53:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:30 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:30 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:30 smithi130 ceph-mon[84265]: Reconfiguring mgr.smithi130.uoqoxl (monmap changed)... 2024-05-13T22:53:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:30 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi130.uoqoxl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:53:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:30 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:53:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:30 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:30 smithi130 ceph-mon[84265]: Reconfiguring daemon mgr.smithi130.uoqoxl on smithi130 2024-05-13T22:53:30.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:30 smithi130 ceph-mon[84265]: pgmap v25: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:30 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:30 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:30 smithi102 ceph-mon[111694]: Reconfiguring mgr.smithi130.uoqoxl (monmap changed)... 2024-05-13T22:53:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:30 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi130.uoqoxl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-13T22:53:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:30 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-13T22:53:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:30 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:30 smithi102 ceph-mon[111694]: Reconfiguring daemon mgr.smithi130.uoqoxl on smithi130 2024-05-13T22:53:30.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:30 smithi102 ceph-mon[111694]: pgmap v25: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:31.837 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:53:31.837 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:31.837 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:31.837 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:53:31.837 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:53:31.837 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:31.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:53:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-13T22:53:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-13T22:53:31.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:32.804 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:32 smithi130 ceph-mon[84265]: Reconfiguring mon.smithi130 (monmap changed)... 2024-05-13T22:53:32.805 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:32 smithi130 ceph-mon[84265]: Reconfiguring daemon mon.smithi130 on smithi130 2024-05-13T22:53:32.805 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:32 smithi130 ceph-mon[84265]: pgmap v26: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:32.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:32 smithi102 ceph-mon[111694]: Reconfiguring mon.smithi130 (monmap changed)... 2024-05-13T22:53:32.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:32 smithi102 ceph-mon[111694]: Reconfiguring daemon mon.smithi130 on smithi130 2024-05-13T22:53:32.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:32 smithi102 ceph-mon[111694]: pgmap v26: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:34.136 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:33 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:34.136 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:33 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:34.136 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:33 smithi130 ceph-mon[84265]: Reconfiguring osd.3 (monmap changed)... 2024-05-13T22:53:34.136 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:33 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-13T22:53:34.136 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:33 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:34.136 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:33 smithi130 ceph-mon[84265]: Reconfiguring daemon osd.3 on smithi130 2024-05-13T22:53:34.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:33 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:34.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:33 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:34.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:33 smithi102 ceph-mon[111694]: Reconfiguring osd.3 (monmap changed)... 2024-05-13T22:53:34.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:33 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-13T22:53:34.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:33 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:34.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:33 smithi102 ceph-mon[111694]: Reconfiguring daemon osd.3 on smithi130 2024-05-13T22:53:34.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.661+0000 7fe74ed25700 1 -- 172.21.15.102:0/474478641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe748101750 msgr2=0x7fe7481057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:34.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.661+0000 7fe74ed25700 1 --2- 172.21.15.102:0/474478641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe748101750 0x7fe7481057a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe734009910 tx=0x7fe734009c20 comp rx=0 tx=0).stop 2024-05-13T22:53:34.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.661+0000 7fe74ed25700 1 -- 172.21.15.102:0/474478641 shutdown_connections 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.661+0000 7fe74ed25700 1 --2- 172.21.15.102:0/474478641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe748101750 0x7fe7481057a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.661+0000 7fe74ed25700 1 --2- 172.21.15.102:0/474478641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe748100da0 0x7fe748101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.661+0000 7fe74ed25700 1 -- 172.21.15.102:0/474478641 >> 172.21.15.102:0/474478641 conn(0x7fe7480fc630 msgr2=0x7fe7480fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.661+0000 7fe74ed25700 1 -- 172.21.15.102:0/474478641 shutdown_connections 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.661+0000 7fe74ed25700 1 -- 172.21.15.102:0/474478641 wait complete. 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.662+0000 7fe74ed25700 1 Processor -- start 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.662+0000 7fe74ed25700 1 -- start start 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.662+0000 7fe74ed25700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe748100da0 0x7fe74819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.662+0000 7fe74ed25700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe748101750 0x7fe74819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.662+0000 7fe74ed25700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74819d0d0 con 0x7fe748100da0 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.662+0000 7fe74ed25700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe748196580 con 0x7fe748101750 2024-05-13T22:53:34.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.662+0000 7fe74cac1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe748100da0 0x7fe74819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.662+0000 7fe74cac1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe748100da0 0x7fe74819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36896/0 (socket says 172.21.15.102:36896) 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.662+0000 7fe74cac1700 1 -- 172.21.15.102:0/4072178462 learned_addr learned my addr 172.21.15.102:0/4072178462 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.662+0000 7fe747fff700 1 --2- 172.21.15.102:0/4072178462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe748101750 0x7fe74819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.663+0000 7fe74cac1700 1 -- 172.21.15.102:0/4072178462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe748101750 msgr2=0x7fe74819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.663+0000 7fe74cac1700 1 --2- 172.21.15.102:0/4072178462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe748101750 0x7fe74819ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.663+0000 7fe74cac1700 1 -- 172.21.15.102:0/4072178462 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7340095f0 con 0x7fe748100da0 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.663+0000 7fe74cac1700 1 --2- 172.21.15.102:0/4072178462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe748100da0 0x7fe74819c500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00e920 tx=0x7fe73c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.663+0000 7fe745ffb700 1 -- 172.21.15.102:0/4072178462 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe73c00c9d0 con 0x7fe748100da0 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.663+0000 7fe745ffb700 1 -- 172.21.15.102:0/4072178462 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe73c010430 con 0x7fe748100da0 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.663+0000 7fe74ed25700 1 -- 172.21.15.102:0/4072178462 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe748196860 con 0x7fe748100da0 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.663+0000 7fe745ffb700 1 -- 172.21.15.102:0/4072178462 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe73c018630 con 0x7fe748100da0 2024-05-13T22:53:34.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.663+0000 7fe74ed25700 1 -- 172.21.15.102:0/4072178462 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe748196d80 con 0x7fe748100da0 2024-05-13T22:53:34.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.664+0000 7fe74ed25700 1 -- 172.21.15.102:0/4072178462 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7481091a0 con 0x7fe748100da0 2024-05-13T22:53:34.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.665+0000 7fe745ffb700 1 -- 172.21.15.102:0/4072178462 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fe73c0105a0 con 0x7fe748100da0 2024-05-13T22:53:34.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.665+0000 7fe745ffb700 1 --2- 172.21.15.102:0/4072178462 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe73806e330 0x7fe7380707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:34.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.665+0000 7fe745ffb700 1 -- 172.21.15.102:0/4072178462 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe73c014070 con 0x7fe748100da0 2024-05-13T22:53:34.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.665+0000 7fe747fff700 1 --2- 172.21.15.102:0/4072178462 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe73806e330 0x7fe7380707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:34.644 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.666+0000 7fe747fff700 1 --2- 172.21.15.102:0/4072178462 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe73806e330 0x7fe7380707f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe734005b40 tx=0x7fe734005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:34.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.670+0000 7fe745ffb700 1 -- 172.21.15.102:0/4072178462 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe73c053850 con 0x7fe748100da0 2024-05-13T22:53:34.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.887+0000 7fe74ed25700 1 -- 172.21.15.102:0/4072178462 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe748197020 con 0x7fe73806e330 2024-05-13T22:53:34.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.888+0000 7fe745ffb700 1 -- 172.21.15.102:0/4072178462 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fe748197020 con 0x7fe73806e330 2024-05-13T22:53:34.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.890+0000 7fe74ed25700 1 -- 172.21.15.102:0/4072178462 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe73806e330 msgr2=0x7fe7380707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:34.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.890+0000 7fe74ed25700 1 --2- 172.21.15.102:0/4072178462 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe73806e330 0x7fe7380707f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe734005b40 tx=0x7fe734005a90 comp rx=0 tx=0).stop 2024-05-13T22:53:34.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.890+0000 7fe74ed25700 1 -- 172.21.15.102:0/4072178462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe748100da0 msgr2=0x7fe74819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:34.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.890+0000 7fe74ed25700 1 --2- 172.21.15.102:0/4072178462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe748100da0 0x7fe74819c500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00e920 tx=0x7fe73c00ec30 comp rx=0 tx=0).stop 2024-05-13T22:53:34.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.890+0000 7fe74ed25700 1 -- 172.21.15.102:0/4072178462 shutdown_connections 2024-05-13T22:53:34.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.890+0000 7fe74ed25700 1 --2- 172.21.15.102:0/4072178462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe748100da0 0x7fe74819c500 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:34.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.890+0000 7fe74ed25700 1 --2- 172.21.15.102:0/4072178462 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe73806e330 0x7fe7380707f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:34.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.890+0000 7fe74ed25700 1 --2- 172.21.15.102:0/4072178462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe748101750 0x7fe74819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:34.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.890+0000 7fe74ed25700 1 -- 172.21.15.102:0/4072178462 >> 172.21.15.102:0/4072178462 conn(0x7fe7480fc630 msgr2=0x7fe7480fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:34.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.891+0000 7fe74ed25700 1 -- 172.21.15.102:0/4072178462 shutdown_connections 2024-05-13T22:53:34.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.891+0000 7fe74ed25700 1 -- 172.21.15.102:0/4072178462 wait complete. 2024-05-13T22:53:34.878 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:53:34.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:34 smithi102 ceph-mon[111694]: pgmap v27: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:34.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:34 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:34.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:34 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:34.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:34 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-13T22:53:34.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:34 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:34.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.996+0000 7fad209f2700 1 -- 172.21.15.102:0/3820985561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffeb0 msgr2=0x7fad1c105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:34.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.996+0000 7fad209f2700 1 --2- 172.21.15.102:0/3820985561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffeb0 0x7fad1c105840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fad10009910 tx=0x7fad10009c20 comp rx=0 tx=0).stop 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.997+0000 7fad209f2700 1 -- 172.21.15.102:0/3820985561 shutdown_connections 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.997+0000 7fad209f2700 1 --2- 172.21.15.102:0/3820985561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffeb0 0x7fad1c105840 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.997+0000 7fad209f2700 1 --2- 172.21.15.102:0/3820985561 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ff500 0x7fad1c0ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.997+0000 7fad209f2700 1 -- 172.21.15.102:0/3820985561 >> 172.21.15.102:0/3820985561 conn(0x7fad1c074d80 msgr2=0x7fad1c075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.997+0000 7fad209f2700 1 -- 172.21.15.102:0/3820985561 shutdown_connections 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.997+0000 7fad209f2700 1 -- 172.21.15.102:0/3820985561 wait complete. 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.997+0000 7fad209f2700 1 Processor -- start 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.997+0000 7fad209f2700 1 -- start start 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad209f2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ff500 0x7fad1c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad209f2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffeb0 0x7fad1c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad209f2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c19d100 con 0x7fad1c0ffeb0 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad209f2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c1965b0 con 0x7fad1c0ff500 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad19d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffeb0 0x7fad1c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:34.976 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad19d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffeb0 0x7fad1c19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36918/0 (socket says 172.21.15.102:36918) 2024-05-13T22:53:34.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad19d9b700 1 -- 172.21.15.102:0/3182148 learned_addr learned my addr 172.21.15.102:0/3182148 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:34.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad1a59c700 1 --2- 172.21.15.102:0/3182148 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ff500 0x7fad1c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:34.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad19d9b700 1 -- 172.21.15.102:0/3182148 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ff500 msgr2=0x7fad1c19c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:34.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad19d9b700 1 --2- 172.21.15.102:0/3182148 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ff500 0x7fad1c19c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:34.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad19d9b700 1 -- 172.21.15.102:0/3182148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad100095f0 con 0x7fad1c0ffeb0 2024-05-13T22:53:34.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad19d9b700 1 --2- 172.21.15.102:0/3182148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffeb0 0x7fad1c19ca70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad100092e0 tx=0x7fad1000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:34.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad0b7fe700 1 -- 172.21.15.102:0/3182148 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad1001d070 con 0x7fad1c0ffeb0 2024-05-13T22:53:34.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad0b7fe700 1 -- 172.21.15.102:0/3182148 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad10005dc0 con 0x7fad1c0ffeb0 2024-05-13T22:53:34.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad209f2700 1 -- 172.21.15.102:0/3182148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c196830 con 0x7fad1c0ffeb0 2024-05-13T22:53:34.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad0b7fe700 1 -- 172.21.15.102:0/3182148 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad10017400 con 0x7fad1c0ffeb0 2024-05-13T22:53:34.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.998+0000 7fad209f2700 1 -- 172.21.15.102:0/3182148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c196d20 con 0x7fad1c0ffeb0 2024-05-13T22:53:34.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:34.999+0000 7fad209f2700 1 -- 172.21.15.102:0/3182148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c1090e0 con 0x7fad1c0ffeb0 2024-05-13T22:53:34.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.000+0000 7fad0b7fe700 1 -- 172.21.15.102:0/3182148 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fad1000f6f0 con 0x7fad1c0ffeb0 2024-05-13T22:53:34.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.000+0000 7fad0b7fe700 1 --2- 172.21.15.102:0/3182148 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fad0406e340 0x7fad04070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:34.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.000+0000 7fad0b7fe700 1 -- 172.21.15.102:0/3182148 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fad1008f750 con 0x7fad1c0ffeb0 2024-05-13T22:53:34.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.001+0000 7fad1a59c700 1 --2- 172.21.15.102:0/3182148 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fad0406e340 0x7fad04070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:34.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.001+0000 7fad1a59c700 1 --2- 172.21.15.102:0/3182148 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fad0406e340 0x7fad04070800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005950 tx=0x7fad0c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:34.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.005+0000 7fad0b7fe700 1 -- 172.21.15.102:0/3182148 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fad1005c3f0 con 0x7fad1c0ffeb0 2024-05-13T22:53:35.113 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:34 smithi130 ceph-mon[84265]: pgmap v27: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:35.113 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:34 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:35.113 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:34 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:35.113 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:34 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-13T22:53:35.113 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:34 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:35.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.220+0000 7fad209f2700 1 -- 172.21.15.102:0/3182148 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad1c02ccc0 con 0x7fad0406e340 2024-05-13T22:53:35.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.221+0000 7fad0b7fe700 1 -- 172.21.15.102:0/3182148 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fad1c02ccc0 con 0x7fad0406e340 2024-05-13T22:53:35.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.224+0000 7fad209f2700 1 -- 172.21.15.102:0/3182148 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fad0406e340 msgr2=0x7fad04070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:35.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.224+0000 7fad209f2700 1 --2- 172.21.15.102:0/3182148 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fad0406e340 0x7fad04070800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fad0c005950 tx=0x7fad0c00b410 comp rx=0 tx=0).stop 2024-05-13T22:53:35.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.224+0000 7fad209f2700 1 -- 172.21.15.102:0/3182148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffeb0 msgr2=0x7fad1c19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:35.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.224+0000 7fad209f2700 1 --2- 172.21.15.102:0/3182148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffeb0 0x7fad1c19ca70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad100092e0 tx=0x7fad1000bd70 comp rx=0 tx=0).stop 2024-05-13T22:53:35.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.224+0000 7fad209f2700 1 -- 172.21.15.102:0/3182148 shutdown_connections 2024-05-13T22:53:35.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.224+0000 7fad209f2700 1 --2- 172.21.15.102:0/3182148 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fad0406e340 0x7fad04070800 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.224+0000 7fad209f2700 1 --2- 172.21.15.102:0/3182148 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad1c0ff500 0x7fad1c19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.224+0000 7fad209f2700 1 --2- 172.21.15.102:0/3182148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad1c0ffeb0 0x7fad1c19ca70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.224+0000 7fad209f2700 1 -- 172.21.15.102:0/3182148 >> 172.21.15.102:0/3182148 conn(0x7fad1c074d80 msgr2=0x7fad1c1050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:35.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.224+0000 7fad209f2700 1 -- 172.21.15.102:0/3182148 shutdown_connections 2024-05-13T22:53:35.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.224+0000 7fad209f2700 1 -- 172.21.15.102:0/3182148 wait complete. 2024-05-13T22:53:35.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.329+0000 7f7d65646700 1 -- 172.21.15.102:0/2601047295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d60105a40 msgr2=0x7f7d60105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:35.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.329+0000 7f7d65646700 1 --2- 172.21.15.102:0/2601047295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d60105a40 0x7f7d60105e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009910 tx=0x7f7d54009c20 comp rx=0 tx=0).stop 2024-05-13T22:53:35.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.329+0000 7f7d65646700 1 -- 172.21.15.102:0/2601047295 shutdown_connections 2024-05-13T22:53:35.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.329+0000 7f7d65646700 1 --2- 172.21.15.102:0/2601047295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d600ffbd0 0x7f7d60100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.329+0000 7f7d65646700 1 --2- 172.21.15.102:0/2601047295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d60105a40 0x7f7d60105e20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.329+0000 7f7d65646700 1 -- 172.21.15.102:0/2601047295 >> 172.21.15.102:0/2601047295 conn(0x7f7d600755a0 msgr2=0x7f7d600759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:35.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.329+0000 7f7d65646700 1 -- 172.21.15.102:0/2601047295 shutdown_connections 2024-05-13T22:53:35.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.330+0000 7f7d65646700 1 -- 172.21.15.102:0/2601047295 wait complete. 2024-05-13T22:53:35.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.330+0000 7f7d65646700 1 Processor -- start 2024-05-13T22:53:35.308 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.330+0000 7f7d65646700 1 -- start start 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d65646700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d600ffbd0 0x7f7d60195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d65646700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d60105a40 0x7f7d60195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d65646700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d60196260 con 0x7f7d60105a40 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d65646700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d60199ff0 con 0x7f7d600ffbd0 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d5e7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d60105a40 0x7f7d60195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d5e7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d60105a40 0x7f7d60195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36926/0 (socket says 172.21.15.102:36926) 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d5effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d600ffbd0 0x7f7d60195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d5e7fc700 1 -- 172.21.15.102:0/2348826100 learned_addr learned my addr 172.21.15.102:0/2348826100 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d5effd700 1 -- 172.21.15.102:0/2348826100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d60105a40 msgr2=0x7f7d60195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d5effd700 1 --2- 172.21.15.102:0/2348826100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d60105a40 0x7f7d60195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d5effd700 1 -- 172.21.15.102:0/2348826100 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540095f0 con 0x7f7d600ffbd0 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d5e7fc700 1 --2- 172.21.15.102:0/2348826100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d60105a40 0x7f7d60195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:53:35.309 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d5effd700 1 --2- 172.21.15.102:0/2348826100 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d600ffbd0 0x7f7d60195640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009910 tx=0x7f7d5400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:35.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.331+0000 7f7d47fff700 1 -- 172.21.15.102:0/2348826100 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d5401d070 con 0x7f7d600ffbd0 2024-05-13T22:53:35.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.332+0000 7f7d47fff700 1 -- 172.21.15.102:0/2348826100 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d54005dc0 con 0x7f7d600ffbd0 2024-05-13T22:53:35.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.332+0000 7f7d47fff700 1 -- 172.21.15.102:0/2348826100 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d5400fe80 con 0x7f7d600ffbd0 2024-05-13T22:53:35.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.332+0000 7f7d65646700 1 -- 172.21.15.102:0/2348826100 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6019a270 con 0x7f7d600ffbd0 2024-05-13T22:53:35.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.332+0000 7f7d65646700 1 -- 172.21.15.102:0/2348826100 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6019a760 con 0x7f7d600ffbd0 2024-05-13T22:53:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.333+0000 7f7d65646700 1 -- 172.21.15.102:0/2348826100 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6004ea90 con 0x7f7d600ffbd0 2024-05-13T22:53:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.333+0000 7f7d47fff700 1 -- 172.21.15.102:0/2348826100 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f7d54003a60 con 0x7f7d600ffbd0 2024-05-13T22:53:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.334+0000 7f7d47fff700 1 --2- 172.21.15.102:0/2348826100 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7d4806e340 0x7f7d48070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.334+0000 7f7d47fff700 1 -- 172.21.15.102:0/2348826100 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7d5408f5d0 con 0x7f7d600ffbd0 2024-05-13T22:53:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.334+0000 7f7d5e7fc700 1 --2- 172.21.15.102:0/2348826100 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7d4806e340 0x7f7d48070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:35.311 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.334+0000 7f7d5e7fc700 1 --2- 172.21.15.102:0/2348826100 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7d4806e340 0x7f7d48070800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7d60196c60 tx=0x7f7d5000d710 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:35.315 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.339+0000 7f7d47fff700 1 -- 172.21.15.102:0/2348826100 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7d540587c0 con 0x7f7d600ffbd0 2024-05-13T22:53:35.521 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.544+0000 7f7d65646700 1 -- 172.21.15.102:0/2348826100 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d6019ac30 con 0x7f7d4806e340 2024-05-13T22:53:35.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.555+0000 7f7d47fff700 1 -- 172.21.15.102:0/2348826100 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7d6019ac30 con 0x7f7d4806e340 2024-05-13T22:53:35.532 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:53:35.532 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (2m) 44s ago 13m 26.7M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:53:35.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (14m) 44s ago 14m 8162k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:53:35.532 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (13m) 28s ago 13m 7889k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:53:35.532 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (14m) 44s ago 14m 6660k - 18.2.1 5be31c24972a 3c4c0d9d8a9f 2024-05-13T22:53:35.532 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (12m) 28s ago 12m 6669k - 18.2.1 5be31c24972a bac13b0fae64 2024-05-13T22:53:35.532 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (2m) 44s ago 13m 75.0M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:53:35.532 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (8m) 44s ago 8m 88.4M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:53:35.532 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (8m) 44s ago 8m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:53:35.532 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (8m) 28s ago 8m 15.8M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (8m) 28s ago 8m 18.8M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (4m) 44s ago 15m 475M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (3m) 28s ago 12m 450M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (59s) 44s ago 15m 31.7M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (33s) 28s ago 12m 27.9M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (3m) 44s ago 13m 16.0M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (3m) 28s ago 12m 16.8M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (11m) 44s ago 11m 224M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (11m) 44s ago 11m 239M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (11m) 44s ago 11m 203M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (10m) 28s ago 10m 292M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (10m) 28s ago 10m 245M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (9m) 28s ago 9m 227M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:53:35.533 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (3m) 44s ago 13m 67.2M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:53:35.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.557+0000 7f7d65646700 1 -- 172.21.15.102:0/2348826100 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7d4806e340 msgr2=0x7f7d48070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:35.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.557+0000 7f7d65646700 1 --2- 172.21.15.102:0/2348826100 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7d4806e340 0x7f7d48070800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7d60196c60 tx=0x7f7d5000d710 comp rx=0 tx=0).stop 2024-05-13T22:53:35.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.557+0000 7f7d65646700 1 -- 172.21.15.102:0/2348826100 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d600ffbd0 msgr2=0x7f7d60195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:35.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.557+0000 7f7d65646700 1 --2- 172.21.15.102:0/2348826100 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d600ffbd0 0x7f7d60195640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009910 tx=0x7f7d5400bd70 comp rx=0 tx=0).stop 2024-05-13T22:53:35.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.557+0000 7f7d65646700 1 -- 172.21.15.102:0/2348826100 shutdown_connections 2024-05-13T22:53:35.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.557+0000 7f7d65646700 1 --2- 172.21.15.102:0/2348826100 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7d4806e340 0x7f7d48070800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.557+0000 7f7d65646700 1 --2- 172.21.15.102:0/2348826100 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d600ffbd0 0x7f7d60195640 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.557+0000 7f7d65646700 1 --2- 172.21.15.102:0/2348826100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d60105a40 0x7f7d60195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.557+0000 7f7d65646700 1 -- 172.21.15.102:0/2348826100 >> 172.21.15.102:0/2348826100 conn(0x7f7d600755a0 msgr2=0x7f7d600fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:35.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.557+0000 7f7d65646700 1 -- 172.21.15.102:0/2348826100 shutdown_connections 2024-05-13T22:53:35.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.557+0000 7f7d65646700 1 -- 172.21.15.102:0/2348826100 wait complete. 2024-05-13T22:53:35.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.661+0000 7f329e19c700 1 -- 172.21.15.102:0/327934981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3298073a80 msgr2=0x7f329810cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.661+0000 7f329e19c700 1 --2- 172.21.15.102:0/327934981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3298073a80 0x7f329810cec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3280009910 tx=0x7f3280009c20 comp rx=0 tx=0).stop 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.661+0000 7f329e19c700 1 -- 172.21.15.102:0/327934981 shutdown_connections 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.661+0000 7f329e19c700 1 --2- 172.21.15.102:0/327934981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3298073a80 0x7f329810cec0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.661+0000 7f329e19c700 1 --2- 172.21.15.102:0/327934981 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3298073160 0x7f3298073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.661+0000 7f329e19c700 1 -- 172.21.15.102:0/327934981 >> 172.21.15.102:0/327934981 conn(0x7f32980fc650 msgr2=0x7f32980fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.661+0000 7f329e19c700 1 -- 172.21.15.102:0/327934981 shutdown_connections 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.661+0000 7f329e19c700 1 -- 172.21.15.102:0/327934981 wait complete. 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f329e19c700 1 Processor -- start 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f329e19c700 1 -- start start 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f329e19c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3298073160 0x7f329819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f329e19c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3298073a80 0x7f329819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f329e19c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329819f350 con 0x7f3298073160 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f329e19c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3298198800 con 0x7f3298073a80 2024-05-13T22:53:35.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f32977fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3298073160 0x7f329819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f32977fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3298073160 0x7f329819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36946/0 (socket says 172.21.15.102:36946) 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f32977fe700 1 -- 172.21.15.102:0/3052591919 learned_addr learned my addr 172.21.15.102:0/3052591919 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f328ffff700 1 --2- 172.21.15.102:0/3052591919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3298073a80 0x7f329819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f32977fe700 1 -- 172.21.15.102:0/3052591919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3298073a80 msgr2=0x7f329819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f32977fe700 1 --2- 172.21.15.102:0/3052591919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3298073a80 0x7f329819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f32977fe700 1 -- 172.21.15.102:0/3052591919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32800095f0 con 0x7f3298073160 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.662+0000 7f32977fe700 1 --2- 172.21.15.102:0/3052591919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3298073160 0x7f329819e780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f328800d8d0 tx=0x7f328800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.663+0000 7f32957fa700 1 -- 172.21.15.102:0/3052591919 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32880098a0 con 0x7f3298073160 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.663+0000 7f329e19c700 1 -- 172.21.15.102:0/3052591919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3298198ae0 con 0x7f3298073160 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.663+0000 7f32957fa700 1 -- 172.21.15.102:0/3052591919 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3288010460 con 0x7f3298073160 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.663+0000 7f32957fa700 1 -- 172.21.15.102:0/3052591919 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f328800f5f0 con 0x7f3298073160 2024-05-13T22:53:35.641 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.663+0000 7f329e19c700 1 -- 172.21.15.102:0/3052591919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3298199000 con 0x7f3298073160 2024-05-13T22:53:35.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.664+0000 7f329e19c700 1 -- 172.21.15.102:0/3052591919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329810a5c0 con 0x7f3298073160 2024-05-13T22:53:35.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.665+0000 7f32957fa700 1 -- 172.21.15.102:0/3052591919 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f3288009a00 con 0x7f3298073160 2024-05-13T22:53:35.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.665+0000 7f32957fa700 1 --2- 172.21.15.102:0/3052591919 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f328406e330 0x7f32840707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:35.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.665+0000 7f32957fa700 1 -- 172.21.15.102:0/3052591919 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f328808e250 con 0x7f3298073160 2024-05-13T22:53:35.642 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.665+0000 7f328ffff700 1 --2- 172.21.15.102:0/3052591919 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f328406e330 0x7f32840707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:35.643 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.666+0000 7f328ffff700 1 --2- 172.21.15.102:0/3052591919 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f328406e330 0x7f32840707f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3280006010 tx=0x7f328001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:35.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.670+0000 7f32957fa700 1 -- 172.21.15.102:0/3052591919 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f328805aef0 con 0x7f3298073160 2024-05-13T22:53:35.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:35 smithi102 ceph-mon[111694]: Reconfiguring osd.4 (monmap changed)... 2024-05-13T22:53:35.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:35 smithi102 ceph-mon[111694]: Reconfiguring daemon osd.4 on smithi130 2024-05-13T22:53:35.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:35 smithi102 ceph-mon[111694]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:53:35.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.957+0000 7f329e19c700 1 -- 172.21.15.102:0/3052591919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f32981992c0 con 0x7f3298073160 2024-05-13T22:53:35.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.958+0000 7f32957fa700 1 -- 172.21.15.102:0/3052591919 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f328805aa80 con 0x7f3298073160 2024-05-13T22:53:35.935 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:53:35.935 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:53:35.935 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:53:35.935 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:53:35.935 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:53:35.935 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:53:35.935 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:53:35.935 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:53:35.935 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-05-13T22:53:35.935 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:53:35.935 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:53:35.936 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:53:35.936 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:53:35.936 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:53:35.936 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-05-13T22:53:35.936 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 4 2024-05-13T22:53:35.936 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:53:35.936 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:53:35.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.960+0000 7f329e19c700 1 -- 172.21.15.102:0/3052591919 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f328406e330 msgr2=0x7f32840707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:35.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.960+0000 7f329e19c700 1 --2- 172.21.15.102:0/3052591919 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f328406e330 0x7f32840707f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3280006010 tx=0x7f328001a040 comp rx=0 tx=0).stop 2024-05-13T22:53:35.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.960+0000 7f329e19c700 1 -- 172.21.15.102:0/3052591919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3298073160 msgr2=0x7f329819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:35.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.960+0000 7f329e19c700 1 --2- 172.21.15.102:0/3052591919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3298073160 0x7f329819e780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f328800d8d0 tx=0x7f328800dbe0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.960+0000 7f329e19c700 1 -- 172.21.15.102:0/3052591919 shutdown_connections 2024-05-13T22:53:35.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.960+0000 7f329e19c700 1 --2- 172.21.15.102:0/3052591919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3298073160 0x7f329819e780 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.960+0000 7f329e19c700 1 --2- 172.21.15.102:0/3052591919 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f328406e330 0x7f32840707f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.960+0000 7f329e19c700 1 --2- 172.21.15.102:0/3052591919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3298073a80 0x7f329819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:35.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.960+0000 7f329e19c700 1 -- 172.21.15.102:0/3052591919 >> 172.21.15.102:0/3052591919 conn(0x7f32980fc650 msgr2=0x7f3298107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:35.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.960+0000 7f329e19c700 1 -- 172.21.15.102:0/3052591919 shutdown_connections 2024-05-13T22:53:35.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:35.960+0000 7f329e19c700 1 -- 172.21.15.102:0/3052591919 wait complete. 2024-05-13T22:53:36.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.066+0000 7f9db31ca700 1 -- 172.21.15.102:0/1845174806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dac105a40 msgr2=0x7f9dac105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:36.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.066+0000 7f9db31ca700 1 --2- 172.21.15.102:0/1845174806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dac105a40 0x7f9dac105e20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009910 tx=0x7f9da0009c20 comp rx=0 tx=0).stop 2024-05-13T22:53:36.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.066+0000 7f9db31ca700 1 -- 172.21.15.102:0/1845174806 shutdown_connections 2024-05-13T22:53:36.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.066+0000 7f9db31ca700 1 --2- 172.21.15.102:0/1845174806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9dac0ffa40 0x7f9dac0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.066+0000 7f9db31ca700 1 --2- 172.21.15.102:0/1845174806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dac105a40 0x7f9dac105e20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.066+0000 7f9db31ca700 1 -- 172.21.15.102:0/1845174806 >> 172.21.15.102:0/1845174806 conn(0x7f9dac0fb530 msgr2=0x7f9dac0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:36.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.066+0000 7f9db31ca700 1 -- 172.21.15.102:0/1845174806 shutdown_connections 2024-05-13T22:53:36.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.066+0000 7f9db31ca700 1 -- 172.21.15.102:0/1845174806 wait complete. 2024-05-13T22:53:36.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.067+0000 7f9db31ca700 1 Processor -- start 2024-05-13T22:53:36.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.067+0000 7f9db31ca700 1 -- start start 2024-05-13T22:53:36.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.067+0000 7f9db31ca700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9dac0ffa40 0x7f9dac19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:36.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.067+0000 7f9db31ca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dac105a40 0x7f9dac19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:36.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.067+0000 7f9db31ca700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac19f2a0 con 0x7f9dac105a40 2024-05-13T22:53:36.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.067+0000 7f9db31ca700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac198750 con 0x7f9dac0ffa40 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.067+0000 7f9dabfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dac105a40 0x7f9dac19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.067+0000 7f9dabfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dac105a40 0x7f9dac19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36964/0 (socket says 172.21.15.102:36964) 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.067+0000 7f9dabfff700 1 -- 172.21.15.102:0/3279147335 learned_addr learned my addr 172.21.15.102:0/3279147335 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.067+0000 7f9db0f66700 1 --2- 172.21.15.102:0/3279147335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9dac0ffa40 0x7f9dac19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.068+0000 7f9dabfff700 1 -- 172.21.15.102:0/3279147335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9dac0ffa40 msgr2=0x7f9dac19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.068+0000 7f9dabfff700 1 --2- 172.21.15.102:0/3279147335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9dac0ffa40 0x7f9dac19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.068+0000 7f9dabfff700 1 -- 172.21.15.102:0/3279147335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da00095f0 con 0x7f9dac105a40 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.068+0000 7f9dabfff700 1 --2- 172.21.15.102:0/3279147335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dac105a40 0x7f9dac19ec10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800c8a0 tx=0x7f9d9800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.068+0000 7f9da9ffb700 1 -- 172.21.15.102:0/3279147335 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d980078e0 con 0x7f9dac105a40 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.068+0000 7f9da9ffb700 1 -- 172.21.15.102:0/3279147335 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d9800f450 con 0x7f9dac105a40 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.068+0000 7f9db31ca700 1 -- 172.21.15.102:0/3279147335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dac198a30 con 0x7f9dac105a40 2024-05-13T22:53:36.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.068+0000 7f9da9ffb700 1 -- 172.21.15.102:0/3279147335 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d9800e5e0 con 0x7f9dac105a40 2024-05-13T22:53:36.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.068+0000 7f9db31ca700 1 -- 172.21.15.102:0/3279147335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dac198f80 con 0x7f9dac105a40 2024-05-13T22:53:36.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.069+0000 7f9db31ca700 1 -- 172.21.15.102:0/3279147335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dac04ea90 con 0x7f9dac105a40 2024-05-13T22:53:36.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.069+0000 7f9da9ffb700 1 -- 172.21.15.102:0/3279147335 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f9d9800f5c0 con 0x7f9dac105a40 2024-05-13T22:53:36.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.070+0000 7f9da9ffb700 1 --2- 172.21.15.102:0/3279147335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9d9c06e330 0x7f9d9c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:36.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.070+0000 7f9da9ffb700 1 -- 172.21.15.102:0/3279147335 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9d9808e1f0 con 0x7f9dac105a40 2024-05-13T22:53:36.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.070+0000 7f9db0f66700 1 --2- 172.21.15.102:0/3279147335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9d9c06e330 0x7f9d9c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:36.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.070+0000 7f9db0f66700 1 --2- 172.21.15.102:0/3279147335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9d9c06e330 0x7f9d9c0707f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9da0005f50 tx=0x7f9da001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:36.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.074+0000 7f9da9ffb700 1 -- 172.21.15.102:0/3279147335 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9d9805af10 con 0x7f9dac105a40 2024-05-13T22:53:36.116 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:35 smithi130 ceph-mon[84265]: Reconfiguring osd.4 (monmap changed)... 2024-05-13T22:53:36.116 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:35 smithi130 ceph-mon[84265]: Reconfiguring daemon osd.4 on smithi130 2024-05-13T22:53:36.116 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:35 smithi130 ceph-mon[84265]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:53:36.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.315+0000 7f9db31ca700 1 -- 172.21.15.102:0/3279147335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9dac199c60 con 0x7f9dac105a40 2024-05-13T22:53:36.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.316+0000 7f9da9ffb700 1 -- 172.21.15.102:0/3279147335 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1909 (secure 0 0 0) 0x7f9d9805aaa0 con 0x7f9dac105a40 2024-05-13T22:53:36.293 INFO:teuthology.orchestra.run.smithi102.stdout:e17 2024-05-13T22:53:36.293 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:53:36.293 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:53:36.293 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:53:36.293 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:53:36.293 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:53:36.293 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 17 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 47 2024-05-13T22:53:36.294 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=14498} 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 0 members: 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:36.295 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:36.296 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:53:36.296 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:53:36.296 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:53:36.296 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:53:36.296 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:36.296 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:53:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.320+0000 7f9db31ca700 1 -- 172.21.15.102:0/3279147335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9d9c06e330 msgr2=0x7f9d9c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.320+0000 7f9db31ca700 1 --2- 172.21.15.102:0/3279147335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9d9c06e330 0x7f9d9c0707f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9da0005f50 tx=0x7f9da001a040 comp rx=0 tx=0).stop 2024-05-13T22:53:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.320+0000 7f9db31ca700 1 -- 172.21.15.102:0/3279147335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dac105a40 msgr2=0x7f9dac19ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.320+0000 7f9db31ca700 1 --2- 172.21.15.102:0/3279147335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dac105a40 0x7f9dac19ec10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800c8a0 tx=0x7f9d9800cbb0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.320+0000 7f9db31ca700 1 -- 172.21.15.102:0/3279147335 shutdown_connections 2024-05-13T22:53:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.320+0000 7f9db31ca700 1 --2- 172.21.15.102:0/3279147335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9d9c06e330 0x7f9d9c0707f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.320+0000 7f9db31ca700 1 --2- 172.21.15.102:0/3279147335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9dac0ffa40 0x7f9dac19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.320+0000 7f9db31ca700 1 --2- 172.21.15.102:0/3279147335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9dac105a40 0x7f9dac19ec10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.320+0000 7f9db31ca700 1 -- 172.21.15.102:0/3279147335 >> 172.21.15.102:0/3279147335 conn(0x7f9dac0fb530 msgr2=0x7f9dac068e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:36.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.320+0000 7f9db31ca700 1 -- 172.21.15.102:0/3279147335 shutdown_connections 2024-05-13T22:53:36.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.320+0000 7f9db31ca700 1 -- 172.21.15.102:0/3279147335 wait complete. 2024-05-13T22:53:36.298 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:53:36.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.426+0000 7fc4245e9700 1 -- 172.21.15.102:0/4085867960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc41c100dd0 msgr2=0x7fc41c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:36.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.426+0000 7fc4245e9700 1 --2- 172.21.15.102:0/4085867960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc41c100dd0 0x7fc41c1011b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc418009910 tx=0x7fc418009c20 comp rx=0 tx=0).stop 2024-05-13T22:53:36.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.426+0000 7fc4245e9700 1 -- 172.21.15.102:0/4085867960 shutdown_connections 2024-05-13T22:53:36.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.426+0000 7fc4245e9700 1 --2- 172.21.15.102:0/4085867960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc41c101780 0x7fc41c1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.426+0000 7fc4245e9700 1 --2- 172.21.15.102:0/4085867960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc41c100dd0 0x7fc41c1011b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.426+0000 7fc4245e9700 1 -- 172.21.15.102:0/4085867960 >> 172.21.15.102:0/4085867960 conn(0x7fc41c0fc640 msgr2=0x7fc41c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:36.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.426+0000 7fc4245e9700 1 -- 172.21.15.102:0/4085867960 shutdown_connections 2024-05-13T22:53:36.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.426+0000 7fc4245e9700 1 -- 172.21.15.102:0/4085867960 wait complete. 2024-05-13T22:53:36.405 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.427+0000 7fc4245e9700 1 Processor -- start 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.427+0000 7fc4245e9700 1 -- start start 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.427+0000 7fc4245e9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc41c100dd0 0x7fc41c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.427+0000 7fc4245e9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc41c101780 0x7fc41c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.427+0000 7fc4245e9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c073b10 con 0x7fc41c101780 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.427+0000 7fc4245e9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c073c50 con 0x7fc41c100dd0 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.427+0000 7fc421b84700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc41c101780 0x7fc41c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.427+0000 7fc422385700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc41c100dd0 0x7fc41c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.427+0000 7fc422385700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc41c100dd0 0x7fc41c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.102:55102/0 (socket says 172.21.15.102:55102) 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.427+0000 7fc421b84700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc41c101780 0x7fc41c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:36978/0 (socket says 172.21.15.102:36978) 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.427+0000 7fc422385700 1 -- 172.21.15.102:0/156607141 learned_addr learned my addr 172.21.15.102:0/156607141 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.428+0000 7fc421b84700 1 -- 172.21.15.102:0/156607141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc41c100dd0 msgr2=0x7fc41c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.428+0000 7fc421b84700 1 --2- 172.21.15.102:0/156607141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc41c100dd0 0x7fc41c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.428+0000 7fc421b84700 1 -- 172.21.15.102:0/156607141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4180095f0 con 0x7fc41c101780 2024-05-13T22:53:36.406 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.428+0000 7fc422385700 1 --2- 172.21.15.102:0/156607141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc41c100dd0 0x7fc41c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:53:36.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.428+0000 7fc421b84700 1 --2- 172.21.15.102:0/156607141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc41c101780 0x7fc41c0735d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc41c06a0d0 tx=0x7fc40c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:36.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.428+0000 7fc4137fe700 1 -- 172.21.15.102:0/156607141 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc40c00f860 con 0x7fc41c101780 2024-05-13T22:53:36.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.428+0000 7fc4137fe700 1 -- 172.21.15.102:0/156607141 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc40c00fe80 con 0x7fc41c101780 2024-05-13T22:53:36.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.428+0000 7fc4137fe700 1 -- 172.21.15.102:0/156607141 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc40c00e5e0 con 0x7fc41c101780 2024-05-13T22:53:36.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.428+0000 7fc4245e9700 1 -- 172.21.15.102:0/156607141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41c1a2910 con 0x7fc41c101780 2024-05-13T22:53:36.407 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.428+0000 7fc4245e9700 1 -- 172.21.15.102:0/156607141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41c1a2d20 con 0x7fc41c101780 2024-05-13T22:53:36.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.429+0000 7fc4245e9700 1 -- 172.21.15.102:0/156607141 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41c109170 con 0x7fc41c101780 2024-05-13T22:53:36.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.429+0000 7fc4137fe700 1 -- 172.21.15.102:0/156607141 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fc40c010460 con 0x7fc41c101780 2024-05-13T22:53:36.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.430+0000 7fc4137fe700 1 --2- 172.21.15.102:0/156607141 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc40806e340 0x7fc408070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:36.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.430+0000 7fc4137fe700 1 -- 172.21.15.102:0/156607141 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc40c08e040 con 0x7fc41c101780 2024-05-13T22:53:36.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.430+0000 7fc422385700 1 --2- 172.21.15.102:0/156607141 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc40806e340 0x7fc408070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:36.408 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.430+0000 7fc422385700 1 --2- 172.21.15.102:0/156607141 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc40806e340 0x7fc408070800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc418005850 tx=0x7fc41801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:36.411 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.435+0000 7fc4137fe700 1 -- 172.21.15.102:0/156607141 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc40c057230 con 0x7fc41c101780 2024-05-13T22:53:36.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.648+0000 7fc4245e9700 1 -- 172.21.15.102:0/156607141 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc41c02cdb0 con 0x7fc40806e340 2024-05-13T22:53:36.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.649+0000 7fc4137fe700 1 -- 172.21.15.102:0/156607141 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fc41c02cdb0 con 0x7fc40806e340 2024-05-13T22:53:36.626 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:53:36.626 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:53:36.626 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:53:36.626 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:53:36.627 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:53:36.627 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr", 2024-05-13T22:53:36.627 INFO:teuthology.orchestra.run.smithi102.stdout: "mon" 2024-05-13T22:53:36.627 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:53:36.627 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "4/23 daemons upgraded", 2024-05-13T22:53:36.627 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading mon daemons", 2024-05-13T22:53:36.627 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:53:36.627 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:53:36.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.652+0000 7fc4245e9700 1 -- 172.21.15.102:0/156607141 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc40806e340 msgr2=0x7fc408070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:36.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.652+0000 7fc4245e9700 1 --2- 172.21.15.102:0/156607141 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc40806e340 0x7fc408070800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc418005850 tx=0x7fc41801a040 comp rx=0 tx=0).stop 2024-05-13T22:53:36.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.652+0000 7fc4245e9700 1 -- 172.21.15.102:0/156607141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc41c101780 msgr2=0x7fc41c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:36.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.652+0000 7fc4245e9700 1 --2- 172.21.15.102:0/156607141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc41c101780 0x7fc41c0735d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc41c06a0d0 tx=0x7fc40c00dbb0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.652+0000 7fc4245e9700 1 -- 172.21.15.102:0/156607141 shutdown_connections 2024-05-13T22:53:36.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.652+0000 7fc4245e9700 1 --2- 172.21.15.102:0/156607141 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc40806e340 0x7fc408070800 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.652+0000 7fc4245e9700 1 --2- 172.21.15.102:0/156607141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc41c100dd0 0x7fc41c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.652+0000 7fc4245e9700 1 --2- 172.21.15.102:0/156607141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc41c101780 0x7fc41c0735d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.652+0000 7fc4245e9700 1 -- 172.21.15.102:0/156607141 >> 172.21.15.102:0/156607141 conn(0x7fc41c0fc640 msgr2=0x7fc41c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:36.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.652+0000 7fc4245e9700 1 -- 172.21.15.102:0/156607141 shutdown_connections 2024-05-13T22:53:36.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.652+0000 7fc4245e9700 1 -- 172.21.15.102:0/156607141 wait complete. 2024-05-13T22:53:36.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.757+0000 7f927a6a2700 1 -- 172.21.15.102:0/3645309061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92740ffee0 msgr2=0x7f9274105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.757+0000 7f927a6a2700 1 --2- 172.21.15.102:0/3645309061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92740ffee0 0x7f9274105870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9268009910 tx=0x7f9268009c20 comp rx=0 tx=0).stop 2024-05-13T22:53:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.757+0000 7f927a6a2700 1 -- 172.21.15.102:0/3645309061 shutdown_connections 2024-05-13T22:53:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.757+0000 7f927a6a2700 1 --2- 172.21.15.102:0/3645309061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92740ffee0 0x7f9274105870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.757+0000 7f927a6a2700 1 --2- 172.21.15.102:0/3645309061 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92740ff530 0x7f92740ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.757+0000 7f927a6a2700 1 -- 172.21.15.102:0/3645309061 >> 172.21.15.102:0/3645309061 conn(0x7f9274074df0 msgr2=0x7f9274075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.757+0000 7f927a6a2700 1 -- 172.21.15.102:0/3645309061 shutdown_connections 2024-05-13T22:53:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.757+0000 7f927a6a2700 1 -- 172.21.15.102:0/3645309061 wait complete. 2024-05-13T22:53:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.757+0000 7f927a6a2700 1 Processor -- start 2024-05-13T22:53:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.757+0000 7f927a6a2700 1 -- start start 2024-05-13T22:53:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f927a6a2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92740ff530 0x7f927419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:36.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f927a6a2700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92740ffee0 0x7f927419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f927a6a2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927419d090 con 0x7f92740ff530 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f927a6a2700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9274196620 con 0x7f92740ffee0 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f9273fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92740ff530 0x7f927419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f9273fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92740ff530 0x7f927419c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:37000/0 (socket says 172.21.15.102:37000) 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f9273fff700 1 -- 172.21.15.102:0/2632214710 learned_addr learned my addr 172.21.15.102:0/2632214710 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f92737fe700 1 --2- 172.21.15.102:0/2632214710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92740ffee0 0x7f927419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f9273fff700 1 -- 172.21.15.102:0/2632214710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92740ffee0 msgr2=0x7f927419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f9273fff700 1 --2- 172.21.15.102:0/2632214710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92740ffee0 0x7f927419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f9273fff700 1 -- 172.21.15.102:0/2632214710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92680095f0 con 0x7f92740ff530 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.758+0000 7f9273fff700 1 --2- 172.21.15.102:0/2632214710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92740ff530 0x7f927419c530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f926400e920 tx=0x7f926400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.759+0000 7f92717fa700 1 -- 172.21.15.102:0/2632214710 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f926400c9d0 con 0x7f92740ff530 2024-05-13T22:53:36.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.759+0000 7f92717fa700 1 -- 172.21.15.102:0/2632214710 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9264010430 con 0x7f92740ff530 2024-05-13T22:53:36.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.759+0000 7f927a6a2700 1 -- 172.21.15.102:0/2632214710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9274196900 con 0x7f92740ff530 2024-05-13T22:53:36.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.759+0000 7f92717fa700 1 -- 172.21.15.102:0/2632214710 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9264018630 con 0x7f92740ff530 2024-05-13T22:53:36.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.759+0000 7f927a6a2700 1 -- 172.21.15.102:0/2632214710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9274196e20 con 0x7f92740ff530 2024-05-13T22:53:36.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.759+0000 7f927a6a2700 1 -- 172.21.15.102:0/2632214710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9274109150 con 0x7f92740ff530 2024-05-13T22:53:36.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.760+0000 7f92717fa700 1 -- 172.21.15.102:0/2632214710 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f92640105a0 con 0x7f92740ff530 2024-05-13T22:53:36.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.761+0000 7f92717fa700 1 --2- 172.21.15.102:0/2632214710 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f925c06e340 0x7f925c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:53:36.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.761+0000 7f92717fa700 1 -- 172.21.15.102:0/2632214710 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9264014070 con 0x7f92740ff530 2024-05-13T22:53:36.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.761+0000 7f92737fe700 1 --2- 172.21.15.102:0/2632214710 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f925c06e340 0x7f925c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:53:36.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.761+0000 7f92737fe700 1 --2- 172.21.15.102:0/2632214710 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f925c06e340 0x7f925c070800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f92680059d0 tx=0x7f9268005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:53:36.742 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:36.765+0000 7f92717fa700 1 -- 172.21.15.102:0/2632214710 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9264053850 con 0x7f92740ff530 2024-05-13T22:53:36.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:36 smithi102 ceph-mon[111694]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:53:36.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:36 smithi102 ceph-mon[111694]: from='client.44109 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:53:36.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:36 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3052591919' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:53:36.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:36 smithi102 ceph-mon[111694]: pgmap v28: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:36.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:36.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:36.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-13T22:53:36.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:36.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:36 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3279147335' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:53:37.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.048+0000 7f927a6a2700 1 -- 172.21.15.102:0/2632214710 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9274197090 con 0x7f92740ff530 2024-05-13T22:53:37.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.048+0000 7f92717fa700 1 -- 172.21.15.102:0/2632214710 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f926405b2d0 con 0x7f92740ff530 2024-05-13T22:53:37.025 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:53:37.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.050+0000 7f927a6a2700 1 -- 172.21.15.102:0/2632214710 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f925c06e340 msgr2=0x7f925c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:37.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.050+0000 7f927a6a2700 1 --2- 172.21.15.102:0/2632214710 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f925c06e340 0x7f925c070800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f92680059d0 tx=0x7f9268005940 comp rx=0 tx=0).stop 2024-05-13T22:53:37.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.050+0000 7f927a6a2700 1 -- 172.21.15.102:0/2632214710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92740ff530 msgr2=0x7f927419c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:53:37.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.050+0000 7f927a6a2700 1 --2- 172.21.15.102:0/2632214710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92740ff530 0x7f927419c530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f926400e920 tx=0x7f926400ec30 comp rx=0 tx=0).stop 2024-05-13T22:53:37.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.050+0000 7f927a6a2700 1 -- 172.21.15.102:0/2632214710 shutdown_connections 2024-05-13T22:53:37.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.050+0000 7f927a6a2700 1 --2- 172.21.15.102:0/2632214710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92740ff530 0x7f927419c530 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:37.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.050+0000 7f927a6a2700 1 --2- 172.21.15.102:0/2632214710 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f925c06e340 0x7f925c070800 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:37.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.050+0000 7f927a6a2700 1 --2- 172.21.15.102:0/2632214710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f92740ffee0 0x7f927419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:53:37.027 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.050+0000 7f927a6a2700 1 -- 172.21.15.102:0/2632214710 >> 172.21.15.102:0/2632214710 conn(0x7f9274074df0 msgr2=0x7f92741050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:53:37.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.050+0000 7f927a6a2700 1 -- 172.21.15.102:0/2632214710 shutdown_connections 2024-05-13T22:53:37.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:53:37.050+0000 7f927a6a2700 1 -- 172.21.15.102:0/2632214710 wait complete. 2024-05-13T22:53:37.097 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:36 smithi130 ceph-mon[84265]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:53:37.097 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:36 smithi130 ceph-mon[84265]: from='client.44109 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:53:37.097 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:36 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3052591919' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:53:37.097 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:36 smithi130 ceph-mon[84265]: pgmap v28: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:37.098 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:37.098 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:37.098 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-13T22:53:37.098 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:37.098 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:36 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3279147335' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:53:38.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:37 smithi130 ceph-mon[84265]: Reconfiguring osd.5 (monmap changed)... 2024-05-13T22:53:38.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:37 smithi130 ceph-mon[84265]: Reconfiguring daemon osd.5 on smithi130 2024-05-13T22:53:38.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:37 smithi130 ceph-mon[84265]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:53:38.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:37 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2632214710' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:53:38.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:37 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:38.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:37 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:38.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:37 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.hzjanp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:53:38.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:37 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:38.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:37 smithi102 ceph-mon[111694]: Reconfiguring osd.5 (monmap changed)... 2024-05-13T22:53:38.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:37 smithi102 ceph-mon[111694]: Reconfiguring daemon osd.5 on smithi130 2024-05-13T22:53:38.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:37 smithi102 ceph-mon[111694]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:53:38.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:37 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2632214710' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:53:38.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:37 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:38.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:37 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:38.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:37 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.hzjanp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:53:38.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:37 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:39.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:38 smithi130 ceph-mon[84265]: Reconfiguring mds.cephfs.smithi130.hzjanp (monmap changed)... 2024-05-13T22:53:39.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:38 smithi130 ceph-mon[84265]: Reconfiguring daemon mds.cephfs.smithi130.hzjanp on smithi130 2024-05-13T22:53:39.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:38 smithi130 ceph-mon[84265]: pgmap v29: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:39.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:38 smithi102 ceph-mon[111694]: Reconfiguring mds.cephfs.smithi130.hzjanp (monmap changed)... 2024-05-13T22:53:39.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:38 smithi102 ceph-mon[111694]: Reconfiguring daemon mds.cephfs.smithi130.hzjanp on smithi130 2024-05-13T22:53:39.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:38 smithi102 ceph-mon[111694]: pgmap v29: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:40.882 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:40.882 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:40.882 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:40 smithi130 ceph-mon[84265]: Reconfiguring mds.cephfs.smithi130.ovlxvg (monmap changed)... 2024-05-13T22:53:40.882 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.ovlxvg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:53:40.882 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:40.882 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:40 smithi130 ceph-mon[84265]: Reconfiguring daemon mds.cephfs.smithi130.ovlxvg on smithi130 2024-05-13T22:53:40.882 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:40 smithi130 ceph-mon[84265]: pgmap v30: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:40.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:40.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:40.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:40 smithi102 ceph-mon[111694]: Reconfiguring mds.cephfs.smithi130.ovlxvg (monmap changed)... 2024-05-13T22:53:40.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.ovlxvg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:53:40.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:40.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:40 smithi102 ceph-mon[111694]: Reconfiguring daemon mds.cephfs.smithi130.ovlxvg on smithi130 2024-05-13T22:53:40.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:40 smithi102 ceph-mon[111694]: pgmap v30: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:42.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:42.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:42.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:53:42.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:53:42.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:53:42.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: Upgrade: Setting container_image for all mon 2024-05-13T22:53:42.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:42.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi102"}]: dispatch 2024-05-13T22:53:42.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi102"}]': finished 2024-05-13T22:53:42.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi130"}]: dispatch 2024-05-13T22:53:42.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi130"}]': finished 2024-05-13T22:53:42.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:42 smithi102 ceph-mon[111694]: pgmap v31: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:53:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:53:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:53:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: Upgrade: Setting container_image for all mon 2024-05-13T22:53:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi102"}]: dispatch 2024-05-13T22:53:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi102"}]': finished 2024-05-13T22:53:42.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi130"}]: dispatch 2024-05-13T22:53:42.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi130"}]': finished 2024-05-13T22:53:42.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:42 smithi130 ceph-mon[84265]: pgmap v31: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:44.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:43 smithi130 ceph-mon[84265]: Upgrade: Updating crash.smithi102 (1/2) 2024-05-13T22:53:44.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:44.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi102", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:53:44.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:44.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:43 smithi130 ceph-mon[84265]: Deploying daemon crash.smithi102 on smithi102 2024-05-13T22:53:44.151 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:43 smithi102 ceph-mon[111694]: Upgrade: Updating crash.smithi102 (1/2) 2024-05-13T22:53:44.152 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:44.152 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi102", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:53:44.152 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:44.152 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:43 smithi102 ceph-mon[111694]: Deploying daemon crash.smithi102 on smithi102 2024-05-13T22:53:45.000 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:44 smithi102 ceph-mon[111694]: pgmap v32: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:45.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:44 smithi130 ceph-mon[84265]: pgmap v32: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:47.369 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:47 smithi130 ceph-mon[84265]: pgmap v33: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:47.369 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:53:47.369 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:47.369 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:47.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:47 smithi102 ceph-mon[111694]: pgmap v33: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:47.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:53:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:49.379 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:49 smithi130 ceph-mon[84265]: pgmap v34: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:49.379 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:49 smithi130 ceph-mon[84265]: Upgrade: Updating crash.smithi130 (2/2) 2024-05-13T22:53:49.379 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:49 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:49.380 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:49 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi130", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:53:49.380 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:49 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:49.380 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:49 smithi130 ceph-mon[84265]: Deploying daemon crash.smithi130 on smithi130 2024-05-13T22:53:49.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:49 smithi102 ceph-mon[111694]: pgmap v34: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:49.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:49 smithi102 ceph-mon[111694]: Upgrade: Updating crash.smithi130 (2/2) 2024-05-13T22:53:49.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:49 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:49.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:49 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi130", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-13T22:53:49.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:49 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:53:49.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:49 smithi102 ceph-mon[111694]: Deploying daemon crash.smithi130 on smithi130 2024-05-13T22:53:51.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:51 smithi130 ceph-mon[84265]: pgmap v35: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:51.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:51 smithi102 ceph-mon[111694]: pgmap v35: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:53.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:52 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:53.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:52 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:53.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:52 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:53:53.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:52 smithi130 ceph-mon[84265]: pgmap v36: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:53.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:52 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:53.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:52 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:53.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:52 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:53:53.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:52 smithi102 ceph-mon[111694]: pgmap v36: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:55.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:55 smithi102 ceph-mon[111694]: pgmap v37: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:55.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:55 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:55.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:55 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:55.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:55 smithi130 ceph-mon[84265]: pgmap v37: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:55.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:55 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:55.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:55 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:57.340 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:56 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:57.341 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:57 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:57.341 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:57 smithi130 ceph-mon[84265]: pgmap v38: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:57.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:56 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:57.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:56 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:57.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:56 smithi102 ceph-mon[111694]: pgmap v38: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:53:58.078 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:58 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:58.079 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:58 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:58.413 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:58 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:58.413 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:58 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:59.000 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:59 smithi102 ceph-mon[111694]: pgmap v39: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:59.000 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:59.000 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:53:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:59.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:59 smithi130 ceph-mon[84265]: pgmap v39: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:53:59.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:53:59.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:53:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:00.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:00 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:00.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:00 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:00.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:00 smithi130 ceph-mon[84265]: pgmap v40: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:00.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:00 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:00.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:00 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:00.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:00 smithi102 ceph-mon[111694]: pgmap v40: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:01.835 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:54:01.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:54:03.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:03.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:03.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:54:03.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:54:03.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:03.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: pgmap v41: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:54:03.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:54:03.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:03.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:03.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: Upgrade: Setting container_image for all crash 2024-05-13T22:54:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi102"}]: dispatch 2024-05-13T22:54:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi102"}]': finished 2024-05-13T22:54:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi130"}]: dispatch 2024-05-13T22:54:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi130"}]': finished 2024-05-13T22:54:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-13T22:54:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-13T22:54:03.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:02 smithi130 ceph-mon[84265]: Upgrade: osd.0 is safe to restart 2024-05-13T22:54:03.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:03.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:03.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:54:03.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:54:03.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:03.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: pgmap v41: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:54:03.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:54:03.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: Upgrade: Setting container_image for all crash 2024-05-13T22:54:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi102"}]: dispatch 2024-05-13T22:54:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi102"}]': finished 2024-05-13T22:54:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi130"}]: dispatch 2024-05-13T22:54:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi130"}]': finished 2024-05-13T22:54:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-13T22:54:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-13T22:54:03.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:02 smithi102 ceph-mon[111694]: Upgrade: osd.0 is safe to restart 2024-05-13T22:54:04.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:04 smithi102 ceph-mon[111694]: Upgrade: Updating osd.0 2024-05-13T22:54:04.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:04 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:04.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:04 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-13T22:54:04.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:04 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:54:04.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:04 smithi102 ceph-mon[111694]: Deploying daemon osd.0 on smithi102 2024-05-13T22:54:04.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:04 smithi102 ceph-mon[111694]: pgmap v42: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:05.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:04 smithi130 ceph-mon[84265]: Upgrade: Updating osd.0 2024-05-13T22:54:05.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:04 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:05.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:04 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-13T22:54:05.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:04 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:54:05.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:04 smithi130 ceph-mon[84265]: Deploying daemon osd.0 on smithi102 2024-05-13T22:54:05.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:04 smithi130 ceph-mon[84265]: pgmap v42: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:07.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.170+0000 7f3cfd698700 1 -- 172.21.15.102:0/3552686075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf8100dc0 msgr2=0x7f3cf81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:07.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.170+0000 7f3cfd698700 1 --2- 172.21.15.102:0/3552686075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf8100dc0 0x7f3cf81011a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009910 tx=0x7f3ce8009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:07.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.170+0000 7f3cfd698700 1 -- 172.21.15.102:0/3552686075 shutdown_connections 2024-05-13T22:54:07.150 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.170+0000 7f3cfd698700 1 --2- 172.21.15.102:0/3552686075 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3cf8101770 0x7f3cf81057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.170+0000 7f3cfd698700 1 --2- 172.21.15.102:0/3552686075 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf8100dc0 0x7f3cf81011a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.170+0000 7f3cfd698700 1 -- 172.21.15.102:0/3552686075 >> 172.21.15.102:0/3552686075 conn(0x7f3cf80fc650 msgr2=0x7f3cf80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.170+0000 7f3cfd698700 1 -- 172.21.15.102:0/3552686075 shutdown_connections 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.170+0000 7f3cfd698700 1 -- 172.21.15.102:0/3552686075 wait complete. 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.171+0000 7f3cfd698700 1 Processor -- start 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.171+0000 7f3cfd698700 1 -- start start 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.171+0000 7f3cfd698700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf8100dc0 0x7f3cf819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.171+0000 7f3cfd698700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3cf8101770 0x7f3cf819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.171+0000 7f3cfd698700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf819d210 con 0x7f3cf8100dc0 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.171+0000 7f3cfd698700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81966c0 con 0x7f3cf8101770 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.171+0000 7f3cf6ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf8100dc0 0x7f3cf819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:07.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cf6ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf8100dc0 0x7f3cf819c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38798/0 (socket says 172.21.15.102:38798) 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cf6ffd700 1 -- 172.21.15.102:0/2087635242 learned_addr learned my addr 172.21.15.102:0/2087635242 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cf67fc700 1 --2- 172.21.15.102:0/2087635242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3cf8101770 0x7f3cf819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cf6ffd700 1 -- 172.21.15.102:0/2087635242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3cf8101770 msgr2=0x7f3cf819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cf6ffd700 1 --2- 172.21.15.102:0/2087635242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3cf8101770 0x7f3cf819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cf6ffd700 1 -- 172.21.15.102:0/2087635242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce80095f0 con 0x7f3cf8100dc0 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cf6ffd700 1 --2- 172.21.15.102:0/2087635242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf8100dc0 0x7f3cf819c640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8006010 tx=0x7f3ce800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cdffff700 1 -- 172.21.15.102:0/2087635242 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce801d070 con 0x7f3cf8100dc0 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cdffff700 1 -- 172.21.15.102:0/2087635242 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ce8004fb0 con 0x7f3cf8100dc0 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cfd698700 1 -- 172.21.15.102:0/2087635242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf8196940 con 0x7f3cf8100dc0 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cdffff700 1 -- 172.21.15.102:0/2087635242 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce8022ea0 con 0x7f3cf8100dc0 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.172+0000 7f3cfd698700 1 -- 172.21.15.102:0/2087635242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf8196e30 con 0x7f3cf8100dc0 2024-05-13T22:54:07.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.173+0000 7f3cfd698700 1 -- 172.21.15.102:0/2087635242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf8109160 con 0x7f3cf8100dc0 2024-05-13T22:54:07.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.174+0000 7f3cdffff700 1 -- 172.21.15.102:0/2087635242 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f3ce8022470 con 0x7f3cf8100dc0 2024-05-13T22:54:07.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.174+0000 7f3cdffff700 1 --2- 172.21.15.102:0/2087635242 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3ce006e340 0x7f3ce0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:07.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.174+0000 7f3cdffff700 1 -- 172.21.15.102:0/2087635242 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3ce808f8f0 con 0x7f3cf8100dc0 2024-05-13T22:54:07.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.174+0000 7f3cf67fc700 1 --2- 172.21.15.102:0/2087635242 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3ce006e340 0x7f3ce0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:07.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.175+0000 7f3cf67fc700 1 --2- 172.21.15.102:0/2087635242 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3ce006e340 0x7f3ce0070800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3cec006fd0 tx=0x7f3cec008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:07.155 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.179+0000 7f3cdffff700 1 -- 172.21.15.102:0/2087635242 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3ce805c640 con 0x7f3cf8100dc0 2024-05-13T22:54:07.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:07 smithi102 ceph-mon[111694]: pgmap v43: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:07.176 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:06 smithi102 systemd[1]: Stopping Ceph osd.0 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:54:07.378 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.401+0000 7f3cfd698700 1 -- 172.21.15.102:0/2087635242 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cf802cce0 con 0x7f3ce006e340 2024-05-13T22:54:07.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.403+0000 7f3cdffff700 1 -- 172.21.15.102:0/2087635242 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3cf802cce0 con 0x7f3ce006e340 2024-05-13T22:54:07.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.405+0000 7f3cfd698700 1 -- 172.21.15.102:0/2087635242 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3ce006e340 msgr2=0x7f3ce0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:07.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.405+0000 7f3cfd698700 1 --2- 172.21.15.102:0/2087635242 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3ce006e340 0x7f3ce0070800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3cec006fd0 tx=0x7f3cec008040 comp rx=0 tx=0).stop 2024-05-13T22:54:07.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.405+0000 7f3cfd698700 1 -- 172.21.15.102:0/2087635242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf8100dc0 msgr2=0x7f3cf819c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:07.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.405+0000 7f3cfd698700 1 --2- 172.21.15.102:0/2087635242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf8100dc0 0x7f3cf819c640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8006010 tx=0x7f3ce800be20 comp rx=0 tx=0).stop 2024-05-13T22:54:07.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.405+0000 7f3cfd698700 1 -- 172.21.15.102:0/2087635242 shutdown_connections 2024-05-13T22:54:07.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.405+0000 7f3cfd698700 1 --2- 172.21.15.102:0/2087635242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cf8100dc0 0x7f3cf819c640 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.405+0000 7f3cfd698700 1 --2- 172.21.15.102:0/2087635242 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3ce006e340 0x7f3ce0070800 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.405+0000 7f3cfd698700 1 --2- 172.21.15.102:0/2087635242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3cf8101770 0x7f3cf819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.405+0000 7f3cfd698700 1 -- 172.21.15.102:0/2087635242 >> 172.21.15.102:0/2087635242 conn(0x7f3cf80fc650 msgr2=0x7f3cf80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:07.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.405+0000 7f3cfd698700 1 -- 172.21.15.102:0/2087635242 shutdown_connections 2024-05-13T22:54:07.382 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.405+0000 7f3cfd698700 1 -- 172.21.15.102:0/2087635242 wait complete. 2024-05-13T22:54:07.392 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:54:07.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:07 smithi130 ceph-mon[84265]: pgmap v43: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:07.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.510+0000 7f71e673f700 1 -- 172.21.15.102:0/2588851332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e0105a40 msgr2=0x7f71e0105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:07.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.510+0000 7f71e673f700 1 --2- 172.21.15.102:0/2588851332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e0105a40 0x7f71e0105e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009910 tx=0x7f71d0009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:07.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.510+0000 7f71e673f700 1 -- 172.21.15.102:0/2588851332 shutdown_connections 2024-05-13T22:54:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.510+0000 7f71e673f700 1 --2- 172.21.15.102:0/2588851332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e00ffa40 0x7f71e00ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.510+0000 7f71e673f700 1 --2- 172.21.15.102:0/2588851332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e0105a40 0x7f71e0105e20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.510+0000 7f71e673f700 1 -- 172.21.15.102:0/2588851332 >> 172.21.15.102:0/2588851332 conn(0x7f71e00fb570 msgr2=0x7f71e00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.510+0000 7f71e673f700 1 -- 172.21.15.102:0/2588851332 shutdown_connections 2024-05-13T22:54:07.488 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.510+0000 7f71e673f700 1 -- 172.21.15.102:0/2588851332 wait complete. 2024-05-13T22:54:07.488 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:07 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[47079]: 2024-05-13T22:54:07.204+0000 7f66f665c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T22:54:07.488 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:07 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[47079]: 2024-05-13T22:54:07.204+0000 7f66f665c700 -1 osd.0 50 *** Got signal Terminated *** 2024-05-13T22:54:07.488 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:07 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[47079]: 2024-05-13T22:54:07.204+0000 7f66f665c700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T22:54:07.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71e673f700 1 Processor -- start 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71e673f700 1 -- start start 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71e673f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e00ffa40 0x7f71e0195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71e673f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e0105a40 0x7f71e0195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71e673f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e0196270 con 0x7f71e0105a40 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71e673f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e019a000 con 0x7f71e00ffa40 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71d7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e0105a40 0x7f71e0195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71d7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e0105a40 0x7f71e0195b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38822/0 (socket says 172.21.15.102:38822) 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71d7fff700 1 -- 172.21.15.102:0/638380325 learned_addr learned my addr 172.21.15.102:0/638380325 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71dffff700 1 --2- 172.21.15.102:0/638380325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e00ffa40 0x7f71e0195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71d7fff700 1 -- 172.21.15.102:0/638380325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e00ffa40 msgr2=0x7f71e0195650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.511+0000 7f71d7fff700 1 --2- 172.21.15.102:0/638380325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e00ffa40 0x7f71e0195650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.512+0000 7f71d7fff700 1 -- 172.21.15.102:0/638380325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d00095f0 con 0x7f71e0105a40 2024-05-13T22:54:07.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.512+0000 7f71d7fff700 1 --2- 172.21.15.102:0/638380325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e0105a40 0x7f71e0195b90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71c800d8d0 tx=0x7f71c800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.512+0000 7f71ddffb700 1 -- 172.21.15.102:0/638380325 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71c80098a0 con 0x7f71e0105a40 2024-05-13T22:54:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.512+0000 7f71ddffb700 1 -- 172.21.15.102:0/638380325 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71c8010460 con 0x7f71e0105a40 2024-05-13T22:54:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.512+0000 7f71e673f700 1 -- 172.21.15.102:0/638380325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e019a2e0 con 0x7f71e0105a40 2024-05-13T22:54:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.512+0000 7f71ddffb700 1 -- 172.21.15.102:0/638380325 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71c800f5f0 con 0x7f71e0105a40 2024-05-13T22:54:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.512+0000 7f71e673f700 1 -- 172.21.15.102:0/638380325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e019a830 con 0x7f71e0105a40 2024-05-13T22:54:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.513+0000 7f71e673f700 1 -- 172.21.15.102:0/638380325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e0108040 con 0x7f71e0105a40 2024-05-13T22:54:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.513+0000 7f71ddffb700 1 -- 172.21.15.102:0/638380325 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f71c8009a00 con 0x7f71e0105a40 2024-05-13T22:54:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.514+0000 7f71ddffb700 1 --2- 172.21.15.102:0/638380325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f71cc06e330 0x7f71cc0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.514+0000 7f71ddffb700 1 -- 172.21.15.102:0/638380325 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f71c808e250 con 0x7f71e0105a40 2024-05-13T22:54:07.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.514+0000 7f71dffff700 1 --2- 172.21.15.102:0/638380325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f71cc06e330 0x7f71cc0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:07.493 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.514+0000 7f71dffff700 1 --2- 172.21.15.102:0/638380325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f71cc06e330 0x7f71cc0707f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f71d000b5c0 tx=0x7f71d001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:07.495 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.518+0000 7f71ddffb700 1 -- 172.21.15.102:0/638380325 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f71c80573c0 con 0x7f71e0105a40 2024-05-13T22:54:07.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.734+0000 7f71e673f700 1 -- 172.21.15.102:0/638380325 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71e019ab40 con 0x7f71cc06e330 2024-05-13T22:54:07.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.735+0000 7f71ddffb700 1 -- 172.21.15.102:0/638380325 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f71e019ab40 con 0x7f71cc06e330 2024-05-13T22:54:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.737+0000 7f71e673f700 1 -- 172.21.15.102:0/638380325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f71cc06e330 msgr2=0x7f71cc0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.737+0000 7f71e673f700 1 --2- 172.21.15.102:0/638380325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f71cc06e330 0x7f71cc0707f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f71d000b5c0 tx=0x7f71d001a040 comp rx=0 tx=0).stop 2024-05-13T22:54:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.737+0000 7f71e673f700 1 -- 172.21.15.102:0/638380325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e0105a40 msgr2=0x7f71e0195b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.737+0000 7f71e673f700 1 --2- 172.21.15.102:0/638380325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e0105a40 0x7f71e0195b90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71c800d8d0 tx=0x7f71c800dbe0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.737+0000 7f71e673f700 1 -- 172.21.15.102:0/638380325 shutdown_connections 2024-05-13T22:54:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.737+0000 7f71e673f700 1 --2- 172.21.15.102:0/638380325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f71cc06e330 0x7f71cc0707f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.737+0000 7f71e673f700 1 --2- 172.21.15.102:0/638380325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71e00ffa40 0x7f71e0195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.737+0000 7f71e673f700 1 --2- 172.21.15.102:0/638380325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71e0105a40 0x7f71e0195b90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.737+0000 7f71e673f700 1 -- 172.21.15.102:0/638380325 >> 172.21.15.102:0/638380325 conn(0x7f71e00fb570 msgr2=0x7f71e00fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:07.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.737+0000 7f71e673f700 1 -- 172.21.15.102:0/638380325 shutdown_connections 2024-05-13T22:54:07.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.737+0000 7f71e673f700 1 -- 172.21.15.102:0/638380325 wait complete. 2024-05-13T22:54:07.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.842+0000 7f07e3f91700 1 -- 172.21.15.102:0/2180321418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07dc105a40 msgr2=0x7f07dc105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:07.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.842+0000 7f07e3f91700 1 --2- 172.21.15.102:0/2180321418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07dc105a40 0x7f07dc105e20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009910 tx=0x7f07d8009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:07.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.842+0000 7f07e3f91700 1 -- 172.21.15.102:0/2180321418 shutdown_connections 2024-05-13T22:54:07.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.842+0000 7f07e3f91700 1 --2- 172.21.15.102:0/2180321418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07dc0ffa40 0x7f07dc0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.842+0000 7f07e3f91700 1 --2- 172.21.15.102:0/2180321418 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07dc105a40 0x7f07dc105e20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.842+0000 7f07e3f91700 1 -- 172.21.15.102:0/2180321418 >> 172.21.15.102:0/2180321418 conn(0x7f07dc0fb570 msgr2=0x7f07dc0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:07.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.842+0000 7f07e3f91700 1 -- 172.21.15.102:0/2180321418 shutdown_connections 2024-05-13T22:54:07.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.842+0000 7f07e3f91700 1 -- 172.21.15.102:0/2180321418 wait complete. 2024-05-13T22:54:07.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e3f91700 1 Processor -- start 2024-05-13T22:54:07.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e3f91700 1 -- start start 2024-05-13T22:54:07.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e3f91700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07dc0ffa40 0x7f07dc195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:07.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e3f91700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07dc105a40 0x7f07dc195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:07.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e3f91700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc196270 con 0x7f07dc0ffa40 2024-05-13T22:54:07.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e3f91700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc19a000 con 0x7f07dc105a40 2024-05-13T22:54:07.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e1d2d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07dc0ffa40 0x7f07dc195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:07.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e1d2d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07dc0ffa40 0x7f07dc195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38830/0 (socket says 172.21.15.102:38830) 2024-05-13T22:54:07.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e1d2d700 1 -- 172.21.15.102:0/579403454 learned_addr learned my addr 172.21.15.102:0/579403454 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:07.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e152c700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07dc105a40 0x7f07dc195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:07.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e1d2d700 1 -- 172.21.15.102:0/579403454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07dc105a40 msgr2=0x7f07dc195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:07.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e1d2d700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07dc105a40 0x7f07dc195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:07.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e1d2d700 1 -- 172.21.15.102:0/579403454 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d80095f0 con 0x7f07dc0ffa40 2024-05-13T22:54:07.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.843+0000 7f07e152c700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07dc105a40 0x7f07dc195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:54:07.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.844+0000 7f07e1d2d700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07dc0ffa40 0x7f07dc195650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07d8006010 tx=0x7f07d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:07.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.844+0000 7f07d2ffd700 1 -- 172.21.15.102:0/579403454 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07d801f070 con 0x7f07dc0ffa40 2024-05-13T22:54:07.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.844+0000 7f07d2ffd700 1 -- 172.21.15.102:0/579403454 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07d8003680 con 0x7f07dc0ffa40 2024-05-13T22:54:07.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.844+0000 7f07e3f91700 1 -- 172.21.15.102:0/579403454 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc19a280 con 0x7f07dc0ffa40 2024-05-13T22:54:07.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.844+0000 7f07d2ffd700 1 -- 172.21.15.102:0/579403454 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07d800f7f0 con 0x7f07dc0ffa40 2024-05-13T22:54:07.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.844+0000 7f07e3f91700 1 -- 172.21.15.102:0/579403454 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc19a770 con 0x7f07dc0ffa40 2024-05-13T22:54:07.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.845+0000 7f07e3f91700 1 -- 172.21.15.102:0/579403454 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc04ea90 con 0x7f07dc0ffa40 2024-05-13T22:54:07.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.846+0000 7f07d2ffd700 1 -- 172.21.15.102:0/579403454 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f07d80037f0 con 0x7f07dc0ffa40 2024-05-13T22:54:07.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.846+0000 7f07d2ffd700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f07c806e340 0x7f07c8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:07.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.846+0000 7f07d2ffd700 1 -- 172.21.15.102:0/579403454 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f07d8091870 con 0x7f07dc0ffa40 2024-05-13T22:54:07.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.846+0000 7f07e152c700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f07c806e340 0x7f07c8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:07.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.846+0000 7f07e152c700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f07c806e340 0x7f07c8070800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07cc005fd0 tx=0x7f07cc005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:07.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:07.851+0000 7f07d2ffd700 1 -- 172.21.15.102:0/579403454 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f07d805a9e0 con 0x7f07dc0ffa40 2024-05-13T22:54:08.034 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.057+0000 7f07e3f91700 1 -- 172.21.15.102:0/579403454 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f07dc19ac70 con 0x7f07c806e340 2024-05-13T22:54:08.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.067+0000 7f07d2ffd700 1 -- 172.21.15.102:0/579403454 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f07dc19ac70 con 0x7f07c806e340 2024-05-13T22:54:08.044 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:54:08.044 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (3m) 11s ago 14m 26.7M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:54:08.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (14m) 11s ago 14m 8250k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:54:08.044 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (13m) 13s ago 13m 7956k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (21s) 11s ago 14m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (16s) 13s ago 13m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (2m) 11s ago 14m 79.1M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (8m) 11s ago 8m 88.0M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (8m) 11s ago 8m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (8m) 13s ago 8m 16.1M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (8m) 13s ago 8m 19.1M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (5m) 11s ago 15m 500M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (4m) 13s ago 13m 451M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (92s) 11s ago 15m 45.7M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (65s) 13s ago 13m 41.6M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (4m) 11s ago 14m 16.4M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (3m) 13s ago 13m 17.1M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:54:08.045 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (12m) 11s ago 12m 225M 1216M 18.2.1 5be31c24972a 0f1608e57bb9 2024-05-13T22:54:08.046 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (12m) 11s ago 12m 240M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:54:08.046 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (11m) 11s ago 11m 204M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:54:08.046 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (11m) 13s ago 11m 293M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:54:08.046 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (10m) 13s ago 10m 245M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:54:08.046 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (10m) 13s ago 10m 227M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:54:08.046 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (3m) 11s ago 13m 69.4M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:54:08.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.069+0000 7f07e3f91700 1 -- 172.21.15.102:0/579403454 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f07c806e340 msgr2=0x7f07c8070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.069+0000 7f07e3f91700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f07c806e340 0x7f07c8070800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07cc005fd0 tx=0x7f07cc005dc0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.069+0000 7f07e3f91700 1 -- 172.21.15.102:0/579403454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07dc0ffa40 msgr2=0x7f07dc195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.069+0000 7f07e3f91700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07dc0ffa40 0x7f07dc195650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07d8006010 tx=0x7f07d800be20 comp rx=0 tx=0).stop 2024-05-13T22:54:08.047 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.069+0000 7f07e3f91700 1 -- 172.21.15.102:0/579403454 shutdown_connections 2024-05-13T22:54:08.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.069+0000 7f07e3f91700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07dc0ffa40 0x7f07dc195650 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.069+0000 7f07e3f91700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f07c806e340 0x7f07c8070800 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.069+0000 7f07e3f91700 1 --2- 172.21.15.102:0/579403454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07dc105a40 0x7f07dc195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.070+0000 7f07e3f91700 1 -- 172.21.15.102:0/579403454 >> 172.21.15.102:0/579403454 conn(0x7f07dc0fb570 msgr2=0x7f07dc0fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:08.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.070+0000 7f07e3f91700 1 -- 172.21.15.102:0/579403454 shutdown_connections 2024-05-13T22:54:08.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.070+0000 7f07e3f91700 1 -- 172.21.15.102:0/579403454 wait complete. 2024-05-13T22:54:08.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.182+0000 7f48ad914700 1 -- 172.21.15.102:0/1107047653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48a8100da0 msgr2=0x7f48a8101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.182+0000 7f48ad914700 1 --2- 172.21.15.102:0/1107047653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48a8100da0 0x7f48a8101180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f489800b0a0 tx=0x7f489800b3b0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.182+0000 7f48ad914700 1 -- 172.21.15.102:0/1107047653 shutdown_connections 2024-05-13T22:54:08.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.182+0000 7f48ad914700 1 --2- 172.21.15.102:0/1107047653 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48a8101750 0x7f48a81057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.182+0000 7f48ad914700 1 --2- 172.21.15.102:0/1107047653 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48a8100da0 0x7f48a8101180 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.159 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.182+0000 7f48ad914700 1 -- 172.21.15.102:0/1107047653 >> 172.21.15.102:0/1107047653 conn(0x7f48a80fc630 msgr2=0x7f48a80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:08.160 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:08 smithi102 ceph-mon[111694]: osd.0 marked itself down and dead 2024-05-13T22:54:08.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.182+0000 7f48ad914700 1 -- 172.21.15.102:0/1107047653 shutdown_connections 2024-05-13T22:54:08.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.182+0000 7f48ad914700 1 -- 172.21.15.102:0/1107047653 wait complete. 2024-05-13T22:54:08.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.183+0000 7f48ad914700 1 Processor -- start 2024-05-13T22:54:08.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.183+0000 7f48ad914700 1 -- start start 2024-05-13T22:54:08.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.183+0000 7f48ad914700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48a8100da0 0x7f48a8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:08.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.183+0000 7f48ad914700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48a8101750 0x7f48a80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:08.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.183+0000 7f48ad914700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8073b10 con 0x7f48a8101750 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.183+0000 7f48ad914700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8073c50 con 0x7f48a8100da0 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48a67fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48a8101750 0x7f48a80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48a67fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48a8101750 0x7f48a80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:38844/0 (socket says 172.21.15.102:38844) 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48a67fc700 1 -- 172.21.15.102:0/673653616 learned_addr learned my addr 172.21.15.102:0/673653616 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48a6ffd700 1 --2- 172.21.15.102:0/673653616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48a8100da0 0x7f48a8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48a67fc700 1 -- 172.21.15.102:0/673653616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48a8100da0 msgr2=0x7f48a8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48a67fc700 1 --2- 172.21.15.102:0/673653616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48a8100da0 0x7f48a8073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48a67fc700 1 -- 172.21.15.102:0/673653616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4898009d00 con 0x7f48a8101750 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48a67fc700 1 --2- 172.21.15.102:0/673653616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48a8101750 0x7f48a80735d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f489000d900 tx=0x7f489000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48ac912700 1 -- 172.21.15.102:0/673653616 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4890009900 con 0x7f48a8101750 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48ac912700 1 -- 172.21.15.102:0/673653616 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4890010460 con 0x7f48a8101750 2024-05-13T22:54:08.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48ad914700 1 -- 172.21.15.102:0/673653616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a81a29a0 con 0x7f48a8101750 2024-05-13T22:54:08.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48ac912700 1 -- 172.21.15.102:0/673653616 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f489000f5f0 con 0x7f48a8101750 2024-05-13T22:54:08.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.184+0000 7f48ad914700 1 -- 172.21.15.102:0/673653616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a81a2db0 con 0x7f48a8101750 2024-05-13T22:54:08.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.186+0000 7f48ad914700 1 -- 172.21.15.102:0/673653616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a8109140 con 0x7f48a8101750 2024-05-13T22:54:08.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.186+0000 7f48ac912700 1 -- 172.21.15.102:0/673653616 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f48900105d0 con 0x7f48a8101750 2024-05-13T22:54:08.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.187+0000 7f48ac912700 1 --2- 172.21.15.102:0/673653616 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f489406e340 0x7f4894070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:08.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.187+0000 7f48ac912700 1 -- 172.21.15.102:0/673653616 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f489008de20 con 0x7f48a8101750 2024-05-13T22:54:08.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.187+0000 7f48a6ffd700 1 --2- 172.21.15.102:0/673653616 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f489406e340 0x7f4894070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:08.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.187+0000 7f48a6ffd700 1 --2- 172.21.15.102:0/673653616 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f489406e340 0x7f4894070800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4898000f80 tx=0x7f48980060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:08.168 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.191+0000 7f48ac912700 1 -- 172.21.15.102:0/673653616 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4890057010 con 0x7f48a8101750 2024-05-13T22:54:08.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:08 smithi130 ceph-mon[84265]: osd.0 marked itself down and dead 2024-05-13T22:54:08.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.477+0000 7f48ad914700 1 -- 172.21.15.102:0/673653616 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f48a804ea90 con 0x7f48a8101750 2024-05-13T22:54:08.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.478+0000 7f48ac912700 1 -- 172.21.15.102:0/673653616 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+687 (secure 0 0 0) 0x7f48900172c0 con 0x7f48a8101750 2024-05-13T22:54:08.455 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:54:08.455 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:54:08.455 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:54:08.455 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:54:08.455 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:54:08.455 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:54:08.455 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:54:08.456 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:54:08.456 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5 2024-05-13T22:54:08.456 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:54:08.456 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:54:08.456 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:54:08.456 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:54:08.456 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:54:08.456 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-05-13T22:54:08.456 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 4 2024-05-13T22:54:08.456 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:54:08.456 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:54:08.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.480+0000 7f48ad914700 1 -- 172.21.15.102:0/673653616 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f489406e340 msgr2=0x7f4894070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.480+0000 7f48ad914700 1 --2- 172.21.15.102:0/673653616 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f489406e340 0x7f4894070800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4898000f80 tx=0x7f48980060c0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.480+0000 7f48ad914700 1 -- 172.21.15.102:0/673653616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48a8101750 msgr2=0x7f48a80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.480+0000 7f48ad914700 1 --2- 172.21.15.102:0/673653616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48a8101750 0x7f48a80735d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f489000d900 tx=0x7f489000dc10 comp rx=0 tx=0).stop 2024-05-13T22:54:08.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.480+0000 7f48ad914700 1 -- 172.21.15.102:0/673653616 shutdown_connections 2024-05-13T22:54:08.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.480+0000 7f48ad914700 1 --2- 172.21.15.102:0/673653616 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f489406e340 0x7f4894070800 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.480+0000 7f48ad914700 1 --2- 172.21.15.102:0/673653616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f48a8100da0 0x7f48a8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.480+0000 7f48ad914700 1 --2- 172.21.15.102:0/673653616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f48a8101750 0x7f48a80735d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.480+0000 7f48ad914700 1 -- 172.21.15.102:0/673653616 >> 172.21.15.102:0/673653616 conn(0x7f48a80fc630 msgr2=0x7f48a80fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:08.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.480+0000 7f48ad914700 1 -- 172.21.15.102:0/673653616 shutdown_connections 2024-05-13T22:54:08.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.481+0000 7f48ad914700 1 -- 172.21.15.102:0/673653616 wait complete. 2024-05-13T22:54:08.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.594+0000 7f07877ed700 1 -- 172.21.15.102:0/888318917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0780100da0 msgr2=0x7f0780101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.594+0000 7f07877ed700 1 --2- 172.21.15.102:0/888318917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0780100da0 0x7f0780101180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f077c009910 tx=0x7f077c009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:08.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.595+0000 7f07877ed700 1 -- 172.21.15.102:0/888318917 shutdown_connections 2024-05-13T22:54:08.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.595+0000 7f07877ed700 1 --2- 172.21.15.102:0/888318917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0780101750 0x7f07801057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.595+0000 7f07877ed700 1 --2- 172.21.15.102:0/888318917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0780100da0 0x7f0780101180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.595+0000 7f07877ed700 1 -- 172.21.15.102:0/888318917 >> 172.21.15.102:0/888318917 conn(0x7f07800fc630 msgr2=0x7f07800fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:08.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.595+0000 7f07877ed700 1 -- 172.21.15.102:0/888318917 shutdown_connections 2024-05-13T22:54:08.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.595+0000 7f07877ed700 1 -- 172.21.15.102:0/888318917 wait complete. 2024-05-13T22:54:08.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.595+0000 7f07877ed700 1 Processor -- start 2024-05-13T22:54:08.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.595+0000 7f07877ed700 1 -- start start 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f07877ed700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0780100da0 0x7f078019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f07877ed700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0780101750 0x7f078019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f07877ed700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078019d170 con 0x7f0780100da0 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f07877ed700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0780196620 con 0x7f0780101750 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f0785589700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0780100da0 0x7f078019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f0785589700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0780100da0 0x7f078019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47038/0 (socket says 172.21.15.102:47038) 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f0785589700 1 -- 172.21.15.102:0/4253128597 learned_addr learned my addr 172.21.15.102:0/4253128597 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f0784d88700 1 --2- 172.21.15.102:0/4253128597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0780101750 0x7f078019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f0785589700 1 -- 172.21.15.102:0/4253128597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0780101750 msgr2=0x7f078019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f0785589700 1 --2- 172.21.15.102:0/4253128597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0780101750 0x7f078019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f0785589700 1 -- 172.21.15.102:0/4253128597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c0095f0 con 0x7f0780100da0 2024-05-13T22:54:08.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f0785589700 1 --2- 172.21.15.102:0/4253128597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0780100da0 0x7f078019c5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f077c006010 tx=0x7f077c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:08.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.596+0000 7f07767fc700 1 -- 172.21.15.102:0/4253128597 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f077c01d070 con 0x7f0780100da0 2024-05-13T22:54:08.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.597+0000 7f07877ed700 1 -- 172.21.15.102:0/4253128597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07801968a0 con 0x7f0780100da0 2024-05-13T22:54:08.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.597+0000 7f07767fc700 1 -- 172.21.15.102:0/4253128597 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f077c00f460 con 0x7f0780100da0 2024-05-13T22:54:08.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.597+0000 7f07767fc700 1 -- 172.21.15.102:0/4253128597 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f077c017610 con 0x7f0780100da0 2024-05-13T22:54:08.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.597+0000 7f07877ed700 1 -- 172.21.15.102:0/4253128597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0780196d90 con 0x7f0780100da0 2024-05-13T22:54:08.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.598+0000 7f07877ed700 1 -- 172.21.15.102:0/4253128597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0780109140 con 0x7f0780100da0 2024-05-13T22:54:08.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.599+0000 7f07767fc700 1 -- 172.21.15.102:0/4253128597 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f077c00f5d0 con 0x7f0780100da0 2024-05-13T22:54:08.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.599+0000 7f07767fc700 1 --2- 172.21.15.102:0/4253128597 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f076c06e340 0x7f076c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:08.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.599+0000 7f07767fc700 1 -- 172.21.15.102:0/4253128597 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f077c08f910 con 0x7f0780100da0 2024-05-13T22:54:08.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.599+0000 7f0784d88700 1 --2- 172.21.15.102:0/4253128597 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f076c06e340 0x7f076c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:08.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.600+0000 7f0784d88700 1 --2- 172.21.15.102:0/4253128597 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f076c06e340 0x7f076c070800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f07800fcdb0 tx=0x7f077000a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:08.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.604+0000 7f07767fc700 1 -- 172.21.15.102:0/4253128597 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f077c05c5b0 con 0x7f0780100da0 2024-05-13T22:54:08.838 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.861+0000 7f07877ed700 1 -- 172.21.15.102:0/4253128597 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f078004ea90 con 0x7f0780100da0 2024-05-13T22:54:08.838 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:08 smithi102 podman[123015]: 2024-05-13 22:54:08.670170503 +0000 UTC m=+1.585386873 container died 0f1608e57bb9b530b6974ab8d56e18eaa2c8823859880bef39f8495b14c6338d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-13T22:54:08.839 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.861+0000 7f07767fc700 1 -- 172.21.15.102:0/4253128597 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1909 (secure 0 0 0) 0x7f077c05c140 con 0x7f0780100da0 2024-05-13T22:54:08.839 INFO:teuthology.orchestra.run.smithi102.stdout:e17 2024-05-13T22:54:08.839 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:54:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:54:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:54:08.840 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:54:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:54:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:54:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 17 2024-05-13T22:54:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:54:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:54:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:54:08.840 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:54:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:54:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:54:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:54:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:54:08.841 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:54:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:54:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:54:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 47 2024-05-13T22:54:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:54:08.842 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=14498} 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 0 members: 2024-05-13T22:54:08.843 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:54:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:54:08.845 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:54:08.845 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:54:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:54:08.845 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:54:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:54:08.845 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:54:08.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.865+0000 7f07877ed700 1 -- 172.21.15.102:0/4253128597 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f076c06e340 msgr2=0x7f076c070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.865+0000 7f07877ed700 1 --2- 172.21.15.102:0/4253128597 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f076c06e340 0x7f076c070800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f07800fcdb0 tx=0x7f077000a300 comp rx=0 tx=0).stop 2024-05-13T22:54:08.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.865+0000 7f07877ed700 1 -- 172.21.15.102:0/4253128597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0780100da0 msgr2=0x7f078019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.865+0000 7f07877ed700 1 --2- 172.21.15.102:0/4253128597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0780100da0 0x7f078019c5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f077c006010 tx=0x7f077c00bd70 comp rx=0 tx=0).stop 2024-05-13T22:54:08.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.865+0000 7f07877ed700 1 -- 172.21.15.102:0/4253128597 shutdown_connections 2024-05-13T22:54:08.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.865+0000 7f07877ed700 1 --2- 172.21.15.102:0/4253128597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0780100da0 0x7f078019c5a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.865+0000 7f07877ed700 1 --2- 172.21.15.102:0/4253128597 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f076c06e340 0x7f076c070800 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.865+0000 7f07877ed700 1 --2- 172.21.15.102:0/4253128597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0780101750 0x7f078019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.865+0000 7f07877ed700 1 -- 172.21.15.102:0/4253128597 >> 172.21.15.102:0/4253128597 conn(0x7f07800fc630 msgr2=0x7f07800fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:08.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.865+0000 7f07877ed700 1 -- 172.21.15.102:0/4253128597 shutdown_connections 2024-05-13T22:54:08.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.865+0000 7f07877ed700 1 -- 172.21.15.102:0/4253128597 wait complete. 2024-05-13T22:54:08.847 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:54:08.958 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.981+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/890530058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a28068670 msgr2=0x7f1a28068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.981+0000 7f1a2ef5f700 1 --2- 172.21.15.102:0/890530058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a28068670 0x7f1a28068a50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009910 tx=0x7f1a1c009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:08.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.981+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/890530058 shutdown_connections 2024-05-13T22:54:08.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.981+0000 7f1a2ef5f700 1 --2- 172.21.15.102:0/890530058 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a28069020 0x7f1a28105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.981+0000 7f1a2ef5f700 1 --2- 172.21.15.102:0/890530058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a28068670 0x7f1a28068a50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.982+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/890530058 >> 172.21.15.102:0/890530058 conn(0x7f1a280755a0 msgr2=0x7f1a280759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:08.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.982+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/890530058 shutdown_connections 2024-05-13T22:54:08.959 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.982+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/890530058 wait complete. 2024-05-13T22:54:08.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.982+0000 7f1a2ef5f700 1 Processor -- start 2024-05-13T22:54:08.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.982+0000 7f1a2ef5f700 1 -- start start 2024-05-13T22:54:08.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a2ef5f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a28068670 0x7f1a2819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:08.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a2ef5f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a28069020 0x7f1a2819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:08.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a2ef5f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2819d0e0 con 0x7f1a28068670 2024-05-13T22:54:08.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a2ef5f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a28196590 con 0x7f1a28069020 2024-05-13T22:54:08.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a2ccfb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a28068670 0x7f1a2819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:08.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a2ccfb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a28068670 0x7f1a2819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47062/0 (socket says 172.21.15.102:47062) 2024-05-13T22:54:08.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a2ccfb700 1 -- 172.21.15.102:0/3910518594 learned_addr learned my addr 172.21.15.102:0/3910518594 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:08.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a27fff700 1 --2- 172.21.15.102:0/3910518594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a28069020 0x7f1a2819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:08.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a2ccfb700 1 -- 172.21.15.102:0/3910518594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a28069020 msgr2=0x7f1a2819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:08.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a2ccfb700 1 --2- 172.21.15.102:0/3910518594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a28069020 0x7f1a2819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:08.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a2ccfb700 1 -- 172.21.15.102:0/3910518594 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c0095f0 con 0x7f1a28068670 2024-05-13T22:54:08.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a2ccfb700 1 --2- 172.21.15.102:0/3910518594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a28068670 0x7f1a2819c510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0092e0 tx=0x7f1a1c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:08.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.983+0000 7f1a25ffb700 1 -- 172.21.15.102:0/3910518594 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1c01d070 con 0x7f1a28068670 2024-05-13T22:54:08.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.984+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/3910518594 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a28196810 con 0x7f1a28068670 2024-05-13T22:54:08.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.984+0000 7f1a25ffb700 1 -- 172.21.15.102:0/3910518594 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a1c005e00 con 0x7f1a28068670 2024-05-13T22:54:08.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.984+0000 7f1a25ffb700 1 -- 172.21.15.102:0/3910518594 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1c017400 con 0x7f1a28068670 2024-05-13T22:54:08.962 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.984+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/3910518594 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a28196d00 con 0x7f1a28068670 2024-05-13T22:54:08.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.985+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/3910518594 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a28109130 con 0x7f1a28068670 2024-05-13T22:54:08.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.985+0000 7f1a25ffb700 1 -- 172.21.15.102:0/3910518594 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f1a1c003a60 con 0x7f1a28068670 2024-05-13T22:54:08.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.986+0000 7f1a25ffb700 1 --2- 172.21.15.102:0/3910518594 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a1806e330 0x7f1a180707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:08.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.986+0000 7f1a25ffb700 1 -- 172.21.15.102:0/3910518594 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1a1c08f870 con 0x7f1a28068670 2024-05-13T22:54:08.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.986+0000 7f1a27fff700 1 --2- 172.21.15.102:0/3910518594 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a1806e330 0x7f1a180707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:08.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.986+0000 7f1a27fff700 1 --2- 172.21.15.102:0/3910518594 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a1806e330 0x7f1a180707f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1a14005950 tx=0x7f1a140058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:08.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:08.995+0000 7f1a25ffb700 1 -- 172.21.15.102:0/3910518594 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1a1c05c510 con 0x7f1a28068670 2024-05-13T22:54:09.090 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:09 smithi102 ceph-mon[111694]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:09.090 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:09 smithi102 ceph-mon[111694]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:09.090 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:09 smithi102 ceph-mon[111694]: pgmap v44: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:54:09.090 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:09 smithi102 ceph-mon[111694]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:09.090 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:09 smithi102 ceph-mon[111694]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:54:09.091 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:09 smithi102 ceph-mon[111694]: osdmap e51: 6 total, 5 up, 6 in 2024-05-13T22:54:09.091 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:09 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/673653616' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:09.091 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:09 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/4253128597' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:54:09.091 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:09 smithi102 podman[123015]: 2024-05-13 22:54:09.099317002 +0000 UTC m=+2.014533374 container cleanup 0f1608e57bb9b530b6974ab8d56e18eaa2c8823859880bef39f8495b14c6338d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=HEAD, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, ceph=True) 2024-05-13T22:54:09.091 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:09 smithi102 bash[123015]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0 2024-05-13T22:54:09.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.238+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/3910518594 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a28196fe0 con 0x7f1a1806e330 2024-05-13T22:54:09.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.240+0000 7f1a25ffb700 1 -- 172.21.15.102:0/3910518594 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1a28196fe0 con 0x7f1a1806e330 2024-05-13T22:54:09.216 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr", 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout: "mon" 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "6/23 daemons upgraded", 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading osd daemons", 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:54:09.217 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:54:09.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.242+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/3910518594 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a1806e330 msgr2=0x7f1a180707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:09.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.242+0000 7f1a2ef5f700 1 --2- 172.21.15.102:0/3910518594 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a1806e330 0x7f1a180707f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1a14005950 tx=0x7f1a140058e0 comp rx=0 tx=0).stop 2024-05-13T22:54:09.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.242+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/3910518594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a28068670 msgr2=0x7f1a2819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:09.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.242+0000 7f1a2ef5f700 1 --2- 172.21.15.102:0/3910518594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a28068670 0x7f1a2819c510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0092e0 tx=0x7f1a1c00be20 comp rx=0 tx=0).stop 2024-05-13T22:54:09.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.242+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/3910518594 shutdown_connections 2024-05-13T22:54:09.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.242+0000 7f1a2ef5f700 1 --2- 172.21.15.102:0/3910518594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a28068670 0x7f1a2819c510 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:09.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.242+0000 7f1a2ef5f700 1 --2- 172.21.15.102:0/3910518594 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a1806e330 0x7f1a180707f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:09.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.242+0000 7f1a2ef5f700 1 --2- 172.21.15.102:0/3910518594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a28069020 0x7f1a2819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:09.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.242+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/3910518594 >> 172.21.15.102:0/3910518594 conn(0x7f1a280755a0 msgr2=0x7f1a280fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:09.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.242+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/3910518594 shutdown_connections 2024-05-13T22:54:09.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.242+0000 7f1a2ef5f700 1 -- 172.21.15.102:0/3910518594 wait complete. 2024-05-13T22:54:09.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.350+0000 7ff32d269700 1 -- 172.21.15.102:0/2350733464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff328101780 msgr2=0x7ff3281057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:09.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.350+0000 7ff32d269700 1 --2- 172.21.15.102:0/2350733464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff328101780 0x7ff3281057d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009910 tx=0x7ff31c009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:09.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.350+0000 7ff32d269700 1 -- 172.21.15.102:0/2350733464 shutdown_connections 2024-05-13T22:54:09.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.350+0000 7ff32d269700 1 --2- 172.21.15.102:0/2350733464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff328101780 0x7ff3281057d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:09.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.350+0000 7ff32d269700 1 --2- 172.21.15.102:0/2350733464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff328100dd0 0x7ff3281011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:09.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.350+0000 7ff32d269700 1 -- 172.21.15.102:0/2350733464 >> 172.21.15.102:0/2350733464 conn(0x7ff3280fc640 msgr2=0x7ff3280fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:09.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.351+0000 7ff32d269700 1 -- 172.21.15.102:0/2350733464 shutdown_connections 2024-05-13T22:54:09.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.351+0000 7ff32d269700 1 -- 172.21.15.102:0/2350733464 wait complete. 2024-05-13T22:54:09.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.351+0000 7ff32d269700 1 Processor -- start 2024-05-13T22:54:09.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.351+0000 7ff32d269700 1 -- start start 2024-05-13T22:54:09.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff32d269700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff328100dd0 0x7ff32819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:09.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff32d269700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff328101780 0x7ff32819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:09.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff32d269700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32819d160 con 0x7ff328100dd0 2024-05-13T22:54:09.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff32d269700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff328196610 con 0x7ff328101780 2024-05-13T22:54:09.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff326ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff328100dd0 0x7ff32819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:09.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff326ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff328100dd0 0x7ff32819c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47088/0 (socket says 172.21.15.102:47088) 2024-05-13T22:54:09.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff326ffd700 1 -- 172.21.15.102:0/319348006 learned_addr learned my addr 172.21.15.102:0/319348006 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:09.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff3267fc700 1 --2- 172.21.15.102:0/319348006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff328101780 0x7ff32819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:09.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff326ffd700 1 -- 172.21.15.102:0/319348006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff328101780 msgr2=0x7ff32819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:09.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff326ffd700 1 --2- 172.21.15.102:0/319348006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff328101780 0x7ff32819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:09.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff326ffd700 1 -- 172.21.15.102:0/319348006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c0095f0 con 0x7ff328100dd0 2024-05-13T22:54:09.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff326ffd700 1 --2- 172.21.15.102:0/319348006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff328100dd0 0x7ff32819c590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff31800d8d0 tx=0x7ff31800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:09.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff30ffff700 1 -- 172.21.15.102:0/319348006 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff318009940 con 0x7ff328100dd0 2024-05-13T22:54:09.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff30ffff700 1 -- 172.21.15.102:0/319348006 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff318010460 con 0x7ff328100dd0 2024-05-13T22:54:09.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff32d269700 1 -- 172.21.15.102:0/319348006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3281968f0 con 0x7ff328100dd0 2024-05-13T22:54:09.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff30ffff700 1 -- 172.21.15.102:0/319348006 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff31800f5f0 con 0x7ff328100dd0 2024-05-13T22:54:09.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.352+0000 7ff32d269700 1 -- 172.21.15.102:0/319348006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff328196e40 con 0x7ff328100dd0 2024-05-13T22:54:09.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.353+0000 7ff32d269700 1 -- 172.21.15.102:0/319348006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3281091d0 con 0x7ff328100dd0 2024-05-13T22:54:09.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.354+0000 7ff30ffff700 1 -- 172.21.15.102:0/319348006 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 37) v1 ==== 92307+0+0 (secure 0 0 0) 0x7ff31800f870 con 0x7ff328100dd0 2024-05-13T22:54:09.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.355+0000 7ff30ffff700 1 --2- 172.21.15.102:0/319348006 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff31006e340 0x7ff310070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:09.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.355+0000 7ff3267fc700 1 --2- 172.21.15.102:0/319348006 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff31006e340 0x7ff310070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:09.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.355+0000 7ff30ffff700 1 -- 172.21.15.102:0/319348006 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff31808e3d0 con 0x7ff328100dd0 2024-05-13T22:54:09.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.355+0000 7ff3267fc700 1 --2- 172.21.15.102:0/319348006 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff31006e340 0x7ff310070800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff3280fcd70 tx=0x7ff31c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:09.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.359+0000 7ff30ffff700 1 -- 172.21.15.102:0/319348006 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff31805b0e0 con 0x7ff328100dd0 2024-05-13T22:54:09.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:09 smithi130 ceph-mon[84265]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:09.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:09 smithi130 ceph-mon[84265]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:09.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:09 smithi130 ceph-mon[84265]: pgmap v44: 65 pgs: 65 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:54:09.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:09 smithi130 ceph-mon[84265]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:09.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:09 smithi130 ceph-mon[84265]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:54:09.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:09 smithi130 ceph-mon[84265]: osdmap e51: 6 total, 5 up, 6 in 2024-05-13T22:54:09.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:09 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/673653616' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:09.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:09 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/4253128597' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:54:09.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.692+0000 7ff32d269700 1 -- 172.21.15.102:0/319348006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff32804ea90 con 0x7ff328100dd0 2024-05-13T22:54:09.670 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:09 smithi102 podman[123155]: 2024-05-13 22:54:09.324769748 +0000 UTC m=+0.658527996 container remove 0f1608e57bb9b530b6974ab8d56e18eaa2c8823859880bef39f8495b14c6338d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0, ceph=True, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD) 2024-05-13T22:54:09.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.692+0000 7ff30ffff700 1 -- 172.21.15.102:0/319348006 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7ff31805ac70 con 0x7ff328100dd0 2024-05-13T22:54:09.670 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_WARN 1 osds down 2024-05-13T22:54:09.670 INFO:teuthology.orchestra.run.smithi102.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-13T22:54:09.671 INFO:teuthology.orchestra.run.smithi102.stdout: osd.0 (root=default,host=smithi102) is down 2024-05-13T22:54:09.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.695+0000 7ff32d269700 1 -- 172.21.15.102:0/319348006 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff31006e340 msgr2=0x7ff310070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:09.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.695+0000 7ff32d269700 1 --2- 172.21.15.102:0/319348006 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff31006e340 0x7ff310070800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff3280fcd70 tx=0x7ff31c01a040 comp rx=0 tx=0).stop 2024-05-13T22:54:09.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.695+0000 7ff32d269700 1 -- 172.21.15.102:0/319348006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff328100dd0 msgr2=0x7ff32819c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:09.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.695+0000 7ff32d269700 1 --2- 172.21.15.102:0/319348006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff328100dd0 0x7ff32819c590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff31800d8d0 tx=0x7ff31800dc90 comp rx=0 tx=0).stop 2024-05-13T22:54:09.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.695+0000 7ff32d269700 1 -- 172.21.15.102:0/319348006 shutdown_connections 2024-05-13T22:54:09.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.695+0000 7ff32d269700 1 --2- 172.21.15.102:0/319348006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff328100dd0 0x7ff32819c590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:09.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.695+0000 7ff32d269700 1 --2- 172.21.15.102:0/319348006 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff31006e340 0x7ff310070800 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:09.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.695+0000 7ff32d269700 1 --2- 172.21.15.102:0/319348006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff328101780 0x7ff32819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:09.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.695+0000 7ff32d269700 1 -- 172.21.15.102:0/319348006 >> 172.21.15.102:0/319348006 conn(0x7ff3280fc640 msgr2=0x7ff3280fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:09.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.696+0000 7ff32d269700 1 -- 172.21.15.102:0/319348006 shutdown_connections 2024-05-13T22:54:09.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:09.696+0000 7ff32d269700 1 -- 172.21.15.102:0/319348006 wait complete. 2024-05-13T22:54:09.922 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:09 smithi102 podman[123291]: 2024-05-13 22:54:09.809288394 +0000 UTC m=+0.112373980 container create 584967f0686381552e0afd1f9b619b7ab304d179470938199a0913ccefd8ac3f (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-13T22:54:09.922 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:09 smithi102 podman[123291]: 2024-05-13 22:54:09.71569087 +0000 UTC m=+0.018776469 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:54:10.291 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:10 smithi102 ceph-mon[111694]: osdmap e52: 6 total, 5 up, 6 in 2024-05-13T22:54:10.291 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:10 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/319348006' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:54:10.291 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:10 smithi102 podman[123291]: 2024-05-13 22:54:10.020174363 +0000 UTC m=+0.323259949 container init 584967f0686381552e0afd1f9b619b7ab304d179470938199a0913ccefd8ac3f (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-13T22:54:10.291 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:10 smithi102 podman[123291]: 2024-05-13 22:54:10.024101821 +0000 UTC m=+0.327187407 container start 584967f0686381552e0afd1f9b619b7ab304d179470938199a0913ccefd8ac3f (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-13T22:54:10.292 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:10 smithi102 podman[123291]: 2024-05-13 22:54:10.111913951 +0000 UTC m=+0.414999557 container attach 584967f0686381552e0afd1f9b619b7ab304d179470938199a0913ccefd8ac3f (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-13T22:54:10.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:10 smithi130 ceph-mon[84265]: osdmap e52: 6 total, 5 up, 6 in 2024-05-13T22:54:10.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:10 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/319348006' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:54:10.575 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:10 smithi102 podman[123291]: 2024-05-13 22:54:10.315102821 +0000 UTC m=+0.618188411 container died 584967f0686381552e0afd1f9b619b7ab304d179470938199a0913ccefd8ac3f (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, RELEASE=squid-c3e5514, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-13T22:54:10.831 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:10 smithi102 podman[123320]: 2024-05-13 22:54:10.840836421 +0000 UTC m=+0.519705348 container remove 584967f0686381552e0afd1f9b619b7ab304d179470938199a0913ccefd8ac3f (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-13T22:54:11.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:11 smithi102 ceph-mon[111694]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:11.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:11 smithi102 ceph-mon[111694]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-13T22:54:11.177 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:10 smithi102 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.0.service: Deactivated successfully. 2024-05-13T22:54:11.177 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:10 smithi102 systemd[1]: Stopped Ceph osd.0 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:54:11.177 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:10 smithi102 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.0.service: Consumed 1min 1.542s CPU time. 2024-05-13T22:54:11.177 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 systemd[1]: Starting Ceph osd.0 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:54:11.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:11 smithi130 ceph-mon[84265]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:11.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:11 smithi130 ceph-mon[84265]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-13T22:54:11.562 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 podman[123420]: 2024-05-13 22:54:11.189736855 +0000 UTC m=+0.018247244 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:54:11.563 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 podman[123420]: 2024-05-13 22:54:11.309111435 +0000 UTC m=+0.137621846 container create d003cd2dc3a3e23092b5fa99be873dc9a2c68b6996b4d302bf05461d464a83c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T22:54:11.563 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 podman[123420]: 2024-05-13 22:54:11.51141168 +0000 UTC m=+0.339922056 container init d003cd2dc3a3e23092b5fa99be873dc9a2c68b6996b4d302bf05461d464a83c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate, GIT_CLEAN=True, RELEASE=squid-c3e5514, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T22:54:11.563 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 podman[123420]: 2024-05-13 22:54:11.515949809 +0000 UTC m=+0.344460197 container start d003cd2dc3a3e23092b5fa99be873dc9a2c68b6996b4d302bf05461d464a83c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-c3e5514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507) 2024-05-13T22:54:11.563 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 podman[123420]: 2024-05-13 22:54:11.586386988 +0000 UTC m=+0.414897373 container attach d003cd2dc3a3e23092b5fa99be873dc9a2c68b6996b4d302bf05461d464a83c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=squid-c3e5514) 2024-05-13T22:54:12.288 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:12 smithi102 ceph-mon[111694]: Health check failed: Degraded data redundancy: 42/231 objects degraded (18.182%), 18 pgs degraded (PG_DEGRADED) 2024-05-13T22:54:12.288 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate[123435]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-13T22:54:12.288 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate[123435]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-13T22:54:12.288 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 bash[123420]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-13T22:54:12.288 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 bash[123420]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-13T22:54:12.288 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 bash[123420]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-13T22:54:12.289 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 bash[123420]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-13T22:54:12.289 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 bash[123420]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-13T22:54:12.289 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 bash[123420]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-13T22:54:12.289 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 bash[123420]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-13T22:54:12.289 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate[123435]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-13T22:54:12.289 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate[123435]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-13T22:54:12.289 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate[123435]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-13T22:54:12.289 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate[123435]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-13T22:54:12.289 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate[123435]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-13T22:54:12.289 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:11 smithi102 podman[123420]: 2024-05-13 22:54:11.987221129 +0000 UTC m=+0.815731516 container died d003cd2dc3a3e23092b5fa99be873dc9a2c68b6996b4d302bf05461d464a83c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-c3e5514) 2024-05-13T22:54:12.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:12 smithi130 ceph-mon[84265]: Health check failed: Degraded data redundancy: 42/231 objects degraded (18.182%), 18 pgs degraded (PG_DEGRADED) 2024-05-13T22:54:12.892 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:12 smithi102 podman[123568]: 2024-05-13 22:54:12.566163271 +0000 UTC m=+0.573241401 container remove d003cd2dc3a3e23092b5fa99be873dc9a2c68b6996b4d302bf05461d464a83c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-13T22:54:12.893 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:12 smithi102 podman[123627]: 2024-05-13 22:54:12.687357255 +0000 UTC m=+0.017863703 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:54:12.893 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:12 smithi102 podman[123627]: 2024-05-13 22:54:12.800717718 +0000 UTC m=+0.131224209 container create 0ada88c777a224b3f5855974176832684e6a7be465b994196cc390d1df0cdde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-13T22:54:13.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:13 smithi102 ceph-mon[111694]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:54:13.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:13 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:13.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:13 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:13.177 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:12 smithi102 podman[123627]: 2024-05-13 22:54:12.994625781 +0000 UTC m=+0.325132229 container init 0ada88c777a224b3f5855974176832684e6a7be465b994196cc390d1df0cdde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-c3e5514, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-13T22:54:13.177 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:12 smithi102 podman[123627]: 2024-05-13 22:54:12.99774776 +0000 UTC m=+0.328254219 container start 0ada88c777a224b3f5855974176832684e6a7be465b994196cc390d1df0cdde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0, RELEASE=squid-c3e5514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-13T22:54:13.177 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:13 smithi102 bash[123627]: 0ada88c777a224b3f5855974176832684e6a7be465b994196cc390d1df0cdde7 2024-05-13T22:54:13.177 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:13 smithi102 systemd[1]: Started Ceph osd.0 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:54:13.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:13 smithi130 ceph-mon[84265]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:54:13.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:13 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:13.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:13 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:13.903 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:13 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[123642]: 2024-05-13T22:54:13.539+0000 7f2e1465a7c0 -1 Falling back to public interface 2024-05-13T22:54:14.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:14 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:54:14.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:14 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:54:15.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:15 smithi130 ceph-mon[84265]: pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:54:15.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:15 smithi102 ceph-mon[111694]: pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:54:17.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:17 smithi130 ceph-mon[84265]: pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:54:17.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:17 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:17.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:17 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:54:17.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:17 smithi102 ceph-mon[111694]: pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:54:17.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:17 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:17.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:17 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:54:18.426 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:17 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[123642]: 2024-05-13T22:54:17.993+0000 7f2e1465a7c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-05-13T22:54:18.903 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:18 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[123642]: 2024-05-13T22:54:18.472+0000 7f2e1465a7c0 -1 osd.0 50 log_to_monitors true 2024-05-13T22:54:19.176 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 22:54:19 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[123642]: 2024-05-13T22:54:19.111+0000 7f2e05ab1700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:54:19.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:19 smithi102 ceph-mon[111694]: pgmap v51: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 309 B/s wr, 2 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:54:19.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:19 smithi102 ceph-mon[111694]: mgrmap e38: smithi102.xcijrj(active, since 92s), standbys: smithi130.uoqoxl 2024-05-13T22:54:19.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:19 smithi102 ceph-mon[111694]: from='osd.0 [v2:172.21.15.102:6802/307502084,v1:172.21.15.102:6803/307502084]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-13T22:54:19.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:19.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:19 smithi130 ceph-mon[84265]: pgmap v51: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 309 B/s wr, 2 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:54:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:19 smithi130 ceph-mon[84265]: mgrmap e38: smithi102.xcijrj(active, since 92s), standbys: smithi130.uoqoxl 2024-05-13T22:54:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:19 smithi130 ceph-mon[84265]: from='osd.0 [v2:172.21.15.102:6802/307502084,v1:172.21.15.102:6803/307502084]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-13T22:54:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:19.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:20.393 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:20 smithi102 ceph-mon[111694]: from='osd.0 [v2:172.21.15.102:6802/307502084,v1:172.21.15.102:6803/307502084]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-13T22:54:20.393 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:20 smithi102 ceph-mon[111694]: osdmap e53: 6 total, 5 up, 6 in 2024-05-13T22:54:20.393 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:20 smithi102 ceph-mon[111694]: from='osd.0 [v2:172.21.15.102:6802/307502084,v1:172.21.15.102:6803/307502084]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:54:20.393 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:20.393 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:20.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:20 smithi130 ceph-mon[84265]: from='osd.0 [v2:172.21.15.102:6802/307502084,v1:172.21.15.102:6803/307502084]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-13T22:54:20.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:20 smithi130 ceph-mon[84265]: osdmap e53: 6 total, 5 up, 6 in 2024-05-13T22:54:20.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:20 smithi130 ceph-mon[84265]: from='osd.0 [v2:172.21.15.102:6802/307502084,v1:172.21.15.102:6803/307502084]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:54:20.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:20.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:21.353 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:21 smithi102 ceph-mon[111694]: pgmap v53: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:54:21.353 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:21 smithi102 ceph-mon[111694]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:54:21.353 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:21 smithi102 ceph-mon[111694]: osd.0 [v2:172.21.15.102:6802/307502084,v1:172.21.15.102:6803/307502084] boot 2024-05-13T22:54:21.353 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:21 smithi102 ceph-mon[111694]: osdmap e54: 6 total, 6 up, 6 in 2024-05-13T22:54:21.353 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:21 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:54:21.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:21 smithi130 ceph-mon[84265]: pgmap v53: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:54:21.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:21 smithi130 ceph-mon[84265]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:54:21.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:21 smithi130 ceph-mon[84265]: osd.0 [v2:172.21.15.102:6802/307502084,v1:172.21.15.102:6803/307502084] boot 2024-05-13T22:54:21.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:21 smithi130 ceph-mon[84265]: osdmap e54: 6 total, 6 up, 6 in 2024-05-13T22:54:21.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:21 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-13T22:54:22.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:22 smithi130 ceph-mon[84265]: osdmap e55: 6 total, 6 up, 6 in 2024-05-13T22:54:22.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:22 smithi102 ceph-mon[111694]: osdmap e55: 6 total, 6 up, 6 in 2024-05-13T22:54:23.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:23 smithi102 ceph-mon[111694]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s; 7/231 objects degraded (3.030%); 46 B/s, 0 objects/s recovering 2024-05-13T22:54:23.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:23 smithi102 ceph-mon[111694]: Health check update: Degraded data redundancy: 7/231 objects degraded (3.030%), 3 pgs degraded (PG_DEGRADED) 2024-05-13T22:54:23.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:23 smithi130 ceph-mon[84265]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s; 7/231 objects degraded (3.030%); 46 B/s, 0 objects/s recovering 2024-05-13T22:54:23.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:23 smithi130 ceph-mon[84265]: Health check update: Degraded data redundancy: 7/231 objects degraded (3.030%), 3 pgs degraded (PG_DEGRADED) 2024-05-13T22:54:24.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:24.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:24.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:54:24.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:54:24.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:24.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:54:24.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:24.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:24.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:24.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-13T22:54:24.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-13T22:54:24.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-05-13T22:54:24.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:24 smithi130 ceph-mon[84265]: pgmap v57: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 7/231 objects degraded (3.030%); 46 B/s, 0 objects/s recovering 2024-05-13T22:54:24.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:24.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:24.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:54:24.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:54:24.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:24.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:54:24.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:24.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:24.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:24.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-13T22:54:24.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-13T22:54:24.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-05-13T22:54:24.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:24 smithi102 ceph-mon[111694]: pgmap v57: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 7/231 objects degraded (3.030%); 46 B/s, 0 objects/s recovering 2024-05-13T22:54:26.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:26 smithi130 ceph-mon[84265]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/231 objects degraded (3.030%), 3 pgs degraded) 2024-05-13T22:54:26.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:26 smithi130 ceph-mon[84265]: Cluster is now healthy 2024-05-13T22:54:26.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:26 smithi102 ceph-mon[111694]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/231 objects degraded (3.030%), 3 pgs degraded) 2024-05-13T22:54:26.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:26 smithi102 ceph-mon[111694]: Cluster is now healthy 2024-05-13T22:54:27.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:27 smithi130 ceph-mon[84265]: pgmap v58: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 589 B/s rd, 1 op/s; 40 B/s, 0 objects/s recovering 2024-05-13T22:54:27.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:27 smithi102 ceph-mon[111694]: pgmap v58: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 589 B/s rd, 1 op/s; 40 B/s, 0 objects/s recovering 2024-05-13T22:54:29.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:29 smithi130 ceph-mon[84265]: pgmap v59: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 35 B/s, 0 objects/s recovering 2024-05-13T22:54:29.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:29 smithi102 ceph-mon[111694]: pgmap v59: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 35 B/s, 0 objects/s recovering 2024-05-13T22:54:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:31 smithi130 ceph-mon[84265]: pgmap v60: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 927 B/s rd, 1 op/s; 28 B/s, 0 objects/s recovering 2024-05-13T22:54:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:31.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:54:31.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:31 smithi102 ceph-mon[111694]: pgmap v60: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 927 B/s rd, 1 op/s; 28 B/s, 0 objects/s recovering 2024-05-13T22:54:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:54:31.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:33.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:33 smithi130 ceph-mon[84265]: pgmap v61: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 25 B/s, 0 objects/s recovering 2024-05-13T22:54:33.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:33 smithi102 ceph-mon[111694]: pgmap v61: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 25 B/s, 0 objects/s recovering 2024-05-13T22:54:35.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:35 smithi130 ceph-mon[84265]: pgmap v62: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:35.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:35 smithi102 ceph-mon[111694]: pgmap v62: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:37.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:37 smithi130 ceph-mon[84265]: pgmap v63: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:37.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:37 smithi102 ceph-mon[111694]: pgmap v63: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:39.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:39 smithi130 ceph-mon[84265]: pgmap v64: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:54:39.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:39 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-13T22:54:39.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:39 smithi102 ceph-mon[111694]: pgmap v64: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:54:39.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:39 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-13T22:54:39.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.811+0000 7f52212d1700 1 -- 172.21.15.102:0/2701535333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f521c1017a0 msgr2=0x7f521c1057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:39.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.811+0000 7f52212d1700 1 --2- 172.21.15.102:0/2701535333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f521c1017a0 0x7f521c1057f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5204009910 tx=0x7f5204009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:39.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.811+0000 7f52212d1700 1 -- 172.21.15.102:0/2701535333 shutdown_connections 2024-05-13T22:54:39.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.811+0000 7f52212d1700 1 --2- 172.21.15.102:0/2701535333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f521c1017a0 0x7f521c1057f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:39.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.811+0000 7f52212d1700 1 --2- 172.21.15.102:0/2701535333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f521c100df0 0x7f521c1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:39.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.811+0000 7f52212d1700 1 -- 172.21.15.102:0/2701535333 >> 172.21.15.102:0/2701535333 conn(0x7f521c0fc6a0 msgr2=0x7f521c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:39.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.811+0000 7f52212d1700 1 -- 172.21.15.102:0/2701535333 shutdown_connections 2024-05-13T22:54:39.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.811+0000 7f52212d1700 1 -- 172.21.15.102:0/2701535333 wait complete. 2024-05-13T22:54:39.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.812+0000 7f52212d1700 1 Processor -- start 2024-05-13T22:54:39.790 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.812+0000 7f52212d1700 1 -- start start 2024-05-13T22:54:39.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f52212d1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f521c100df0 0x7f521c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:39.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f52212d1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f521c1017a0 0x7f521c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:39.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f52212d1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c19d260 con 0x7f521c1017a0 2024-05-13T22:54:39.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f52212d1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c196710 con 0x7f521c100df0 2024-05-13T22:54:39.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f521a7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f521c1017a0 0x7f521c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:39.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f521a7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f521c1017a0 0x7f521c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50794/0 (socket says 172.21.15.102:50794) 2024-05-13T22:54:39.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f521a7fc700 1 -- 172.21.15.102:0/3056930361 learned_addr learned my addr 172.21.15.102:0/3056930361 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:39.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f521a7fc700 1 -- 172.21.15.102:0/3056930361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f521c100df0 msgr2=0x7f521c19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:39.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f521a7fc700 1 --2- 172.21.15.102:0/3056930361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f521c100df0 0x7f521c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:39.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f521a7fc700 1 -- 172.21.15.102:0/3056930361 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52040095f0 con 0x7f521c1017a0 2024-05-13T22:54:39.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f521a7fc700 1 --2- 172.21.15.102:0/3056930361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f521c1017a0 0x7f521c19cbd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52040092e0 tx=0x7f520400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:39.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f5213fff700 1 -- 172.21.15.102:0/3056930361 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f520401d070 con 0x7f521c1017a0 2024-05-13T22:54:39.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.813+0000 7f5213fff700 1 -- 172.21.15.102:0/3056930361 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f520400f460 con 0x7f521c1017a0 2024-05-13T22:54:39.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.814+0000 7f52212d1700 1 -- 172.21.15.102:0/3056930361 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c196990 con 0x7f521c1017a0 2024-05-13T22:54:39.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.814+0000 7f5213fff700 1 -- 172.21.15.102:0/3056930361 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52040175f0 con 0x7f521c1017a0 2024-05-13T22:54:39.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.814+0000 7f52212d1700 1 -- 172.21.15.102:0/3056930361 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c196e80 con 0x7f521c1017a0 2024-05-13T22:54:39.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.815+0000 7f52212d1700 1 -- 172.21.15.102:0/3056930361 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c109140 con 0x7f521c1017a0 2024-05-13T22:54:39.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.815+0000 7f5213fff700 1 -- 172.21.15.102:0/3056930361 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f520400f5d0 con 0x7f521c1017a0 2024-05-13T22:54:39.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.816+0000 7f5213fff700 1 --2- 172.21.15.102:0/3056930361 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f520806e200 0x7f52080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:39.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.816+0000 7f5213fff700 1 -- 172.21.15.102:0/3056930361 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f520408f530 con 0x7f521c1017a0 2024-05-13T22:54:39.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.816+0000 7f521affd700 1 --2- 172.21.15.102:0/3056930361 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f520806e200 0x7f52080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:39.793 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.816+0000 7f521affd700 1 --2- 172.21.15.102:0/3056930361 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f520806e200 0x7f52080706c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f520c0097b0 tx=0x7f520c006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:39.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:39.821+0000 7f5213fff700 1 -- 172.21.15.102:0/3056930361 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f520405c470 con 0x7f521c1017a0 2024-05-13T22:54:40.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.036+0000 7f52212d1700 1 -- 172.21.15.102:0/3056930361 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f521c02cce0 con 0x7f520806e200 2024-05-13T22:54:40.014 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.038+0000 7f5213fff700 1 -- 172.21.15.102:0/3056930361 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f521c02cce0 con 0x7f520806e200 2024-05-13T22:54:40.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.040+0000 7f52212d1700 1 -- 172.21.15.102:0/3056930361 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f520806e200 msgr2=0x7f52080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.040+0000 7f52212d1700 1 --2- 172.21.15.102:0/3056930361 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f520806e200 0x7f52080706c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f520c0097b0 tx=0x7f520c006d20 comp rx=0 tx=0).stop 2024-05-13T22:54:40.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.040+0000 7f52212d1700 1 -- 172.21.15.102:0/3056930361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f521c1017a0 msgr2=0x7f521c19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.040+0000 7f52212d1700 1 --2- 172.21.15.102:0/3056930361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f521c1017a0 0x7f521c19cbd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52040092e0 tx=0x7f520400be20 comp rx=0 tx=0).stop 2024-05-13T22:54:40.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.040+0000 7f52212d1700 1 -- 172.21.15.102:0/3056930361 shutdown_connections 2024-05-13T22:54:40.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.040+0000 7f52212d1700 1 --2- 172.21.15.102:0/3056930361 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f520806e200 0x7f52080706c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.040+0000 7f52212d1700 1 --2- 172.21.15.102:0/3056930361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f521c100df0 0x7f521c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.040+0000 7f52212d1700 1 --2- 172.21.15.102:0/3056930361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f521c1017a0 0x7f521c19cbd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.017 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.040+0000 7f52212d1700 1 -- 172.21.15.102:0/3056930361 >> 172.21.15.102:0/3056930361 conn(0x7f521c0fc6a0 msgr2=0x7f521c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:40.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.040+0000 7f52212d1700 1 -- 172.21.15.102:0/3056930361 shutdown_connections 2024-05-13T22:54:40.018 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.040+0000 7f52212d1700 1 -- 172.21.15.102:0/3056930361 wait complete. 2024-05-13T22:54:40.027 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:54:40.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.148+0000 7f01a38db700 1 -- 172.21.15.102:0/2392206762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019c104a50 msgr2=0x7f019c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.148+0000 7f01a38db700 1 --2- 172.21.15.102:0/2392206762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019c104a50 0x7f019c104e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0198009910 tx=0x7f0198009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:40.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.148+0000 7f01a38db700 1 -- 172.21.15.102:0/2392206762 shutdown_connections 2024-05-13T22:54:40.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.148+0000 7f01a38db700 1 --2- 172.21.15.102:0/2392206762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f019c100140 0x7f019c1005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.148+0000 7f01a38db700 1 --2- 172.21.15.102:0/2392206762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019c104a50 0x7f019c104e30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.148+0000 7f01a38db700 1 -- 172.21.15.102:0/2392206762 >> 172.21.15.102:0/2392206762 conn(0x7f019c075e30 msgr2=0x7f019c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.148+0000 7f01a38db700 1 -- 172.21.15.102:0/2392206762 shutdown_connections 2024-05-13T22:54:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.149+0000 7f01a38db700 1 -- 172.21.15.102:0/2392206762 wait complete. 2024-05-13T22:54:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.149+0000 7f01a38db700 1 Processor -- start 2024-05-13T22:54:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.149+0000 7f01a38db700 1 -- start start 2024-05-13T22:54:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.149+0000 7f01a38db700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019c100140 0x7f019c1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.149+0000 7f01a38db700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f019c104a50 0x7f019c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.149+0000 7f01a38db700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c198050 con 0x7f019c100140 2024-05-13T22:54:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.149+0000 7f01a38db700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c198190 con 0x7f019c104a50 2024-05-13T22:54:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01a1677700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019c100140 0x7f019c1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:40.129 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01a1677700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019c100140 0x7f019c1974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50806/0 (socket says 172.21.15.102:50806) 2024-05-13T22:54:40.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01a1677700 1 -- 172.21.15.102:0/1914119192 learned_addr learned my addr 172.21.15.102:0/1914119192 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:40.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01a1677700 1 -- 172.21.15.102:0/1914119192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f019c104a50 msgr2=0x7f019c197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01a1677700 1 --2- 172.21.15.102:0/1914119192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f019c104a50 0x7f019c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01a1677700 1 -- 172.21.15.102:0/1914119192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01980095f0 con 0x7f019c100140 2024-05-13T22:54:40.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01a1677700 1 --2- 172.21.15.102:0/1914119192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019c100140 0x7f019c1974f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f01980092e0 tx=0x7f019800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:40.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01927fc700 1 -- 172.21.15.102:0/1914119192 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019801d070 con 0x7f019c100140 2024-05-13T22:54:40.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01927fc700 1 -- 172.21.15.102:0/1914119192 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0198005e00 con 0x7f019c100140 2024-05-13T22:54:40.130 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01927fc700 1 -- 172.21.15.102:0/1914119192 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0198017400 con 0x7f019c100140 2024-05-13T22:54:40.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01a38db700 1 -- 172.21.15.102:0/1914119192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019c1a0560 con 0x7f019c100140 2024-05-13T22:54:40.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.150+0000 7f01a38db700 1 -- 172.21.15.102:0/1914119192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019c1a09f0 con 0x7f019c100140 2024-05-13T22:54:40.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.151+0000 7f01a38db700 1 -- 172.21.15.102:0/1914119192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019c108020 con 0x7f019c100140 2024-05-13T22:54:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.152+0000 7f01927fc700 1 -- 172.21.15.102:0/1914119192 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f019800f460 con 0x7f019c100140 2024-05-13T22:54:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.153+0000 7f01927fc700 1 --2- 172.21.15.102:0/1914119192 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f018806e200 0x7f01880706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.153+0000 7f01927fc700 1 -- 172.21.15.102:0/1914119192 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f019808f620 con 0x7f019c100140 2024-05-13T22:54:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.153+0000 7f01a0e76700 1 --2- 172.21.15.102:0/1914119192 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f018806e200 0x7f01880706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:40.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.153+0000 7f01a0e76700 1 --2- 172.21.15.102:0/1914119192 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f018806e200 0x7f01880706c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f018c005950 tx=0x7f018c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:40.137 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.160+0000 7f01927fc700 1 -- 172.21.15.102:0/1914119192 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f01980589b0 con 0x7f019c100140 2024-05-13T22:54:40.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.391+0000 7f01a38db700 1 -- 172.21.15.102:0/1914119192 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f019c02cdd0 con 0x7f018806e200 2024-05-13T22:54:40.368 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:40 smithi102 ceph-mon[111694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-13T22:54:40.368 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:40 smithi102 ceph-mon[111694]: Upgrade: osd.1 is safe to restart 2024-05-13T22:54:40.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.392+0000 7f01927fc700 1 -- 172.21.15.102:0/1914119192 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f019c02cdd0 con 0x7f018806e200 2024-05-13T22:54:40.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.394+0000 7f01a38db700 1 -- 172.21.15.102:0/1914119192 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f018806e200 msgr2=0x7f01880706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.394+0000 7f01a38db700 1 --2- 172.21.15.102:0/1914119192 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f018806e200 0x7f01880706c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f018c005950 tx=0x7f018c0058e0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.395+0000 7f01a38db700 1 -- 172.21.15.102:0/1914119192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019c100140 msgr2=0x7f019c1974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.395+0000 7f01a38db700 1 --2- 172.21.15.102:0/1914119192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019c100140 0x7f019c1974f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f01980092e0 tx=0x7f019800be20 comp rx=0 tx=0).stop 2024-05-13T22:54:40.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.395+0000 7f01a38db700 1 -- 172.21.15.102:0/1914119192 shutdown_connections 2024-05-13T22:54:40.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.395+0000 7f01a38db700 1 --2- 172.21.15.102:0/1914119192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f019c100140 0x7f019c1974f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.395+0000 7f01a38db700 1 --2- 172.21.15.102:0/1914119192 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f018806e200 0x7f01880706c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.395+0000 7f01a38db700 1 --2- 172.21.15.102:0/1914119192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f019c104a50 0x7f019c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.395+0000 7f01a38db700 1 -- 172.21.15.102:0/1914119192 >> 172.21.15.102:0/1914119192 conn(0x7f019c075e30 msgr2=0x7f019c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:40.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.395+0000 7f01a38db700 1 -- 172.21.15.102:0/1914119192 shutdown_connections 2024-05-13T22:54:40.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.395+0000 7f01a38db700 1 -- 172.21.15.102:0/1914119192 wait complete. 2024-05-13T22:54:40.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:40 smithi130 ceph-mon[84265]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-13T22:54:40.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:40 smithi130 ceph-mon[84265]: Upgrade: osd.1 is safe to restart 2024-05-13T22:54:40.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.502+0000 7f592562e700 1 -- 172.21.15.102:0/3960570803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920068670 msgr2=0x7f5920068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.502+0000 7f592562e700 1 --2- 172.21.15.102:0/3960570803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920068670 0x7f5920068a50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5914009910 tx=0x7f5914009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:40.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.502+0000 7f592562e700 1 -- 172.21.15.102:0/3960570803 shutdown_connections 2024-05-13T22:54:40.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.502+0000 7f592562e700 1 --2- 172.21.15.102:0/3960570803 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920069020 0x7f5920105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.502+0000 7f592562e700 1 --2- 172.21.15.102:0/3960570803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920068670 0x7f5920068a50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.502+0000 7f592562e700 1 -- 172.21.15.102:0/3960570803 >> 172.21.15.102:0/3960570803 conn(0x7f59200755a0 msgr2=0x7f59200759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:40.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.502+0000 7f592562e700 1 -- 172.21.15.102:0/3960570803 shutdown_connections 2024-05-13T22:54:40.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.502+0000 7f592562e700 1 -- 172.21.15.102:0/3960570803 wait complete. 2024-05-13T22:54:40.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.503+0000 7f592562e700 1 Processor -- start 2024-05-13T22:54:40.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.503+0000 7f592562e700 1 -- start start 2024-05-13T22:54:40.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.503+0000 7f592562e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920068670 0x7f592019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:40.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.503+0000 7f592562e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920069020 0x7f592019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:40.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.503+0000 7f592562e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592019d0e0 con 0x7f5920068670 2024-05-13T22:54:40.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.503+0000 7f592562e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5920196590 con 0x7f5920069020 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.503+0000 7f591effd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920068670 0x7f592019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.503+0000 7f591effd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920068670 0x7f592019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50824/0 (socket says 172.21.15.102:50824) 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.503+0000 7f591effd700 1 -- 172.21.15.102:0/1405818359 learned_addr learned my addr 172.21.15.102:0/1405818359 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.504+0000 7f591e7fc700 1 --2- 172.21.15.102:0/1405818359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920069020 0x7f592019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.504+0000 7f591effd700 1 -- 172.21.15.102:0/1405818359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920069020 msgr2=0x7f592019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.504+0000 7f591effd700 1 --2- 172.21.15.102:0/1405818359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920069020 0x7f592019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.504+0000 7f591effd700 1 -- 172.21.15.102:0/1405818359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59140095f0 con 0x7f5920068670 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.504+0000 7f591effd700 1 --2- 172.21.15.102:0/1405818359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920068670 0x7f592019c510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5914006010 tx=0x7f591400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.504+0000 7f5907fff700 1 -- 172.21.15.102:0/1405818359 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f591401d070 con 0x7f5920068670 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.504+0000 7f592562e700 1 -- 172.21.15.102:0/1405818359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5920196810 con 0x7f5920068670 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.504+0000 7f592562e700 1 -- 172.21.15.102:0/1405818359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5920196d00 con 0x7f5920068670 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.504+0000 7f5907fff700 1 -- 172.21.15.102:0/1405818359 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f591400f460 con 0x7f5920068670 2024-05-13T22:54:40.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.504+0000 7f5907fff700 1 -- 172.21.15.102:0/1405818359 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5914017630 con 0x7f5920068670 2024-05-13T22:54:40.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.505+0000 7f592562e700 1 -- 172.21.15.102:0/1405818359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5920109130 con 0x7f5920068670 2024-05-13T22:54:40.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.506+0000 7f5907fff700 1 -- 172.21.15.102:0/1405818359 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f591400f5d0 con 0x7f5920068670 2024-05-13T22:54:40.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.506+0000 7f5907fff700 1 --2- 172.21.15.102:0/1405818359 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f590806e200 0x7f59080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:40.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.506+0000 7f5907fff700 1 -- 172.21.15.102:0/1405818359 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f591408f4f0 con 0x7f5920068670 2024-05-13T22:54:40.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.506+0000 7f591e7fc700 1 --2- 172.21.15.102:0/1405818359 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f590806e200 0x7f59080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:40.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.507+0000 7f591e7fc700 1 --2- 172.21.15.102:0/1405818359 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f590806e200 0x7f59080706c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5910005950 tx=0x7f59100058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:40.492 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.515+0000 7f5907fff700 1 -- 172.21.15.102:0/1405818359 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f591405c430 con 0x7f5920068670 2024-05-13T22:54:40.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.730+0000 7f592562e700 1 -- 172.21.15.102:0/1405818359 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5920196f70 con 0x7f590806e200 2024-05-13T22:54:40.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.741+0000 7f5907fff700 1 -- 172.21.15.102:0/1405818359 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5920196f70 con 0x7f590806e200 2024-05-13T22:54:40.718 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:54:40.718 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (3m) 22s ago 14m 26.7M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:54:40.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (15m) 22s ago 15m 8274k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:54:40.718 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (14m) 45s ago 14m 7956k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:54:40.718 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (54s) 22s ago 15m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T22:54:40.718 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (49s) 45s ago 14m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T22:54:40.718 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (3m) 22s ago 14m 79.8M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:54:40.718 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (9m) 22s ago 9m 88.0M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:54:40.718 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (9m) 22s ago 9m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:54:40.718 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (9m) 45s ago 9m 16.1M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:54:40.718 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (9m) 45s ago 9m 19.1M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (6m) 22s ago 16m 503M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (5m) 45s ago 13m 451M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (2m) 22s ago 16m 47.3M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (98s) 45s ago 13m 41.6M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (4m) 22s ago 15m 15.8M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (4m) 45s ago 13m 17.1M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (28s) 22s ago 13m 14.8M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 0ada88c777a2 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (12m) 22s ago 12m 240M 1216M 18.2.1 5be31c24972a e83df24af944 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (12m) 22s ago 12m 204M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (11m) 45s ago 11m 293M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (11m) 45s ago 11m 245M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (10m) 45s ago 10m 227M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:54:40.719 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (4m) 22s ago 14m 69.4M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:54:40.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.743+0000 7f592562e700 1 -- 172.21.15.102:0/1405818359 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f590806e200 msgr2=0x7f59080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.743+0000 7f592562e700 1 --2- 172.21.15.102:0/1405818359 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f590806e200 0x7f59080706c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5910005950 tx=0x7f59100058e0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.743+0000 7f592562e700 1 -- 172.21.15.102:0/1405818359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920068670 msgr2=0x7f592019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.743+0000 7f592562e700 1 --2- 172.21.15.102:0/1405818359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920068670 0x7f592019c510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5914006010 tx=0x7f591400be20 comp rx=0 tx=0).stop 2024-05-13T22:54:40.720 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.743+0000 7f592562e700 1 -- 172.21.15.102:0/1405818359 shutdown_connections 2024-05-13T22:54:40.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.743+0000 7f592562e700 1 --2- 172.21.15.102:0/1405818359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5920068670 0x7f592019c510 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.743+0000 7f592562e700 1 --2- 172.21.15.102:0/1405818359 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f590806e200 0x7f59080706c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.743+0000 7f592562e700 1 --2- 172.21.15.102:0/1405818359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5920069020 0x7f592019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.743+0000 7f592562e700 1 -- 172.21.15.102:0/1405818359 >> 172.21.15.102:0/1405818359 conn(0x7f59200755a0 msgr2=0x7f59200fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:40.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.743+0000 7f592562e700 1 -- 172.21.15.102:0/1405818359 shutdown_connections 2024-05-13T22:54:40.721 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.743+0000 7f592562e700 1 -- 172.21.15.102:0/1405818359 wait complete. 2024-05-13T22:54:40.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.849+0000 7f9943e85700 1 -- 172.21.15.102:0/2417963300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f993c100dc0 msgr2=0x7f993c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.849+0000 7f9943e85700 1 --2- 172.21.15.102:0/2417963300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f993c100dc0 0x7f993c1011a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9938009910 tx=0x7f9938009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:40.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 -- 172.21.15.102:0/2417963300 shutdown_connections 2024-05-13T22:54:40.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 --2- 172.21.15.102:0/2417963300 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f993c101770 0x7f993c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 --2- 172.21.15.102:0/2417963300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f993c100dc0 0x7f993c1011a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.828 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 -- 172.21.15.102:0/2417963300 >> 172.21.15.102:0/2417963300 conn(0x7f993c0fc650 msgr2=0x7f993c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 -- 172.21.15.102:0/2417963300 shutdown_connections 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 -- 172.21.15.102:0/2417963300 wait complete. 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 Processor -- start 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 -- start start 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f993c100dc0 0x7f993c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f993c101770 0x7f993c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c073b10 con 0x7f993c101770 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.850+0000 7f9943e85700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c073c50 con 0x7f993c100dc0 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9941420700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f993c101770 0x7f993c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9941420700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f993c101770 0x7f993c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50846/0 (socket says 172.21.15.102:50846) 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9941420700 1 -- 172.21.15.102:0/1709853474 learned_addr learned my addr 172.21.15.102:0/1709853474 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9941c21700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f993c100dc0 0x7f993c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9941420700 1 -- 172.21.15.102:0/1709853474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f993c100dc0 msgr2=0x7f993c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:40.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9941420700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f993c100dc0 0x7f993c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:40.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9941420700 1 -- 172.21.15.102:0/1709853474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99380095f0 con 0x7f993c101770 2024-05-13T22:54:40.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9941c21700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f993c100dc0 0x7f993c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:54:40.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9941420700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f993c101770 0x7f993c0735d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b480 tx=0x7f992c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:40.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9932ffd700 1 -- 172.21.15.102:0/1709853474 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f992c0118c0 con 0x7f993c101770 2024-05-13T22:54:40.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9932ffd700 1 -- 172.21.15.102:0/1709853474 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f992c010380 con 0x7f993c101770 2024-05-13T22:54:40.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9932ffd700 1 -- 172.21.15.102:0/1709853474 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f992c00f4f0 con 0x7f993c101770 2024-05-13T22:54:40.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9943e85700 1 -- 172.21.15.102:0/1709853474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993c1a2a40 con 0x7f993c101770 2024-05-13T22:54:40.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.851+0000 7f9943e85700 1 -- 172.21.15.102:0/1709853474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c1a2e50 con 0x7f993c101770 2024-05-13T22:54:40.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.852+0000 7f9943e85700 1 -- 172.21.15.102:0/1709853474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c109160 con 0x7f993c101770 2024-05-13T22:54:40.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.853+0000 7f9932ffd700 1 -- 172.21.15.102:0/1709853474 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f992c0104f0 con 0x7f993c101770 2024-05-13T22:54:40.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.853+0000 7f9932ffd700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f992806e200 0x7f99280706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:40.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.853+0000 7f9932ffd700 1 -- 172.21.15.102:0/1709853474 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f992c08dd30 con 0x7f993c101770 2024-05-13T22:54:40.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.853+0000 7f9941c21700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f992806e200 0x7f99280706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:40.831 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.854+0000 7f9941c21700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f992806e200 0x7f99280706c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f993800b5c0 tx=0x7f9938005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:40.835 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:40.858+0000 7f9932ffd700 1 -- 172.21.15.102:0/1709853474 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f992c0570c0 con 0x7f993c101770 2024-05-13T22:54:41.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.147+0000 7f9943e85700 1 -- 172.21.15.102:0/1709853474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f993c04ea90 con 0x7f993c101770 2024-05-13T22:54:41.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.148+0000 7f9932ffd700 1 -- 172.21.15.102:0/1709853474 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f992c05a780 con 0x7f993c101770 2024-05-13T22:54:41.125 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:54:41.125 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:54:41.125 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:54:41.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:54:41.125 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:54:41.125 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:54:41.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:54:41.125 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:54:41.125 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-05-13T22:54:41.125 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 1 2024-05-13T22:54:41.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:54:41.126 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:54:41.126 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:54:41.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:54:41.126 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:54:41.126 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-05-13T22:54:41.126 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 5 2024-05-13T22:54:41.126 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:54:41.126 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:54:41.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.150+0000 7f9943e85700 1 -- 172.21.15.102:0/1709853474 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f992806e200 msgr2=0x7f99280706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.150+0000 7f9943e85700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f992806e200 0x7f99280706c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f993800b5c0 tx=0x7f9938005480 comp rx=0 tx=0).stop 2024-05-13T22:54:41.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.150+0000 7f9943e85700 1 -- 172.21.15.102:0/1709853474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f993c101770 msgr2=0x7f993c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.150+0000 7f9943e85700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f993c101770 0x7f993c0735d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b480 tx=0x7f992c00b840 comp rx=0 tx=0).stop 2024-05-13T22:54:41.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.150+0000 7f9943e85700 1 -- 172.21.15.102:0/1709853474 shutdown_connections 2024-05-13T22:54:41.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.150+0000 7f9943e85700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f992806e200 0x7f99280706c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.150+0000 7f9943e85700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f993c100dc0 0x7f993c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.150+0000 7f9943e85700 1 --2- 172.21.15.102:0/1709853474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f993c101770 0x7f993c0735d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.150+0000 7f9943e85700 1 -- 172.21.15.102:0/1709853474 >> 172.21.15.102:0/1709853474 conn(0x7f993c0fc650 msgr2=0x7f993c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:41.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.150+0000 7f9943e85700 1 -- 172.21.15.102:0/1709853474 shutdown_connections 2024-05-13T22:54:41.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.150+0000 7f9943e85700 1 -- 172.21.15.102:0/1709853474 wait complete. 2024-05-13T22:54:41.233 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.256+0000 7ffa50c16700 1 -- 172.21.15.102:0/1620492420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c101780 msgr2=0x7ffa4c1057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.256+0000 7ffa50c16700 1 --2- 172.21.15.102:0/1620492420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c101780 0x7ffa4c1057d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009910 tx=0x7ffa40009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:41.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.256+0000 7ffa50c16700 1 -- 172.21.15.102:0/1620492420 shutdown_connections 2024-05-13T22:54:41.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.256+0000 7ffa50c16700 1 --2- 172.21.15.102:0/1620492420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c101780 0x7ffa4c1057d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.256+0000 7ffa50c16700 1 --2- 172.21.15.102:0/1620492420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa4c100dd0 0x7ffa4c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.256+0000 7ffa50c16700 1 -- 172.21.15.102:0/1620492420 >> 172.21.15.102:0/1620492420 conn(0x7ffa4c0fc640 msgr2=0x7ffa4c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:41.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.256+0000 7ffa50c16700 1 -- 172.21.15.102:0/1620492420 shutdown_connections 2024-05-13T22:54:41.234 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.256+0000 7ffa50c16700 1 -- 172.21.15.102:0/1620492420 wait complete. 2024-05-13T22:54:41.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.257+0000 7ffa50c16700 1 Processor -- start 2024-05-13T22:54:41.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.257+0000 7ffa50c16700 1 -- start start 2024-05-13T22:54:41.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.257+0000 7ffa50c16700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c100dd0 0x7ffa4c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:41.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.257+0000 7ffa50c16700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa4c101780 0x7ffa4c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:41.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.257+0000 7ffa50c16700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1995b0 con 0x7ffa4c100dd0 2024-05-13T22:54:41.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.257+0000 7ffa50c16700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c19d350 con 0x7ffa4c101780 2024-05-13T22:54:41.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa4a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c100dd0 0x7ffa4c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:41.236 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa4a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c100dd0 0x7ffa4c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50870/0 (socket says 172.21.15.102:50870) 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa4a59c700 1 -- 172.21.15.102:0/3425402157 learned_addr learned my addr 172.21.15.102:0/3425402157 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa49d9b700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa4c101780 0x7ffa4c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa4a59c700 1 -- 172.21.15.102:0/3425402157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa4c101780 msgr2=0x7ffa4c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa4a59c700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa4c101780 0x7ffa4c198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa4a59c700 1 -- 172.21.15.102:0/3425402157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400095f0 con 0x7ffa4c100dd0 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa49d9b700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa4c101780 0x7ffa4c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa4a59c700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c100dd0 0x7ffa4c1989e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00d8d0 tx=0x7ffa3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa3b7fe700 1 -- 172.21.15.102:0/3425402157 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3c009940 con 0x7ffa4c100dd0 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa50c16700 1 -- 172.21.15.102:0/3425402157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c19d630 con 0x7ffa4c100dd0 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa3b7fe700 1 -- 172.21.15.102:0/3425402157 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa3c010460 con 0x7ffa4c100dd0 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa3b7fe700 1 -- 172.21.15.102:0/3425402157 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3c00f5f0 con 0x7ffa4c100dd0 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.258+0000 7ffa50c16700 1 -- 172.21.15.102:0/3425402157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c19db80 con 0x7ffa4c100dd0 2024-05-13T22:54:41.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.259+0000 7ffa50c16700 1 -- 172.21.15.102:0/3425402157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c104f50 con 0x7ffa4c100dd0 2024-05-13T22:54:41.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.260+0000 7ffa3b7fe700 1 -- 172.21.15.102:0/3425402157 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ffa3c0105d0 con 0x7ffa4c100dd0 2024-05-13T22:54:41.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.260+0000 7ffa3b7fe700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ffa3406e200 0x7ffa340706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:41.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.261+0000 7ffa3b7fe700 1 -- 172.21.15.102:0/3425402157 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ffa3c08e190 con 0x7ffa4c100dd0 2024-05-13T22:54:41.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.261+0000 7ffa49d9b700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ffa3406e200 0x7ffa340706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:41.239 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.261+0000 7ffa49d9b700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ffa3406e200 0x7ffa340706c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000b5c0 tx=0x7ffa400053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:41.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.265+0000 7ffa3b7fe700 1 -- 172.21.15.102:0/3425402157 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ffa3c05afb0 con 0x7ffa4c100dd0 2024-05-13T22:54:41.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:41 smithi130 ceph-mon[84265]: pgmap v65: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:41.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:41 smithi130 ceph-mon[84265]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:41.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:41 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:41.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:41 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-13T22:54:41.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:41 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:54:41.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:41 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1709853474' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:41.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:41 smithi102 ceph-mon[111694]: pgmap v65: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:41.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:41 smithi102 ceph-mon[111694]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:41.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:41 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:41.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:41 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-13T22:54:41.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:41 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:54:41.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:41 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1709853474' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:54:41.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.509+0000 7ffa50c16700 1 -- 172.21.15.102:0/3425402157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffa4c19dda0 con 0x7ffa4c100dd0 2024-05-13T22:54:41.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.509+0000 7ffa3b7fe700 1 -- 172.21.15.102:0/3425402157 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1909 (secure 0 0 0) 0x7ffa3c05ab40 con 0x7ffa4c100dd0 2024-05-13T22:54:41.486 INFO:teuthology.orchestra.run.smithi102.stdout:e17 2024-05-13T22:54:41.486 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:54:41.486 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:54:41.486 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 17 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:54:41.487 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 47 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=14498} 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:54:41.488 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:54:41.489 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:54:41.489 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 0 members: 2024-05-13T22:54:41.489 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:54:41.489 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:54:41.489 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:54:41.489 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:54:41.489 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:54:41.489 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:54:41.489 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:54:41.489 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:54:41.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.511+0000 7ffa50c16700 1 -- 172.21.15.102:0/3425402157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ffa3406e200 msgr2=0x7ffa340706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.511+0000 7ffa50c16700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ffa3406e200 0x7ffa340706c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000b5c0 tx=0x7ffa400053a0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.511+0000 7ffa50c16700 1 -- 172.21.15.102:0/3425402157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c100dd0 msgr2=0x7ffa4c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.511+0000 7ffa50c16700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c100dd0 0x7ffa4c1989e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00d8d0 tx=0x7ffa3c00dc90 comp rx=0 tx=0).stop 2024-05-13T22:54:41.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.511+0000 7ffa50c16700 1 -- 172.21.15.102:0/3425402157 shutdown_connections 2024-05-13T22:54:41.490 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.511+0000 7ffa50c16700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa4c100dd0 0x7ffa4c1989e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.511+0000 7ffa50c16700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ffa3406e200 0x7ffa340706c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.511+0000 7ffa50c16700 1 --2- 172.21.15.102:0/3425402157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa4c101780 0x7ffa4c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.511+0000 7ffa50c16700 1 -- 172.21.15.102:0/3425402157 >> 172.21.15.102:0/3425402157 conn(0x7ffa4c0fc640 msgr2=0x7ffa4c0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:41.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.512+0000 7ffa50c16700 1 -- 172.21.15.102:0/3425402157 shutdown_connections 2024-05-13T22:54:41.491 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.512+0000 7ffa50c16700 1 -- 172.21.15.102:0/3425402157 wait complete. 2024-05-13T22:54:41.491 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:54:41.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.617+0000 7f085ce27700 1 -- 172.21.15.102:0/2135342497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0858100d70 msgr2=0x7f08581048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.617+0000 7f085ce27700 1 --2- 172.21.15.102:0/2135342497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0858100d70 0x7f08581048a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0848009910 tx=0x7f0848009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:41.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.617+0000 7f085ce27700 1 -- 172.21.15.102:0/2135342497 shutdown_connections 2024-05-13T22:54:41.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.617+0000 7f085ce27700 1 --2- 172.21.15.102:0/2135342497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0858100d70 0x7f08581048a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.617+0000 7f085ce27700 1 --2- 172.21.15.102:0/2135342497 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08581003c0 0x7f08581007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.617+0000 7f085ce27700 1 -- 172.21.15.102:0/2135342497 >> 172.21.15.102:0/2135342497 conn(0x7f0858075e30 msgr2=0x7f08580fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:41.594 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.617+0000 7f085ce27700 1 -- 172.21.15.102:0/2135342497 shutdown_connections 2024-05-13T22:54:41.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.617+0000 7f085ce27700 1 -- 172.21.15.102:0/2135342497 wait complete. 2024-05-13T22:54:41.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.618+0000 7f085ce27700 1 Processor -- start 2024-05-13T22:54:41.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.618+0000 7f085ce27700 1 -- start start 2024-05-13T22:54:41.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.618+0000 7f085ce27700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08581003c0 0x7f0858197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:41.598 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.618+0000 7f085ce27700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0858100d70 0x7f0858197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.618+0000 7f085ce27700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08581982d0 con 0x7f0858100d70 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.618+0000 7f085ce27700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0858198440 con 0x7f08581003c0 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.618+0000 7f0855d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0858100d70 0x7f0858197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f0855d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0858100d70 0x7f0858197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50890/0 (socket says 172.21.15.102:50890) 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f0855d9b700 1 -- 172.21.15.102:0/4050967096 learned_addr learned my addr 172.21.15.102:0/4050967096 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f085659c700 1 --2- 172.21.15.102:0/4050967096 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08581003c0 0x7f0858197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f0855d9b700 1 -- 172.21.15.102:0/4050967096 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08581003c0 msgr2=0x7f0858197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f0855d9b700 1 --2- 172.21.15.102:0/4050967096 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08581003c0 0x7f0858197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f0855d9b700 1 -- 172.21.15.102:0/4050967096 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08480095f0 con 0x7f0858100d70 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f0855d9b700 1 --2- 172.21.15.102:0/4050967096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0858100d70 0x7f0858197cb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0848000c00 tx=0x7f084800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f08477fe700 1 -- 172.21.15.102:0/4050967096 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084801d070 con 0x7f0858100d70 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f08477fe700 1 -- 172.21.15.102:0/4050967096 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0848005dc0 con 0x7f0858100d70 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f08477fe700 1 -- 172.21.15.102:0/4050967096 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0848017400 con 0x7f0858100d70 2024-05-13T22:54:41.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f085ce27700 1 -- 172.21.15.102:0/4050967096 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08581a07e0 con 0x7f0858100d70 2024-05-13T22:54:41.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.619+0000 7f085ce27700 1 -- 172.21.15.102:0/4050967096 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08581a0bf0 con 0x7f0858100d70 2024-05-13T22:54:41.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.620+0000 7f085ce27700 1 -- 172.21.15.102:0/4050967096 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08581082a0 con 0x7f0858100d70 2024-05-13T22:54:41.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.621+0000 7f08477fe700 1 -- 172.21.15.102:0/4050967096 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f084800f460 con 0x7f0858100d70 2024-05-13T22:54:41.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.621+0000 7f08477fe700 1 --2- 172.21.15.102:0/4050967096 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f084006e200 0x7f08400706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:41.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.621+0000 7f08477fe700 1 -- 172.21.15.102:0/4050967096 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f084808f550 con 0x7f0858100d70 2024-05-13T22:54:41.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.621+0000 7f085659c700 1 --2- 172.21.15.102:0/4050967096 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f084006e200 0x7f08400706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:41.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.622+0000 7f085659c700 1 --2- 172.21.15.102:0/4050967096 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f084006e200 0x7f08400706c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f084c007be0 tx=0x7f084c0073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:41.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.626+0000 7f08477fe700 1 -- 172.21.15.102:0/4050967096 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f08480588e0 con 0x7f0858100d70 2024-05-13T22:54:41.820 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.844+0000 7f085ce27700 1 -- 172.21.15.102:0/4050967096 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f08580ff770 con 0x7f084006e200 2024-05-13T22:54:41.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.845+0000 7f08477fe700 1 -- 172.21.15.102:0/4050967096 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f08580ff770 con 0x7f084006e200 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr", 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout: "mon" 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "7/23 daemons upgraded", 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading osd daemons", 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:54:41.822 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:54:41.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.847+0000 7f085ce27700 1 -- 172.21.15.102:0/4050967096 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f084006e200 msgr2=0x7f08400706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.847+0000 7f085ce27700 1 --2- 172.21.15.102:0/4050967096 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f084006e200 0x7f08400706c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f084c007be0 tx=0x7f084c0073d0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.847+0000 7f085ce27700 1 -- 172.21.15.102:0/4050967096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0858100d70 msgr2=0x7f0858197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.847+0000 7f085ce27700 1 --2- 172.21.15.102:0/4050967096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0858100d70 0x7f0858197cb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0848000c00 tx=0x7f084800bd70 comp rx=0 tx=0).stop 2024-05-13T22:54:41.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.847+0000 7f085ce27700 1 -- 172.21.15.102:0/4050967096 shutdown_connections 2024-05-13T22:54:41.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.847+0000 7f085ce27700 1 --2- 172.21.15.102:0/4050967096 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f084006e200 0x7f08400706c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.847+0000 7f085ce27700 1 --2- 172.21.15.102:0/4050967096 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08581003c0 0x7f0858197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.847+0000 7f085ce27700 1 --2- 172.21.15.102:0/4050967096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0858100d70 0x7f0858197cb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.847+0000 7f085ce27700 1 -- 172.21.15.102:0/4050967096 >> 172.21.15.102:0/4050967096 conn(0x7f0858075e30 msgr2=0x7f0858103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:41.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.847+0000 7f085ce27700 1 -- 172.21.15.102:0/4050967096 shutdown_connections 2024-05-13T22:54:41.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.847+0000 7f085ce27700 1 -- 172.21.15.102:0/4050967096 wait complete. 2024-05-13T22:54:41.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.957+0000 7f101bbeb700 1 -- 172.21.15.102:0/1510777889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10141001c0 msgr2=0x7f1014100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.957+0000 7f101bbeb700 1 --2- 172.21.15.102:0/1510777889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10141001c0 0x7f1014100640 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1004009910 tx=0x7f1004009c20 comp rx=0 tx=0).stop 2024-05-13T22:54:41.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.957+0000 7f101bbeb700 1 -- 172.21.15.102:0/1510777889 shutdown_connections 2024-05-13T22:54:41.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.957+0000 7f101bbeb700 1 --2- 172.21.15.102:0/1510777889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10141001c0 0x7f1014100640 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.957+0000 7f101bbeb700 1 --2- 172.21.15.102:0/1510777889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1014104a10 0x7f1014104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.957+0000 7f101bbeb700 1 -- 172.21.15.102:0/1510777889 >> 172.21.15.102:0/1510777889 conn(0x7f1014075e30 msgr2=0x7f10140fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:41.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.957+0000 7f101bbeb700 1 -- 172.21.15.102:0/1510777889 shutdown_connections 2024-05-13T22:54:41.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.957+0000 7f101bbeb700 1 -- 172.21.15.102:0/1510777889 wait complete. 2024-05-13T22:54:41.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.958+0000 7f101bbeb700 1 Processor -- start 2024-05-13T22:54:41.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.958+0000 7f101bbeb700 1 -- start start 2024-05-13T22:54:41.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.958+0000 7f101bbeb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10141001c0 0x7f101419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:41.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.958+0000 7f101bbeb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1014104a10 0x7f101419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.958+0000 7f101bbeb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101419d0d0 con 0x7f1014104a10 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.958+0000 7f101bbeb700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1014196580 con 0x7f10141001c0 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.958+0000 7f1019186700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1014104a10 0x7f101419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.958+0000 7f1019186700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1014104a10 0x7f101419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50906/0 (socket says 172.21.15.102:50906) 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.958+0000 7f1019186700 1 -- 172.21.15.102:0/3440446306 learned_addr learned my addr 172.21.15.102:0/3440446306 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.958+0000 7f1019987700 1 --2- 172.21.15.102:0/3440446306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10141001c0 0x7f101419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.959+0000 7f1019186700 1 -- 172.21.15.102:0/3440446306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10141001c0 msgr2=0x7f101419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.959+0000 7f1019186700 1 --2- 172.21.15.102:0/3440446306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10141001c0 0x7f101419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.959+0000 7f1019186700 1 -- 172.21.15.102:0/3440446306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10040095f0 con 0x7f1014104a10 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.959+0000 7f1019186700 1 --2- 172.21.15.102:0/3440446306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1014104a10 0x7f101419ca40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1004006010 tx=0x7f100400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.959+0000 7f100affd700 1 -- 172.21.15.102:0/3440446306 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100401d070 con 0x7f1014104a10 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.959+0000 7f100affd700 1 -- 172.21.15.102:0/3440446306 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1004004fb0 con 0x7f1014104a10 2024-05-13T22:54:41.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.959+0000 7f100affd700 1 -- 172.21.15.102:0/3440446306 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100400f690 con 0x7f1014104a10 2024-05-13T22:54:41.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.959+0000 7f101bbeb700 1 -- 172.21.15.102:0/3440446306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1014196800 con 0x7f1014104a10 2024-05-13T22:54:41.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.959+0000 7f101bbeb700 1 -- 172.21.15.102:0/3440446306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1014196cf0 con 0x7f1014104a10 2024-05-13T22:54:41.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.960+0000 7f101bbeb700 1 -- 172.21.15.102:0/3440446306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1014108f30 con 0x7f1014104a10 2024-05-13T22:54:41.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.961+0000 7f100affd700 1 -- 172.21.15.102:0/3440446306 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f10040228a0 con 0x7f1014104a10 2024-05-13T22:54:41.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.961+0000 7f100affd700 1 --2- 172.21.15.102:0/3440446306 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f100006e200 0x7f10000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:54:41.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.961+0000 7f100affd700 1 -- 172.21.15.102:0/3440446306 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f100408f940 con 0x7f1014104a10 2024-05-13T22:54:41.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.961+0000 7f1019987700 1 --2- 172.21.15.102:0/3440446306 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f100006e200 0x7f10000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:54:41.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.962+0000 7f1019987700 1 --2- 172.21.15.102:0/3440446306 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f100006e200 0x7f10000706c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1010005fd0 tx=0x7f101000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:54:41.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:41.966+0000 7f100affd700 1 -- 172.21.15.102:0/3440446306 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f100405c800 con 0x7f1014104a10 2024-05-13T22:54:42.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.258+0000 7f101bbeb700 1 -- 172.21.15.102:0/3440446306 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1014108910 con 0x7f1014104a10 2024-05-13T22:54:42.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.258+0000 7f100affd700 1 -- 172.21.15.102:0/3440446306 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f100405c390 con 0x7f1014104a10 2024-05-13T22:54:42.235 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:54:42.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.261+0000 7f101bbeb700 1 -- 172.21.15.102:0/3440446306 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f100006e200 msgr2=0x7f10000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:42.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.261+0000 7f101bbeb700 1 --2- 172.21.15.102:0/3440446306 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f100006e200 0x7f10000706c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1010005fd0 tx=0x7f101000a560 comp rx=0 tx=0).stop 2024-05-13T22:54:42.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.261+0000 7f101bbeb700 1 -- 172.21.15.102:0/3440446306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1014104a10 msgr2=0x7f101419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:54:42.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.261+0000 7f101bbeb700 1 --2- 172.21.15.102:0/3440446306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1014104a10 0x7f101419ca40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1004006010 tx=0x7f100400be20 comp rx=0 tx=0).stop 2024-05-13T22:54:42.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.261+0000 7f101bbeb700 1 -- 172.21.15.102:0/3440446306 shutdown_connections 2024-05-13T22:54:42.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.261+0000 7f101bbeb700 1 --2- 172.21.15.102:0/3440446306 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f100006e200 0x7f10000706c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:42.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.261+0000 7f101bbeb700 1 --2- 172.21.15.102:0/3440446306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10141001c0 0x7f101419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:42.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.261+0000 7f101bbeb700 1 --2- 172.21.15.102:0/3440446306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1014104a10 0x7f101419ca40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:54:42.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.261+0000 7f101bbeb700 1 -- 172.21.15.102:0/3440446306 >> 172.21.15.102:0/3440446306 conn(0x7f1014075e30 msgr2=0x7f10140ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:54:42.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.261+0000 7f101bbeb700 1 -- 172.21.15.102:0/3440446306 shutdown_connections 2024-05-13T22:54:42.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:54:42.261+0000 7f101bbeb700 1 -- 172.21.15.102:0/3440446306 wait complete. 2024-05-13T22:54:42.375 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:42 smithi102 ceph-mon[111694]: Upgrade: Updating osd.1 2024-05-13T22:54:42.375 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:42 smithi102 ceph-mon[111694]: Deploying daemon osd.1 on smithi102 2024-05-13T22:54:42.375 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:42 smithi102 ceph-mon[111694]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:42.375 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:42 smithi102 ceph-mon[111694]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:42.375 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:42 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3425402157' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:54:42.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:42 smithi130 ceph-mon[84265]: Upgrade: Updating osd.1 2024-05-13T22:54:42.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:42 smithi130 ceph-mon[84265]: Deploying daemon osd.1 on smithi102 2024-05-13T22:54:42.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:42 smithi130 ceph-mon[84265]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:42.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:42 smithi130 ceph-mon[84265]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:42.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:42 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3425402157' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:54:43.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:43 smithi130 ceph-mon[84265]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:43.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:43 smithi130 ceph-mon[84265]: pgmap v66: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:54:43.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:43 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3440446306' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:54:43.420 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:43 smithi102 ceph-mon[111694]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:54:43.420 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:43 smithi102 ceph-mon[111694]: pgmap v66: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:54:43.420 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:43 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3440446306' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:54:43.676 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:43 smithi102 systemd[1]: Stopping Ceph osd.1 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:54:44.150 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:43 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[54781]: 2024-05-13T22:54:43.714+0000 7f8a2e814700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T22:54:44.150 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:43 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[54781]: 2024-05-13T22:54:43.714+0000 7f8a2e814700 -1 osd.1 55 *** Got signal Terminated *** 2024-05-13T22:54:44.150 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:43 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[54781]: 2024-05-13T22:54:43.714+0000 7f8a2e814700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T22:54:44.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:44 smithi102 ceph-mon[111694]: osd.1 marked itself down and dead 2024-05-13T22:54:44.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:44 smithi130 ceph-mon[84265]: osd.1 marked itself down and dead 2024-05-13T22:54:44.909 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:44 smithi102 podman[129018]: 2024-05-13 22:54:44.657614533 +0000 UTC m=+1.064397241 container died e83df24af944169d258fcb447354c423c2afe1bc11a37902af4b2a927d827915 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-13T22:54:45.176 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:45 smithi102 podman[129018]: 2024-05-13 22:54:45.02353243 +0000 UTC m=+1.430315138 container cleanup e83df24af944169d258fcb447354c423c2afe1bc11a37902af4b2a927d827915 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=HEAD, ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.29.1) 2024-05-13T22:54:45.176 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:45 smithi102 bash[129018]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1 2024-05-13T22:54:45.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:45 smithi102 ceph-mon[111694]: pgmap v67: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:45.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:45 smithi102 ceph-mon[111694]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:54:45.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:45 smithi102 ceph-mon[111694]: osdmap e56: 6 total, 5 up, 6 in 2024-05-13T22:54:45.627 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:45 smithi102 podman[129035]: 2024-05-13 22:54:45.325315977 +0000 UTC m=+0.661958734 container remove e83df24af944169d258fcb447354c423c2afe1bc11a37902af4b2a927d827915 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=-18.2.1) 2024-05-13T22:54:45.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:45 smithi130 ceph-mon[84265]: pgmap v67: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:54:45.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:45 smithi130 ceph-mon[84265]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:54:45.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:45 smithi130 ceph-mon[84265]: osdmap e56: 6 total, 5 up, 6 in 2024-05-13T22:54:45.888 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:45 smithi102 podman[129122]: 2024-05-13 22:54:45.651099645 +0000 UTC m=+0.112658076 container create 96a5bcefbcd3cc5119a77251242f715bcbce264e0d0ae33dc2b24f6f56a790f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-13T22:54:45.888 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:45 smithi102 podman[129122]: 2024-05-13 22:54:45.555648118 +0000 UTC m=+0.017206547 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:54:45.888 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:45 smithi102 podman[129122]: 2024-05-13 22:54:45.912236777 +0000 UTC m=+0.373795206 container init 96a5bcefbcd3cc5119a77251242f715bcbce264e0d0ae33dc2b24f6f56a790f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-c3e5514, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-13T22:54:46.170 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:45 smithi102 podman[129122]: 2024-05-13 22:54:45.915616429 +0000 UTC m=+0.377174863 container start 96a5bcefbcd3cc5119a77251242f715bcbce264e0d0ae33dc2b24f6f56a790f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-deactivate, RELEASE=squid-c3e5514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-13T22:54:46.170 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:45 smithi102 podman[129122]: 2024-05-13 22:54:45.995444031 +0000 UTC m=+0.457002462 container attach 96a5bcefbcd3cc5119a77251242f715bcbce264e0d0ae33dc2b24f6f56a790f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-13T22:54:46.426 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:46 smithi102 podman[129122]: 2024-05-13 22:54:46.222568889 +0000 UTC m=+0.684127318 container died 96a5bcefbcd3cc5119a77251242f715bcbce264e0d0ae33dc2b24f6f56a790f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T22:54:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: osdmap e57: 6 total, 5 up, 6 in 2024-05-13T22:54:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [0, 1]}]: dispatch 2024-05-13T22:54:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 2]}]: dispatch 2024-05-13T22:54:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]: dispatch 2024-05-13T22:54:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 1]}]: dispatch 2024-05-13T22:54:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [3, 2]}]: dispatch 2024-05-13T22:54:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 1]}]: dispatch 2024-05-13T22:54:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 2]}]: dispatch 2024-05-13T22:54:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [4, 5]}]: dispatch 2024-05-13T22:54:46.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 1]}]: dispatch 2024-05-13T22:54:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 0]}]: dispatch 2024-05-13T22:54:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:46.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:54:46.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: osdmap e57: 6 total, 5 up, 6 in 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [0, 1]}]: dispatch 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 2]}]: dispatch 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]: dispatch 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 1]}]: dispatch 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [3, 2]}]: dispatch 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 1]}]: dispatch 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 2]}]: dispatch 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [4, 5]}]: dispatch 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 1]}]: dispatch 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 0]}]: dispatch 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:46.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:54:47.175 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:46 smithi102 podman[129148]: 2024-05-13 22:54:46.840986724 +0000 UTC m=+0.612761535 container remove 96a5bcefbcd3cc5119a77251242f715bcbce264e0d0ae33dc2b24f6f56a790f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-13T22:54:47.175 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:46 smithi102 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.1.service: Deactivated successfully. 2024-05-13T22:54:47.175 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:46 smithi102 systemd[1]: Stopped Ceph osd.1 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:54:47.175 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:46 smithi102 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.1.service: Consumed 1min 9.062s CPU time. 2024-05-13T22:54:47.175 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:47 smithi102 systemd[1]: Starting Ceph osd.1 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:54:47.426 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:47 smithi102 podman[129250]: 2024-05-13 22:54:47.300980216 +0000 UTC m=+0.116360216 container create cd80628b0c511dd38a2ba9fd421f77ad65033c4ab67fc3550e0201541d752f15 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T22:54:47.426 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:47 smithi102 podman[129250]: 2024-05-13 22:54:47.205441932 +0000 UTC m=+0.020821955 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:54:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-13T22:54:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [0, 1]}]': finished 2024-05-13T22:54:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 2]}]': finished 2024-05-13T22:54:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]': finished 2024-05-13T22:54:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 1]}]': finished 2024-05-13T22:54:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [3, 2]}]': finished 2024-05-13T22:54:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 1]}]': finished 2024-05-13T22:54:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 2]}]': finished 2024-05-13T22:54:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [4, 5]}]': finished 2024-05-13T22:54:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 1]}]': finished 2024-05-13T22:54:47.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 0]}]': finished 2024-05-13T22:54:47.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:47 smithi102 ceph-mon[111694]: osdmap e58: 6 total, 5 up, 6 in 2024-05-13T22:54:47.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: pgmap v70: 65 pgs: 12 stale+active+clean, 53 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-13T22:54:47.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [0, 1]}]': finished 2024-05-13T22:54:47.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [3, 2]}]': finished 2024-05-13T22:54:47.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [0, 1]}]': finished 2024-05-13T22:54:47.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 1]}]': finished 2024-05-13T22:54:47.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [3, 2]}]': finished 2024-05-13T22:54:47.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [3, 1]}]': finished 2024-05-13T22:54:47.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 2]}]': finished 2024-05-13T22:54:47.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [4, 5]}]': finished 2024-05-13T22:54:47.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 1]}]': finished 2024-05-13T22:54:47.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 0]}]': finished 2024-05-13T22:54:47.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:47 smithi130 ceph-mon[84265]: osdmap e58: 6 total, 5 up, 6 in 2024-05-13T22:54:47.927 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:47 smithi102 podman[129250]: 2024-05-13 22:54:47.553863372 +0000 UTC m=+0.369243379 container init cd80628b0c511dd38a2ba9fd421f77ad65033c4ab67fc3550e0201541d752f15 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514) 2024-05-13T22:54:47.927 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:47 smithi102 podman[129250]: 2024-05-13 22:54:47.557315693 +0000 UTC m=+0.372695702 container start cd80628b0c511dd38a2ba9fd421f77ad65033c4ab67fc3550e0201541d752f15 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-13T22:54:47.927 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:47 smithi102 podman[129250]: 2024-05-13 22:54:47.628690105 +0000 UTC m=+0.444070116 container attach cd80628b0c511dd38a2ba9fd421f77ad65033c4ab67fc3550e0201541d752f15 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T22:54:48.326 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:48 smithi102 ceph-mon[111694]: osdmap e59: 6 total, 5 up, 6 in 2024-05-13T22:54:48.327 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:48 smithi102 ceph-mon[111694]: osdmap e60: 6 total, 5 up, 6 in 2024-05-13T22:54:48.327 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate[129266]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-13T22:54:48.327 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate[129266]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-13T22:54:48.327 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 bash[129250]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-13T22:54:48.327 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 bash[129250]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-13T22:54:48.327 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 bash[129250]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-13T22:54:48.327 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 bash[129250]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-13T22:54:48.327 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 bash[129250]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-13T22:54:48.327 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 bash[129250]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-13T22:54:48.328 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 bash[129250]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-13T22:54:48.328 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate[129266]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-13T22:54:48.328 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate[129266]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-13T22:54:48.328 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate[129266]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-13T22:54:48.328 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate[129266]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-13T22:54:48.328 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate[129266]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-13T22:54:48.328 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 podman[129250]: 2024-05-13 22:54:48.033916444 +0000 UTC m=+0.849296449 container died cd80628b0c511dd38a2ba9fd421f77ad65033c4ab67fc3550e0201541d752f15 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=squid-c3e5514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T22:54:48.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:48 smithi130 ceph-mon[84265]: osdmap e59: 6 total, 5 up, 6 in 2024-05-13T22:54:48.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:48 smithi130 ceph-mon[84265]: osdmap e60: 6 total, 5 up, 6 in 2024-05-13T22:54:48.926 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 podman[129390]: 2024-05-13 22:54:48.650150943 +0000 UTC m=+0.610383763 container remove cd80628b0c511dd38a2ba9fd421f77ad65033c4ab67fc3550e0201541d752f15 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-activate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-c3e5514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-13T22:54:48.926 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 podman[129451]: 2024-05-13 22:54:48.774786239 +0000 UTC m=+0.018797714 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:54:48.926 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:48 smithi102 podman[129451]: 2024-05-13 22:54:48.89280593 +0000 UTC m=+0.136817390 container create c3e9e68b06fd3826d5f699ef464e00e44c4540f0cf1d8c4dcd755e148579e7d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS) 2024-05-13T22:54:49.282 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:49 smithi102 ceph-mon[111694]: pgmap v73: 65 pgs: 5 unknown, 4 peering, 9 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 29 active+clean; 199 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21/210 objects degraded (10.000%) 2024-05-13T22:54:49.282 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:49 smithi102 ceph-mon[111694]: Health check failed: Degraded data redundancy: 21/210 objects degraded (10.000%), 9 pgs degraded (PG_DEGRADED) 2024-05-13T22:54:49.550 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:49 smithi102 podman[129451]: 2024-05-13 22:54:49.312392907 +0000 UTC m=+0.556404368 container init c3e9e68b06fd3826d5f699ef464e00e44c4540f0cf1d8c4dcd755e148579e7d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-c3e5514, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-13T22:54:49.550 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:49 smithi102 podman[129451]: 2024-05-13 22:54:49.316623773 +0000 UTC m=+0.560635242 container start c3e9e68b06fd3826d5f699ef464e00e44c4540f0cf1d8c4dcd755e148579e7d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-c3e5514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-13T22:54:49.550 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:49 smithi102 ceph-osd[129470]: -- 172.21.15.102:0/2363651540 <== mon.0 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d89465680 con 0x558d8942f000 2024-05-13T22:54:49.550 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:49 smithi102 bash[129451]: c3e9e68b06fd3826d5f699ef464e00e44c4540f0cf1d8c4dcd755e148579e7d7 2024-05-13T22:54:49.550 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:49 smithi102 systemd[1]: Started Ceph osd.1 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:54:49.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:49 smithi130 ceph-mon[84265]: pgmap v73: 65 pgs: 5 unknown, 4 peering, 9 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 29 active+clean; 199 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21/210 objects degraded (10.000%) 2024-05-13T22:54:49.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:49 smithi130 ceph-mon[84265]: Health check failed: Degraded data redundancy: 21/210 objects degraded (10.000%), 9 pgs degraded (PG_DEGRADED) 2024-05-13T22:54:50.176 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:49 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[129465]: 2024-05-13T22:54:49.859+0000 7f8a7eba87c0 -1 Falling back to public interface 2024-05-13T22:54:50.519 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:50 smithi102 ceph-mon[111694]: osdmap e61: 6 total, 5 up, 6 in 2024-05-13T22:54:50.520 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:50 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:50.520 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:50 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:50.520 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:50 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:54:50.520 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:50 smithi102 ceph-mon[111694]: pgmap v76: 65 pgs: 1 active+clean+remapped, 26 active+clean, 19 active+undersized, 1 active+undersized+remapped, 10 active+undersized+degraded, 4 peering, 4 unknown; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/228 objects degraded (9.649%); 6/228 objects misplaced (2.632%); 3.2 MiB/s, 1 objects/s recovering 2024-05-13T22:54:50.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:50 smithi130 ceph-mon[84265]: osdmap e61: 6 total, 5 up, 6 in 2024-05-13T22:54:50.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:50 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:50.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:50 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:50.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:50 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:54:50.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:50 smithi130 ceph-mon[84265]: pgmap v76: 65 pgs: 1 active+clean+remapped, 26 active+clean, 19 active+undersized, 1 active+undersized+remapped, 10 active+undersized+degraded, 4 peering, 4 unknown; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/228 objects degraded (9.649%); 6/228 objects misplaced (2.632%); 3.2 MiB/s, 1 objects/s recovering 2024-05-13T22:54:51.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:51 smithi102 ceph-mon[111694]: osdmap e62: 6 total, 5 up, 6 in 2024-05-13T22:54:51.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:51 smithi102 ceph-mon[111694]: osdmap e63: 6 total, 5 up, 6 in 2024-05-13T22:54:51.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:51 smithi130 ceph-mon[84265]: osdmap e62: 6 total, 5 up, 6 in 2024-05-13T22:54:51.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:51 smithi130 ceph-mon[84265]: osdmap e63: 6 total, 5 up, 6 in 2024-05-13T22:54:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:53 smithi130 ceph-mon[84265]: pgmap v79: 65 pgs: 2 remapped+peering, 8 active+clean+remapped, 26 active+clean, 19 active+undersized, 10 active+undersized+degraded; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 MiB/s rd, 88 MiB/s wr, 840 op/s; 22/231 objects degraded (9.524%); 17/231 objects misplaced (7.359%); 3.2 MiB/s, 351 objects/s recovering 2024-05-13T22:54:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:53 smithi130 ceph-mon[84265]: osdmap e64: 6 total, 5 up, 6 in 2024-05-13T22:54:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:53 smithi130 ceph-mon[84265]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-13T22:54:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:53 smithi102 ceph-mon[111694]: pgmap v79: 65 pgs: 2 remapped+peering, 8 active+clean+remapped, 26 active+clean, 19 active+undersized, 10 active+undersized+degraded; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 MiB/s rd, 88 MiB/s wr, 840 op/s; 22/231 objects degraded (9.524%); 17/231 objects misplaced (7.359%); 3.2 MiB/s, 351 objects/s recovering 2024-05-13T22:54:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:53 smithi102 ceph-mon[111694]: osdmap e64: 6 total, 5 up, 6 in 2024-05-13T22:54:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:53 smithi102 ceph-mon[111694]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-13T22:54:54.676 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:54 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[129465]: 2024-05-13T22:54:54.372+0000 7f8a7eba87c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-05-13T22:54:55.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:55 smithi102 ceph-mon[111694]: pgmap v81: 65 pgs: 2 remapped+peering, 8 active+clean+remapped, 26 active+clean, 19 active+undersized, 10 active+undersized+degraded; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 70 MiB/s wr, 398 op/s; 22/231 objects degraded (9.524%); 17/231 objects misplaced (7.359%); 0 B/s, 289 objects/s recovering 2024-05-13T22:54:55.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:55 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:55.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:55 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:55.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:55 smithi102 ceph-mon[111694]: from='osd.1 [v2:172.21.15.102:6810/272360893,v1:172.21.15.102:6811/272360893]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-13T22:54:55.177 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:54 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[129465]: 2024-05-13T22:54:54.937+0000 7f8a7eba87c0 -1 osd.1 55 log_to_monitors true 2024-05-13T22:54:55.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:55 smithi130 ceph-mon[84265]: pgmap v81: 65 pgs: 2 remapped+peering, 8 active+clean+remapped, 26 active+clean, 19 active+undersized, 10 active+undersized+degraded; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 70 MiB/s wr, 398 op/s; 22/231 objects degraded (9.524%); 17/231 objects misplaced (7.359%); 0 B/s, 289 objects/s recovering 2024-05-13T22:54:55.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:55 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:55.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:55 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:55.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:55 smithi130 ceph-mon[84265]: from='osd.1 [v2:172.21.15.102:6810/272360893,v1:172.21.15.102:6811/272360893]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-13T22:54:56.176 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 22:54:55 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[129465]: 2024-05-13T22:54:55.764+0000 7f8a6ffff700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:54:56.734 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:56 smithi102 ceph-mon[111694]: from='osd.1 [v2:172.21.15.102:6810/272360893,v1:172.21.15.102:6811/272360893]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-13T22:54:56.734 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:56 smithi102 ceph-mon[111694]: osdmap e65: 6 total, 5 up, 6 in 2024-05-13T22:54:56.734 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:56 smithi102 ceph-mon[111694]: from='osd.1 [v2:172.21.15.102:6810/272360893,v1:172.21.15.102:6811/272360893]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:54:56.734 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:56 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:56.735 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:56 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:56.735 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:56 smithi102 ceph-mon[111694]: pgmap v83: 65 pgs: 2 remapped+peering, 8 active+clean+remapped, 26 active+clean, 19 active+undersized, 10 active+undersized+degraded; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 58 MiB/s wr, 330 op/s; 22/231 objects degraded (9.524%); 17/231 objects misplaced (7.359%); 0 B/s, 240 objects/s recovering 2024-05-13T22:54:56.735 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:56 smithi102 ceph-mon[111694]: Health check update: Degraded data redundancy: 22/231 objects degraded (9.524%), 10 pgs degraded (PG_DEGRADED) 2024-05-13T22:54:57.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:56 smithi130 ceph-mon[84265]: from='osd.1 [v2:172.21.15.102:6810/272360893,v1:172.21.15.102:6811/272360893]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-13T22:54:57.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:56 smithi130 ceph-mon[84265]: osdmap e65: 6 total, 5 up, 6 in 2024-05-13T22:54:57.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:56 smithi130 ceph-mon[84265]: from='osd.1 [v2:172.21.15.102:6810/272360893,v1:172.21.15.102:6811/272360893]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:54:57.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:56 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:57.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:56 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:54:57.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:56 smithi130 ceph-mon[84265]: pgmap v83: 65 pgs: 2 remapped+peering, 8 active+clean+remapped, 26 active+clean, 19 active+undersized, 10 active+undersized+degraded; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 58 MiB/s wr, 330 op/s; 22/231 objects degraded (9.524%); 17/231 objects misplaced (7.359%); 0 B/s, 240 objects/s recovering 2024-05-13T22:54:57.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:56 smithi130 ceph-mon[84265]: Health check update: Degraded data redundancy: 22/231 objects degraded (9.524%), 10 pgs degraded (PG_DEGRADED) 2024-05-13T22:54:57.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:57 smithi102 ceph-mon[111694]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:54:57.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:57 smithi102 ceph-mon[111694]: osd.1 [v2:172.21.15.102:6810/272360893,v1:172.21.15.102:6811/272360893] boot 2024-05-13T22:54:57.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:57 smithi102 ceph-mon[111694]: osdmap e66: 6 total, 6 up, 6 in 2024-05-13T22:54:57.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:57 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:54:58.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:57 smithi130 ceph-mon[84265]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:54:58.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:57 smithi130 ceph-mon[84265]: osd.1 [v2:172.21.15.102:6810/272360893,v1:172.21.15.102:6811/272360893] boot 2024-05-13T22:54:58.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:57 smithi130 ceph-mon[84265]: osdmap e66: 6 total, 6 up, 6 in 2024-05-13T22:54:58.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:57 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-13T22:54:59.146 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:58 smithi102 ceph-mon[111694]: osdmap e67: 6 total, 6 up, 6 in 2024-05-13T22:54:59.146 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:58 smithi102 ceph-mon[111694]: pgmap v86: 65 pgs: 19 peering, 9 remapped+peering, 1 active+clean+remapped, 26 active+clean, 6 active+undersized, 4 active+undersized+degraded; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 866 B/s rd, 1 op/s; 8/231 objects degraded (3.463%); 0 B/s, 119 objects/s recovering 2024-05-13T22:54:59.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:58 smithi130 ceph-mon[84265]: osdmap e67: 6 total, 6 up, 6 in 2024-05-13T22:54:59.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:58 smithi130 ceph-mon[84265]: pgmap v86: 65 pgs: 19 peering, 9 remapped+peering, 1 active+clean+remapped, 26 active+clean, 6 active+undersized, 4 active+undersized+degraded; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 866 B/s rd, 1 op/s; 8/231 objects degraded (3.463%); 0 B/s, 119 objects/s recovering 2024-05-13T22:55:00.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-13T22:55:00.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: osdmap e68: 6 total, 6 up, 6 in 2024-05-13T22:55:00.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:00.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:00.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:55:00.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:55:00.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:00.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:55:00.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:00.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:00.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:00.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:54:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-13T22:55:00.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-13T22:55:00.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: osdmap e68: 6 total, 6 up, 6 in 2024-05-13T22:55:00.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:00.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:00.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:55:00.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:55:00.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:00.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:55:00.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:00.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:00.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:00.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:54:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-13T22:55:01.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:00 smithi130 ceph-mon[84265]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-13T22:55:01.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:00 smithi130 ceph-mon[84265]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-05-13T22:55:01.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:00 smithi130 ceph-mon[84265]: osdmap e69: 6 total, 6 up, 6 in 2024-05-13T22:55:01.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:00 smithi130 ceph-mon[84265]: pgmap v89: 65 pgs: 19 peering, 9 remapped+peering, 28 active+clean, 1 active+recovering+undersized+remapped, 4 active+undersized, 4 active+undersized+degraded; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 8/231 objects degraded (3.463%); 0 B/s, 194 objects/s recovering 2024-05-13T22:55:01.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:00 smithi102 ceph-mon[111694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-13T22:55:01.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:00 smithi102 ceph-mon[111694]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-05-13T22:55:01.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:00 smithi102 ceph-mon[111694]: osdmap e69: 6 total, 6 up, 6 in 2024-05-13T22:55:01.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:00 smithi102 ceph-mon[111694]: pgmap v89: 65 pgs: 19 peering, 9 remapped+peering, 28 active+clean, 1 active+recovering+undersized+remapped, 4 active+undersized, 4 active+undersized+degraded; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 8/231 objects degraded (3.463%); 0 B/s, 194 objects/s recovering 2024-05-13T22:55:02.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:01 smithi130 ceph-mon[84265]: osdmap e70: 6 total, 6 up, 6 in 2024-05-13T22:55:02.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:02.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:55:02.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:01 smithi130 ceph-mon[84265]: Health check update: Degraded data redundancy: 8/231 objects degraded (3.463%), 4 pgs degraded (PG_DEGRADED) 2024-05-13T22:55:02.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:01 smithi130 ceph-mon[84265]: osdmap e71: 6 total, 6 up, 6 in 2024-05-13T22:55:02.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:01 smithi102 ceph-mon[111694]: osdmap e70: 6 total, 6 up, 6 in 2024-05-13T22:55:02.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:02.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:55:02.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:01 smithi102 ceph-mon[111694]: Health check update: Degraded data redundancy: 8/231 objects degraded (3.463%), 4 pgs degraded (PG_DEGRADED) 2024-05-13T22:55:02.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:01 smithi102 ceph-mon[111694]: osdmap e71: 6 total, 6 up, 6 in 2024-05-13T22:55:03.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:02 smithi130 ceph-mon[84265]: pgmap v92: 65 pgs: 1 peering, 63 active+clean, 1 active+recovering+undersized+remapped; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s, 0 objects/s recovering 2024-05-13T22:55:03.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:02 smithi102 ceph-mon[111694]: pgmap v92: 65 pgs: 1 peering, 63 active+clean, 1 active+recovering+undersized+remapped; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s, 0 objects/s recovering 2024-05-13T22:55:04.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:03 smithi130 ceph-mon[84265]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-13T22:55:04.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:03 smithi130 ceph-mon[84265]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/231 objects degraded (3.463%), 4 pgs degraded) 2024-05-13T22:55:04.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:03 smithi102 ceph-mon[111694]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-13T22:55:04.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:03 smithi102 ceph-mon[111694]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/231 objects degraded (3.463%), 4 pgs degraded) 2024-05-13T22:55:05.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:04 smithi130 ceph-mon[84265]: pgmap v93: 65 pgs: 1 peering, 63 active+clean, 1 active+recovering+undersized+remapped; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.8 MiB/s, 0 objects/s recovering 2024-05-13T22:55:05.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:04 smithi102 ceph-mon[111694]: pgmap v93: 65 pgs: 1 peering, 63 active+clean, 1 active+recovering+undersized+remapped; 212 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.8 MiB/s, 0 objects/s recovering 2024-05-13T22:55:06.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:06 smithi130 ceph-mon[84265]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-13T22:55:06.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:06 smithi130 ceph-mon[84265]: Cluster is now healthy 2024-05-13T22:55:06.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:06 smithi102 ceph-mon[111694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-13T22:55:06.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:06 smithi102 ceph-mon[111694]: Cluster is now healthy 2024-05-13T22:55:07.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:07 smithi130 ceph-mon[84265]: pgmap v94: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 489 B/s rd, 0 op/s; 3.2 MiB/s, 0 objects/s recovering 2024-05-13T22:55:07.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:07 smithi102 ceph-mon[111694]: pgmap v94: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 489 B/s rd, 0 op/s; 3.2 MiB/s, 0 objects/s recovering 2024-05-13T22:55:09.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:09 smithi130 ceph-mon[84265]: pgmap v95: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 2.5 MiB/s, 0 objects/s recovering 2024-05-13T22:55:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:09 smithi102 ceph-mon[111694]: pgmap v95: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 2.5 MiB/s, 0 objects/s recovering 2024-05-13T22:55:11.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:11 smithi130 ceph-mon[84265]: pgmap v96: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 883 B/s rd, 1 op/s; 2.2 MiB/s, 70 objects/s recovering 2024-05-13T22:55:11.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:11 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:11.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:11 smithi102 ceph-mon[111694]: pgmap v96: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 883 B/s rd, 1 op/s; 2.2 MiB/s, 70 objects/s recovering 2024-05-13T22:55:11.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:11 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.377+0000 7f4590bec700 1 -- 172.21.15.102:0/2991851433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f458c0ffa40 msgr2=0x7f458c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.377+0000 7f4590bec700 1 --2- 172.21.15.102:0/2991851433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f458c0ffa40 0x7f458c0ffec0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f457c009910 tx=0x7f457c009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.377+0000 7f4590bec700 1 -- 172.21.15.102:0/2991851433 shutdown_connections 2024-05-13T22:55:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.377+0000 7f4590bec700 1 --2- 172.21.15.102:0/2991851433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f458c0ffa40 0x7f458c0ffec0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.377+0000 7f4590bec700 1 --2- 172.21.15.102:0/2991851433 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f458c105a40 0x7f458c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.377+0000 7f4590bec700 1 -- 172.21.15.102:0/2991851433 >> 172.21.15.102:0/2991851433 conn(0x7f458c0fb550 msgr2=0x7f458c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.377+0000 7f4590bec700 1 -- 172.21.15.102:0/2991851433 shutdown_connections 2024-05-13T22:55:12.356 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.377+0000 7f4590bec700 1 -- 172.21.15.102:0/2991851433 wait complete. 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f4590bec700 1 Processor -- start 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f4590bec700 1 -- start start 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f4590bec700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f458c0ffa40 0x7f458c195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f4590bec700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f458c105a40 0x7f458c195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f4590bec700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c196260 con 0x7f458c0ffa40 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f4590bec700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c199ff0 con 0x7f458c105a40 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f458a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f458c0ffa40 0x7f458c195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f458a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f458c0ffa40 0x7f458c195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51570/0 (socket says 172.21.15.102:51570) 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f458a59c700 1 -- 172.21.15.102:0/935408763 learned_addr learned my addr 172.21.15.102:0/935408763 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f4589d9b700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f458c105a40 0x7f458c195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f458a59c700 1 -- 172.21.15.102:0/935408763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f458c105a40 msgr2=0x7f458c195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f458a59c700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f458c105a40 0x7f458c195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.357 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f458a59c700 1 -- 172.21.15.102:0/935408763 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c0095f0 con 0x7f458c0ffa40 2024-05-13T22:55:12.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.378+0000 7f4589d9b700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f458c105a40 0x7f458c195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:55:12.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.379+0000 7f458a59c700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f458c0ffa40 0x7f458c195640 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4580007b50 tx=0x7f4580007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:12.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.379+0000 7f457b7fe700 1 -- 172.21.15.102:0/935408763 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4580011070 con 0x7f458c0ffa40 2024-05-13T22:55:12.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.379+0000 7f4590bec700 1 -- 172.21.15.102:0/935408763 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c19a270 con 0x7f458c0ffa40 2024-05-13T22:55:12.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.379+0000 7f457b7fe700 1 -- 172.21.15.102:0/935408763 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f458000c2e0 con 0x7f458c0ffa40 2024-05-13T22:55:12.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.379+0000 7f457b7fe700 1 -- 172.21.15.102:0/935408763 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458000e800 con 0x7f458c0ffa40 2024-05-13T22:55:12.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.379+0000 7f4590bec700 1 -- 172.21.15.102:0/935408763 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c19a7c0 con 0x7f458c0ffa40 2024-05-13T22:55:12.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.380+0000 7f4590bec700 1 -- 172.21.15.102:0/935408763 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c04ea90 con 0x7f458c0ffa40 2024-05-13T22:55:12.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.380+0000 7f457b7fe700 1 -- 172.21.15.102:0/935408763 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4580005340 con 0x7f458c0ffa40 2024-05-13T22:55:12.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.381+0000 7f457b7fe700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f457406e200 0x7f45740706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:12.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.381+0000 7f457b7fe700 1 -- 172.21.15.102:0/935408763 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f458008f890 con 0x7f458c0ffa40 2024-05-13T22:55:12.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.381+0000 7f4589d9b700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f457406e200 0x7f45740706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:12.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.381+0000 7f4589d9b700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f457406e200 0x7f45740706c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f457c00b5c0 tx=0x7f457c005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:12.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.386+0000 7f457b7fe700 1 -- 172.21.15.102:0/935408763 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4580058b00 con 0x7f458c0ffa40 2024-05-13T22:55:12.578 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.601+0000 7f4590bec700 1 -- 172.21.15.102:0/935408763 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f458c19ac90 con 0x7f457406e200 2024-05-13T22:55:12.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.603+0000 7f457b7fe700 1 -- 172.21.15.102:0/935408763 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f458c19ac90 con 0x7f457406e200 2024-05-13T22:55:12.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.605+0000 7f4590bec700 1 -- 172.21.15.102:0/935408763 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f457406e200 msgr2=0x7f45740706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:12.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.605+0000 7f4590bec700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f457406e200 0x7f45740706c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f457c00b5c0 tx=0x7f457c005480 comp rx=0 tx=0).stop 2024-05-13T22:55:12.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.605+0000 7f4590bec700 1 -- 172.21.15.102:0/935408763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f458c0ffa40 msgr2=0x7f458c195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:12.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.605+0000 7f4590bec700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f458c0ffa40 0x7f458c195640 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4580007b50 tx=0x7f4580007f10 comp rx=0 tx=0).stop 2024-05-13T22:55:12.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.605+0000 7f4590bec700 1 -- 172.21.15.102:0/935408763 shutdown_connections 2024-05-13T22:55:12.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.605+0000 7f4590bec700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f458c0ffa40 0x7f458c195640 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.605+0000 7f4590bec700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f457406e200 0x7f45740706c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.605+0000 7f4590bec700 1 --2- 172.21.15.102:0/935408763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f458c105a40 0x7f458c195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.582 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.605+0000 7f4590bec700 1 -- 172.21.15.102:0/935408763 >> 172.21.15.102:0/935408763 conn(0x7f458c0fb550 msgr2=0x7f458c0fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:12.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.605+0000 7f4590bec700 1 -- 172.21.15.102:0/935408763 shutdown_connections 2024-05-13T22:55:12.583 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.605+0000 7f4590bec700 1 -- 172.21.15.102:0/935408763 wait complete. 2024-05-13T22:55:12.592 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:55:12.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.711+0000 7f08349bb700 1 -- 172.21.15.102:0/2238805738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0830101780 msgr2=0x7f08301057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:12.690 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.711+0000 7f08349bb700 1 --2- 172.21.15.102:0/2238805738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0830101780 0x7f08301057d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0824009910 tx=0x7f0824009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:12.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.711+0000 7f08349bb700 1 -- 172.21.15.102:0/2238805738 shutdown_connections 2024-05-13T22:55:12.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.711+0000 7f08349bb700 1 --2- 172.21.15.102:0/2238805738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0830101780 0x7f08301057d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.711+0000 7f08349bb700 1 --2- 172.21.15.102:0/2238805738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0830100dd0 0x7f08301011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.711+0000 7f08349bb700 1 -- 172.21.15.102:0/2238805738 >> 172.21.15.102:0/2238805738 conn(0x7f08300fc640 msgr2=0x7f08300fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:12.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.712+0000 7f08349bb700 1 -- 172.21.15.102:0/2238805738 shutdown_connections 2024-05-13T22:55:12.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.712+0000 7f08349bb700 1 -- 172.21.15.102:0/2238805738 wait complete. 2024-05-13T22:55:12.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.712+0000 7f08349bb700 1 Processor -- start 2024-05-13T22:55:12.692 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.712+0000 7f08349bb700 1 -- start start 2024-05-13T22:55:12.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f08349bb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0830100dd0 0x7f083019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:12.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f08349bb700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0830101780 0x7f083019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:12.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f08349bb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083019f470 con 0x7f0830100dd0 2024-05-13T22:55:12.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f08349bb700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0830198920 con 0x7f0830101780 2024-05-13T22:55:12.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f082e59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0830100dd0 0x7f083019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:12.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f082e59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0830100dd0 0x7f083019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51582/0 (socket says 172.21.15.102:51582) 2024-05-13T22:55:12.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f082e59c700 1 -- 172.21.15.102:0/3183362126 learned_addr learned my addr 172.21.15.102:0/3183362126 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:12.693 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f082dd9b700 1 --2- 172.21.15.102:0/3183362126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0830101780 0x7f083019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f082e59c700 1 -- 172.21.15.102:0/3183362126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0830101780 msgr2=0x7f083019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f082e59c700 1 --2- 172.21.15.102:0/3183362126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0830101780 0x7f083019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f082e59c700 1 -- 172.21.15.102:0/3183362126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08240095f0 con 0x7f0830100dd0 2024-05-13T22:55:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f082e59c700 1 --2- 172.21.15.102:0/3183362126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0830100dd0 0x7f083019e8a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f082000b4b0 tx=0x7f082000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f081f7fe700 1 -- 172.21.15.102:0/3183362126 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0820011800 con 0x7f0830100dd0 2024-05-13T22:55:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f081f7fe700 1 -- 172.21.15.102:0/3183362126 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0820011e20 con 0x7f0830100dd0 2024-05-13T22:55:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f08349bb700 1 -- 172.21.15.102:0/3183362126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0830198c00 con 0x7f0830100dd0 2024-05-13T22:55:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f081f7fe700 1 -- 172.21.15.102:0/3183362126 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f082000f2e0 con 0x7f0830100dd0 2024-05-13T22:55:12.694 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.713+0000 7f08349bb700 1 -- 172.21.15.102:0/3183362126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0830199150 con 0x7f0830100dd0 2024-05-13T22:55:12.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.714+0000 7f08349bb700 1 -- 172.21.15.102:0/3183362126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0830104f50 con 0x7f0830100dd0 2024-05-13T22:55:12.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.715+0000 7f081f7fe700 1 -- 172.21.15.102:0/3183362126 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0820010380 con 0x7f0830100dd0 2024-05-13T22:55:12.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.715+0000 7f081f7fe700 1 --2- 172.21.15.102:0/3183362126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f081806e200 0x7f08180706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:12.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.715+0000 7f081f7fe700 1 -- 172.21.15.102:0/3183362126 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f082008dc20 con 0x7f0830100dd0 2024-05-13T22:55:12.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.715+0000 7f082dd9b700 1 --2- 172.21.15.102:0/3183362126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f081806e200 0x7f08180706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:12.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.716+0000 7f082dd9b700 1 --2- 172.21.15.102:0/3183362126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f081806e200 0x7f08180706c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f08300fce80 tx=0x7f082401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:12.696 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.720+0000 7f081f7fe700 1 -- 172.21.15.102:0/3183362126 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f082005aa40 con 0x7f0830100dd0 2024-05-13T22:55:12.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.941+0000 7f08349bb700 1 -- 172.21.15.102:0/3183362126 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0830199420 con 0x7f081806e200 2024-05-13T22:55:12.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.942+0000 7f081f7fe700 1 -- 172.21.15.102:0/3183362126 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0830199420 con 0x7f081806e200 2024-05-13T22:55:12.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.944+0000 7f08349bb700 1 -- 172.21.15.102:0/3183362126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f081806e200 msgr2=0x7f08180706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:12.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.944+0000 7f08349bb700 1 --2- 172.21.15.102:0/3183362126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f081806e200 0x7f08180706c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f08300fce80 tx=0x7f082401a040 comp rx=0 tx=0).stop 2024-05-13T22:55:12.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.944+0000 7f08349bb700 1 -- 172.21.15.102:0/3183362126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0830100dd0 msgr2=0x7f083019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:12.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.944+0000 7f08349bb700 1 --2- 172.21.15.102:0/3183362126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0830100dd0 0x7f083019e8a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f082000b4b0 tx=0x7f082000b7c0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.944+0000 7f08349bb700 1 -- 172.21.15.102:0/3183362126 shutdown_connections 2024-05-13T22:55:12.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.944+0000 7f08349bb700 1 --2- 172.21.15.102:0/3183362126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0830100dd0 0x7f083019e8a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.944+0000 7f08349bb700 1 --2- 172.21.15.102:0/3183362126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f081806e200 0x7f08180706c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.944+0000 7f08349bb700 1 --2- 172.21.15.102:0/3183362126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0830101780 0x7f083019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:12.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.944+0000 7f08349bb700 1 -- 172.21.15.102:0/3183362126 >> 172.21.15.102:0/3183362126 conn(0x7f08300fc640 msgr2=0x7f08300fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:12.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.945+0000 7f08349bb700 1 -- 172.21.15.102:0/3183362126 shutdown_connections 2024-05-13T22:55:12.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:12.945+0000 7f08349bb700 1 -- 172.21.15.102:0/3183362126 wait complete. 2024-05-13T22:55:13.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.048+0000 7f3d81697700 1 -- 172.21.15.102:0/27165135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d7c0ffbd0 msgr2=0x7f3d7c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:13.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.048+0000 7f3d81697700 1 --2- 172.21.15.102:0/27165135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d7c0ffbd0 0x7f3d7c100050 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009910 tx=0x7f3d6c009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:13.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.048+0000 7f3d81697700 1 -- 172.21.15.102:0/27165135 shutdown_connections 2024-05-13T22:55:13.025 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.048+0000 7f3d81697700 1 --2- 172.21.15.102:0/27165135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d7c0ffbd0 0x7f3d7c100050 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.048+0000 7f3d81697700 1 --2- 172.21.15.102:0/27165135 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c105a40 0x7f3d7c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.048+0000 7f3d81697700 1 -- 172.21.15.102:0/27165135 >> 172.21.15.102:0/27165135 conn(0x7f3d7c0755a0 msgr2=0x7f3d7c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:13.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.048+0000 7f3d81697700 1 -- 172.21.15.102:0/27165135 shutdown_connections 2024-05-13T22:55:13.026 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.048+0000 7f3d81697700 1 -- 172.21.15.102:0/27165135 wait complete. 2024-05-13T22:55:13.028 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.049+0000 7f3d81697700 1 Processor -- start 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.049+0000 7f3d81697700 1 -- start start 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.049+0000 7f3d81697700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d7c0ffbd0 0x7f3d7c197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.049+0000 7f3d81697700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c105a40 0x7f3d7c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.049+0000 7f3d81697700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c198190 con 0x7f3d7c0ffbd0 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.049+0000 7f3d81697700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c198300 con 0x7f3d7c105a40 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.049+0000 7f3d7affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d7c0ffbd0 0x7f3d7c197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.049+0000 7f3d7affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d7c0ffbd0 0x7f3d7c197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51610/0 (socket says 172.21.15.102:51610) 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.049+0000 7f3d7affd700 1 -- 172.21.15.102:0/12180668 learned_addr learned my addr 172.21.15.102:0/12180668 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.049+0000 7f3d7a7fc700 1 --2- 172.21.15.102:0/12180668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c105a40 0x7f3d7c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.050+0000 7f3d7affd700 1 -- 172.21.15.102:0/12180668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c105a40 msgr2=0x7f3d7c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.050+0000 7f3d7affd700 1 --2- 172.21.15.102:0/12180668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c105a40 0x7f3d7c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.050+0000 7f3d7affd700 1 -- 172.21.15.102:0/12180668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c0095f0 con 0x7f3d7c0ffbd0 2024-05-13T22:55:13.029 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.050+0000 7f3d7affd700 1 --2- 172.21.15.102:0/12180668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d7c0ffbd0 0x7f3d7c197570 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000d8d0 tx=0x7f3d7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:13.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.050+0000 7f3d63fff700 1 -- 172.21.15.102:0/12180668 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d70009940 con 0x7f3d7c0ffbd0 2024-05-13T22:55:13.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.050+0000 7f3d63fff700 1 -- 172.21.15.102:0/12180668 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d70010460 con 0x7f3d7c0ffbd0 2024-05-13T22:55:13.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.050+0000 7f3d63fff700 1 -- 172.21.15.102:0/12180668 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d7000f5f0 con 0x7f3d7c0ffbd0 2024-05-13T22:55:13.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.050+0000 7f3d81697700 1 -- 172.21.15.102:0/12180668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c1a05e0 con 0x7f3d7c0ffbd0 2024-05-13T22:55:13.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.050+0000 7f3d81697700 1 -- 172.21.15.102:0/12180668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c1a0b30 con 0x7f3d7c0ffbd0 2024-05-13T22:55:13.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.051+0000 7f3d81697700 1 -- 172.21.15.102:0/12180668 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c107fd0 con 0x7f3d7c0ffbd0 2024-05-13T22:55:13.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.052+0000 7f3d63fff700 1 -- 172.21.15.102:0/12180668 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3d700105d0 con 0x7f3d7c0ffbd0 2024-05-13T22:55:13.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.052+0000 7f3d63fff700 1 --2- 172.21.15.102:0/12180668 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3d6406e200 0x7f3d640706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:13.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.052+0000 7f3d63fff700 1 -- 172.21.15.102:0/12180668 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f3d7008e290 con 0x7f3d7c0ffbd0 2024-05-13T22:55:13.030 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.052+0000 7f3d7a7fc700 1 --2- 172.21.15.102:0/12180668 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3d6406e200 0x7f3d640706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:13.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.053+0000 7f3d7a7fc700 1 --2- 172.21.15.102:0/12180668 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3d6406e200 0x7f3d640706c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c005b40 tx=0x7f3d6c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:13.033 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.057+0000 7f3d63fff700 1 -- 172.21.15.102:0/12180668 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3d70057480 con 0x7f3d7c0ffbd0 2024-05-13T22:55:13.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:13 smithi102 ceph-mon[111694]: pgmap v97: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 1.9 MiB/s, 63 objects/s recovering 2024-05-13T22:55:13.240 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.264+0000 7f3d81697700 1 -- 172.21.15.102:0/12180668 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d7c02cd80 con 0x7f3d6406e200 2024-05-13T22:55:13.251 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.274+0000 7f3d63fff700 1 -- 172.21.15.102:0/12180668 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3d7c02cd80 con 0x7f3d6406e200 2024-05-13T22:55:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:55:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (4m) 18s ago 15m 26.7M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:55:13.252 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (15m) 18s ago 15m 8514k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:55:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (14m) 78s ago 14m 7956k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:55:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (87s) 18s ago 15m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T22:55:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (81s) 78s ago 14m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T22:55:13.253 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (3m) 18s ago 15m 77.5M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:55:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (9m) 18s ago 9m 88.3M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:55:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (9m) 18s ago 9m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:55:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (9m) 78s ago 9m 16.1M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:55:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (9m) 78s ago 9m 19.1M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:55:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (6m) 18s ago 16m 505M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:55:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (5m) 78s ago 14m 451M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:55:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (2m) 18s ago 16m 50.0M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:55:13.254 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (2m) 78s ago 14m 41.6M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:55:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (5m) 18s ago 15m 15.9M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:55:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (5m) 78s ago 14m 17.1M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:55:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (60s) 18s ago 13m 103M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 0ada88c777a2 2024-05-13T22:55:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (24s) 18s ago 13m 12.4M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 c3e9e68b06fd 2024-05-13T22:55:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (12m) 18s ago 12m 214M 1216M 18.2.1 5be31c24972a e183a93d33d2 2024-05-13T22:55:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (12m) 78s ago 12m 293M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:55:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (11m) 78s ago 11m 245M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:55:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (11m) 78s ago 11m 227M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:55:13.255 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (4m) 18s ago 14m 73.7M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:55:13.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.277+0000 7f3d81697700 1 -- 172.21.15.102:0/12180668 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3d6406e200 msgr2=0x7f3d640706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:13.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.277+0000 7f3d81697700 1 --2- 172.21.15.102:0/12180668 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3d6406e200 0x7f3d640706c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c005b40 tx=0x7f3d6c01a040 comp rx=0 tx=0).stop 2024-05-13T22:55:13.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.277+0000 7f3d81697700 1 -- 172.21.15.102:0/12180668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d7c0ffbd0 msgr2=0x7f3d7c197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:13.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.277+0000 7f3d81697700 1 --2- 172.21.15.102:0/12180668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d7c0ffbd0 0x7f3d7c197570 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000d8d0 tx=0x7f3d7000dc90 comp rx=0 tx=0).stop 2024-05-13T22:55:13.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.277+0000 7f3d81697700 1 -- 172.21.15.102:0/12180668 shutdown_connections 2024-05-13T22:55:13.256 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.277+0000 7f3d81697700 1 --2- 172.21.15.102:0/12180668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d7c0ffbd0 0x7f3d7c197570 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.277+0000 7f3d81697700 1 --2- 172.21.15.102:0/12180668 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3d6406e200 0x7f3d640706c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.277+0000 7f3d81697700 1 --2- 172.21.15.102:0/12180668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d7c105a40 0x7f3d7c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.277+0000 7f3d81697700 1 -- 172.21.15.102:0/12180668 >> 172.21.15.102:0/12180668 conn(0x7f3d7c0755a0 msgr2=0x7f3d7c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:13.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.277+0000 7f3d81697700 1 -- 172.21.15.102:0/12180668 shutdown_connections 2024-05-13T22:55:13.257 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.277+0000 7f3d81697700 1 -- 172.21.15.102:0/12180668 wait complete. 2024-05-13T22:55:13.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.380+0000 7f29b42fd700 1 -- 172.21.15.102:0/3362005497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29ac105a40 msgr2=0x7f29ac105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:13.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.380+0000 7f29b42fd700 1 --2- 172.21.15.102:0/3362005497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29ac105a40 0x7f29ac105e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009910 tx=0x7f29a8009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:13.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.380+0000 7f29b42fd700 1 -- 172.21.15.102:0/3362005497 shutdown_connections 2024-05-13T22:55:13.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.380+0000 7f29b42fd700 1 --2- 172.21.15.102:0/3362005497 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.380+0000 7f29b42fd700 1 --2- 172.21.15.102:0/3362005497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29ac105a40 0x7f29ac105e20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.380+0000 7f29b42fd700 1 -- 172.21.15.102:0/3362005497 >> 172.21.15.102:0/3362005497 conn(0x7f29ac0755a0 msgr2=0x7f29ac0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:13.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.380+0000 7f29b42fd700 1 -- 172.21.15.102:0/3362005497 shutdown_connections 2024-05-13T22:55:13.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.380+0000 7f29b42fd700 1 -- 172.21.15.102:0/3362005497 wait complete. 2024-05-13T22:55:13.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.380+0000 7f29b42fd700 1 Processor -- start 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.380+0000 7f29b42fd700 1 -- start start 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b42fd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b42fd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29ac105a40 0x7f29ac19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b42fd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac19d110 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b42fd700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac1965c0 con 0x7f29ac105a40 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b2099700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b2099700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51636/0 (socket says 172.21.15.102:51636) 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b2099700 1 -- 172.21.15.102:0/2633039576 learned_addr learned my addr 172.21.15.102:0/2633039576 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b1898700 1 --2- 172.21.15.102:0/2633039576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29ac105a40 0x7f29ac19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b2099700 1 -- 172.21.15.102:0/2633039576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29ac105a40 msgr2=0x7f29ac19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b2099700 1 --2- 172.21.15.102:0/2633039576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29ac105a40 0x7f29ac19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b2099700 1 -- 172.21.15.102:0/2633039576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a80095f0 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b2099700 1 --2- 172.21.15.102:0/2633039576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac19c540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29a800b5c0 tx=0x7f29a800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:13.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29a37fe700 1 -- 172.21.15.102:0/2633039576 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29a801d070 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29a37fe700 1 -- 172.21.15.102:0/2633039576 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29a8005e00 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b42fd700 1 -- 172.21.15.102:0/2633039576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ac196840 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29a37fe700 1 -- 172.21.15.102:0/2633039576 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29a8017400 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.360 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.381+0000 7f29b42fd700 1 -- 172.21.15.102:0/2633039576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ac196d30 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.382+0000 7f29b42fd700 1 -- 172.21.15.102:0/2633039576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ac109050 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.383+0000 7f29a37fe700 1 -- 172.21.15.102:0/2633039576 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f29a800f6f0 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.383+0000 7f29a37fe700 1 --2- 172.21.15.102:0/2633039576 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f299806e200 0x7f29980706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:13.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.383+0000 7f29a37fe700 1 -- 172.21.15.102:0/2633039576 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f29a808f6b0 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.383+0000 7f29b1898700 1 --2- 172.21.15.102:0/2633039576 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f299806e200 0x7f29980706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:13.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.384+0000 7f29b1898700 1 --2- 172.21.15.102:0/2633039576 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f299806e200 0x7f29980706c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f29ac075bb0 tx=0x7f299c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:13.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.388+0000 7f29a37fe700 1 -- 172.21.15.102:0/2633039576 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f29a805c450 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:13 smithi130 ceph-mon[84265]: pgmap v97: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 1.9 MiB/s, 63 objects/s recovering 2024-05-13T22:55:13.646 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.670+0000 7f29b42fd700 1 -- 172.21.15.102:0/2633039576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f29ac04ea90 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.670+0000 7f29a37fe700 1 -- 172.21.15.102:0/2633039576 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f29a805bfe0 con 0x7f29ac0ffbd0 2024-05-13T22:55:13.647 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:55:13.647 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:55:13.647 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:55:13.647 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:55:13.647 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:55:13.647 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 6 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:55:13.648 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:55:13.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.672+0000 7f29b42fd700 1 -- 172.21.15.102:0/2633039576 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f299806e200 msgr2=0x7f29980706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:13.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.672+0000 7f29b42fd700 1 --2- 172.21.15.102:0/2633039576 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f299806e200 0x7f29980706c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f29ac075bb0 tx=0x7f299c00b410 comp rx=0 tx=0).stop 2024-05-13T22:55:13.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.672+0000 7f29b42fd700 1 -- 172.21.15.102:0/2633039576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29ac0ffbd0 msgr2=0x7f29ac19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:13.649 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.672+0000 7f29b42fd700 1 --2- 172.21.15.102:0/2633039576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac19c540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29a800b5c0 tx=0x7f29a800bd70 comp rx=0 tx=0).stop 2024-05-13T22:55:13.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.672+0000 7f29b42fd700 1 -- 172.21.15.102:0/2633039576 shutdown_connections 2024-05-13T22:55:13.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.672+0000 7f29b42fd700 1 --2- 172.21.15.102:0/2633039576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29ac0ffbd0 0x7f29ac19c540 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.672+0000 7f29b42fd700 1 --2- 172.21.15.102:0/2633039576 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f299806e200 0x7f29980706c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.672+0000 7f29b42fd700 1 --2- 172.21.15.102:0/2633039576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29ac105a40 0x7f29ac19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.672+0000 7f29b42fd700 1 -- 172.21.15.102:0/2633039576 >> 172.21.15.102:0/2633039576 conn(0x7f29ac0755a0 msgr2=0x7f29ac0685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:13.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.673+0000 7f29b42fd700 1 -- 172.21.15.102:0/2633039576 shutdown_connections 2024-05-13T22:55:13.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.673+0000 7f29b42fd700 1 -- 172.21.15.102:0/2633039576 wait complete. 2024-05-13T22:55:13.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.778+0000 7fc356cbc700 1 -- 172.21.15.102:0/3000935616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350101790 msgr2=0x7fc3501057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:13.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.778+0000 7fc356cbc700 1 --2- 172.21.15.102:0/3000935616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350101790 0x7fc3501057e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009910 tx=0x7fc33c009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:13.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.778+0000 7fc356cbc700 1 -- 172.21.15.102:0/3000935616 shutdown_connections 2024-05-13T22:55:13.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.778+0000 7fc356cbc700 1 --2- 172.21.15.102:0/3000935616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350101790 0x7fc3501057e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.778+0000 7fc356cbc700 1 --2- 172.21.15.102:0/3000935616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc350100de0 0x7fc3501011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.778+0000 7fc356cbc700 1 -- 172.21.15.102:0/3000935616 >> 172.21.15.102:0/3000935616 conn(0x7fc3500fc670 msgr2=0x7fc3500fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:13.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.778+0000 7fc356cbc700 1 -- 172.21.15.102:0/3000935616 shutdown_connections 2024-05-13T22:55:13.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.778+0000 7fc356cbc700 1 -- 172.21.15.102:0/3000935616 wait complete. 2024-05-13T22:55:13.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.779+0000 7fc356cbc700 1 Processor -- start 2024-05-13T22:55:13.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.779+0000 7fc356cbc700 1 -- start start 2024-05-13T22:55:13.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.779+0000 7fc356cbc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350100de0 0x7fc35019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:13.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.779+0000 7fc356cbc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc350101790 0x7fc35019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:13.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.779+0000 7fc356cbc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35019d1f0 con 0x7fc350100de0 2024-05-13T22:55:13.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.779+0000 7fc356cbc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3501966a0 con 0x7fc350101790 2024-05-13T22:55:13.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.779+0000 7fc354a58700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350100de0 0x7fc35019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:13.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.780+0000 7fc354a58700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350100de0 0x7fc35019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51650/0 (socket says 172.21.15.102:51650) 2024-05-13T22:55:13.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.780+0000 7fc354a58700 1 -- 172.21.15.102:0/4229228390 learned_addr learned my addr 172.21.15.102:0/4229228390 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:13.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.780+0000 7fc34ffff700 1 --2- 172.21.15.102:0/4229228390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc350101790 0x7fc35019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:13.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.780+0000 7fc354a58700 1 -- 172.21.15.102:0/4229228390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc350101790 msgr2=0x7fc35019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:13.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.780+0000 7fc354a58700 1 --2- 172.21.15.102:0/4229228390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc350101790 0x7fc35019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:13.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.780+0000 7fc354a58700 1 -- 172.21.15.102:0/4229228390 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c0095f0 con 0x7fc350100de0 2024-05-13T22:55:13.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.780+0000 7fc354a58700 1 --2- 172.21.15.102:0/4229228390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350100de0 0x7fc35019c620 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc34400b4b0 tx=0x7fc34400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:13.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.780+0000 7fc34dffb700 1 -- 172.21.15.102:0/4229228390 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc344011800 con 0x7fc350100de0 2024-05-13T22:55:13.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.781+0000 7fc34dffb700 1 -- 172.21.15.102:0/4229228390 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc344011e20 con 0x7fc350100de0 2024-05-13T22:55:13.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.781+0000 7fc34dffb700 1 -- 172.21.15.102:0/4229228390 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc34400f2e0 con 0x7fc350100de0 2024-05-13T22:55:13.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.781+0000 7fc356cbc700 1 -- 172.21.15.102:0/4229228390 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc350196980 con 0x7fc350100de0 2024-05-13T22:55:13.762 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.781+0000 7fc356cbc700 1 -- 172.21.15.102:0/4229228390 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc350196e70 con 0x7fc350100de0 2024-05-13T22:55:13.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.782+0000 7fc34dffb700 1 -- 172.21.15.102:0/4229228390 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc344010380 con 0x7fc350100de0 2024-05-13T22:55:13.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.783+0000 7fc356cbc700 1 -- 172.21.15.102:0/4229228390 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc350109130 con 0x7fc350100de0 2024-05-13T22:55:13.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.783+0000 7fc34dffb700 1 --2- 172.21.15.102:0/4229228390 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc34006e200 0x7fc3400706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:13.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.783+0000 7fc34dffb700 1 -- 172.21.15.102:0/4229228390 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fc34408daa0 con 0x7fc350100de0 2024-05-13T22:55:13.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.783+0000 7fc34ffff700 1 --2- 172.21.15.102:0/4229228390 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc34006e200 0x7fc3400706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:13.763 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.783+0000 7fc34ffff700 1 --2- 172.21.15.102:0/4229228390 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc34006e200 0x7fc3400706c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc3500fcf70 tx=0x7fc33c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:13.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:13.788+0000 7fc34dffb700 1 -- 172.21.15.102:0/4229228390 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc34405a840 con 0x7fc350100de0 2024-05-13T22:55:14.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.026+0000 7fc356cbc700 1 -- 172.21.15.102:0/4229228390 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc350197170 con 0x7fc350100de0 2024-05-13T22:55:14.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.027+0000 7fc34dffb700 1 -- 172.21.15.102:0/4229228390 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1909 (secure 0 0 0) 0x7fc34405a3d0 con 0x7fc350100de0 2024-05-13T22:55:14.003 INFO:teuthology.orchestra.run.smithi102.stdout:e17 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 17 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:55:14.004 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 47 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=14498} 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:55:14.005 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 0 members: 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:55:14.006 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:55:14.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.029+0000 7fc356cbc700 1 -- 172.21.15.102:0/4229228390 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc34006e200 msgr2=0x7fc3400706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:14.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.029+0000 7fc356cbc700 1 --2- 172.21.15.102:0/4229228390 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc34006e200 0x7fc3400706c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc3500fcf70 tx=0x7fc33c01a040 comp rx=0 tx=0).stop 2024-05-13T22:55:14.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.029+0000 7fc356cbc700 1 -- 172.21.15.102:0/4229228390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350100de0 msgr2=0x7fc35019c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:14.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.029+0000 7fc356cbc700 1 --2- 172.21.15.102:0/4229228390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350100de0 0x7fc35019c620 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc34400b4b0 tx=0x7fc34400b870 comp rx=0 tx=0).stop 2024-05-13T22:55:14.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.029+0000 7fc356cbc700 1 -- 172.21.15.102:0/4229228390 shutdown_connections 2024-05-13T22:55:14.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.029+0000 7fc356cbc700 1 --2- 172.21.15.102:0/4229228390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc350100de0 0x7fc35019c620 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.029+0000 7fc356cbc700 1 --2- 172.21.15.102:0/4229228390 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc34006e200 0x7fc3400706c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.029+0000 7fc356cbc700 1 --2- 172.21.15.102:0/4229228390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc350101790 0x7fc35019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.029+0000 7fc356cbc700 1 -- 172.21.15.102:0/4229228390 >> 172.21.15.102:0/4229228390 conn(0x7fc3500fc670 msgr2=0x7fc3500fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:14.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.029+0000 7fc356cbc700 1 -- 172.21.15.102:0/4229228390 shutdown_connections 2024-05-13T22:55:14.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.029+0000 7fc356cbc700 1 -- 172.21.15.102:0/4229228390 wait complete. 2024-05-13T22:55:14.008 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:55:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.135+0000 7f90a0e37700 1 -- 172.21.15.102:0/2060641848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f909c101770 msgr2=0x7f909c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.135+0000 7f90a0e37700 1 --2- 172.21.15.102:0/2060641848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f909c101770 0x7f909c1057c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9090009910 tx=0x7f9090009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.135+0000 7f90a0e37700 1 -- 172.21.15.102:0/2060641848 shutdown_connections 2024-05-13T22:55:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.135+0000 7f90a0e37700 1 --2- 172.21.15.102:0/2060641848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f909c101770 0x7f909c1057c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.135+0000 7f90a0e37700 1 --2- 172.21.15.102:0/2060641848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f909c100dc0 0x7f909c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.135+0000 7f90a0e37700 1 -- 172.21.15.102:0/2060641848 >> 172.21.15.102:0/2060641848 conn(0x7f909c0fc650 msgr2=0x7f909c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:14.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.135+0000 7f90a0e37700 1 -- 172.21.15.102:0/2060641848 shutdown_connections 2024-05-13T22:55:14.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.135+0000 7f90a0e37700 1 -- 172.21.15.102:0/2060641848 wait complete. 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.136+0000 7f90a0e37700 1 Processor -- start 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.136+0000 7f90a0e37700 1 -- start start 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.136+0000 7f90a0e37700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f909c100dc0 0x7f909c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.136+0000 7f90a0e37700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f909c101770 0x7f909c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.136+0000 7f90a0e37700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c073b10 con 0x7f909c100dc0 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.136+0000 7f90a0e37700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c073c50 con 0x7f909c101770 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.136+0000 7f909a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f909c100dc0 0x7f909c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.136+0000 7f909a59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f909c100dc0 0x7f909c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51668/0 (socket says 172.21.15.102:51668) 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.137+0000 7f909a59c700 1 -- 172.21.15.102:0/4242135001 learned_addr learned my addr 172.21.15.102:0/4242135001 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.137+0000 7f909a59c700 1 -- 172.21.15.102:0/4242135001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f909c101770 msgr2=0x7f909c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.137+0000 7f909a59c700 1 --2- 172.21.15.102:0/4242135001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f909c101770 0x7f909c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.137+0000 7f909a59c700 1 -- 172.21.15.102:0/4242135001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90900095f0 con 0x7f909c100dc0 2024-05-13T22:55:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.137+0000 7f909a59c700 1 --2- 172.21.15.102:0/4242135001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f909c100dc0 0x7f909c073090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f908c00b4b0 tx=0x7f908c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:14.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.137+0000 7f908b7fe700 1 -- 172.21.15.102:0/4242135001 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f908c011800 con 0x7f909c100dc0 2024-05-13T22:55:14.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.137+0000 7f908b7fe700 1 -- 172.21.15.102:0/4242135001 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f908c011e20 con 0x7f909c100dc0 2024-05-13T22:55:14.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.137+0000 7f90a0e37700 1 -- 172.21.15.102:0/4242135001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909c1a2a70 con 0x7f909c100dc0 2024-05-13T22:55:14.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.137+0000 7f908b7fe700 1 -- 172.21.15.102:0/4242135001 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f908c00f2e0 con 0x7f909c100dc0 2024-05-13T22:55:14.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.137+0000 7f90a0e37700 1 -- 172.21.15.102:0/4242135001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c1a2e80 con 0x7f909c100dc0 2024-05-13T22:55:14.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.138+0000 7f90a0e37700 1 -- 172.21.15.102:0/4242135001 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c1090f0 con 0x7f909c100dc0 2024-05-13T22:55:14.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.139+0000 7f908b7fe700 1 -- 172.21.15.102:0/4242135001 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f908c010380 con 0x7f909c100dc0 2024-05-13T22:55:14.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.139+0000 7f908b7fe700 1 --2- 172.21.15.102:0/4242135001 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f908406e200 0x7f90840706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:14.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.139+0000 7f908b7fe700 1 -- 172.21.15.102:0/4242135001 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f908c08dd10 con 0x7f909c100dc0 2024-05-13T22:55:14.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.139+0000 7f9099d9b700 1 --2- 172.21.15.102:0/4242135001 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f908406e200 0x7f90840706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:14.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.140+0000 7f9099d9b700 1 --2- 172.21.15.102:0/4242135001 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f908406e200 0x7f90840706c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f909000b5c0 tx=0x7f9090005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:14.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.144+0000 7f908b7fe700 1 -- 172.21.15.102:0/4242135001 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f908c057000 con 0x7f909c100dc0 2024-05-13T22:55:14.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.362+0000 7f90a0e37700 1 -- 172.21.15.102:0/4242135001 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f909c02cce0 con 0x7f908406e200 2024-05-13T22:55:14.339 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:14 smithi102 ceph-mon[111694]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:14.339 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:14 smithi102 ceph-mon[111694]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:14.339 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:14 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2633039576' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:14.339 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:14 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/4229228390' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:55:14.340 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.363+0000 7f908b7fe700 1 -- 172.21.15.102:0/4242135001 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f909c02cce0 con 0x7f908406e200 2024-05-13T22:55:14.340 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:55:14.340 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:55:14.340 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:55:14.340 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:55:14.340 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:55:14.340 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-05-13T22:55:14.340 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr", 2024-05-13T22:55:14.340 INFO:teuthology.orchestra.run.smithi102.stdout: "mon" 2024-05-13T22:55:14.340 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:55:14.340 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "8/23 daemons upgraded", 2024-05-13T22:55:14.341 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading osd daemons", 2024-05-13T22:55:14.341 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:55:14.341 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:55:14.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.365+0000 7f90a0e37700 1 -- 172.21.15.102:0/4242135001 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f908406e200 msgr2=0x7f90840706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:14.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.365+0000 7f90a0e37700 1 --2- 172.21.15.102:0/4242135001 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f908406e200 0x7f90840706c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f909000b5c0 tx=0x7f9090005960 comp rx=0 tx=0).stop 2024-05-13T22:55:14.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.365+0000 7f90a0e37700 1 -- 172.21.15.102:0/4242135001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f909c100dc0 msgr2=0x7f909c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:14.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.365+0000 7f90a0e37700 1 --2- 172.21.15.102:0/4242135001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f909c100dc0 0x7f909c073090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f908c00b4b0 tx=0x7f908c00b870 comp rx=0 tx=0).stop 2024-05-13T22:55:14.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.365+0000 7f90a0e37700 1 -- 172.21.15.102:0/4242135001 shutdown_connections 2024-05-13T22:55:14.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.365+0000 7f90a0e37700 1 --2- 172.21.15.102:0/4242135001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f909c100dc0 0x7f909c073090 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.365+0000 7f90a0e37700 1 --2- 172.21.15.102:0/4242135001 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f908406e200 0x7f90840706c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.365+0000 7f90a0e37700 1 --2- 172.21.15.102:0/4242135001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f909c101770 0x7f909c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.365+0000 7f90a0e37700 1 -- 172.21.15.102:0/4242135001 >> 172.21.15.102:0/4242135001 conn(0x7f909c0fc650 msgr2=0x7f909c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:14.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.365+0000 7f90a0e37700 1 -- 172.21.15.102:0/4242135001 shutdown_connections 2024-05-13T22:55:14.343 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.365+0000 7f90a0e37700 1 -- 172.21.15.102:0/4242135001 wait complete. 2024-05-13T22:55:14.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:14 smithi130 ceph-mon[84265]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:14.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:14 smithi130 ceph-mon[84265]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:14.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:14 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2633039576' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:14.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:14 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/4229228390' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:55:14.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.470+0000 7f624c71e700 1 -- 172.21.15.102:0/3812557871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6244105a40 msgr2=0x7f6244105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:14.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.470+0000 7f624c71e700 1 --2- 172.21.15.102:0/3812557871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6244105a40 0x7f6244105e20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6234009910 tx=0x7f6234009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:14.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.470+0000 7f624c71e700 1 -- 172.21.15.102:0/3812557871 shutdown_connections 2024-05-13T22:55:14.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.470+0000 7f624c71e700 1 --2- 172.21.15.102:0/3812557871 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62440ffbd0 0x7f6244100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.470+0000 7f624c71e700 1 --2- 172.21.15.102:0/3812557871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6244105a40 0x7f6244105e20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.470+0000 7f624c71e700 1 -- 172.21.15.102:0/3812557871 >> 172.21.15.102:0/3812557871 conn(0x7f62440755a0 msgr2=0x7f62440759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:14.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.470+0000 7f624c71e700 1 -- 172.21.15.102:0/3812557871 shutdown_connections 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.470+0000 7f624c71e700 1 -- 172.21.15.102:0/3812557871 wait complete. 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f624c71e700 1 Processor -- start 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f624c71e700 1 -- start start 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f624c71e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62440ffbd0 0x7f62441975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f624c71e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6244105a40 0x7f6244197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f624c71e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6244198110 con 0x7f6244105a40 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f624c71e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6244198250 con 0x7f62440ffbd0 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f6249cb9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6244105a40 0x7f6244197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f6249cb9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6244105a40 0x7f6244197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51694/0 (socket says 172.21.15.102:51694) 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f6249cb9700 1 -- 172.21.15.102:0/3182020235 learned_addr learned my addr 172.21.15.102:0/3182020235 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f624a4ba700 1 --2- 172.21.15.102:0/3182020235 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62440ffbd0 0x7f62441975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:14.450 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f6249cb9700 1 -- 172.21.15.102:0/3182020235 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62440ffbd0 msgr2=0x7f62441975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:14.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f6249cb9700 1 --2- 172.21.15.102:0/3182020235 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62440ffbd0 0x7f62441975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.471+0000 7f6249cb9700 1 -- 172.21.15.102:0/3182020235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62340095f0 con 0x7f6244105a40 2024-05-13T22:55:14.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.472+0000 7f6249cb9700 1 --2- 172.21.15.102:0/3182020235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6244105a40 0x7f6244197af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f624000d8d0 tx=0x7f624000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:14.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.472+0000 7f623b7fe700 1 -- 172.21.15.102:0/3182020235 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6240009940 con 0x7f6244105a40 2024-05-13T22:55:14.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.472+0000 7f623b7fe700 1 -- 172.21.15.102:0/3182020235 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6240010460 con 0x7f6244105a40 2024-05-13T22:55:14.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.472+0000 7f624c71e700 1 -- 172.21.15.102:0/3182020235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62441a0620 con 0x7f6244105a40 2024-05-13T22:55:14.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.472+0000 7f623b7fe700 1 -- 172.21.15.102:0/3182020235 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f624000f5f0 con 0x7f6244105a40 2024-05-13T22:55:14.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.472+0000 7f624c71e700 1 -- 172.21.15.102:0/3182020235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62441a0ae0 con 0x7f6244105a40 2024-05-13T22:55:14.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.473+0000 7f624c71e700 1 -- 172.21.15.102:0/3182020235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6244108040 con 0x7f6244105a40 2024-05-13T22:55:14.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.473+0000 7f623b7fe700 1 -- 172.21.15.102:0/3182020235 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6240009aa0 con 0x7f6244105a40 2024-05-13T22:55:14.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.474+0000 7f623b7fe700 1 --2- 172.21.15.102:0/3182020235 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f623006e200 0x7f62300706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:14.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.474+0000 7f623b7fe700 1 -- 172.21.15.102:0/3182020235 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f624008e140 con 0x7f6244105a40 2024-05-13T22:55:14.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.474+0000 7f624a4ba700 1 --2- 172.21.15.102:0/3182020235 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f623006e200 0x7f62300706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:14.452 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.474+0000 7f624a4ba700 1 --2- 172.21.15.102:0/3182020235 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f623006e200 0x7f62300706c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f623400b5c0 tx=0x7f6234005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:14.455 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.478+0000 7f623b7fe700 1 -- 172.21.15.102:0/3182020235 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6240057430 con 0x7f6244105a40 2024-05-13T22:55:14.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.760+0000 7f624c71e700 1 -- 172.21.15.102:0/3182020235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f624404ea90 con 0x7f6244105a40 2024-05-13T22:55:14.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.761+0000 7f623b7fe700 1 -- 172.21.15.102:0/3182020235 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f624005aaf0 con 0x7f6244105a40 2024-05-13T22:55:14.737 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:55:14.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.763+0000 7f624c71e700 1 -- 172.21.15.102:0/3182020235 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f623006e200 msgr2=0x7f62300706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.763+0000 7f624c71e700 1 --2- 172.21.15.102:0/3182020235 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f623006e200 0x7f62300706c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f623400b5c0 tx=0x7f6234005960 comp rx=0 tx=0).stop 2024-05-13T22:55:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.763+0000 7f624c71e700 1 -- 172.21.15.102:0/3182020235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6244105a40 msgr2=0x7f6244197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.763+0000 7f624c71e700 1 --2- 172.21.15.102:0/3182020235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6244105a40 0x7f6244197af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f624000d8d0 tx=0x7f624000dc90 comp rx=0 tx=0).stop 2024-05-13T22:55:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.763+0000 7f624c71e700 1 -- 172.21.15.102:0/3182020235 shutdown_connections 2024-05-13T22:55:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.763+0000 7f624c71e700 1 --2- 172.21.15.102:0/3182020235 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f623006e200 0x7f62300706c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.763+0000 7f624c71e700 1 --2- 172.21.15.102:0/3182020235 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62440ffbd0 0x7f62441975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.763+0000 7f624c71e700 1 --2- 172.21.15.102:0/3182020235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6244105a40 0x7f6244197af0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.763+0000 7f624c71e700 1 -- 172.21.15.102:0/3182020235 >> 172.21.15.102:0/3182020235 conn(0x7f62440755a0 msgr2=0x7f62440fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.763+0000 7f624c71e700 1 -- 172.21.15.102:0/3182020235 shutdown_connections 2024-05-13T22:55:14.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:14.763+0000 7f624c71e700 1 -- 172.21.15.102:0/3182020235 wait complete. 2024-05-13T22:55:15.139 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:15 smithi102 ceph-mon[111694]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:15.139 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:15 smithi102 ceph-mon[111694]: pgmap v98: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 0 B/s, 53 objects/s recovering 2024-05-13T22:55:15.139 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:15 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-13T22:55:15.139 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:15 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3182020235' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:55:15.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:15 smithi130 ceph-mon[84265]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:15.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:15 smithi130 ceph-mon[84265]: pgmap v98: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 0 B/s, 53 objects/s recovering 2024-05-13T22:55:15.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:15 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-13T22:55:15.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:15 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3182020235' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:55:16.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:16 smithi130 ceph-mon[84265]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:16.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:16 smithi130 ceph-mon[84265]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-13T22:55:16.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:16 smithi130 ceph-mon[84265]: Upgrade: osd.2 is safe to restart 2024-05-13T22:55:16.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:16.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:55:16.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:16 smithi102 ceph-mon[111694]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:16.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:16 smithi102 ceph-mon[111694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-13T22:55:16.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:16 smithi102 ceph-mon[111694]: Upgrade: osd.2 is safe to restart 2024-05-13T22:55:16.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:16.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:55:17.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:17 smithi130 ceph-mon[84265]: pgmap v99: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 0 B/s, 53 objects/s recovering 2024-05-13T22:55:17.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:17 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:17.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:17 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-13T22:55:17.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:17 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:55:17.408 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:17 smithi102 ceph-mon[111694]: pgmap v99: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 0 B/s, 53 objects/s recovering 2024-05-13T22:55:17.408 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:17 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:17.408 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:17 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-13T22:55:17.408 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:17 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:55:18.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:18 smithi102 ceph-mon[111694]: Upgrade: Updating osd.2 2024-05-13T22:55:18.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:18 smithi102 ceph-mon[111694]: Deploying daemon osd.2 on smithi102 2024-05-13T22:55:18.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:18 smithi130 ceph-mon[84265]: Upgrade: Updating osd.2 2024-05-13T22:55:18.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:18 smithi130 ceph-mon[84265]: Deploying daemon osd.2 on smithi102 2024-05-13T22:55:19.296 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:19 smithi102 ceph-mon[111694]: pgmap v100: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:55:19.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:19 smithi130 ceph-mon[84265]: pgmap v100: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:55:19.926 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:19 smithi102 systemd[1]: Stopping Ceph osd.2 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:55:20.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:20 smithi102 ceph-mon[111694]: osd.2 marked itself down and dead 2024-05-13T22:55:20.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:20 smithi102 ceph-mon[111694]: pgmap v101: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:55:20.427 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:20 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[62121]: 2024-05-13T22:55:20.007+0000 7fd507741700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T22:55:20.427 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:20 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[62121]: 2024-05-13T22:55:20.007+0000 7fd507741700 -1 osd.2 71 *** Got signal Terminated *** 2024-05-13T22:55:20.427 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:20 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[62121]: 2024-05-13T22:55:20.007+0000 7fd507741700 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T22:55:20.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:20 smithi130 ceph-mon[84265]: osd.2 marked itself down and dead 2024-05-13T22:55:20.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:20 smithi130 ceph-mon[84265]: pgmap v101: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:55:20.902 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:20 smithi102 podman[134383]: 2024-05-13 22:55:20.660914793 +0000 UTC m=+0.814760246 container died e183a93d33d2873aa40af1307ca712bb1033ebb67371814f1fc5cea9806b34f2 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-05-13T22:55:21.176 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:21 smithi102 podman[134383]: 2024-05-13 22:55:21.016227393 +0000 UTC m=+1.170072840 container cleanup e183a93d33d2873aa40af1307ca712bb1033ebb67371814f1fc5cea9806b34f2 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-13T22:55:21.176 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:21 smithi102 bash[134383]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2 2024-05-13T22:55:21.616 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:21 smithi102 ceph-mon[111694]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:55:21.616 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:21 smithi102 ceph-mon[111694]: osdmap e72: 6 total, 5 up, 6 in 2024-05-13T22:55:21.616 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:21 smithi102 podman[134401]: 2024-05-13 22:55:21.309686324 +0000 UTC m=+0.651693779 container remove e183a93d33d2873aa40af1307ca712bb1033ebb67371814f1fc5cea9806b34f2 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2, io.buildah.version=1.29.1, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-13T22:55:21.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:21 smithi130 ceph-mon[84265]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:55:21.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:21 smithi130 ceph-mon[84265]: osdmap e72: 6 total, 5 up, 6 in 2024-05-13T22:55:21.926 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:21 smithi102 podman[134489]: 2024-05-13 22:55:21.542699276 +0000 UTC m=+0.017189468 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:55:21.926 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:21 smithi102 podman[134489]: 2024-05-13 22:55:21.660738482 +0000 UTC m=+0.135228664 container create e0309a4d43f947426fbdfbde52c3f0c34029a7562452d2ea73d89b7d9759ddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-c3e5514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-13T22:55:21.926 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:21 smithi102 podman[134489]: 2024-05-13 22:55:21.863962222 +0000 UTC m=+0.338452413 container init e0309a4d43f947426fbdfbde52c3f0c34029a7562452d2ea73d89b7d9759ddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-c3e5514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-13T22:55:21.926 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:21 smithi102 podman[134489]: 2024-05-13 22:55:21.867650628 +0000 UTC m=+0.342140829 container start e0309a4d43f947426fbdfbde52c3f0c34029a7562452d2ea73d89b7d9759ddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-13T22:55:22.209 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:21 smithi102 podman[134489]: 2024-05-13 22:55:21.95492375 +0000 UTC m=+0.429413954 container attach e0309a4d43f947426fbdfbde52c3f0c34029a7562452d2ea73d89b7d9759ddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-13T22:55:22.209 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:22 smithi102 podman[134489]: 2024-05-13 22:55:22.158841024 +0000 UTC m=+0.633331206 container died e0309a4d43f947426fbdfbde52c3f0c34029a7562452d2ea73d89b7d9759ddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-13T22:55:22.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:22 smithi130 ceph-mon[84265]: osdmap e73: 6 total, 5 up, 6 in 2024-05-13T22:55:22.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:22 smithi130 ceph-mon[84265]: pgmap v104: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 11 peering, 2 stale+active+clean, 40 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 10/231 objects degraded (4.329%) 2024-05-13T22:55:22.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:22 smithi102 ceph-mon[111694]: osdmap e73: 6 total, 5 up, 6 in 2024-05-13T22:55:22.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:22 smithi102 ceph-mon[111694]: pgmap v104: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 11 peering, 2 stale+active+clean, 40 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 10/231 objects degraded (4.329%) 2024-05-13T22:55:23.149 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:22 smithi102 podman[134516]: 2024-05-13 22:55:22.725737918 +0000 UTC m=+0.561329427 container remove e0309a4d43f947426fbdfbde52c3f0c34029a7562452d2ea73d89b7d9759ddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-13T22:55:23.149 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:22 smithi102 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.2.service: Deactivated successfully. 2024-05-13T22:55:23.149 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:22 smithi102 systemd[1]: Stopped Ceph osd.2 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:55:23.150 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:22 smithi102 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.2.service: Consumed 57.869s CPU time. 2024-05-13T22:55:23.150 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:22 smithi102 systemd[1]: Starting Ceph osd.2 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:55:23.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:23 smithi102 ceph-mon[111694]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-13T22:55:23.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:23 smithi102 ceph-mon[111694]: Health check failed: Degraded data redundancy: 10/231 objects degraded (4.329%), 5 pgs degraded (PG_DEGRADED) 2024-05-13T22:55:23.426 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 podman[134617]: 2024-05-13 22:55:23.07616657 +0000 UTC m=+0.017300100 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:55:23.427 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 podman[134617]: 2024-05-13 22:55:23.218570849 +0000 UTC m=+0.159704399 container create 6c51b85da5d2f140a21cc0c25a2f71cbdb2e80f6c69bcd8eeef91c634f407e2d (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-13T22:55:23.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:23 smithi130 ceph-mon[84265]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-13T22:55:23.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:23 smithi130 ceph-mon[84265]: Health check failed: Degraded data redundancy: 10/231 objects degraded (4.329%), 5 pgs degraded (PG_DEGRADED) 2024-05-13T22:55:23.899 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 podman[134617]: 2024-05-13 22:55:23.471543883 +0000 UTC m=+0.412677385 container init 6c51b85da5d2f140a21cc0c25a2f71cbdb2e80f6c69bcd8eeef91c634f407e2d (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T22:55:23.899 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 podman[134617]: 2024-05-13 22:55:23.475021992 +0000 UTC m=+0.416155506 container start 6c51b85da5d2f140a21cc0c25a2f71cbdb2e80f6c69bcd8eeef91c634f407e2d (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-13T22:55:23.899 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 podman[134617]: 2024-05-13 22:55:23.546469089 +0000 UTC m=+0.487602597 container attach 6c51b85da5d2f140a21cc0c25a2f71cbdb2e80f6c69bcd8eeef91c634f407e2d (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-13T22:55:24.176 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate[134632]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate[134632]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 bash[134617]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 bash[134617]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 bash[134617]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 bash[134617]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 bash[134617]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 bash[134617]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 bash[134617]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate[134632]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate[134632]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate[134632]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-13T22:55:24.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate[134632]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-13T22:55:24.178 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate[134632]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-13T22:55:24.178 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:23 smithi102 podman[134617]: 2024-05-13 22:55:23.94697439 +0000 UTC m=+0.888107900 container died 6c51b85da5d2f140a21cc0c25a2f71cbdb2e80f6c69bcd8eeef91c634f407e2d (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T22:55:24.577 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:24 smithi102 ceph-mon[111694]: pgmap v105: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 11 peering, 2 stale+active+clean, 40 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 0 op/s; 10/231 objects degraded (4.329%) 2024-05-13T22:55:24.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:24 smithi130 ceph-mon[84265]: pgmap v105: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 11 peering, 2 stale+active+clean, 40 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 0 op/s; 10/231 objects degraded (4.329%) 2024-05-13T22:55:24.909 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:24 smithi102 podman[134771]: 2024-05-13 22:55:24.601045593 +0000 UTC m=+0.648476708 container remove 6c51b85da5d2f140a21cc0c25a2f71cbdb2e80f6c69bcd8eeef91c634f407e2d (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-activate, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-13T22:55:24.910 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:24 smithi102 podman[134831]: 2024-05-13 22:55:24.717070744 +0000 UTC m=+0.017424997 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:55:24.910 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:24 smithi102 podman[134831]: 2024-05-13 22:55:24.835433339 +0000 UTC m=+0.135787626 container create 1b85728dccf5a1959cbab622b485fec7fd27d1341d8d84c032313d317121bfe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=squid-c3e5514) 2024-05-13T22:55:25.176 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:25 smithi102 podman[134831]: 2024-05-13 22:55:25.046497712 +0000 UTC m=+0.346851964 container init 1b85728dccf5a1959cbab622b485fec7fd27d1341d8d84c032313d317121bfe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-c3e5514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD) 2024-05-13T22:55:25.176 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:25 smithi102 podman[134831]: 2024-05-13 22:55:25.049838773 +0000 UTC m=+0.350193020 container start 1b85728dccf5a1959cbab622b485fec7fd27d1341d8d84c032313d317121bfe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-c3e5514) 2024-05-13T22:55:25.176 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:25 smithi102 ceph-osd[134850]: -- 172.21.15.102:0/4046150448 <== mon.0 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f2b225680 con 0x561f2b1ef000 2024-05-13T22:55:25.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:25 smithi102 bash[134831]: 1b85728dccf5a1959cbab622b485fec7fd27d1341d8d84c032313d317121bfe5 2024-05-13T22:55:25.177 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:25 smithi102 systemd[1]: Started Ceph osd.2 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:55:25.926 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:25 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[134846]: 2024-05-13T22:55:25.588+0000 7fbd8a8077c0 -1 Falling back to public interface 2024-05-13T22:55:26.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:26 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:26.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:26 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:26.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:26 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:55:26.453 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:26 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:26.454 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:26 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:26.454 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:26 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:55:27.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:27 smithi102 ceph-mon[111694]: pgmap v106: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 37/231 objects degraded (16.017%) 2024-05-13T22:55:27.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:27 smithi102 ceph-mon[111694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-13T22:55:27.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:27 smithi130 ceph-mon[84265]: pgmap v106: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 37/231 objects degraded (16.017%) 2024-05-13T22:55:27.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:27 smithi130 ceph-mon[84265]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-13T22:55:29.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:29 smithi102 ceph-mon[111694]: pgmap v107: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 37/231 objects degraded (16.017%) 2024-05-13T22:55:29.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:29 smithi130 ceph-mon[84265]: pgmap v107: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 37/231 objects degraded (16.017%) 2024-05-13T22:55:30.426 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:30 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[134846]: 2024-05-13T22:55:30.025+0000 7fbd8a8077c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-05-13T22:55:30.926 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:30 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[134846]: 2024-05-13T22:55:30.508+0000 7fbd8a8077c0 -1 osd.2 71 log_to_monitors true 2024-05-13T22:55:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:31 smithi102 ceph-mon[111694]: pgmap v108: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 729 B/s rd, 1 op/s; 37/231 objects degraded (16.017%) 2024-05-13T22:55:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:31 smithi102 ceph-mon[111694]: from='osd.2 [v2:172.21.15.102:6818/3530038065,v1:172.21.15.102:6819/3530038065]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-13T22:55:31.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:31.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:31.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:31.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:55:31.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:31 smithi102 ceph-mon[111694]: Health check update: Degraded data redundancy: 37/231 objects degraded (16.017%), 14 pgs degraded (PG_DEGRADED) 2024-05-13T22:55:31.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:31 smithi130 ceph-mon[84265]: pgmap v108: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 729 B/s rd, 1 op/s; 37/231 objects degraded (16.017%) 2024-05-13T22:55:31.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:31 smithi130 ceph-mon[84265]: from='osd.2 [v2:172.21.15.102:6818/3530038065,v1:172.21.15.102:6819/3530038065]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-13T22:55:31.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:31.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:31.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:31.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:55:31.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:31 smithi130 ceph-mon[84265]: Health check update: Degraded data redundancy: 37/231 objects degraded (16.017%), 14 pgs degraded (PG_DEGRADED) 2024-05-13T22:55:32.426 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 22:55:32 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[134846]: 2024-05-13T22:55:32.070+0000 7fbd77c56700 -1 osd.2 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:55:32.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:32 smithi102 ceph-mon[111694]: from='osd.2 [v2:172.21.15.102:6818/3530038065,v1:172.21.15.102:6819/3530038065]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-13T22:55:32.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:32 smithi102 ceph-mon[111694]: osdmap e74: 6 total, 5 up, 6 in 2024-05-13T22:55:32.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:32 smithi102 ceph-mon[111694]: from='osd.2 [v2:172.21.15.102:6818/3530038065,v1:172.21.15.102:6819/3530038065]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:55:32.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:32 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:32.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:32 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:32.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:32 smithi102 ceph-mon[111694]: from='osd.2 [v2:172.21.15.102:6818/3530038065,v1:172.21.15.102:6819/3530038065]' entity='osd.2' 2024-05-13T22:55:32.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:32 smithi130 ceph-mon[84265]: from='osd.2 [v2:172.21.15.102:6818/3530038065,v1:172.21.15.102:6819/3530038065]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-13T22:55:32.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:32 smithi130 ceph-mon[84265]: osdmap e74: 6 total, 5 up, 6 in 2024-05-13T22:55:32.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:32 smithi130 ceph-mon[84265]: from='osd.2 [v2:172.21.15.102:6818/3530038065,v1:172.21.15.102:6819/3530038065]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-05-13T22:55:32.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:32 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:32.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:32 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:32.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:32 smithi130 ceph-mon[84265]: from='osd.2 [v2:172.21.15.102:6818/3530038065,v1:172.21.15.102:6819/3530038065]' entity='osd.2' 2024-05-13T22:55:33.458 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:33 smithi102 ceph-mon[111694]: pgmap v110: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 37/231 objects degraded (16.017%) 2024-05-13T22:55:33.458 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:33 smithi102 ceph-mon[111694]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:55:33.458 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:33 smithi102 ceph-mon[111694]: osd.2 [v2:172.21.15.102:6818/3530038065,v1:172.21.15.102:6819/3530038065] boot 2024-05-13T22:55:33.458 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:33 smithi102 ceph-mon[111694]: osdmap e75: 6 total, 6 up, 6 in 2024-05-13T22:55:33.458 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:33 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:55:33.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:33 smithi130 ceph-mon[84265]: pgmap v110: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 37/231 objects degraded (16.017%) 2024-05-13T22:55:33.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:33 smithi130 ceph-mon[84265]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:55:33.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:33 smithi130 ceph-mon[84265]: osd.2 [v2:172.21.15.102:6818/3530038065,v1:172.21.15.102:6819/3530038065] boot 2024-05-13T22:55:33.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:33 smithi130 ceph-mon[84265]: osdmap e75: 6 total, 6 up, 6 in 2024-05-13T22:55:33.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:33 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-13T22:55:35.320 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:35 smithi102 ceph-mon[111694]: pgmap v112: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 37/231 objects degraded (16.017%) 2024-05-13T22:55:35.321 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:35 smithi102 ceph-mon[111694]: osdmap e76: 6 total, 6 up, 6 in 2024-05-13T22:55:35.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:35 smithi130 ceph-mon[84265]: pgmap v112: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 37/231 objects degraded (16.017%) 2024-05-13T22:55:35.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:35 smithi130 ceph-mon[84265]: osdmap e76: 6 total, 6 up, 6 in 2024-05-13T22:55:36.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-13T22:55:36.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:36 smithi130 ceph-mon[84265]: pgmap v114: 65 pgs: 10 active+undersized, 9 active+undersized+degraded, 46 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 23/231 objects degraded (9.957%) 2024-05-13T22:55:36.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:36.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:36.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:55:36.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:55:36.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:36.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:55:36.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:36.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:36.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:36.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-13T22:55:36.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-13T22:55:36.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-13T22:55:36.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:36 smithi102 ceph-mon[111694]: pgmap v114: 65 pgs: 10 active+undersized, 9 active+undersized+degraded, 46 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 23/231 objects degraded (9.957%) 2024-05-13T22:55:37.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:37 smithi130 ceph-mon[84265]: Health check update: Degraded data redundancy: 23/231 objects degraded (9.957%), 9 pgs degraded (PG_DEGRADED) 2024-05-13T22:55:37.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:37 smithi102 ceph-mon[111694]: Health check update: Degraded data redundancy: 23/231 objects degraded (9.957%), 9 pgs degraded (PG_DEGRADED) 2024-05-13T22:55:38.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:38 smithi130 ceph-mon[84265]: pgmap v115: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-05-13T22:55:38.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:38 smithi102 ceph-mon[111694]: pgmap v115: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-05-13T22:55:39.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:39 smithi130 ceph-mon[84265]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/231 objects degraded (9.957%), 9 pgs degraded) 2024-05-13T22:55:39.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:39 smithi130 ceph-mon[84265]: Cluster is now healthy 2024-05-13T22:55:39.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:39 smithi102 ceph-mon[111694]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/231 objects degraded (9.957%), 9 pgs degraded) 2024-05-13T22:55:39.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:39 smithi102 ceph-mon[111694]: Cluster is now healthy 2024-05-13T22:55:40.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:40 smithi130 ceph-mon[84265]: pgmap v116: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-05-13T22:55:40.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:40 smithi102 ceph-mon[111694]: pgmap v116: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-05-13T22:55:43.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:43 smithi130 ceph-mon[84265]: pgmap v117: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-05-13T22:55:43.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:43 smithi102 ceph-mon[111694]: pgmap v117: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-05-13T22:55:44.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.880+0000 7fa78856f700 1 -- 172.21.15.102:0/3980632454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa780105a40 msgr2=0x7fa780105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:44.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.880+0000 7fa78856f700 1 --2- 172.21.15.102:0/3980632454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa780105a40 0x7fa780105e20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00b0a0 tx=0x7fa77c00b3b0 comp rx=0 tx=0).stop 2024-05-13T22:55:44.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.880+0000 7fa78856f700 1 -- 172.21.15.102:0/3980632454 shutdown_connections 2024-05-13T22:55:44.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.880+0000 7fa78856f700 1 --2- 172.21.15.102:0/3980632454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7800ffbd0 0x7fa780100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:44.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.880+0000 7fa78856f700 1 --2- 172.21.15.102:0/3980632454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa780105a40 0x7fa780105e20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:44.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.880+0000 7fa78856f700 1 -- 172.21.15.102:0/3980632454 >> 172.21.15.102:0/3980632454 conn(0x7fa7800755a0 msgr2=0x7fa7800759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:44.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.880+0000 7fa78856f700 1 -- 172.21.15.102:0/3980632454 shutdown_connections 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.880+0000 7fa78856f700 1 -- 172.21.15.102:0/3980632454 wait complete. 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.881+0000 7fa78856f700 1 Processor -- start 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.881+0000 7fa78856f700 1 -- start start 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.881+0000 7fa78856f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7800ffbd0 0x7fa78019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.881+0000 7fa78856f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa780105a40 0x7fa78019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.881+0000 7fa78856f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78019cfd0 con 0x7fa780105a40 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.881+0000 7fa78856f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa780196520 con 0x7fa7800ffbd0 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.881+0000 7fa785b0a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa780105a40 0x7fa78019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.881+0000 7fa785b0a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa780105a40 0x7fa78019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60066/0 (socket says 172.21.15.102:60066) 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.881+0000 7fa785b0a700 1 -- 172.21.15.102:0/1932377101 learned_addr learned my addr 172.21.15.102:0/1932377101 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.881+0000 7fa78630b700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7800ffbd0 0x7fa78019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:44.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.882+0000 7fa785b0a700 1 -- 172.21.15.102:0/1932377101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7800ffbd0 msgr2=0x7fa78019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:44.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.882+0000 7fa785b0a700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7800ffbd0 0x7fa78019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:44.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.882+0000 7fa785b0a700 1 -- 172.21.15.102:0/1932377101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c009d00 con 0x7fa780105a40 2024-05-13T22:55:44.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.882+0000 7fa78630b700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7800ffbd0 0x7fa78019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:55:44.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.882+0000 7fa785b0a700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa780105a40 0x7fa78019c980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa77000c930 tx=0x7fa77000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:44.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.882+0000 7fa7777fe700 1 -- 172.21.15.102:0/1932377101 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa770007ad0 con 0x7fa780105a40 2024-05-13T22:55:44.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.882+0000 7fa7777fe700 1 -- 172.21.15.102:0/1932377101 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa770007c30 con 0x7fa780105a40 2024-05-13T22:55:44.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.882+0000 7fa78856f700 1 -- 172.21.15.102:0/1932377101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa780196800 con 0x7fa780105a40 2024-05-13T22:55:44.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.882+0000 7fa7777fe700 1 -- 172.21.15.102:0/1932377101 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa770018610 con 0x7fa780105a40 2024-05-13T22:55:44.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.882+0000 7fa78856f700 1 -- 172.21.15.102:0/1932377101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa780196d50 con 0x7fa780105a40 2024-05-13T22:55:44.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.883+0000 7fa78856f700 1 -- 172.21.15.102:0/1932377101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa780109050 con 0x7fa780105a40 2024-05-13T22:55:44.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.884+0000 7fa7777fe700 1 -- 172.21.15.102:0/1932377101 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa77000f450 con 0x7fa780105a40 2024-05-13T22:55:44.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.884+0000 7fa7777fe700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa76c06e200 0x7fa76c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:44.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.884+0000 7fa7777fe700 1 -- 172.21.15.102:0/1932377101 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fa77008e410 con 0x7fa780105a40 2024-05-13T22:55:44.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.884+0000 7fa78630b700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa76c06e200 0x7fa76c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:44.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.885+0000 7fa78630b700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa76c06e200 0x7fa76c0706c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0062a0 tx=0x7fa77c006210 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:44.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:44.889+0000 7fa7777fe700 1 -- 172.21.15.102:0/1932377101 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa77005b1b0 con 0x7fa780105a40 2024-05-13T22:55:45.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.109+0000 7fa78856f700 1 -- 172.21.15.102:0/1932377101 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa780196fd0 con 0x7fa76c06e200 2024-05-13T22:55:45.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.111+0000 7fa7777fe700 1 -- 172.21.15.102:0/1932377101 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa780196fd0 con 0x7fa76c06e200 2024-05-13T22:55:45.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.113+0000 7fa78856f700 1 -- 172.21.15.102:0/1932377101 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa76c06e200 msgr2=0x7fa76c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.113+0000 7fa78856f700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa76c06e200 0x7fa76c0706c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0062a0 tx=0x7fa77c006210 comp rx=0 tx=0).stop 2024-05-13T22:55:45.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.113+0000 7fa78856f700 1 -- 172.21.15.102:0/1932377101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa780105a40 msgr2=0x7fa78019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.113+0000 7fa78856f700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa780105a40 0x7fa78019c980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa77000c930 tx=0x7fa77000ccf0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.113+0000 7fa78856f700 1 -- 172.21.15.102:0/1932377101 shutdown_connections 2024-05-13T22:55:45.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.113+0000 7fa78856f700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa76c06e200 0x7fa76c0706c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.113+0000 7fa78856f700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7800ffbd0 0x7fa78019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.113+0000 7fa78856f700 1 --2- 172.21.15.102:0/1932377101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa780105a40 0x7fa78019c980 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.113+0000 7fa78856f700 1 -- 172.21.15.102:0/1932377101 >> 172.21.15.102:0/1932377101 conn(0x7fa7800755a0 msgr2=0x7fa7800685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:45.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.113+0000 7fa78856f700 1 -- 172.21.15.102:0/1932377101 shutdown_connections 2024-05-13T22:55:45.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.113+0000 7fa78856f700 1 -- 172.21.15.102:0/1932377101 wait complete. 2024-05-13T22:55:45.100 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:55:45.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:45 smithi102 ceph-mon[111694]: pgmap v118: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-05-13T22:55:45.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.220+0000 7f3e68b17700 1 -- 172.21.15.102:0/3892912769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e64101770 msgr2=0x7f3e641057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.220+0000 7f3e68b17700 1 --2- 172.21.15.102:0/3892912769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e64101770 0x7f3e641057c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009910 tx=0x7f3e58009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:45.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.220+0000 7f3e68b17700 1 -- 172.21.15.102:0/3892912769 shutdown_connections 2024-05-13T22:55:45.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.220+0000 7f3e68b17700 1 --2- 172.21.15.102:0/3892912769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e64101770 0x7f3e641057c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.220+0000 7f3e68b17700 1 --2- 172.21.15.102:0/3892912769 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e64100dc0 0x7f3e641011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.220+0000 7f3e68b17700 1 -- 172.21.15.102:0/3892912769 >> 172.21.15.102:0/3892912769 conn(0x7f3e640fc650 msgr2=0x7f3e640fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:45.197 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.220+0000 7f3e68b17700 1 -- 172.21.15.102:0/3892912769 shutdown_connections 2024-05-13T22:55:45.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.220+0000 7f3e68b17700 1 -- 172.21.15.102:0/3892912769 wait complete. 2024-05-13T22:55:45.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.221+0000 7f3e68b17700 1 Processor -- start 2024-05-13T22:55:45.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.221+0000 7f3e68b17700 1 -- start start 2024-05-13T22:55:45.199 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.221+0000 7f3e68b17700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e64100dc0 0x7f3e6419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.221+0000 7f3e68b17700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e64101770 0x7f3e6419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.221+0000 7f3e68b17700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6419d240 con 0x7f3e64101770 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.221+0000 7f3e68b17700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e641966f0 con 0x7f3e64100dc0 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.221+0000 7f3e61d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e64101770 0x7f3e6419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.221+0000 7f3e61d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e64101770 0x7f3e6419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60084/0 (socket says 172.21.15.102:60084) 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.222+0000 7f3e61d9b700 1 -- 172.21.15.102:0/1639226078 learned_addr learned my addr 172.21.15.102:0/1639226078 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.222+0000 7f3e6259c700 1 --2- 172.21.15.102:0/1639226078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e64100dc0 0x7f3e6419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.222+0000 7f3e61d9b700 1 -- 172.21.15.102:0/1639226078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e64100dc0 msgr2=0x7f3e6419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.222+0000 7f3e61d9b700 1 --2- 172.21.15.102:0/1639226078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e64100dc0 0x7f3e6419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.222+0000 7f3e61d9b700 1 -- 172.21.15.102:0/1639226078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e580095f0 con 0x7f3e64101770 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.222+0000 7f3e61d9b700 1 --2- 172.21.15.102:0/1639226078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e64101770 0x7f3e6419cbb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3e580092e0 tx=0x7f3e5800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.222+0000 7f3e537fe700 1 -- 172.21.15.102:0/1639226078 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e5801d070 con 0x7f3e64101770 2024-05-13T22:55:45.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.222+0000 7f3e537fe700 1 -- 172.21.15.102:0/1639226078 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e5800f460 con 0x7f3e64101770 2024-05-13T22:55:45.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.222+0000 7f3e68b17700 1 -- 172.21.15.102:0/1639226078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e64196970 con 0x7f3e64101770 2024-05-13T22:55:45.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.222+0000 7f3e537fe700 1 -- 172.21.15.102:0/1639226078 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e580175d0 con 0x7f3e64101770 2024-05-13T22:55:45.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.222+0000 7f3e68b17700 1 -- 172.21.15.102:0/1639226078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e64196e60 con 0x7f3e64101770 2024-05-13T22:55:45.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.223+0000 7f3e68b17700 1 -- 172.21.15.102:0/1639226078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e64109110 con 0x7f3e64101770 2024-05-13T22:55:45.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.224+0000 7f3e537fe700 1 -- 172.21.15.102:0/1639226078 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3e5800f5d0 con 0x7f3e64101770 2024-05-13T22:55:45.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.224+0000 7f3e537fe700 1 --2- 172.21.15.102:0/1639226078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3e4c06e200 0x7f3e4c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:45.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.224+0000 7f3e537fe700 1 -- 172.21.15.102:0/1639226078 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f3e5808f700 con 0x7f3e64101770 2024-05-13T22:55:45.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.224+0000 7f3e6259c700 1 --2- 172.21.15.102:0/1639226078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3e4c06e200 0x7f3e4c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:45.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.225+0000 7f3e6259c700 1 --2- 172.21.15.102:0/1639226078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3e4c06e200 0x7f3e4c0706c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009a10 tx=0x7f3e54009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:45.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.229+0000 7f3e537fe700 1 -- 172.21.15.102:0/1639226078 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3e5805c4a0 con 0x7f3e64101770 2024-05-13T22:55:45.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:45 smithi130 ceph-mon[84265]: pgmap v118: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-05-13T22:55:45.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.444+0000 7f3e68b17700 1 -- 172.21.15.102:0/1639226078 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e6402cce0 con 0x7f3e4c06e200 2024-05-13T22:55:45.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.445+0000 7f3e537fe700 1 -- 172.21.15.102:0/1639226078 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3e6402cce0 con 0x7f3e4c06e200 2024-05-13T22:55:45.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.447+0000 7f3e68b17700 1 -- 172.21.15.102:0/1639226078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3e4c06e200 msgr2=0x7f3e4c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.447+0000 7f3e68b17700 1 --2- 172.21.15.102:0/1639226078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3e4c06e200 0x7f3e4c0706c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009a10 tx=0x7f3e54009190 comp rx=0 tx=0).stop 2024-05-13T22:55:45.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.447+0000 7f3e68b17700 1 -- 172.21.15.102:0/1639226078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e64101770 msgr2=0x7f3e6419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.447+0000 7f3e68b17700 1 --2- 172.21.15.102:0/1639226078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e64101770 0x7f3e6419cbb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3e580092e0 tx=0x7f3e5800bd70 comp rx=0 tx=0).stop 2024-05-13T22:55:45.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.447+0000 7f3e68b17700 1 -- 172.21.15.102:0/1639226078 shutdown_connections 2024-05-13T22:55:45.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.447+0000 7f3e68b17700 1 --2- 172.21.15.102:0/1639226078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f3e4c06e200 0x7f3e4c0706c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.447+0000 7f3e68b17700 1 --2- 172.21.15.102:0/1639226078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e64100dc0 0x7f3e6419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.447+0000 7f3e68b17700 1 --2- 172.21.15.102:0/1639226078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e64101770 0x7f3e6419cbb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.447+0000 7f3e68b17700 1 -- 172.21.15.102:0/1639226078 >> 172.21.15.102:0/1639226078 conn(0x7f3e640fc650 msgr2=0x7f3e640fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:45.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.448+0000 7f3e68b17700 1 -- 172.21.15.102:0/1639226078 shutdown_connections 2024-05-13T22:55:45.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.448+0000 7f3e68b17700 1 -- 172.21.15.102:0/1639226078 wait complete. 2024-05-13T22:55:45.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.551+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/3989746162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b14100da0 msgr2=0x7f9b14101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.551+0000 7f9b1b9a0700 1 --2- 172.21.15.102:0/3989746162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b14100da0 0x7f9b14101180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009910 tx=0x7f9b10009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:45.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.552+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/3989746162 shutdown_connections 2024-05-13T22:55:45.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.552+0000 7f9b1b9a0700 1 --2- 172.21.15.102:0/3989746162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b14101750 0x7f9b141057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.552+0000 7f9b1b9a0700 1 --2- 172.21.15.102:0/3989746162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b14100da0 0x7f9b14101180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.552+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/3989746162 >> 172.21.15.102:0/3989746162 conn(0x7f9b140fc650 msgr2=0x7f9b140fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:45.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.552+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/3989746162 shutdown_connections 2024-05-13T22:55:45.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.552+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/3989746162 wait complete. 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.552+0000 7f9b1b9a0700 1 Processor -- start 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b1b9a0700 1 -- start start 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b1b9a0700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b14100da0 0x7f9b1419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b1b9a0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b14101750 0x7f9b1419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b1b9a0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1419d110 con 0x7f9b14101750 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b1b9a0700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b141965c0 con 0x7f9b14100da0 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b18f3b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b14101750 0x7f9b1419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b18f3b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b14101750 0x7f9b1419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60112/0 (socket says 172.21.15.102:60112) 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b18f3b700 1 -- 172.21.15.102:0/1938638055 learned_addr learned my addr 172.21.15.102:0/1938638055 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b1973c700 1 --2- 172.21.15.102:0/1938638055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b14100da0 0x7f9b1419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b18f3b700 1 -- 172.21.15.102:0/1938638055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b14100da0 msgr2=0x7f9b1419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.531 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b18f3b700 1 --2- 172.21.15.102:0/1938638055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b14100da0 0x7f9b1419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.553+0000 7f9b18f3b700 1 -- 172.21.15.102:0/1938638055 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b100095f0 con 0x7f9b14101750 2024-05-13T22:55:45.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.554+0000 7f9b18f3b700 1 --2- 172.21.15.102:0/1938638055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b14101750 0x7f9b1419ca80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400d8d0 tx=0x7f9b0400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:45.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.554+0000 7f9b0a7fc700 1 -- 172.21.15.102:0/1938638055 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b04004a00 con 0x7f9b14101750 2024-05-13T22:55:45.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.554+0000 7f9b0a7fc700 1 -- 172.21.15.102:0/1938638055 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b040054e0 con 0x7f9b14101750 2024-05-13T22:55:45.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.554+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/1938638055 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b141968a0 con 0x7f9b14101750 2024-05-13T22:55:45.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.554+0000 7f9b0a7fc700 1 -- 172.21.15.102:0/1938638055 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b0400fc20 con 0x7f9b14101750 2024-05-13T22:55:45.532 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.554+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/1938638055 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b14196df0 con 0x7f9b14101750 2024-05-13T22:55:45.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.555+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/1938638055 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b14109140 con 0x7f9b14101750 2024-05-13T22:55:45.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.556+0000 7f9b0a7fc700 1 -- 172.21.15.102:0/1938638055 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9b04005020 con 0x7f9b14101750 2024-05-13T22:55:45.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.556+0000 7f9b0a7fc700 1 --2- 172.21.15.102:0/1938638055 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b0006e200 0x7f9b000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:45.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.556+0000 7f9b0a7fc700 1 -- 172.21.15.102:0/1938638055 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f9b0408daa0 con 0x7f9b14101750 2024-05-13T22:55:45.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.556+0000 7f9b1973c700 1 --2- 172.21.15.102:0/1938638055 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b0006e200 0x7f9b000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:45.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.557+0000 7f9b1973c700 1 --2- 172.21.15.102:0/1938638055 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b0006e200 0x7f9b000706c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b10006010 tx=0x7f9b1001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:45.537 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.561+0000 7f9b0a7fc700 1 -- 172.21.15.102:0/1938638055 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9b0405a840 con 0x7f9b14101750 2024-05-13T22:55:45.747 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.771+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/1938638055 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b14197040 con 0x7f9b0006e200 2024-05-13T22:55:45.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.782+0000 7f9b0a7fc700 1 -- 172.21.15.102:0/1938638055 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f9b14197040 con 0x7f9b0006e200 2024-05-13T22:55:45.758 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:55:45.758 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (4m) 15s ago 15m 26.7M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:55:45.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (16m) 15s ago 16m 8740k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:55:45.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (15m) 110s ago 15m 7956k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (119s) 15s ago 16m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (114s) 110s ago 15m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (4m) 15s ago 15m 79.9M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (10m) 15s ago 10m 88.3M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (10m) 15s ago 10m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (10m) 110s ago 10m 16.1M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (10m) 110s ago 10m 19.1M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (7m) 15s ago 17m 506M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (6m) 110s ago 14m 451M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (3m) 15s ago 17m 51.7M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (2m) 110s ago 14m 41.6M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (5m) 15s ago 16m 16.0M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (5m) 110s ago 15m 17.1M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:55:45.759 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (93s) 15s ago 14m 133M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 0ada88c777a2 2024-05-13T22:55:45.760 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (56s) 15s ago 13m 130M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 c3e9e68b06fd 2024-05-13T22:55:45.760 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (21s) 15s ago 13m 15.1M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 1b85728dccf5 2024-05-13T22:55:45.760 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (12m) 110s ago 12m 293M 2240M 18.2.1 5be31c24972a 3b510d692e3f 2024-05-13T22:55:45.760 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (12m) 110s ago 12m 245M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:55:45.760 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (11m) 110s ago 11m 227M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:55:45.760 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (5m) 15s ago 15m 73.7M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:55:45.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.784+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/1938638055 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b0006e200 msgr2=0x7f9b000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.784+0000 7f9b1b9a0700 1 --2- 172.21.15.102:0/1938638055 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b0006e200 0x7f9b000706c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b10006010 tx=0x7f9b1001a040 comp rx=0 tx=0).stop 2024-05-13T22:55:45.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.784+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/1938638055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b14101750 msgr2=0x7f9b1419ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.784+0000 7f9b1b9a0700 1 --2- 172.21.15.102:0/1938638055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b14101750 0x7f9b1419ca80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400d8d0 tx=0x7f9b0400dbe0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.784+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/1938638055 shutdown_connections 2024-05-13T22:55:45.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.784+0000 7f9b1b9a0700 1 --2- 172.21.15.102:0/1938638055 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b0006e200 0x7f9b000706c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.784+0000 7f9b1b9a0700 1 --2- 172.21.15.102:0/1938638055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b14100da0 0x7f9b1419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.784+0000 7f9b1b9a0700 1 --2- 172.21.15.102:0/1938638055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b14101750 0x7f9b1419ca80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.784+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/1938638055 >> 172.21.15.102:0/1938638055 conn(0x7f9b140fc650 msgr2=0x7f9b140fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:45.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.784+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/1938638055 shutdown_connections 2024-05-13T22:55:45.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.784+0000 7f9b1b9a0700 1 -- 172.21.15.102:0/1938638055 wait complete. 2024-05-13T22:55:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.888+0000 7ff52ff52700 1 -- 172.21.15.102:0/4088561583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5280ffbd0 msgr2=0x7ff528100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.888+0000 7ff52ff52700 1 --2- 172.21.15.102:0/4088561583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5280ffbd0 0x7ff528100050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff524009910 tx=0x7ff524009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.888+0000 7ff52ff52700 1 -- 172.21.15.102:0/4088561583 shutdown_connections 2024-05-13T22:55:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.888+0000 7ff52ff52700 1 --2- 172.21.15.102:0/4088561583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5280ffbd0 0x7ff528100050 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.888+0000 7ff52ff52700 1 --2- 172.21.15.102:0/4088561583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff528105a40 0x7ff528105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.888+0000 7ff52ff52700 1 -- 172.21.15.102:0/4088561583 >> 172.21.15.102:0/4088561583 conn(0x7ff5280755a0 msgr2=0x7ff5280759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.888+0000 7ff52ff52700 1 -- 172.21.15.102:0/4088561583 shutdown_connections 2024-05-13T22:55:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.888+0000 7ff52ff52700 1 -- 172.21.15.102:0/4088561583 wait complete. 2024-05-13T22:55:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.889+0000 7ff52ff52700 1 Processor -- start 2024-05-13T22:55:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.889+0000 7ff52ff52700 1 -- start start 2024-05-13T22:55:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.889+0000 7ff52ff52700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5280ffbd0 0x7ff528195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.889+0000 7ff52ff52700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff528105a40 0x7ff528195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.889+0000 7ff52ff52700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5281961b0 con 0x7ff528105a40 2024-05-13T22:55:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.889+0000 7ff52ff52700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5281962f0 con 0x7ff5280ffbd0 2024-05-13T22:55:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.889+0000 7ff52d4ed700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff528105a40 0x7ff528195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.889+0000 7ff52d4ed700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff528105a40 0x7ff528195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60142/0 (socket says 172.21.15.102:60142) 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.889+0000 7ff52d4ed700 1 -- 172.21.15.102:0/2083347167 learned_addr learned my addr 172.21.15.102:0/2083347167 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.890+0000 7ff52dcee700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5280ffbd0 0x7ff528195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.890+0000 7ff52d4ed700 1 -- 172.21.15.102:0/2083347167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5280ffbd0 msgr2=0x7ff528195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.890+0000 7ff52d4ed700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5280ffbd0 0x7ff528195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.890+0000 7ff52d4ed700 1 -- 172.21.15.102:0/2083347167 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5240095f0 con 0x7ff528105a40 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.890+0000 7ff52dcee700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5280ffbd0 0x7ff528195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.890+0000 7ff52d4ed700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff528105a40 0x7ff528195b60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff524006010 tx=0x7ff52400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.890+0000 7ff51effd700 1 -- 172.21.15.102:0/2083347167 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff52401d070 con 0x7ff528105a40 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.890+0000 7ff52ff52700 1 -- 172.21.15.102:0/2083347167 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52819a0e0 con 0x7ff528105a40 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.890+0000 7ff51effd700 1 -- 172.21.15.102:0/2083347167 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff524004fb0 con 0x7ff528105a40 2024-05-13T22:55:45.868 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.890+0000 7ff51effd700 1 -- 172.21.15.102:0/2083347167 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff524022ea0 con 0x7ff528105a40 2024-05-13T22:55:45.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.890+0000 7ff52ff52700 1 -- 172.21.15.102:0/2083347167 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52819a5d0 con 0x7ff528105a40 2024-05-13T22:55:45.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.891+0000 7ff52ff52700 1 -- 172.21.15.102:0/2083347167 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff528107fd0 con 0x7ff528105a40 2024-05-13T22:55:45.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.892+0000 7ff51effd700 1 -- 172.21.15.102:0/2083347167 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff524022470 con 0x7ff528105a40 2024-05-13T22:55:45.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.892+0000 7ff51effd700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff51406e200 0x7ff5140706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:45.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.892+0000 7ff51effd700 1 -- 172.21.15.102:0/2083347167 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7ff52408f6b0 con 0x7ff528105a40 2024-05-13T22:55:45.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.892+0000 7ff52dcee700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff51406e200 0x7ff5140706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:45.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.893+0000 7ff52dcee700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff51406e200 0x7ff5140706c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff518005fd0 tx=0x7ff518005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:45.873 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:45.897+0000 7ff51effd700 1 -- 172.21.15.102:0/2083347167 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff524058a50 con 0x7ff528105a40 2024-05-13T22:55:46.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.183+0000 7ff52ff52700 1 -- 172.21.15.102:0/2083347167 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff52804ea90 con 0x7ff528105a40 2024-05-13T22:55:46.160 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:46 smithi102 ceph-mon[111694]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:46.160 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:46.161 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:55:46.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.184+0000 7ff51effd700 1 -- 172.21.15.102:0/2083347167 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7ff524022720 con 0x7ff528105a40 2024-05-13T22:55:46.161 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:55:46.161 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:55:46.161 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:55:46.161 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:55:46.161 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:55:46.161 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 3 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 7 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:55:46.162 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:55:46.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.186+0000 7ff52ff52700 1 -- 172.21.15.102:0/2083347167 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff51406e200 msgr2=0x7ff5140706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.186+0000 7ff52ff52700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff51406e200 0x7ff5140706c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff518005fd0 tx=0x7ff518005e20 comp rx=0 tx=0).stop 2024-05-13T22:55:46.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.186+0000 7ff52ff52700 1 -- 172.21.15.102:0/2083347167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff528105a40 msgr2=0x7ff528195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.186+0000 7ff52ff52700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff528105a40 0x7ff528195b60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff524006010 tx=0x7ff52400be20 comp rx=0 tx=0).stop 2024-05-13T22:55:46.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.186+0000 7ff52ff52700 1 -- 172.21.15.102:0/2083347167 shutdown_connections 2024-05-13T22:55:46.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.186+0000 7ff52ff52700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff51406e200 0x7ff5140706c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.186+0000 7ff52ff52700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5280ffbd0 0x7ff528195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.186+0000 7ff52ff52700 1 --2- 172.21.15.102:0/2083347167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff528105a40 0x7ff528195b60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.186+0000 7ff52ff52700 1 -- 172.21.15.102:0/2083347167 >> 172.21.15.102:0/2083347167 conn(0x7ff5280755a0 msgr2=0x7ff5280fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:46.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.186+0000 7ff52ff52700 1 -- 172.21.15.102:0/2083347167 shutdown_connections 2024-05-13T22:55:46.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.186+0000 7ff52ff52700 1 -- 172.21.15.102:0/2083347167 wait complete. 2024-05-13T22:55:46.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.297+0000 7f0738cee700 1 -- 172.21.15.102:0/2139759640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0734105a40 msgr2=0x7f0734105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.297+0000 7f0738cee700 1 --2- 172.21.15.102:0/2139759640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0734105a40 0x7f0734105e20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0728009910 tx=0x7f0728009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:46.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.297+0000 7f0738cee700 1 -- 172.21.15.102:0/2139759640 shutdown_connections 2024-05-13T22:55:46.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.297+0000 7f0738cee700 1 --2- 172.21.15.102:0/2139759640 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07340ffa40 0x7f07340ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.297+0000 7f0738cee700 1 --2- 172.21.15.102:0/2139759640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0734105a40 0x7f0734105e20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.297+0000 7f0738cee700 1 -- 172.21.15.102:0/2139759640 >> 172.21.15.102:0/2139759640 conn(0x7f07340fb550 msgr2=0x7f07340fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:46.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.297+0000 7f0738cee700 1 -- 172.21.15.102:0/2139759640 shutdown_connections 2024-05-13T22:55:46.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.297+0000 7f0738cee700 1 -- 172.21.15.102:0/2139759640 wait complete. 2024-05-13T22:55:46.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.298+0000 7f0738cee700 1 Processor -- start 2024-05-13T22:55:46.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.298+0000 7f0738cee700 1 -- start start 2024-05-13T22:55:46.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.298+0000 7f0738cee700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07340ffa40 0x7f07340fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.298+0000 7f0738cee700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0734105a40 0x7f07340fc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.298+0000 7f0738cee700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340fc8a0 con 0x7f0734105a40 2024-05-13T22:55:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.298+0000 7f0738cee700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340fc9e0 con 0x7f07340ffa40 2024-05-13T22:55:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.298+0000 7f0731d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0734105a40 0x7f07340fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.298+0000 7f0731d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0734105a40 0x7f07340fc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60156/0 (socket says 172.21.15.102:60156) 2024-05-13T22:55:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.298+0000 7f0731d9b700 1 -- 172.21.15.102:0/1864119915 learned_addr learned my addr 172.21.15.102:0/1864119915 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.298+0000 7f073259c700 1 --2- 172.21.15.102:0/1864119915 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07340ffa40 0x7f07340fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.299+0000 7f0731d9b700 1 -- 172.21.15.102:0/1864119915 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07340ffa40 msgr2=0x7f07340fbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.299+0000 7f0731d9b700 1 --2- 172.21.15.102:0/1864119915 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07340ffa40 0x7f07340fbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.299+0000 7f0731d9b700 1 -- 172.21.15.102:0/1864119915 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07280095f0 con 0x7f0734105a40 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.299+0000 7f0731d9b700 1 --2- 172.21.15.102:0/1864119915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0734105a40 0x7f07340fc360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f072400c930 tx=0x7f072400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.299+0000 7f07237fe700 1 -- 172.21.15.102:0/1864119915 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0724007ad0 con 0x7f0734105a40 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.299+0000 7f07237fe700 1 -- 172.21.15.102:0/1864119915 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0724007c30 con 0x7f0734105a40 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.299+0000 7f07237fe700 1 -- 172.21.15.102:0/1864119915 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0724018610 con 0x7f0734105a40 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.299+0000 7f0738cee700 1 -- 172.21.15.102:0/1864119915 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07341a29f0 con 0x7f0734105a40 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.299+0000 7f0738cee700 1 -- 172.21.15.102:0/1864119915 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07341a2d50 con 0x7f0734105a40 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.300+0000 7f0738cee700 1 -- 172.21.15.102:0/1864119915 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0734109050 con 0x7f0734105a40 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.300+0000 7f07237fe700 1 -- 172.21.15.102:0/1864119915 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f072400f450 con 0x7f0734105a40 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.301+0000 7f07237fe700 1 --2- 172.21.15.102:0/1864119915 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f071c06e200 0x7f071c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.301+0000 7f07237fe700 1 -- 172.21.15.102:0/1864119915 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f072408e340 con 0x7f0734105a40 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.301+0000 7f073259c700 1 --2- 172.21.15.102:0/1864119915 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f071c06e200 0x7f071c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:46.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.301+0000 7f073259c700 1 --2- 172.21.15.102:0/1864119915 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f071c06e200 0x7f071c0706c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f072800b5c0 tx=0x7f072801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:46.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.306+0000 7f07237fe700 1 -- 172.21.15.102:0/1864119915 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0724057630 con 0x7f0734105a40 2024-05-13T22:55:46.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:46 smithi130 ceph-mon[84265]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:46.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:46.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:55:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.547+0000 7f0738cee700 1 -- 172.21.15.102:0/1864119915 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f073404ea90 con 0x7f0734105a40 2024-05-13T22:55:46.524 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.548+0000 7f07237fe700 1 -- 172.21.15.102:0/1864119915 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1909 (secure 0 0 0) 0x7f072405acf0 con 0x7f0734105a40 2024-05-13T22:55:46.524 INFO:teuthology.orchestra.run.smithi102.stdout:e17 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 17 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:55:46.525 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 47 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=14498} 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:55:46.526 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 0 members: 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:55:46.527 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:55:46.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.550+0000 7f0738cee700 1 -- 172.21.15.102:0/1864119915 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f071c06e200 msgr2=0x7f071c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.550+0000 7f0738cee700 1 --2- 172.21.15.102:0/1864119915 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f071c06e200 0x7f071c0706c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f072800b5c0 tx=0x7f072801a040 comp rx=0 tx=0).stop 2024-05-13T22:55:46.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.550+0000 7f0738cee700 1 -- 172.21.15.102:0/1864119915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0734105a40 msgr2=0x7f07340fc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.550+0000 7f0738cee700 1 --2- 172.21.15.102:0/1864119915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0734105a40 0x7f07340fc360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f072400c930 tx=0x7f072400ccf0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.550+0000 7f0738cee700 1 -- 172.21.15.102:0/1864119915 shutdown_connections 2024-05-13T22:55:46.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.550+0000 7f0738cee700 1 --2- 172.21.15.102:0/1864119915 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f071c06e200 0x7f071c0706c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.550+0000 7f0738cee700 1 --2- 172.21.15.102:0/1864119915 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07340ffa40 0x7f07340fbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.550+0000 7f0738cee700 1 --2- 172.21.15.102:0/1864119915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0734105a40 0x7f07340fc360 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.550+0000 7f0738cee700 1 -- 172.21.15.102:0/1864119915 >> 172.21.15.102:0/1864119915 conn(0x7f07340fb550 msgr2=0x7f0734069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:46.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.550+0000 7f0738cee700 1 -- 172.21.15.102:0/1864119915 shutdown_connections 2024-05-13T22:55:46.529 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.550+0000 7f0738cee700 1 -- 172.21.15.102:0/1864119915 wait complete. 2024-05-13T22:55:46.529 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:55:46.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.655+0000 7f94e3cb4700 1 -- 172.21.15.102:0/3844798412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94dc0ffec0 msgr2=0x7f94dc105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.655+0000 7f94e3cb4700 1 --2- 172.21.15.102:0/3844798412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94dc0ffec0 0x7f94dc105850 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009910 tx=0x7f94d8009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:46.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.655+0000 7f94e3cb4700 1 -- 172.21.15.102:0/3844798412 shutdown_connections 2024-05-13T22:55:46.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.655+0000 7f94e3cb4700 1 --2- 172.21.15.102:0/3844798412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94dc0ffec0 0x7f94dc105850 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.655+0000 7f94e3cb4700 1 --2- 172.21.15.102:0/3844798412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94dc0ff510 0x7f94dc0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.655+0000 7f94e3cb4700 1 -- 172.21.15.102:0/3844798412 >> 172.21.15.102:0/3844798412 conn(0x7f94dc074d80 msgr2=0x7f94dc075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:46.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.655+0000 7f94e3cb4700 1 -- 172.21.15.102:0/3844798412 shutdown_connections 2024-05-13T22:55:46.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.655+0000 7f94e3cb4700 1 -- 172.21.15.102:0/3844798412 wait complete. 2024-05-13T22:55:46.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.655+0000 7f94e3cb4700 1 Processor -- start 2024-05-13T22:55:46.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.655+0000 7f94e3cb4700 1 -- start start 2024-05-13T22:55:46.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e3cb4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94dc0ff510 0x7f94dc19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:46.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e3cb4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94dc0ffec0 0x7f94dc19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:46.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e3cb4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc19d0e0 con 0x7f94dc0ffec0 2024-05-13T22:55:46.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e3cb4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc196590 con 0x7f94dc0ff510 2024-05-13T22:55:46.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e124f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94dc0ffec0 0x7f94dc19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:46.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e124f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94dc0ffec0 0x7f94dc19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60164/0 (socket says 172.21.15.102:60164) 2024-05-13T22:55:46.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e124f700 1 -- 172.21.15.102:0/2955853729 learned_addr learned my addr 172.21.15.102:0/2955853729 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:46.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e1a50700 1 --2- 172.21.15.102:0/2955853729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94dc0ff510 0x7f94dc19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:46.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e124f700 1 -- 172.21.15.102:0/2955853729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94dc0ff510 msgr2=0x7f94dc19c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e124f700 1 --2- 172.21.15.102:0/2955853729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94dc0ff510 0x7f94dc19c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e124f700 1 -- 172.21.15.102:0/2955853729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d80095f0 con 0x7f94dc0ffec0 2024-05-13T22:55:46.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e124f700 1 --2- 172.21.15.102:0/2955853729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94dc0ffec0 0x7f94dc19ca50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f94d8000c00 tx=0x7f94d800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:46.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94d2ffd700 1 -- 172.21.15.102:0/2955853729 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94d801d070 con 0x7f94dc0ffec0 2024-05-13T22:55:46.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94d2ffd700 1 -- 172.21.15.102:0/2955853729 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94d8005dc0 con 0x7f94dc0ffec0 2024-05-13T22:55:46.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94e3cb4700 1 -- 172.21.15.102:0/2955853729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc196810 con 0x7f94dc0ffec0 2024-05-13T22:55:46.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.656+0000 7f94d2ffd700 1 -- 172.21.15.102:0/2955853729 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94d800fe70 con 0x7f94dc0ffec0 2024-05-13T22:55:46.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.657+0000 7f94e3cb4700 1 -- 172.21.15.102:0/2955853729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc196d00 con 0x7f94dc0ffec0 2024-05-13T22:55:46.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.657+0000 7f94e3cb4700 1 -- 172.21.15.102:0/2955853729 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc109180 con 0x7f94dc0ffec0 2024-05-13T22:55:46.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.658+0000 7f94d2ffd700 1 -- 172.21.15.102:0/2955853729 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f94d800f6b0 con 0x7f94dc0ffec0 2024-05-13T22:55:46.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.658+0000 7f94d2ffd700 1 --2- 172.21.15.102:0/2955853729 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f94c806e200 0x7f94c80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:46.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.658+0000 7f94d2ffd700 1 -- 172.21.15.102:0/2955853729 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f94d808f640 con 0x7f94dc0ffec0 2024-05-13T22:55:46.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.658+0000 7f94e1a50700 1 --2- 172.21.15.102:0/2955853729 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f94c806e200 0x7f94c80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:46.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.659+0000 7f94e1a50700 1 --2- 172.21.15.102:0/2955853729 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f94c806e200 0x7f94c80706c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94cc005950 tx=0x7f94cc00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:46.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.663+0000 7f94d2ffd700 1 -- 172.21.15.102:0/2955853729 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f94d805c3e0 con 0x7f94dc0ffec0 2024-05-13T22:55:46.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.881+0000 7f94e3cb4700 1 -- 172.21.15.102:0/2955853729 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94dc196fe0 con 0x7f94c806e200 2024-05-13T22:55:46.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.882+0000 7f94d2ffd700 1 -- 172.21.15.102:0/2955853729 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f94dc196fe0 con 0x7f94c806e200 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr", 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout: "mon" 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "9/23 daemons upgraded", 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading osd daemons", 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:55:46.859 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:55:46.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.884+0000 7f94e3cb4700 1 -- 172.21.15.102:0/2955853729 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f94c806e200 msgr2=0x7f94c80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.884+0000 7f94e3cb4700 1 --2- 172.21.15.102:0/2955853729 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f94c806e200 0x7f94c80706c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94cc005950 tx=0x7f94cc00b410 comp rx=0 tx=0).stop 2024-05-13T22:55:46.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.884+0000 7f94e3cb4700 1 -- 172.21.15.102:0/2955853729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94dc0ffec0 msgr2=0x7f94dc19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.884+0000 7f94e3cb4700 1 --2- 172.21.15.102:0/2955853729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94dc0ffec0 0x7f94dc19ca50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f94d8000c00 tx=0x7f94d800bd70 comp rx=0 tx=0).stop 2024-05-13T22:55:46.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.885+0000 7f94e3cb4700 1 -- 172.21.15.102:0/2955853729 shutdown_connections 2024-05-13T22:55:46.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.885+0000 7f94e3cb4700 1 --2- 172.21.15.102:0/2955853729 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f94c806e200 0x7f94c80706c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.885+0000 7f94e3cb4700 1 --2- 172.21.15.102:0/2955853729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94dc0ff510 0x7f94dc19c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.885+0000 7f94e3cb4700 1 --2- 172.21.15.102:0/2955853729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f94dc0ffec0 0x7f94dc19ca50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.885+0000 7f94e3cb4700 1 -- 172.21.15.102:0/2955853729 >> 172.21.15.102:0/2955853729 conn(0x7f94dc074d80 msgr2=0x7f94dc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:46.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.885+0000 7f94e3cb4700 1 -- 172.21.15.102:0/2955853729 shutdown_connections 2024-05-13T22:55:46.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.885+0000 7f94e3cb4700 1 -- 172.21.15.102:0/2955853729 wait complete. 2024-05-13T22:55:46.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.989+0000 7fb90ec35700 1 -- 172.21.15.102:0/421513296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb908101da0 msgr2=0x7fb908102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.989+0000 7fb90ec35700 1 --2- 172.21.15.102:0/421513296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb908101da0 0x7fb908102180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009910 tx=0x7fb8fc009c20 comp rx=0 tx=0).stop 2024-05-13T22:55:46.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.989+0000 7fb90ec35700 1 -- 172.21.15.102:0/421513296 shutdown_connections 2024-05-13T22:55:46.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.989+0000 7fb90ec35700 1 --2- 172.21.15.102:0/421513296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9081026c0 0x7fb90810abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.989+0000 7fb90ec35700 1 --2- 172.21.15.102:0/421513296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb908101da0 0x7fb908102180 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.989+0000 7fb90ec35700 1 -- 172.21.15.102:0/421513296 >> 172.21.15.102:0/421513296 conn(0x7fb9080fb570 msgr2=0x7fb9080fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:46.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.989+0000 7fb90ec35700 1 -- 172.21.15.102:0/421513296 shutdown_connections 2024-05-13T22:55:46.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.989+0000 7fb90ec35700 1 -- 172.21.15.102:0/421513296 wait complete. 2024-05-13T22:55:46.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.990+0000 7fb90ec35700 1 Processor -- start 2024-05-13T22:55:46.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.990+0000 7fb90ec35700 1 -- start start 2024-05-13T22:55:46.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.990+0000 7fb90ec35700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb908101da0 0x7fb90819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:46.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.990+0000 7fb90ec35700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9081026c0 0x7fb90819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:46.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.990+0000 7fb90ec35700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90819d120 con 0x7fb908101da0 2024-05-13T22:55:46.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.990+0000 7fb90ec35700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9081965d0 con 0x7fb9081026c0 2024-05-13T22:55:46.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.990+0000 7fb90c9d1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb908101da0 0x7fb90819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:46.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.990+0000 7fb90c9d1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb908101da0 0x7fb90819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:60186/0 (socket says 172.21.15.102:60186) 2024-05-13T22:55:46.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.990+0000 7fb90c9d1700 1 -- 172.21.15.102:0/71130059 learned_addr learned my addr 172.21.15.102:0/71130059 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:55:46.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.990+0000 7fb907fff700 1 --2- 172.21.15.102:0/71130059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9081026c0 0x7fb90819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:46.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.991+0000 7fb90c9d1700 1 -- 172.21.15.102:0/71130059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9081026c0 msgr2=0x7fb90819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:46.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.991+0000 7fb90c9d1700 1 --2- 172.21.15.102:0/71130059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9081026c0 0x7fb90819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:46.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.991+0000 7fb90c9d1700 1 -- 172.21.15.102:0/71130059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc0095f0 con 0x7fb908101da0 2024-05-13T22:55:46.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.991+0000 7fb90c9d1700 1 --2- 172.21.15.102:0/71130059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb908101da0 0x7fb90819c550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0092e0 tx=0x7fb8fc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:46.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.991+0000 7fb905ffb700 1 -- 172.21.15.102:0/71130059 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8fc01d070 con 0x7fb908101da0 2024-05-13T22:55:46.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.991+0000 7fb905ffb700 1 -- 172.21.15.102:0/71130059 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8fc005e00 con 0x7fb908101da0 2024-05-13T22:55:46.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.991+0000 7fb905ffb700 1 -- 172.21.15.102:0/71130059 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8fc017400 con 0x7fb908101da0 2024-05-13T22:55:46.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.991+0000 7fb90ec35700 1 -- 172.21.15.102:0/71130059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb908196850 con 0x7fb908101da0 2024-05-13T22:55:46.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.991+0000 7fb90ec35700 1 -- 172.21.15.102:0/71130059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb908196d40 con 0x7fb908101da0 2024-05-13T22:55:46.971 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.992+0000 7fb90ec35700 1 -- 172.21.15.102:0/71130059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9081082b0 con 0x7fb908101da0 2024-05-13T22:55:46.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.993+0000 7fb905ffb700 1 -- 172.21.15.102:0/71130059 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb8fc00f6f0 con 0x7fb908101da0 2024-05-13T22:55:46.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.993+0000 7fb905ffb700 1 --2- 172.21.15.102:0/71130059 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb8f806e200 0x7fb8f80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:55:46.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.993+0000 7fb905ffb700 1 -- 172.21.15.102:0/71130059 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fb8fc08f6c0 con 0x7fb908101da0 2024-05-13T22:55:46.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.993+0000 7fb907fff700 1 --2- 172.21.15.102:0/71130059 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb8f806e200 0x7fb8f80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:55:46.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.994+0000 7fb907fff700 1 --2- 172.21.15.102:0/71130059 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb8f806e200 0x7fb8f80706c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb9080fcc00 tx=0x7fb8f400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:55:46.974 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:46.998+0000 7fb905ffb700 1 -- 172.21.15.102:0/71130059 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb8fc05c460 con 0x7fb908101da0 2024-05-13T22:55:47.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:47 smithi102 ceph-mon[111694]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:47.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:47 smithi102 ceph-mon[111694]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:47.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:47 smithi102 ceph-mon[111694]: pgmap v119: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 768 B/s rd, 1 op/s 2024-05-13T22:55:47.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:47.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:47 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2083347167' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:47.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:47 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1864119915' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:55:47.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.282+0000 7fb90ec35700 1 -- 172.21.15.102:0/71130059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb90802ccc0 con 0x7fb908101da0 2024-05-13T22:55:47.259 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.283+0000 7fb905ffb700 1 -- 172.21.15.102:0/71130059 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb8fc05bff0 con 0x7fb908101da0 2024-05-13T22:55:47.260 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:55:47.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.285+0000 7fb90ec35700 1 -- 172.21.15.102:0/71130059 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb8f806e200 msgr2=0x7fb8f80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:47.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.285+0000 7fb90ec35700 1 --2- 172.21.15.102:0/71130059 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb8f806e200 0x7fb8f80706c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb9080fcc00 tx=0x7fb8f400b410 comp rx=0 tx=0).stop 2024-05-13T22:55:47.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.285+0000 7fb90ec35700 1 -- 172.21.15.102:0/71130059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb908101da0 msgr2=0x7fb90819c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:55:47.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.285+0000 7fb90ec35700 1 --2- 172.21.15.102:0/71130059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb908101da0 0x7fb90819c550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0092e0 tx=0x7fb8fc00bd70 comp rx=0 tx=0).stop 2024-05-13T22:55:47.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.285+0000 7fb90ec35700 1 -- 172.21.15.102:0/71130059 shutdown_connections 2024-05-13T22:55:47.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.285+0000 7fb90ec35700 1 --2- 172.21.15.102:0/71130059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb908101da0 0x7fb90819c550 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:47.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.285+0000 7fb90ec35700 1 --2- 172.21.15.102:0/71130059 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb8f806e200 0x7fb8f80706c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:47.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.285+0000 7fb90ec35700 1 --2- 172.21.15.102:0/71130059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9081026c0 0x7fb90819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:55:47.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.285+0000 7fb90ec35700 1 -- 172.21.15.102:0/71130059 >> 172.21.15.102:0/71130059 conn(0x7fb9080fb570 msgr2=0x7fb9081053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:55:47.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.285+0000 7fb90ec35700 1 -- 172.21.15.102:0/71130059 shutdown_connections 2024-05-13T22:55:47.262 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:55:47.285+0000 7fb90ec35700 1 -- 172.21.15.102:0/71130059 wait complete. 2024-05-13T22:55:47.389 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:47 smithi130 ceph-mon[84265]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:47.389 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:47 smithi130 ceph-mon[84265]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:47.389 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:47 smithi130 ceph-mon[84265]: pgmap v119: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 768 B/s rd, 1 op/s 2024-05-13T22:55:47.389 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:47.389 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:47 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2083347167' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:55:47.389 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:47 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1864119915' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:55:48.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:48 smithi130 ceph-mon[84265]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:48.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:48 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/71130059' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:55:48.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:48 smithi102 ceph-mon[111694]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:55:48.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:48 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/71130059' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:55:49.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:49 smithi130 ceph-mon[84265]: pgmap v120: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-13T22:55:49.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:49 smithi102 ceph-mon[111694]: pgmap v120: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-13T22:55:51.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:51 smithi130 ceph-mon[84265]: pgmap v121: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:55:51.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:51 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-13T22:55:51.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:51 smithi102 ceph-mon[111694]: pgmap v121: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:55:51.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:51 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-13T22:55:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:52 smithi130 ceph-mon[84265]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-13T22:55:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:52 smithi130 ceph-mon[84265]: Upgrade: osd.3 is safe to restart 2024-05-13T22:55:52.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:52 smithi102 ceph-mon[111694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-13T22:55:52.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:52 smithi102 ceph-mon[111694]: Upgrade: osd.3 is safe to restart 2024-05-13T22:55:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:53 smithi130 ceph-mon[84265]: pgmap v122: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:55:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:53 smithi130 ceph-mon[84265]: Upgrade: Updating osd.3 2024-05-13T22:55:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:53 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:53.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:53 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-13T22:55:53.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:53 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:55:53.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:53 smithi130 ceph-mon[84265]: Deploying daemon osd.3 on smithi130 2024-05-13T22:55:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:53 smithi102 ceph-mon[111694]: pgmap v122: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:55:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:53 smithi102 ceph-mon[111694]: Upgrade: Updating osd.3 2024-05-13T22:55:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:53 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:55:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:53 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-13T22:55:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:53 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:55:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:53 smithi102 ceph-mon[111694]: Deploying daemon osd.3 on smithi130 2024-05-13T22:55:55.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:55 smithi130 ceph-mon[84265]: pgmap v123: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:55:55.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:55 smithi102 ceph-mon[111694]: pgmap v123: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:55:55.720 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:55 smithi130 systemd[1]: Stopping Ceph osd.3 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:55:56.149 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:55 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[42623]: 2024-05-13T22:55:55.777+0000 7fd4e6256700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T22:55:56.149 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:55 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[42623]: 2024-05-13T22:55:55.777+0000 7fd4e6256700 -1 osd.3 76 *** Got signal Terminated *** 2024-05-13T22:55:56.150 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:55 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[42623]: 2024-05-13T22:55:55.777+0000 7fd4e6256700 -1 osd.3 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T22:55:56.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:56 smithi102 ceph-mon[111694]: osd.3 marked itself down and dead 2024-05-13T22:55:56.527 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:56 smithi130 ceph-mon[84265]: osd.3 marked itself down and dead 2024-05-13T22:55:56.797 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:56 smithi130 podman[92989]: 2024-05-13 22:55:56.551611619 +0000 UTC m=+0.899171094 container died 3b510d692e3f1fbcd0d8a681942dc52d158b560c7f1ece6d01952f6b4dac06ef (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=HEAD) 2024-05-13T22:55:57.149 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:56 smithi130 podman[92989]: 2024-05-13 22:55:56.911165368 +0000 UTC m=+1.258724835 container cleanup 3b510d692e3f1fbcd0d8a681942dc52d158b560c7f1ece6d01952f6b4dac06ef (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-05-13T22:55:57.149 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:56 smithi130 bash[92989]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3 2024-05-13T22:55:57.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:57 smithi102 ceph-mon[111694]: pgmap v124: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:55:57.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:57 smithi102 ceph-mon[111694]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:55:57.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:57 smithi102 ceph-mon[111694]: osdmap e77: 6 total, 5 up, 6 in 2024-05-13T22:55:57.483 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:57 smithi130 ceph-mon[84265]: pgmap v124: 65 pgs: 65 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:55:57.483 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:57 smithi130 ceph-mon[84265]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:55:57.483 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:57 smithi130 ceph-mon[84265]: osdmap e77: 6 total, 5 up, 6 in 2024-05-13T22:55:57.483 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:57 smithi130 podman[93017]: 2024-05-13 22:55:57.278997093 +0000 UTC m=+0.726150432 container remove 3b510d692e3f1fbcd0d8a681942dc52d158b560c7f1ece6d01952f6b4dac06ef (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, ceph=True, RELEASE=HEAD, org.label-schema.schema-version=1.0) 2024-05-13T22:55:57.779 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:57 smithi130 podman[93105]: 2024-05-13 22:55:57.58335861 +0000 UTC m=+0.017837112 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:55:57.779 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:57 smithi130 podman[93105]: 2024-05-13 22:55:57.680937719 +0000 UTC m=+0.115416200 container create 0ffc0c0abb808388e94b8565cdb64e665cd18a8ecac6578221bd617c8aef0a49 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-c3e5514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-13T22:55:58.149 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:57 smithi130 podman[93105]: 2024-05-13 22:55:57.892205638 +0000 UTC m=+0.326684141 container init 0ffc0c0abb808388e94b8565cdb64e665cd18a8ecac6578221bd617c8aef0a49 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-deactivate, ceph=True, RELEASE=squid-c3e5514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-13T22:55:58.149 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:57 smithi130 podman[93105]: 2024-05-13 22:55:57.895949565 +0000 UTC m=+0.330428068 container start 0ffc0c0abb808388e94b8565cdb64e665cd18a8ecac6578221bd617c8aef0a49 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-deactivate, RELEASE=squid-c3e5514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-13T22:55:58.149 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:57 smithi130 podman[93105]: 2024-05-13 22:55:57.950148069 +0000 UTC m=+0.384626556 container attach 0ffc0c0abb808388e94b8565cdb64e665cd18a8ecac6578221bd617c8aef0a49 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-13T22:55:58.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:58 smithi102 ceph-mon[111694]: osdmap e78: 6 total, 5 up, 6 in 2024-05-13T22:55:58.479 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:58 smithi130 ceph-mon[84265]: osdmap e78: 6 total, 5 up, 6 in 2024-05-13T22:55:58.479 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:58 smithi130 podman[93105]: 2024-05-13 22:55:58.19297154 +0000 UTC m=+0.627450046 container died 0ffc0c0abb808388e94b8565cdb64e665cd18a8ecac6578221bd617c8aef0a49 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-13T22:55:59.149 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:58 smithi130 podman[93132]: 2024-05-13 22:55:58.779113762 +0000 UTC m=+0.579394645 container remove 0ffc0c0abb808388e94b8565cdb64e665cd18a8ecac6578221bd617c8aef0a49 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=squid-c3e5514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-05-13T22:55:59.149 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:58 smithi130 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.3.service: Deactivated successfully. 2024-05-13T22:55:59.149 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:58 smithi130 systemd[1]: Stopped Ceph osd.3 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:55:59.150 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:58 smithi130 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.3.service: Consumed 1min 19.970s CPU time. 2024-05-13T22:55:59.150 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 systemd[1]: Starting Ceph osd.3 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:55:59.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:59 smithi102 ceph-mon[111694]: pgmap v127: 65 pgs: 21 peering, 5 stale+active+clean, 39 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-13T22:55:59.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:55:59 smithi102 ceph-mon[111694]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-05-13T22:55:59.458 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:59 smithi130 ceph-mon[84265]: pgmap v127: 65 pgs: 21 peering, 5 stale+active+clean, 39 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-13T22:55:59.458 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:55:59 smithi130 ceph-mon[84265]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-05-13T22:55:59.459 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 podman[93231]: 2024-05-13 22:55:59.124404149 +0000 UTC m=+0.017430564 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:55:59.459 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 podman[93231]: 2024-05-13 22:55:59.238660167 +0000 UTC m=+0.131686561 container create 1482427f704c4abd29bf6586483bdde37d8ab7e068c87b2a222261fbc9e998b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-13T22:55:59.459 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 podman[93231]: 2024-05-13 22:55:59.482822668 +0000 UTC m=+0.375849070 container init 1482427f704c4abd29bf6586483bdde37d8ab7e068c87b2a222261fbc9e998b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-c3e5514, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-13T22:55:59.899 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 podman[93231]: 2024-05-13 22:55:59.486642037 +0000 UTC m=+0.379668447 container start 1482427f704c4abd29bf6586483bdde37d8ab7e068c87b2a222261fbc9e998b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True) 2024-05-13T22:55:59.899 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 podman[93231]: 2024-05-13 22:55:59.557806928 +0000 UTC m=+0.450833332 container attach 1482427f704c4abd29bf6586483bdde37d8ab7e068c87b2a222261fbc9e998b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-c3e5514) 2024-05-13T22:56:00.224 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate[93245]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate[93245]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 bash[93231]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 bash[93231]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 bash[93231]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 bash[93231]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 bash[93231]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 bash[93231]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 bash[93231]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate[93245]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate[93245]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate[93245]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-13T22:56:00.225 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate[93245]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-13T22:56:00.226 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate[93245]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-13T22:56:00.226 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:55:59 smithi130 podman[93231]: 2024-05-13 22:55:59.966198354 +0000 UTC m=+0.859224769 container died 1482427f704c4abd29bf6586483bdde37d8ab7e068c87b2a222261fbc9e998b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-13T22:56:00.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:00 smithi102 ceph-mon[111694]: pgmap v128: 65 pgs: 3 active+undersized, 2 active+undersized+degraded, 21 peering, 3 stale+active+clean, 36 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5/231 objects degraded (2.165%) 2024-05-13T22:56:00.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:00 smithi102 ceph-mon[111694]: Health check failed: Degraded data redundancy: 5/231 objects degraded (2.165%), 2 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:00.562 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:00 smithi130 ceph-mon[84265]: pgmap v128: 65 pgs: 3 active+undersized, 2 active+undersized+degraded, 21 peering, 3 stale+active+clean, 36 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5/231 objects degraded (2.165%) 2024-05-13T22:56:00.562 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:00 smithi130 ceph-mon[84265]: Health check failed: Degraded data redundancy: 5/231 objects degraded (2.165%), 2 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:00.899 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:00 smithi130 podman[93366]: 2024-05-13 22:56:00.586692118 +0000 UTC m=+0.613996842 container remove 1482427f704c4abd29bf6586483bdde37d8ab7e068c87b2a222261fbc9e998b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-activate, RELEASE=squid-c3e5514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-05-13T22:56:00.900 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:00 smithi130 podman[93425]: 2024-05-13 22:56:00.703181518 +0000 UTC m=+0.017870027 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:56:00.900 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:00 smithi130 podman[93425]: 2024-05-13 22:56:00.812806835 +0000 UTC m=+0.127495325 container create 66e804d4b801eac156e2356e42f8aad96012b9c00a0e0a87d4fe31d89693f480 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-13T22:56:01.399 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:01 smithi130 podman[93425]: 2024-05-13 22:56:01.073789973 +0000 UTC m=+0.388478456 container init 66e804d4b801eac156e2356e42f8aad96012b9c00a0e0a87d4fe31d89693f480 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-c3e5514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-13T22:56:01.399 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:01 smithi130 podman[93425]: 2024-05-13 22:56:01.076984654 +0000 UTC m=+0.391673148 container start 66e804d4b801eac156e2356e42f8aad96012b9c00a0e0a87d4fe31d89693f480 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-13T22:56:01.399 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:01 smithi130 ceph-osd[93445]: -- 172.21.15.130:0/2442216087 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2621b9680 con 0x55b262182c00 2024-05-13T22:56:01.400 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:01 smithi130 bash[93425]: 66e804d4b801eac156e2356e42f8aad96012b9c00a0e0a87d4fe31d89693f480 2024-05-13T22:56:01.400 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:01 smithi130 systemd[1]: Started Ceph osd.3 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:56:01.899 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:01 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[93441]: 2024-05-13T22:56:01.627+0000 7f55ca90f7c0 -1 Falling back to public interface 2024-05-13T22:56:02.286 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:02.286 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:56:02.286 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:02.286 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:02.286 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:56:02.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:02.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:56:02.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:02.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:02.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:56:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:03 smithi130 ceph-mon[84265]: pgmap v129: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/231 objects degraded (19.481%) 2024-05-13T22:56:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:03 smithi130 ceph-mon[84265]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs peering) 2024-05-13T22:56:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:03 smithi102 ceph-mon[111694]: pgmap v129: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/231 objects degraded (19.481%) 2024-05-13T22:56:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:03 smithi102 ceph-mon[111694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs peering) 2024-05-13T22:56:05.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:05 smithi102 ceph-mon[111694]: pgmap v130: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/231 objects degraded (19.481%) 2024-05-13T22:56:05.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:05 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:05.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:05 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:05.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:05 smithi130 ceph-mon[84265]: pgmap v130: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/231 objects degraded (19.481%) 2024-05-13T22:56:05.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:05 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:05.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:05 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:06.649 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:06 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[93441]: 2024-05-13T22:56:06.480+0000 7f55ca90f7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-05-13T22:56:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:06 smithi130 ceph-mon[84265]: pgmap v131: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/231 objects degraded (19.481%) 2024-05-13T22:56:06.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:06 smithi130 ceph-mon[84265]: Health check update: Degraded data redundancy: 45/231 objects degraded (19.481%), 15 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:06 smithi102 ceph-mon[111694]: pgmap v131: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 212 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/231 objects degraded (19.481%) 2024-05-13T22:56:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:06 smithi102 ceph-mon[111694]: Health check update: Degraded data redundancy: 45/231 objects degraded (19.481%), 15 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:07.236 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:06 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[93441]: 2024-05-13T22:56:06.981+0000 7f55ca90f7c0 -1 osd.3 76 log_to_monitors true 2024-05-13T22:56:07.543 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 22:56:07 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[93441]: 2024-05-13T22:56:07.391+0000 7f55bbd66700 -1 osd.3 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:56:07.543 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:07 smithi130 ceph-mon[84265]: from='osd.3 [v2:172.21.15.130:6800/3670165065,v1:172.21.15.130:6801/3670165065]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-13T22:56:07.543 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:07 smithi130 ceph-mon[84265]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-13T22:56:07.576 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:07 smithi102 ceph-mon[111694]: from='osd.3 [v2:172.21.15.130:6800/3670165065,v1:172.21.15.130:6801/3670165065]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-13T22:56:07.576 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:07 smithi102 ceph-mon[111694]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-13T22:56:08.613 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:08 smithi130 ceph-mon[84265]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-13T22:56:08.614 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:08 smithi130 ceph-mon[84265]: osdmap e79: 6 total, 5 up, 6 in 2024-05-13T22:56:08.614 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:08 smithi130 ceph-mon[84265]: from='osd.3 [v2:172.21.15.130:6800/3670165065,v1:172.21.15.130:6801/3670165065]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:56:08.614 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:08 smithi130 ceph-mon[84265]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:56:08.614 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:08 smithi130 ceph-mon[84265]: pgmap v133: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 212 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/231 objects degraded (19.481%) 2024-05-13T22:56:08.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:08 smithi102 ceph-mon[111694]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-13T22:56:08.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:08 smithi102 ceph-mon[111694]: osdmap e79: 6 total, 5 up, 6 in 2024-05-13T22:56:08.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:08 smithi102 ceph-mon[111694]: from='osd.3 [v2:172.21.15.130:6800/3670165065,v1:172.21.15.130:6801/3670165065]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:56:08.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:08 smithi102 ceph-mon[111694]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:56:08.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:08 smithi102 ceph-mon[111694]: pgmap v133: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 212 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/231 objects degraded (19.481%) 2024-05-13T22:56:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:56:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: osd.3 [v2:172.21.15.130:6800/3670165065,v1:172.21.15.130:6801/3670165065] boot 2024-05-13T22:56:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: osdmap e80: 6 total, 6 up, 6 in 2024-05-13T22:56:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:56:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:56:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:56:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:56:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-13T22:56:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-13T22:56:09.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:09 smithi102 ceph-mon[111694]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-13T22:56:09.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:56:09.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: osd.3 [v2:172.21.15.130:6800/3670165065,v1:172.21.15.130:6801/3670165065] boot 2024-05-13T22:56:09.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: osdmap e80: 6 total, 6 up, 6 in 2024-05-13T22:56:09.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-13T22:56:09.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:09.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:09.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:56:09.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:56:09.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:09.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:56:09.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:09.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:09.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:09.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-13T22:56:09.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-13T22:56:09.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:09 smithi130 ceph-mon[84265]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-13T22:56:10.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:10 smithi130 ceph-mon[84265]: osdmap e81: 6 total, 6 up, 6 in 2024-05-13T22:56:10.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:10 smithi130 ceph-mon[84265]: pgmap v136: 65 pgs: 3 peering, 16 active+undersized, 14 active+undersized+degraded, 32 active+clean; 212 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:56:10.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:10 smithi102 ceph-mon[111694]: osdmap e81: 6 total, 6 up, 6 in 2024-05-13T22:56:10.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:10 smithi102 ceph-mon[111694]: pgmap v136: 65 pgs: 3 peering, 16 active+undersized, 14 active+undersized+degraded, 32 active+clean; 212 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 42/231 objects degraded (18.182%) 2024-05-13T22:56:11.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:11 smithi130 ceph-mon[84265]: Health check update: Degraded data redundancy: 42/231 objects degraded (18.182%), 14 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:11.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:11 smithi102 ceph-mon[111694]: Health check update: Degraded data redundancy: 42/231 objects degraded (18.182%), 14 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:12.620 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:12 smithi130 ceph-mon[84265]: pgmap v137: 65 pgs: 3 peering, 7 active+undersized, 8 active+undersized+degraded, 47 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 23/231 objects degraded (9.957%) 2024-05-13T22:56:12.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:12 smithi102 ceph-mon[111694]: pgmap v137: 65 pgs: 3 peering, 7 active+undersized, 8 active+undersized+degraded, 47 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 23/231 objects degraded (9.957%) 2024-05-13T22:56:14.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:14 smithi130 ceph-mon[84265]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/231 objects degraded (9.957%), 8 pgs degraded) 2024-05-13T22:56:14.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:14 smithi130 ceph-mon[84265]: Cluster is now healthy 2024-05-13T22:56:14.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:14 smithi102 ceph-mon[111694]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/231 objects degraded (9.957%), 8 pgs degraded) 2024-05-13T22:56:14.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:14 smithi102 ceph-mon[111694]: Cluster is now healthy 2024-05-13T22:56:15.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:15 smithi130 ceph-mon[84265]: pgmap v138: 65 pgs: 3 peering, 62 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 611 B/s rd, 1 op/s 2024-05-13T22:56:15.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:15 smithi102 ceph-mon[111694]: pgmap v138: 65 pgs: 3 peering, 62 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 611 B/s rd, 1 op/s 2024-05-13T22:56:17.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.402+0000 7f4b11d66700 1 -- 172.21.15.102:0/1345251838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b0c102610 msgr2=0x7f4b0c10ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:17.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.402+0000 7f4b11d66700 1 --2- 172.21.15.102:0/1345251838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b0c102610 0x7f4b0c10ab00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009910 tx=0x7f4b00009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:17.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.403+0000 7f4b11d66700 1 -- 172.21.15.102:0/1345251838 shutdown_connections 2024-05-13T22:56:17.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.403+0000 7f4b11d66700 1 --2- 172.21.15.102:0/1345251838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b0c102610 0x7f4b0c10ab00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.403+0000 7f4b11d66700 1 --2- 172.21.15.102:0/1345251838 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b0c101c60 0x7f4b0c102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.403+0000 7f4b11d66700 1 -- 172.21.15.102:0/1345251838 >> 172.21.15.102:0/1345251838 conn(0x7f4b0c0fb540 msgr2=0x7f4b0c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:17.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.403+0000 7f4b11d66700 1 -- 172.21.15.102:0/1345251838 shutdown_connections 2024-05-13T22:56:17.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.403+0000 7f4b11d66700 1 -- 172.21.15.102:0/1345251838 wait complete. 2024-05-13T22:56:17.380 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:17 smithi102 ceph-mon[111694]: pgmap v139: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s 2024-05-13T22:56:17.380 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:17 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:17.381 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:17 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:56:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.403+0000 7f4b11d66700 1 Processor -- start 2024-05-13T22:56:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.403+0000 7f4b11d66700 1 -- start start 2024-05-13T22:56:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b11d66700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b0c101c60 0x7f4b0c19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b11d66700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b0c102610 0x7f4b0c19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b11d66700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c19d180 con 0x7f4b0c102610 2024-05-13T22:56:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b11d66700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c196630 con 0x7f4b0c101c60 2024-05-13T22:56:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b0affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b0c102610 0x7f4b0c19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:17.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b0affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b0c102610 0x7f4b0c19caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58724/0 (socket says 172.21.15.102:58724) 2024-05-13T22:56:17.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b0affd700 1 -- 172.21.15.102:0/3570366493 learned_addr learned my addr 172.21.15.102:0/3570366493 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:17.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b0b7fe700 1 --2- 172.21.15.102:0/3570366493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b0c101c60 0x7f4b0c19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b0affd700 1 -- 172.21.15.102:0/3570366493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b0c101c60 msgr2=0x7f4b0c19c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b0affd700 1 --2- 172.21.15.102:0/3570366493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b0c101c60 0x7f4b0c19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b0affd700 1 -- 172.21.15.102:0/3570366493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b000095f0 con 0x7f4b0c102610 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b0affd700 1 --2- 172.21.15.102:0/3570366493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b0c102610 0x7f4b0c19caf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b000092e0 tx=0x7f4b0000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b08ff9700 1 -- 172.21.15.102:0/3570366493 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b0001d070 con 0x7f4b0c102610 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.404+0000 7f4b08ff9700 1 -- 172.21.15.102:0/3570366493 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b0000f460 con 0x7f4b0c102610 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.405+0000 7f4b11d66700 1 -- 172.21.15.102:0/3570366493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0c1968b0 con 0x7f4b0c102610 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.405+0000 7f4b08ff9700 1 -- 172.21.15.102:0/3570366493 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b000175f0 con 0x7f4b0c102610 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.405+0000 7f4b11d66700 1 -- 172.21.15.102:0/3570366493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0c196da0 con 0x7f4b0c102610 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.406+0000 7f4b11d66700 1 -- 172.21.15.102:0/3570366493 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0c108200 con 0x7f4b0c102610 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.406+0000 7f4b08ff9700 1 -- 172.21.15.102:0/3570366493 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4b0000f5d0 con 0x7f4b0c102610 2024-05-13T22:56:17.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.407+0000 7f4b08ff9700 1 --2- 172.21.15.102:0/3570366493 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4af406e200 0x7f4af40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:17.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.407+0000 7f4b08ff9700 1 -- 172.21.15.102:0/3570366493 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f4b0008f6a0 con 0x7f4b0c102610 2024-05-13T22:56:17.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.407+0000 7f4b0b7fe700 1 --2- 172.21.15.102:0/3570366493 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4af406e200 0x7f4af40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:17.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.407+0000 7f4b0b7fe700 1 --2- 172.21.15.102:0/3570366493 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4af406e200 0x7f4af40706c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4afc005950 tx=0x7f4afc005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:17.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.412+0000 7f4b08ff9700 1 -- 172.21.15.102:0/3570366493 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4b0005c4c0 con 0x7f4b0c102610 2024-05-13T22:56:17.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:17 smithi130 ceph-mon[84265]: pgmap v139: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s 2024-05-13T22:56:17.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:17 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:17.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:17 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:56:17.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.627+0000 7f4b11d66700 1 -- 172.21.15.102:0/3570366493 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b0c1970d0 con 0x7f4af406e200 2024-05-13T22:56:17.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.628+0000 7f4b08ff9700 1 -- 172.21.15.102:0/3570366493 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f4b0c1970d0 con 0x7f4af406e200 2024-05-13T22:56:17.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.631+0000 7f4b11d66700 1 -- 172.21.15.102:0/3570366493 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4af406e200 msgr2=0x7f4af40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:17.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.631+0000 7f4b11d66700 1 --2- 172.21.15.102:0/3570366493 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4af406e200 0x7f4af40706c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4afc005950 tx=0x7f4afc005cf0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.631+0000 7f4b11d66700 1 -- 172.21.15.102:0/3570366493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b0c102610 msgr2=0x7f4b0c19caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:17.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.631+0000 7f4b11d66700 1 --2- 172.21.15.102:0/3570366493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b0c102610 0x7f4b0c19caf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b000092e0 tx=0x7f4b0000be20 comp rx=0 tx=0).stop 2024-05-13T22:56:17.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.631+0000 7f4b11d66700 1 -- 172.21.15.102:0/3570366493 shutdown_connections 2024-05-13T22:56:17.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.631+0000 7f4b11d66700 1 --2- 172.21.15.102:0/3570366493 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4af406e200 0x7f4af40706c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.631+0000 7f4b11d66700 1 --2- 172.21.15.102:0/3570366493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b0c101c60 0x7f4b0c19c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.631+0000 7f4b11d66700 1 --2- 172.21.15.102:0/3570366493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b0c102610 0x7f4b0c19caf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.631+0000 7f4b11d66700 1 -- 172.21.15.102:0/3570366493 >> 172.21.15.102:0/3570366493 conn(0x7f4b0c0fb540 msgr2=0x7f4b0c105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:17.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.631+0000 7f4b11d66700 1 -- 172.21.15.102:0/3570366493 shutdown_connections 2024-05-13T22:56:17.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.631+0000 7f4b11d66700 1 -- 172.21.15.102:0/3570366493 wait complete. 2024-05-13T22:56:17.618 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:56:17.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.737+0000 7ff89daa5700 1 -- 172.21.15.102:0/2904666719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff898105a40 msgr2=0x7ff898105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:17.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.737+0000 7ff89daa5700 1 --2- 172.21.15.102:0/2904666719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff898105a40 0x7ff898105e20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff888009910 tx=0x7ff888009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:17.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.737+0000 7ff89daa5700 1 -- 172.21.15.102:0/2904666719 shutdown_connections 2024-05-13T22:56:17.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.737+0000 7ff89daa5700 1 --2- 172.21.15.102:0/2904666719 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8980ffbd0 0x7ff898100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.737+0000 7ff89daa5700 1 --2- 172.21.15.102:0/2904666719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff898105a40 0x7ff898105e20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.715 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.737+0000 7ff89daa5700 1 -- 172.21.15.102:0/2904666719 >> 172.21.15.102:0/2904666719 conn(0x7ff8980755a0 msgr2=0x7ff8980759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.737+0000 7ff89daa5700 1 -- 172.21.15.102:0/2904666719 shutdown_connections 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.737+0000 7ff89daa5700 1 -- 172.21.15.102:0/2904666719 wait complete. 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff89daa5700 1 Processor -- start 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff89daa5700 1 -- start start 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff89daa5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8980ffbd0 0x7ff898197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff89daa5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff898105a40 0x7ff898197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff89daa5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8981980f0 con 0x7ff8980ffbd0 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff89daa5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff898198260 con 0x7ff898105a40 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff8977fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8980ffbd0 0x7ff898197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff8977fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8980ffbd0 0x7ff898197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58742/0 (socket says 172.21.15.102:58742) 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff8977fe700 1 -- 172.21.15.102:0/2203228671 learned_addr learned my addr 172.21.15.102:0/2203228671 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff88ffff700 1 --2- 172.21.15.102:0/2203228671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff898105a40 0x7ff898197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:17.716 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff8977fe700 1 -- 172.21.15.102:0/2203228671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff898105a40 msgr2=0x7ff898197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:17.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff8977fe700 1 --2- 172.21.15.102:0/2203228671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff898105a40 0x7ff898197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.738+0000 7ff8977fe700 1 -- 172.21.15.102:0/2203228671 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8880095f0 con 0x7ff8980ffbd0 2024-05-13T22:56:17.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.739+0000 7ff8977fe700 1 --2- 172.21.15.102:0/2203228671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8980ffbd0 0x7ff898197590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff8880092e0 tx=0x7ff88800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:17.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.739+0000 7ff8957fa700 1 -- 172.21.15.102:0/2203228671 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff88801d070 con 0x7ff8980ffbd0 2024-05-13T22:56:17.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.739+0000 7ff89daa5700 1 -- 172.21.15.102:0/2203228671 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8981a0600 con 0x7ff8980ffbd0 2024-05-13T22:56:17.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.739+0000 7ff8957fa700 1 -- 172.21.15.102:0/2203228671 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff888005e00 con 0x7ff8980ffbd0 2024-05-13T22:56:17.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.739+0000 7ff8957fa700 1 -- 172.21.15.102:0/2203228671 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff888017400 con 0x7ff8980ffbd0 2024-05-13T22:56:17.717 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.739+0000 7ff89daa5700 1 -- 172.21.15.102:0/2203228671 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8981a0a70 con 0x7ff8980ffbd0 2024-05-13T22:56:17.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.740+0000 7ff89daa5700 1 -- 172.21.15.102:0/2203228671 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff898108040 con 0x7ff8980ffbd0 2024-05-13T22:56:17.718 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.740+0000 7ff8957fa700 1 -- 172.21.15.102:0/2203228671 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff88800f460 con 0x7ff8980ffbd0 2024-05-13T22:56:17.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.741+0000 7ff8957fa700 1 --2- 172.21.15.102:0/2203228671 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff88406e200 0x7ff8840706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:17.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.741+0000 7ff8957fa700 1 -- 172.21.15.102:0/2203228671 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7ff88808f720 con 0x7ff8980ffbd0 2024-05-13T22:56:17.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.741+0000 7ff88ffff700 1 --2- 172.21.15.102:0/2203228671 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff88406e200 0x7ff8840706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:17.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.741+0000 7ff88ffff700 1 --2- 172.21.15.102:0/2203228671 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff88406e200 0x7ff8840706c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff898068ce0 tx=0x7ff88000b2c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:17.722 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.746+0000 7ff8957fa700 1 -- 172.21.15.102:0/2203228671 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff888058990 con 0x7ff8980ffbd0 2024-05-13T22:56:17.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.966+0000 7ff89daa5700 1 -- 172.21.15.102:0/2203228671 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8980611d0 con 0x7ff88406e200 2024-05-13T22:56:17.944 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.968+0000 7ff8957fa700 1 -- 172.21.15.102:0/2203228671 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ff8980611d0 con 0x7ff88406e200 2024-05-13T22:56:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.970+0000 7ff89daa5700 1 -- 172.21.15.102:0/2203228671 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff88406e200 msgr2=0x7ff8840706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.970+0000 7ff89daa5700 1 --2- 172.21.15.102:0/2203228671 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff88406e200 0x7ff8840706c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff898068ce0 tx=0x7ff88000b2c0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.970+0000 7ff89daa5700 1 -- 172.21.15.102:0/2203228671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8980ffbd0 msgr2=0x7ff898197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.970+0000 7ff89daa5700 1 --2- 172.21.15.102:0/2203228671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8980ffbd0 0x7ff898197590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff8880092e0 tx=0x7ff88800be20 comp rx=0 tx=0).stop 2024-05-13T22:56:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.970+0000 7ff89daa5700 1 -- 172.21.15.102:0/2203228671 shutdown_connections 2024-05-13T22:56:17.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.970+0000 7ff89daa5700 1 --2- 172.21.15.102:0/2203228671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8980ffbd0 0x7ff898197590 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.970+0000 7ff89daa5700 1 --2- 172.21.15.102:0/2203228671 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff88406e200 0x7ff8840706c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.970+0000 7ff89daa5700 1 --2- 172.21.15.102:0/2203228671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff898105a40 0x7ff898197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.970+0000 7ff89daa5700 1 -- 172.21.15.102:0/2203228671 >> 172.21.15.102:0/2203228671 conn(0x7ff8980755a0 msgr2=0x7ff8980fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.970+0000 7ff89daa5700 1 -- 172.21.15.102:0/2203228671 shutdown_connections 2024-05-13T22:56:17.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:17.970+0000 7ff89daa5700 1 -- 172.21.15.102:0/2203228671 wait complete. 2024-05-13T22:56:18.051 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.075+0000 7fedebcbc700 1 -- 172.21.15.102:0/827576106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede4068670 msgr2=0x7fede4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:18.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.075+0000 7fedebcbc700 1 --2- 172.21.15.102:0/827576106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede4068670 0x7fede4068a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fede0009910 tx=0x7fede0009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:18.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.075+0000 7fedebcbc700 1 -- 172.21.15.102:0/827576106 shutdown_connections 2024-05-13T22:56:18.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.075+0000 7fedebcbc700 1 --2- 172.21.15.102:0/827576106 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fede4069020 0x7fede4105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.075+0000 7fedebcbc700 1 --2- 172.21.15.102:0/827576106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede4068670 0x7fede4068a50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.075+0000 7fedebcbc700 1 -- 172.21.15.102:0/827576106 >> 172.21.15.102:0/827576106 conn(0x7fede40755a0 msgr2=0x7fede40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:18.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.075+0000 7fedebcbc700 1 -- 172.21.15.102:0/827576106 shutdown_connections 2024-05-13T22:56:18.052 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.075+0000 7fedebcbc700 1 -- 172.21.15.102:0/827576106 wait complete. 2024-05-13T22:56:18.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fedebcbc700 1 Processor -- start 2024-05-13T22:56:18.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fedebcbc700 1 -- start start 2024-05-13T22:56:18.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fedebcbc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede4068670 0x7fede419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:18.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fedebcbc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fede4069020 0x7fede419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:18.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fedebcbc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede419f2e0 con 0x7fede4068670 2024-05-13T22:56:18.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fedebcbc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede4198790 con 0x7fede4069020 2024-05-13T22:56:18.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fede9a58700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede4068670 0x7fede419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:18.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fede9a58700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede4068670 0x7fede419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58750/0 (socket says 172.21.15.102:58750) 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fede9a58700 1 -- 172.21.15.102:0/3610798414 learned_addr learned my addr 172.21.15.102:0/3610798414 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fede9257700 1 --2- 172.21.15.102:0/3610798414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fede4069020 0x7fede419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fede9a58700 1 -- 172.21.15.102:0/3610798414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fede4069020 msgr2=0x7fede419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fede9a58700 1 --2- 172.21.15.102:0/3610798414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fede4069020 0x7fede419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.076+0000 7fede9a58700 1 -- 172.21.15.102:0/3610798414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede00095f0 con 0x7fede4068670 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.077+0000 7fede9a58700 1 --2- 172.21.15.102:0/3610798414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede4068670 0x7fede419e710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fede0006010 tx=0x7fede000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.077+0000 7feddaffd700 1 -- 172.21.15.102:0/3610798414 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede001d070 con 0x7fede4068670 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.077+0000 7fedebcbc700 1 -- 172.21.15.102:0/3610798414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede4198a10 con 0x7fede4068670 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.077+0000 7feddaffd700 1 -- 172.21.15.102:0/3610798414 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fede000f460 con 0x7fede4068670 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.077+0000 7feddaffd700 1 -- 172.21.15.102:0/3610798414 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede0017630 con 0x7fede4068670 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.077+0000 7fedebcbc700 1 -- 172.21.15.102:0/3610798414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede4198f00 con 0x7fede4068670 2024-05-13T22:56:18.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.078+0000 7fedebcbc700 1 -- 172.21.15.102:0/3610798414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede404ea90 con 0x7fede4068670 2024-05-13T22:56:18.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.078+0000 7feddaffd700 1 -- 172.21.15.102:0/3610798414 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fede000f5d0 con 0x7fede4068670 2024-05-13T22:56:18.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.079+0000 7feddaffd700 1 --2- 172.21.15.102:0/3610798414 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedd006e200 0x7fedd00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:18.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.079+0000 7feddaffd700 1 -- 172.21.15.102:0/3610798414 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fede008f6b0 con 0x7fede4068670 2024-05-13T22:56:18.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.079+0000 7fede9257700 1 --2- 172.21.15.102:0/3610798414 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedd006e200 0x7fedd00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:18.056 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.079+0000 7fede9257700 1 --2- 172.21.15.102:0/3610798414 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedd006e200 0x7fedd00706c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fedd4005950 tx=0x7fedd40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:18.060 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.084+0000 7feddaffd700 1 -- 172.21.15.102:0/3610798414 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fede005c450 con 0x7fede4068670 2024-05-13T22:56:18.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.291+0000 7fedebcbc700 1 -- 172.21.15.102:0/3610798414 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fede41992f0 con 0x7fedd006e200 2024-05-13T22:56:18.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.301+0000 7feddaffd700 1 -- 172.21.15.102:0/3610798414 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fede41992f0 con 0x7fedd006e200 2024-05-13T22:56:18.278 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:56:18.278 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (5m) 47s ago 16m 26.7M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:56:18.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (16m) 47s ago 16m 8740k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:56:18.278 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (15m) 14s ago 15m 8124k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:56:18.278 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (2m) 47s ago 16m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T22:56:18.278 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (2m) 14s ago 15m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T22:56:18.278 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (5m) 47s ago 16m 79.9M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:56:18.278 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (11m) 47s ago 11m 88.3M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:56:18.278 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (10m) 47s ago 10m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:56:18.278 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (11m) 14s ago 11m 17.1M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:56:18.278 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (10m) 14s ago 10m 20.0M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:56:18.279 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (7m) 47s ago 17m 506M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:56:18.279 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (6m) 14s ago 15m 452M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:56:18.279 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (3m) 47s ago 18m 51.7M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:56:18.279 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (3m) 14s ago 15m 42.4M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:56:18.279 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (6m) 47s ago 16m 16.0M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:56:18.279 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (6m) 14s ago 15m 17.0M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:56:18.279 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (2m) 47s ago 14m 133M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 0ada88c777a2 2024-05-13T22:56:18.279 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (89s) 47s ago 14m 130M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 c3e9e68b06fd 2024-05-13T22:56:18.279 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (53s) 47s ago 13m 15.1M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 1b85728dccf5 2024-05-13T22:56:18.279 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (17s) 14s ago 13m 12.7M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 66e804d4b801 2024-05-13T22:56:18.279 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (12m) 14s ago 12m 248M 2240M 18.2.1 5be31c24972a 5af6d8af2b7c 2024-05-13T22:56:18.282 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (12m) 14s ago 12m 282M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:56:18.282 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (5m) 47s ago 16m 73.7M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:56:18.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.303+0000 7fedebcbc700 1 -- 172.21.15.102:0/3610798414 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedd006e200 msgr2=0x7fedd00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:18.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.303+0000 7fedebcbc700 1 --2- 172.21.15.102:0/3610798414 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedd006e200 0x7fedd00706c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fedd4005950 tx=0x7fedd40058e0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.303+0000 7fedebcbc700 1 -- 172.21.15.102:0/3610798414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede4068670 msgr2=0x7fede419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:18.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.303+0000 7fedebcbc700 1 --2- 172.21.15.102:0/3610798414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede4068670 0x7fede419e710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fede0006010 tx=0x7fede000be20 comp rx=0 tx=0).stop 2024-05-13T22:56:18.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.303+0000 7fedebcbc700 1 -- 172.21.15.102:0/3610798414 shutdown_connections 2024-05-13T22:56:18.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.303+0000 7fedebcbc700 1 --2- 172.21.15.102:0/3610798414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fede4068670 0x7fede419e710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.303+0000 7fedebcbc700 1 --2- 172.21.15.102:0/3610798414 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fedd006e200 0x7fedd00706c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.303+0000 7fedebcbc700 1 --2- 172.21.15.102:0/3610798414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fede4069020 0x7fede419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.303+0000 7fedebcbc700 1 -- 172.21.15.102:0/3610798414 >> 172.21.15.102:0/3610798414 conn(0x7fede40755a0 msgr2=0x7fede4102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:18.283 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.303+0000 7fedebcbc700 1 -- 172.21.15.102:0/3610798414 shutdown_connections 2024-05-13T22:56:18.284 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.303+0000 7fedebcbc700 1 -- 172.21.15.102:0/3610798414 wait complete. 2024-05-13T22:56:18.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.406+0000 7fadcebfe700 1 -- 172.21.15.102:0/3986279634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc8105a40 msgr2=0x7fadc8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:18.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.406+0000 7fadcebfe700 1 --2- 172.21.15.102:0/3986279634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc8105a40 0x7fadc8105e20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009910 tx=0x7fadc0009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:18.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.406+0000 7fadcebfe700 1 -- 172.21.15.102:0/3986279634 shutdown_connections 2024-05-13T22:56:18.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.406+0000 7fadcebfe700 1 --2- 172.21.15.102:0/3986279634 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc80ffbd0 0x7fadc8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.406+0000 7fadcebfe700 1 --2- 172.21.15.102:0/3986279634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc8105a40 0x7fadc8105e20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.406+0000 7fadcebfe700 1 -- 172.21.15.102:0/3986279634 >> 172.21.15.102:0/3986279634 conn(0x7fadc80755a0 msgr2=0x7fadc80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:18.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.406+0000 7fadcebfe700 1 -- 172.21.15.102:0/3986279634 shutdown_connections 2024-05-13T22:56:18.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.406+0000 7fadcebfe700 1 -- 172.21.15.102:0/3986279634 wait complete. 2024-05-13T22:56:18.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.407+0000 7fadcebfe700 1 Processor -- start 2024-05-13T22:56:18.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.407+0000 7fadcebfe700 1 -- start start 2024-05-13T22:56:18.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.407+0000 7fadcebfe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc80ffbd0 0x7fadc8197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:18.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.407+0000 7fadcebfe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc8105a40 0x7fadc8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:18.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.407+0000 7fadcebfe700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc81980f0 con 0x7fadc80ffbd0 2024-05-13T22:56:18.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.407+0000 7fadcebfe700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8198260 con 0x7fadc8105a40 2024-05-13T22:56:18.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.407+0000 7fadcc99a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc80ffbd0 0x7fadc8197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:18.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.407+0000 7fadcc99a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc80ffbd0 0x7fadc8197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58776/0 (socket says 172.21.15.102:58776) 2024-05-13T22:56:18.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.407+0000 7fadcc99a700 1 -- 172.21.15.102:0/1502877146 learned_addr learned my addr 172.21.15.102:0/1502877146 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:18.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.407+0000 7fadc7fff700 1 --2- 172.21.15.102:0/1502877146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc8105a40 0x7fadc8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:18.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.408+0000 7fadcc99a700 1 -- 172.21.15.102:0/1502877146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc8105a40 msgr2=0x7fadc8197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:18.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.408+0000 7fadcc99a700 1 --2- 172.21.15.102:0/1502877146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc8105a40 0x7fadc8197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.408+0000 7fadcc99a700 1 -- 172.21.15.102:0/1502877146 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00095f0 con 0x7fadc80ffbd0 2024-05-13T22:56:18.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.408+0000 7fadcc99a700 1 --2- 172.21.15.102:0/1502877146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc80ffbd0 0x7fadc8197590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fadc0006010 tx=0x7fadc000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:18.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.408+0000 7fadc5ffb700 1 -- 172.21.15.102:0/1502877146 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc001d070 con 0x7fadc80ffbd0 2024-05-13T22:56:18.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.408+0000 7fadc5ffb700 1 -- 172.21.15.102:0/1502877146 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadc0004fb0 con 0x7fadc80ffbd0 2024-05-13T22:56:18.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.408+0000 7fadcebfe700 1 -- 172.21.15.102:0/1502877146 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc81a0600 con 0x7fadc80ffbd0 2024-05-13T22:56:18.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.408+0000 7fadc5ffb700 1 -- 172.21.15.102:0/1502877146 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc0022ea0 con 0x7fadc80ffbd0 2024-05-13T22:56:18.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.408+0000 7fadcebfe700 1 -- 172.21.15.102:0/1502877146 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc81a0a70 con 0x7fadc80ffbd0 2024-05-13T22:56:18.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.409+0000 7fadcebfe700 1 -- 172.21.15.102:0/1502877146 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc8108040 con 0x7fadc80ffbd0 2024-05-13T22:56:18.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.410+0000 7fadc5ffb700 1 -- 172.21.15.102:0/1502877146 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fadc0022470 con 0x7fadc80ffbd0 2024-05-13T22:56:18.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.411+0000 7fadc5ffb700 1 --2- 172.21.15.102:0/1502877146 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fadb406e200 0x7fadb40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:18.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.411+0000 7fadc5ffb700 1 -- 172.21.15.102:0/1502877146 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fadc008fb30 con 0x7fadc80ffbd0 2024-05-13T22:56:18.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.411+0000 7fadc7fff700 1 --2- 172.21.15.102:0/1502877146 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fadb406e200 0x7fadb40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:18.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.412+0000 7fadc7fff700 1 --2- 172.21.15.102:0/1502877146 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fadb406e200 0x7fadb40706c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fadc8068ce0 tx=0x7fadb80077f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:18.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.415+0000 7fadc5ffb700 1 -- 172.21.15.102:0/1502877146 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fadc0058f90 con 0x7fadc80ffbd0 2024-05-13T22:56:18.671 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.695+0000 7fadcebfe700 1 -- 172.21.15.102:0/1502877146 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fadc804ea90 con 0x7fadc80ffbd0 2024-05-13T22:56:18.672 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.695+0000 7fadc5ffb700 1 -- 172.21.15.102:0/1502877146 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fadc0022720 con 0x7fadc80ffbd0 2024-05-13T22:56:18.672 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:56:18.672 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:56:18.672 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:56:18.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:56:18.672 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:56:18.672 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:56:18.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:56:18.672 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:56:18.672 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-05-13T22:56:18.672 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 4 2024-05-13T22:56:18.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:56:18.673 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:56:18.673 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:56:18.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:56:18.673 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:56:18.673 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-05-13T22:56:18.673 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 8 2024-05-13T22:56:18.674 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:56:18.674 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:56:18.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.697+0000 7fadcebfe700 1 -- 172.21.15.102:0/1502877146 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fadb406e200 msgr2=0x7fadb40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:18.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.697+0000 7fadcebfe700 1 --2- 172.21.15.102:0/1502877146 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fadb406e200 0x7fadb40706c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fadc8068ce0 tx=0x7fadb80077f0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.697+0000 7fadcebfe700 1 -- 172.21.15.102:0/1502877146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc80ffbd0 msgr2=0x7fadc8197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:18.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.697+0000 7fadcebfe700 1 --2- 172.21.15.102:0/1502877146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc80ffbd0 0x7fadc8197590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fadc0006010 tx=0x7fadc000be20 comp rx=0 tx=0).stop 2024-05-13T22:56:18.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.697+0000 7fadcebfe700 1 -- 172.21.15.102:0/1502877146 shutdown_connections 2024-05-13T22:56:18.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.697+0000 7fadcebfe700 1 --2- 172.21.15.102:0/1502877146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc80ffbd0 0x7fadc8197590 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.697+0000 7fadcebfe700 1 --2- 172.21.15.102:0/1502877146 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fadb406e200 0x7fadb40706c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.697+0000 7fadcebfe700 1 --2- 172.21.15.102:0/1502877146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fadc8105a40 0x7fadc8197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.697+0000 7fadcebfe700 1 -- 172.21.15.102:0/1502877146 >> 172.21.15.102:0/1502877146 conn(0x7fadc80755a0 msgr2=0x7fadc80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:18.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.698+0000 7fadcebfe700 1 -- 172.21.15.102:0/1502877146 shutdown_connections 2024-05-13T22:56:18.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.698+0000 7fadcebfe700 1 -- 172.21.15.102:0/1502877146 wait complete. 2024-05-13T22:56:18.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.803+0000 7fc7991c5700 1 -- 172.21.15.102:0/4021377949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc794105a60 msgr2=0x7fc794105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:18.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.803+0000 7fc7991c5700 1 --2- 172.21.15.102:0/4021377949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc794105a60 0x7fc794105e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc784009910 tx=0x7fc784009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:18.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.803+0000 7fc7991c5700 1 -- 172.21.15.102:0/4021377949 shutdown_connections 2024-05-13T22:56:18.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.803+0000 7fc7991c5700 1 --2- 172.21.15.102:0/4021377949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7940ffa60 0x7fc7940ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.803+0000 7fc7991c5700 1 --2- 172.21.15.102:0/4021377949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc794105a60 0x7fc794105e40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.803+0000 7fc7991c5700 1 -- 172.21.15.102:0/4021377949 >> 172.21.15.102:0/4021377949 conn(0x7fc7940fb550 msgr2=0x7fc7940fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:18.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.803+0000 7fc7991c5700 1 -- 172.21.15.102:0/4021377949 shutdown_connections 2024-05-13T22:56:18.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.803+0000 7fc7991c5700 1 -- 172.21.15.102:0/4021377949 wait complete. 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.804+0000 7fc7991c5700 1 Processor -- start 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.804+0000 7fc7991c5700 1 -- start start 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.804+0000 7fc7991c5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7940ffa60 0x7fc794197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.804+0000 7fc7991c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc794105a60 0x7fc794197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.804+0000 7fc7991c5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7941981a0 con 0x7fc794105a60 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.804+0000 7fc7991c5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794198310 con 0x7fc7940ffa60 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.804+0000 7fc78bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc794105a60 0x7fc794197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.804+0000 7fc78bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc794105a60 0x7fc794197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57986/0 (socket says 172.21.15.102:57986) 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.804+0000 7fc78bfff700 1 -- 172.21.15.102:0/1609698914 learned_addr learned my addr 172.21.15.102:0/1609698914 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.805+0000 7fc78bfff700 1 -- 172.21.15.102:0/1609698914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7940ffa60 msgr2=0x7fc794197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.805+0000 7fc792d9d700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7940ffa60 0x7fc794197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:18.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.805+0000 7fc78bfff700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7940ffa60 0x7fc794197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:18.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.805+0000 7fc78bfff700 1 -- 172.21.15.102:0/1609698914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7840095f0 con 0x7fc794105a60 2024-05-13T22:56:18.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.805+0000 7fc792d9d700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7940ffa60 0x7fc794197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:56:18.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.805+0000 7fc78bfff700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc794105a60 0x7fc794197ac0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00d8d0 tx=0x7fc77c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:18.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.805+0000 7fc790d99700 1 -- 172.21.15.102:0/1609698914 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc77c009940 con 0x7fc794105a60 2024-05-13T22:56:18.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.805+0000 7fc7991c5700 1 -- 172.21.15.102:0/1609698914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7941a05f0 con 0x7fc794105a60 2024-05-13T22:56:18.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.805+0000 7fc790d99700 1 -- 172.21.15.102:0/1609698914 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc77c010460 con 0x7fc794105a60 2024-05-13T22:56:18.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.805+0000 7fc790d99700 1 -- 172.21.15.102:0/1609698914 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc77c00f5f0 con 0x7fc794105a60 2024-05-13T22:56:18.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.805+0000 7fc7991c5700 1 -- 172.21.15.102:0/1609698914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7941a0b40 con 0x7fc794105a60 2024-05-13T22:56:18.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.806+0000 7fc7991c5700 1 -- 172.21.15.102:0/1609698914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc794108060 con 0x7fc794105a60 2024-05-13T22:56:18.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.807+0000 7fc790d99700 1 -- 172.21.15.102:0/1609698914 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc77c0105d0 con 0x7fc794105a60 2024-05-13T22:56:18.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.807+0000 7fc790d99700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc78006e200 0x7fc7800706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:18.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.807+0000 7fc790d99700 1 -- 172.21.15.102:0/1609698914 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fc77c08e210 con 0x7fc794105a60 2024-05-13T22:56:18.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.807+0000 7fc792d9d700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc78006e200 0x7fc7800706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:18.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.807+0000 7fc792d9d700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc78006e200 0x7fc7800706c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc78400b5c0 tx=0x7fc7840053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:18.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:18.812+0000 7fc790d99700 1 -- 172.21.15.102:0/1609698914 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc77c057480 con 0x7fc794105a60 2024-05-13T22:56:19.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.055+0000 7fc7991c5700 1 -- 172.21.15.102:0/1609698914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc79404ea90 con 0x7fc794105a60 2024-05-13T22:56:19.031 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.055+0000 7fc790d99700 1 -- 172.21.15.102:0/1609698914 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1909 (secure 0 0 0) 0x7fc77c05ab40 con 0x7fc794105a60 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:e17 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 17 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:56:19.032 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:56:19.033 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:56:19.033 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:56:19.033 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:56:19.033 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:56:19.033 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:56:19.033 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:56:19.033 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 47 2024-05-13T22:56:19.033 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:56:19.033 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:56:19.033 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:56:19.034 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=14498} 2024-05-13T22:56:19.034 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:56:19.034 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:56:19.034 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:56:19.034 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:56:19.034 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:56:19.034 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:56:19.035 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:56:19.035 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:56:19.035 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:56:19.036 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 0 members: 2024-05-13T22:56:19.036 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:56:19.036 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:56:19.036 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:56:19.036 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:56:19.037 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:56:19.037 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:56:19.037 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:56:19.037 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:56:19.039 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.058+0000 7fc7991c5700 1 -- 172.21.15.102:0/1609698914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc78006e200 msgr2=0x7fc7800706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:19.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.058+0000 7fc7991c5700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc78006e200 0x7fc7800706c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc78400b5c0 tx=0x7fc7840053a0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.058+0000 7fc7991c5700 1 -- 172.21.15.102:0/1609698914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc794105a60 msgr2=0x7fc794197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:19.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.058+0000 7fc7991c5700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc794105a60 0x7fc794197ac0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00d8d0 tx=0x7fc77c00dc90 comp rx=0 tx=0).stop 2024-05-13T22:56:19.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.058+0000 7fc7991c5700 1 -- 172.21.15.102:0/1609698914 shutdown_connections 2024-05-13T22:56:19.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.058+0000 7fc7991c5700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc78006e200 0x7fc7800706c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.058+0000 7fc7991c5700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7940ffa60 0x7fc794197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.058+0000 7fc7991c5700 1 --2- 172.21.15.102:0/1609698914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc794105a60 0x7fc794197ac0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.058+0000 7fc7991c5700 1 -- 172.21.15.102:0/1609698914 >> 172.21.15.102:0/1609698914 conn(0x7fc7940fb550 msgr2=0x7fc7940fbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:19.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.058+0000 7fc7991c5700 1 -- 172.21.15.102:0/1609698914 shutdown_connections 2024-05-13T22:56:19.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.058+0000 7fc7991c5700 1 -- 172.21.15.102:0/1609698914 wait complete. 2024-05-13T22:56:19.043 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:56:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.165+0000 7fba82e8c700 1 -- 172.21.15.102:0/1967407054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba7c100de0 msgr2=0x7fba7c1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.165+0000 7fba82e8c700 1 --2- 172.21.15.102:0/1967407054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba7c100de0 0x7fba7c1011c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba70009910 tx=0x7fba70009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.165+0000 7fba82e8c700 1 -- 172.21.15.102:0/1967407054 shutdown_connections 2024-05-13T22:56:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.165+0000 7fba82e8c700 1 --2- 172.21.15.102:0/1967407054 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba7c101790 0x7fba7c1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.165+0000 7fba82e8c700 1 --2- 172.21.15.102:0/1967407054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba7c100de0 0x7fba7c1011c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.165+0000 7fba82e8c700 1 -- 172.21.15.102:0/1967407054 >> 172.21.15.102:0/1967407054 conn(0x7fba7c0fc670 msgr2=0x7fba7c0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.165+0000 7fba82e8c700 1 -- 172.21.15.102:0/1967407054 shutdown_connections 2024-05-13T22:56:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.165+0000 7fba82e8c700 1 -- 172.21.15.102:0/1967407054 wait complete. 2024-05-13T22:56:19.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.165+0000 7fba82e8c700 1 Processor -- start 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.165+0000 7fba82e8c700 1 -- start start 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba82e8c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba7c100de0 0x7fba7c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba82e8c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba7c101790 0x7fba7c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba82e8c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c19d1f0 con 0x7fba7c101790 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba82e8c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1966a0 con 0x7fba7c100de0 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba7bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba7c101790 0x7fba7c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba7bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba7c101790 0x7fba7c19cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58016/0 (socket says 172.21.15.102:58016) 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba7bfff700 1 -- 172.21.15.102:0/222440546 learned_addr learned my addr 172.21.15.102:0/222440546 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba80c28700 1 --2- 172.21.15.102:0/222440546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba7c100de0 0x7fba7c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba7bfff700 1 -- 172.21.15.102:0/222440546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba7c100de0 msgr2=0x7fba7c19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba7bfff700 1 --2- 172.21.15.102:0/222440546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba7c100de0 0x7fba7c19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba7bfff700 1 -- 172.21.15.102:0/222440546 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba700095f0 con 0x7fba7c101790 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba7bfff700 1 --2- 172.21.15.102:0/222440546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba7c101790 0x7fba7c19cb60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fba6800b480 tx=0x7fba6800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:19.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba79ffb700 1 -- 172.21.15.102:0/222440546 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba68011800 con 0x7fba7c101790 2024-05-13T22:56:19.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba82e8c700 1 -- 172.21.15.102:0/222440546 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c196980 con 0x7fba7c101790 2024-05-13T22:56:19.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba79ffb700 1 -- 172.21.15.102:0/222440546 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba68011e20 con 0x7fba7c101790 2024-05-13T22:56:19.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.166+0000 7fba79ffb700 1 -- 172.21.15.102:0/222440546 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba68010ea0 con 0x7fba7c101790 2024-05-13T22:56:19.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.167+0000 7fba82e8c700 1 -- 172.21.15.102:0/222440546 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c196ed0 con 0x7fba7c101790 2024-05-13T22:56:19.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.167+0000 7fba82e8c700 1 -- 172.21.15.102:0/222440546 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7c109180 con 0x7fba7c101790 2024-05-13T22:56:19.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.168+0000 7fba79ffb700 1 -- 172.21.15.102:0/222440546 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fba68010380 con 0x7fba7c101790 2024-05-13T22:56:19.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.168+0000 7fba79ffb700 1 --2- 172.21.15.102:0/222440546 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fba6c06e200 0x7fba6c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:19.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.168+0000 7fba79ffb700 1 -- 172.21.15.102:0/222440546 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fba6808dd00 con 0x7fba7c101790 2024-05-13T22:56:19.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.168+0000 7fba80c28700 1 --2- 172.21.15.102:0/222440546 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fba6c06e200 0x7fba6c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:19.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.169+0000 7fba80c28700 1 --2- 172.21.15.102:0/222440546 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fba6c06e200 0x7fba6c0706c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba70005f50 tx=0x7fba7001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:19.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.173+0000 7fba79ffb700 1 -- 172.21.15.102:0/222440546 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fba6805aaa0 con 0x7fba7c101790 2024-05-13T22:56:19.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.387+0000 7fba82e8c700 1 -- 172.21.15.102:0/222440546 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fba7c1971c0 con 0x7fba6c06e200 2024-05-13T22:56:19.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:19 smithi102 ceph-mon[111694]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:19.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:19 smithi102 ceph-mon[111694]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:19.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:19 smithi102 ceph-mon[111694]: pgmap v140: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 950 B/s rd, 1 op/s 2024-05-13T22:56:19.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:19 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1502877146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:19.365 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:19 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1609698914' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:56:19.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.388+0000 7fba79ffb700 1 -- 172.21.15.102:0/222440546 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fba7c1971c0 con 0x7fba6c06e200 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr", 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout: "mon" 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "10/23 daemons upgraded", 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading osd daemons", 2024-05-13T22:56:19.366 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:56:19.367 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:56:19.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.391+0000 7fba82e8c700 1 -- 172.21.15.102:0/222440546 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fba6c06e200 msgr2=0x7fba6c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:19.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.391+0000 7fba82e8c700 1 --2- 172.21.15.102:0/222440546 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fba6c06e200 0x7fba6c0706c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba70005f50 tx=0x7fba7001a040 comp rx=0 tx=0).stop 2024-05-13T22:56:19.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.391+0000 7fba82e8c700 1 -- 172.21.15.102:0/222440546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba7c101790 msgr2=0x7fba7c19cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:19.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.391+0000 7fba82e8c700 1 --2- 172.21.15.102:0/222440546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba7c101790 0x7fba7c19cb60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fba6800b480 tx=0x7fba6800b790 comp rx=0 tx=0).stop 2024-05-13T22:56:19.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.391+0000 7fba82e8c700 1 -- 172.21.15.102:0/222440546 shutdown_connections 2024-05-13T22:56:19.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.391+0000 7fba82e8c700 1 --2- 172.21.15.102:0/222440546 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fba6c06e200 0x7fba6c0706c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.391+0000 7fba82e8c700 1 --2- 172.21.15.102:0/222440546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba7c100de0 0x7fba7c19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.391+0000 7fba82e8c700 1 --2- 172.21.15.102:0/222440546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba7c101790 0x7fba7c19cb60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.391+0000 7fba82e8c700 1 -- 172.21.15.102:0/222440546 >> 172.21.15.102:0/222440546 conn(0x7fba7c0fc670 msgr2=0x7fba7c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:19.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.391+0000 7fba82e8c700 1 -- 172.21.15.102:0/222440546 shutdown_connections 2024-05-13T22:56:19.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.391+0000 7fba82e8c700 1 -- 172.21.15.102:0/222440546 wait complete. 2024-05-13T22:56:19.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:19 smithi130 ceph-mon[84265]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:19.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:19 smithi130 ceph-mon[84265]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:19.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:19 smithi130 ceph-mon[84265]: pgmap v140: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 950 B/s rd, 1 op/s 2024-05-13T22:56:19.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:19 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1502877146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:19.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:19 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1609698914' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:56:19.473 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.497+0000 7fa304304700 1 -- 172.21.15.102:0/281424863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2fc105a40 msgr2=0x7fa2fc105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:19.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.497+0000 7fa304304700 1 --2- 172.21.15.102:0/281424863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2fc105a40 0x7fa2fc105e20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009910 tx=0x7fa2f0009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:19.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.497+0000 7fa304304700 1 -- 172.21.15.102:0/281424863 shutdown_connections 2024-05-13T22:56:19.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.497+0000 7fa304304700 1 --2- 172.21.15.102:0/281424863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2fc0ffbd0 0x7fa2fc100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.497+0000 7fa304304700 1 --2- 172.21.15.102:0/281424863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2fc105a40 0x7fa2fc105e20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.497+0000 7fa304304700 1 -- 172.21.15.102:0/281424863 >> 172.21.15.102:0/281424863 conn(0x7fa2fc0755a0 msgr2=0x7fa2fc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:19.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.497+0000 7fa304304700 1 -- 172.21.15.102:0/281424863 shutdown_connections 2024-05-13T22:56:19.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.497+0000 7fa304304700 1 -- 172.21.15.102:0/281424863 wait complete. 2024-05-13T22:56:19.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.498+0000 7fa304304700 1 Processor -- start 2024-05-13T22:56:19.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.498+0000 7fa304304700 1 -- start start 2024-05-13T22:56:19.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.498+0000 7fa304304700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2fc0ffbd0 0x7fa2fc195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:19.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.498+0000 7fa304304700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2fc105a40 0x7fa2fc195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:19.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.498+0000 7fa304304700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc196260 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.498+0000 7fa304304700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc199ff0 con 0x7fa2fc105a40 2024-05-13T22:56:19.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.498+0000 7fa3020a0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2fc0ffbd0 0x7fa2fc195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:19.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.498+0000 7fa3020a0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2fc0ffbd0 0x7fa2fc195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:58038/0 (socket says 172.21.15.102:58038) 2024-05-13T22:56:19.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.498+0000 7fa3020a0700 1 -- 172.21.15.102:0/3130899954 learned_addr learned my addr 172.21.15.102:0/3130899954 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:19.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.499+0000 7fa3020a0700 1 -- 172.21.15.102:0/3130899954 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2fc105a40 msgr2=0x7fa2fc195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:19.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.499+0000 7fa3020a0700 1 --2- 172.21.15.102:0/3130899954 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2fc105a40 0x7fa2fc195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.499+0000 7fa3020a0700 1 -- 172.21.15.102:0/3130899954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f00095f0 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.499+0000 7fa3020a0700 1 --2- 172.21.15.102:0/3130899954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2fc0ffbd0 0x7fa2fc195640 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0006010 tx=0x7fa2f000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:19.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.499+0000 7fa2ef7fe700 1 -- 172.21.15.102:0/3130899954 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2f001d070 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.499+0000 7fa2ef7fe700 1 -- 172.21.15.102:0/3130899954 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2f000f460 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.499+0000 7fa304304700 1 -- 172.21.15.102:0/3130899954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2fc19a270 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.499+0000 7fa2ef7fe700 1 -- 172.21.15.102:0/3130899954 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2f0017630 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.499+0000 7fa304304700 1 -- 172.21.15.102:0/3130899954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc19a760 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.500+0000 7fa304304700 1 -- 172.21.15.102:0/3130899954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2fc108040 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.501+0000 7fa2ef7fe700 1 -- 172.21.15.102:0/3130899954 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa2f000f5d0 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.501+0000 7fa2ef7fe700 1 --2- 172.21.15.102:0/3130899954 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa2e806e200 0x7fa2e80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:19.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.501+0000 7fa2ef7fe700 1 -- 172.21.15.102:0/3130899954 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fa2f008f6b0 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.501+0000 7fa30189f700 1 --2- 172.21.15.102:0/3130899954 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa2e806e200 0x7fa2e80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:19.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.501+0000 7fa30189f700 1 --2- 172.21.15.102:0/3130899954 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa2e806e200 0x7fa2e80706c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8005950 tx=0x7fa2f80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:19.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.506+0000 7fa2ef7fe700 1 -- 172.21.15.102:0/3130899954 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa2f0058920 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.781+0000 7fa304304700 1 -- 172.21.15.102:0/3130899954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa2fc19ab00 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.758 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.781+0000 7fa2ef7fe700 1 -- 172.21.15.102:0/3130899954 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa2f005bfe0 con 0x7fa2fc0ffbd0 2024-05-13T22:56:19.758 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:56:19.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.783+0000 7fa304304700 1 -- 172.21.15.102:0/3130899954 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa2e806e200 msgr2=0x7fa2e80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:19.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.783+0000 7fa304304700 1 --2- 172.21.15.102:0/3130899954 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa2e806e200 0x7fa2e80706c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8005950 tx=0x7fa2f80058e0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.783+0000 7fa304304700 1 -- 172.21.15.102:0/3130899954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2fc0ffbd0 msgr2=0x7fa2fc195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:19.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.783+0000 7fa304304700 1 --2- 172.21.15.102:0/3130899954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2fc0ffbd0 0x7fa2fc195640 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0006010 tx=0x7fa2f000be20 comp rx=0 tx=0).stop 2024-05-13T22:56:19.760 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.783+0000 7fa304304700 1 -- 172.21.15.102:0/3130899954 shutdown_connections 2024-05-13T22:56:19.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.783+0000 7fa304304700 1 --2- 172.21.15.102:0/3130899954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2fc0ffbd0 0x7fa2fc195640 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.783+0000 7fa304304700 1 --2- 172.21.15.102:0/3130899954 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa2e806e200 0x7fa2e80706c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.783+0000 7fa304304700 1 --2- 172.21.15.102:0/3130899954 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2fc105a40 0x7fa2fc195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:19.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.783+0000 7fa304304700 1 -- 172.21.15.102:0/3130899954 >> 172.21.15.102:0/3130899954 conn(0x7fa2fc0755a0 msgr2=0x7fa2fc0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:19.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.783+0000 7fa304304700 1 -- 172.21.15.102:0/3130899954 shutdown_connections 2024-05-13T22:56:19.761 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:19.783+0000 7fa304304700 1 -- 172.21.15.102:0/3130899954 wait complete. 2024-05-13T22:56:20.343 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:20 smithi102 ceph-mon[111694]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:20.343 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:20 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3130899954' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:56:20.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:20 smithi130 ceph-mon[84265]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:20.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:20 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3130899954' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:56:21.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:21 smithi130 ceph-mon[84265]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:21.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:21 smithi130 ceph-mon[84265]: pgmap v141: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 862 B/s rd, 1 op/s 2024-05-13T22:56:21.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:21 smithi102 ceph-mon[111694]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:21.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:21 smithi102 ceph-mon[111694]: pgmap v141: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 862 B/s rd, 1 op/s 2024-05-13T22:56:22.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:22 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:22.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:22 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:23.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:23 smithi130 ceph-mon[84265]: pgmap v142: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T22:56:23.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:23 smithi102 ceph-mon[111694]: pgmap v142: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T22:56:25.362 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:25 smithi130 ceph-mon[84265]: pgmap v143: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-13T22:56:25.363 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:25 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-13T22:56:25.363 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:25 smithi130 ceph-mon[84265]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-13T22:56:25.363 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:25 smithi130 ceph-mon[84265]: Upgrade: osd.4 is safe to restart 2024-05-13T22:56:25.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:25 smithi102 ceph-mon[111694]: pgmap v143: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-13T22:56:25.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:25 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-13T22:56:25.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:25 smithi102 ceph-mon[111694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-13T22:56:25.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:25 smithi102 ceph-mon[111694]: Upgrade: osd.4 is safe to restart 2024-05-13T22:56:27.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:26 smithi130 ceph-mon[84265]: Upgrade: Updating osd.4 2024-05-13T22:56:27.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:26 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:27.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:26 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-13T22:56:27.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:26 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:56:27.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:26 smithi130 ceph-mon[84265]: Deploying daemon osd.4 on smithi130 2024-05-13T22:56:27.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:26 smithi130 ceph-mon[84265]: pgmap v144: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:56:27.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:26 smithi102 ceph-mon[111694]: Upgrade: Updating osd.4 2024-05-13T22:56:27.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:26 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:27.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:26 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-13T22:56:27.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:26 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:56:27.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:26 smithi102 ceph-mon[111694]: Deploying daemon osd.4 on smithi130 2024-05-13T22:56:27.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:26 smithi102 ceph-mon[111694]: pgmap v144: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:56:29.350 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:29 smithi130 ceph-mon[84265]: pgmap v145: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:56:29.350 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:29 smithi130 systemd[1]: Stopping Ceph osd.4 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:56:29.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:29 smithi102 ceph-mon[111694]: pgmap v145: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:56:29.649 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:29 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[49455]: 2024-05-13T22:56:29.373+0000 7ff23b3c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T22:56:29.649 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:29 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[49455]: 2024-05-13T22:56:29.373+0000 7ff23b3c4700 -1 osd.4 81 *** Got signal Terminated *** 2024-05-13T22:56:29.649 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:29 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[49455]: 2024-05-13T22:56:29.373+0000 7ff23b3c4700 -1 osd.4 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T22:56:30.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:30 smithi130 ceph-mon[84265]: osd.4 marked itself down and dead 2024-05-13T22:56:30.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:30 smithi102 ceph-mon[111694]: osd.4 marked itself down and dead 2024-05-13T22:56:30.943 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:30 smithi130 podman[98167]: 2024-05-13 22:56:30.713135054 +0000 UTC m=+1.443057181 container died 5af6d8af2b7cd96efdcceb933e294c3001f4efb698859c6a977248f631d2c5cf (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4, org.label-schema.license=GPLv2, RELEASE=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1) 2024-05-13T22:56:31.351 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:31 smithi130 ceph-mon[84265]: pgmap v146: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:56:31.351 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:31 smithi130 ceph-mon[84265]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:56:31.351 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:31 smithi130 ceph-mon[84265]: osdmap e82: 6 total, 5 up, 6 in 2024-05-13T22:56:31.351 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:31.351 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:56:31.351 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:31 smithi130 podman[98167]: 2024-05-13 22:56:31.073664451 +0000 UTC m=+1.803586578 container cleanup 5af6d8af2b7cd96efdcceb933e294c3001f4efb698859c6a977248f631d2c5cf (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=HEAD) 2024-05-13T22:56:31.351 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:31 smithi130 bash[98167]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4 2024-05-13T22:56:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:31 smithi102 ceph-mon[111694]: pgmap v146: 65 pgs: 65 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:56:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:31 smithi102 ceph-mon[111694]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:56:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:31 smithi102 ceph-mon[111694]: osdmap e82: 6 total, 5 up, 6 in 2024-05-13T22:56:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:56:31.649 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:31 smithi130 podman[98185]: 2024-05-13 22:56:31.374935632 +0000 UTC m=+0.660224367 container remove 5af6d8af2b7cd96efdcceb933e294c3001f4efb698859c6a977248f631d2c5cf (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-13T22:56:31.955 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:31 smithi130 podman[98274]: 2024-05-13 22:56:31.626609244 +0000 UTC m=+0.017025611 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:56:31.955 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:31 smithi130 podman[98274]: 2024-05-13 22:56:31.725905737 +0000 UTC m=+0.116322100 container create 7fc3bdba5fe23a99dce2650ed3593c38dd63cb2ffcb35bb3f7576defa0678465 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-13T22:56:31.955 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:31 smithi130 podman[98274]: 2024-05-13 22:56:31.979034235 +0000 UTC m=+0.369450606 container init 7fc3bdba5fe23a99dce2650ed3593c38dd63cb2ffcb35bb3f7576defa0678465 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=squid-c3e5514) 2024-05-13T22:56:32.247 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:32 smithi130 ceph-mon[84265]: osdmap e83: 6 total, 5 up, 6 in 2024-05-13T22:56:32.247 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:31 smithi130 podman[98274]: 2024-05-13 22:56:31.982635753 +0000 UTC m=+0.373052118 container start 7fc3bdba5fe23a99dce2650ed3593c38dd63cb2ffcb35bb3f7576defa0678465 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=squid-c3e5514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-13T22:56:32.247 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:32 smithi130 podman[98274]: 2024-05-13 22:56:32.120285807 +0000 UTC m=+0.510702171 container attach 7fc3bdba5fe23a99dce2650ed3593c38dd63cb2ffcb35bb3f7576defa0678465 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-c3e5514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-13T22:56:32.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:32 smithi102 ceph-mon[111694]: osdmap e83: 6 total, 5 up, 6 in 2024-05-13T22:56:32.540 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:32 smithi130 podman[98274]: 2024-05-13 22:56:32.271776244 +0000 UTC m=+0.662192613 container died 7fc3bdba5fe23a99dce2650ed3593c38dd63cb2ffcb35bb3f7576defa0678465 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, maintainer=Guillaume Abrioux ) 2024-05-13T22:56:33.107 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:32 smithi130 podman[98300]: 2024-05-13 22:56:32.874181703 +0000 UTC m=+0.595950837 container remove 7fc3bdba5fe23a99dce2650ed3593c38dd63cb2ffcb35bb3f7576defa0678465 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-c3e5514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-05-13T22:56:33.107 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:32 smithi130 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-13T22:56:33.107 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:32 smithi130 systemd[1]: Stopped Ceph osd.4 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:56:33.108 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:32 smithi130 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.4.service: Consumed 1min 5.815s CPU time. 2024-05-13T22:56:33.108 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 systemd[1]: Starting Ceph osd.4 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:56:33.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:33 smithi130 ceph-mon[84265]: pgmap v149: 65 pgs: 4 active+undersized, 2 active+undersized+degraded, 8 peering, 2 stale+active+clean, 49 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 7/231 objects degraded (3.030%) 2024-05-13T22:56:33.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:33 smithi130 ceph-mon[84265]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-05-13T22:56:33.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:33 smithi130 ceph-mon[84265]: Health check failed: Degraded data redundancy: 7/231 objects degraded (3.030%), 2 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:33.399 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 podman[98398]: 2024-05-13 22:56:33.212568378 +0000 UTC m=+0.017457348 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:56:33.399 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 podman[98398]: 2024-05-13 22:56:33.325750624 +0000 UTC m=+0.130639594 container create d09139207f24eabf34521e7bea696ea9df1b65b7e03585ae53faa868ca93cd6c (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-c3e5514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-13T22:56:33.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:33 smithi102 ceph-mon[111694]: pgmap v149: 65 pgs: 4 active+undersized, 2 active+undersized+degraded, 8 peering, 2 stale+active+clean, 49 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 7/231 objects degraded (3.030%) 2024-05-13T22:56:33.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:33 smithi102 ceph-mon[111694]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-05-13T22:56:33.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:33 smithi102 ceph-mon[111694]: Health check failed: Degraded data redundancy: 7/231 objects degraded (3.030%), 2 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:33.899 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 podman[98398]: 2024-05-13 22:56:33.536747926 +0000 UTC m=+0.341636896 container init d09139207f24eabf34521e7bea696ea9df1b65b7e03585ae53faa868ca93cd6c (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-c3e5514, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-13T22:56:33.899 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 podman[98398]: 2024-05-13 22:56:33.540406205 +0000 UTC m=+0.345295177 container start d09139207f24eabf34521e7bea696ea9df1b65b7e03585ae53faa868ca93cd6c (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=squid-c3e5514, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-13T22:56:33.899 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 podman[98398]: 2024-05-13 22:56:33.594777854 +0000 UTC m=+0.399666837 container attach d09139207f24eabf34521e7bea696ea9df1b65b7e03585ae53faa868ca93cd6c (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-c3e5514, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-13T22:56:34.257 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate[98414]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-13T22:56:34.257 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate[98414]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-13T22:56:34.257 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 bash[98398]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-13T22:56:34.257 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 bash[98398]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-13T22:56:34.257 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 bash[98398]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-13T22:56:34.258 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 bash[98398]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-13T22:56:34.258 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 bash[98398]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-13T22:56:34.258 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 bash[98398]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-13T22:56:34.262 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 bash[98398]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-13T22:56:34.262 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate[98414]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-13T22:56:34.262 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate[98414]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-13T22:56:34.262 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate[98414]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-13T22:56:34.262 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate[98414]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-13T22:56:34.262 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:33 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate[98414]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-13T22:56:34.262 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:34 smithi130 podman[98398]: 2024-05-13 22:56:34.013886699 +0000 UTC m=+0.818775677 container died d09139207f24eabf34521e7bea696ea9df1b65b7e03585ae53faa868ca93cd6c (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-13T22:56:34.843 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:34 smithi130 podman[98533]: 2024-05-13 22:56:34.565623122 +0000 UTC m=+0.542058432 container remove d09139207f24eabf34521e7bea696ea9df1b65b7e03585ae53faa868ca93cd6c (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-13T22:56:34.843 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:34 smithi130 podman[98593]: 2024-05-13 22:56:34.774838342 +0000 UTC m=+0.110751107 container create 5bb5a8f94042c809b83d3e35fe89b3a5e23a860495bb291bb0210f93a8946e20 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-13T22:56:34.843 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:34 smithi130 podman[98593]: 2024-05-13 22:56:34.681693961 +0000 UTC m=+0.017606737 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:56:35.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:35 smithi130 ceph-mon[84265]: pgmap v150: 65 pgs: 10 active+undersized, 7 active+undersized+degraded, 8 peering, 1 stale+active+clean, 39 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 25/231 objects degraded (10.823%) 2024-05-13T22:56:35.399 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:34 smithi130 podman[98593]: 2024-05-13 22:56:34.994082607 +0000 UTC m=+0.329995381 container init 5bb5a8f94042c809b83d3e35fe89b3a5e23a860495bb291bb0210f93a8946e20 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-13T22:56:35.400 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:34 smithi130 podman[98593]: 2024-05-13 22:56:34.997391442 +0000 UTC m=+0.333304214 container start 5bb5a8f94042c809b83d3e35fe89b3a5e23a860495bb291bb0210f93a8946e20 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-13T22:56:35.400 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:35 smithi130 ceph-osd[98611]: -- 172.21.15.130:0/3624795438 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4c65d7680 con 0x55a4c65a1000 2024-05-13T22:56:35.400 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:35 smithi130 bash[98593]: 5bb5a8f94042c809b83d3e35fe89b3a5e23a860495bb291bb0210f93a8946e20 2024-05-13T22:56:35.400 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:35 smithi130 systemd[1]: Started Ceph osd.4 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:56:35.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:35 smithi102 ceph-mon[111694]: pgmap v150: 65 pgs: 10 active+undersized, 7 active+undersized+degraded, 8 peering, 1 stale+active+clean, 39 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 25/231 objects degraded (10.823%) 2024-05-13T22:56:35.891 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:35 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T22:56:35.539+0000 7ff4d261f7c0 -1 Falling back to public interface 2024-05-13T22:56:36.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:36.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:36.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:36 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:56:36.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:36.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:36.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:36 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:56:37.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:37 smithi130 ceph-mon[84265]: pgmap v151: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 40/231 objects degraded (17.316%) 2024-05-13T22:56:37.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:37 smithi130 ceph-mon[84265]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-05-13T22:56:37.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:37 smithi102 ceph-mon[111694]: pgmap v151: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 40/231 objects degraded (17.316%) 2024-05-13T22:56:37.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:37 smithi102 ceph-mon[111694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-05-13T22:56:39.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:39 smithi130 ceph-mon[84265]: pgmap v152: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 40/231 objects degraded (17.316%) 2024-05-13T22:56:39.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:39 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:39.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:39 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:39.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:39 smithi102 ceph-mon[111694]: pgmap v152: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 40/231 objects degraded (17.316%) 2024-05-13T22:56:39.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:39 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:39.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:39 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:40.251 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:40 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T22:56:40.078+0000 7ff4d261f7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-05-13T22:56:40.542 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:40.542 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:40.542 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:40 smithi130 ceph-mon[84265]: pgmap v153: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 514 B/s rd, 1 op/s; 40/231 objects degraded (17.316%) 2024-05-13T22:56:40.582 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:40.582 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:40.582 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:40 smithi102 ceph-mon[111694]: pgmap v153: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 212 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 514 B/s rd, 1 op/s; 40/231 objects degraded (17.316%) 2024-05-13T22:56:40.899 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:40 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T22:56:40.564+0000 7ff4d261f7c0 -1 osd.4 81 log_to_monitors true 2024-05-13T22:56:41.525 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:41 smithi130 ceph-mon[84265]: from='osd.4 [v2:172.21.15.130:6808/3241450581,v1:172.21.15.130:6809/3241450581]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-13T22:56:41.525 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:41 smithi130 ceph-mon[84265]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-13T22:56:41.525 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:41 smithi130 ceph-mon[84265]: Health check update: Degraded data redundancy: 40/231 objects degraded (17.316%), 15 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:41.525 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 22:56:41 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T22:56:41.301+0000 7ff4c3a76700 -1 osd.4 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:56:41.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:41 smithi102 ceph-mon[111694]: from='osd.4 [v2:172.21.15.130:6808/3241450581,v1:172.21.15.130:6809/3241450581]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-13T22:56:41.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:41 smithi102 ceph-mon[111694]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-13T22:56:41.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:41 smithi102 ceph-mon[111694]: Health check update: Degraded data redundancy: 40/231 objects degraded (17.316%), 15 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:42.521 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:42 smithi130 ceph-mon[84265]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-13T22:56:42.521 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:42 smithi130 ceph-mon[84265]: from='osd.4 [v2:172.21.15.130:6808/3241450581,v1:172.21.15.130:6809/3241450581]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:56:42.521 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:42 smithi130 ceph-mon[84265]: osdmap e84: 6 total, 5 up, 6 in 2024-05-13T22:56:42.522 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:42 smithi130 ceph-mon[84265]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:56:42.522 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:42 smithi130 ceph-mon[84265]: pgmap v155: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 40/231 objects degraded (17.316%) 2024-05-13T22:56:42.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:42 smithi102 ceph-mon[111694]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-13T22:56:42.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:42 smithi102 ceph-mon[111694]: from='osd.4 [v2:172.21.15.130:6808/3241450581,v1:172.21.15.130:6809/3241450581]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:56:42.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:42 smithi102 ceph-mon[111694]: osdmap e84: 6 total, 5 up, 6 in 2024-05-13T22:56:42.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:42 smithi102 ceph-mon[111694]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:56:42.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:42 smithi102 ceph-mon[111694]: pgmap v155: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 40/231 objects degraded (17.316%) 2024-05-13T22:56:43.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:56:43.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: osd.4 [v2:172.21.15.130:6808/3241450581,v1:172.21.15.130:6809/3241450581] boot 2024-05-13T22:56:43.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: osdmap e85: 6 total, 6 up, 6 in 2024-05-13T22:56:43.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:56:43.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:43.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:43.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:56:43.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:56:43.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:43.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:56:43.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:43.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:43.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:43.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-13T22:56:43.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-13T22:56:43.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:43 smithi102 ceph-mon[111694]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-05-13T22:56:43.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:56:43.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: osd.4 [v2:172.21.15.130:6808/3241450581,v1:172.21.15.130:6809/3241450581] boot 2024-05-13T22:56:43.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: osdmap e85: 6 total, 6 up, 6 in 2024-05-13T22:56:43.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-13T22:56:43.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:43.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:43.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:56:43.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:56:43.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:43.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:56:43.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:43.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:43.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:43.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-13T22:56:43.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-13T22:56:43.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:43 smithi130 ceph-mon[84265]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-05-13T22:56:44.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:44 smithi130 ceph-mon[84265]: osdmap e86: 6 total, 6 up, 6 in 2024-05-13T22:56:44.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:44 smithi130 ceph-mon[84265]: pgmap v158: 65 pgs: 10 peering, 12 active+undersized, 10 active+undersized+degraded, 33 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 22/231 objects degraded (9.524%) 2024-05-13T22:56:44.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:44 smithi102 ceph-mon[111694]: osdmap e86: 6 total, 6 up, 6 in 2024-05-13T22:56:44.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:44 smithi102 ceph-mon[111694]: pgmap v158: 65 pgs: 10 peering, 12 active+undersized, 10 active+undersized+degraded, 33 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 22/231 objects degraded (9.524%) 2024-05-13T22:56:47.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:47 smithi130 ceph-mon[84265]: pgmap v159: 65 pgs: 10 peering, 7 active+undersized, 3 active+undersized+degraded, 45 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 2 op/s; 9/231 objects degraded (3.896%) 2024-05-13T22:56:47.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:47.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:56:47.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:47 smithi130 ceph-mon[84265]: Health check update: Degraded data redundancy: 9/231 objects degraded (3.896%), 3 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:47.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:47 smithi102 ceph-mon[111694]: pgmap v159: 65 pgs: 10 peering, 7 active+undersized, 3 active+undersized+degraded, 45 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 2 op/s; 9/231 objects degraded (3.896%) 2024-05-13T22:56:47.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:47.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:56:47.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:47 smithi102 ceph-mon[111694]: Health check update: Degraded data redundancy: 9/231 objects degraded (3.896%), 3 pgs degraded (PG_DEGRADED) 2024-05-13T22:56:48.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:48 smithi130 ceph-mon[84265]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/231 objects degraded (3.896%), 3 pgs degraded) 2024-05-13T22:56:48.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:48 smithi130 ceph-mon[84265]: Cluster is now healthy 2024-05-13T22:56:48.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:48 smithi102 ceph-mon[111694]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9/231 objects degraded (3.896%), 3 pgs degraded) 2024-05-13T22:56:48.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:48 smithi102 ceph-mon[111694]: Cluster is now healthy 2024-05-13T22:56:49.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:49 smithi130 ceph-mon[84265]: pgmap v160: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-13T22:56:49.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:49 smithi102 ceph-mon[111694]: pgmap v160: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-13T22:56:49.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.904+0000 7f8c7ca81700 1 -- 172.21.15.102:0/2502662632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c78101770 msgr2=0x7f8c781057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:49.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.904+0000 7f8c7ca81700 1 --2- 172.21.15.102:0/2502662632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c78101770 0x7f8c781057c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009910 tx=0x7f8c6c009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:49.882 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.904+0000 7f8c7ca81700 1 -- 172.21.15.102:0/2502662632 shutdown_connections 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.904+0000 7f8c7ca81700 1 --2- 172.21.15.102:0/2502662632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c78101770 0x7f8c781057c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.904+0000 7f8c7ca81700 1 --2- 172.21.15.102:0/2502662632 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c78100dc0 0x7f8c781011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.904+0000 7f8c7ca81700 1 -- 172.21.15.102:0/2502662632 >> 172.21.15.102:0/2502662632 conn(0x7f8c780fc650 msgr2=0x7f8c780fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.904+0000 7f8c7ca81700 1 -- 172.21.15.102:0/2502662632 shutdown_connections 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.904+0000 7f8c7ca81700 1 -- 172.21.15.102:0/2502662632 wait complete. 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c7ca81700 1 Processor -- start 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c7ca81700 1 -- start start 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c7ca81700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c78100dc0 0x7f8c78073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c7ca81700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c78101770 0x7f8c780735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c7ca81700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78073b10 con 0x7f8c78101770 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c7ca81700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78073c50 con 0x7f8c78100dc0 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c75d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c78101770 0x7f8c780735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c75d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c78101770 0x7f8c780735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40670/0 (socket says 172.21.15.102:40670) 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c75d9b700 1 -- 172.21.15.102:0/3955185253 learned_addr learned my addr 172.21.15.102:0/3955185253 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c7659c700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c78100dc0 0x7f8c78073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c75d9b700 1 -- 172.21.15.102:0/3955185253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c78100dc0 msgr2=0x7f8c78073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c75d9b700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c78100dc0 0x7f8c78073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c75d9b700 1 -- 172.21.15.102:0/3955185253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c0095f0 con 0x7f8c78101770 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.905+0000 7f8c7659c700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c78100dc0 0x7f8c78073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.906+0000 7f8c75d9b700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c78101770 0x7f8c780735d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0092e0 tx=0x7f8c6c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.906+0000 7f8c677fe700 1 -- 172.21.15.102:0/3955185253 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c6c01d070 con 0x7f8c78101770 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.906+0000 7f8c7ca81700 1 -- 172.21.15.102:0/3955185253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c78073ed0 con 0x7f8c78101770 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.906+0000 7f8c677fe700 1 -- 172.21.15.102:0/3955185253 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c6c00f460 con 0x7f8c78101770 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.906+0000 7f8c677fe700 1 -- 172.21.15.102:0/3955185253 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c6c0175f0 con 0x7f8c78101770 2024-05-13T22:56:49.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.906+0000 7f8c7ca81700 1 -- 172.21.15.102:0/3955185253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c781a2c90 con 0x7f8c78101770 2024-05-13T22:56:49.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.907+0000 7f8c7ca81700 1 -- 172.21.15.102:0/3955185253 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c781091c0 con 0x7f8c78101770 2024-05-13T22:56:49.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.907+0000 7f8c677fe700 1 -- 172.21.15.102:0/3955185253 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8c6c00f5d0 con 0x7f8c78101770 2024-05-13T22:56:49.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.908+0000 7f8c677fe700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8c6006e200 0x7f8c600706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:49.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.908+0000 7f8c677fe700 1 -- 172.21.15.102:0/3955185253 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f8c6c08f720 con 0x7f8c78101770 2024-05-13T22:56:49.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.908+0000 7f8c7659c700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8c6006e200 0x7f8c600706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:49.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.908+0000 7f8c7659c700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8c6006e200 0x7f8c600706c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8c68005fd0 tx=0x7f8c68005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:49.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:49.913+0000 7f8c677fe700 1 -- 172.21.15.102:0/3955185253 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8c6c058990 con 0x7f8c78101770 2024-05-13T22:56:50.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.127+0000 7f8c7ca81700 1 -- 172.21.15.102:0/3955185253 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c7802cce0 con 0x7f8c6006e200 2024-05-13T22:56:50.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.128+0000 7f8c677fe700 1 -- 172.21.15.102:0/3955185253 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f8c7802cce0 con 0x7f8c6006e200 2024-05-13T22:56:50.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.130+0000 7f8c7ca81700 1 -- 172.21.15.102:0/3955185253 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8c6006e200 msgr2=0x7f8c600706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.130+0000 7f8c7ca81700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8c6006e200 0x7f8c600706c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8c68005fd0 tx=0x7f8c68005dc0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.130+0000 7f8c7ca81700 1 -- 172.21.15.102:0/3955185253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c78101770 msgr2=0x7f8c780735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.130+0000 7f8c7ca81700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c78101770 0x7f8c780735d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0092e0 tx=0x7f8c6c00be20 comp rx=0 tx=0).stop 2024-05-13T22:56:50.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.130+0000 7f8c7ca81700 1 -- 172.21.15.102:0/3955185253 shutdown_connections 2024-05-13T22:56:50.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.130+0000 7f8c7ca81700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8c6006e200 0x7f8c600706c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.130+0000 7f8c7ca81700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c78100dc0 0x7f8c78073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.130+0000 7f8c7ca81700 1 --2- 172.21.15.102:0/3955185253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c78101770 0x7f8c780735d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.130+0000 7f8c7ca81700 1 -- 172.21.15.102:0/3955185253 >> 172.21.15.102:0/3955185253 conn(0x7f8c780fc650 msgr2=0x7f8c780fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:50.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.130+0000 7f8c7ca81700 1 -- 172.21.15.102:0/3955185253 shutdown_connections 2024-05-13T22:56:50.107 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.130+0000 7f8c7ca81700 1 -- 172.21.15.102:0/3955185253 wait complete. 2024-05-13T22:56:50.117 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:56:50.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.236+0000 7f18abc06700 1 -- 172.21.15.102:0/948405432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18a4069020 msgr2=0x7f18a4105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.236+0000 7f18abc06700 1 --2- 172.21.15.102:0/948405432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18a4069020 0x7f18a4105790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1898009910 tx=0x7f1898009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.236+0000 7f18abc06700 1 -- 172.21.15.102:0/948405432 shutdown_connections 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.236+0000 7f18abc06700 1 --2- 172.21.15.102:0/948405432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18a4069020 0x7f18a4105790 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.236+0000 7f18abc06700 1 --2- 172.21.15.102:0/948405432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18a4068670 0x7f18a4068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.236+0000 7f18abc06700 1 -- 172.21.15.102:0/948405432 >> 172.21.15.102:0/948405432 conn(0x7f18a40755a0 msgr2=0x7f18a40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.236+0000 7f18abc06700 1 -- 172.21.15.102:0/948405432 shutdown_connections 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.236+0000 7f18abc06700 1 -- 172.21.15.102:0/948405432 wait complete. 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18abc06700 1 Processor -- start 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18abc06700 1 -- start start 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18abc06700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18a4068670 0x7f18a4196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18abc06700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18a4069020 0x7f18a4196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18abc06700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a4197240 con 0x7f18a4069020 2024-05-13T22:56:50.334 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18abc06700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a41973b0 con 0x7f18a4068670 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18a91a1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18a4069020 0x7f18a4196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18a91a1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18a4069020 0x7f18a4196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40680/0 (socket says 172.21.15.102:40680) 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18a91a1700 1 -- 172.21.15.102:0/563229899 learned_addr learned my addr 172.21.15.102:0/563229899 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18a99a2700 1 --2- 172.21.15.102:0/563229899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18a4068670 0x7f18a4196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18a91a1700 1 -- 172.21.15.102:0/563229899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18a4068670 msgr2=0x7f18a4196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18a91a1700 1 --2- 172.21.15.102:0/563229899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18a4068670 0x7f18a4196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.237+0000 7f18a91a1700 1 -- 172.21.15.102:0/563229899 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18980095f0 con 0x7f18a4069020 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.238+0000 7f18a91a1700 1 --2- 172.21.15.102:0/563229899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18a4069020 0x7f18a4196c40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f189800bda0 tx=0x7f189800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.238+0000 7f18a2ffd700 1 -- 172.21.15.102:0/563229899 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f189801d070 con 0x7f18a4069020 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.238+0000 7f18a2ffd700 1 -- 172.21.15.102:0/563229899 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f189800f460 con 0x7f18a4069020 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.238+0000 7f18abc06700 1 -- 172.21.15.102:0/563229899 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a419a0f0 con 0x7f18a4069020 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.238+0000 7f18a2ffd700 1 -- 172.21.15.102:0/563229899 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1898017630 con 0x7f18a4069020 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.238+0000 7f18abc06700 1 -- 172.21.15.102:0/563229899 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a419a5e0 con 0x7f18a4069020 2024-05-13T22:56:50.335 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.239+0000 7f18abc06700 1 -- 172.21.15.102:0/563229899 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a404ea90 con 0x7f18a4069020 2024-05-13T22:56:50.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.239+0000 7f18a2ffd700 1 -- 172.21.15.102:0/563229899 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f189800fab0 con 0x7f18a4069020 2024-05-13T22:56:50.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.240+0000 7f18a2ffd700 1 --2- 172.21.15.102:0/563229899 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f188c06e200 0x7f188c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:50.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.240+0000 7f18a99a2700 1 --2- 172.21.15.102:0/563229899 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f188c06e200 0x7f188c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:50.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.240+0000 7f18a2ffd700 1 -- 172.21.15.102:0/563229899 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f189808f610 con 0x7f18a4069020 2024-05-13T22:56:50.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.240+0000 7f18a99a2700 1 --2- 172.21.15.102:0/563229899 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f188c06e200 0x7f188c0706c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f189c00ac50 tx=0x7f189c00a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:50.337 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.244+0000 7f18a2ffd700 1 -- 172.21.15.102:0/563229899 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1898058900 con 0x7f18a4069020 2024-05-13T22:56:50.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.463+0000 7f18abc06700 1 -- 172.21.15.102:0/563229899 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18a4108c70 con 0x7f188c06e200 2024-05-13T22:56:50.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.465+0000 7f18a2ffd700 1 -- 172.21.15.102:0/563229899 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f18a4108c70 con 0x7f188c06e200 2024-05-13T22:56:50.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.467+0000 7f18abc06700 1 -- 172.21.15.102:0/563229899 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f188c06e200 msgr2=0x7f188c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.467+0000 7f18abc06700 1 --2- 172.21.15.102:0/563229899 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f188c06e200 0x7f188c0706c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f189c00ac50 tx=0x7f189c00a380 comp rx=0 tx=0).stop 2024-05-13T22:56:50.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.467+0000 7f18abc06700 1 -- 172.21.15.102:0/563229899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18a4069020 msgr2=0x7f18a4196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.467+0000 7f18abc06700 1 --2- 172.21.15.102:0/563229899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18a4069020 0x7f18a4196c40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f189800bda0 tx=0x7f189800bdd0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.467+0000 7f18abc06700 1 -- 172.21.15.102:0/563229899 shutdown_connections 2024-05-13T22:56:50.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.467+0000 7f18abc06700 1 --2- 172.21.15.102:0/563229899 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f188c06e200 0x7f188c0706c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.467+0000 7f18abc06700 1 --2- 172.21.15.102:0/563229899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18a4068670 0x7f18a4196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.467+0000 7f18abc06700 1 --2- 172.21.15.102:0/563229899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18a4069020 0x7f18a4196c40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.467+0000 7f18abc06700 1 -- 172.21.15.102:0/563229899 >> 172.21.15.102:0/563229899 conn(0x7f18a40755a0 msgr2=0x7f18a4105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:50.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.467+0000 7f18abc06700 1 -- 172.21.15.102:0/563229899 shutdown_connections 2024-05-13T22:56:50.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.467+0000 7f18abc06700 1 -- 172.21.15.102:0/563229899 wait complete. 2024-05-13T22:56:50.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.572+0000 7f80c400d700 1 -- 172.21.15.102:0/1195559072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80bc0ff510 msgr2=0x7f80bc0ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.572+0000 7f80c400d700 1 --2- 172.21.15.102:0/1195559072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80bc0ff510 0x7f80bc0ff8f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009910 tx=0x7f80b8009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:50.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.573+0000 7f80c400d700 1 -- 172.21.15.102:0/1195559072 shutdown_connections 2024-05-13T22:56:50.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.573+0000 7f80c400d700 1 --2- 172.21.15.102:0/1195559072 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80bc0ffec0 0x7f80bc1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.573+0000 7f80c400d700 1 --2- 172.21.15.102:0/1195559072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80bc0ff510 0x7f80bc0ff8f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.573+0000 7f80c400d700 1 -- 172.21.15.102:0/1195559072 >> 172.21.15.102:0/1195559072 conn(0x7f80bc074e00 msgr2=0x7f80bc075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:50.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.573+0000 7f80c400d700 1 -- 172.21.15.102:0/1195559072 shutdown_connections 2024-05-13T22:56:50.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.573+0000 7f80c400d700 1 -- 172.21.15.102:0/1195559072 wait complete. 2024-05-13T22:56:50.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.573+0000 7f80c400d700 1 Processor -- start 2024-05-13T22:56:50.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.573+0000 7f80c400d700 1 -- start start 2024-05-13T22:56:50.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c400d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80bc0ff510 0x7f80bc19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:50.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c400d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80bc0ffec0 0x7f80bc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:50.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c400d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc19d240 con 0x7f80bc0ff510 2024-05-13T22:56:50.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c400d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc1966f0 con 0x7f80bc0ffec0 2024-05-13T22:56:50.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c1da9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80bc0ff510 0x7f80bc19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:50.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c1da9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80bc0ff510 0x7f80bc19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40710/0 (socket says 172.21.15.102:40710) 2024-05-13T22:56:50.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c1da9700 1 -- 172.21.15.102:0/4052220556 learned_addr learned my addr 172.21.15.102:0/4052220556 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:50.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c15a8700 1 --2- 172.21.15.102:0/4052220556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80bc0ffec0 0x7f80bc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:50.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c1da9700 1 -- 172.21.15.102:0/4052220556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80bc0ffec0 msgr2=0x7f80bc19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c1da9700 1 --2- 172.21.15.102:0/4052220556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80bc0ffec0 0x7f80bc19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c1da9700 1 -- 172.21.15.102:0/4052220556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b80095f0 con 0x7f80bc0ff510 2024-05-13T22:56:50.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c1da9700 1 --2- 172.21.15.102:0/4052220556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80bc0ff510 0x7f80bc19c670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f80b8006010 tx=0x7f80b800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:50.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80b2ffd700 1 -- 172.21.15.102:0/4052220556 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80b801d070 con 0x7f80bc0ff510 2024-05-13T22:56:50.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80b2ffd700 1 -- 172.21.15.102:0/4052220556 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80b800f460 con 0x7f80bc0ff510 2024-05-13T22:56:50.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80b2ffd700 1 -- 172.21.15.102:0/4052220556 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80b8017610 con 0x7f80bc0ff510 2024-05-13T22:56:50.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c400d700 1 -- 172.21.15.102:0/4052220556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80bc196970 con 0x7f80bc0ff510 2024-05-13T22:56:50.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.574+0000 7f80c400d700 1 -- 172.21.15.102:0/4052220556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80bc196e60 con 0x7f80bc0ff510 2024-05-13T22:56:50.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.575+0000 7f80c400d700 1 -- 172.21.15.102:0/4052220556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80bc1091e0 con 0x7f80bc0ff510 2024-05-13T22:56:50.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.576+0000 7f80b2ffd700 1 -- 172.21.15.102:0/4052220556 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f80b800f5d0 con 0x7f80bc0ff510 2024-05-13T22:56:50.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.576+0000 7f80b2ffd700 1 --2- 172.21.15.102:0/4052220556 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f80a806e200 0x7f80a80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:50.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.576+0000 7f80b2ffd700 1 -- 172.21.15.102:0/4052220556 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f80b808f6a0 con 0x7f80bc0ff510 2024-05-13T22:56:50.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.576+0000 7f80c15a8700 1 --2- 172.21.15.102:0/4052220556 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f80a806e200 0x7f80a80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:50.554 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.577+0000 7f80c15a8700 1 --2- 172.21.15.102:0/4052220556 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f80a806e200 0x7f80a80706c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f80bc197b60 tx=0x7f80ac009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:50.557 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.581+0000 7f80b2ffd700 1 -- 172.21.15.102:0/4052220556 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f80b805c4c0 con 0x7f80bc0ff510 2024-05-13T22:56:50.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.805+0000 7f80c400d700 1 -- 172.21.15.102:0/4052220556 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f80bc02cce0 con 0x7f80a806e200 2024-05-13T22:56:50.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.815+0000 7f80b2ffd700 1 -- 172.21.15.102:0/4052220556 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f80bc02cce0 con 0x7f80a806e200 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (6m) 80s ago 17m 26.7M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (17m) 80s ago 17m 8740k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (16m) 12s ago 16m 8238k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (3m) 80s ago 17m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (2m) 12s ago 16m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (5m) 80s ago 16m 79.9M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (11m) 80s ago 11m 88.3M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (11m) 80s ago 11m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (11m) 12s ago 11m 17.3M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (11m) 12s ago 11m 20.3M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:56:50.792 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (8m) 80s ago 18m 506M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (7m) 12s ago 16m 453M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (4m) 80s ago 18m 51.7M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (3m) 12s ago 15m 43.2M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (6m) 80s ago 17m 16.0M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (6m) 12s ago 16m 16.8M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (2m) 80s ago 15m 133M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 0ada88c777a2 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (2m) 80s ago 14m 130M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 c3e9e68b06fd 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (86s) 80s ago 14m 15.1M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 1b85728dccf5 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (50s) 12s ago 13m 98.1M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 66e804d4b801 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (16s) 12s ago 13m 14.9M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5bb5a8f94042 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (13m) 12s ago 13m 283M 2240M 18.2.1 5be31c24972a eb9310deec60 2024-05-13T22:56:50.793 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (6m) 80s ago 16m 73.7M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:56:50.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.817+0000 7f80c400d700 1 -- 172.21.15.102:0/4052220556 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f80a806e200 msgr2=0x7f80a80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.817+0000 7f80c400d700 1 --2- 172.21.15.102:0/4052220556 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f80a806e200 0x7f80a80706c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f80bc197b60 tx=0x7f80ac009190 comp rx=0 tx=0).stop 2024-05-13T22:56:50.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.817+0000 7f80c400d700 1 -- 172.21.15.102:0/4052220556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80bc0ff510 msgr2=0x7f80bc19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.817+0000 7f80c400d700 1 --2- 172.21.15.102:0/4052220556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80bc0ff510 0x7f80bc19c670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f80b8006010 tx=0x7f80b800bd70 comp rx=0 tx=0).stop 2024-05-13T22:56:50.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.818+0000 7f80c400d700 1 -- 172.21.15.102:0/4052220556 shutdown_connections 2024-05-13T22:56:50.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.818+0000 7f80c400d700 1 --2- 172.21.15.102:0/4052220556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80bc0ff510 0x7f80bc19c670 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.794 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.818+0000 7f80c400d700 1 --2- 172.21.15.102:0/4052220556 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f80a806e200 0x7f80a80706c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.818+0000 7f80c400d700 1 --2- 172.21.15.102:0/4052220556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f80bc0ffec0 0x7f80bc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.818+0000 7f80c400d700 1 -- 172.21.15.102:0/4052220556 >> 172.21.15.102:0/4052220556 conn(0x7f80bc074e00 msgr2=0x7f80bc0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:50.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.818+0000 7f80c400d700 1 -- 172.21.15.102:0/4052220556 shutdown_connections 2024-05-13T22:56:50.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.818+0000 7f80c400d700 1 -- 172.21.15.102:0/4052220556 wait complete. 2024-05-13T22:56:50.898 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.920+0000 7f2b772a4700 1 -- 172.21.15.102:0/3718630608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b70105a40 msgr2=0x7f2b70105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.920+0000 7f2b772a4700 1 --2- 172.21.15.102:0/3718630608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b70105a40 0x7f2b70105e20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009910 tx=0x7f2b6c009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:50.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.920+0000 7f2b772a4700 1 -- 172.21.15.102:0/3718630608 shutdown_connections 2024-05-13T22:56:50.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.920+0000 7f2b772a4700 1 --2- 172.21.15.102:0/3718630608 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2b700ffa40 0x7f2b700ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.920+0000 7f2b772a4700 1 --2- 172.21.15.102:0/3718630608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b70105a40 0x7f2b70105e20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.920+0000 7f2b772a4700 1 -- 172.21.15.102:0/3718630608 >> 172.21.15.102:0/3718630608 conn(0x7f2b700fb550 msgr2=0x7f2b700fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:50.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.921+0000 7f2b772a4700 1 -- 172.21.15.102:0/3718630608 shutdown_connections 2024-05-13T22:56:50.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.921+0000 7f2b772a4700 1 -- 172.21.15.102:0/3718630608 wait complete. 2024-05-13T22:56:50.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.921+0000 7f2b772a4700 1 Processor -- start 2024-05-13T22:56:50.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.921+0000 7f2b772a4700 1 -- start start 2024-05-13T22:56:50.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.921+0000 7f2b772a4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2b700ffa40 0x7f2b7019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:50.899 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.921+0000 7f2b772a4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b70105a40 0x7f2b7019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.921+0000 7f2b772a4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7019d150 con 0x7f2b70105a40 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.921+0000 7f2b772a4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70196670 con 0x7f2b700ffa40 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.921+0000 7f2b7483f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b70105a40 0x7f2b7019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b7483f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b70105a40 0x7f2b7019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40726/0 (socket says 172.21.15.102:40726) 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b7483f700 1 -- 172.21.15.102:0/2370130020 learned_addr learned my addr 172.21.15.102:0/2370130020 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b75040700 1 --2- 172.21.15.102:0/2370130020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2b700ffa40 0x7f2b7019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b7483f700 1 -- 172.21.15.102:0/2370130020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2b700ffa40 msgr2=0x7f2b7019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b7483f700 1 --2- 172.21.15.102:0/2370130020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2b700ffa40 0x7f2b7019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b7483f700 1 -- 172.21.15.102:0/2370130020 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c0095f0 con 0x7f2b70105a40 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b7483f700 1 --2- 172.21.15.102:0/2370130020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b70105a40 0x7f2b7019cb30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000e920 tx=0x7f2b6000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b667fc700 1 -- 172.21.15.102:0/2370130020 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b6000c9d0 con 0x7f2b70105a40 2024-05-13T22:56:50.900 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b667fc700 1 -- 172.21.15.102:0/2370130020 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b60010430 con 0x7f2b70105a40 2024-05-13T22:56:50.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b667fc700 1 -- 172.21.15.102:0/2370130020 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b60018630 con 0x7f2b70105a40 2024-05-13T22:56:50.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b772a4700 1 -- 172.21.15.102:0/2370130020 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b70196950 con 0x7f2b70105a40 2024-05-13T22:56:50.901 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.922+0000 7f2b772a4700 1 -- 172.21.15.102:0/2370130020 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b70196ea0 con 0x7f2b70105a40 2024-05-13T22:56:50.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.923+0000 7f2b772a4700 1 -- 172.21.15.102:0/2370130020 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b70109050 con 0x7f2b70105a40 2024-05-13T22:56:50.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.924+0000 7f2b667fc700 1 -- 172.21.15.102:0/2370130020 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2b600105a0 con 0x7f2b70105a40 2024-05-13T22:56:50.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.924+0000 7f2b667fc700 1 --2- 172.21.15.102:0/2370130020 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2b5c06e200 0x7f2b5c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:50.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.924+0000 7f2b75040700 1 --2- 172.21.15.102:0/2370130020 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2b5c06e200 0x7f2b5c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:50.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.924+0000 7f2b667fc700 1 -- 172.21.15.102:0/2370130020 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f2b60014070 con 0x7f2b70105a40 2024-05-13T22:56:50.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.925+0000 7f2b75040700 1 --2- 172.21.15.102:0/2370130020 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2b5c06e200 0x7f2b5c0706c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0059d0 tx=0x7f2b6c005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:50.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:50.929+0000 7f2b667fc700 1 -- 172.21.15.102:0/2370130020 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2b60053850 con 0x7f2b70105a40 2024-05-13T22:56:51.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.211+0000 7f2b772a4700 1 -- 172.21.15.102:0/2370130020 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2b7004ea90 con 0x7f2b70105a40 2024-05-13T22:56:51.187 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.211+0000 7f2b667fc700 1 -- 172.21.15.102:0/2370130020 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f2b6005b260 con 0x7f2b70105a40 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 5 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:56:51.188 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:56:51.189 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-05-13T22:56:51.189 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:56:51.189 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:56:51.189 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-05-13T22:56:51.189 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 9 2024-05-13T22:56:51.189 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:56:51.189 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:56:51.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.213+0000 7f2b772a4700 1 -- 172.21.15.102:0/2370130020 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2b5c06e200 msgr2=0x7f2b5c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.213+0000 7f2b772a4700 1 --2- 172.21.15.102:0/2370130020 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2b5c06e200 0x7f2b5c0706c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0059d0 tx=0x7f2b6c005940 comp rx=0 tx=0).stop 2024-05-13T22:56:51.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.213+0000 7f2b772a4700 1 -- 172.21.15.102:0/2370130020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b70105a40 msgr2=0x7f2b7019cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.213+0000 7f2b772a4700 1 --2- 172.21.15.102:0/2370130020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b70105a40 0x7f2b7019cb30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000e920 tx=0x7f2b6000ec30 comp rx=0 tx=0).stop 2024-05-13T22:56:51.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.213+0000 7f2b772a4700 1 -- 172.21.15.102:0/2370130020 shutdown_connections 2024-05-13T22:56:51.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.213+0000 7f2b772a4700 1 --2- 172.21.15.102:0/2370130020 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2b5c06e200 0x7f2b5c0706c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.213+0000 7f2b772a4700 1 --2- 172.21.15.102:0/2370130020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2b700ffa40 0x7f2b7019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.213+0000 7f2b772a4700 1 --2- 172.21.15.102:0/2370130020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b70105a40 0x7f2b7019cb30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.213+0000 7f2b772a4700 1 -- 172.21.15.102:0/2370130020 >> 172.21.15.102:0/2370130020 conn(0x7f2b700fb550 msgr2=0x7f2b70069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:51.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.214+0000 7f2b772a4700 1 -- 172.21.15.102:0/2370130020 shutdown_connections 2024-05-13T22:56:51.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.214+0000 7f2b772a4700 1 -- 172.21.15.102:0/2370130020 wait complete. 2024-05-13T22:56:51.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.319+0000 7f1b99bda700 1 -- 172.21.15.102:0/2275916137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b94100dc0 msgr2=0x7f1b941011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.319+0000 7f1b99bda700 1 --2- 172.21.15.102:0/2275916137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b94100dc0 0x7f1b941011a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b0a0 tx=0x7f1b8c00b3b0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.319+0000 7f1b99bda700 1 -- 172.21.15.102:0/2275916137 shutdown_connections 2024-05-13T22:56:51.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.319+0000 7f1b99bda700 1 --2- 172.21.15.102:0/2275916137 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b94101770 0x7f1b941057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.319+0000 7f1b99bda700 1 --2- 172.21.15.102:0/2275916137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b94100dc0 0x7f1b941011a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.319+0000 7f1b99bda700 1 -- 172.21.15.102:0/2275916137 >> 172.21.15.102:0/2275916137 conn(0x7f1b940fc670 msgr2=0x7f1b940fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:51.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.319+0000 7f1b99bda700 1 -- 172.21.15.102:0/2275916137 shutdown_connections 2024-05-13T22:56:51.296 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.319+0000 7f1b99bda700 1 -- 172.21.15.102:0/2275916137 wait complete. 2024-05-13T22:56:51.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.320+0000 7f1b99bda700 1 Processor -- start 2024-05-13T22:56:51.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.320+0000 7f1b99bda700 1 -- start start 2024-05-13T22:56:51.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.320+0000 7f1b99bda700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b94100dc0 0x7f1b94073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.320+0000 7f1b99bda700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b94101770 0x7f1b940735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.320+0000 7f1b99bda700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b94073b10 con 0x7f1b94101770 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.320+0000 7f1b99bda700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b94073c50 con 0x7f1b94100dc0 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.320+0000 7f1b92ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b94101770 0x7f1b940735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.320+0000 7f1b92ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b94101770 0x7f1b940735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40746/0 (socket says 172.21.15.102:40746) 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.320+0000 7f1b92ffd700 1 -- 172.21.15.102:0/1021235297 learned_addr learned my addr 172.21.15.102:0/1021235297 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.320+0000 7f1b937fe700 1 --2- 172.21.15.102:0/1021235297 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b94100dc0 0x7f1b94073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.321+0000 7f1b92ffd700 1 -- 172.21.15.102:0/1021235297 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b94100dc0 msgr2=0x7f1b94073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.321+0000 7f1b92ffd700 1 --2- 172.21.15.102:0/1021235297 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b94100dc0 0x7f1b94073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.321+0000 7f1b92ffd700 1 -- 172.21.15.102:0/1021235297 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c009d00 con 0x7f1b94101770 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.321+0000 7f1b92ffd700 1 --2- 172.21.15.102:0/1021235297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b94101770 0x7f1b940735d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00d900 tx=0x7f1b7c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.321+0000 7f1b90ff9700 1 -- 172.21.15.102:0/1021235297 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b7c009900 con 0x7f1b94101770 2024-05-13T22:56:51.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.321+0000 7f1b90ff9700 1 -- 172.21.15.102:0/1021235297 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b7c010460 con 0x7f1b94101770 2024-05-13T22:56:51.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.321+0000 7f1b99bda700 1 -- 172.21.15.102:0/1021235297 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b941a29c0 con 0x7f1b94101770 2024-05-13T22:56:51.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.321+0000 7f1b90ff9700 1 -- 172.21.15.102:0/1021235297 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b7c00f5f0 con 0x7f1b94101770 2024-05-13T22:56:51.299 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.321+0000 7f1b99bda700 1 -- 172.21.15.102:0/1021235297 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b941a2dd0 con 0x7f1b94101770 2024-05-13T22:56:51.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.322+0000 7f1b99bda700 1 -- 172.21.15.102:0/1021235297 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b94109160 con 0x7f1b94101770 2024-05-13T22:56:51.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.323+0000 7f1b90ff9700 1 -- 172.21.15.102:0/1021235297 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1b7c0105d0 con 0x7f1b94101770 2024-05-13T22:56:51.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.323+0000 7f1b90ff9700 1 --2- 172.21.15.102:0/1021235297 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1b8006e200 0x7f1b800706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:51.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.323+0000 7f1b90ff9700 1 -- 172.21.15.102:0/1021235297 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f1b7c08ddd0 con 0x7f1b94101770 2024-05-13T22:56:51.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.323+0000 7f1b937fe700 1 --2- 172.21.15.102:0/1021235297 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1b8006e200 0x7f1b800706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:51.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.324+0000 7f1b937fe700 1 --2- 172.21.15.102:0/1021235297 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1b8006e200 0x7f1b800706c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c000f80 tx=0x7f1b8c0060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:51.304 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.328+0000 7f1b90ff9700 1 -- 172.21.15.102:0/1021235297 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1b7c057040 con 0x7f1b94101770 2024-05-13T22:56:51.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:51 smithi130 ceph-mon[84265]: pgmap v161: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-13T22:56:51.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:51 smithi130 ceph-mon[84265]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:51.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:51 smithi102 ceph-mon[111694]: pgmap v161: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-13T22:56:51.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:51 smithi102 ceph-mon[111694]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:51.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.571+0000 7f1b99bda700 1 -- 172.21.15.102:0/1021235297 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1b94073e10 con 0x7f1b94101770 2024-05-13T22:56:51.548 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.572+0000 7f1b90ff9700 1 -- 172.21.15.102:0/1021235297 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1909 (secure 0 0 0) 0x7f1b7c017790 con 0x7f1b94101770 2024-05-13T22:56:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:e17 2024-05-13T22:56:51.548 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 17 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:45:28.142444+0000 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:56:51.549 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 47 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=14498} 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:56:51.550 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 0 members: 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:14514} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/1826944061,v1:172.21.15.102:6827/1826944061] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:56:51.551 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:56:51.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.574+0000 7f1b99bda700 1 -- 172.21.15.102:0/1021235297 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1b8006e200 msgr2=0x7f1b800706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.574+0000 7f1b99bda700 1 --2- 172.21.15.102:0/1021235297 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1b8006e200 0x7f1b800706c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c000f80 tx=0x7f1b8c0060c0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.574+0000 7f1b99bda700 1 -- 172.21.15.102:0/1021235297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b94101770 msgr2=0x7f1b940735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.574+0000 7f1b99bda700 1 --2- 172.21.15.102:0/1021235297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b94101770 0x7f1b940735d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00d900 tx=0x7f1b7c00dc10 comp rx=0 tx=0).stop 2024-05-13T22:56:51.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.574+0000 7f1b99bda700 1 -- 172.21.15.102:0/1021235297 shutdown_connections 2024-05-13T22:56:51.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.574+0000 7f1b99bda700 1 --2- 172.21.15.102:0/1021235297 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1b8006e200 0x7f1b800706c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.574+0000 7f1b99bda700 1 --2- 172.21.15.102:0/1021235297 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b94100dc0 0x7f1b94073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.574+0000 7f1b99bda700 1 --2- 172.21.15.102:0/1021235297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b94101770 0x7f1b940735d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.574+0000 7f1b99bda700 1 -- 172.21.15.102:0/1021235297 >> 172.21.15.102:0/1021235297 conn(0x7f1b940fc670 msgr2=0x7f1b940fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:51.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.575+0000 7f1b99bda700 1 -- 172.21.15.102:0/1021235297 shutdown_connections 2024-05-13T22:56:51.553 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.575+0000 7f1b99bda700 1 -- 172.21.15.102:0/1021235297 wait complete. 2024-05-13T22:56:51.553 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 17 2024-05-13T22:56:51.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.681+0000 7fa7d425d700 1 -- 172.21.15.102:0/3293364589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7cc101770 msgr2=0x7fa7cc1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.681+0000 7fa7d425d700 1 --2- 172.21.15.102:0/3293364589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7cc101770 0x7fa7cc1057c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009910 tx=0x7fa7bc009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.681+0000 7fa7d425d700 1 -- 172.21.15.102:0/3293364589 shutdown_connections 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.681+0000 7fa7d425d700 1 --2- 172.21.15.102:0/3293364589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7cc101770 0x7fa7cc1057c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.681+0000 7fa7d425d700 1 --2- 172.21.15.102:0/3293364589 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7cc100dc0 0x7fa7cc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.681+0000 7fa7d425d700 1 -- 172.21.15.102:0/3293364589 >> 172.21.15.102:0/3293364589 conn(0x7fa7cc0fc650 msgr2=0x7fa7cc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.681+0000 7fa7d425d700 1 -- 172.21.15.102:0/3293364589 shutdown_connections 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.681+0000 7fa7d425d700 1 -- 172.21.15.102:0/3293364589 wait complete. 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d425d700 1 Processor -- start 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d425d700 1 -- start start 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d425d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7cc100dc0 0x7fa7cc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d425d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7cc101770 0x7fa7cc19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:51.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d425d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc19d110 con 0x7fa7cc101770 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d425d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1965c0 con 0x7fa7cc100dc0 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d17f8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7cc101770 0x7fa7cc19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d17f8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7cc101770 0x7fa7cc19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40754/0 (socket says 172.21.15.102:40754) 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d17f8700 1 -- 172.21.15.102:0/713427591 learned_addr learned my addr 172.21.15.102:0/713427591 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d1ff9700 1 --2- 172.21.15.102:0/713427591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7cc100dc0 0x7fa7cc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d17f8700 1 -- 172.21.15.102:0/713427591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7cc100dc0 msgr2=0x7fa7cc19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d17f8700 1 --2- 172.21.15.102:0/713427591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7cc100dc0 0x7fa7cc19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d17f8700 1 -- 172.21.15.102:0/713427591 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc0095f0 con 0x7fa7cc101770 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7d17f8700 1 --2- 172.21.15.102:0/713427591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7cc101770 0x7fa7cc19ca80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0059d0 tx=0x7fa7bc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.682+0000 7fa7c2ffd700 1 -- 172.21.15.102:0/713427591 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7bc01d070 con 0x7fa7cc101770 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.683+0000 7fa7c2ffd700 1 -- 172.21.15.102:0/713427591 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7bc00b610 con 0x7fa7cc101770 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.683+0000 7fa7d425d700 1 -- 172.21.15.102:0/713427591 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7cc196840 con 0x7fa7cc101770 2024-05-13T22:56:51.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.683+0000 7fa7c2ffd700 1 -- 172.21.15.102:0/713427591 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7bc022470 con 0x7fa7cc101770 2024-05-13T22:56:51.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.683+0000 7fa7d425d700 1 -- 172.21.15.102:0/713427591 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc196d30 con 0x7fa7cc101770 2024-05-13T22:56:51.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.684+0000 7fa7d425d700 1 -- 172.21.15.102:0/713427591 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7cc1091c0 con 0x7fa7cc101770 2024-05-13T22:56:51.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.684+0000 7fa7c2ffd700 1 -- 172.21.15.102:0/713427591 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa7bc00fc70 con 0x7fa7cc101770 2024-05-13T22:56:51.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.685+0000 7fa7c2ffd700 1 --2- 172.21.15.102:0/713427591 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa7b806e200 0x7fa7b80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:51.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.685+0000 7fa7c2ffd700 1 -- 172.21.15.102:0/713427591 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fa7bc08f5b0 con 0x7fa7cc101770 2024-05-13T22:56:51.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.685+0000 7fa7d1ff9700 1 --2- 172.21.15.102:0/713427591 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa7b806e200 0x7fa7b80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:51.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.685+0000 7fa7d1ff9700 1 --2- 172.21.15.102:0/713427591 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa7b806e200 0x7fa7b80706c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009bf0 tx=0x7fa7c8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:51.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.689+0000 7fa7c2ffd700 1 -- 172.21.15.102:0/713427591 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa7bc05c3d0 con 0x7fa7cc101770 2024-05-13T22:56:51.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.908+0000 7fa7d425d700 1 -- 172.21.15.102:0/713427591 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7cc02ccf0 con 0x7fa7b806e200 2024-05-13T22:56:51.885 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.909+0000 7fa7c2ffd700 1 -- 172.21.15.102:0/713427591 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa7cc02ccf0 con 0x7fa7b806e200 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr", 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout: "mon" 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "11/23 daemons upgraded", 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading osd daemons", 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:56:51.886 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:56:51.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.911+0000 7fa7d425d700 1 -- 172.21.15.102:0/713427591 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa7b806e200 msgr2=0x7fa7b80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.911+0000 7fa7d425d700 1 --2- 172.21.15.102:0/713427591 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa7b806e200 0x7fa7b80706c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009bf0 tx=0x7fa7c8009260 comp rx=0 tx=0).stop 2024-05-13T22:56:51.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.911+0000 7fa7d425d700 1 -- 172.21.15.102:0/713427591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7cc101770 msgr2=0x7fa7cc19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.911+0000 7fa7d425d700 1 --2- 172.21.15.102:0/713427591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7cc101770 0x7fa7cc19ca80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0059d0 tx=0x7fa7bc00f740 comp rx=0 tx=0).stop 2024-05-13T22:56:51.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.911+0000 7fa7d425d700 1 -- 172.21.15.102:0/713427591 shutdown_connections 2024-05-13T22:56:51.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.911+0000 7fa7d425d700 1 --2- 172.21.15.102:0/713427591 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa7b806e200 0x7fa7b80706c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.911+0000 7fa7d425d700 1 --2- 172.21.15.102:0/713427591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7cc100dc0 0x7fa7cc19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.911+0000 7fa7d425d700 1 --2- 172.21.15.102:0/713427591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7cc101770 0x7fa7cc19ca80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.911+0000 7fa7d425d700 1 -- 172.21.15.102:0/713427591 >> 172.21.15.102:0/713427591 conn(0x7fa7cc0fc650 msgr2=0x7fa7cc0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:51.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.911+0000 7fa7d425d700 1 -- 172.21.15.102:0/713427591 shutdown_connections 2024-05-13T22:56:51.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:51.911+0000 7fa7d425d700 1 -- 172.21.15.102:0/713427591 wait complete. 2024-05-13T22:56:51.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.015+0000 7fb921dd8700 1 -- 172.21.15.102:0/3250872112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0ff400 msgr2=0x7fb91c0ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.015+0000 7fb921dd8700 1 --2- 172.21.15.102:0/3250872112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0ff400 0x7fb91c0ff7e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009910 tx=0x7fb90c009c20 comp rx=0 tx=0).stop 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.015+0000 7fb921dd8700 1 -- 172.21.15.102:0/3250872112 shutdown_connections 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.015+0000 7fb921dd8700 1 --2- 172.21.15.102:0/3250872112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb91c0ffdb0 0x7fb91c103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.015+0000 7fb921dd8700 1 --2- 172.21.15.102:0/3250872112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0ff400 0x7fb91c0ff7e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.015+0000 7fb921dd8700 1 -- 172.21.15.102:0/3250872112 >> 172.21.15.102:0/3250872112 conn(0x7fb91c075e30 msgr2=0x7fb91c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.015+0000 7fb921dd8700 1 -- 172.21.15.102:0/3250872112 shutdown_connections 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.015+0000 7fb921dd8700 1 -- 172.21.15.102:0/3250872112 wait complete. 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.016+0000 7fb921dd8700 1 Processor -- start 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.016+0000 7fb921dd8700 1 -- start start 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.016+0000 7fb921dd8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0ff400 0x7fb91c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.016+0000 7fb921dd8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb91c0ffdb0 0x7fb91c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:51.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.016+0000 7fb921dd8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c19d0f0 con 0x7fb91c0ff400 2024-05-13T22:56:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.016+0000 7fb921dd8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c1965a0 con 0x7fb91c0ffdb0 2024-05-13T22:56:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb91b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0ff400 0x7fb91c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb91b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0ff400 0x7fb91c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:40764/0 (socket says 172.21.15.102:40764) 2024-05-13T22:56:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb91b7fe700 1 -- 172.21.15.102:0/2820442507 learned_addr learned my addr 172.21.15.102:0/2820442507 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:56:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb91affd700 1 --2- 172.21.15.102:0/2820442507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb91c0ffdb0 0x7fb91c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb91b7fe700 1 -- 172.21.15.102:0/2820442507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb91c0ffdb0 msgr2=0x7fb91c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb91b7fe700 1 --2- 172.21.15.102:0/2820442507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb91c0ffdb0 0x7fb91c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb91b7fe700 1 -- 172.21.15.102:0/2820442507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c0095f0 con 0x7fb91c0ff400 2024-05-13T22:56:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb91b7fe700 1 --2- 172.21.15.102:0/2820442507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0ff400 0x7fb91c19c520 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006010 tx=0x7fb90c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb918ff9700 1 -- 172.21.15.102:0/2820442507 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb90c01d070 con 0x7fb91c0ff400 2024-05-13T22:56:51.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb921dd8700 1 -- 172.21.15.102:0/2820442507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91c196820 con 0x7fb91c0ff400 2024-05-13T22:56:51.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb918ff9700 1 -- 172.21.15.102:0/2820442507 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb90c00f460 con 0x7fb91c0ff400 2024-05-13T22:56:51.996 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb918ff9700 1 -- 172.21.15.102:0/2820442507 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb90c017630 con 0x7fb91c0ff400 2024-05-13T22:56:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.017+0000 7fb921dd8700 1 -- 172.21.15.102:0/2820442507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91c196d10 con 0x7fb91c0ff400 2024-05-13T22:56:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.018+0000 7fb921dd8700 1 -- 172.21.15.102:0/2820442507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91c1097b0 con 0x7fb91c0ff400 2024-05-13T22:56:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.019+0000 7fb918ff9700 1 -- 172.21.15.102:0/2820442507 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb90c00f5d0 con 0x7fb91c0ff400 2024-05-13T22:56:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.019+0000 7fb918ff9700 1 --2- 172.21.15.102:0/2820442507 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb90406e200 0x7fb9040706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:56:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.019+0000 7fb918ff9700 1 -- 172.21.15.102:0/2820442507 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fb90c08f690 con 0x7fb91c0ff400 2024-05-13T22:56:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.019+0000 7fb91affd700 1 --2- 172.21.15.102:0/2820442507 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb90406e200 0x7fb9040706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:56:51.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.020+0000 7fb91affd700 1 --2- 172.21.15.102:0/2820442507 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb90406e200 0x7fb9040706c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb910005950 tx=0x7fb9100058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:56:52.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.024+0000 7fb918ff9700 1 -- 172.21.15.102:0/2820442507 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb90c05c430 con 0x7fb91c0ff400 2024-05-13T22:56:52.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:52 smithi102 ceph-mon[111694]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:52.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:52 smithi102 ceph-mon[111694]: from='client.34356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:52.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:52 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2370130020' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:52.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:52 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1021235297' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:56:52.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.303+0000 7fb921dd8700 1 -- 172.21.15.102:0/2820442507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb91c04ea90 con 0x7fb91c0ff400 2024-05-13T22:56:52.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.303+0000 7fb918ff9700 1 -- 172.21.15.102:0/2820442507 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb90c05bfc0 con 0x7fb91c0ff400 2024-05-13T22:56:52.279 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:56:52.281 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.305+0000 7fb921dd8700 1 -- 172.21.15.102:0/2820442507 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb90406e200 msgr2=0x7fb9040706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:52.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.305+0000 7fb921dd8700 1 --2- 172.21.15.102:0/2820442507 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb90406e200 0x7fb9040706c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb910005950 tx=0x7fb9100058e0 comp rx=0 tx=0).stop 2024-05-13T22:56:52.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.305+0000 7fb921dd8700 1 -- 172.21.15.102:0/2820442507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0ff400 msgr2=0x7fb91c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:56:52.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.305+0000 7fb921dd8700 1 --2- 172.21.15.102:0/2820442507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0ff400 0x7fb91c19c520 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006010 tx=0x7fb90c00be20 comp rx=0 tx=0).stop 2024-05-13T22:56:52.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.305+0000 7fb921dd8700 1 -- 172.21.15.102:0/2820442507 shutdown_connections 2024-05-13T22:56:52.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.305+0000 7fb921dd8700 1 --2- 172.21.15.102:0/2820442507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb91c0ff400 0x7fb91c19c520 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:52.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.305+0000 7fb921dd8700 1 --2- 172.21.15.102:0/2820442507 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb90406e200 0x7fb9040706c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:52.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.305+0000 7fb921dd8700 1 --2- 172.21.15.102:0/2820442507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb91c0ffdb0 0x7fb91c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:56:52.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.305+0000 7fb921dd8700 1 -- 172.21.15.102:0/2820442507 >> 172.21.15.102:0/2820442507 conn(0x7fb91c075e30 msgr2=0x7fb91c0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:56:52.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.305+0000 7fb921dd8700 1 -- 172.21.15.102:0/2820442507 shutdown_connections 2024-05-13T22:56:52.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:56:52.305+0000 7fb921dd8700 1 -- 172.21.15.102:0/2820442507 wait complete. 2024-05-13T22:56:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:52 smithi130 ceph-mon[84265]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:52 smithi130 ceph-mon[84265]: from='client.34356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:52 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2370130020' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:56:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:52 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1021235297' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:56:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:53 smithi130 ceph-mon[84265]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:53 smithi130 ceph-mon[84265]: pgmap v162: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 943 B/s rd, 1 op/s 2024-05-13T22:56:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:53 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2820442507' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:56:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:53 smithi102 ceph-mon[111694]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:56:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:53 smithi102 ceph-mon[111694]: pgmap v162: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 943 B/s rd, 1 op/s 2024-05-13T22:56:53.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:53 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2820442507' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:56:55.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:55 smithi130 ceph-mon[84265]: pgmap v163: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 951 B/s rd, 1 op/s 2024-05-13T22:56:55.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:55 smithi102 ceph-mon[111694]: pgmap v163: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 951 B/s rd, 1 op/s 2024-05-13T22:56:57.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:57 smithi130 ceph-mon[84265]: pgmap v164: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:56:57.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:57 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:57.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:57 smithi102 ceph-mon[111694]: pgmap v164: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:56:57.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:57 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:56:58.353 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:58 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-13T22:56:58.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:58 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-13T22:56:59.323 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:59 smithi130 ceph-mon[84265]: pgmap v165: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:56:59.323 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:59 smithi130 ceph-mon[84265]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-13T22:56:59.323 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:56:59 smithi130 ceph-mon[84265]: Upgrade: osd.5 is safe to restart 2024-05-13T22:56:59.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:59 smithi102 ceph-mon[111694]: pgmap v165: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:56:59.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:59 smithi102 ceph-mon[111694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-13T22:56:59.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:56:59 smithi102 ceph-mon[111694]: Upgrade: osd.5 is safe to restart 2024-05-13T22:57:00.884 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:00 smithi130 ceph-mon[84265]: Upgrade: Updating osd.5 2024-05-13T22:57:00.884 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:00 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:00.884 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:00 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-13T22:57:00.885 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:00 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:00.885 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:00 smithi130 ceph-mon[84265]: Deploying daemon osd.5 on smithi130 2024-05-13T22:57:00.885 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:00 smithi130 ceph-mon[84265]: pgmap v166: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:57:00.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:00 smithi102 ceph-mon[111694]: Upgrade: Updating osd.5 2024-05-13T22:57:00.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:00 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:00.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:00 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-13T22:57:00.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:00 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:00.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:00 smithi102 ceph-mon[111694]: Deploying daemon osd.5 on smithi130 2024-05-13T22:57:00.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:00 smithi102 ceph-mon[111694]: pgmap v166: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:57:02.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:02.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:57:02.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:02.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:57:03.098 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:02 smithi130 systemd[1]: Stopping Ceph osd.5 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:57:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:03 smithi130 ceph-mon[84265]: pgmap v167: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:57:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:03 smithi130 ceph-mon[84265]: osd.5 marked itself down and dead 2024-05-13T22:57:03.399 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:03 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[56354]: 2024-05-13T22:57:03.121+0000 7f2a06544700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T22:57:03.399 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:03 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[56354]: 2024-05-13T22:57:03.121+0000 7f2a06544700 -1 osd.5 86 *** Got signal Terminated *** 2024-05-13T22:57:03.399 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:03 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[56354]: 2024-05-13T22:57:03.121+0000 7f2a06544700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T22:57:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:03 smithi102 ceph-mon[111694]: pgmap v167: 65 pgs: 65 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:57:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:03 smithi102 ceph-mon[111694]: osd.5 marked itself down and dead 2024-05-13T22:57:03.948 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:03 smithi130 podman[103466]: 2024-05-13 22:57:03.714863752 +0000 UTC m=+0.700190924 container died eb9310deec60132e2630f5e390bb880d226b8e2c751fda7075dcbade796a97b6 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, ceph=True, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T22:57:04.224 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:04 smithi130 ceph-mon[84265]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:57:04.224 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:04 smithi130 ceph-mon[84265]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-13T22:57:04.225 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:04 smithi130 ceph-mon[84265]: osdmap e87: 6 total, 5 up, 6 in 2024-05-13T22:57:04.225 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:04 smithi130 podman[103466]: 2024-05-13 22:57:04.063464241 +0000 UTC m=+1.048791412 container cleanup eb9310deec60132e2630f5e390bb880d226b8e2c751fda7075dcbade796a97b6 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-13T22:57:04.225 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:04 smithi130 bash[103466]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5 2024-05-13T22:57:04.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:04 smithi102 ceph-mon[111694]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-13T22:57:04.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:04 smithi102 ceph-mon[111694]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-13T22:57:04.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:04 smithi102 ceph-mon[111694]: osdmap e87: 6 total, 5 up, 6 in 2024-05-13T22:57:04.649 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:04 smithi130 podman[103481]: 2024-05-13 22:57:04.248488378 +0000 UTC m=+0.526199331 container remove eb9310deec60132e2630f5e390bb880d226b8e2c751fda7075dcbade796a97b6 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222) 2024-05-13T22:57:04.982 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:04 smithi130 podman[103572]: 2024-05-13 22:57:04.587873937 +0000 UTC m=+0.017868031 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:57:04.982 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:04 smithi130 podman[103572]: 2024-05-13 22:57:04.699358217 +0000 UTC m=+0.129352321 container create ba91122f7e7b1a7e3a3780ed4c99761b696d3b112ac8d2d0dc7b92525a3a3591 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=squid-c3e5514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T22:57:04.983 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:04 smithi130 podman[103572]: 2024-05-13 22:57:04.91849854 +0000 UTC m=+0.348492635 container init ba91122f7e7b1a7e3a3780ed4c99761b696d3b112ac8d2d0dc7b92525a3a3591 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=squid-c3e5514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T22:57:04.983 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:04 smithi130 podman[103572]: 2024-05-13 22:57:04.922121188 +0000 UTC m=+0.352115281 container start ba91122f7e7b1a7e3a3780ed4c99761b696d3b112ac8d2d0dc7b92525a3a3591 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-c3e5514, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-13T22:57:04.983 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:05 smithi130 podman[103572]: 2024-05-13 22:57:05.001674245 +0000 UTC m=+0.431668344 container attach ba91122f7e7b1a7e3a3780ed4c99761b696d3b112ac8d2d0dc7b92525a3a3591 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, ceph=True) 2024-05-13T22:57:05.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:05 smithi130 ceph-mon[84265]: pgmap v169: 65 pgs: 13 stale+active+clean, 52 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-13T22:57:05.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:05 smithi130 ceph-mon[84265]: osdmap e88: 6 total, 5 up, 6 in 2024-05-13T22:57:05.399 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:05 smithi130 podman[103572]: 2024-05-13 22:57:05.214545501 +0000 UTC m=+0.644539582 container died ba91122f7e7b1a7e3a3780ed4c99761b696d3b112ac8d2d0dc7b92525a3a3591 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-c3e5514) 2024-05-13T22:57:05.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:05 smithi102 ceph-mon[111694]: pgmap v169: 65 pgs: 13 stale+active+clean, 52 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-13T22:57:05.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:05 smithi102 ceph-mon[111694]: osdmap e88: 6 total, 5 up, 6 in 2024-05-13T22:57:05.983 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:05 smithi130 podman[103598]: 2024-05-13 22:57:05.789018497 +0000 UTC m=+0.567043299 container remove ba91122f7e7b1a7e3a3780ed4c99761b696d3b112ac8d2d0dc7b92525a3a3591 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2) 2024-05-13T22:57:05.984 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:05 smithi130 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.5.service: Deactivated successfully. 2024-05-13T22:57:05.984 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:05 smithi130 systemd[1]: Stopped Ceph osd.5 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:57:05.984 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:05 smithi130 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.5.service: Consumed 1min 7.013s CPU time. 2024-05-13T22:57:05.984 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 systemd[1]: Starting Ceph osd.5 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T22:57:06.326 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:06 smithi130 ceph-mon[84265]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-05-13T22:57:06.327 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:06 smithi130 ceph-mon[84265]: Health check failed: Degraded data redundancy: 4/231 objects degraded (1.732%), 2 pgs degraded (PG_DEGRADED) 2024-05-13T22:57:06.327 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 podman[103698]: 2024-05-13 22:57:06.130553472 +0000 UTC m=+0.017306858 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:57:06.327 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 podman[103698]: 2024-05-13 22:57:06.248316198 +0000 UTC m=+0.135069576 container create f50a58fb5c98bf7e576a600614d9540e0282e62c0679d45f800657797964385a (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-13T22:57:06.649 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 podman[103698]: 2024-05-13 22:57:06.459213766 +0000 UTC m=+0.345967157 container init f50a58fb5c98bf7e576a600614d9540e0282e62c0679d45f800657797964385a (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-c3e5514, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-13T22:57:06.649 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 podman[103698]: 2024-05-13 22:57:06.462738811 +0000 UTC m=+0.349492189 container start f50a58fb5c98bf7e576a600614d9540e0282e62c0679d45f800657797964385a (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-c3e5514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-13T22:57:06.649 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 podman[103698]: 2024-05-13 22:57:06.542412462 +0000 UTC m=+0.429165842 container attach f50a58fb5c98bf7e576a600614d9540e0282e62c0679d45f800657797964385a (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-c3e5514, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-13T22:57:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:06 smithi102 ceph-mon[111694]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-05-13T22:57:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:06 smithi102 ceph-mon[111694]: Health check failed: Degraded data redundancy: 4/231 objects degraded (1.732%), 2 pgs degraded (PG_DEGRADED) 2024-05-13T22:57:07.149 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate[103712]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-13T22:57:07.149 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate[103712]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-13T22:57:07.149 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 bash[103698]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-13T22:57:07.149 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 bash[103698]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-13T22:57:07.150 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 bash[103698]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-13T22:57:07.150 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 bash[103698]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-13T22:57:07.150 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 bash[103698]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-13T22:57:07.150 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 bash[103698]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-13T22:57:07.150 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 bash[103698]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-13T22:57:07.150 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate[103712]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-13T22:57:07.150 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate[103712]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-13T22:57:07.150 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate[103712]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-13T22:57:07.150 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate[103712]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-13T22:57:07.150 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate[103712]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-13T22:57:07.150 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:06 smithi130 podman[103698]: 2024-05-13 22:57:06.939618026 +0000 UTC m=+0.826371403 container died f50a58fb5c98bf7e576a600614d9540e0282e62c0679d45f800657797964385a (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-c3e5514, io.buildah.version=1.33.5) 2024-05-13T22:57:07.507 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:07 smithi130 ceph-mon[84265]: pgmap v171: 65 pgs: 3 active+undersized, 2 active+undersized+degraded, 4 peering, 9 stale+active+clean, 47 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 4/231 objects degraded (1.732%) 2024-05-13T22:57:07.577 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:07 smithi102 ceph-mon[111694]: pgmap v171: 65 pgs: 3 active+undersized, 2 active+undersized+degraded, 4 peering, 9 stale+active+clean, 47 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 4/231 objects degraded (1.732%) 2024-05-13T22:57:07.847 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:07 smithi130 podman[103847]: 2024-05-13 22:57:07.532076312 +0000 UTC m=+0.586786552 container remove f50a58fb5c98bf7e576a600614d9540e0282e62c0679d45f800657797964385a (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-activate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-13T22:57:07.847 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:07 smithi130 podman[103907]: 2024-05-13 22:57:07.64943144 +0000 UTC m=+0.017239379 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T22:57:07.847 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:07 smithi130 podman[103907]: 2024-05-13 22:57:07.766332797 +0000 UTC m=+0.134140735 container create 5d983135967dfba3d8086f23b0b4d81fbfb968ed66316d0c7abf2c53810eb270 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-c3e5514) 2024-05-13T22:57:08.399 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:07 smithi130 podman[103907]: 2024-05-13 22:57:07.986923963 +0000 UTC m=+0.354731883 container init 5d983135967dfba3d8086f23b0b4d81fbfb968ed66316d0c7abf2c53810eb270 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-13T22:57:08.399 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:07 smithi130 podman[103907]: 2024-05-13 22:57:07.990223258 +0000 UTC m=+0.358031177 container start 5d983135967dfba3d8086f23b0b4d81fbfb968ed66316d0c7abf2c53810eb270 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5, RELEASE=squid-c3e5514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-13T22:57:08.399 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:08 smithi130 ceph-osd[103926]: -- 172.21.15.130:0/1285783034 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56391a6f5680 con 0x56391a6bec00 2024-05-13T22:57:08.399 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:08 smithi130 bash[103907]: 5d983135967dfba3d8086f23b0b4d81fbfb968ed66316d0c7abf2c53810eb270 2024-05-13T22:57:08.399 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:08 smithi130 systemd[1]: Started Ceph osd.5 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T22:57:08.899 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:08 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T22:57:08.535+0000 7f4ae02a17c0 -1 Falling back to public interface 2024-05-13T22:57:09.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:09 smithi130 ceph-mon[84265]: pgmap v172: 65 pgs: 13 active+undersized, 12 active+undersized+degraded, 4 peering, 36 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 35/231 objects degraded (15.152%) 2024-05-13T22:57:09.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:09.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:09.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:09 smithi102 ceph-mon[111694]: pgmap v172: 65 pgs: 13 active+undersized, 12 active+undersized+degraded, 4 peering, 36 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 35/231 objects degraded (15.152%) 2024-05-13T22:57:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:09.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:10.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:10 smithi102 ceph-mon[111694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-05-13T22:57:10.523 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:10 smithi130 ceph-mon[84265]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-05-13T22:57:11.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:11 smithi102 ceph-mon[111694]: pgmap v173: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 35/231 objects degraded (15.152%) 2024-05-13T22:57:11.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:11 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:11.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:11 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:11.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:11 smithi130 ceph-mon[84265]: pgmap v173: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 35/231 objects degraded (15.152%) 2024-05-13T22:57:11.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:11 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:11.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:11 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:13.134 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:13 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T22:57:13.013+0000 7f4ae02a17c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-05-13T22:57:13.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:13 smithi130 ceph-mon[84265]: pgmap v174: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/231 objects degraded (15.152%) 2024-05-13T22:57:13.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:13 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:13.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:13 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:13.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:13 smithi102 ceph-mon[111694]: pgmap v174: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/231 objects degraded (15.152%) 2024-05-13T22:57:13.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:13 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:13.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:13 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:13.899 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:13 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T22:57:13.480+0000 7f4ae02a17c0 -1 osd.5 86 log_to_monitors true 2024-05-13T22:57:14.280 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:14 smithi130 ceph-mon[84265]: from='osd.5 [v2:172.21.15.130:6816/3573323293,v1:172.21.15.130:6817/3573323293]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-13T22:57:14.280 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:14 smithi130 ceph-mon[84265]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-13T22:57:14.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:14 smithi102 ceph-mon[111694]: from='osd.5 [v2:172.21.15.130:6816/3573323293,v1:172.21.15.130:6817/3573323293]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-13T22:57:14.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:14 smithi102 ceph-mon[111694]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-13T22:57:15.399 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 22:57:15 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T22:57:15.058+0000 7f4acd6f0700 -1 osd.5 86 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-13T22:57:15.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:15 smithi130 ceph-mon[84265]: pgmap v175: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 35/231 objects degraded (15.152%) 2024-05-13T22:57:15.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:15 smithi130 ceph-mon[84265]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-13T22:57:15.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:15 smithi130 ceph-mon[84265]: osdmap e89: 6 total, 5 up, 6 in 2024-05-13T22:57:15.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:15 smithi130 ceph-mon[84265]: from='osd.5 [v2:172.21.15.130:6816/3573323293,v1:172.21.15.130:6817/3573323293]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:57:15.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:15 smithi130 ceph-mon[84265]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:57:15.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:15 smithi130 ceph-mon[84265]: from='osd.5 ' entity='osd.5' 2024-05-13T22:57:15.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:15 smithi102 ceph-mon[111694]: pgmap v175: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 212 MiB data, 984 MiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 35/231 objects degraded (15.152%) 2024-05-13T22:57:15.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:15 smithi102 ceph-mon[111694]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-13T22:57:15.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:15 smithi102 ceph-mon[111694]: osdmap e89: 6 total, 5 up, 6 in 2024-05-13T22:57:15.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:15 smithi102 ceph-mon[111694]: from='osd.5 [v2:172.21.15.130:6816/3573323293,v1:172.21.15.130:6817/3573323293]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:57:15.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:15 smithi102 ceph-mon[111694]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-05-13T22:57:15.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:15 smithi102 ceph-mon[111694]: from='osd.5 ' entity='osd.5' 2024-05-13T22:57:17.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:17.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: Upgrade: Setting container_image for all osd 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-13T22:57:17.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: Upgrade: Setting require_osd_release to 19 squid 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: osd.5 [v2:172.21.15.130:6816/3573323293,v1:172.21.15.130:6817/3573323293] boot 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: osdmap e90: 6 total, 6 up, 6 in 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:57:17.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: pgmap v178: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 212 MiB data, 904 MiB used, 536 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 35/231 objects degraded (15.152%) 2024-05-13T22:57:17.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:17.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:57:17.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:16 smithi130 ceph-mon[84265]: Health check update: Degraded data redundancy: 35/231 objects degraded (15.152%), 12 pgs degraded (PG_DEGRADED) 2024-05-13T22:57:17.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:17.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:17.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:17.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:57:17.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:17.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:17.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:17.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:17.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: Upgrade: Setting container_image for all osd 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-13T22:57:17.178 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: Upgrade: Setting require_osd_release to 19 squid 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: osd.5 [v2:172.21.15.130:6816/3573323293,v1:172.21.15.130:6817/3573323293] boot 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: osdmap e90: 6 total, 6 up, 6 in 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: pgmap v178: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 212 MiB data, 904 MiB used, 536 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 35/231 objects degraded (15.152%) 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:57:17.179 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:16 smithi102 ceph-mon[111694]: Health check update: Degraded data redundancy: 35/231 objects degraded (15.152%), 12 pgs degraded (PG_DEGRADED) 2024-05-13T22:57:18.095 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:17 smithi102 ceph-mon[111694]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-13T22:57:18.095 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:17 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-13T22:57:18.095 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:17 smithi102 ceph-mon[111694]: osdmap e91: 6 total, 6 up, 6 in 2024-05-13T22:57:18.095 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:17 smithi102 ceph-mon[111694]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-13T22:57:18.095 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:17 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:18.095 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:17 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-13T22:57:18.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:17 smithi130 ceph-mon[84265]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-13T22:57:18.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:17 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-13T22:57:18.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:17 smithi130 ceph-mon[84265]: osdmap e91: 6 total, 6 up, 6 in 2024-05-13T22:57:18.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:17 smithi130 ceph-mon[84265]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-13T22:57:18.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:17 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:18.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:17 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-13T22:57:19.124 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: osdmap e92: 6 total, 6 up, 6 in 2024-05-13T22:57:19.124 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-13T22:57:19.124 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: pgmap v181: 65 pgs: 5 peering, 13 active+undersized, 5 active+undersized+degraded, 42 active+clean; 212 MiB data, 904 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 14/231 objects degraded (6.061%) 2024-05-13T22:57:19.124 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 2 up:standby 2024-05-13T22:57:19.124 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:19.125 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:19.125 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:57:19.125 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:19.125 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:19.125 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:19.125 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:19.125 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:19.125 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:18 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:19.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: osdmap e92: 6 total, 6 up, 6 in 2024-05-13T22:57:19.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-13T22:57:19.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: pgmap v181: 65 pgs: 5 peering, 13 active+undersized, 5 active+undersized+degraded, 42 active+clean; 212 MiB data, 904 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 14/231 objects degraded (6.061%) 2024-05-13T22:57:19.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 2 up:standby 2024-05-13T22:57:19.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:19.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:19.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:57:19.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:19.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:19.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:19.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:19.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:19.150 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:18 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:20.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:20 smithi130 ceph-mon[84265]: mds.? [v2:172.21.15.102:6826/1720794729,v1:172.21.15.102:6827/1720794729] up:boot 2024-05-13T22:57:20.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:20 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 3 up:standby 2024-05-13T22:57:20.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:57:20.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:20.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.cgkcwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:57:20.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:20.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:20 smithi102 ceph-mon[111694]: mds.? [v2:172.21.15.102:6826/1720794729,v1:172.21.15.102:6827/1720794729] up:boot 2024-05-13T22:57:20.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:20 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 3 up:standby 2024-05-13T22:57:20.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:57:20.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:20.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.cgkcwz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:57:20.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:21.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:21 smithi130 ceph-mon[84265]: Upgrade: Updating mds.cephfs.smithi102.cgkcwz 2024-05-13T22:57:21.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:21 smithi130 ceph-mon[84265]: Deploying daemon mds.cephfs.smithi102.cgkcwz on smithi102 2024-05-13T22:57:21.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:21 smithi130 ceph-mon[84265]: pgmap v182: 65 pgs: 5 peering, 3 active+undersized, 1 active+undersized+degraded, 56 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 522 B/s rd, 1 op/s; 3/231 objects degraded (1.299%) 2024-05-13T22:57:21.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:21 smithi102 ceph-mon[111694]: Upgrade: Updating mds.cephfs.smithi102.cgkcwz 2024-05-13T22:57:21.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:21 smithi102 ceph-mon[111694]: Deploying daemon mds.cephfs.smithi102.cgkcwz on smithi102 2024-05-13T22:57:21.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:21 smithi102 ceph-mon[111694]: pgmap v182: 65 pgs: 5 peering, 3 active+undersized, 1 active+undersized+degraded, 56 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 522 B/s rd, 1 op/s; 3/231 objects degraded (1.299%) 2024-05-13T22:57:22.149 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:22 smithi102 ceph-mon[111694]: Health check update: Degraded data redundancy: 3/231 objects degraded (1.299%), 1 pg degraded (PG_DEGRADED) 2024-05-13T22:57:22.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.412+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/721227253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3b0101c70 msgr2=0x7ff3b0102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:22.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.412+0000 7ff3b7e2f700 1 --2- 172.21.15.102:0/721227253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3b0101c70 0x7ff3b0102050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009910 tx=0x7ff3ac009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:22.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.412+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/721227253 shutdown_connections 2024-05-13T22:57:22.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.412+0000 7ff3b7e2f700 1 --2- 172.21.15.102:0/721227253 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3b0102620 0x7ff3b010ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.412+0000 7ff3b7e2f700 1 --2- 172.21.15.102:0/721227253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3b0101c70 0x7ff3b0102050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.412+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/721227253 >> 172.21.15.102:0/721227253 conn(0x7ff3b00fb530 msgr2=0x7ff3b00fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:22.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.412+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/721227253 shutdown_connections 2024-05-13T22:57:22.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.412+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/721227253 wait complete. 2024-05-13T22:57:22.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b7e2f700 1 Processor -- start 2024-05-13T22:57:22.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b7e2f700 1 -- start start 2024-05-13T22:57:22.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b7e2f700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3b0101c70 0x7ff3b019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:22.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b7e2f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3b0102620 0x7ff3b019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:22.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b7e2f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b019d250 con 0x7ff3b0102620 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b7e2f700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0196700 con 0x7ff3b0101c70 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b53ca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3b0102620 0x7ff3b019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b53ca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3b0102620 0x7ff3b019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42312/0 (socket says 172.21.15.102:42312) 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b53ca700 1 -- 172.21.15.102:0/789614925 learned_addr learned my addr 172.21.15.102:0/789614925 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b5bcb700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3b0101c70 0x7ff3b019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b53ca700 1 -- 172.21.15.102:0/789614925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3b0101c70 msgr2=0x7ff3b019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b53ca700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3b0101c70 0x7ff3b019c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b53ca700 1 -- 172.21.15.102:0/789614925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0095f0 con 0x7ff3b0102620 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.413+0000 7ff3b5bcb700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3b0101c70 0x7ff3b019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.414+0000 7ff3b53ca700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3b0102620 0x7ff3b019cbc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000b480 tx=0x7ff3a000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.414+0000 7ff3a6ffd700 1 -- 172.21.15.102:0/789614925 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3a00118c0 con 0x7ff3b0102620 2024-05-13T22:57:22.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.414+0000 7ff3a6ffd700 1 -- 172.21.15.102:0/789614925 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3a0010380 con 0x7ff3b0102620 2024-05-13T22:57:22.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.414+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/789614925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b01969e0 con 0x7ff3b0102620 2024-05-13T22:57:22.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.414+0000 7ff3a6ffd700 1 -- 172.21.15.102:0/789614925 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3a000f4f0 con 0x7ff3b0102620 2024-05-13T22:57:22.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.414+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/789614925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b0196f30 con 0x7ff3b0102620 2024-05-13T22:57:22.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.415+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/789614925 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b0108210 con 0x7ff3b0102620 2024-05-13T22:57:22.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.415+0000 7ff3a6ffd700 1 -- 172.21.15.102:0/789614925 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff3a00104f0 con 0x7ff3b0102620 2024-05-13T22:57:22.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.416+0000 7ff3a6ffd700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff39c06e200 0x7ff39c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:22.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.416+0000 7ff3a6ffd700 1 -- 172.21.15.102:0/789614925 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6704+0+0 (secure 0 0 0) 0x7ff3a008dd90 con 0x7ff3b0102620 2024-05-13T22:57:22.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.416+0000 7ff3b5bcb700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff39c06e200 0x7ff39c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:22.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.416+0000 7ff3b5bcb700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff39c06e200 0x7ff39c0706c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00b5c0 tx=0x7ff3ac005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:22.397 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.420+0000 7ff3a6ffd700 1 -- 172.21.15.102:0/789614925 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7ff3a005abb0 con 0x7ff3b0102620 2024-05-13T22:57:22.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:22 smithi130 ceph-mon[84265]: Health check update: Degraded data redundancy: 3/231 objects degraded (1.299%), 1 pg degraded (PG_DEGRADED) 2024-05-13T22:57:22.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.644+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/789614925 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3b0197270 con 0x7ff39c06e200 2024-05-13T22:57:22.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.645+0000 7ff3a6ffd700 1 -- 172.21.15.102:0/789614925 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff3b0197270 con 0x7ff39c06e200 2024-05-13T22:57:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.647+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/789614925 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff39c06e200 msgr2=0x7ff39c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.647+0000 7ff3b7e2f700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff39c06e200 0x7ff39c0706c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00b5c0 tx=0x7ff3ac005480 comp rx=0 tx=0).stop 2024-05-13T22:57:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.647+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/789614925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3b0102620 msgr2=0x7ff3b019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.647+0000 7ff3b7e2f700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3b0102620 0x7ff3b019cbc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000b480 tx=0x7ff3a000b840 comp rx=0 tx=0).stop 2024-05-13T22:57:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.647+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/789614925 shutdown_connections 2024-05-13T22:57:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.647+0000 7ff3b7e2f700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7ff39c06e200 0x7ff39c0706c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.647+0000 7ff3b7e2f700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3b0101c70 0x7ff3b019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.647+0000 7ff3b7e2f700 1 --2- 172.21.15.102:0/789614925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3b0102620 0x7ff3b019cbc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.647+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/789614925 >> 172.21.15.102:0/789614925 conn(0x7ff3b00fb530 msgr2=0x7ff3b0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.648+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/789614925 shutdown_connections 2024-05-13T22:57:22.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.648+0000 7ff3b7e2f700 1 -- 172.21.15.102:0/789614925 wait complete. 2024-05-13T22:57:22.635 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:57:22.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.757+0000 7f4d666db700 1 -- 172.21.15.102:0/3326269742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d600ff500 msgr2=0x7f4d600ff8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:22.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.757+0000 7f4d666db700 1 --2- 172.21.15.102:0/3326269742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d600ff500 0x7f4d600ff8e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4d50009910 tx=0x7f4d50009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:22.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.757+0000 7f4d666db700 1 -- 172.21.15.102:0/3326269742 shutdown_connections 2024-05-13T22:57:22.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.757+0000 7f4d666db700 1 --2- 172.21.15.102:0/3326269742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d600ffeb0 0x7f4d60105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.757+0000 7f4d666db700 1 --2- 172.21.15.102:0/3326269742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d600ff500 0x7f4d600ff8e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.757+0000 7f4d666db700 1 -- 172.21.15.102:0/3326269742 >> 172.21.15.102:0/3326269742 conn(0x7f4d60074d80 msgr2=0x7f4d60075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:22.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.757+0000 7f4d666db700 1 -- 172.21.15.102:0/3326269742 shutdown_connections 2024-05-13T22:57:22.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.757+0000 7f4d666db700 1 -- 172.21.15.102:0/3326269742 wait complete. 2024-05-13T22:57:22.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.758+0000 7f4d666db700 1 Processor -- start 2024-05-13T22:57:22.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.758+0000 7f4d666db700 1 -- start start 2024-05-13T22:57:22.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.758+0000 7f4d666db700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d600ff500 0x7f4d6019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.758+0000 7f4d666db700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d600ffeb0 0x7f4d6019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.758+0000 7f4d666db700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6019d1e0 con 0x7f4d600ffeb0 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.758+0000 7f4d666db700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60196690 con 0x7f4d600ff500 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.758+0000 7f4d57fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d600ffeb0 0x7f4d6019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.758+0000 7f4d57fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d600ffeb0 0x7f4d6019cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42330/0 (socket says 172.21.15.102:42330) 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.758+0000 7f4d57fff700 1 -- 172.21.15.102:0/3913279551 learned_addr learned my addr 172.21.15.102:0/3913279551 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.758+0000 7f4d5ffff700 1 --2- 172.21.15.102:0/3913279551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d600ff500 0x7f4d6019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.759+0000 7f4d57fff700 1 -- 172.21.15.102:0/3913279551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d600ff500 msgr2=0x7f4d6019c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.759+0000 7f4d57fff700 1 --2- 172.21.15.102:0/3913279551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d600ff500 0x7f4d6019c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.759+0000 7f4d57fff700 1 -- 172.21.15.102:0/3913279551 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d500095f0 con 0x7f4d600ffeb0 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.759+0000 7f4d57fff700 1 --2- 172.21.15.102:0/3913279551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d600ffeb0 0x7f4d6019cb50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800c8f0 tx=0x7f4d4800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.759+0000 7f4d5dffb700 1 -- 172.21.15.102:0/3913279551 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d480078a0 con 0x7f4d600ffeb0 2024-05-13T22:57:22.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.759+0000 7f4d5dffb700 1 -- 172.21.15.102:0/3913279551 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d4800f450 con 0x7f4d600ffeb0 2024-05-13T22:57:22.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.759+0000 7f4d666db700 1 -- 172.21.15.102:0/3913279551 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d60196970 con 0x7f4d600ffeb0 2024-05-13T22:57:22.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.759+0000 7f4d5dffb700 1 -- 172.21.15.102:0/3913279551 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d48018610 con 0x7f4d600ffeb0 2024-05-13T22:57:22.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.759+0000 7f4d666db700 1 -- 172.21.15.102:0/3913279551 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d60196ec0 con 0x7f4d600ffeb0 2024-05-13T22:57:22.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.760+0000 7f4d666db700 1 -- 172.21.15.102:0/3913279551 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d60109190 con 0x7f4d600ffeb0 2024-05-13T22:57:22.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.761+0000 7f4d5dffb700 1 -- 172.21.15.102:0/3913279551 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4d4800f5c0 con 0x7f4d600ffeb0 2024-05-13T22:57:22.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.761+0000 7f4d5dffb700 1 --2- 172.21.15.102:0/3913279551 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4d4c06e200 0x7f4d4c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:22.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.761+0000 7f4d5ffff700 1 --2- 172.21.15.102:0/3913279551 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4d4c06e200 0x7f4d4c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:22.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.761+0000 7f4d5dffb700 1 -- 172.21.15.102:0/3913279551 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f4d4808e150 con 0x7f4d600ffeb0 2024-05-13T22:57:22.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.762+0000 7f4d5ffff700 1 --2- 172.21.15.102:0/3913279551 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4d4c06e200 0x7f4d4c0706c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d50006010 tx=0x7f4d5001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:22.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.766+0000 7f4d5dffb700 1 -- 172.21.15.102:0/3913279551 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4d4805aef0 con 0x7f4d600ffeb0 2024-05-13T22:57:22.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.985+0000 7f4d666db700 1 -- 172.21.15.102:0/3913279551 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d60197140 con 0x7f4d4c06e200 2024-05-13T22:57:22.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.987+0000 7f4d5dffb700 1 -- 172.21.15.102:0/3913279551 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4d60197140 con 0x7f4d4c06e200 2024-05-13T22:57:22.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.989+0000 7f4d666db700 1 -- 172.21.15.102:0/3913279551 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4d4c06e200 msgr2=0x7f4d4c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:22.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.989+0000 7f4d666db700 1 --2- 172.21.15.102:0/3913279551 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4d4c06e200 0x7f4d4c0706c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d50006010 tx=0x7f4d5001a040 comp rx=0 tx=0).stop 2024-05-13T22:57:22.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.989+0000 7f4d666db700 1 -- 172.21.15.102:0/3913279551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d600ffeb0 msgr2=0x7f4d6019cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:22.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.989+0000 7f4d666db700 1 --2- 172.21.15.102:0/3913279551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d600ffeb0 0x7f4d6019cb50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800c8f0 tx=0x7f4d4800cc00 comp rx=0 tx=0).stop 2024-05-13T22:57:22.966 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.989+0000 7f4d666db700 1 -- 172.21.15.102:0/3913279551 shutdown_connections 2024-05-13T22:57:22.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.989+0000 7f4d666db700 1 --2- 172.21.15.102:0/3913279551 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f4d4c06e200 0x7f4d4c0706c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.989+0000 7f4d666db700 1 --2- 172.21.15.102:0/3913279551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d600ff500 0x7f4d6019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.989+0000 7f4d666db700 1 --2- 172.21.15.102:0/3913279551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d600ffeb0 0x7f4d6019cb50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:22.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.989+0000 7f4d666db700 1 -- 172.21.15.102:0/3913279551 >> 172.21.15.102:0/3913279551 conn(0x7f4d60074d80 msgr2=0x7f4d600fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:22.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.989+0000 7f4d666db700 1 -- 172.21.15.102:0/3913279551 shutdown_connections 2024-05-13T22:57:22.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:22.989+0000 7f4d666db700 1 -- 172.21.15.102:0/3913279551 wait complete. 2024-05-13T22:57:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.094+0000 7fd596c41700 1 -- 172.21.15.102:0/300911058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd590100de0 msgr2=0x7fd5901011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.094+0000 7fd596c41700 1 --2- 172.21.15.102:0/300911058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd590100de0 0x7fd5901011c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd584009910 tx=0x7fd584009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.094+0000 7fd596c41700 1 -- 172.21.15.102:0/300911058 shutdown_connections 2024-05-13T22:57:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.094+0000 7fd596c41700 1 --2- 172.21.15.102:0/300911058 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd590101790 0x7fd5901057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.094+0000 7fd596c41700 1 --2- 172.21.15.102:0/300911058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd590100de0 0x7fd5901011c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.094+0000 7fd596c41700 1 -- 172.21.15.102:0/300911058 >> 172.21.15.102:0/300911058 conn(0x7fd5900fc6b0 msgr2=0x7fd5900fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:23.072 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.094+0000 7fd596c41700 1 -- 172.21.15.102:0/300911058 shutdown_connections 2024-05-13T22:57:23.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.094+0000 7fd596c41700 1 -- 172.21.15.102:0/300911058 wait complete. 2024-05-13T22:57:23.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.095+0000 7fd596c41700 1 Processor -- start 2024-05-13T22:57:23.073 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.095+0000 7fd596c41700 1 -- start start 2024-05-13T22:57:23.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.095+0000 7fd596c41700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd590100de0 0x7fd59019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:23.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.095+0000 7fd596c41700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd590101790 0x7fd59019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:23.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.095+0000 7fd596c41700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59019d230 con 0x7fd590101790 2024-05-13T22:57:23.074 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.095+0000 7fd596c41700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5901966e0 con 0x7fd590100de0 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.095+0000 7fd58ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd590101790 0x7fd59019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd58ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd590101790 0x7fd59019cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42346/0 (socket says 172.21.15.102:42346) 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd58ffff700 1 -- 172.21.15.102:0/2430286708 learned_addr learned my addr 172.21.15.102:0/2430286708 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd5949dd700 1 --2- 172.21.15.102:0/2430286708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd590100de0 0x7fd59019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd58ffff700 1 -- 172.21.15.102:0/2430286708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd590100de0 msgr2=0x7fd59019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd58ffff700 1 --2- 172.21.15.102:0/2430286708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd590100de0 0x7fd59019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd58ffff700 1 -- 172.21.15.102:0/2430286708 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5840095f0 con 0x7fd590101790 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd58ffff700 1 --2- 172.21.15.102:0/2430286708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd590101790 0x7fd59019cba0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00b480 tx=0x7fd57c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd58dffb700 1 -- 172.21.15.102:0/2430286708 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd57c0118c0 con 0x7fd590101790 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd596c41700 1 -- 172.21.15.102:0/2430286708 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5901969c0 con 0x7fd590101790 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd58dffb700 1 -- 172.21.15.102:0/2430286708 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd57c010380 con 0x7fd590101790 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd58dffb700 1 -- 172.21.15.102:0/2430286708 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd57c00f4f0 con 0x7fd590101790 2024-05-13T22:57:23.075 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.096+0000 7fd596c41700 1 -- 172.21.15.102:0/2430286708 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd590196f10 con 0x7fd590101790 2024-05-13T22:57:23.076 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.097+0000 7fd596c41700 1 -- 172.21.15.102:0/2430286708 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd590109180 con 0x7fd590101790 2024-05-13T22:57:23.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.102+0000 7fd58dffb700 1 -- 172.21.15.102:0/2430286708 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd57c0104f0 con 0x7fd590101790 2024-05-13T22:57:23.079 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.102+0000 7fd58dffb700 1 --2- 172.21.15.102:0/2430286708 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd58006e200 0x7fd5800706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:23.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.102+0000 7fd58dffb700 1 -- 172.21.15.102:0/2430286708 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fd57c08e170 con 0x7fd590101790 2024-05-13T22:57:23.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.103+0000 7fd5949dd700 1 --2- 172.21.15.102:0/2430286708 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd58006e200 0x7fd5800706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:23.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.103+0000 7fd5949dd700 1 --2- 172.21.15.102:0/2430286708 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd58006e200 0x7fd5800706c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd584006010 tx=0x7fd58401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:23.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.108+0000 7fd58dffb700 1 -- 172.21.15.102:0/2430286708 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd57c05b050 con 0x7fd590101790 2024-05-13T22:57:23.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.320+0000 7fd596c41700 1 -- 172.21.15.102:0/2430286708 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5901972c0 con 0x7fd58006e200 2024-05-13T22:57:23.297 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:23 smithi102 ceph-mon[111694]: pgmap v183: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 510 B/s rd, 0 op/s 2024-05-13T22:57:23.297 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:23 smithi102 ceph-mon[111694]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/231 objects degraded (1.299%), 1 pg degraded) 2024-05-13T22:57:23.297 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:23 smithi102 ceph-mon[111694]: Cluster is now healthy 2024-05-13T22:57:23.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.333+0000 7fd58dffb700 1 -- 172.21.15.102:0/2430286708 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd5901972c0 con 0x7fd58006e200 2024-05-13T22:57:23.310 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:57:23.310 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (6m) 112s ago 17m 26.7M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:57:23.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (17m) 112s ago 17m 8740k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:57:23.310 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (16m) 12s ago 16m 8476k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:57:23.310 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (3m) 112s ago 17m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T22:57:23.310 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (3m) 12s ago 16m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T22:57:23.310 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (6m) 112s ago 17m 79.9M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:57:23.310 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (12m) 112s ago 12m 88.3M - 18.2.1 5be31c24972a 8fdb206d4cb8 2024-05-13T22:57:23.310 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (12m) 112s ago 12m 146M - 18.2.1 5be31c24972a 75d583d3639f 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (12m) 12s ago 12m 17.5M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (12m) 12s ago 11m 20.5M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (8m) 112s ago 19m 506M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (7m) 12s ago 16m 453M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (4m) 112s ago 19m 51.7M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (4m) 12s ago 16m 44.2M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (7m) 112s ago 17m 16.0M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (7m) 12s ago 16m 17.3M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (3m) 112s ago 15m 133M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 0ada88c777a2 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (2m) 112s ago 15m 130M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 c3e9e68b06fd 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (118s) 112s ago 14m 15.1M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 1b85728dccf5 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (82s) 12s ago 14m 100M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 66e804d4b801 2024-05-13T22:57:23.311 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (48s) 12s ago 14m 94.7M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5bb5a8f94042 2024-05-13T22:57:23.312 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (15s) 12s ago 13m 13.9M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5d983135967d 2024-05-13T22:57:23.312 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (6m) 112s ago 17m 73.7M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:57:23.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.335+0000 7fd596c41700 1 -- 172.21.15.102:0/2430286708 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd58006e200 msgr2=0x7fd5800706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:23.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.335+0000 7fd596c41700 1 --2- 172.21.15.102:0/2430286708 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd58006e200 0x7fd5800706c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd584006010 tx=0x7fd58401a040 comp rx=0 tx=0).stop 2024-05-13T22:57:23.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.335+0000 7fd596c41700 1 -- 172.21.15.102:0/2430286708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd590101790 msgr2=0x7fd59019cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:23.312 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.335+0000 7fd596c41700 1 --2- 172.21.15.102:0/2430286708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd590101790 0x7fd59019cba0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00b480 tx=0x7fd57c00b840 comp rx=0 tx=0).stop 2024-05-13T22:57:23.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.335+0000 7fd596c41700 1 -- 172.21.15.102:0/2430286708 shutdown_connections 2024-05-13T22:57:23.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.335+0000 7fd596c41700 1 --2- 172.21.15.102:0/2430286708 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd58006e200 0x7fd5800706c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.335+0000 7fd596c41700 1 --2- 172.21.15.102:0/2430286708 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd590100de0 0x7fd59019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.335+0000 7fd596c41700 1 --2- 172.21.15.102:0/2430286708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd590101790 0x7fd59019cba0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.335+0000 7fd596c41700 1 -- 172.21.15.102:0/2430286708 >> 172.21.15.102:0/2430286708 conn(0x7fd5900fc6b0 msgr2=0x7fd5900fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:23.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.335+0000 7fd596c41700 1 -- 172.21.15.102:0/2430286708 shutdown_connections 2024-05-13T22:57:23.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.335+0000 7fd596c41700 1 -- 172.21.15.102:0/2430286708 wait complete. 2024-05-13T22:57:23.341 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:23 smithi130 ceph-mon[84265]: pgmap v183: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 510 B/s rd, 0 op/s 2024-05-13T22:57:23.341 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:23 smithi130 ceph-mon[84265]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/231 objects degraded (1.299%), 1 pg degraded) 2024-05-13T22:57:23.341 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:23 smithi130 ceph-mon[84265]: Cluster is now healthy 2024-05-13T22:57:23.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.438+0000 7fc8626f1700 1 -- 172.21.15.102:0/977841813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0fff20 msgr2=0x7fc85c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.438+0000 7fc8626f1700 1 --2- 172.21.15.102:0/977841813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0fff20 0x7fc85c1015b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc850009910 tx=0x7fc850009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.438+0000 7fc8626f1700 1 -- 172.21.15.102:0/977841813 shutdown_connections 2024-05-13T22:57:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.438+0000 7fc8626f1700 1 --2- 172.21.15.102:0/977841813 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0fff20 0x7fc85c1015b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.438+0000 7fc8626f1700 1 --2- 172.21.15.102:0/977841813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc85c0ff570 0x7fc85c0ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.438+0000 7fc8626f1700 1 -- 172.21.15.102:0/977841813 >> 172.21.15.102:0/977841813 conn(0x7fc85c074bf0 msgr2=0x7fc85c075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.438+0000 7fc8626f1700 1 -- 172.21.15.102:0/977841813 shutdown_connections 2024-05-13T22:57:23.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.438+0000 7fc8626f1700 1 -- 172.21.15.102:0/977841813 wait complete. 2024-05-13T22:57:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.439+0000 7fc8626f1700 1 Processor -- start 2024-05-13T22:57:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.439+0000 7fc8626f1700 1 -- start start 2024-05-13T22:57:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.439+0000 7fc8626f1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ff570 0x7fc85c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.439+0000 7fc8626f1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc85c0fff20 0x7fc85c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.439+0000 7fc8626f1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c19f360 con 0x7fc85c0ff570 2024-05-13T22:57:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.439+0000 7fc8626f1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c198810 con 0x7fc85c0fff20 2024-05-13T22:57:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.439+0000 7fc85bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ff570 0x7fc85c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc85bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ff570 0x7fc85c19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42362/0 (socket says 172.21.15.102:42362) 2024-05-13T22:57:23.418 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc85bfff700 1 -- 172.21.15.102:0/3032322307 learned_addr learned my addr 172.21.15.102:0/3032322307 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc85b7fe700 1 --2- 172.21.15.102:0/3032322307 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc85c0fff20 0x7fc85c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc85bfff700 1 -- 172.21.15.102:0/3032322307 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc85c0fff20 msgr2=0x7fc85c19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc85bfff700 1 --2- 172.21.15.102:0/3032322307 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc85c0fff20 0x7fc85c19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc85bfff700 1 -- 172.21.15.102:0/3032322307 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8500095f0 con 0x7fc85c0ff570 2024-05-13T22:57:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc85bfff700 1 --2- 172.21.15.102:0/3032322307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ff570 0x7fc85c19e790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00b4b0 tx=0x7fc84c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc8597fa700 1 -- 172.21.15.102:0/3032322307 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc84c011800 con 0x7fc85c0ff570 2024-05-13T22:57:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc8626f1700 1 -- 172.21.15.102:0/3032322307 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c198af0 con 0x7fc85c0ff570 2024-05-13T22:57:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc8597fa700 1 -- 172.21.15.102:0/3032322307 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc84c011e20 con 0x7fc85c0ff570 2024-05-13T22:57:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc8597fa700 1 -- 172.21.15.102:0/3032322307 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc84c00f2e0 con 0x7fc85c0ff570 2024-05-13T22:57:23.419 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.440+0000 7fc8626f1700 1 -- 172.21.15.102:0/3032322307 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c1990c0 con 0x7fc85c0ff570 2024-05-13T22:57:23.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.441+0000 7fc8626f1700 1 -- 172.21.15.102:0/3032322307 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c04ea90 con 0x7fc85c0ff570 2024-05-13T22:57:23.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.442+0000 7fc8597fa700 1 -- 172.21.15.102:0/3032322307 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc84c010380 con 0x7fc85c0ff570 2024-05-13T22:57:23.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.442+0000 7fc8597fa700 1 --2- 172.21.15.102:0/3032322307 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc84406e200 0x7fc8440706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:23.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.442+0000 7fc8597fa700 1 -- 172.21.15.102:0/3032322307 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fc84c08dd40 con 0x7fc85c0ff570 2024-05-13T22:57:23.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.442+0000 7fc85b7fe700 1 --2- 172.21.15.102:0/3032322307 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc84406e200 0x7fc8440706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:23.421 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.443+0000 7fc85b7fe700 1 --2- 172.21.15.102:0/3032322307 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc84406e200 0x7fc8440706c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc85c1092d0 tx=0x7fc85001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:23.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.448+0000 7fc8597fa700 1 -- 172.21.15.102:0/3032322307 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc84c091030 con 0x7fc85c0ff570 2024-05-13T22:57:23.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.731+0000 7fc8626f1700 1 -- 172.21.15.102:0/3032322307 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc85c199440 con 0x7fc85c0ff570 2024-05-13T22:57:23.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.731+0000 7fc8597fa700 1 -- 172.21.15.102:0/3032322307 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+701 (secure 0 0 0) 0x7fc84c05aae0 con 0x7fc85c0ff570 2024-05-13T22:57:23.708 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 6 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:57:23.709 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:57:23.710 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-05-13T22:57:23.710 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 10 2024-05-13T22:57:23.710 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:57:23.710 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:57:23.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.733+0000 7fc8626f1700 1 -- 172.21.15.102:0/3032322307 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc84406e200 msgr2=0x7fc8440706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.733+0000 7fc8626f1700 1 --2- 172.21.15.102:0/3032322307 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc84406e200 0x7fc8440706c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc85c1092d0 tx=0x7fc85001a040 comp rx=0 tx=0).stop 2024-05-13T22:57:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.733+0000 7fc8626f1700 1 -- 172.21.15.102:0/3032322307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ff570 msgr2=0x7fc85c19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.733+0000 7fc8626f1700 1 --2- 172.21.15.102:0/3032322307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ff570 0x7fc85c19e790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00b4b0 tx=0x7fc84c00b7c0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.733+0000 7fc8626f1700 1 -- 172.21.15.102:0/3032322307 shutdown_connections 2024-05-13T22:57:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.733+0000 7fc8626f1700 1 --2- 172.21.15.102:0/3032322307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc85c0ff570 0x7fc85c19e790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.733+0000 7fc8626f1700 1 --2- 172.21.15.102:0/3032322307 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc84406e200 0x7fc8440706c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.733+0000 7fc8626f1700 1 --2- 172.21.15.102:0/3032322307 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc85c0fff20 0x7fc85c19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.733+0000 7fc8626f1700 1 -- 172.21.15.102:0/3032322307 >> 172.21.15.102:0/3032322307 conn(0x7fc85c074bf0 msgr2=0x7fc85c106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.733+0000 7fc8626f1700 1 -- 172.21.15.102:0/3032322307 shutdown_connections 2024-05-13T22:57:23.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.733+0000 7fc8626f1700 1 -- 172.21.15.102:0/3032322307 wait complete. 2024-05-13T22:57:23.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.839+0000 7fdd534ef700 1 -- 172.21.15.102:0/1528492070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4c069020 msgr2=0x7fdd4c105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:23.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.839+0000 7fdd534ef700 1 --2- 172.21.15.102:0/1528492070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4c069020 0x7fdd4c105790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009910 tx=0x7fdd48009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:23.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.839+0000 7fdd534ef700 1 -- 172.21.15.102:0/1528492070 shutdown_connections 2024-05-13T22:57:23.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.839+0000 7fdd534ef700 1 --2- 172.21.15.102:0/1528492070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4c069020 0x7fdd4c105790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.839+0000 7fdd534ef700 1 --2- 172.21.15.102:0/1528492070 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd4c068670 0x7fdd4c068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.839+0000 7fdd534ef700 1 -- 172.21.15.102:0/1528492070 >> 172.21.15.102:0/1528492070 conn(0x7fdd4c0755a0 msgr2=0x7fdd4c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:23.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.839+0000 7fdd534ef700 1 -- 172.21.15.102:0/1528492070 shutdown_connections 2024-05-13T22:57:23.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.839+0000 7fdd534ef700 1 -- 172.21.15.102:0/1528492070 wait complete. 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd534ef700 1 Processor -- start 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd534ef700 1 -- start start 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd534ef700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4c068670 0x7fdd4c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd534ef700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd4c069020 0x7fdd4c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd534ef700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c19d220 con 0x7fdd4c068670 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd534ef700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c1966d0 con 0x7fdd4c069020 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd5128b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4c068670 0x7fdd4c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd5128b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4c068670 0x7fdd4c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42388/0 (socket says 172.21.15.102:42388) 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd5128b700 1 -- 172.21.15.102:0/3202480843 learned_addr learned my addr 172.21.15.102:0/3202480843 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd50a8a700 1 --2- 172.21.15.102:0/3202480843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd4c069020 0x7fdd4c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd5128b700 1 -- 172.21.15.102:0/3202480843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd4c069020 msgr2=0x7fdd4c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd5128b700 1 --2- 172.21.15.102:0/3202480843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd4c069020 0x7fdd4c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.840+0000 7fdd5128b700 1 -- 172.21.15.102:0/3202480843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd480095f0 con 0x7fdd4c068670 2024-05-13T22:57:23.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.841+0000 7fdd5128b700 1 --2- 172.21.15.102:0/3202480843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4c068670 0x7fdd4c19c650 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00b4b0 tx=0x7fdd3c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.841+0000 7fdd427fc700 1 -- 172.21.15.102:0/3202480843 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd3c011800 con 0x7fdd4c068670 2024-05-13T22:57:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.841+0000 7fdd534ef700 1 -- 172.21.15.102:0/3202480843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4c1969b0 con 0x7fdd4c068670 2024-05-13T22:57:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.841+0000 7fdd427fc700 1 -- 172.21.15.102:0/3202480843 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd3c011e20 con 0x7fdd4c068670 2024-05-13T22:57:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.841+0000 7fdd427fc700 1 -- 172.21.15.102:0/3202480843 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd3c00f2e0 con 0x7fdd4c068670 2024-05-13T22:57:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.841+0000 7fdd534ef700 1 -- 172.21.15.102:0/3202480843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c196f00 con 0x7fdd4c068670 2024-05-13T22:57:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.842+0000 7fdd534ef700 1 -- 172.21.15.102:0/3202480843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4c1090c0 con 0x7fdd4c068670 2024-05-13T22:57:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.842+0000 7fdd427fc700 1 -- 172.21.15.102:0/3202480843 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdd3c010380 con 0x7fdd4c068670 2024-05-13T22:57:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.843+0000 7fdd427fc700 1 --2- 172.21.15.102:0/3202480843 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdd3806e200 0x7fdd380706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.843+0000 7fdd427fc700 1 -- 172.21.15.102:0/3202480843 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fdd3c08ddc0 con 0x7fdd4c068670 2024-05-13T22:57:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.843+0000 7fdd50a8a700 1 --2- 172.21.15.102:0/3202480843 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdd3806e200 0x7fdd380706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:23.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.843+0000 7fdd50a8a700 1 --2- 172.21.15.102:0/3202480843 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdd3806e200 0x7fdd380706c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c197b40 tx=0x7fdd4801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:23.825 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:23.847+0000 7fdd427fc700 1 -- 172.21.15.102:0/3202480843 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdd3c05ab60 con 0x7fdd4c068670 2024-05-13T22:57:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.087+0000 7fdd534ef700 1 -- 172.21.15.102:0/3202480843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdd4c04ea90 con 0x7fdd4c068670 2024-05-13T22:57:24.065 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.088+0000 7fdd427fc700 1 -- 172.21.15.102:0/3202480843 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1712 (secure 0 0 0) 0x7fdd3c05a6f0 con 0x7fdd4c068670 2024-05-13T22:57:24.065 INFO:teuthology.orchestra.run.smithi102.stdout:e20 2024-05-13T22:57:24.065 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:57:24.065 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:57:24.065 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 18 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:57:18.071027+0000 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:57:24.066 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 0 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=14498} 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:57:24.067 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:57:24.068 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:57:24.068 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 14498 members: 14498 2024-05-13T22:57:24.068 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:14498} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.102:6828/2977397703,v1:172.21.15.102:6829/2977397703] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:57:24.068 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:57:24.068 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:57:24.068 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:57:24.068 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:57:24.068 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:57:24.068 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:57:24.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.091+0000 7fdd534ef700 1 -- 172.21.15.102:0/3202480843 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdd3806e200 msgr2=0x7fdd380706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:24.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.091+0000 7fdd534ef700 1 --2- 172.21.15.102:0/3202480843 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdd3806e200 0x7fdd380706c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c197b40 tx=0x7fdd4801a040 comp rx=0 tx=0).stop 2024-05-13T22:57:24.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.091+0000 7fdd534ef700 1 -- 172.21.15.102:0/3202480843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4c068670 msgr2=0x7fdd4c19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:24.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.091+0000 7fdd534ef700 1 --2- 172.21.15.102:0/3202480843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4c068670 0x7fdd4c19c650 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00b4b0 tx=0x7fdd3c00b870 comp rx=0 tx=0).stop 2024-05-13T22:57:24.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.091+0000 7fdd534ef700 1 -- 172.21.15.102:0/3202480843 shutdown_connections 2024-05-13T22:57:24.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.091+0000 7fdd534ef700 1 --2- 172.21.15.102:0/3202480843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdd4c068670 0x7fdd4c19c650 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.091+0000 7fdd534ef700 1 --2- 172.21.15.102:0/3202480843 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdd3806e200 0x7fdd380706c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.091+0000 7fdd534ef700 1 --2- 172.21.15.102:0/3202480843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd4c069020 0x7fdd4c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.091+0000 7fdd534ef700 1 -- 172.21.15.102:0/3202480843 >> 172.21.15.102:0/3202480843 conn(0x7fdd4c0755a0 msgr2=0x7fdd4c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:24.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.091+0000 7fdd534ef700 1 -- 172.21.15.102:0/3202480843 shutdown_connections 2024-05-13T22:57:24.070 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.091+0000 7fdd534ef700 1 -- 172.21.15.102:0/3202480843 wait complete. 2024-05-13T22:57:24.070 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 20 2024-05-13T22:57:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.198+0000 7fd10ac73700 1 -- 172.21.15.102:0/1576740132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd104101770 msgr2=0x7fd1041057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.198+0000 7fd10ac73700 1 --2- 172.21.15.102:0/1576740132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd104101770 0x7fd1041057c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009910 tx=0x7fd0f0009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.199+0000 7fd10ac73700 1 -- 172.21.15.102:0/1576740132 shutdown_connections 2024-05-13T22:57:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.199+0000 7fd10ac73700 1 --2- 172.21.15.102:0/1576740132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd104101770 0x7fd1041057c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.199+0000 7fd10ac73700 1 --2- 172.21.15.102:0/1576740132 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd104100dc0 0x7fd1041011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.199+0000 7fd10ac73700 1 -- 172.21.15.102:0/1576740132 >> 172.21.15.102:0/1576740132 conn(0x7fd1040fc650 msgr2=0x7fd1040fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.199+0000 7fd10ac73700 1 -- 172.21.15.102:0/1576740132 shutdown_connections 2024-05-13T22:57:24.176 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.199+0000 7fd10ac73700 1 -- 172.21.15.102:0/1576740132 wait complete. 2024-05-13T22:57:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.199+0000 7fd10ac73700 1 Processor -- start 2024-05-13T22:57:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.199+0000 7fd10ac73700 1 -- start start 2024-05-13T22:57:24.178 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd10ac73700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd104100dc0 0x7fd10419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd10ac73700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd104101770 0x7fd10419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd10ac73700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10419d240 con 0x7fd104100dc0 2024-05-13T22:57:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd10ac73700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041966f0 con 0x7fd104101770 2024-05-13T22:57:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd108a0f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd104100dc0 0x7fd10419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:24.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd108a0f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd104100dc0 0x7fd10419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42402/0 (socket says 172.21.15.102:42402) 2024-05-13T22:57:24.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd108a0f700 1 -- 172.21.15.102:0/2032568746 learned_addr learned my addr 172.21.15.102:0/2032568746 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:24.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd103fff700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd104101770 0x7fd10419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:24.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd108a0f700 1 -- 172.21.15.102:0/2032568746 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd104101770 msgr2=0x7fd10419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:24.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd108a0f700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd104101770 0x7fd10419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd108a0f700 1 -- 172.21.15.102:0/2032568746 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f00095f0 con 0x7fd104100dc0 2024-05-13T22:57:24.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd103fff700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd104101770 0x7fd10419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:57:24.180 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd108a0f700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd104100dc0 0x7fd10419c670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800b4b0 tx=0x7fd0f800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:24.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd101ffb700 1 -- 172.21.15.102:0/2032568746 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0f8011800 con 0x7fd104100dc0 2024-05-13T22:57:24.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd101ffb700 1 -- 172.21.15.102:0/2032568746 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0f8011e20 con 0x7fd104100dc0 2024-05-13T22:57:24.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd10ac73700 1 -- 172.21.15.102:0/2032568746 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1041969d0 con 0x7fd104100dc0 2024-05-13T22:57:24.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.200+0000 7fd101ffb700 1 -- 172.21.15.102:0/2032568746 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0f800f2e0 con 0x7fd104100dc0 2024-05-13T22:57:24.181 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.201+0000 7fd10ac73700 1 -- 172.21.15.102:0/2032568746 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd104196f20 con 0x7fd104100dc0 2024-05-13T22:57:24.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.202+0000 7fd10ac73700 1 -- 172.21.15.102:0/2032568746 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1041090f0 con 0x7fd104100dc0 2024-05-13T22:57:24.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.202+0000 7fd101ffb700 1 -- 172.21.15.102:0/2032568746 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd0f8010380 con 0x7fd104100dc0 2024-05-13T22:57:24.183 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.202+0000 7fd101ffb700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0f406e200 0x7fd0f40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:24.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.202+0000 7fd101ffb700 1 -- 172.21.15.102:0/2032568746 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fd0f808ddc0 con 0x7fd104100dc0 2024-05-13T22:57:24.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.202+0000 7fd103fff700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0f406e200 0x7fd0f40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:24.184 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.203+0000 7fd103fff700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0f406e200 0x7fd0f40706c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd104197b60 tx=0x7fd0f001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:24.185 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.207+0000 7fd101ffb700 1 -- 172.21.15.102:0/2032568746 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd0f805ab60 con 0x7fd104100dc0 2024-05-13T22:57:24.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:24 smithi130 ceph-mon[84265]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:24.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:24 smithi130 ceph-mon[84265]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:24.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:24 smithi130 ceph-mon[84265]: osdmap e93: 6 total, 6 up, 6 in 2024-05-13T22:57:24.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:24 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 2 up:standby 2024-05-13T22:57:24.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:24 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3032322307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:24.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:24 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3202480843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:57:24.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:24 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:57:24.400 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.422+0000 7fd10ac73700 1 -- 172.21.15.102:0/2032568746 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1041971a0 con 0x7fd0f406e200 2024-05-13T22:57:24.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:24 smithi102 ceph-mon[111694]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:24.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:24 smithi102 ceph-mon[111694]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:24.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:24 smithi102 ceph-mon[111694]: osdmap e93: 6 total, 6 up, 6 in 2024-05-13T22:57:24.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:24 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 2 up:standby 2024-05-13T22:57:24.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:24 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3032322307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:24.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:24 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3202480843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:57:24.401 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:24 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:57:24.401 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.423+0000 7fd101ffb700 1 -- 172.21.15.102:0/2032568746 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd1041971a0 con 0x7fd0f406e200 2024-05-13T22:57:24.401 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:57:24.401 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:57:24.401 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:57:24.401 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:57:24.401 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:57:24.401 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-05-13T22:57:24.402 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr", 2024-05-13T22:57:24.402 INFO:teuthology.orchestra.run.smithi102.stdout: "mon", 2024-05-13T22:57:24.402 INFO:teuthology.orchestra.run.smithi102.stdout: "osd" 2024-05-13T22:57:24.402 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:57:24.402 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "12/23 daemons upgraded", 2024-05-13T22:57:24.402 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading mds daemons", 2024-05-13T22:57:24.402 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:57:24.402 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:57:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.426+0000 7fd10ac73700 1 -- 172.21.15.102:0/2032568746 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0f406e200 msgr2=0x7fd0f40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.426+0000 7fd10ac73700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0f406e200 0x7fd0f40706c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd104197b60 tx=0x7fd0f001a040 comp rx=0 tx=0).stop 2024-05-13T22:57:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.426+0000 7fd10ac73700 1 -- 172.21.15.102:0/2032568746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd104100dc0 msgr2=0x7fd10419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.426+0000 7fd10ac73700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd104100dc0 0x7fd10419c670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800b4b0 tx=0x7fd0f800b870 comp rx=0 tx=0).stop 2024-05-13T22:57:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.426+0000 7fd10ac73700 1 -- 172.21.15.102:0/2032568746 shutdown_connections 2024-05-13T22:57:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.426+0000 7fd10ac73700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd104100dc0 0x7fd10419c670 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.426+0000 7fd10ac73700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0f406e200 0x7fd0f40706c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.426+0000 7fd10ac73700 1 --2- 172.21.15.102:0/2032568746 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd104101770 0x7fd10419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.426+0000 7fd10ac73700 1 -- 172.21.15.102:0/2032568746 >> 172.21.15.102:0/2032568746 conn(0x7fd1040fc650 msgr2=0x7fd1040fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.426+0000 7fd10ac73700 1 -- 172.21.15.102:0/2032568746 shutdown_connections 2024-05-13T22:57:24.403 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.426+0000 7fd10ac73700 1 -- 172.21.15.102:0/2032568746 wait complete. 2024-05-13T22:57:24.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.531+0000 7f8f4695a700 1 -- 172.21.15.102:0/4058610774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f400ffa40 msgr2=0x7f8f400ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:24.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.531+0000 7f8f4695a700 1 --2- 172.21.15.102:0/4058610774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f400ffa40 0x7f8f400ffec0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009910 tx=0x7f8f34009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:24.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.531+0000 7f8f4695a700 1 -- 172.21.15.102:0/4058610774 shutdown_connections 2024-05-13T22:57:24.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.531+0000 7f8f4695a700 1 --2- 172.21.15.102:0/4058610774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f400ffa40 0x7f8f400ffec0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.531+0000 7f8f4695a700 1 --2- 172.21.15.102:0/4058610774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f40105a40 0x7f8f40105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.531+0000 7f8f4695a700 1 -- 172.21.15.102:0/4058610774 >> 172.21.15.102:0/4058610774 conn(0x7f8f400fb530 msgr2=0x7f8f400fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:24.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.531+0000 7f8f4695a700 1 -- 172.21.15.102:0/4058610774 shutdown_connections 2024-05-13T22:57:24.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.531+0000 7f8f4695a700 1 -- 172.21.15.102:0/4058610774 wait complete. 2024-05-13T22:57:24.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.531+0000 7f8f4695a700 1 Processor -- start 2024-05-13T22:57:24.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.531+0000 7f8f4695a700 1 -- start start 2024-05-13T22:57:24.511 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f4695a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f400ffa40 0x7f8f4019c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f4695a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f40105a40 0x7f8f4019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f4695a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019d1d0 con 0x7f8f40105a40 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f4695a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f40196680 con 0x7f8f400ffa40 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f3f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f40105a40 0x7f8f4019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f3f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f40105a40 0x7f8f4019cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42432/0 (socket says 172.21.15.102:42432) 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f3f7fe700 1 -- 172.21.15.102:0/2007481171 learned_addr learned my addr 172.21.15.102:0/2007481171 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f3ffff700 1 --2- 172.21.15.102:0/2007481171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f400ffa40 0x7f8f4019c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f3f7fe700 1 -- 172.21.15.102:0/2007481171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f400ffa40 msgr2=0x7f8f4019c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f3f7fe700 1 --2- 172.21.15.102:0/2007481171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f400ffa40 0x7f8f4019c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f3f7fe700 1 -- 172.21.15.102:0/2007481171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f340095f0 con 0x7f8f40105a40 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f3f7fe700 1 --2- 172.21.15.102:0/2007481171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f40105a40 0x7f8f4019cb40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f34000c00 tx=0x7f8f3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f3d7fa700 1 -- 172.21.15.102:0/2007481171 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f3401d070 con 0x7f8f40105a40 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f3d7fa700 1 -- 172.21.15.102:0/2007481171 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f3400f460 con 0x7f8f40105a40 2024-05-13T22:57:24.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f4695a700 1 -- 172.21.15.102:0/2007481171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f40196900 con 0x7f8f40105a40 2024-05-13T22:57:24.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.532+0000 7f8f3d7fa700 1 -- 172.21.15.102:0/2007481171 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f34017630 con 0x7f8f40105a40 2024-05-13T22:57:24.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.533+0000 7f8f4695a700 1 -- 172.21.15.102:0/2007481171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f40196df0 con 0x7f8f40105a40 2024-05-13T22:57:24.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.533+0000 7f8f4695a700 1 -- 172.21.15.102:0/2007481171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f40109070 con 0x7f8f40105a40 2024-05-13T22:57:24.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.534+0000 7f8f3d7fa700 1 -- 172.21.15.102:0/2007481171 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8f3400f5d0 con 0x7f8f40105a40 2024-05-13T22:57:24.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.535+0000 7f8f3d7fa700 1 --2- 172.21.15.102:0/2007481171 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f2806e200 0x7f8f280706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:24.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.535+0000 7f8f3ffff700 1 --2- 172.21.15.102:0/2007481171 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f2806e200 0x7f8f280706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:24.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.535+0000 7f8f3d7fa700 1 -- 172.21.15.102:0/2007481171 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f8f3408f800 con 0x7f8f40105a40 2024-05-13T22:57:24.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.535+0000 7f8f3ffff700 1 --2- 172.21.15.102:0/2007481171 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f2806e200 0x7f8f280706c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f30006fd0 tx=0x7f8f30008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:24.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.540+0000 7f8f3d7fa700 1 -- 172.21.15.102:0/2007481171 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8f3405c5a0 con 0x7f8f40105a40 2024-05-13T22:57:24.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.819+0000 7f8f4695a700 1 -- 172.21.15.102:0/2007481171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8f4004ea90 con 0x7f8f40105a40 2024-05-13T22:57:24.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.819+0000 7f8f3d7fa700 1 -- 172.21.15.102:0/2007481171 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f8f3405c130 con 0x7f8f40105a40 2024-05-13T22:57:24.795 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:57:24.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.821+0000 7f8f4695a700 1 -- 172.21.15.102:0/2007481171 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f2806e200 msgr2=0x7f8f280706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:24.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.821+0000 7f8f4695a700 1 --2- 172.21.15.102:0/2007481171 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f2806e200 0x7f8f280706c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f30006fd0 tx=0x7f8f30008040 comp rx=0 tx=0).stop 2024-05-13T22:57:24.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.821+0000 7f8f4695a700 1 -- 172.21.15.102:0/2007481171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f40105a40 msgr2=0x7f8f4019cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:24.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.821+0000 7f8f4695a700 1 --2- 172.21.15.102:0/2007481171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f40105a40 0x7f8f4019cb40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f34000c00 tx=0x7f8f3400be20 comp rx=0 tx=0).stop 2024-05-13T22:57:24.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.821+0000 7f8f4695a700 1 -- 172.21.15.102:0/2007481171 shutdown_connections 2024-05-13T22:57:24.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.821+0000 7f8f4695a700 1 --2- 172.21.15.102:0/2007481171 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8f2806e200 0x7f8f280706c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.821+0000 7f8f4695a700 1 --2- 172.21.15.102:0/2007481171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f400ffa40 0x7f8f4019c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.821+0000 7f8f4695a700 1 --2- 172.21.15.102:0/2007481171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8f40105a40 0x7f8f4019cb40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:24.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.821+0000 7f8f4695a700 1 -- 172.21.15.102:0/2007481171 >> 172.21.15.102:0/2007481171 conn(0x7f8f400fb530 msgr2=0x7f8f400690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:24.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.821+0000 7f8f4695a700 1 -- 172.21.15.102:0/2007481171 shutdown_connections 2024-05-13T22:57:24.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:24.821+0000 7f8f4695a700 1 -- 172.21.15.102:0/2007481171 wait complete. 2024-05-13T22:57:25.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:25 smithi130 ceph-mon[84265]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:25.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:25 smithi130 ceph-mon[84265]: pgmap v185: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 438 B/s rd, 0 op/s 2024-05-13T22:57:25.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:25 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2007481171' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:57:25.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:25 smithi102 ceph-mon[111694]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:25.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:25 smithi102 ceph-mon[111694]: pgmap v185: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 438 B/s rd, 0 op/s 2024-05-13T22:57:25.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:25 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2007481171' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:57:26.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:26 smithi130 ceph-mon[84265]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:26.400 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:26 smithi102 ceph-mon[111694]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:27.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:27 smithi130 ceph-mon[84265]: pgmap v186: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-13T22:57:27.422 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:27 smithi102 ceph-mon[111694]: pgmap v186: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-13T22:57:28.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:28 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:28.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:28 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:28.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:28 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:28.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:28 smithi102 ceph-mon[111694]: pgmap v187: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-13T22:57:28.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:28 smithi102 ceph-mon[111694]: mds.? [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] up:boot 2024-05-13T22:57:28.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:28 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 3 up:standby 2024-05-13T22:57:28.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:28 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:57:29.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:28 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:29.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:28 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:29.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:28 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:29.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:28 smithi130 ceph-mon[84265]: pgmap v187: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-13T22:57:29.151 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:28 smithi130 ceph-mon[84265]: mds.? [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] up:boot 2024-05-13T22:57:29.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:28 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi102.sdylgv=up:active} 3 up:standby 2024-05-13T22:57:29.152 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:28 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:57:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:31 smithi130 ceph-mon[84265]: pgmap v188: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-13T22:57:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:57:31.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:31 smithi102 ceph-mon[111694]: pgmap v188: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-13T22:57:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:57:32.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:32 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:32.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:32 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:33.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:33 smithi130 ceph-mon[84265]: pgmap v189: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:33.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:33 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:33.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:33 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:33.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:33 smithi102 ceph-mon[111694]: pgmap v189: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:33.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:33 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:33.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:33 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:35.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:35 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:35.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:35 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:35.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:35 smithi102 ceph-mon[111694]: pgmap v190: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:35.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:35 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:35.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:35 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:35.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:35 smithi130 ceph-mon[84265]: pgmap v190: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:37.285 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:37 smithi102 ceph-mon[111694]: pgmap v191: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:37.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:37 smithi130 ceph-mon[84265]: pgmap v191: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:38.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:38.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:38.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:38.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:57:38.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:38.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:38.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:38.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:38.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:38.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:38.900 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:38 smithi130 ceph-mon[84265]: pgmap v192: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:38.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:38.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:38.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:38.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:57:38.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:38.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:38.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:38.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:38.927 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:38.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:38.928 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:38 smithi102 ceph-mon[111694]: pgmap v192: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:40.617 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:40 smithi102 ceph-mon[111694]: Upgrade: Updating mds.cephfs.smithi102.sdylgv 2024-05-13T22:57:40.617 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:40.617 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.sdylgv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:57:40.617 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:40.617 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:40 smithi102 ceph-mon[111694]: Deploying daemon mds.cephfs.smithi102.sdylgv on smithi102 2024-05-13T22:57:40.617 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:40 smithi102 ceph-mon[111694]: pgmap v193: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:40.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:40 smithi130 ceph-mon[84265]: Upgrade: Updating mds.cephfs.smithi102.sdylgv 2024-05-13T22:57:40.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:40.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi102.sdylgv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:57:40.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:57:40.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:40 smithi130 ceph-mon[84265]: Deploying daemon mds.cephfs.smithi102.sdylgv on smithi102 2024-05-13T22:57:40.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:40 smithi130 ceph-mon[84265]: pgmap v193: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:41.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:41 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102[111690]: 2024-05-13T22:57:41.438+0000 7fa332a0f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-13T22:57:41.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:41 smithi102 ceph-mon[111694]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-13T22:57:41.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:41 smithi102 ceph-mon[111694]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-13T22:57:41.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:41 smithi102 ceph-mon[111694]: osdmap e94: 6 total, 6 up, 6 in 2024-05-13T22:57:41.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:41 smithi102 ceph-mon[111694]: Standby daemon mds.cephfs.smithi130.ovlxvg assigned to filesystem cephfs as rank 0 2024-05-13T22:57:41.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:41 smithi102 ceph-mon[111694]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-13T22:57:41.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:41 smithi102 ceph-mon[111694]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-13T22:57:41.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:41 smithi102 ceph-mon[111694]: fsmap cephfs:1/1 {0=cephfs.smithi130.ovlxvg=up:replay} 2 up:standby 2024-05-13T22:57:41.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:41 smithi130 ceph-mon[84265]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-13T22:57:41.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:41 smithi130 ceph-mon[84265]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-13T22:57:41.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:41 smithi130 ceph-mon[84265]: osdmap e94: 6 total, 6 up, 6 in 2024-05-13T22:57:41.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:41 smithi130 ceph-mon[84265]: Standby daemon mds.cephfs.smithi130.ovlxvg assigned to filesystem cephfs as rank 0 2024-05-13T22:57:41.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:41 smithi130 ceph-mon[84265]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-13T22:57:41.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:41 smithi130 ceph-mon[84265]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-13T22:57:41.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:41 smithi130 ceph-mon[84265]: fsmap cephfs:1/1 {0=cephfs.smithi130.ovlxvg=up:replay} 2 up:standby 2024-05-13T22:57:42.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:42 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:57:42.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:42 smithi102 ceph-mon[111694]: pgmap v195: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:42.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:42 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:57:42.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:42 smithi130 ceph-mon[84265]: pgmap v195: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:57:45.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:45 smithi130 ceph-mon[84265]: pgmap v196: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 0 op/s 2024-05-13T22:57:45.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:45 smithi102 ceph-mon[111694]: pgmap v196: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 0 op/s 2024-05-13T22:57:47.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:47 smithi130 ceph-mon[84265]: pgmap v197: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 9.0 MiB/s rd, 3 op/s 2024-05-13T22:57:47.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:47.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:47 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:57:47.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:47 smithi102 ceph-mon[111694]: pgmap v197: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 9.0 MiB/s rd, 3 op/s 2024-05-13T22:57:47.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:47.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:47 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:57:49.371 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:49 smithi102 ceph-mon[111694]: pgmap v198: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-05-13T22:57:49.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:49 smithi130 ceph-mon[84265]: pgmap v198: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-05-13T22:57:51.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:51 smithi130 ceph-mon[84265]: pgmap v199: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-05-13T22:57:51.418 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:51 smithi102 ceph-mon[111694]: pgmap v199: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-05-13T22:57:52.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:52 smithi130 ceph-mon[84265]: reconnect by client.24321 192.168.0.1:0/2826286483 after 0.000999983 2024-05-13T22:57:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:52 smithi130 ceph-mon[84265]: reconnect by client.14554 192.168.0.1:0/2305339701 after 0.000999983 2024-05-13T22:57:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:52 smithi130 ceph-mon[84265]: mds.? [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] up:reconnect 2024-05-13T22:57:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:52 smithi130 ceph-mon[84265]: fsmap cephfs:1/1 {0=cephfs.smithi130.ovlxvg=up:reconnect} 2 up:standby 2024-05-13T22:57:52.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:52 smithi102 ceph-mon[111694]: reconnect by client.24321 192.168.0.1:0/2826286483 after 0.000999983 2024-05-13T22:57:52.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:52 smithi102 ceph-mon[111694]: reconnect by client.14554 192.168.0.1:0/2305339701 after 0.000999983 2024-05-13T22:57:52.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:52 smithi102 ceph-mon[111694]: mds.? [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] up:reconnect 2024-05-13T22:57:52.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:52 smithi102 ceph-mon[111694]: fsmap cephfs:1/1 {0=cephfs.smithi130.ovlxvg=up:reconnect} 2 up:standby 2024-05-13T22:57:53.175 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:53 smithi102 ceph-mon[111694]: pgmap v200: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-05-13T22:57:53.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:53 smithi102 ceph-mon[111694]: mds.? [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] up:rejoin 2024-05-13T22:57:53.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:53 smithi102 ceph-mon[111694]: fsmap cephfs:1/1 {0=cephfs.smithi130.ovlxvg=up:rejoin} 2 up:standby 2024-05-13T22:57:53.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:53 smithi102 ceph-mon[111694]: daemon mds.cephfs.smithi130.ovlxvg is now active in filesystem cephfs as rank 0 2024-05-13T22:57:53.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:53 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:53.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:53 smithi130 ceph-mon[84265]: pgmap v200: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-05-13T22:57:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:53 smithi130 ceph-mon[84265]: mds.? [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] up:rejoin 2024-05-13T22:57:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:53 smithi130 ceph-mon[84265]: fsmap cephfs:1/1 {0=cephfs.smithi130.ovlxvg=up:rejoin} 2 up:standby 2024-05-13T22:57:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:53 smithi130 ceph-mon[84265]: daemon mds.cephfs.smithi130.ovlxvg is now active in filesystem cephfs as rank 0 2024-05-13T22:57:53.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:53 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:54.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:54 smithi102 ceph-mon[111694]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-13T22:57:54.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:54 smithi102 ceph-mon[111694]: Cluster is now healthy 2024-05-13T22:57:54.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:54 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:54.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:54 smithi102 ceph-mon[111694]: mds.? [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] up:active 2024-05-13T22:57:54.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:54 smithi102 ceph-mon[111694]: mds.? [v2:172.21.15.102:6828/1256538594,v1:172.21.15.102:6829/1256538594] up:boot 2024-05-13T22:57:54.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:54 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi130.ovlxvg=up:active} 3 up:standby 2024-05-13T22:57:54.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:54 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:57:54.177 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:54 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:54.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:54 smithi130 ceph-mon[84265]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-13T22:57:54.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:54 smithi130 ceph-mon[84265]: Cluster is now healthy 2024-05-13T22:57:54.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:54 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:54.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:54 smithi130 ceph-mon[84265]: mds.? [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] up:active 2024-05-13T22:57:54.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:54 smithi130 ceph-mon[84265]: mds.? [v2:172.21.15.102:6828/1256538594,v1:172.21.15.102:6829/1256538594] up:boot 2024-05-13T22:57:54.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:54 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi130.ovlxvg=up:active} 3 up:standby 2024-05-13T22:57:54.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:54 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:57:54.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:54 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:57:54.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.929+0000 7fe9416b5700 1 -- 172.21.15.102:0/3124989485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe93c100dc0 msgr2=0x7fe93c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:54.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.929+0000 7fe9416b5700 1 --2- 172.21.15.102:0/3124989485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe93c100dc0 0x7fe93c1011a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009910 tx=0x7fe92c009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:54.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.929+0000 7fe9416b5700 1 -- 172.21.15.102:0/3124989485 shutdown_connections 2024-05-13T22:57:54.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.929+0000 7fe9416b5700 1 --2- 172.21.15.102:0/3124989485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe93c101770 0x7fe93c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:54.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.929+0000 7fe9416b5700 1 --2- 172.21.15.102:0/3124989485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe93c100dc0 0x7fe93c1011a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:54.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.929+0000 7fe9416b5700 1 -- 172.21.15.102:0/3124989485 >> 172.21.15.102:0/3124989485 conn(0x7fe93c0fc650 msgr2=0x7fe93c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:54.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.929+0000 7fe9416b5700 1 -- 172.21.15.102:0/3124989485 shutdown_connections 2024-05-13T22:57:54.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.929+0000 7fe9416b5700 1 -- 172.21.15.102:0/3124989485 wait complete. 2024-05-13T22:57:54.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.930+0000 7fe9416b5700 1 Processor -- start 2024-05-13T22:57:54.907 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.930+0000 7fe9416b5700 1 -- start start 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.930+0000 7fe9416b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe93c100dc0 0x7fe93c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.930+0000 7fe9416b5700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe93c101770 0x7fe93c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.930+0000 7fe9416b5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c19f420 con 0x7fe93c100dc0 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.930+0000 7fe9416b5700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c1988d0 con 0x7fe93c101770 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe93affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe93c100dc0 0x7fe93c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe93affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe93c100dc0 0x7fe93c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47548/0 (socket says 172.21.15.102:47548) 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe93affd700 1 -- 172.21.15.102:0/768492946 learned_addr learned my addr 172.21.15.102:0/768492946 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe93a7fc700 1 --2- 172.21.15.102:0/768492946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe93c101770 0x7fe93c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe93affd700 1 -- 172.21.15.102:0/768492946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe93c101770 msgr2=0x7fe93c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe93affd700 1 --2- 172.21.15.102:0/768492946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe93c101770 0x7fe93c19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe93affd700 1 -- 172.21.15.102:0/768492946 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c0095f0 con 0x7fe93c100dc0 2024-05-13T22:57:54.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe93affd700 1 --2- 172.21.15.102:0/768492946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe93c100dc0 0x7fe93c19e850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe92c006010 tx=0x7fe92c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:54.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe923fff700 1 -- 172.21.15.102:0/768492946 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe92c01d070 con 0x7fe93c100dc0 2024-05-13T22:57:54.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe923fff700 1 -- 172.21.15.102:0/768492946 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe92c004fb0 con 0x7fe93c100dc0 2024-05-13T22:57:54.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe9416b5700 1 -- 172.21.15.102:0/768492946 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93c198b50 con 0x7fe93c100dc0 2024-05-13T22:57:54.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe923fff700 1 -- 172.21.15.102:0/768492946 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe92c022ea0 con 0x7fe93c100dc0 2024-05-13T22:57:54.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.931+0000 7fe9416b5700 1 -- 172.21.15.102:0/768492946 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93c199040 con 0x7fe93c100dc0 2024-05-13T22:57:54.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.932+0000 7fe9416b5700 1 -- 172.21.15.102:0/768492946 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe93c104f40 con 0x7fe93c100dc0 2024-05-13T22:57:54.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.933+0000 7fe923fff700 1 -- 172.21.15.102:0/768492946 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe92c022470 con 0x7fe93c100dc0 2024-05-13T22:57:54.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.933+0000 7fe923fff700 1 --2- 172.21.15.102:0/768492946 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92406e200 0x7fe9240706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:54.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.933+0000 7fe923fff700 1 -- 172.21.15.102:0/768492946 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fe92c08f6f0 con 0x7fe93c100dc0 2024-05-13T22:57:54.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.933+0000 7fe93a7fc700 1 --2- 172.21.15.102:0/768492946 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92406e200 0x7fe9240706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:54.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.934+0000 7fe93a7fc700 1 --2- 172.21.15.102:0/768492946 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92406e200 0x7fe9240706c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe930006fd0 tx=0x7fe930008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:54.914 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:54.938+0000 7fe923fff700 1 -- 172.21.15.102:0/768492946 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe92c05c560 con 0x7fe93c100dc0 2024-05-13T22:57:55.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.184+0000 7fe9416b5700 1 -- 172.21.15.102:0/768492946 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe93c02ccc0 con 0x7fe92406e200 2024-05-13T22:57:55.161 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:55 smithi102 ceph-mon[111694]: pgmap v201: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 85 B/s wr, 5 op/s 2024-05-13T22:57:55.162 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.186+0000 7fe923fff700 1 -- 172.21.15.102:0/768492946 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe93c02ccc0 con 0x7fe92406e200 2024-05-13T22:57:55.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.188+0000 7fe9416b5700 1 -- 172.21.15.102:0/768492946 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92406e200 msgr2=0x7fe9240706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.188+0000 7fe9416b5700 1 --2- 172.21.15.102:0/768492946 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92406e200 0x7fe9240706c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe930006fd0 tx=0x7fe930008040 comp rx=0 tx=0).stop 2024-05-13T22:57:55.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.188+0000 7fe9416b5700 1 -- 172.21.15.102:0/768492946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe93c100dc0 msgr2=0x7fe93c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.188+0000 7fe9416b5700 1 --2- 172.21.15.102:0/768492946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe93c100dc0 0x7fe93c19e850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe92c006010 tx=0x7fe92c00be20 comp rx=0 tx=0).stop 2024-05-13T22:57:55.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.188+0000 7fe9416b5700 1 -- 172.21.15.102:0/768492946 shutdown_connections 2024-05-13T22:57:55.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.188+0000 7fe9416b5700 1 --2- 172.21.15.102:0/768492946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe93c100dc0 0x7fe93c19e850 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.188+0000 7fe9416b5700 1 --2- 172.21.15.102:0/768492946 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92406e200 0x7fe9240706c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.188+0000 7fe9416b5700 1 --2- 172.21.15.102:0/768492946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe93c101770 0x7fe93c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.188+0000 7fe9416b5700 1 -- 172.21.15.102:0/768492946 >> 172.21.15.102:0/768492946 conn(0x7fe93c0fc650 msgr2=0x7fe93c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:55.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.188+0000 7fe9416b5700 1 -- 172.21.15.102:0/768492946 shutdown_connections 2024-05-13T22:57:55.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.188+0000 7fe9416b5700 1 -- 172.21.15.102:0/768492946 wait complete. 2024-05-13T22:57:55.175 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:57:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.294+0000 7fcc99512700 1 -- 172.21.15.102:0/2628379139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc94101750 msgr2=0x7fcc941057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.294+0000 7fcc99512700 1 --2- 172.21.15.102:0/2628379139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc94101750 0x7fcc941057a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009910 tx=0x7fcc7c009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.294+0000 7fcc99512700 1 -- 172.21.15.102:0/2628379139 shutdown_connections 2024-05-13T22:57:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.294+0000 7fcc99512700 1 --2- 172.21.15.102:0/2628379139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc94101750 0x7fcc941057a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.294+0000 7fcc99512700 1 --2- 172.21.15.102:0/2628379139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc94100da0 0x7fcc94101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.294+0000 7fcc99512700 1 -- 172.21.15.102:0/2628379139 >> 172.21.15.102:0/2628379139 conn(0x7fcc940fc630 msgr2=0x7fcc940fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:55.272 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.294+0000 7fcc99512700 1 -- 172.21.15.102:0/2628379139 shutdown_connections 2024-05-13T22:57:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.294+0000 7fcc99512700 1 -- 172.21.15.102:0/2628379139 wait complete. 2024-05-13T22:57:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.295+0000 7fcc99512700 1 Processor -- start 2024-05-13T22:57:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.295+0000 7fcc99512700 1 -- start start 2024-05-13T22:57:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.295+0000 7fcc99512700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc94100da0 0x7fcc9419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.295+0000 7fcc99512700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc94101750 0x7fcc9419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.295+0000 7fcc99512700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9419d0d0 con 0x7fcc94100da0 2024-05-13T22:57:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.295+0000 7fcc99512700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94196580 con 0x7fcc94101750 2024-05-13T22:57:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.295+0000 7fcc92ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc94100da0 0x7fcc9419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:55.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.295+0000 7fcc92ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc94100da0 0x7fcc9419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47576/0 (socket says 172.21.15.102:47576) 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.295+0000 7fcc92ffd700 1 -- 172.21.15.102:0/1280597150 learned_addr learned my addr 172.21.15.102:0/1280597150 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.295+0000 7fcc927fc700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc94101750 0x7fcc9419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.296+0000 7fcc92ffd700 1 -- 172.21.15.102:0/1280597150 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc94101750 msgr2=0x7fcc9419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.296+0000 7fcc92ffd700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc94101750 0x7fcc9419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.296+0000 7fcc92ffd700 1 -- 172.21.15.102:0/1280597150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0095f0 con 0x7fcc94100da0 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.296+0000 7fcc927fc700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc94101750 0x7fcc9419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.296+0000 7fcc92ffd700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc94100da0 0x7fcc9419c500 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c8a0 tx=0x7fcc8400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.296+0000 7fcc8bfff700 1 -- 172.21.15.102:0/1280597150 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc8400cea0 con 0x7fcc94100da0 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.296+0000 7fcc8bfff700 1 -- 172.21.15.102:0/1280597150 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc84004830 con 0x7fcc94100da0 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.296+0000 7fcc8bfff700 1 -- 172.21.15.102:0/1280597150 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc840055b0 con 0x7fcc94100da0 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.296+0000 7fcc99512700 1 -- 172.21.15.102:0/1280597150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc94196860 con 0x7fcc94100da0 2024-05-13T22:57:55.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.296+0000 7fcc99512700 1 -- 172.21.15.102:0/1280597150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc94196db0 con 0x7fcc94100da0 2024-05-13T22:57:55.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.297+0000 7fcc99512700 1 -- 172.21.15.102:0/1280597150 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc941091a0 con 0x7fcc94100da0 2024-05-13T22:57:55.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.297+0000 7fcc8bfff700 1 -- 172.21.15.102:0/1280597150 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcc84007ad0 con 0x7fcc94100da0 2024-05-13T22:57:55.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.298+0000 7fcc8bfff700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc8006e200 0x7fcc800706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:55.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.298+0000 7fcc8bfff700 1 -- 172.21.15.102:0/1280597150 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fcc8408dff0 con 0x7fcc94100da0 2024-05-13T22:57:55.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.298+0000 7fcc927fc700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc8006e200 0x7fcc800706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:55.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.298+0000 7fcc927fc700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc8006e200 0x7fcc800706c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc940fce50 tx=0x7fcc7c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:55.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.302+0000 7fcc8bfff700 1 -- 172.21.15.102:0/1280597150 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcc8405ad10 con 0x7fcc94100da0 2024-05-13T22:57:55.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:55 smithi130 ceph-mon[84265]: pgmap v201: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 85 B/s wr, 5 op/s 2024-05-13T22:57:55.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.532+0000 7fcc99512700 1 -- 172.21.15.102:0/1280597150 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc94197090 con 0x7fcc8006e200 2024-05-13T22:57:55.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.533+0000 7fcc8bfff700 1 -- 172.21.15.102:0/1280597150 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcc94197090 con 0x7fcc8006e200 2024-05-13T22:57:55.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.535+0000 7fcc99512700 1 -- 172.21.15.102:0/1280597150 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc8006e200 msgr2=0x7fcc800706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.535+0000 7fcc99512700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc8006e200 0x7fcc800706c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc940fce50 tx=0x7fcc7c01a040 comp rx=0 tx=0).stop 2024-05-13T22:57:55.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.535+0000 7fcc99512700 1 -- 172.21.15.102:0/1280597150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc94100da0 msgr2=0x7fcc9419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.535+0000 7fcc99512700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc94100da0 0x7fcc9419c500 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c8a0 tx=0x7fcc8400cc60 comp rx=0 tx=0).stop 2024-05-13T22:57:55.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.535+0000 7fcc99512700 1 -- 172.21.15.102:0/1280597150 shutdown_connections 2024-05-13T22:57:55.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.535+0000 7fcc99512700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc94100da0 0x7fcc9419c500 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.535+0000 7fcc99512700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc8006e200 0x7fcc800706c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.535+0000 7fcc99512700 1 --2- 172.21.15.102:0/1280597150 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc94101750 0x7fcc9419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.535+0000 7fcc99512700 1 -- 172.21.15.102:0/1280597150 >> 172.21.15.102:0/1280597150 conn(0x7fcc940fc630 msgr2=0x7fcc940fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:55.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.536+0000 7fcc99512700 1 -- 172.21.15.102:0/1280597150 shutdown_connections 2024-05-13T22:57:55.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.536+0000 7fcc99512700 1 -- 172.21.15.102:0/1280597150 wait complete. 2024-05-13T22:57:55.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.642+0000 7f851a761700 1 -- 172.21.15.102:0/841224690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8514105a40 msgr2=0x7f8514105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.642+0000 7f851a761700 1 --2- 172.21.15.102:0/841224690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8514105a40 0x7f8514105e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8504009910 tx=0x7f8504009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:55.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.642+0000 7f851a761700 1 -- 172.21.15.102:0/841224690 shutdown_connections 2024-05-13T22:57:55.621 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.642+0000 7f851a761700 1 --2- 172.21.15.102:0/841224690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85140ffbd0 0x7f8514100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.642+0000 7f851a761700 1 --2- 172.21.15.102:0/841224690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8514105a40 0x7f8514105e20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.642+0000 7f851a761700 1 -- 172.21.15.102:0/841224690 >> 172.21.15.102:0/841224690 conn(0x7f85140755a0 msgr2=0x7f85140759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:55.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.642+0000 7f851a761700 1 -- 172.21.15.102:0/841224690 shutdown_connections 2024-05-13T22:57:55.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.642+0000 7f851a761700 1 -- 172.21.15.102:0/841224690 wait complete. 2024-05-13T22:57:55.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f851a761700 1 Processor -- start 2024-05-13T22:57:55.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f851a761700 1 -- start start 2024-05-13T22:57:55.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f851a761700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85140ffbd0 0x7f85141954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:55.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f851a761700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8514105a40 0x7f8514195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:55.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f851a761700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8514195f70 con 0x7f8514105a40 2024-05-13T22:57:55.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f851a761700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85141960e0 con 0x7f85140ffbd0 2024-05-13T22:57:55.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f85137fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8514105a40 0x7f8514195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:55.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f85137fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8514105a40 0x7f8514195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47596/0 (socket says 172.21.15.102:47596) 2024-05-13T22:57:55.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f85137fe700 1 -- 172.21.15.102:0/794168928 learned_addr learned my addr 172.21.15.102:0/794168928 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:55.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f8513fff700 1 --2- 172.21.15.102:0/794168928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85140ffbd0 0x7f85141954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:55.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f85137fe700 1 -- 172.21.15.102:0/794168928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85140ffbd0 msgr2=0x7f85141954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f85137fe700 1 --2- 172.21.15.102:0/794168928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85140ffbd0 0x7f85141954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.643+0000 7f85137fe700 1 -- 172.21.15.102:0/794168928 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85040095f0 con 0x7f8514105a40 2024-05-13T22:57:55.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.644+0000 7f85137fe700 1 --2- 172.21.15.102:0/794168928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8514105a40 0x7f8514195a30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f850800b4b0 tx=0x7f850800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:55.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.644+0000 7f85117fa700 1 -- 172.21.15.102:0/794168928 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8508011800 con 0x7f8514105a40 2024-05-13T22:57:55.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.644+0000 7f851a761700 1 -- 172.21.15.102:0/794168928 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85141a2b60 con 0x7f8514105a40 2024-05-13T22:57:55.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.644+0000 7f85117fa700 1 -- 172.21.15.102:0/794168928 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8508011e20 con 0x7f8514105a40 2024-05-13T22:57:55.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.644+0000 7f85117fa700 1 -- 172.21.15.102:0/794168928 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f850800f2e0 con 0x7f8514105a40 2024-05-13T22:57:55.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.644+0000 7f851a761700 1 -- 172.21.15.102:0/794168928 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85141a2ec0 con 0x7f8514105a40 2024-05-13T22:57:55.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.645+0000 7f851a761700 1 -- 172.21.15.102:0/794168928 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8514108040 con 0x7f8514105a40 2024-05-13T22:57:55.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.645+0000 7f85117fa700 1 -- 172.21.15.102:0/794168928 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8508010380 con 0x7f8514105a40 2024-05-13T22:57:55.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.646+0000 7f85117fa700 1 --2- 172.21.15.102:0/794168928 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f84fc06e200 0x7f84fc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:55.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.646+0000 7f85117fa700 1 -- 172.21.15.102:0/794168928 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f850808ddc0 con 0x7f8514105a40 2024-05-13T22:57:55.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.646+0000 7f8513fff700 1 --2- 172.21.15.102:0/794168928 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f84fc06e200 0x7f84fc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:55.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.646+0000 7f8513fff700 1 --2- 172.21.15.102:0/794168928 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f84fc06e200 0x7f84fc0706c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f850400b5c0 tx=0x7f850401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:55.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.650+0000 7f85117fa700 1 -- 172.21.15.102:0/794168928 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8508056fd0 con 0x7f8514105a40 2024-05-13T22:57:55.841 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.865+0000 7f851a761700 1 -- 172.21.15.102:0/794168928 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f851402cce0 con 0x7f84fc06e200 2024-05-13T22:57:55.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.876+0000 7f85117fa700 1 -- 172.21.15.102:0/794168928 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f851402cce0 con 0x7f84fc06e200 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (7m) 22s ago 18m 26.7M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (18m) 22s ago 18m 9084k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (17m) 44s ago 17m 8476k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (4m) 22s ago 18m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (4m) 44s ago 17m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (6m) 22s ago 17m 75.2M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (28s) 22s ago 12m 14.9M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1a34f8c7d234 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 starting - - - - 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (12m) 44s ago 12m 17.5M - 18.2.1 5be31c24972a 26cc76cac865 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (12m) 44s ago 12m 20.5M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:57:55.853 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (9m) 22s ago 19m 511M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (8m) 44s ago 17m 453M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (5m) 22s ago 19m 56.5M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (4m) 44s ago 17m 44.2M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (7m) 22s ago 18m 15.9M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (7m) 44s ago 17m 17.3M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (3m) 22s ago 16m 137M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 0ada88c777a2 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (3m) 22s ago 15m 129M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 c3e9e68b06fd 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (2m) 22s ago 15m 97.2M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 1b85728dccf5 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (115s) 44s ago 15m 100M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 66e804d4b801 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (81s) 44s ago 14m 94.7M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5bb5a8f94042 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (48s) 44s ago 14m 13.9M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5d983135967d 2024-05-13T22:57:55.854 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (7m) 22s ago 17m 75.9M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:57:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.879+0000 7f851a761700 1 -- 172.21.15.102:0/794168928 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f84fc06e200 msgr2=0x7f84fc0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.879+0000 7f851a761700 1 --2- 172.21.15.102:0/794168928 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f84fc06e200 0x7f84fc0706c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f850400b5c0 tx=0x7f850401a040 comp rx=0 tx=0).stop 2024-05-13T22:57:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.879+0000 7f851a761700 1 -- 172.21.15.102:0/794168928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8514105a40 msgr2=0x7f8514195a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.879+0000 7f851a761700 1 --2- 172.21.15.102:0/794168928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8514105a40 0x7f8514195a30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f850800b4b0 tx=0x7f850800b870 comp rx=0 tx=0).stop 2024-05-13T22:57:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.879+0000 7f851a761700 1 -- 172.21.15.102:0/794168928 shutdown_connections 2024-05-13T22:57:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.879+0000 7f851a761700 1 --2- 172.21.15.102:0/794168928 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f84fc06e200 0x7f84fc0706c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.855 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.879+0000 7f851a761700 1 --2- 172.21.15.102:0/794168928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f85140ffbd0 0x7f85141954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.879+0000 7f851a761700 1 --2- 172.21.15.102:0/794168928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8514105a40 0x7f8514195a30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.879+0000 7f851a761700 1 -- 172.21.15.102:0/794168928 >> 172.21.15.102:0/794168928 conn(0x7f85140755a0 msgr2=0x7f85140fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.879+0000 7f851a761700 1 -- 172.21.15.102:0/794168928 shutdown_connections 2024-05-13T22:57:55.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.879+0000 7f851a761700 1 -- 172.21.15.102:0/794168928 wait complete. 2024-05-13T22:57:55.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.984+0000 7fc344d26700 1 -- 172.21.15.102:0/587076242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc340100da0 msgr2=0x7fc340101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.960 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.984+0000 7fc344d26700 1 --2- 172.21.15.102:0/587076242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc340100da0 0x7fc340101180 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc33800b0a0 tx=0x7fc33800b3b0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.984+0000 7fc344d26700 1 -- 172.21.15.102:0/587076242 shutdown_connections 2024-05-13T22:57:55.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.984+0000 7fc344d26700 1 --2- 172.21.15.102:0/587076242 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc340101750 0x7fc3401057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.984+0000 7fc344d26700 1 --2- 172.21.15.102:0/587076242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc340100da0 0x7fc340101180 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.984+0000 7fc344d26700 1 -- 172.21.15.102:0/587076242 >> 172.21.15.102:0/587076242 conn(0x7fc3400fc650 msgr2=0x7fc3400fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:55.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.984+0000 7fc344d26700 1 -- 172.21.15.102:0/587076242 shutdown_connections 2024-05-13T22:57:55.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.984+0000 7fc344d26700 1 -- 172.21.15.102:0/587076242 wait complete. 2024-05-13T22:57:55.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.985+0000 7fc344d26700 1 Processor -- start 2024-05-13T22:57:55.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.985+0000 7fc344d26700 1 -- start start 2024-05-13T22:57:55.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.985+0000 7fc344d26700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc340100da0 0x7fc34019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:55.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.985+0000 7fc344d26700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc340101750 0x7fc34019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:55.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.985+0000 7fc344d26700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34019d1f0 con 0x7fc340101750 2024-05-13T22:57:55.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.985+0000 7fc344d26700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3401966a0 con 0x7fc340100da0 2024-05-13T22:57:55.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.985+0000 7fc33dd9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc340101750 0x7fc34019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:55.963 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.985+0000 7fc33dd9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc340101750 0x7fc34019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47614/0 (socket says 172.21.15.102:47614) 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.985+0000 7fc33dd9b700 1 -- 172.21.15.102:0/1308138711 learned_addr learned my addr 172.21.15.102:0/1308138711 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.986+0000 7fc33e59c700 1 --2- 172.21.15.102:0/1308138711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc340100da0 0x7fc34019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.986+0000 7fc33dd9b700 1 -- 172.21.15.102:0/1308138711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc340100da0 msgr2=0x7fc34019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.986+0000 7fc33dd9b700 1 --2- 172.21.15.102:0/1308138711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc340100da0 0x7fc34019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.986+0000 7fc33dd9b700 1 -- 172.21.15.102:0/1308138711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338009d00 con 0x7fc340101750 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.986+0000 7fc33dd9b700 1 --2- 172.21.15.102:0/1308138711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc340101750 0x7fc34019cb60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc32800b480 tx=0x7fc32800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.986+0000 7fc3377fe700 1 -- 172.21.15.102:0/1308138711 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3280118c0 con 0x7fc340101750 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.986+0000 7fc3377fe700 1 -- 172.21.15.102:0/1308138711 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc328010380 con 0x7fc340101750 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.986+0000 7fc344d26700 1 -- 172.21.15.102:0/1308138711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc340196980 con 0x7fc340101750 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.986+0000 7fc3377fe700 1 -- 172.21.15.102:0/1308138711 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc32800f4f0 con 0x7fc340101750 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.986+0000 7fc344d26700 1 -- 172.21.15.102:0/1308138711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc340196ed0 con 0x7fc340101750 2024-05-13T22:57:55.964 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.987+0000 7fc344d26700 1 -- 172.21.15.102:0/1308138711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc340109140 con 0x7fc340101750 2024-05-13T22:57:55.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.988+0000 7fc3377fe700 1 -- 172.21.15.102:0/1308138711 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc3280104f0 con 0x7fc340101750 2024-05-13T22:57:55.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.988+0000 7fc3377fe700 1 --2- 172.21.15.102:0/1308138711 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc32c06e200 0x7fc32c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:55.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.988+0000 7fc3377fe700 1 -- 172.21.15.102:0/1308138711 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fc32808ddf0 con 0x7fc340101750 2024-05-13T22:57:55.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.988+0000 7fc33e59c700 1 --2- 172.21.15.102:0/1308138711 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc32c06e200 0x7fc32c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:55.965 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.988+0000 7fc33e59c700 1 --2- 172.21.15.102:0/1308138711 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc32c06e200 0x7fc32c0706c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc33800b830 tx=0x7fc3380097c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:55.968 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:55.993+0000 7fc3377fe700 1 -- 172.21.15.102:0/1308138711 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc32805abb0 con 0x7fc340101750 2024-05-13T22:57:56.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.281+0000 7fc344d26700 1 -- 172.21.15.102:0/1308138711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc34004ea90 con 0x7fc340101750 2024-05-13T22:57:56.258 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.282+0000 7fc3377fe700 1 -- 172.21.15.102:0/1308138711 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fc32805a740 con 0x7fc340101750 2024-05-13T22:57:56.258 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:57:56.258 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:57:56.258 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:57:56.258 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:57:56.258 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:57:56.258 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:57:56.258 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 6 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 12 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:57:56.259 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:57:56.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.284+0000 7fc344d26700 1 -- 172.21.15.102:0/1308138711 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc32c06e200 msgr2=0x7fc32c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:56.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.284+0000 7fc344d26700 1 --2- 172.21.15.102:0/1308138711 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc32c06e200 0x7fc32c0706c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc33800b830 tx=0x7fc3380097c0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.260 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.284+0000 7fc344d26700 1 -- 172.21.15.102:0/1308138711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc340101750 msgr2=0x7fc34019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:56.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.284+0000 7fc344d26700 1 --2- 172.21.15.102:0/1308138711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc340101750 0x7fc34019cb60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc32800b480 tx=0x7fc32800b840 comp rx=0 tx=0).stop 2024-05-13T22:57:56.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.284+0000 7fc344d26700 1 -- 172.21.15.102:0/1308138711 shutdown_connections 2024-05-13T22:57:56.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.284+0000 7fc344d26700 1 --2- 172.21.15.102:0/1308138711 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc32c06e200 0x7fc32c0706c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.284+0000 7fc344d26700 1 --2- 172.21.15.102:0/1308138711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc340100da0 0x7fc34019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.284+0000 7fc344d26700 1 --2- 172.21.15.102:0/1308138711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc340101750 0x7fc34019cb60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.284+0000 7fc344d26700 1 -- 172.21.15.102:0/1308138711 >> 172.21.15.102:0/1308138711 conn(0x7fc3400fc650 msgr2=0x7fc3400fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:56.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.284+0000 7fc344d26700 1 -- 172.21.15.102:0/1308138711 shutdown_connections 2024-05-13T22:57:56.261 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.284+0000 7fc344d26700 1 -- 172.21.15.102:0/1308138711 wait complete. 2024-05-13T22:57:56.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.391+0000 7fb83679a700 1 -- 172.21.15.102:0/3676651881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb830100da0 msgr2=0x7fb830101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:56.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.391+0000 7fb83679a700 1 --2- 172.21.15.102:0/3676651881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb830100da0 0x7fb830101180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb818009910 tx=0x7fb818009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:56.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.391+0000 7fb83679a700 1 -- 172.21.15.102:0/3676651881 shutdown_connections 2024-05-13T22:57:56.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.391+0000 7fb83679a700 1 --2- 172.21.15.102:0/3676651881 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb830101750 0x7fb8301057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.391+0000 7fb83679a700 1 --2- 172.21.15.102:0/3676651881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb830100da0 0x7fb830101180 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.368 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.391+0000 7fb83679a700 1 -- 172.21.15.102:0/3676651881 >> 172.21.15.102:0/3676651881 conn(0x7fb8300fc630 msgr2=0x7fb8300fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:56.368 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:56 smithi102 ceph-mon[111694]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:56.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.391+0000 7fb83679a700 1 -- 172.21.15.102:0/3676651881 shutdown_connections 2024-05-13T22:57:56.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.391+0000 7fb83679a700 1 -- 172.21.15.102:0/3676651881 wait complete. 2024-05-13T22:57:56.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.392+0000 7fb83679a700 1 Processor -- start 2024-05-13T22:57:56.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.392+0000 7fb83679a700 1 -- start start 2024-05-13T22:57:56.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.392+0000 7fb83679a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb830100da0 0x7fb830073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:56.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.392+0000 7fb83679a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb830101750 0x7fb8300735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:56.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.392+0000 7fb83679a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb830073b10 con 0x7fb830100da0 2024-05-13T22:57:56.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.392+0000 7fb83679a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb830073c50 con 0x7fb830101750 2024-05-13T22:57:56.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.392+0000 7fb82ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb830100da0 0x7fb830073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:56.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.392+0000 7fb82ffff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb830100da0 0x7fb830073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47626/0 (socket says 172.21.15.102:47626) 2024-05-13T22:57:56.370 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.392+0000 7fb82ffff700 1 -- 172.21.15.102:0/372413799 learned_addr learned my addr 172.21.15.102:0/372413799 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:56.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.393+0000 7fb82ffff700 1 -- 172.21.15.102:0/372413799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb830101750 msgr2=0x7fb8300735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:56.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.393+0000 7fb82ffff700 1 --2- 172.21.15.102:0/372413799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb830101750 0x7fb8300735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.393+0000 7fb82ffff700 1 -- 172.21.15.102:0/372413799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8180095f0 con 0x7fb830100da0 2024-05-13T22:57:56.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.393+0000 7fb82ffff700 1 --2- 172.21.15.102:0/372413799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb830100da0 0x7fb830073090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb818000c00 tx=0x7fb81800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:56.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.393+0000 7fb82d7fa700 1 -- 172.21.15.102:0/372413799 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb81801d070 con 0x7fb830100da0 2024-05-13T22:57:56.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.393+0000 7fb82d7fa700 1 -- 172.21.15.102:0/372413799 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb81800f460 con 0x7fb830100da0 2024-05-13T22:57:56.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.393+0000 7fb83679a700 1 -- 172.21.15.102:0/372413799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb830073ed0 con 0x7fb830100da0 2024-05-13T22:57:56.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.393+0000 7fb82d7fa700 1 -- 172.21.15.102:0/372413799 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb818017630 con 0x7fb830100da0 2024-05-13T22:57:56.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.393+0000 7fb83679a700 1 -- 172.21.15.102:0/372413799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8301a2cd0 con 0x7fb830100da0 2024-05-13T22:57:56.371 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.394+0000 7fb83679a700 1 -- 172.21.15.102:0/372413799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb830109140 con 0x7fb830100da0 2024-05-13T22:57:56.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.395+0000 7fb82d7fa700 1 -- 172.21.15.102:0/372413799 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb81800f5d0 con 0x7fb830100da0 2024-05-13T22:57:56.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.395+0000 7fb82d7fa700 1 --2- 172.21.15.102:0/372413799 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb81c06e200 0x7fb81c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:56.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.395+0000 7fb82f7fe700 1 --2- 172.21.15.102:0/372413799 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb81c06e200 0x7fb81c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:56.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.395+0000 7fb82d7fa700 1 -- 172.21.15.102:0/372413799 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fb81808f840 con 0x7fb830100da0 2024-05-13T22:57:56.372 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.396+0000 7fb82f7fe700 1 --2- 172.21.15.102:0/372413799 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb81c06e200 0x7fb81c0706c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb830074880 tx=0x7fb820005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:56.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.400+0000 7fb82d7fa700 1 -- 172.21.15.102:0/372413799 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb818058a50 con 0x7fb830100da0 2024-05-13T22:57:56.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:56 smithi130 ceph-mon[84265]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:56.622 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.646+0000 7fb83679a700 1 -- 172.21.15.102:0/372413799 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb83002cd30 con 0x7fb830100da0 2024-05-13T22:57:56.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.647+0000 7fb82d7fa700 1 -- 172.21.15.102:0/372413799 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1895 (secure 0 0 0) 0x7fb81805c110 con 0x7fb830100da0 2024-05-13T22:57:56.623 INFO:teuthology.orchestra.run.smithi102.stdout:e26 2024-05-13T22:57:56.623 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:57:56.623 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:57:56.623 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:57:56.623 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:57:56.623 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:57:56.623 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:57:56.623 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 26 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:57:53.095951+0000 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 94 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:57:56.624 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=24275} 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 24275 members: 24275 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{0:24275} state up:active seq 190 join_fscid=1 addr [v2:172.21.15.130:6826/633282433,v1:172.21.15.130:6827/633282433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:57:56.625 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:57:56.626 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:57:56.626 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:57:56.626 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:24281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/244897459,v1:172.21.15.130:6825/244897459] compat {c=[1],r=[1],i=[7ff]}] 2024-05-13T22:57:56.626 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{-1:34410} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T22:57:56.626 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.102:6828/1256538594,v1:172.21.15.102:6829/1256538594] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T22:57:56.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.649+0000 7fb83679a700 1 -- 172.21.15.102:0/372413799 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb81c06e200 msgr2=0x7fb81c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:56.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.649+0000 7fb83679a700 1 --2- 172.21.15.102:0/372413799 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb81c06e200 0x7fb81c0706c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb830074880 tx=0x7fb820005c80 comp rx=0 tx=0).stop 2024-05-13T22:57:56.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.649+0000 7fb83679a700 1 -- 172.21.15.102:0/372413799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb830100da0 msgr2=0x7fb830073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:56.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.649+0000 7fb83679a700 1 --2- 172.21.15.102:0/372413799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb830100da0 0x7fb830073090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb818000c00 tx=0x7fb81800be20 comp rx=0 tx=0).stop 2024-05-13T22:57:56.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.649+0000 7fb83679a700 1 -- 172.21.15.102:0/372413799 shutdown_connections 2024-05-13T22:57:56.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.649+0000 7fb83679a700 1 --2- 172.21.15.102:0/372413799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb830100da0 0x7fb830073090 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.649+0000 7fb83679a700 1 --2- 172.21.15.102:0/372413799 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb81c06e200 0x7fb81c0706c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.649+0000 7fb83679a700 1 --2- 172.21.15.102:0/372413799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb830101750 0x7fb8300735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.649+0000 7fb83679a700 1 -- 172.21.15.102:0/372413799 >> 172.21.15.102:0/372413799 conn(0x7fb8300fc630 msgr2=0x7fb8300fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:56.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.649+0000 7fb83679a700 1 -- 172.21.15.102:0/372413799 shutdown_connections 2024-05-13T22:57:56.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.649+0000 7fb83679a700 1 -- 172.21.15.102:0/372413799 wait complete. 2024-05-13T22:57:56.628 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 26 2024-05-13T22:57:56.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.759+0000 7fd605cae700 1 -- 172.21.15.102:0/3048491457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd600101750 msgr2=0x7fd6001057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:56.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.759+0000 7fd605cae700 1 --2- 172.21.15.102:0/3048491457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd600101750 0x7fd6001057a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009910 tx=0x7fd5f4009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:56.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.759+0000 7fd605cae700 1 -- 172.21.15.102:0/3048491457 shutdown_connections 2024-05-13T22:57:56.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.759+0000 7fd605cae700 1 --2- 172.21.15.102:0/3048491457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd600101750 0x7fd6001057a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.759+0000 7fd605cae700 1 --2- 172.21.15.102:0/3048491457 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd600100da0 0x7fd600101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.759+0000 7fd605cae700 1 -- 172.21.15.102:0/3048491457 >> 172.21.15.102:0/3048491457 conn(0x7fd6000fc650 msgr2=0x7fd6000fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:56.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.759+0000 7fd605cae700 1 -- 172.21.15.102:0/3048491457 shutdown_connections 2024-05-13T22:57:56.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.759+0000 7fd605cae700 1 -- 172.21.15.102:0/3048491457 wait complete. 2024-05-13T22:57:56.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.760+0000 7fd605cae700 1 Processor -- start 2024-05-13T22:57:56.737 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.760+0000 7fd605cae700 1 -- start start 2024-05-13T22:57:56.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.760+0000 7fd605cae700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd600100da0 0x7fd600073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:56.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.760+0000 7fd605cae700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd600101750 0x7fd6000735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:56.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.760+0000 7fd605cae700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd600073b10 con 0x7fd600100da0 2024-05-13T22:57:56.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.760+0000 7fd605cae700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd600073c50 con 0x7fd600101750 2024-05-13T22:57:56.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd5ff7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd600100da0 0x7fd600073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:56.738 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd5ff7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd600100da0 0x7fd600073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47638/0 (socket says 172.21.15.102:47638) 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd5ff7fe700 1 -- 172.21.15.102:0/4288131367 learned_addr learned my addr 172.21.15.102:0/4288131367 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd5feffd700 1 --2- 172.21.15.102:0/4288131367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd600101750 0x7fd6000735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd5ff7fe700 1 -- 172.21.15.102:0/4288131367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd600101750 msgr2=0x7fd6000735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd5ff7fe700 1 --2- 172.21.15.102:0/4288131367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd600101750 0x7fd6000735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd5ff7fe700 1 -- 172.21.15.102:0/4288131367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f40095f0 con 0x7fd600100da0 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd5ff7fe700 1 --2- 172.21.15.102:0/4288131367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd600100da0 0x7fd600073090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000d8d0 tx=0x7fd5f000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd5fcff9700 1 -- 172.21.15.102:0/4288131367 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f00098a0 con 0x7fd600100da0 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd5fcff9700 1 -- 172.21.15.102:0/4288131367 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5f0010460 con 0x7fd600100da0 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd605cae700 1 -- 172.21.15.102:0/4288131367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6001a2910 con 0x7fd600100da0 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd5fcff9700 1 -- 172.21.15.102:0/4288131367 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f000f5f0 con 0x7fd600100da0 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.761+0000 7fd605cae700 1 -- 172.21.15.102:0/4288131367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6001a2d20 con 0x7fd600100da0 2024-05-13T22:57:56.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.762+0000 7fd605cae700 1 -- 172.21.15.102:0/4288131367 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6001091a0 con 0x7fd600100da0 2024-05-13T22:57:56.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.763+0000 7fd5fcff9700 1 -- 172.21.15.102:0/4288131367 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd5f0009a00 con 0x7fd600100da0 2024-05-13T22:57:56.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.763+0000 7fd5fcff9700 1 --2- 172.21.15.102:0/4288131367 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd5e806e200 0x7fd5e80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:56.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.763+0000 7fd5fcff9700 1 -- 172.21.15.102:0/4288131367 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7fd5f008e040 con 0x7fd600100da0 2024-05-13T22:57:56.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.763+0000 7fd5feffd700 1 --2- 172.21.15.102:0/4288131367 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd5e806e200 0x7fd5e80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:56.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.764+0000 7fd5feffd700 1 --2- 172.21.15.102:0/4288131367 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd5e806e200 0x7fd5e80706c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd600074880 tx=0x7fd5f401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:56.744 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:56.768+0000 7fd5fcff9700 1 -- 172.21.15.102:0/4288131367 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd5f00572d0 con 0x7fd600100da0 2024-05-13T22:57:56.977 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.001+0000 7fd605cae700 1 -- 172.21.15.102:0/4288131367 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd60002cdd0 con 0x7fd5e806e200 2024-05-13T22:57:56.978 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.002+0000 7fd5fcff9700 1 -- 172.21.15.102:0/4288131367 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd60002cdd0 con 0x7fd5e806e200 2024-05-13T22:57:56.978 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:57:56.978 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:57:56.978 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:57:56.978 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:57:56.978 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:57:56.979 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-05-13T22:57:56.979 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr", 2024-05-13T22:57:56.979 INFO:teuthology.orchestra.run.smithi102.stdout: "mon", 2024-05-13T22:57:56.979 INFO:teuthology.orchestra.run.smithi102.stdout: "osd" 2024-05-13T22:57:56.979 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:57:56.979 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "13/23 daemons upgraded", 2024-05-13T22:57:56.979 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading mds daemons", 2024-05-13T22:57:56.979 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:57:56.979 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:57:56.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.005+0000 7fd605cae700 1 -- 172.21.15.102:0/4288131367 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd5e806e200 msgr2=0x7fd5e80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:56.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.005+0000 7fd605cae700 1 --2- 172.21.15.102:0/4288131367 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd5e806e200 0x7fd5e80706c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd600074880 tx=0x7fd5f401a040 comp rx=0 tx=0).stop 2024-05-13T22:57:56.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.005+0000 7fd605cae700 1 -- 172.21.15.102:0/4288131367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd600100da0 msgr2=0x7fd600073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:56.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.005+0000 7fd605cae700 1 --2- 172.21.15.102:0/4288131367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd600100da0 0x7fd600073090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000d8d0 tx=0x7fd5f000dbe0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.005+0000 7fd605cae700 1 -- 172.21.15.102:0/4288131367 shutdown_connections 2024-05-13T22:57:56.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.005+0000 7fd605cae700 1 --2- 172.21.15.102:0/4288131367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd600100da0 0x7fd600073090 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.005+0000 7fd605cae700 1 --2- 172.21.15.102:0/4288131367 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd5e806e200 0x7fd5e80706c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.005+0000 7fd605cae700 1 --2- 172.21.15.102:0/4288131367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd600101750 0x7fd6000735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:56.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.005+0000 7fd605cae700 1 -- 172.21.15.102:0/4288131367 >> 172.21.15.102:0/4288131367 conn(0x7fd6000fc650 msgr2=0x7fd6000fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:56.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.005+0000 7fd605cae700 1 -- 172.21.15.102:0/4288131367 shutdown_connections 2024-05-13T22:57:56.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.005+0000 7fd605cae700 1 -- 172.21.15.102:0/4288131367 wait complete. 2024-05-13T22:57:57.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.115+0000 7f9b993d3700 1 -- 172.21.15.102:0/1759701510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b940ffbd0 msgr2=0x7f9b94100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:57.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.115+0000 7f9b993d3700 1 --2- 172.21.15.102:0/1759701510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b940ffbd0 0x7f9b94100050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009910 tx=0x7f9b7c009c20 comp rx=0 tx=0).stop 2024-05-13T22:57:57.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.115+0000 7f9b993d3700 1 -- 172.21.15.102:0/1759701510 shutdown_connections 2024-05-13T22:57:57.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.115+0000 7f9b993d3700 1 --2- 172.21.15.102:0/1759701510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b940ffbd0 0x7f9b94100050 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:57.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.115+0000 7f9b993d3700 1 --2- 172.21.15.102:0/1759701510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94105a40 0x7f9b94105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:57.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.115+0000 7f9b993d3700 1 -- 172.21.15.102:0/1759701510 >> 172.21.15.102:0/1759701510 conn(0x7f9b940755a0 msgr2=0x7f9b940759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:57.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.115+0000 7f9b993d3700 1 -- 172.21.15.102:0/1759701510 shutdown_connections 2024-05-13T22:57:57.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.115+0000 7f9b993d3700 1 -- 172.21.15.102:0/1759701510 wait complete. 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b993d3700 1 Processor -- start 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b993d3700 1 -- start start 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b993d3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b940ffbd0 0x7f9b94197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b993d3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94105a40 0x7f9b94197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b993d3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b94198190 con 0x7f9b940ffbd0 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b993d3700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b94198300 con 0x7f9b94105a40 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b92ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b940ffbd0 0x7f9b94197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b92ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b940ffbd0 0x7f9b94197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47658/0 (socket says 172.21.15.102:47658) 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b92ffd700 1 -- 172.21.15.102:0/303652534 learned_addr learned my addr 172.21.15.102:0/303652534 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b8bfff700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94105a40 0x7f9b94197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b92ffd700 1 -- 172.21.15.102:0/303652534 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94105a40 msgr2=0x7f9b94197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b92ffd700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94105a40 0x7f9b94197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b92ffd700 1 -- 172.21.15.102:0/303652534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c0095f0 con 0x7f9b940ffbd0 2024-05-13T22:57:57.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b8bfff700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94105a40 0x7f9b94197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T22:57:57.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.116+0000 7f9b92ffd700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b940ffbd0 0x7f9b94197570 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400c8a0 tx=0x7f9b8400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:57.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.117+0000 7f9b90ff9700 1 -- 172.21.15.102:0/303652534 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b8400cea0 con 0x7f9b940ffbd0 2024-05-13T22:57:57.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.117+0000 7f9b90ff9700 1 -- 172.21.15.102:0/303652534 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b84004830 con 0x7f9b940ffbd0 2024-05-13T22:57:57.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.117+0000 7f9b993d3700 1 -- 172.21.15.102:0/303652534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b941a05e0 con 0x7f9b940ffbd0 2024-05-13T22:57:57.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.117+0000 7f9b90ff9700 1 -- 172.21.15.102:0/303652534 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b840055b0 con 0x7f9b940ffbd0 2024-05-13T22:57:57.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.117+0000 7f9b993d3700 1 -- 172.21.15.102:0/303652534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b941a0b30 con 0x7f9b940ffbd0 2024-05-13T22:57:57.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.118+0000 7f9b993d3700 1 -- 172.21.15.102:0/303652534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b94107fd0 con 0x7f9b940ffbd0 2024-05-13T22:57:57.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.118+0000 7f9b90ff9700 1 -- 172.21.15.102:0/303652534 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9b840049a0 con 0x7f9b940ffbd0 2024-05-13T22:57:57.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.119+0000 7f9b90ff9700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b8006e200 0x7f9b800706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:57:57.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.119+0000 7f9b90ff9700 1 -- 172.21.15.102:0/303652534 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6790+0+0 (secure 0 0 0) 0x7f9b8408df10 con 0x7f9b940ffbd0 2024-05-13T22:57:57.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.119+0000 7f9b8bfff700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b8006e200 0x7f9b800706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:57:57.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.119+0000 7f9b8bfff700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b8006e200 0x7f9b800706c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00b5c0 tx=0x7f9b7c005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:57:57.099 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.123+0000 7f9b90ff9700 1 -- 172.21.15.102:0/303652534 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f9b84057120 con 0x7f9b940ffbd0 2024-05-13T22:57:57.300 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:57 smithi130 ceph-mon[84265]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:57.300 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:57 smithi130 ceph-mon[84265]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:57.300 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:57 smithi130 ceph-mon[84265]: pgmap v202: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-13T22:57:57.300 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:57 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1308138711' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:57.300 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:57 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/372413799' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:57:57.392 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.416+0000 7f9b993d3700 1 -- 172.21.15.102:0/303652534 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9b941079b0 con 0x7f9b940ffbd0 2024-05-13T22:57:57.392 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:57 smithi102 ceph-mon[111694]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:57.393 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:57 smithi102 ceph-mon[111694]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:57.393 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:57 smithi102 ceph-mon[111694]: pgmap v202: 65 pgs: 65 active+clean; 212 MiB data, 908 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-13T22:57:57.393 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:57 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1308138711' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:57:57.393 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:57 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/372413799' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:57:57.394 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:57:57.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.416+0000 7f9b90ff9700 1 -- 172.21.15.102:0/303652534 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f9b8401d090 con 0x7f9b940ffbd0 2024-05-13T22:57:57.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.419+0000 7f9b993d3700 1 -- 172.21.15.102:0/303652534 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b8006e200 msgr2=0x7f9b800706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:57.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.419+0000 7f9b993d3700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b8006e200 0x7f9b800706c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00b5c0 tx=0x7f9b7c005480 comp rx=0 tx=0).stop 2024-05-13T22:57:57.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.419+0000 7f9b993d3700 1 -- 172.21.15.102:0/303652534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b940ffbd0 msgr2=0x7f9b94197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:57:57.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.419+0000 7f9b993d3700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b940ffbd0 0x7f9b94197570 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400c8a0 tx=0x7f9b8400cc60 comp rx=0 tx=0).stop 2024-05-13T22:57:57.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.419+0000 7f9b993d3700 1 -- 172.21.15.102:0/303652534 shutdown_connections 2024-05-13T22:57:57.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.419+0000 7f9b993d3700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b940ffbd0 0x7f9b94197570 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:57.395 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.419+0000 7f9b993d3700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f9b8006e200 0x7f9b800706c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:57.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.419+0000 7f9b993d3700 1 --2- 172.21.15.102:0/303652534 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b94105a40 0x7f9b94197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:57:57.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.419+0000 7f9b993d3700 1 -- 172.21.15.102:0/303652534 >> 172.21.15.102:0/303652534 conn(0x7f9b940755a0 msgr2=0x7f9b940fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:57:57.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.419+0000 7f9b993d3700 1 -- 172.21.15.102:0/303652534 shutdown_connections 2024-05-13T22:57:57.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:57:57.419+0000 7f9b993d3700 1 -- 172.21.15.102:0/303652534 wait complete. 2024-05-13T22:57:58.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:58 smithi130 ceph-mon[84265]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:58.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:58 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/303652534' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:57:58.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:58 smithi102 ceph-mon[111694]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:57:58.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:58 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/303652534' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:57:59.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:59 smithi130 ceph-mon[84265]: pgmap v203: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 9.7 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-13T22:57:59.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:59.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:57:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:59.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:59 smithi102 ceph-mon[111694]: pgmap v203: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 9.7 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-13T22:57:59.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:57:59.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:57:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:00.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:00 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:00.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:00 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:00.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:00 smithi102 ceph-mon[111694]: pgmap v204: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-05-13T22:58:00.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:00 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:00.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:00 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:00.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:00 smithi130 ceph-mon[84265]: pgmap v204: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-05-13T22:58:02.310 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:02.310 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:58:02.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:02.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:58:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:03 smithi130 ceph-mon[84265]: pgmap v205: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-05-13T22:58:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:03 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:03 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:03 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:03 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:58:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:03 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:03 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:03 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:03 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:03 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:03.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:03 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:03.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:03 smithi102 ceph-mon[111694]: pgmap v205: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-05-13T22:58:03.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:03 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:03 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:03 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:03 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:58:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:03 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:03 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:03 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:03 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:03 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:03.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:03 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:05.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:05 smithi130 ceph-mon[84265]: pgmap v206: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 345 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-05-13T22:58:05.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:05 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:05.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:05 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.hzjanp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:58:05.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:05 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:05.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:05 smithi102 ceph-mon[111694]: pgmap v206: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 345 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-05-13T22:58:05.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:05 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:05.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:05 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.hzjanp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:58:05.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:05 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:06.301 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:06 smithi130 ceph-mon[84265]: Upgrade: Updating mds.cephfs.smithi130.hzjanp 2024-05-13T22:58:06.301 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:06 smithi130 ceph-mon[84265]: Deploying daemon mds.cephfs.smithi130.hzjanp on smithi130 2024-05-13T22:58:06.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:06 smithi102 ceph-mon[111694]: Upgrade: Updating mds.cephfs.smithi130.hzjanp 2024-05-13T22:58:06.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:06 smithi102 ceph-mon[111694]: Deploying daemon mds.cephfs.smithi130.hzjanp on smithi130 2024-05-13T22:58:07.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:07 smithi130 ceph-mon[84265]: pgmap v207: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-05-13T22:58:07.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:07 smithi102 ceph-mon[111694]: pgmap v207: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-05-13T22:58:08.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:08 smithi130 ceph-mon[84265]: osdmap e95: 6 total, 6 up, 6 in 2024-05-13T22:58:08.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:08 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi130.ovlxvg=up:active} 2 up:standby 2024-05-13T22:58:08.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:08 smithi102 ceph-mon[111694]: osdmap e95: 6 total, 6 up, 6 in 2024-05-13T22:58:08.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:08 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi130.ovlxvg=up:active} 2 up:standby 2024-05-13T22:58:09.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:09 smithi130 ceph-mon[84265]: pgmap v209: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:09.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:09 smithi102 ceph-mon[111694]: pgmap v209: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:11.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:11 smithi130 ceph-mon[84265]: pgmap v210: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:11.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:11 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:11.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:11 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:11.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:11 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:11.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:11 smithi102 ceph-mon[111694]: pgmap v210: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:11.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:11 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:11.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:11 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:11.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:11 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:12.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:12 smithi130 ceph-mon[84265]: mds.? [v2:172.21.15.130:6824/3836279348,v1:172.21.15.130:6825/3836279348] up:boot 2024-05-13T22:58:12.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:12 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi130.ovlxvg=up:active} 3 up:standby 2024-05-13T22:58:12.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:58:12.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:12 smithi102 ceph-mon[111694]: mds.? [v2:172.21.15.130:6824/3836279348,v1:172.21.15.130:6825/3836279348] up:boot 2024-05-13T22:58:12.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:12 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi130.ovlxvg=up:active} 3 up:standby 2024-05-13T22:58:12.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:58:13.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:13 smithi130 ceph-mon[84265]: pgmap v211: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:13.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:13 smithi102 ceph-mon[111694]: pgmap v211: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:15.148 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:14 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:15.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:14 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:15.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:14 smithi130 ceph-mon[84265]: pgmap v212: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:15.175 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:14 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:15.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:14 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:15.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:14 smithi102 ceph-mon[111694]: pgmap v212: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:15.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:15 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:15.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:15 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:16.175 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:15 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:16.175 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:15 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:17.148 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:58:17.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:16 smithi130 ceph-mon[84265]: pgmap v213: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:17.175 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:58:17.175 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:16 smithi102 ceph-mon[111694]: pgmap v213: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:19.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:19 smithi102 ceph-mon[111694]: pgmap v214: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:58:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:19 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:19.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:19 smithi130 ceph-mon[84265]: pgmap v214: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:19.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:19.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:19.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:19.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:58:19.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:19.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:19.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:19.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:19.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:19.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:19 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:21.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:20 smithi130 ceph-mon[84265]: Upgrade: Updating mds.cephfs.smithi130.ovlxvg 2024-05-13T22:58:21.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:21.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.ovlxvg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:58:21.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:20 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:21.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:20 smithi130 ceph-mon[84265]: Deploying daemon mds.cephfs.smithi130.ovlxvg on smithi130 2024-05-13T22:58:21.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:20 smithi130 ceph-mon[84265]: pgmap v215: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:21.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:20 smithi102 ceph-mon[111694]: Upgrade: Updating mds.cephfs.smithi130.ovlxvg 2024-05-13T22:58:21.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:21.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi130.ovlxvg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-13T22:58:21.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:20 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:21.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:20 smithi102 ceph-mon[111694]: Deploying daemon mds.cephfs.smithi130.ovlxvg on smithi130 2024-05-13T22:58:21.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:20 smithi102 ceph-mon[111694]: pgmap v215: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:22.400 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:22 smithi130 ceph-mon[84265]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-13T22:58:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:22 smithi130 ceph-mon[84265]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-13T22:58:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:22 smithi130 ceph-mon[84265]: osdmap e96: 6 total, 6 up, 6 in 2024-05-13T22:58:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:22 smithi130 ceph-mon[84265]: Standby daemon mds.cephfs.smithi102.cgkcwz assigned to filesystem cephfs as rank 0 2024-05-13T22:58:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:22 smithi130 ceph-mon[84265]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-13T22:58:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:22 smithi130 ceph-mon[84265]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-13T22:58:22.401 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:22 smithi130 ceph-mon[84265]: fsmap cephfs:1/1 {0=cephfs.smithi102.cgkcwz=up:replay} 2 up:standby 2024-05-13T22:58:22.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:22 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102[111690]: 2024-05-13T22:58:22.091+0000 7fa332a0f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-13T22:58:22.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:22 smithi102 ceph-mon[111694]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-13T22:58:22.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:22 smithi102 ceph-mon[111694]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-13T22:58:22.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:22 smithi102 ceph-mon[111694]: osdmap e96: 6 total, 6 up, 6 in 2024-05-13T22:58:22.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:22 smithi102 ceph-mon[111694]: Standby daemon mds.cephfs.smithi102.cgkcwz assigned to filesystem cephfs as rank 0 2024-05-13T22:58:22.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:22 smithi102 ceph-mon[111694]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-13T22:58:22.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:22 smithi102 ceph-mon[111694]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-13T22:58:22.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:22 smithi102 ceph-mon[111694]: fsmap cephfs:1/1 {0=cephfs.smithi102.cgkcwz=up:replay} 2 up:standby 2024-05-13T22:58:23.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:23 smithi130 ceph-mon[84265]: pgmap v216: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:23.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:23 smithi102 ceph-mon[111694]: pgmap v216: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail 2024-05-13T22:58:25.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:25 smithi130 ceph-mon[84265]: pgmap v218: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 0 op/s 2024-05-13T22:58:25.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:25 smithi102 ceph-mon[111694]: pgmap v218: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 0 op/s 2024-05-13T22:58:27.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:27 smithi130 ceph-mon[84265]: pgmap v219: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s rd, 2 op/s 2024-05-13T22:58:27.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:27 smithi102 ceph-mon[111694]: pgmap v219: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s rd, 2 op/s 2024-05-13T22:58:27.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.531+0000 7fcc7adff700 1 -- 172.21.15.102:0/465296933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc74101d00 msgr2=0x7fcc741020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:27.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.531+0000 7fcc7adff700 1 --2- 172.21.15.102:0/465296933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc74101d00 0x7fcc741020e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009910 tx=0x7fcc68009c20 comp rx=0 tx=0).stop 2024-05-13T22:58:27.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.531+0000 7fcc7adff700 1 -- 172.21.15.102:0/465296933 shutdown_connections 2024-05-13T22:58:27.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.531+0000 7fcc7adff700 1 --2- 172.21.15.102:0/465296933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc74102620 0x7fcc7410ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:27.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.531+0000 7fcc7adff700 1 --2- 172.21.15.102:0/465296933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc74101d00 0x7fcc741020e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:27.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.531+0000 7fcc7adff700 1 -- 172.21.15.102:0/465296933 >> 172.21.15.102:0/465296933 conn(0x7fcc740fb570 msgr2=0x7fcc740fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:27.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.531+0000 7fcc7adff700 1 -- 172.21.15.102:0/465296933 shutdown_connections 2024-05-13T22:58:27.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.531+0000 7fcc7adff700 1 -- 172.21.15.102:0/465296933 wait complete. 2024-05-13T22:58:27.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.532+0000 7fcc7adff700 1 Processor -- start 2024-05-13T22:58:27.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.532+0000 7fcc7adff700 1 -- start start 2024-05-13T22:58:27.509 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.532+0000 7fcc7adff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc74101d00 0x7fcc7419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:27.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.532+0000 7fcc7adff700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc74102620 0x7fcc7419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:27.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.532+0000 7fcc7adff700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7419d100 con 0x7fcc74101d00 2024-05-13T22:58:27.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.532+0000 7fcc7adff700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc741965b0 con 0x7fcc74102620 2024-05-13T22:58:27.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.532+0000 7fcc78b9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc74101d00 0x7fcc7419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:27.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc78b9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc74101d00 0x7fcc7419c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57646/0 (socket says 172.21.15.102:57646) 2024-05-13T22:58:27.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc78b9b700 1 -- 172.21.15.102:0/376621157 learned_addr learned my addr 172.21.15.102:0/376621157 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:58:27.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc73fff700 1 --2- 172.21.15.102:0/376621157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc74102620 0x7fcc7419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:27.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc78b9b700 1 -- 172.21.15.102:0/376621157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc74102620 msgr2=0x7fcc7419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:27.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc78b9b700 1 --2- 172.21.15.102:0/376621157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc74102620 0x7fcc7419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:27.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc78b9b700 1 -- 172.21.15.102:0/376621157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680095f0 con 0x7fcc74101d00 2024-05-13T22:58:27.510 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc78b9b700 1 --2- 172.21.15.102:0/376621157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc74101d00 0x7fcc7419c530 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc68005f50 tx=0x7fcc6800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:27.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc71ffb700 1 -- 172.21.15.102:0/376621157 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc680052a0 con 0x7fcc74101d00 2024-05-13T22:58:27.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc7adff700 1 -- 172.21.15.102:0/376621157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc74196830 con 0x7fcc74101d00 2024-05-13T22:58:27.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc71ffb700 1 -- 172.21.15.102:0/376621157 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc68023070 con 0x7fcc74101d00 2024-05-13T22:58:27.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc71ffb700 1 -- 172.21.15.102:0/376621157 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc68003cf0 con 0x7fcc74101d00 2024-05-13T22:58:27.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.533+0000 7fcc7adff700 1 -- 172.21.15.102:0/376621157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc74196d20 con 0x7fcc74101d00 2024-05-13T22:58:27.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.535+0000 7fcc7adff700 1 -- 172.21.15.102:0/376621157 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc74108210 con 0x7fcc74101d00 2024-05-13T22:58:27.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.535+0000 7fcc71ffb700 1 -- 172.21.15.102:0/376621157 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcc68025030 con 0x7fcc74101d00 2024-05-13T22:58:27.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.535+0000 7fcc71ffb700 1 --2- 172.21.15.102:0/376621157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc6406e200 0x7fcc640706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:27.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.536+0000 7fcc71ffb700 1 -- 172.21.15.102:0/376621157 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fcc680932c0 con 0x7fcc74101d00 2024-05-13T22:58:27.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.536+0000 7fcc73fff700 1 --2- 172.21.15.102:0/376621157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc6406e200 0x7fcc640706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:27.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.536+0000 7fcc73fff700 1 --2- 172.21.15.102:0/376621157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc6406e200 0x7fcc640706c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcc740fcc90 tx=0x7fcc60009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:27.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.540+0000 7fcc71ffb700 1 -- 172.21.15.102:0/376621157 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fcc68060030 con 0x7fcc74101d00 2024-05-13T22:58:27.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.765+0000 7fcc7adff700 1 -- 172.21.15.102:0/376621157 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc74197050 con 0x7fcc6406e200 2024-05-13T22:58:27.743 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.766+0000 7fcc71ffb700 1 -- 172.21.15.102:0/376621157 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcc74197050 con 0x7fcc6406e200 2024-05-13T22:58:27.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.769+0000 7fcc7adff700 1 -- 172.21.15.102:0/376621157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc6406e200 msgr2=0x7fcc640706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:27.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.769+0000 7fcc7adff700 1 --2- 172.21.15.102:0/376621157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc6406e200 0x7fcc640706c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcc740fcc90 tx=0x7fcc60009260 comp rx=0 tx=0).stop 2024-05-13T22:58:27.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.769+0000 7fcc7adff700 1 -- 172.21.15.102:0/376621157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc74101d00 msgr2=0x7fcc7419c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:27.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.769+0000 7fcc7adff700 1 --2- 172.21.15.102:0/376621157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc74101d00 0x7fcc7419c530 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc68005f50 tx=0x7fcc6800b610 comp rx=0 tx=0).stop 2024-05-13T22:58:27.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.769+0000 7fcc7adff700 1 -- 172.21.15.102:0/376621157 shutdown_connections 2024-05-13T22:58:27.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.769+0000 7fcc7adff700 1 --2- 172.21.15.102:0/376621157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcc74101d00 0x7fcc7419c530 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:27.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.769+0000 7fcc7adff700 1 --2- 172.21.15.102:0/376621157 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fcc6406e200 0x7fcc640706c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:27.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.769+0000 7fcc7adff700 1 --2- 172.21.15.102:0/376621157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc74102620 0x7fcc7419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:27.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.769+0000 7fcc7adff700 1 -- 172.21.15.102:0/376621157 >> 172.21.15.102:0/376621157 conn(0x7fcc740fb570 msgr2=0x7fcc74105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:27.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.769+0000 7fcc7adff700 1 -- 172.21.15.102:0/376621157 shutdown_connections 2024-05-13T22:58:27.745 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.769+0000 7fcc7adff700 1 -- 172.21.15.102:0/376621157 wait complete. 2024-05-13T22:58:27.755 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:58:27.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.877+0000 7fd490419700 1 -- 172.21.15.102:0/3186992191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4880ffbd0 msgr2=0x7fd488100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:27.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.877+0000 7fd490419700 1 --2- 172.21.15.102:0/3186992191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4880ffbd0 0x7fd488100050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd478009910 tx=0x7fd478009c20 comp rx=0 tx=0).stop 2024-05-13T22:58:27.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.877+0000 7fd490419700 1 -- 172.21.15.102:0/3186992191 shutdown_connections 2024-05-13T22:58:27.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.877+0000 7fd490419700 1 --2- 172.21.15.102:0/3186992191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4880ffbd0 0x7fd488100050 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:27.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.877+0000 7fd490419700 1 --2- 172.21.15.102:0/3186992191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd488105a40 0x7fd488105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:27.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.877+0000 7fd490419700 1 -- 172.21.15.102:0/3186992191 >> 172.21.15.102:0/3186992191 conn(0x7fd4880755a0 msgr2=0x7fd4880759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:27.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.877+0000 7fd490419700 1 -- 172.21.15.102:0/3186992191 shutdown_connections 2024-05-13T22:58:27.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.877+0000 7fd490419700 1 -- 172.21.15.102:0/3186992191 wait complete. 2024-05-13T22:58:27.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.878+0000 7fd490419700 1 Processor -- start 2024-05-13T22:58:27.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.878+0000 7fd490419700 1 -- start start 2024-05-13T22:58:27.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.878+0000 7fd490419700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4880ffbd0 0x7fd48819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:27.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.878+0000 7fd490419700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd488105a40 0x7fd48819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:27.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.878+0000 7fd490419700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48819f2b0 con 0x7fd4880ffbd0 2024-05-13T22:58:27.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.878+0000 7fd490419700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd488198730 con 0x7fd488105a40 2024-05-13T22:58:27.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.878+0000 7fd48e1b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4880ffbd0 0x7fd48819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:27.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.878+0000 7fd48e1b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4880ffbd0 0x7fd48819e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57656/0 (socket says 172.21.15.102:57656) 2024-05-13T22:58:27.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.878+0000 7fd48e1b5700 1 -- 172.21.15.102:0/4190458 learned_addr learned my addr 172.21.15.102:0/4190458 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:58:27.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.879+0000 7fd48d9b4700 1 --2- 172.21.15.102:0/4190458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd488105a40 0x7fd48819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:27.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.879+0000 7fd48e1b5700 1 -- 172.21.15.102:0/4190458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd488105a40 msgr2=0x7fd48819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:27.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.879+0000 7fd48e1b5700 1 --2- 172.21.15.102:0/4190458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd488105a40 0x7fd48819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:27.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.879+0000 7fd48e1b5700 1 -- 172.21.15.102:0/4190458 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4780095f0 con 0x7fd4880ffbd0 2024-05-13T22:58:27.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.879+0000 7fd48e1b5700 1 --2- 172.21.15.102:0/4190458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4880ffbd0 0x7fd48819e690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd48400d900 tx=0x7fd48400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:27.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.879+0000 7fd47f7fe700 1 -- 172.21.15.102:0/4190458 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4840041f0 con 0x7fd4880ffbd0 2024-05-13T22:58:27.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.879+0000 7fd490419700 1 -- 172.21.15.102:0/4190458 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd488198a10 con 0x7fd4880ffbd0 2024-05-13T22:58:27.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.879+0000 7fd490419700 1 -- 172.21.15.102:0/4190458 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd488198f60 con 0x7fd4880ffbd0 2024-05-13T22:58:27.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.879+0000 7fd47f7fe700 1 -- 172.21.15.102:0/4190458 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd484004cf0 con 0x7fd4880ffbd0 2024-05-13T22:58:27.857 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.880+0000 7fd47f7fe700 1 -- 172.21.15.102:0/4190458 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd48400b750 con 0x7fd4880ffbd0 2024-05-13T22:58:27.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.881+0000 7fd490419700 1 -- 172.21.15.102:0/4190458 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c005320 con 0x7fd4880ffbd0 2024-05-13T22:58:27.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.881+0000 7fd47f7fe700 1 -- 172.21.15.102:0/4190458 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd484004830 con 0x7fd4880ffbd0 2024-05-13T22:58:27.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.882+0000 7fd47f7fe700 1 --2- 172.21.15.102:0/4190458 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd47406e200 0x7fd4740706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:27.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.882+0000 7fd47f7fe700 1 -- 172.21.15.102:0/4190458 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd48408e030 con 0x7fd4880ffbd0 2024-05-13T22:58:27.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.882+0000 7fd48d9b4700 1 --2- 172.21.15.102:0/4190458 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd47406e200 0x7fd4740706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:27.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.882+0000 7fd48d9b4700 1 --2- 172.21.15.102:0/4190458 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd47406e200 0x7fd4740706c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd488199f20 tx=0x7fd47801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:27.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:27.887+0000 7fd47f7fe700 1 -- 172.21.15.102:0/4190458 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd48405ada0 con 0x7fd4880ffbd0 2024-05-13T22:58:28.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.112+0000 7fd490419700 1 -- 172.21.15.102:0/4190458 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd46c000bf0 con 0x7fd47406e200 2024-05-13T22:58:28.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.113+0000 7fd47f7fe700 1 -- 172.21.15.102:0/4190458 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd46c000bf0 con 0x7fd47406e200 2024-05-13T22:58:28.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.115+0000 7fd490419700 1 -- 172.21.15.102:0/4190458 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd47406e200 msgr2=0x7fd4740706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.115+0000 7fd490419700 1 --2- 172.21.15.102:0/4190458 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd47406e200 0x7fd4740706c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd488199f20 tx=0x7fd47801a040 comp rx=0 tx=0).stop 2024-05-13T22:58:28.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.115+0000 7fd490419700 1 -- 172.21.15.102:0/4190458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4880ffbd0 msgr2=0x7fd48819e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.115+0000 7fd490419700 1 --2- 172.21.15.102:0/4190458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4880ffbd0 0x7fd48819e690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd48400d900 tx=0x7fd48400dc10 comp rx=0 tx=0).stop 2024-05-13T22:58:28.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.115+0000 7fd490419700 1 -- 172.21.15.102:0/4190458 shutdown_connections 2024-05-13T22:58:28.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.115+0000 7fd490419700 1 --2- 172.21.15.102:0/4190458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4880ffbd0 0x7fd48819e690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.115+0000 7fd490419700 1 --2- 172.21.15.102:0/4190458 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd47406e200 0x7fd4740706c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.115+0000 7fd490419700 1 --2- 172.21.15.102:0/4190458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd488105a40 0x7fd48819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.115+0000 7fd490419700 1 -- 172.21.15.102:0/4190458 >> 172.21.15.102:0/4190458 conn(0x7fd4880755a0 msgr2=0x7fd4880684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:28.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.116+0000 7fd490419700 1 -- 172.21.15.102:0/4190458 shutdown_connections 2024-05-13T22:58:28.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.116+0000 7fd490419700 1 -- 172.21.15.102:0/4190458 wait complete. 2024-05-13T22:58:28.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.223+0000 7f75b8408700 1 -- 172.21.15.102:0/2820582599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b0101d00 msgr2=0x7f75b01020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.223+0000 7f75b8408700 1 --2- 172.21.15.102:0/2820582599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b0101d00 0x7f75b01020e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009910 tx=0x7f75ac009c20 comp rx=0 tx=0).stop 2024-05-13T22:58:28.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.223+0000 7f75b8408700 1 -- 172.21.15.102:0/2820582599 shutdown_connections 2024-05-13T22:58:28.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.223+0000 7f75b8408700 1 --2- 172.21.15.102:0/2820582599 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b0102620 0x7f75b010ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.223+0000 7f75b8408700 1 --2- 172.21.15.102:0/2820582599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b0101d00 0x7f75b01020e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.223+0000 7f75b8408700 1 -- 172.21.15.102:0/2820582599 >> 172.21.15.102:0/2820582599 conn(0x7f75b00fb570 msgr2=0x7f75b00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:28.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.223+0000 7f75b8408700 1 -- 172.21.15.102:0/2820582599 shutdown_connections 2024-05-13T22:58:28.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.223+0000 7f75b8408700 1 -- 172.21.15.102:0/2820582599 wait complete. 2024-05-13T22:58:28.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b8408700 1 Processor -- start 2024-05-13T22:58:28.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b8408700 1 -- start start 2024-05-13T22:58:28.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b8408700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b0101d00 0x7f75b019c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:28.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b8408700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b0102620 0x7f75b019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:28.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b8408700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b019d100 con 0x7f75b0101d00 2024-05-13T22:58:28.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b8408700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b01965b0 con 0x7f75b0102620 2024-05-13T22:58:28.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b61a4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b0101d00 0x7f75b019c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:28.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b61a4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b0101d00 0x7f75b019c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57682/0 (socket says 172.21.15.102:57682) 2024-05-13T22:58:28.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b61a4700 1 -- 172.21.15.102:0/2541419480 learned_addr learned my addr 172.21.15.102:0/2541419480 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:58:28.207 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b59a3700 1 --2- 172.21.15.102:0/2541419480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b0102620 0x7f75b019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:28.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b61a4700 1 -- 172.21.15.102:0/2541419480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b0102620 msgr2=0x7f75b019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b61a4700 1 --2- 172.21.15.102:0/2541419480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b0102620 0x7f75b019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.224+0000 7f75b61a4700 1 -- 172.21.15.102:0/2541419480 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac0095f0 con 0x7f75b0101d00 2024-05-13T22:58:28.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.225+0000 7f75b61a4700 1 --2- 172.21.15.102:0/2541419480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b0101d00 0x7f75b019c530 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f75ac006010 tx=0x7f75ac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:28.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.225+0000 7f75a77fe700 1 -- 172.21.15.102:0/2541419480 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75ac01d070 con 0x7f75b0101d00 2024-05-13T22:58:28.208 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.225+0000 7f75a77fe700 1 -- 172.21.15.102:0/2541419480 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75ac005dc0 con 0x7f75b0101d00 2024-05-13T22:58:28.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.225+0000 7f75b8408700 1 -- 172.21.15.102:0/2541419480 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b0196830 con 0x7f75b0101d00 2024-05-13T22:58:28.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.225+0000 7f75a77fe700 1 -- 172.21.15.102:0/2541419480 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75ac017400 con 0x7f75b0101d00 2024-05-13T22:58:28.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.225+0000 7f75b8408700 1 -- 172.21.15.102:0/2541419480 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b0196d20 con 0x7f75b0101d00 2024-05-13T22:58:28.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.226+0000 7f75b8408700 1 -- 172.21.15.102:0/2541419480 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b0108210 con 0x7f75b0101d00 2024-05-13T22:58:28.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.227+0000 7f75a77fe700 1 -- 172.21.15.102:0/2541419480 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f75ac003a60 con 0x7f75b0101d00 2024-05-13T22:58:28.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.227+0000 7f75a77fe700 1 --2- 172.21.15.102:0/2541419480 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f759c06e200 0x7f759c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:28.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.227+0000 7f75a77fe700 1 -- 172.21.15.102:0/2541419480 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f75ac08f670 con 0x7f75b0101d00 2024-05-13T22:58:28.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.227+0000 7f75b59a3700 1 --2- 172.21.15.102:0/2541419480 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f759c06e200 0x7f759c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:28.210 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.228+0000 7f75b59a3700 1 --2- 172.21.15.102:0/2541419480 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f759c06e200 0x7f759c0706c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f75b00fcc90 tx=0x7f75a0007320 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:28.211 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.232+0000 7f75a77fe700 1 -- 172.21.15.102:0/2541419480 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f75ac05c3e0 con 0x7f75b0101d00 2024-05-13T22:58:28.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.453+0000 7f75b8408700 1 -- 172.21.15.102:0/2541419480 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f75b002cd00 con 0x7f759c06e200 2024-05-13T22:58:28.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.473+0000 7f75a77fe700 1 -- 172.21.15.102:0/2541419480 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f75b002cd00 con 0x7f759c06e200 2024-05-13T22:58:28.450 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:58:28.450 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (7m) 30s ago 18m 26.7M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:58:28.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (18m) 30s ago 18m 9114k - 18.2.1 5be31c24972a aafec7f3815d 2024-05-13T22:58:28.450 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (17m) 14s ago 17m 8950k - 18.2.1 5be31c24972a 169b0543484d 2024-05-13T22:58:28.450 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (4m) 30s ago 18m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T22:58:28.450 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (4m) 14s ago 17m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T22:58:28.450 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (7m) 30s ago 18m 79.9M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:58:28.450 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (61s) 30s ago 13m 15.9M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1a34f8c7d234 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (35s) 30s ago 13m 11.7M - 19.0.0-2565-gc3e55145 b1b56bed7c91 2b50dedff13c 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (18s) 14s ago 13m 12.5M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1a45ebbee664 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (13m) 14s ago 13m 86.1M - 18.2.1 5be31c24972a ba00d6d7be4a 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (9m) 30s ago 20m 511M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (8m) 14s ago 17m 453M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (5m) 30s ago 20m 57.1M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (5m) 14s ago 17m 47.6M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (8m) 30s ago 18m 16.0M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (8m) 14s ago 17m 17.1M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (4m) 30s ago 16m 141M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 0ada88c777a2 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (3m) 30s ago 16m 118M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 c3e9e68b06fd 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (3m) 30s ago 15m 104M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 1b85728dccf5 2024-05-13T22:58:28.451 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (2m) 14s ago 15m 108M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 66e804d4b801 2024-05-13T22:58:28.452 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (113s) 14s ago 15m 99.3M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5bb5a8f94042 2024-05-13T22:58:28.452 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (80s) 14s ago 14m 100M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5d983135967d 2024-05-13T22:58:28.452 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (7m) 30s ago 18m 75.9M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:58:28.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.477+0000 7f75b8408700 1 -- 172.21.15.102:0/2541419480 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f759c06e200 msgr2=0x7f759c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.477+0000 7f75b8408700 1 --2- 172.21.15.102:0/2541419480 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f759c06e200 0x7f759c0706c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f75b00fcc90 tx=0x7f75a0007320 comp rx=0 tx=0).stop 2024-05-13T22:58:28.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.477+0000 7f75b8408700 1 -- 172.21.15.102:0/2541419480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b0101d00 msgr2=0x7f75b019c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.477+0000 7f75b8408700 1 --2- 172.21.15.102:0/2541419480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b0101d00 0x7f75b019c530 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f75ac006010 tx=0x7f75ac00bd70 comp rx=0 tx=0).stop 2024-05-13T22:58:28.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.477+0000 7f75b8408700 1 -- 172.21.15.102:0/2541419480 shutdown_connections 2024-05-13T22:58:28.453 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.477+0000 7f75b8408700 1 --2- 172.21.15.102:0/2541419480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b0101d00 0x7f75b019c530 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.477+0000 7f75b8408700 1 --2- 172.21.15.102:0/2541419480 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f759c06e200 0x7f759c0706c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.477+0000 7f75b8408700 1 --2- 172.21.15.102:0/2541419480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b0102620 0x7f75b019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.477+0000 7f75b8408700 1 -- 172.21.15.102:0/2541419480 >> 172.21.15.102:0/2541419480 conn(0x7f75b00fb570 msgr2=0x7f75b0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:28.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.477+0000 7f75b8408700 1 -- 172.21.15.102:0/2541419480 shutdown_connections 2024-05-13T22:58:28.454 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.477+0000 7f75b8408700 1 -- 172.21.15.102:0/2541419480 wait complete. 2024-05-13T22:58:28.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.586+0000 7fa98e001700 1 -- 172.21.15.102:0/1337612562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa988101750 msgr2=0x7fa9881057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.586+0000 7fa98e001700 1 --2- 172.21.15.102:0/1337612562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa988101750 0x7fa9881057a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009910 tx=0x7fa97c009c20 comp rx=0 tx=0).stop 2024-05-13T22:58:28.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.586+0000 7fa98e001700 1 -- 172.21.15.102:0/1337612562 shutdown_connections 2024-05-13T22:58:28.562 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.586+0000 7fa98e001700 1 --2- 172.21.15.102:0/1337612562 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa988101750 0x7fa9881057a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.586+0000 7fa98e001700 1 --2- 172.21.15.102:0/1337612562 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa988100da0 0x7fa988101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.586+0000 7fa98e001700 1 -- 172.21.15.102:0/1337612562 >> 172.21.15.102:0/1337612562 conn(0x7fa9880fc630 msgr2=0x7fa9880fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:28.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.586+0000 7fa98e001700 1 -- 172.21.15.102:0/1337612562 shutdown_connections 2024-05-13T22:58:28.563 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.586+0000 7fa98e001700 1 -- 172.21.15.102:0/1337612562 wait complete. 2024-05-13T22:58:28.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.587+0000 7fa98e001700 1 Processor -- start 2024-05-13T22:58:28.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.587+0000 7fa98e001700 1 -- start start 2024-05-13T22:58:28.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa98e001700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa988100da0 0x7fa98819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa98e001700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa988101750 0x7fa98819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa98e001700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98819d0d0 con 0x7fa988101750 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa98e001700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa988196580 con 0x7fa988100da0 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa986ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa988101750 0x7fa98819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa986ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa988101750 0x7fa98819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44072/0 (socket says 172.21.15.102:44072) 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa9877fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa988100da0 0x7fa98819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa986ffd700 1 -- 172.21.15.102:0/1321385126 learned_addr learned my addr 172.21.15.102:0/1321385126 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa9877fe700 1 -- 172.21.15.102:0/1321385126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa988101750 msgr2=0x7fa98819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa9877fe700 1 --2- 172.21.15.102:0/1321385126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa988101750 0x7fa98819ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa9877fe700 1 -- 172.21.15.102:0/1321385126 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c0095f0 con 0x7fa988100da0 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa986ffd700 1 --2- 172.21.15.102:0/1321385126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa988101750 0x7fa98819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.588+0000 7fa9877fe700 1 --2- 172.21.15.102:0/1321385126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa988100da0 0x7fa98819c500 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa97800e970 tx=0x7fa97800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:28.565 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.589+0000 7fa984ff9700 1 -- 172.21.15.102:0/1321385126 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa97800c9d0 con 0x7fa988100da0 2024-05-13T22:58:28.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.589+0000 7fa984ff9700 1 -- 172.21.15.102:0/1321385126 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa978010430 con 0x7fa988100da0 2024-05-13T22:58:28.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.589+0000 7fa98e001700 1 -- 172.21.15.102:0/1321385126 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa988196860 con 0x7fa988100da0 2024-05-13T22:58:28.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.589+0000 7fa984ff9700 1 -- 172.21.15.102:0/1321385126 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa978018630 con 0x7fa988100da0 2024-05-13T22:58:28.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.589+0000 7fa98e001700 1 -- 172.21.15.102:0/1321385126 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa988196db0 con 0x7fa988100da0 2024-05-13T22:58:28.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.590+0000 7fa98e001700 1 -- 172.21.15.102:0/1321385126 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9881091a0 con 0x7fa988100da0 2024-05-13T22:58:28.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.590+0000 7fa984ff9700 1 -- 172.21.15.102:0/1321385126 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa9780188a0 con 0x7fa988100da0 2024-05-13T22:58:28.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.591+0000 7fa984ff9700 1 --2- 172.21.15.102:0/1321385126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa97006e200 0x7fa9700706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:28.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.591+0000 7fa984ff9700 1 -- 172.21.15.102:0/1321385126 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa978014070 con 0x7fa988100da0 2024-05-13T22:58:28.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.591+0000 7fa986ffd700 1 --2- 172.21.15.102:0/1321385126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa97006e200 0x7fa9700706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:28.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.591+0000 7fa986ffd700 1 --2- 172.21.15.102:0/1321385126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa97006e200 0x7fa9700706c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa9880fce50 tx=0x7fa97c005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:28.571 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.596+0000 7fa984ff9700 1 -- 172.21.15.102:0/1321385126 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fa978053850 con 0x7fa988100da0 2024-05-13T22:58:28.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.890+0000 7fa98e001700 1 -- 172.21.15.102:0/1321385126 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa9881970a0 con 0x7fa988100da0 2024-05-13T22:58:28.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.891+0000 7fa984ff9700 1 -- 172.21.15.102:0/1321385126 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa97805b150 con 0x7fa988100da0 2024-05-13T22:58:28.866 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 6 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 3 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:58:28.867 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:58:28.868 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 13 2024-05-13T22:58:28.868 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:58:28.868 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:58:28.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.893+0000 7fa98e001700 1 -- 172.21.15.102:0/1321385126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa97006e200 msgr2=0x7fa9700706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.893+0000 7fa98e001700 1 --2- 172.21.15.102:0/1321385126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa97006e200 0x7fa9700706c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa9880fce50 tx=0x7fa97c005a90 comp rx=0 tx=0).stop 2024-05-13T22:58:28.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.893+0000 7fa98e001700 1 -- 172.21.15.102:0/1321385126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa988100da0 msgr2=0x7fa98819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.893+0000 7fa98e001700 1 --2- 172.21.15.102:0/1321385126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa988100da0 0x7fa98819c500 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa97800e970 tx=0x7fa97800ec80 comp rx=0 tx=0).stop 2024-05-13T22:58:28.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.893+0000 7fa98e001700 1 -- 172.21.15.102:0/1321385126 shutdown_connections 2024-05-13T22:58:28.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.893+0000 7fa98e001700 1 --2- 172.21.15.102:0/1321385126 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fa97006e200 0x7fa9700706c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.893+0000 7fa98e001700 1 --2- 172.21.15.102:0/1321385126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa988100da0 0x7fa98819c500 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.893+0000 7fa98e001700 1 --2- 172.21.15.102:0/1321385126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa988101750 0x7fa98819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.893+0000 7fa98e001700 1 -- 172.21.15.102:0/1321385126 >> 172.21.15.102:0/1321385126 conn(0x7fa9880fc630 msgr2=0x7fa9880fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:28.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.893+0000 7fa98e001700 1 -- 172.21.15.102:0/1321385126 shutdown_connections 2024-05-13T22:58:28.870 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:28.893+0000 7fa98e001700 1 -- 172.21.15.102:0/1321385126 wait complete. 2024-05-13T22:58:28.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.007+0000 7f78ca0b6700 1 -- 172.21.15.102:0/909468549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c4100dd0 msgr2=0x7f78c41011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.007+0000 7f78ca0b6700 1 --2- 172.21.15.102:0/909468549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c4100dd0 0x7f78c41011b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009910 tx=0x7f78b4009c20 comp rx=0 tx=0).stop 2024-05-13T22:58:28.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.007+0000 7f78ca0b6700 1 -- 172.21.15.102:0/909468549 shutdown_connections 2024-05-13T22:58:28.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.007+0000 7f78ca0b6700 1 --2- 172.21.15.102:0/909468549 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c4101780 0x7f78c41057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.007+0000 7f78ca0b6700 1 --2- 172.21.15.102:0/909468549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c4100dd0 0x7f78c41011b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.007+0000 7f78ca0b6700 1 -- 172.21.15.102:0/909468549 >> 172.21.15.102:0/909468549 conn(0x7f78c40fc640 msgr2=0x7f78c40fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:28.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.007+0000 7f78ca0b6700 1 -- 172.21.15.102:0/909468549 shutdown_connections 2024-05-13T22:58:28.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.007+0000 7f78ca0b6700 1 -- 172.21.15.102:0/909468549 wait complete. 2024-05-13T22:58:28.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.008+0000 7f78ca0b6700 1 Processor -- start 2024-05-13T22:58:28.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.008+0000 7f78ca0b6700 1 -- start start 2024-05-13T22:58:28.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.009+0000 7f78ca0b6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c4100dd0 0x7f78c419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:28.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.009+0000 7f78ca0b6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c4101780 0x7f78c419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:28.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.009+0000 7f78ca0b6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c419d0e0 con 0x7f78c4100dd0 2024-05-13T22:58:28.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.009+0000 7f78ca0b6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c4196590 con 0x7f78c4101780 2024-05-13T22:58:28.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.009+0000 7f78c37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c4100dd0 0x7f78c419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:28.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.009+0000 7f78c37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c4100dd0 0x7f78c419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44098/0 (socket says 172.21.15.102:44098) 2024-05-13T22:58:28.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.009+0000 7f78c37fe700 1 -- 172.21.15.102:0/939182962 learned_addr learned my addr 172.21.15.102:0/939182962 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:58:28.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.009+0000 7f78c2ffd700 1 --2- 172.21.15.102:0/939182962 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c4101780 0x7f78c419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:28.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.009+0000 7f78c37fe700 1 -- 172.21.15.102:0/939182962 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c4101780 msgr2=0x7f78c419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:28.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.009+0000 7f78c37fe700 1 --2- 172.21.15.102:0/939182962 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c4101780 0x7f78c419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:28.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.009+0000 7f78c37fe700 1 -- 172.21.15.102:0/939182962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40095f0 con 0x7f78c4100dd0 2024-05-13T22:58:28.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.010+0000 7f78c37fe700 1 --2- 172.21.15.102:0/939182962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c4100dd0 0x7f78c419c510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f78b400f690 tx=0x7f78b400f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:28.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.010+0000 7f78c0ff9700 1 -- 172.21.15.102:0/939182962 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78b401d070 con 0x7f78c4100dd0 2024-05-13T22:58:28.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.010+0000 7f78c0ff9700 1 -- 172.21.15.102:0/939182962 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78b40050d0 con 0x7f78c4100dd0 2024-05-13T22:58:28.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.010+0000 7f78ca0b6700 1 -- 172.21.15.102:0/939182962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c4196810 con 0x7f78c4100dd0 2024-05-13T22:58:28.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.010+0000 7f78ca0b6700 1 -- 172.21.15.102:0/939182962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c4196d00 con 0x7f78c4100dd0 2024-05-13T22:58:28.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.010+0000 7f78c0ff9700 1 -- 172.21.15.102:0/939182962 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78b4022470 con 0x7f78c4100dd0 2024-05-13T22:58:28.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.011+0000 7f78ca0b6700 1 -- 172.21.15.102:0/939182962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c4109170 con 0x7f78c4100dd0 2024-05-13T22:58:28.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.012+0000 7f78c0ff9700 1 -- 172.21.15.102:0/939182962 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f78b4003780 con 0x7f78c4100dd0 2024-05-13T22:58:28.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.012+0000 7f78c0ff9700 1 --2- 172.21.15.102:0/939182962 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f78ac06e200 0x7f78ac0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:28.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.013+0000 7f78c0ff9700 1 -- 172.21.15.102:0/939182962 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f78b408f6b0 con 0x7f78c4100dd0 2024-05-13T22:58:28.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.013+0000 7f78c2ffd700 1 --2- 172.21.15.102:0/939182962 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f78ac06e200 0x7f78ac0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:28.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.013+0000 7f78c2ffd700 1 --2- 172.21.15.102:0/939182962 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f78ac06e200 0x7f78ac0706c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f78c40fce60 tx=0x7f78b8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:28.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.017+0000 7f78c0ff9700 1 -- 172.21.15.102:0/939182962 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f78b405c3a0 con 0x7f78c4100dd0 2024-05-13T22:58:29.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.271+0000 7f78ca0b6700 1 -- 172.21.15.102:0/939182962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f78c404ea90 con 0x7f78c4100dd0 2024-05-13T22:58:29.247 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.272+0000 7f78c0ff9700 1 -- 172.21.15.102:0/939182962 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1755 (secure 0 0 0) 0x7f78b405bf30 con 0x7f78c4100dd0 2024-05-13T22:58:29.248 INFO:teuthology.orchestra.run.smithi102.stdout:e30 2024-05-13T22:58:29.248 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:58:29.248 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:58:29.248 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:58:29.248 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:58:29.248 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:58:29.248 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:58:29.248 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 30 2024-05-13T22:58:29.248 INFO:teuthology.orchestra.run.smithi102.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-13T22:58:29.248 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:58:29.248 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:58:22.097012+0000 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 96 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:58:29.249 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=34410} 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 0 members: 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:34410} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T22:58:29.250 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:58:29.251 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:58:29.251 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:58:29.251 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:58:29.251 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.102:6828/1256538594,v1:172.21.15.102:6829/1256538594] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T22:58:29.251 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:44271} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/3836279348,v1:172.21.15.130:6825/3836279348] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.274+0000 7f78ca0b6700 1 -- 172.21.15.102:0/939182962 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f78ac06e200 msgr2=0x7f78ac0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.274+0000 7f78ca0b6700 1 --2- 172.21.15.102:0/939182962 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f78ac06e200 0x7f78ac0706c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f78c40fce60 tx=0x7f78b8009260 comp rx=0 tx=0).stop 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.274+0000 7f78ca0b6700 1 -- 172.21.15.102:0/939182962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c4100dd0 msgr2=0x7f78c419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.274+0000 7f78ca0b6700 1 --2- 172.21.15.102:0/939182962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c4100dd0 0x7f78c419c510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f78b400f690 tx=0x7f78b400f6c0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.274+0000 7f78ca0b6700 1 -- 172.21.15.102:0/939182962 shutdown_connections 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.274+0000 7f78ca0b6700 1 --2- 172.21.15.102:0/939182962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78c4100dd0 0x7f78c419c510 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.274+0000 7f78ca0b6700 1 --2- 172.21.15.102:0/939182962 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f78ac06e200 0x7f78ac0706c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.274+0000 7f78ca0b6700 1 --2- 172.21.15.102:0/939182962 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f78c4101780 0x7f78c419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.274+0000 7f78ca0b6700 1 -- 172.21.15.102:0/939182962 >> 172.21.15.102:0/939182962 conn(0x7f78c40fc640 msgr2=0x7f78c40fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.274+0000 7f78ca0b6700 1 -- 172.21.15.102:0/939182962 shutdown_connections 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.274+0000 7f78ca0b6700 1 -- 172.21.15.102:0/939182962 wait complete. 2024-05-13T22:58:29.252 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 30 2024-05-13T22:58:29.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.382+0000 7f536e7dc700 1 -- 172.21.15.102:0/1030505561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368068670 msgr2=0x7f5368068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:29.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.382+0000 7f536e7dc700 1 --2- 172.21.15.102:0/1030505561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368068670 0x7f5368068a50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5358009910 tx=0x7f5358009c20 comp rx=0 tx=0).stop 2024-05-13T22:58:29.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.382+0000 7f536e7dc700 1 -- 172.21.15.102:0/1030505561 shutdown_connections 2024-05-13T22:58:29.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.382+0000 7f536e7dc700 1 --2- 172.21.15.102:0/1030505561 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5368069020 0x7f5368105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.382+0000 7f536e7dc700 1 --2- 172.21.15.102:0/1030505561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368068670 0x7f5368068a50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.382+0000 7f536e7dc700 1 -- 172.21.15.102:0/1030505561 >> 172.21.15.102:0/1030505561 conn(0x7f53680755a0 msgr2=0x7f53680759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:29.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.382+0000 7f536e7dc700 1 -- 172.21.15.102:0/1030505561 shutdown_connections 2024-05-13T22:58:29.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.382+0000 7f536e7dc700 1 -- 172.21.15.102:0/1030505561 wait complete. 2024-05-13T22:58:29.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.383+0000 7f536e7dc700 1 Processor -- start 2024-05-13T22:58:29.359 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.383+0000 7f536e7dc700 1 -- start start 2024-05-13T22:58:29.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.383+0000 7f536e7dc700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5368068670 0x7f536819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:29.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.383+0000 7f536e7dc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368069020 0x7f536819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:29.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.383+0000 7f536e7dc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536819d290 con 0x7f5368069020 2024-05-13T22:58:29.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.383+0000 7f536e7dc700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5368196740 con 0x7f5368068670 2024-05-13T22:58:29.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.383+0000 7f53677fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368069020 0x7f536819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:29.361 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.383+0000 7f53677fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368069020 0x7f536819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44108/0 (socket says 172.21.15.102:44108) 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.383+0000 7f53677fe700 1 -- 172.21.15.102:0/330052272 learned_addr learned my addr 172.21.15.102:0/330052272 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.383+0000 7f5367fff700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5368068670 0x7f536819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.384+0000 7f53677fe700 1 -- 172.21.15.102:0/330052272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5368068670 msgr2=0x7f536819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.384+0000 7f53677fe700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5368068670 0x7f536819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.384+0000 7f53677fe700 1 -- 172.21.15.102:0/330052272 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53580095f0 con 0x7f5368069020 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.384+0000 7f5367fff700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5368068670 0x7f536819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.384+0000 7f53677fe700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368069020 0x7f536819cc00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f535c00b480 tx=0x7f535c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.384+0000 7f53657fa700 1 -- 172.21.15.102:0/330052272 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f535c0118c0 con 0x7f5368069020 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.384+0000 7f53657fa700 1 -- 172.21.15.102:0/330052272 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f535c010380 con 0x7f5368069020 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.384+0000 7f536e7dc700 1 -- 172.21.15.102:0/330052272 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5368196a20 con 0x7f5368069020 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.384+0000 7f53657fa700 1 -- 172.21.15.102:0/330052272 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f535c00f4f0 con 0x7f5368069020 2024-05-13T22:58:29.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.384+0000 7f536e7dc700 1 -- 172.21.15.102:0/330052272 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5368196f70 con 0x7f5368069020 2024-05-13T22:58:29.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.385+0000 7f536e7dc700 1 -- 172.21.15.102:0/330052272 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5368109130 con 0x7f5368069020 2024-05-13T22:58:29.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.386+0000 7f53657fa700 1 -- 172.21.15.102:0/330052272 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f535c0104f0 con 0x7f5368069020 2024-05-13T22:58:29.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.387+0000 7f53657fa700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f535006e200 0x7f53500706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:29.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.387+0000 7f5367fff700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f535006e200 0x7f53500706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:29.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.387+0000 7f53657fa700 1 -- 172.21.15.102:0/330052272 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f535c08dee0 con 0x7f5368069020 2024-05-13T22:58:29.364 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.387+0000 7f5367fff700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f535006e200 0x7f53500706c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f53580059d0 tx=0x7f535801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:29.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.391+0000 7f53657fa700 1 -- 172.21.15.102:0/330052272 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f535c05abd0 con 0x7f5368069020 2024-05-13T22:58:29.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:29 smithi130 ceph-mon[84265]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:58:29.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:29 smithi130 ceph-mon[84265]: pgmap v220: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-05-13T22:58:29.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:29 smithi130 ceph-mon[84265]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:58:29.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:29 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1321385126' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:29.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:29 smithi102 ceph-mon[111694]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:58:29.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:29 smithi102 ceph-mon[111694]: pgmap v220: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-05-13T22:58:29.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:29 smithi102 ceph-mon[111694]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:58:29.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:29 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1321385126' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:29.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.619+0000 7f536e7dc700 1 -- 172.21.15.102:0/330052272 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5368197250 con 0x7f535006e200 2024-05-13T22:58:29.596 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.620+0000 7f53657fa700 1 -- 172.21.15.102:0/330052272 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5368197250 con 0x7f535006e200 2024-05-13T22:58:29.596 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:58:29.596 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:58:29.596 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:58:29.596 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:58:29.597 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:58:29.597 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-05-13T22:58:29.597 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr", 2024-05-13T22:58:29.597 INFO:teuthology.orchestra.run.smithi102.stdout: "mon", 2024-05-13T22:58:29.597 INFO:teuthology.orchestra.run.smithi102.stdout: "osd" 2024-05-13T22:58:29.597 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:58:29.597 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "15/23 daemons upgraded", 2024-05-13T22:58:29.597 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading mds daemons", 2024-05-13T22:58:29.597 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:58:29.597 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:58:29.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.623+0000 7f536e7dc700 1 -- 172.21.15.102:0/330052272 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f535006e200 msgr2=0x7f53500706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:29.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.623+0000 7f536e7dc700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f535006e200 0x7f53500706c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f53580059d0 tx=0x7f535801a040 comp rx=0 tx=0).stop 2024-05-13T22:58:29.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.623+0000 7f536e7dc700 1 -- 172.21.15.102:0/330052272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368069020 msgr2=0x7f536819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:29.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.623+0000 7f536e7dc700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368069020 0x7f536819cc00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f535c00b480 tx=0x7f535c00b840 comp rx=0 tx=0).stop 2024-05-13T22:58:29.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.623+0000 7f536e7dc700 1 -- 172.21.15.102:0/330052272 shutdown_connections 2024-05-13T22:58:29.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.623+0000 7f536e7dc700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f535006e200 0x7f53500706c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.623+0000 7f536e7dc700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5368068670 0x7f536819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.623+0000 7f536e7dc700 1 --2- 172.21.15.102:0/330052272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5368069020 0x7f536819cc00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.623+0000 7f536e7dc700 1 -- 172.21.15.102:0/330052272 >> 172.21.15.102:0/330052272 conn(0x7f53680755a0 msgr2=0x7f53680fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:29.599 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.623+0000 7f536e7dc700 1 -- 172.21.15.102:0/330052272 shutdown_connections 2024-05-13T22:58:29.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.623+0000 7f536e7dc700 1 -- 172.21.15.102:0/330052272 wait complete. 2024-05-13T22:58:29.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.730+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/1785779365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbac073ae0 msgr2=0x7fdbac10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:29.706 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.730+0000 7fdbb0cdd700 1 --2- 172.21.15.102:0/1785779365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbac073ae0 0x7fdbac10ce40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009910 tx=0x7fdba0009c20 comp rx=0 tx=0).stop 2024-05-13T22:58:29.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.730+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/1785779365 shutdown_connections 2024-05-13T22:58:29.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.730+0000 7fdbb0cdd700 1 --2- 172.21.15.102:0/1785779365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbac073ae0 0x7fdbac10ce40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.730+0000 7fdbb0cdd700 1 --2- 172.21.15.102:0/1785779365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdbac0731c0 0x7fdbac0735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.730+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/1785779365 >> 172.21.15.102:0/1785779365 conn(0x7fdbac0fc630 msgr2=0x7fdbac0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:29.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.730+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/1785779365 shutdown_connections 2024-05-13T22:58:29.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.730+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/1785779365 wait complete. 2024-05-13T22:58:29.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.731+0000 7fdbb0cdd700 1 Processor -- start 2024-05-13T22:58:29.707 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.731+0000 7fdbb0cdd700 1 -- start start 2024-05-13T22:58:29.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.731+0000 7fdbb0cdd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdbac0731c0 0x7fdbac198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:29.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.731+0000 7fdbb0cdd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbac073ae0 0x7fdbac198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.731+0000 7fdbb0cdd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac199530 con 0x7fdbac073ae0 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.731+0000 7fdbb0cdd700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac19d340 con 0x7fdbac0731c0 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.731+0000 7fdba9d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbac073ae0 0x7fdbac198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.731+0000 7fdba9d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbac073ae0 0x7fdbac198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44124/0 (socket says 172.21.15.102:44124) 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.731+0000 7fdba9d9b700 1 -- 172.21.15.102:0/2514337403 learned_addr learned my addr 172.21.15.102:0/2514337403 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.731+0000 7fdbaa59c700 1 --2- 172.21.15.102:0/2514337403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdbac0731c0 0x7fdbac198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.732+0000 7fdba9d9b700 1 -- 172.21.15.102:0/2514337403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdbac0731c0 msgr2=0x7fdbac198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.732+0000 7fdba9d9b700 1 --2- 172.21.15.102:0/2514337403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdbac0731c0 0x7fdbac198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.732+0000 7fdba9d9b700 1 -- 172.21.15.102:0/2514337403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba00095f0 con 0x7fdbac073ae0 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.732+0000 7fdba9d9b700 1 --2- 172.21.15.102:0/2514337403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbac073ae0 0x7fdbac198ea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdba000b7d0 tx=0x7fdba000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.732+0000 7fdb9b7fe700 1 -- 172.21.15.102:0/2514337403 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba001d070 con 0x7fdbac073ae0 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.732+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/2514337403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac19d5c0 con 0x7fdbac073ae0 2024-05-13T22:58:29.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.732+0000 7fdb9b7fe700 1 -- 172.21.15.102:0/2514337403 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdba00052a0 con 0x7fdbac073ae0 2024-05-13T22:58:29.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.732+0000 7fdb9b7fe700 1 -- 172.21.15.102:0/2514337403 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba0022470 con 0x7fdbac073ae0 2024-05-13T22:58:29.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.732+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/2514337403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac19dab0 con 0x7fdbac073ae0 2024-05-13T22:58:29.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.733+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/2514337403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbac10a540 con 0x7fdbac073ae0 2024-05-13T22:58:29.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.734+0000 7fdb9b7fe700 1 -- 172.21.15.102:0/2514337403 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdba000f850 con 0x7fdbac073ae0 2024-05-13T22:58:29.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.734+0000 7fdb9b7fe700 1 --2- 172.21.15.102:0/2514337403 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdb9406e200 0x7fdb940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:58:29.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.735+0000 7fdb9b7fe700 1 -- 172.21.15.102:0/2514337403 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fdba008f7d0 con 0x7fdbac073ae0 2024-05-13T22:58:29.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.735+0000 7fdbaa59c700 1 --2- 172.21.15.102:0/2514337403 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdb9406e200 0x7fdb940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:58:29.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.735+0000 7fdbaa59c700 1 --2- 172.21.15.102:0/2514337403 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdb9406e200 0x7fdb940706c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009c60 tx=0x7fdb9c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:58:29.719 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:29.743+0000 7fdb9b7fe700 1 -- 172.21.15.102:0/2514337403 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fdba005c4c0 con 0x7fdbac073ae0 2024-05-13T22:58:30.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.056+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/2514337403 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdbac04ea90 con 0x7fdbac073ae0 2024-05-13T22:58:30.032 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.056+0000 7fdb9b7fe700 1 -- 172.21.15.102:0/2514337403 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7fdba005c050 con 0x7fdbac073ae0 2024-05-13T22:58:30.032 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_WARN 1 filesystem is degraded 2024-05-13T22:58:30.033 INFO:teuthology.orchestra.run.smithi102.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-13T22:58:30.033 INFO:teuthology.orchestra.run.smithi102.stdout: fs cephfs is degraded 2024-05-13T22:58:30.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.059+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/2514337403 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdb9406e200 msgr2=0x7fdb940706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:30.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.059+0000 7fdbb0cdd700 1 --2- 172.21.15.102:0/2514337403 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdb9406e200 0x7fdb940706c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009c60 tx=0x7fdb9c009260 comp rx=0 tx=0).stop 2024-05-13T22:58:30.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.059+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/2514337403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbac073ae0 msgr2=0x7fdbac198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:58:30.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.059+0000 7fdbb0cdd700 1 --2- 172.21.15.102:0/2514337403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbac073ae0 0x7fdbac198ea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdba000b7d0 tx=0x7fdba000bf90 comp rx=0 tx=0).stop 2024-05-13T22:58:30.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.059+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/2514337403 shutdown_connections 2024-05-13T22:58:30.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.059+0000 7fdbb0cdd700 1 --2- 172.21.15.102:0/2514337403 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fdb9406e200 0x7fdb940706c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:30.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.059+0000 7fdbb0cdd700 1 --2- 172.21.15.102:0/2514337403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdbac0731c0 0x7fdbac198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:30.035 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.059+0000 7fdbb0cdd700 1 --2- 172.21.15.102:0/2514337403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbac073ae0 0x7fdbac198ea0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:58:30.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.059+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/2514337403 >> 172.21.15.102:0/2514337403 conn(0x7fdbac0fc630 msgr2=0x7fdbac107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:58:30.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.059+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/2514337403 shutdown_connections 2024-05-13T22:58:30.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:58:30.059+0000 7fdbb0cdd700 1 -- 172.21.15.102:0/2514337403 wait complete. 2024-05-13T22:58:30.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:30 smithi130 ceph-mon[84265]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:58:30.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:30 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/939182962' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:58:30.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:30 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:30.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:30 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:30.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:30 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:30.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:30 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2514337403' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:58:30.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:30 smithi102 ceph-mon[111694]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:58:30.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:30 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/939182962' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:58:30.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:30 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:30.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:30 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:30.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:30 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:30.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:30 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2514337403' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:58:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:31 smithi130 ceph-mon[84265]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:58:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:31 smithi130 ceph-mon[84265]: pgmap v221: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-05-13T22:58:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:31 smithi130 ceph-mon[84265]: mds.? [v2:172.21.15.130:6826/3806306097,v1:172.21.15.130:6827/3806306097] up:boot 2024-05-13T22:58:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:31 smithi130 ceph-mon[84265]: fsmap cephfs:1/1 {0=cephfs.smithi102.cgkcwz=up:replay} 3 up:standby 2024-05-13T22:58:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:58:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:31.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:58:31.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:31 smithi102 ceph-mon[111694]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:58:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:31 smithi102 ceph-mon[111694]: pgmap v221: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-05-13T22:58:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:31 smithi102 ceph-mon[111694]: mds.? [v2:172.21.15.130:6826/3806306097,v1:172.21.15.130:6827/3806306097] up:boot 2024-05-13T22:58:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:31 smithi102 ceph-mon[111694]: fsmap cephfs:1/1 {0=cephfs.smithi102.cgkcwz=up:replay} 3 up:standby 2024-05-13T22:58:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:58:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:31.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:58:33.148 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:32 smithi130 ceph-mon[84265]: reconnect by client.14554 192.168.0.1:0/2305339701 after 0 2024-05-13T22:58:33.148 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:32 smithi130 ceph-mon[84265]: reconnect by client.24321 192.168.0.1:0/2826286483 after 0.00099998 2024-05-13T22:58:33.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:32 smithi130 ceph-mon[84265]: mds.? [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] up:reconnect 2024-05-13T22:58:33.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:32 smithi130 ceph-mon[84265]: fsmap cephfs:1/1 {0=cephfs.smithi102.cgkcwz=up:reconnect} 3 up:standby 2024-05-13T22:58:33.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:32 smithi130 ceph-mon[84265]: pgmap v222: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-05-13T22:58:33.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:32 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:33.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:32 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:33.175 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:32 smithi102 ceph-mon[111694]: reconnect by client.14554 192.168.0.1:0/2305339701 after 0 2024-05-13T22:58:33.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:32 smithi102 ceph-mon[111694]: reconnect by client.24321 192.168.0.1:0/2826286483 after 0.00099998 2024-05-13T22:58:33.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:32 smithi102 ceph-mon[111694]: mds.? [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] up:reconnect 2024-05-13T22:58:33.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:32 smithi102 ceph-mon[111694]: fsmap cephfs:1/1 {0=cephfs.smithi102.cgkcwz=up:reconnect} 3 up:standby 2024-05-13T22:58:33.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:32 smithi102 ceph-mon[111694]: pgmap v222: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-05-13T22:58:33.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:32 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:33.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:32 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:34.141 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:33 smithi130 ceph-mon[84265]: mds.? [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] up:rejoin 2024-05-13T22:58:34.142 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:33 smithi130 ceph-mon[84265]: fsmap cephfs:1/1 {0=cephfs.smithi102.cgkcwz=up:rejoin} 3 up:standby 2024-05-13T22:58:34.142 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:33 smithi130 ceph-mon[84265]: daemon mds.cephfs.smithi102.cgkcwz is now active in filesystem cephfs as rank 0 2024-05-13T22:58:34.142 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:33 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:34.142 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:33 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:34.175 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:33 smithi102 ceph-mon[111694]: mds.? [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] up:rejoin 2024-05-13T22:58:34.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:33 smithi102 ceph-mon[111694]: fsmap cephfs:1/1 {0=cephfs.smithi102.cgkcwz=up:rejoin} 3 up:standby 2024-05-13T22:58:34.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:33 smithi102 ceph-mon[111694]: daemon mds.cephfs.smithi102.cgkcwz is now active in filesystem cephfs as rank 0 2024-05-13T22:58:34.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:33 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:34.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:33 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:35.148 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:34 smithi130 ceph-mon[84265]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-13T22:58:35.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:34 smithi130 ceph-mon[84265]: Cluster is now healthy 2024-05-13T22:58:35.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:34 smithi130 ceph-mon[84265]: mds.? [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] up:active 2024-05-13T22:58:35.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:34 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 3 up:standby 2024-05-13T22:58:35.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:34 smithi130 ceph-mon[84265]: pgmap v223: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-13T22:58:35.175 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:34 smithi102 ceph-mon[111694]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-13T22:58:35.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:34 smithi102 ceph-mon[111694]: Cluster is now healthy 2024-05-13T22:58:35.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:34 smithi102 ceph-mon[111694]: mds.? [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] up:active 2024-05-13T22:58:35.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:34 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 3 up:standby 2024-05-13T22:58:35.176 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:34 smithi102 ceph-mon[111694]: pgmap v223: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-05-13T22:58:37.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:37 smithi130 ceph-mon[84265]: pgmap v224: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 6 op/s 2024-05-13T22:58:37.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:37 smithi102 ceph-mon[111694]: pgmap v224: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 6 op/s 2024-05-13T22:58:38.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:38.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:38.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:38.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:58:38.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:38.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:38.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:38.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: Upgrade: Setting container_image for all mds 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi102.cgkcwz"}]': finished 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi102.sdylgv"}]': finished 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi130.hzjanp"}]': finished 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi130.ovlxvg"}]': finished 2024-05-13T22:58:38.427 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: Upgrade: Setting filesystem cephfs Joinable 2024-05-13T22:58:38.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-13T22:58:38.428 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:38 smithi102 ceph-mon[111694]: pgmap v225: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 6 op/s 2024-05-13T22:58:38.480 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:38.480 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: Upgrade: Setting container_image for all mds 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi102.cgkcwz"}]: dispatch 2024-05-13T22:58:38.481 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi102.cgkcwz"}]': finished 2024-05-13T22:58:38.482 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi102.sdylgv"}]: dispatch 2024-05-13T22:58:38.482 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi102.sdylgv"}]': finished 2024-05-13T22:58:38.482 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi130.hzjanp"}]: dispatch 2024-05-13T22:58:38.482 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi130.hzjanp"}]': finished 2024-05-13T22:58:38.482 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi130.ovlxvg"}]: dispatch 2024-05-13T22:58:38.482 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi130.ovlxvg"}]': finished 2024-05-13T22:58:38.482 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: Upgrade: Setting filesystem cephfs Joinable 2024-05-13T22:58:38.482 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-13T22:58:38.482 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:38 smithi130 ceph-mon[84265]: pgmap v225: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 6 op/s 2024-05-13T22:58:39.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:39 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-13T22:58:39.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:39 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 3 up:standby 2024-05-13T22:58:39.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:39 smithi130 ceph-mon[84265]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-13T22:58:39.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:39 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-13T22:58:39.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:39 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-13T22:58:39.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:39 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 3 up:standby 2024-05-13T22:58:39.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:39 smithi102 ceph-mon[111694]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-13T22:58:39.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:39 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-13T22:58:40.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-13T22:58:40.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 3 up:standby 2024-05-13T22:58:40.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:58:40.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:40.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:40.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: Upgrade: Setting container_image for all rgw 2024-05-13T22:58:40.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:40.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:40.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: Upgrade: Setting container_image for all rbd-mirror 2024-05-13T22:58:40.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:40.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:40.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:40 smithi102 ceph-mon[111694]: pgmap v226: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 170 B/s wr, 4 op/s 2024-05-13T22:58:40.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-13T22:58:40.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 3 up:standby 2024-05-13T22:58:40.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:58:40.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:40.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:40.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: Upgrade: Setting container_image for all rgw 2024-05-13T22:58:40.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:40.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:40.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: Upgrade: Setting container_image for all rbd-mirror 2024-05-13T22:58:40.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:40.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:58:40.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:40 smithi130 ceph-mon[84265]: pgmap v226: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 170 B/s wr, 4 op/s 2024-05-13T22:58:43.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:43 smithi130 ceph-mon[84265]: pgmap v227: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-13T22:58:43.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:43 smithi102 ceph-mon[111694]: pgmap v227: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-13T22:58:45.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:45 smithi130 ceph-mon[84265]: pgmap v228: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-13T22:58:45.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:45 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:45.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:45 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:58:45.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:45 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:45.410 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:45 smithi102 ceph-mon[111694]: pgmap v228: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-05-13T22:58:45.410 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:45 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:45.410 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:45 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi102", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:58:45.410 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:45 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:46.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:46 smithi130 ceph-mon[84265]: Upgrade: Updating ceph-exporter.smithi102 (1/2) 2024-05-13T22:58:46.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:46 smithi130 ceph-mon[84265]: Deploying daemon ceph-exporter.smithi102 on smithi102 2024-05-13T22:58:46.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:46.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:58:46.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:46 smithi102 ceph-mon[111694]: Upgrade: Updating ceph-exporter.smithi102 (1/2) 2024-05-13T22:58:46.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:46 smithi102 ceph-mon[111694]: Deploying daemon ceph-exporter.smithi102 on smithi102 2024-05-13T22:58:46.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:46.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:58:47.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:47 smithi130 ceph-mon[84265]: pgmap v229: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-13T22:58:47.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:47 smithi102 ceph-mon[111694]: pgmap v229: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-13T22:58:49.384 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:49 smithi102 ceph-mon[111694]: pgmap v230: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-13T22:58:49.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:49 smithi130 ceph-mon[84265]: pgmap v230: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-13T22:58:50.768 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:50 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:50.769 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:50 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:50.769 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:50 smithi102 ceph-mon[111694]: pgmap v231: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-05-13T22:58:50.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:50 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:50.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:50 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:50.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:50 smithi130 ceph-mon[84265]: pgmap v231: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-05-13T22:58:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:52 smithi130 ceph-mon[84265]: Upgrade: Updating ceph-exporter.smithi130 (2/2) 2024-05-13T22:58:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:52 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:52 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:58:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:52 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:52.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:52 smithi130 ceph-mon[84265]: Deploying daemon ceph-exporter.smithi130 on smithi130 2024-05-13T22:58:52.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:52 smithi102 ceph-mon[111694]: Upgrade: Updating ceph-exporter.smithi130 (2/2) 2024-05-13T22:58:52.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:52 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:52.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:52 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi130", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-13T22:58:52.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:52 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:58:52.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:52 smithi102 ceph-mon[111694]: Deploying daemon ceph-exporter.smithi130 on smithi130 2024-05-13T22:58:53.386 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:53 smithi130 ceph-mon[84265]: pgmap v232: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-05-13T22:58:53.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:53 smithi102 ceph-mon[111694]: pgmap v232: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-05-13T22:58:55.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:55 smithi130 ceph-mon[84265]: pgmap v233: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-05-13T22:58:55.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:55 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:55.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:55 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:55.399 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:55 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:55.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:55 smithi102 ceph-mon[111694]: pgmap v233: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-05-13T22:58:55.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:55 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:55.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:55 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:55.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:55 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:58:57.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:57 smithi102 ceph-mon[111694]: pgmap v234: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-05-13T22:58:57.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:57 smithi130 ceph-mon[84265]: pgmap v234: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-05-13T22:58:58.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:58 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:58.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:58 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:58.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:58 smithi130 ceph-mon[84265]: pgmap v235: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s rd, 2 op/s 2024-05-13T22:58:58.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:58 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:58.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:58 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:58.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:58 smithi102 ceph-mon[111694]: pgmap v235: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s rd, 2 op/s 2024-05-13T22:58:59.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:59.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:58:59 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:59.790 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:58:59.790 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:58:59 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:00.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.167+0000 7fe234e8a700 1 -- 172.21.15.102:0/2918489969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2300ffec0 msgr2=0x7fe230105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:00.144 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.167+0000 7fe234e8a700 1 --2- 172.21.15.102:0/2918489969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2300ffec0 0x7fe230105850 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe224009910 tx=0x7fe224009c20 comp rx=0 tx=0).stop 2024-05-13T22:59:00.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.167+0000 7fe234e8a700 1 -- 172.21.15.102:0/2918489969 shutdown_connections 2024-05-13T22:59:00.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.167+0000 7fe234e8a700 1 --2- 172.21.15.102:0/2918489969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2300ffec0 0x7fe230105850 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.167+0000 7fe234e8a700 1 --2- 172.21.15.102:0/2918489969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2300ff510 0x7fe2300ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.167+0000 7fe234e8a700 1 -- 172.21.15.102:0/2918489969 >> 172.21.15.102:0/2918489969 conn(0x7fe230074d80 msgr2=0x7fe230075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:00.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.167+0000 7fe234e8a700 1 -- 172.21.15.102:0/2918489969 shutdown_connections 2024-05-13T22:59:00.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.167+0000 7fe234e8a700 1 -- 172.21.15.102:0/2918489969 wait complete. 2024-05-13T22:59:00.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.168+0000 7fe234e8a700 1 Processor -- start 2024-05-13T22:59:00.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.168+0000 7fe234e8a700 1 -- start start 2024-05-13T22:59:00.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.168+0000 7fe234e8a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2300ff510 0x7fe23019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:00.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.168+0000 7fe234e8a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2300ffec0 0x7fe23019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:00.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe234e8a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019d0e0 con 0x7fe2300ff510 2024-05-13T22:59:00.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe234e8a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe230196590 con 0x7fe2300ffec0 2024-05-13T22:59:00.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe22e59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2300ff510 0x7fe23019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:00.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe22e59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2300ff510 0x7fe23019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43310/0 (socket says 172.21.15.102:43310) 2024-05-13T22:59:00.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe22e59c700 1 -- 172.21.15.102:0/2228140972 learned_addr learned my addr 172.21.15.102:0/2228140972 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:00.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe22dd9b700 1 --2- 172.21.15.102:0/2228140972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2300ffec0 0x7fe23019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:00.147 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe22e59c700 1 -- 172.21.15.102:0/2228140972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2300ffec0 msgr2=0x7fe23019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:00.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe22e59c700 1 --2- 172.21.15.102:0/2228140972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2300ffec0 0x7fe23019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe22e59c700 1 -- 172.21.15.102:0/2228140972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2240095f0 con 0x7fe2300ff510 2024-05-13T22:59:00.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe22e59c700 1 --2- 172.21.15.102:0/2228140972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2300ff510 0x7fe23019c510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe22000d8d0 tx=0x7fe22000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:00.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe21f7fe700 1 -- 172.21.15.102:0/2228140972 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2200098a0 con 0x7fe2300ff510 2024-05-13T22:59:00.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe21f7fe700 1 -- 172.21.15.102:0/2228140972 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe220010460 con 0x7fe2300ff510 2024-05-13T22:59:00.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe234e8a700 1 -- 172.21.15.102:0/2228140972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe230196870 con 0x7fe2300ff510 2024-05-13T22:59:00.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe21f7fe700 1 -- 172.21.15.102:0/2228140972 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe22000f5f0 con 0x7fe2300ff510 2024-05-13T22:59:00.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.169+0000 7fe234e8a700 1 -- 172.21.15.102:0/2228140972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe230196d90 con 0x7fe2300ff510 2024-05-13T22:59:00.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.171+0000 7fe234e8a700 1 -- 172.21.15.102:0/2228140972 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe230109180 con 0x7fe2300ff510 2024-05-13T22:59:00.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.171+0000 7fe21f7fe700 1 -- 172.21.15.102:0/2228140972 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe220009a00 con 0x7fe2300ff510 2024-05-13T22:59:00.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.171+0000 7fe21f7fe700 1 --2- 172.21.15.102:0/2228140972 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe21806e200 0x7fe2180706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:00.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.171+0000 7fe21f7fe700 1 -- 172.21.15.102:0/2228140972 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe22008e090 con 0x7fe2300ff510 2024-05-13T22:59:00.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.171+0000 7fe22dd9b700 1 --2- 172.21.15.102:0/2228140972 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe21806e200 0x7fe2180706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:00.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.172+0000 7fe22dd9b700 1 --2- 172.21.15.102:0/2228140972 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe21806e200 0x7fe2180706c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe224006010 tx=0x7fe22401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:00.153 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.176+0000 7fe21f7fe700 1 -- 172.21.15.102:0/2228140972 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe22005ae00 con 0x7fe2300ff510 2024-05-13T22:59:00.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.402+0000 7fe234e8a700 1 -- 172.21.15.102:0/2228140972 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe230197030 con 0x7fe21806e200 2024-05-13T22:59:00.380 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.403+0000 7fe21f7fe700 1 -- 172.21.15.102:0/2228140972 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fe230197030 con 0x7fe21806e200 2024-05-13T22:59:00.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.405+0000 7fe234e8a700 1 -- 172.21.15.102:0/2228140972 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe21806e200 msgr2=0x7fe2180706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:00.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.405+0000 7fe234e8a700 1 --2- 172.21.15.102:0/2228140972 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe21806e200 0x7fe2180706c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe224006010 tx=0x7fe22401a040 comp rx=0 tx=0).stop 2024-05-13T22:59:00.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.405+0000 7fe234e8a700 1 -- 172.21.15.102:0/2228140972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2300ff510 msgr2=0x7fe23019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:00.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.405+0000 7fe234e8a700 1 --2- 172.21.15.102:0/2228140972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2300ff510 0x7fe23019c510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe22000d8d0 tx=0x7fe22000dbe0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.406+0000 7fe234e8a700 1 -- 172.21.15.102:0/2228140972 shutdown_connections 2024-05-13T22:59:00.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.406+0000 7fe234e8a700 1 --2- 172.21.15.102:0/2228140972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2300ff510 0x7fe23019c510 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.406+0000 7fe234e8a700 1 --2- 172.21.15.102:0/2228140972 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe21806e200 0x7fe2180706c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.406+0000 7fe234e8a700 1 --2- 172.21.15.102:0/2228140972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2300ffec0 0x7fe23019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.406+0000 7fe234e8a700 1 -- 172.21.15.102:0/2228140972 >> 172.21.15.102:0/2228140972 conn(0x7fe230074d80 msgr2=0x7fe2300feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:00.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.406+0000 7fe234e8a700 1 -- 172.21.15.102:0/2228140972 shutdown_connections 2024-05-13T22:59:00.383 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.406+0000 7fe234e8a700 1 -- 172.21.15.102:0/2228140972 wait complete. 2024-05-13T22:59:00.394 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T22:59:00.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.521+0000 7efc759e3700 1 -- 172.21.15.102:0/3752839654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc70105bb0 msgr2=0x7efc70105f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:00.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.521+0000 7efc759e3700 1 --2- 172.21.15.102:0/3752839654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc70105bb0 0x7efc70105f90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc60009910 tx=0x7efc60009c20 comp rx=0 tx=0).stop 2024-05-13T22:59:00.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.521+0000 7efc759e3700 1 -- 172.21.15.102:0/3752839654 shutdown_connections 2024-05-13T22:59:00.498 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.521+0000 7efc759e3700 1 --2- 172.21.15.102:0/3752839654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc70100120 0x7efc701005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.521+0000 7efc759e3700 1 --2- 172.21.15.102:0/3752839654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc70105bb0 0x7efc70105f90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.521+0000 7efc759e3700 1 -- 172.21.15.102:0/3752839654 >> 172.21.15.102:0/3752839654 conn(0x7efc70075e30 msgr2=0x7efc700fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:00.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.521+0000 7efc759e3700 1 -- 172.21.15.102:0/3752839654 shutdown_connections 2024-05-13T22:59:00.499 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.521+0000 7efc759e3700 1 -- 172.21.15.102:0/3752839654 wait complete. 2024-05-13T22:59:00.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.522+0000 7efc759e3700 1 Processor -- start 2024-05-13T22:59:00.500 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.522+0000 7efc759e3700 1 -- start start 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc759e3700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc70100120 0x7efc701953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc759e3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc70105bb0 0x7efc70195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc6e7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc70105bb0 0x7efc70195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc6effd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc70100120 0x7efc701953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc6e7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc70105bb0 0x7efc70195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43334/0 (socket says 172.21.15.102:43334) 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc6e7fc700 1 -- 172.21.15.102:0/1818674956 learned_addr learned my addr 172.21.15.102:0/1818674956 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc70195fe0 con 0x7efc70105bb0 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc70196120 con 0x7efc70100120 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc6effd700 1 -- 172.21.15.102:0/1818674956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc70105bb0 msgr2=0x7efc70195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc6effd700 1 --2- 172.21.15.102:0/1818674956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc70105bb0 0x7efc70195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc6effd700 1 -- 172.21.15.102:0/1818674956 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc600095f0 con 0x7efc70100120 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc6effd700 1 --2- 172.21.15.102:0/1818674956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc70100120 0x7efc701953c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fd0 tx=0x7efc6000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:00.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.523+0000 7efc749e1700 1 -- 172.21.15.102:0/1818674956 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc6001d070 con 0x7efc70100120 2024-05-13T22:59:00.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.524+0000 7efc749e1700 1 -- 172.21.15.102:0/1818674956 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc600050d0 con 0x7efc70100120 2024-05-13T22:59:00.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.524+0000 7efc749e1700 1 -- 172.21.15.102:0/1818674956 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc60003e90 con 0x7efc70100120 2024-05-13T22:59:00.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.524+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7019a410 con 0x7efc70100120 2024-05-13T22:59:00.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.524+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7019a8d0 con 0x7efc70100120 2024-05-13T22:59:00.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.525+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7004ea90 con 0x7efc70100120 2024-05-13T22:59:00.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.525+0000 7efc749e1700 1 -- 172.21.15.102:0/1818674956 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7efc6000fa10 con 0x7efc70100120 2024-05-13T22:59:00.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.526+0000 7efc749e1700 1 --2- 172.21.15.102:0/1818674956 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7efc5806e200 0x7efc580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:00.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.526+0000 7efc749e1700 1 -- 172.21.15.102:0/1818674956 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7efc6008f630 con 0x7efc70100120 2024-05-13T22:59:00.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.526+0000 7efc6e7fc700 1 --2- 172.21.15.102:0/1818674956 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7efc5806e200 0x7efc580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:00.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.526+0000 7efc6e7fc700 1 --2- 172.21.15.102:0/1818674956 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7efc5806e200 0x7efc580706c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efc700ff990 tx=0x7efc64009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:00.507 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.531+0000 7efc749e1700 1 -- 172.21.15.102:0/1818674956 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7efc60058870 con 0x7efc70100120 2024-05-13T22:59:00.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:00 smithi102 ceph-mon[111694]: pgmap v236: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1 op/s 2024-05-13T22:59:00.735 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.758+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc7019b420 con 0x7efc5806e200 2024-05-13T22:59:00.736 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.760+0000 7efc749e1700 1 -- 172.21.15.102:0/1818674956 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7efc7019b420 con 0x7efc5806e200 2024-05-13T22:59:00.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.762+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7efc5806e200 msgr2=0x7efc580706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:00.739 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.762+0000 7efc759e3700 1 --2- 172.21.15.102:0/1818674956 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7efc5806e200 0x7efc580706c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efc700ff990 tx=0x7efc64009260 comp rx=0 tx=0).stop 2024-05-13T22:59:00.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.762+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc70100120 msgr2=0x7efc701953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:00.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.762+0000 7efc759e3700 1 --2- 172.21.15.102:0/1818674956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc70100120 0x7efc701953c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efc60009fd0 tx=0x7efc6000f690 comp rx=0 tx=0).stop 2024-05-13T22:59:00.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.762+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 shutdown_connections 2024-05-13T22:59:00.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.762+0000 7efc759e3700 1 --2- 172.21.15.102:0/1818674956 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7efc5806e200 0x7efc580706c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.762+0000 7efc759e3700 1 --2- 172.21.15.102:0/1818674956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc70100120 0x7efc701953c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.740 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.762+0000 7efc759e3700 1 --2- 172.21.15.102:0/1818674956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc70105bb0 0x7efc70195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.762+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 >> 172.21.15.102:0/1818674956 conn(0x7efc70075e30 msgr2=0x7efc70103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:00.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.762+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 shutdown_connections 2024-05-13T22:59:00.741 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.762+0000 7efc759e3700 1 -- 172.21.15.102:0/1818674956 wait complete. 2024-05-13T22:59:00.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.869+0000 7f482576e700 1 -- 172.21.15.102:0/3786498421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4820101780 msgr2=0x7f48201057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:00.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.869+0000 7f482576e700 1 --2- 172.21.15.102:0/3786498421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4820101780 0x7f48201057d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4810009910 tx=0x7f4810009c20 comp rx=0 tx=0).stop 2024-05-13T22:59:00.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.869+0000 7f482576e700 1 -- 172.21.15.102:0/3786498421 shutdown_connections 2024-05-13T22:59:00.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.869+0000 7f482576e700 1 --2- 172.21.15.102:0/3786498421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4820101780 0x7f48201057d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.846 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.869+0000 7f482576e700 1 --2- 172.21.15.102:0/3786498421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4820100dd0 0x7f48201011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.869+0000 7f482576e700 1 -- 172.21.15.102:0/3786498421 >> 172.21.15.102:0/3786498421 conn(0x7f48200fc640 msgr2=0x7f48200fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:00.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.869+0000 7f482576e700 1 -- 172.21.15.102:0/3786498421 shutdown_connections 2024-05-13T22:59:00.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.869+0000 7f482576e700 1 -- 172.21.15.102:0/3786498421 wait complete. 2024-05-13T22:59:00.847 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.870+0000 7f482576e700 1 Processor -- start 2024-05-13T22:59:00.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.870+0000 7f482576e700 1 -- start start 2024-05-13T22:59:00.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.870+0000 7f482576e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4820100dd0 0x7f482019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:00.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.870+0000 7f482576e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4820101780 0x7f482019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:00.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.870+0000 7f482576e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482019d160 con 0x7f4820101780 2024-05-13T22:59:00.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.870+0000 7f482576e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4820196610 con 0x7f4820100dd0 2024-05-13T22:59:00.848 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.870+0000 7f481e7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4820101780 0x7f482019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.870+0000 7f481e7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4820101780 0x7f482019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43358/0 (socket says 172.21.15.102:43358) 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.870+0000 7f481e7fc700 1 -- 172.21.15.102:0/2269052136 learned_addr learned my addr 172.21.15.102:0/2269052136 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.870+0000 7f481effd700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4820100dd0 0x7f482019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.871+0000 7f481effd700 1 -- 172.21.15.102:0/2269052136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4820101780 msgr2=0x7f482019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.871+0000 7f481effd700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4820101780 0x7f482019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.871+0000 7f481effd700 1 -- 172.21.15.102:0/2269052136 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100095f0 con 0x7f4820100dd0 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.871+0000 7f481e7fc700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4820101780 0x7f482019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.871+0000 7f481effd700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4820100dd0 0x7f482019c590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f481400e970 tx=0x7f481400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.871+0000 7f4807fff700 1 -- 172.21.15.102:0/2269052136 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f481400c9d0 con 0x7f4820100dd0 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.871+0000 7f482576e700 1 -- 172.21.15.102:0/2269052136 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48201968f0 con 0x7f4820100dd0 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.871+0000 7f4807fff700 1 -- 172.21.15.102:0/2269052136 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4814010430 con 0x7f4820100dd0 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.871+0000 7f4807fff700 1 -- 172.21.15.102:0/2269052136 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4814018630 con 0x7f4820100dd0 2024-05-13T22:59:00.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.871+0000 7f482576e700 1 -- 172.21.15.102:0/2269052136 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4820196e40 con 0x7f4820100dd0 2024-05-13T22:59:00.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.872+0000 7f482576e700 1 -- 172.21.15.102:0/2269052136 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48201091d0 con 0x7f4820100dd0 2024-05-13T22:59:00.850 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.872+0000 7f4807fff700 1 -- 172.21.15.102:0/2269052136 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f48140188f0 con 0x7f4820100dd0 2024-05-13T22:59:00.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.873+0000 7f4807fff700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f480806e200 0x7f48080706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:00.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.873+0000 7f4807fff700 1 -- 172.21.15.102:0/2269052136 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f4814014070 con 0x7f4820100dd0 2024-05-13T22:59:00.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.873+0000 7f481e7fc700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f480806e200 0x7f48080706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:00.851 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.873+0000 7f481e7fc700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f480806e200 0x7f48080706c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f481000b760 tx=0x7f4810005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:00.854 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:00.878+0000 7f4807fff700 1 -- 172.21.15.102:0/2269052136 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f4814053850 con 0x7f4820100dd0 2024-05-13T22:59:00.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:00 smithi130 ceph-mon[84265]: pgmap v236: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1 op/s 2024-05-13T22:59:01.069 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.092+0000 7f482576e700 1 -- 172.21.15.102:0/2269052136 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4820197060 con 0x7f480806e200 2024-05-13T22:59:01.083 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.106+0000 7f4807fff700 1 -- 172.21.15.102:0/2269052136 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f4820197060 con 0x7f480806e200 2024-05-13T22:59:01.083 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:59:01.084 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (8m) 0s ago 19m 26.0M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:59:01.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (12s) 0s ago 19m 7936k - 19.0.0-2565-gc3e55145 b1b56bed7c91 bfcb10ba7c2a 2024-05-13T22:59:01.084 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (6s) 3s ago 18m 7004k - 19.0.0-2565-gc3e55145 b1b56bed7c91 035d93e33eb8 2024-05-13T22:59:01.084 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (5m) 0s ago 19m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T22:59:01.084 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (5m) 3s ago 18m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T22:59:01.084 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (7m) 0s ago 18m 79.9M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:59:01.084 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (93s) 0s ago 13m 89.5M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1a34f8c7d234 2024-05-13T22:59:01.084 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (68s) 0s ago 13m 110M - 19.0.0-2565-gc3e55145 b1b56bed7c91 2b50dedff13c 2024-05-13T22:59:01.084 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (50s) 3s ago 13m 13.7M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1a45ebbee664 2024-05-13T22:59:01.084 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (31s) 3s ago 13m 15.7M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1d727b21600b 2024-05-13T22:59:01.084 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (10m) 0s ago 20m 513M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:59:01.085 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (9m) 3s ago 18m 454M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:59:01.085 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (6m) 0s ago 20m 60.6M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:59:01.085 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (5m) 3s ago 18m 49.2M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:59:01.085 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (8m) 0s ago 19m 16.2M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:59:01.085 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (8m) 3s ago 18m 17.1M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:59:01.085 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (4m) 0s ago 17m 142M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 0ada88c777a2 2024-05-13T22:59:01.085 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (4m) 0s ago 16m 119M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 c3e9e68b06fd 2024-05-13T22:59:01.085 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (3m) 0s ago 16m 106M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 1b85728dccf5 2024-05-13T22:59:01.085 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (3m) 3s ago 16m 110M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 66e804d4b801 2024-05-13T22:59:01.085 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (2m) 3s ago 15m 101M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5bb5a8f94042 2024-05-13T22:59:01.085 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (113s) 3s ago 15m 102M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5d983135967d 2024-05-13T22:59:01.086 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (8m) 0s ago 18m 80.2M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:59:01.086 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.109+0000 7f482576e700 1 -- 172.21.15.102:0/2269052136 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f480806e200 msgr2=0x7f48080706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.110+0000 7f482576e700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f480806e200 0x7f48080706c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f481000b760 tx=0x7f4810005940 comp rx=0 tx=0).stop 2024-05-13T22:59:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.110+0000 7f482576e700 1 -- 172.21.15.102:0/2269052136 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4820100dd0 msgr2=0x7f482019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.110+0000 7f482576e700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4820100dd0 0x7f482019c590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f481400e970 tx=0x7f481400ec80 comp rx=0 tx=0).stop 2024-05-13T22:59:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.110+0000 7f482576e700 1 -- 172.21.15.102:0/2269052136 shutdown_connections 2024-05-13T22:59:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.110+0000 7f482576e700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f480806e200 0x7f48080706c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.110+0000 7f482576e700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4820100dd0 0x7f482019c590 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.110+0000 7f482576e700 1 --2- 172.21.15.102:0/2269052136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4820101780 0x7f482019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.110+0000 7f482576e700 1 -- 172.21.15.102:0/2269052136 >> 172.21.15.102:0/2269052136 conn(0x7f48200fc640 msgr2=0x7f48200fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:01.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.110+0000 7f482576e700 1 -- 172.21.15.102:0/2269052136 shutdown_connections 2024-05-13T22:59:01.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.110+0000 7f482576e700 1 -- 172.21.15.102:0/2269052136 wait complete. 2024-05-13T22:59:01.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.235+0000 7fea6bfc7700 1 -- 172.21.15.102:0/2035900658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea64105a40 msgr2=0x7fea64105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.235+0000 7fea6bfc7700 1 --2- 172.21.15.102:0/2035900658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea64105a40 0x7fea64105e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea60009910 tx=0x7fea60009c20 comp rx=0 tx=0).stop 2024-05-13T22:59:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.235+0000 7fea6bfc7700 1 -- 172.21.15.102:0/2035900658 shutdown_connections 2024-05-13T22:59:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.235+0000 7fea6bfc7700 1 --2- 172.21.15.102:0/2035900658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea640ffa40 0x7fea640ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.235+0000 7fea6bfc7700 1 --2- 172.21.15.102:0/2035900658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea64105a40 0x7fea64105e20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.235+0000 7fea6bfc7700 1 -- 172.21.15.102:0/2035900658 >> 172.21.15.102:0/2035900658 conn(0x7fea640fb570 msgr2=0x7fea640fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.236+0000 7fea6bfc7700 1 -- 172.21.15.102:0/2035900658 shutdown_connections 2024-05-13T22:59:01.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.236+0000 7fea6bfc7700 1 -- 172.21.15.102:0/2035900658 wait complete. 2024-05-13T22:59:01.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.236+0000 7fea6bfc7700 1 Processor -- start 2024-05-13T22:59:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.236+0000 7fea6bfc7700 1 -- start start 2024-05-13T22:59:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.236+0000 7fea6bfc7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea640ffa40 0x7fea641975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.236+0000 7fea6bfc7700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea64105a40 0x7fea64197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.236+0000 7fea6bfc7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea641980b0 con 0x7fea640ffa40 2024-05-13T22:59:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.236+0000 7fea6bfc7700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea641981f0 con 0x7fea64105a40 2024-05-13T22:59:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea69d63700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea640ffa40 0x7fea641975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea69d63700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea640ffa40 0x7fea641975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43368/0 (socket says 172.21.15.102:43368) 2024-05-13T22:59:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea69d63700 1 -- 172.21.15.102:0/668061374 learned_addr learned my addr 172.21.15.102:0/668061374 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:01.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea69562700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea64105a40 0x7fea64197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:01.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea69d63700 1 -- 172.21.15.102:0/668061374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea64105a40 msgr2=0x7fea64197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea69d63700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea64105a40 0x7fea64197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea69d63700 1 -- 172.21.15.102:0/668061374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea600095f0 con 0x7fea640ffa40 2024-05-13T22:59:01.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea69562700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea64105a40 0x7fea64197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T22:59:01.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea69d63700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea640ffa40 0x7fea641975a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fea60005f50 tx=0x7fea6000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:01.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea5affd700 1 -- 172.21.15.102:0/668061374 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea6001d070 con 0x7fea640ffa40 2024-05-13T22:59:01.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea5affd700 1 -- 172.21.15.102:0/668061374 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea6000b610 con 0x7fea640ffa40 2024-05-13T22:59:01.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea5affd700 1 -- 172.21.15.102:0/668061374 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea60022470 con 0x7fea640ffa40 2024-05-13T22:59:01.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.237+0000 7fea6bfc7700 1 -- 172.21.15.102:0/668061374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea641a0610 con 0x7fea640ffa40 2024-05-13T22:59:01.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.238+0000 7fea6bfc7700 1 -- 172.21.15.102:0/668061374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea641a09c0 con 0x7fea640ffa40 2024-05-13T22:59:01.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.238+0000 7fea6bfc7700 1 -- 172.21.15.102:0/668061374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea64108040 con 0x7fea640ffa40 2024-05-13T22:59:01.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.239+0000 7fea5affd700 1 -- 172.21.15.102:0/668061374 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fea60003780 con 0x7fea640ffa40 2024-05-13T22:59:01.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.240+0000 7fea5affd700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fea5006e200 0x7fea500706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:01.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.240+0000 7fea5affd700 1 -- 172.21.15.102:0/668061374 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fea6008f570 con 0x7fea640ffa40 2024-05-13T22:59:01.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.240+0000 7fea69562700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fea5006e200 0x7fea500706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:01.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.240+0000 7fea69562700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fea5006e200 0x7fea500706c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fea64069500 tx=0x7fea54009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:01.221 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.245+0000 7fea5affd700 1 -- 172.21.15.102:0/668061374 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fea600587b0 con 0x7fea640ffa40 2024-05-13T22:59:01.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.535+0000 7fea6bfc7700 1 -- 172.21.15.102:0/668061374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fea6402cc70 con 0x7fea640ffa40 2024-05-13T22:59:01.512 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.535+0000 7fea5affd700 1 -- 172.21.15.102:0/668061374 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fea6005be70 con 0x7fea640ffa40 2024-05-13T22:59:01.512 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:59:01.512 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:59:01.512 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:59:01.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:59:01.512 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:59:01.512 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:59:01.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:59:01.513 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:59:01.513 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 6 2024-05-13T22:59:01.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:59:01.513 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:59:01.513 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 4 2024-05-13T22:59:01.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:59:01.513 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:59:01.513 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 14 2024-05-13T22:59:01.513 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:59:01.513 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:59:01.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.537+0000 7fea6bfc7700 1 -- 172.21.15.102:0/668061374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fea5006e200 msgr2=0x7fea500706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.537+0000 7fea6bfc7700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fea5006e200 0x7fea500706c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fea64069500 tx=0x7fea54009260 comp rx=0 tx=0).stop 2024-05-13T22:59:01.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.537+0000 7fea6bfc7700 1 -- 172.21.15.102:0/668061374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea640ffa40 msgr2=0x7fea641975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.537+0000 7fea6bfc7700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea640ffa40 0x7fea641975a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fea60005f50 tx=0x7fea6000f690 comp rx=0 tx=0).stop 2024-05-13T22:59:01.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.538+0000 7fea6bfc7700 1 -- 172.21.15.102:0/668061374 shutdown_connections 2024-05-13T22:59:01.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.538+0000 7fea6bfc7700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea640ffa40 0x7fea641975a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.538+0000 7fea6bfc7700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fea5006e200 0x7fea500706c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.538+0000 7fea6bfc7700 1 --2- 172.21.15.102:0/668061374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fea64105a40 0x7fea64197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.538+0000 7fea6bfc7700 1 -- 172.21.15.102:0/668061374 >> 172.21.15.102:0/668061374 conn(0x7fea640fb570 msgr2=0x7fea640fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:01.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.538+0000 7fea6bfc7700 1 -- 172.21.15.102:0/668061374 shutdown_connections 2024-05-13T22:59:01.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.538+0000 7fea6bfc7700 1 -- 172.21.15.102:0/668061374 wait complete. 2024-05-13T22:59:01.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.646+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1856554620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d8101770 msgr2=0x7fd0d81057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.646+0000 7fd0dc91a700 1 --2- 172.21.15.102:0/1856554620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d8101770 0x7fd0d81057c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009910 tx=0x7fd0cc009c20 comp rx=0 tx=0).stop 2024-05-13T22:59:01.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.646+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1856554620 shutdown_connections 2024-05-13T22:59:01.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.646+0000 7fd0dc91a700 1 --2- 172.21.15.102:0/1856554620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d8101770 0x7fd0d81057c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.646+0000 7fd0dc91a700 1 --2- 172.21.15.102:0/1856554620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0d8100dc0 0x7fd0d81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.646+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1856554620 >> 172.21.15.102:0/1856554620 conn(0x7fd0d80fc670 msgr2=0x7fd0d80fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:01.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.646+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1856554620 shutdown_connections 2024-05-13T22:59:01.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.646+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1856554620 wait complete. 2024-05-13T22:59:01.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.647+0000 7fd0dc91a700 1 Processor -- start 2024-05-13T22:59:01.624 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.647+0000 7fd0dc91a700 1 -- start start 2024-05-13T22:59:01.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.647+0000 7fd0dc91a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d8100dc0 0x7fd0d819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:01.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.647+0000 7fd0dc91a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0d8101770 0x7fd0d819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:01.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.647+0000 7fd0dc91a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d819f2c0 con 0x7fd0d8100dc0 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.647+0000 7fd0dc91a700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d8198800 con 0x7fd0d8101770 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0d659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d8100dc0 0x7fd0d819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0d659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d8100dc0 0x7fd0d819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43390/0 (socket says 172.21.15.102:43390) 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0d659c700 1 -- 172.21.15.102:0/1041619554 learned_addr learned my addr 172.21.15.102:0/1041619554 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0d5d9b700 1 --2- 172.21.15.102:0/1041619554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0d8101770 0x7fd0d819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0d659c700 1 -- 172.21.15.102:0/1041619554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0d8101770 msgr2=0x7fd0d819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0d659c700 1 --2- 172.21.15.102:0/1041619554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0d8101770 0x7fd0d819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0d659c700 1 -- 172.21.15.102:0/1041619554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc0095f0 con 0x7fd0d8100dc0 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0d659c700 1 --2- 172.21.15.102:0/1041619554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d8100dc0 0x7fd0d819e780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800c930 tx=0x7fd0c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0c77fe700 1 -- 172.21.15.102:0/1041619554 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c8007ad0 con 0x7fd0d8100dc0 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1041619554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d8198ae0 con 0x7fd0d8100dc0 2024-05-13T22:59:01.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1041619554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d8199030 con 0x7fd0d8100dc0 2024-05-13T22:59:01.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0c77fe700 1 -- 172.21.15.102:0/1041619554 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0c800ce80 con 0x7fd0d8100dc0 2024-05-13T22:59:01.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.648+0000 7fd0c77fe700 1 -- 172.21.15.102:0/1041619554 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c8018630 con 0x7fd0d8100dc0 2024-05-13T22:59:01.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.649+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1041619554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b8005320 con 0x7fd0d8100dc0 2024-05-13T22:59:01.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.650+0000 7fd0c77fe700 1 -- 172.21.15.102:0/1041619554 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd0c8007c30 con 0x7fd0d8100dc0 2024-05-13T22:59:01.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.650+0000 7fd0c77fe700 1 --2- 172.21.15.102:0/1041619554 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0c006e200 0x7fd0c00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:01.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.650+0000 7fd0c77fe700 1 -- 172.21.15.102:0/1041619554 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd0c808e1c0 con 0x7fd0d8100dc0 2024-05-13T22:59:01.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.650+0000 7fd0d5d9b700 1 --2- 172.21.15.102:0/1041619554 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0c006e200 0x7fd0c00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:01.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.651+0000 7fd0d5d9b700 1 --2- 172.21.15.102:0/1041619554 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0c006e200 0x7fd0c00706c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80fce40 tx=0x7fd0cc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:01.633 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.656+0000 7fd0c77fe700 1 -- 172.21.15.102:0/1041619554 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fd0c805aeb0 con 0x7fd0d8100dc0 2024-05-13T22:59:01.771 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:01 smithi102 ceph-mon[111694]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:01.771 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:01.771 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:01.771 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:01 smithi102 ceph-mon[111694]: from='client.44313 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:01.771 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:59:01.771 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:01 smithi102 ceph-mon[111694]: from='client.44317 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:01.771 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:01 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/668061374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:01.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.907+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1041619554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd0b8006200 con 0x7fd0d8100dc0 2024-05-13T22:59:01.884 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.907+0000 7fd0c77fe700 1 -- 172.21.15.102:0/1041619554 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1971 (secure 0 0 0) 0x7fd0c805aa40 con 0x7fd0d8100dc0 2024-05-13T22:59:01.885 INFO:teuthology.orchestra.run.smithi102.stdout:e37 2024-05-13T22:59:01.885 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T22:59:01.885 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T22:59:01.885 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T22:59:01.885 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:59:01.885 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T22:59:01.885 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T22:59:01.885 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 37 2024-05-13T22:59:01.885 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T22:59:01.885 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T22:59:01.885 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:58:39.442205+0000 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 96 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=34410} 2024-05-13T22:59:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 34410 members: 34410 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:34410} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:34414} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6828/1256538594,v1:172.21.15.102:6829/1256538594] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T22:59:01.887 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T22:59:01.888 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:44271} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/3836279348,v1:172.21.15.130:6825/3836279348] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T22:59:01.888 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6826/3806306097,v1:172.21.15.130:6827/3806306097] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.910+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1041619554 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0c006e200 msgr2=0x7fd0c00706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.910+0000 7fd0dc91a700 1 --2- 172.21.15.102:0/1041619554 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0c006e200 0x7fd0c00706c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80fce40 tx=0x7fd0cc01a040 comp rx=0 tx=0).stop 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.910+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1041619554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d8100dc0 msgr2=0x7fd0d819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.910+0000 7fd0dc91a700 1 --2- 172.21.15.102:0/1041619554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d8100dc0 0x7fd0d819e780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800c930 tx=0x7fd0c800cc40 comp rx=0 tx=0).stop 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.910+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1041619554 shutdown_connections 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.910+0000 7fd0dc91a700 1 --2- 172.21.15.102:0/1041619554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d8100dc0 0x7fd0d819e780 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.910+0000 7fd0dc91a700 1 --2- 172.21.15.102:0/1041619554 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fd0c006e200 0x7fd0c00706c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.910+0000 7fd0dc91a700 1 --2- 172.21.15.102:0/1041619554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0d8101770 0x7fd0d819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.910+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1041619554 >> 172.21.15.102:0/1041619554 conn(0x7fd0d80fc670 msgr2=0x7fd0d80fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.910+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1041619554 shutdown_connections 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:01.910+0000 7fd0dc91a700 1 -- 172.21.15.102:0/1041619554 wait complete. 2024-05-13T22:59:01.889 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 37 2024-05-13T22:59:01.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:01 smithi130 ceph-mon[84265]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:01.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:01.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:01.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:01 smithi130 ceph-mon[84265]: from='client.44313 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:01.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:59:01.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:01 smithi130 ceph-mon[84265]: from='client.44317 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:01.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:01 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/668061374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.019+0000 7f44adbad700 1 -- 172.21.15.102:0/733469993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a8101d60 msgr2=0x7f44a8102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.019+0000 7f44adbad700 1 --2- 172.21.15.102:0/733469993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a8101d60 0x7f44a8102140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4498009940 tx=0x7f4498009c50 comp rx=0 tx=0).stop 2024-05-13T22:59:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.020+0000 7f44adbad700 1 -- 172.21.15.102:0/733469993 shutdown_connections 2024-05-13T22:59:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.020+0000 7f44adbad700 1 --2- 172.21.15.102:0/733469993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a8102680 0x7f44a810ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.020+0000 7f44adbad700 1 --2- 172.21.15.102:0/733469993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a8101d60 0x7f44a8102140 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.020+0000 7f44adbad700 1 -- 172.21.15.102:0/733469993 >> 172.21.15.102:0/733469993 conn(0x7f44a80fb570 msgr2=0x7f44a80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.020+0000 7f44adbad700 1 -- 172.21.15.102:0/733469993 shutdown_connections 2024-05-13T22:59:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.020+0000 7f44adbad700 1 -- 172.21.15.102:0/733469993 wait complete. 2024-05-13T22:59:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.020+0000 7f44adbad700 1 Processor -- start 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.020+0000 7f44adbad700 1 -- start start 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44adbad700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a8101d60 0x7f44a819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44adbad700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a8102680 0x7f44a819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44adbad700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a819d100 con 0x7f44a8101d60 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44adbad700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a81965b0 con 0x7f44a8102680 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44a77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a8101d60 0x7f44a819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44a77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a8101d60 0x7f44a819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43410/0 (socket says 172.21.15.102:43410) 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44a77fe700 1 -- 172.21.15.102:0/1771889940 learned_addr learned my addr 172.21.15.102:0/1771889940 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44a6ffd700 1 --2- 172.21.15.102:0/1771889940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a8102680 0x7f44a819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44a77fe700 1 -- 172.21.15.102:0/1771889940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a8102680 msgr2=0x7f44a819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44a77fe700 1 --2- 172.21.15.102:0/1771889940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a8102680 0x7f44a819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:01.999 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44a77fe700 1 -- 172.21.15.102:0/1771889940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44980095f0 con 0x7f44a8101d60 2024-05-13T22:59:02.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.021+0000 7f44a77fe700 1 --2- 172.21.15.102:0/1771889940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a8101d60 0x7f44a819c530 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44980055f0 tx=0x7f4498005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:02.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.022+0000 7f44a4ff9700 1 -- 172.21.15.102:0/1771889940 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f449801d070 con 0x7f44a8101d60 2024-05-13T22:59:02.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.022+0000 7f44adbad700 1 -- 172.21.15.102:0/1771889940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a8196830 con 0x7f44a8101d60 2024-05-13T22:59:02.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.022+0000 7f44adbad700 1 -- 172.21.15.102:0/1771889940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a8196d20 con 0x7f44a8101d60 2024-05-13T22:59:02.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.022+0000 7f44a4ff9700 1 -- 172.21.15.102:0/1771889940 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4498005dc0 con 0x7f44a8101d60 2024-05-13T22:59:02.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.022+0000 7f44a4ff9700 1 -- 172.21.15.102:0/1771889940 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4498021410 con 0x7f44a8101d60 2024-05-13T22:59:02.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.023+0000 7f44adbad700 1 -- 172.21.15.102:0/1771889940 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4494005320 con 0x7f44a8101d60 2024-05-13T22:59:02.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.024+0000 7f44a4ff9700 1 -- 172.21.15.102:0/1771889940 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f449802b440 con 0x7f44a8101d60 2024-05-13T22:59:02.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.024+0000 7f44a4ff9700 1 --2- 172.21.15.102:0/1771889940 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f449006e200 0x7f44900706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:02.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.024+0000 7f44a4ff9700 1 -- 172.21.15.102:0/1771889940 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f449808f390 con 0x7f44a8101d60 2024-05-13T22:59:02.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.024+0000 7f44a6ffd700 1 --2- 172.21.15.102:0/1771889940 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f449006e200 0x7f44900706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:02.002 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.025+0000 7f44a6ffd700 1 --2- 172.21.15.102:0/1771889940 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f449006e200 0x7f44900706c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f44a80fcc40 tx=0x7f449c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:02.006 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.030+0000 7f44a4ff9700 1 -- 172.21.15.102:0/1771889940 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f449805c080 con 0x7f44a8101d60 2024-05-13T22:59:02.237 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.260+0000 7f44adbad700 1 -- 172.21.15.102:0/1771889940 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4494000bf0 con 0x7f449006e200 2024-05-13T22:59:02.238 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.262+0000 7f44a4ff9700 1 -- 172.21.15.102:0/1771889940 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f4494000bf0 con 0x7f449006e200 2024-05-13T22:59:02.239 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:59:02.239 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64", 2024-05-13T22:59:02.239 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": true, 2024-05-13T22:59:02.239 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-13T22:59:02.239 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [ 2024-05-13T22:59:02.239 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph-exporter", 2024-05-13T22:59:02.239 INFO:teuthology.orchestra.run.smithi102.stdout: "mon", 2024-05-13T22:59:02.239 INFO:teuthology.orchestra.run.smithi102.stdout: "mds", 2024-05-13T22:59:02.239 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr", 2024-05-13T22:59:02.240 INFO:teuthology.orchestra.run.smithi102.stdout: "crash", 2024-05-13T22:59:02.240 INFO:teuthology.orchestra.run.smithi102.stdout: "osd" 2024-05-13T22:59:02.240 INFO:teuthology.orchestra.run.smithi102.stdout: ], 2024-05-13T22:59:02.240 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": "18/23 daemons upgraded", 2024-05-13T22:59:02.240 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-05-13T22:59:02.240 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:59:02.240 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:59:02.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.264+0000 7f44adbad700 1 -- 172.21.15.102:0/1771889940 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f449006e200 msgr2=0x7f44900706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:02.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.264+0000 7f44adbad700 1 --2- 172.21.15.102:0/1771889940 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f449006e200 0x7f44900706c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f44a80fcc40 tx=0x7f449c00b410 comp rx=0 tx=0).stop 2024-05-13T22:59:02.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.264+0000 7f44adbad700 1 -- 172.21.15.102:0/1771889940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a8101d60 msgr2=0x7f44a819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:02.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.264+0000 7f44adbad700 1 --2- 172.21.15.102:0/1771889940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a8101d60 0x7f44a819c530 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44980055f0 tx=0x7f4498005620 comp rx=0 tx=0).stop 2024-05-13T22:59:02.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.264+0000 7f44adbad700 1 -- 172.21.15.102:0/1771889940 shutdown_connections 2024-05-13T22:59:02.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.264+0000 7f44adbad700 1 --2- 172.21.15.102:0/1771889940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44a8101d60 0x7f44a819c530 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:02.241 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.264+0000 7f44adbad700 1 --2- 172.21.15.102:0/1771889940 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f449006e200 0x7f44900706c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:02.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.264+0000 7f44adbad700 1 --2- 172.21.15.102:0/1771889940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f44a8102680 0x7f44a819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:02.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.264+0000 7f44adbad700 1 -- 172.21.15.102:0/1771889940 >> 172.21.15.102:0/1771889940 conn(0x7f44a80fb570 msgr2=0x7f44a81053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:02.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.264+0000 7f44adbad700 1 -- 172.21.15.102:0/1771889940 shutdown_connections 2024-05-13T22:59:02.242 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.264+0000 7f44adbad700 1 -- 172.21.15.102:0/1771889940 wait complete. 2024-05-13T22:59:02.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.373+0000 7f1a1fda9700 1 -- 172.21.15.102:0/1579621509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a180ffa40 msgr2=0x7f1a180ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:02.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.373+0000 7f1a1fda9700 1 --2- 172.21.15.102:0/1579621509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a180ffa40 0x7f1a180ffec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009910 tx=0x7f1a0c009c20 comp rx=0 tx=0).stop 2024-05-13T22:59:02.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.373+0000 7f1a1fda9700 1 -- 172.21.15.102:0/1579621509 shutdown_connections 2024-05-13T22:59:02.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.373+0000 7f1a1fda9700 1 --2- 172.21.15.102:0/1579621509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a180ffa40 0x7f1a180ffec0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:02.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.373+0000 7f1a1fda9700 1 --2- 172.21.15.102:0/1579621509 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a18105a40 0x7f1a18105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:02.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.373+0000 7f1a1fda9700 1 -- 172.21.15.102:0/1579621509 >> 172.21.15.102:0/1579621509 conn(0x7f1a180fb570 msgr2=0x7f1a180fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:02.350 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.373+0000 7f1a1fda9700 1 -- 172.21.15.102:0/1579621509 shutdown_connections 2024-05-13T22:59:02.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.373+0000 7f1a1fda9700 1 -- 172.21.15.102:0/1579621509 wait complete. 2024-05-13T22:59:02.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1fda9700 1 Processor -- start 2024-05-13T22:59:02.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1fda9700 1 -- start start 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1fda9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a180ffa40 0x7f1a18197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1fda9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a18105a40 0x7f1a18197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1fda9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a181980d0 con 0x7f1a18105a40 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1fda9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a18198240 con 0x7f1a180ffa40 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1d344700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a18105a40 0x7f1a18197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1d344700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a18105a40 0x7f1a18197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:43432/0 (socket says 172.21.15.102:43432) 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1d344700 1 -- 172.21.15.102:0/3073651234 learned_addr learned my addr 172.21.15.102:0/3073651234 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1db45700 1 --2- 172.21.15.102:0/3073651234 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a180ffa40 0x7f1a18197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1d344700 1 -- 172.21.15.102:0/3073651234 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a180ffa40 msgr2=0x7f1a18197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1d344700 1 --2- 172.21.15.102:0/3073651234 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a180ffa40 0x7f1a18197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.374+0000 7f1a1d344700 1 -- 172.21.15.102:0/3073651234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c0095f0 con 0x7f1a18105a40 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.375+0000 7f1a1d344700 1 --2- 172.21.15.102:0/3073651234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a18105a40 0x7f1a18197ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c005f50 tx=0x7f1a0c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.375+0000 7f1a16ffd700 1 -- 172.21.15.102:0/3073651234 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a0c01d070 con 0x7f1a18105a40 2024-05-13T22:59:02.353 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.375+0000 7f1a16ffd700 1 -- 172.21.15.102:0/3073651234 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a0c00b610 con 0x7f1a18105a40 2024-05-13T22:59:02.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.375+0000 7f1a1fda9700 1 -- 172.21.15.102:0/3073651234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a181a05e0 con 0x7f1a18105a40 2024-05-13T22:59:02.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.375+0000 7f1a16ffd700 1 -- 172.21.15.102:0/3073651234 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a0c022470 con 0x7f1a18105a40 2024-05-13T22:59:02.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.375+0000 7f1a1fda9700 1 -- 172.21.15.102:0/3073651234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a181a0a50 con 0x7f1a18105a40 2024-05-13T22:59:02.354 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.376+0000 7f1a1fda9700 1 -- 172.21.15.102:0/3073651234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a18107fd0 con 0x7f1a18105a40 2024-05-13T22:59:02.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.376+0000 7f1a16ffd700 1 -- 172.21.15.102:0/3073651234 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1a0c003780 con 0x7f1a18105a40 2024-05-13T22:59:02.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.377+0000 7f1a16ffd700 1 --2- 172.21.15.102:0/3073651234 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a0006e200 0x7f1a000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:02.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.377+0000 7f1a16ffd700 1 -- 172.21.15.102:0/3073651234 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1a0c08f6a0 con 0x7f1a18105a40 2024-05-13T22:59:02.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.377+0000 7f1a1db45700 1 --2- 172.21.15.102:0/3073651234 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a0006e200 0x7f1a000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:02.355 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.377+0000 7f1a1db45700 1 --2- 172.21.15.102:0/3073651234 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a0006e200 0x7f1a000706c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000f540 tx=0x7f1a1000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:02.358 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.382+0000 7f1a16ffd700 1 -- 172.21.15.102:0/3073651234 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1a0c058860 con 0x7f1a18105a40 2024-05-13T22:59:02.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.675+0000 7f1a1fda9700 1 -- 172.21.15.102:0/3073651234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1a1804ea90 con 0x7f1a18105a40 2024-05-13T22:59:02.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.675+0000 7f1a16ffd700 1 -- 172.21.15.102:0/3073651234 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f1a0c05bf20 con 0x7f1a18105a40 2024-05-13T22:59:02.652 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:59:02.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.678+0000 7f1a1fda9700 1 -- 172.21.15.102:0/3073651234 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a0006e200 msgr2=0x7f1a000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:02.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.678+0000 7f1a1fda9700 1 --2- 172.21.15.102:0/3073651234 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a0006e200 0x7f1a000706c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000f540 tx=0x7f1a1000b040 comp rx=0 tx=0).stop 2024-05-13T22:59:02.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.678+0000 7f1a1fda9700 1 -- 172.21.15.102:0/3073651234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a18105a40 msgr2=0x7f1a18197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:02.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.678+0000 7f1a1fda9700 1 --2- 172.21.15.102:0/3073651234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a18105a40 0x7f1a18197ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c005f50 tx=0x7f1a0c00f690 comp rx=0 tx=0).stop 2024-05-13T22:59:02.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.678+0000 7f1a1fda9700 1 -- 172.21.15.102:0/3073651234 shutdown_connections 2024-05-13T22:59:02.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.678+0000 7f1a1fda9700 1 --2- 172.21.15.102:0/3073651234 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a0006e200 0x7f1a000706c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:02.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.678+0000 7f1a1fda9700 1 --2- 172.21.15.102:0/3073651234 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a180ffa40 0x7f1a18197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:02.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.678+0000 7f1a1fda9700 1 --2- 172.21.15.102:0/3073651234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a18105a40 0x7f1a18197ab0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:02.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.678+0000 7f1a1fda9700 1 -- 172.21.15.102:0/3073651234 >> 172.21.15.102:0/3073651234 conn(0x7f1a180fb570 msgr2=0x7f1a180fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:02.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.678+0000 7f1a1fda9700 1 -- 172.21.15.102:0/3073651234 shutdown_connections 2024-05-13T22:59:02.655 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:02.678+0000 7f1a1fda9700 1 -- 172.21.15.102:0/3073651234 wait complete. 2024-05-13T22:59:02.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:02.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:02.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:02 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1041619554' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:59:02.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:02.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:02 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:02.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:02 smithi102 ceph-mon[111694]: pgmap v237: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 413 KiB/s rd, 1 op/s 2024-05-13T22:59:02.926 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:02 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3073651234' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:59:03.148 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:03.148 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:03.148 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:02 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1041619554' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T22:59:03.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:03.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:02 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:03.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:02 smithi130 ceph-mon[84265]: pgmap v237: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 413 KiB/s rd, 1 op/s 2024-05-13T22:59:03.149 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:02 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3073651234' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:59:04.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:04 smithi130 ceph-mon[84265]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:04.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:04 smithi102 ceph-mon[111694]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:05.175 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:05 smithi102 ceph-mon[111694]: pgmap v238: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-13T22:59:05.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:05 smithi130 ceph-mon[84265]: pgmap v238: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-13T22:59:06.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:59:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:59:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:59:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:06.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: Upgrade: Setting filesystem cephfs Joinable 2024-05-13T22:59:06.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-13T22:59:06.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:06 smithi130 ceph-mon[84265]: pgmap v239: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:06.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:06.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:06.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:59:06.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:59:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:59:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: Upgrade: Setting filesystem cephfs Joinable 2024-05-13T22:59:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-13T22:59:06.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:06 smithi102 ceph-mon[111694]: pgmap v239: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:07.593 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-13T22:59:07.593 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:59:07.593 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.593 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.593 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.593 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.593 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: Upgrade: Setting container_image for all ceph-exporter 2024-05-13T22:59:07.593 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:07.593 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi102"}]: dispatch 2024-05-13T22:59:07.593 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi102"}]': finished 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi130"}]: dispatch 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi130"}]': finished 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: Upgrade: Setting container_image for all iscsi 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: Upgrade: Setting container_image for all nfs 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: Upgrade: Setting container_image for all nvmeof 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.594 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: Upgrade: Finalizing container_image settings 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-13T22:59:07.595 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-13T22:59:07.596 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: Upgrade: Complete! 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:07.597 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:59:07.598 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:59:07.598 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:59:07.598 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:07.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-13T22:59:07.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: fsmap cephfs:1 {0=cephfs.smithi102.cgkcwz=up:active} 1 up:standby-replay 2 up:standby 2024-05-13T22:59:07.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: Upgrade: Setting container_image for all ceph-exporter 2024-05-13T22:59:07.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi102"}]: dispatch 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi102"}]': finished 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi130"}]: dispatch 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi130"}]': finished 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: Upgrade: Setting container_image for all iscsi 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: Upgrade: Setting container_image for all nfs 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: Upgrade: Setting container_image for all nvmeof 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: Upgrade: Finalizing container_image settings 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-13T22:59:07.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-13T22:59:07.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: Upgrade: Complete! 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-13T22:59:07.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-13T22:59:07.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:59:07.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:59:07.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:59:07.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:07.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T22:59:07.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T22:59:07.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T22:59:07.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:08.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:08 smithi130 ceph-mon[84265]: pgmap v240: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:08.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:08 smithi102 ceph-mon[111694]: pgmap v240: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:11.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:11 smithi130 ceph-mon[84265]: pgmap v241: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:59:11.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:11 smithi102 ceph-mon[111694]: pgmap v241: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:59:12.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:12.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T22:59:13.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:13 smithi130 ceph-mon[84265]: pgmap v242: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:13.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:13 smithi102 ceph-mon[111694]: pgmap v242: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:15.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:15 smithi130 ceph-mon[84265]: pgmap v243: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:15.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:15 smithi102 ceph-mon[111694]: pgmap v243: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:16.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:59:16.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:59:17.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:17 smithi102 ceph-mon[111694]: pgmap v244: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:17.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:17 smithi130 ceph-mon[84265]: pgmap v244: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:19.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:19 smithi102 ceph-mon[111694]: pgmap v245: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:19.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:19 smithi130 ceph-mon[84265]: pgmap v245: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:21.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:21 smithi102 ceph-mon[111694]: pgmap v246: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:59:21.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:21 smithi130 ceph-mon[84265]: pgmap v246: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:59:23.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:23 smithi102 ceph-mon[111694]: pgmap v247: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:23.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:23 smithi130 ceph-mon[84265]: pgmap v247: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:25.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:25 smithi102 ceph-mon[111694]: pgmap v248: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:25.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:25 smithi130 ceph-mon[84265]: pgmap v248: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:27.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:27 smithi102 ceph-mon[111694]: pgmap v249: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:27.451 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:27 smithi130 ceph-mon[84265]: pgmap v249: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:29.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:29 smithi102 ceph-mon[111694]: pgmap v250: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:29.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:29 smithi130 ceph-mon[84265]: pgmap v250: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:31.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:31 smithi102 ceph-mon[111694]: pgmap v251: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:59:31.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:59:31.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:31 smithi130 ceph-mon[84265]: pgmap v251: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:59:31.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:59:32.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.799+0000 7f10a79d8700 1 -- 172.21.15.102:0/2306462818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a00ffa40 msgr2=0x7f10a00ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:32.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.799+0000 7f10a79d8700 1 --2- 172.21.15.102:0/2306462818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a00ffa40 0x7f10a00ffec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1090009910 tx=0x7f1090009c20 comp rx=0 tx=0).stop 2024-05-13T22:59:32.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.799+0000 7f10a79d8700 1 -- 172.21.15.102:0/2306462818 shutdown_connections 2024-05-13T22:59:32.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.799+0000 7f10a79d8700 1 --2- 172.21.15.102:0/2306462818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a00ffa40 0x7f10a00ffec0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:32.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.799+0000 7f10a79d8700 1 --2- 172.21.15.102:0/2306462818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10a0105a40 0x7f10a0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:32.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.799+0000 7f10a79d8700 1 -- 172.21.15.102:0/2306462818 >> 172.21.15.102:0/2306462818 conn(0x7f10a00fb530 msgr2=0x7f10a00fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:32.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.799+0000 7f10a79d8700 1 -- 172.21.15.102:0/2306462818 shutdown_connections 2024-05-13T22:59:32.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.799+0000 7f10a79d8700 1 -- 172.21.15.102:0/2306462818 wait complete. 2024-05-13T22:59:32.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a79d8700 1 Processor -- start 2024-05-13T22:59:32.777 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a79d8700 1 -- start start 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a79d8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a00ffa40 0x7f10a0196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a79d8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10a0105a40 0x7f10a0196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a79d8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a0197250 con 0x7f10a00ffa40 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a79d8700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01973c0 con 0x7f10a0105a40 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a5774700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a00ffa40 0x7f10a0196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a5774700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a00ffa40 0x7f10a0196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35370/0 (socket says 172.21.15.102:35370) 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a5774700 1 -- 172.21.15.102:0/3313875270 learned_addr learned my addr 172.21.15.102:0/3313875270 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a5774700 1 -- 172.21.15.102:0/3313875270 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10a0105a40 msgr2=0x7f10a0196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a5774700 1 --2- 172.21.15.102:0/3313875270 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10a0105a40 0x7f10a0196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a5774700 1 -- 172.21.15.102:0/3313875270 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900095f0 con 0x7f10a00ffa40 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.800+0000 7f10a5774700 1 --2- 172.21.15.102:0/3313875270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a00ffa40 0x7f10a0196710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f109c00c930 tx=0x7f109c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.801+0000 7f10967fc700 1 -- 172.21.15.102:0/3313875270 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f109c007ad0 con 0x7f10a00ffa40 2024-05-13T22:59:32.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.801+0000 7f10967fc700 1 -- 172.21.15.102:0/3313875270 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f109c007c30 con 0x7f10a00ffa40 2024-05-13T22:59:32.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.801+0000 7f10a79d8700 1 -- 172.21.15.102:0/3313875270 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a019a160 con 0x7f10a00ffa40 2024-05-13T22:59:32.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.801+0000 7f10967fc700 1 -- 172.21.15.102:0/3313875270 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f109c018610 con 0x7f10a00ffa40 2024-05-13T22:59:32.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.801+0000 7f10a79d8700 1 -- 172.21.15.102:0/3313875270 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a019a6b0 con 0x7f10a00ffa40 2024-05-13T22:59:32.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.802+0000 7f10a79d8700 1 -- 172.21.15.102:0/3313875270 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a0108fe0 con 0x7f10a00ffa40 2024-05-13T22:59:32.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.802+0000 7f10967fc700 1 -- 172.21.15.102:0/3313875270 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f109c00f450 con 0x7f10a00ffa40 2024-05-13T22:59:32.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.803+0000 7f10967fc700 1 --2- 172.21.15.102:0/3313875270 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f108c06e200 0x7f108c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:32.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.803+0000 7f10967fc700 1 -- 172.21.15.102:0/3313875270 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f109c08e4c0 con 0x7f10a00ffa40 2024-05-13T22:59:32.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.803+0000 7f10a4f73700 1 --2- 172.21.15.102:0/3313875270 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f108c06e200 0x7f108c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:32.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.803+0000 7f10a4f73700 1 --2- 172.21.15.102:0/3313875270 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f108c06e200 0x7f108c0706c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f109000b5c0 tx=0x7f1090005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:32.783 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:32.807+0000 7f10967fc700 1 -- 172.21.15.102:0/3313875270 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f109c057680 con 0x7f10a00ffa40 2024-05-13T22:59:32.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.021+0000 7f10a79d8700 1 -- 172.21.15.102:0/3313875270 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f10a002cce0 con 0x7f108c06e200 2024-05-13T22:59:32.998 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.022+0000 7f10967fc700 1 -- 172.21.15.102:0/3313875270 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f10a002cce0 con 0x7f108c06e200 2024-05-13T22:59:33.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.025+0000 7f10a79d8700 1 -- 172.21.15.102:0/3313875270 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f108c06e200 msgr2=0x7f108c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:33.000 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.025+0000 7f10a79d8700 1 --2- 172.21.15.102:0/3313875270 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f108c06e200 0x7f108c0706c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f109000b5c0 tx=0x7f1090005960 comp rx=0 tx=0).stop 2024-05-13T22:59:33.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.025+0000 7f10a79d8700 1 -- 172.21.15.102:0/3313875270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a00ffa40 msgr2=0x7f10a0196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:33.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.025+0000 7f10a79d8700 1 --2- 172.21.15.102:0/3313875270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a00ffa40 0x7f10a0196710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f109c00c930 tx=0x7f109c00ccf0 comp rx=0 tx=0).stop 2024-05-13T22:59:33.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.025+0000 7f10a79d8700 1 -- 172.21.15.102:0/3313875270 shutdown_connections 2024-05-13T22:59:33.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.025+0000 7f10a79d8700 1 --2- 172.21.15.102:0/3313875270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10a00ffa40 0x7f10a0196710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:33.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.025+0000 7f10a79d8700 1 --2- 172.21.15.102:0/3313875270 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f108c06e200 0x7f108c0706c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:33.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.025+0000 7f10a79d8700 1 --2- 172.21.15.102:0/3313875270 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10a0105a40 0x7f10a0196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:33.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.025+0000 7f10a79d8700 1 -- 172.21.15.102:0/3313875270 >> 172.21.15.102:0/3313875270 conn(0x7f10a00fb530 msgr2=0x7f10a00690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:33.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.025+0000 7f10a79d8700 1 -- 172.21.15.102:0/3313875270 shutdown_connections 2024-05-13T22:59:33.001 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:33.025+0000 7f10a79d8700 1 -- 172.21.15.102:0/3313875270 wait complete. 2024-05-13T22:59:33.398 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:33 smithi102 ceph-mon[111694]: pgmap v252: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:33.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:33 smithi130 ceph-mon[84265]: pgmap v252: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:33.765 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph orch ps' 2024-05-13T22:59:33.976 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:59:34.351 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:34 smithi102 ceph-mon[111694]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:34.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:34 smithi130 ceph-mon[84265]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:35.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:35 smithi102 ceph-mon[111694]: pgmap v253: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:35.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.646+0000 7fae6df10700 1 -- 172.21.15.102:0/542986841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae68100410 msgr2=0x7fae681048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:35.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.646+0000 7fae6df10700 1 --2- 172.21.15.102:0/542986841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae68100410 0x7fae681048e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009910 tx=0x7fae5c009c20 comp rx=0 tx=0).stop 2024-05-13T22:59:35.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.647+0000 7fae6df10700 1 -- 172.21.15.102:0/542986841 shutdown_connections 2024-05-13T22:59:35.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.647+0000 7fae6df10700 1 --2- 172.21.15.102:0/542986841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae68100410 0x7fae681048e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:35.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.647+0000 7fae6df10700 1 --2- 172.21.15.102:0/542986841 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae680ffa60 0x7fae680ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:35.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.647+0000 7fae6df10700 1 -- 172.21.15.102:0/542986841 >> 172.21.15.102:0/542986841 conn(0x7fae680fb550 msgr2=0x7fae680fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:35.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.647+0000 7fae6df10700 1 -- 172.21.15.102:0/542986841 shutdown_connections 2024-05-13T22:59:35.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.647+0000 7fae6df10700 1 -- 172.21.15.102:0/542986841 wait complete. 2024-05-13T22:59:35.623 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.647+0000 7fae6df10700 1 Processor -- start 2024-05-13T22:59:35.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae6df10700 1 -- start start 2024-05-13T22:59:35.625 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae6df10700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae680ffa60 0x7fae6819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae6df10700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae68100410 0x7fae6819d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae6df10700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6819d970 con 0x7fae680ffa60 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae6df10700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae681a1700 con 0x7fae68100410 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae677fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae680ffa60 0x7fae6819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae677fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae680ffa60 0x7fae6819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35396/0 (socket says 172.21.15.102:35396) 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae677fe700 1 -- 172.21.15.102:0/3505614742 learned_addr learned my addr 172.21.15.102:0/3505614742 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae66ffd700 1 --2- 172.21.15.102:0/3505614742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae68100410 0x7fae6819d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae677fe700 1 -- 172.21.15.102:0/3505614742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae68100410 msgr2=0x7fae6819d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae677fe700 1 --2- 172.21.15.102:0/3505614742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae68100410 0x7fae6819d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae677fe700 1 -- 172.21.15.102:0/3505614742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0095f0 con 0x7fae680ffa60 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.648+0000 7fae677fe700 1 --2- 172.21.15.102:0/3505614742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae680ffa60 0x7fae6819cd50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae5800d8d0 tx=0x7fae5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.649+0000 7fae64ff9700 1 -- 172.21.15.102:0/3505614742 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae58009940 con 0x7fae680ffa60 2024-05-13T22:59:35.626 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.649+0000 7fae6df10700 1 -- 172.21.15.102:0/3505614742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae681a19e0 con 0x7fae680ffa60 2024-05-13T22:59:35.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.649+0000 7fae64ff9700 1 -- 172.21.15.102:0/3505614742 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae58010460 con 0x7fae680ffa60 2024-05-13T22:59:35.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.649+0000 7fae64ff9700 1 -- 172.21.15.102:0/3505614742 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae5800f5f0 con 0x7fae680ffa60 2024-05-13T22:59:35.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.649+0000 7fae6df10700 1 -- 172.21.15.102:0/3505614742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae681a1f30 con 0x7fae680ffa60 2024-05-13T22:59:35.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.650+0000 7fae6df10700 1 -- 172.21.15.102:0/3505614742 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae68102040 con 0x7fae680ffa60 2024-05-13T22:59:35.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.650+0000 7fae64ff9700 1 -- 172.21.15.102:0/3505614742 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fae580105d0 con 0x7fae680ffa60 2024-05-13T22:59:35.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.651+0000 7fae64ff9700 1 --2- 172.21.15.102:0/3505614742 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fae5006e200 0x7fae500706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:35.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.651+0000 7fae64ff9700 1 -- 172.21.15.102:0/3505614742 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fae5808e2c0 con 0x7fae680ffa60 2024-05-13T22:59:35.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.651+0000 7fae66ffd700 1 --2- 172.21.15.102:0/3505614742 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fae5006e200 0x7fae500706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:35.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.651+0000 7fae66ffd700 1 --2- 172.21.15.102:0/3505614742 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fae5006e200 0x7fae500706c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fae6819e370 tx=0x7fae5c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:35.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.655+0000 7fae64ff9700 1 -- 172.21.15.102:0/3505614742 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fae58057480 con 0x7fae680ffa60 2024-05-13T22:59:35.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:35 smithi130 ceph-mon[84265]: pgmap v253: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:35.836 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.861+0000 7fae6df10700 1 -- 172.21.15.102:0/3505614742 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fae681a2260 con 0x7fae5006e200 2024-05-13T22:59:35.849 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.871+0000 7fae64ff9700 1 -- 172.21.15.102:0/3505614742 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fae681a2260 con 0x7fae5006e200 2024-05-13T22:59:35.850 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T22:59:35.850 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (8m) 35s ago 19m 26.0M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T22:59:35.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (46s) 35s ago 20m 7936k - 19.0.0-2565-gc3e55145 b1b56bed7c91 bfcb10ba7c2a 2024-05-13T22:59:35.850 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (41s) 38s ago 19m 7004k - 19.0.0-2565-gc3e55145 b1b56bed7c91 035d93e33eb8 2024-05-13T22:59:35.850 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (5m) 35s ago 20m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T22:59:35.850 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (5m) 38s ago 18m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T22:59:35.850 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (8m) 35s ago 19m 79.9M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (2m) 35s ago 14m 89.5M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1a34f8c7d234 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (103s) 35s ago 14m 110M - 19.0.0-2565-gc3e55145 b1b56bed7c91 2b50dedff13c 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (85s) 38s ago 14m 13.7M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1a45ebbee664 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (66s) 38s ago 14m 15.7M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1d727b21600b 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (11m) 35s ago 21m 513M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (9m) 38s ago 18m 454M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (7m) 35s ago 21m 60.6M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (6m) 38s ago 18m 49.2M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (9m) 35s ago 19m 16.2M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (9m) 38s ago 18m 17.1M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (5m) 35s ago 17m 142M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 0ada88c777a2 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (4m) 35s ago 17m 119M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 c3e9e68b06fd 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (4m) 35s ago 17m 106M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 1b85728dccf5 2024-05-13T22:59:35.851 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (3m) 38s ago 16m 110M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 66e804d4b801 2024-05-13T22:59:35.852 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (3m) 38s ago 16m 101M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5bb5a8f94042 2024-05-13T22:59:35.852 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (2m) 38s ago 15m 102M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5d983135967d 2024-05-13T22:59:35.852 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (9m) 35s ago 19m 80.2M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T22:59:35.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.873+0000 7fae6df10700 1 -- 172.21.15.102:0/3505614742 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fae5006e200 msgr2=0x7fae500706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:35.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.873+0000 7fae6df10700 1 --2- 172.21.15.102:0/3505614742 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fae5006e200 0x7fae500706c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fae6819e370 tx=0x7fae5c01a040 comp rx=0 tx=0).stop 2024-05-13T22:59:35.852 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.873+0000 7fae6df10700 1 -- 172.21.15.102:0/3505614742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae680ffa60 msgr2=0x7fae6819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:35.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.873+0000 7fae6df10700 1 --2- 172.21.15.102:0/3505614742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae680ffa60 0x7fae6819cd50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae5800d8d0 tx=0x7fae5800dc90 comp rx=0 tx=0).stop 2024-05-13T22:59:35.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.873+0000 7fae6df10700 1 -- 172.21.15.102:0/3505614742 shutdown_connections 2024-05-13T22:59:35.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.873+0000 7fae6df10700 1 --2- 172.21.15.102:0/3505614742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae680ffa60 0x7fae6819cd50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:35.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.873+0000 7fae6df10700 1 --2- 172.21.15.102:0/3505614742 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fae5006e200 0x7fae500706c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:35.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.873+0000 7fae6df10700 1 --2- 172.21.15.102:0/3505614742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae68100410 0x7fae6819d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:35.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.873+0000 7fae6df10700 1 -- 172.21.15.102:0/3505614742 >> 172.21.15.102:0/3505614742 conn(0x7fae680fb550 msgr2=0x7fae68110f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:35.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.874+0000 7fae6df10700 1 -- 172.21.15.102:0/3505614742 shutdown_connections 2024-05-13T22:59:35.853 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:35.874+0000 7fae6df10700 1 -- 172.21.15.102:0/3505614742 wait complete. 2024-05-13T22:59:36.479 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph orch upgrade status' 2024-05-13T22:59:36.705 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:59:37.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:37 smithi102 ceph-mon[111694]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:37.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:37 smithi102 ceph-mon[111694]: pgmap v254: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:37.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:37 smithi130 ceph-mon[84265]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:37.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:37 smithi130 ceph-mon[84265]: pgmap v254: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:38.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.596+0000 7f9603ff4700 1 -- 172.21.15.102:0/2869088849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95fc101780 msgr2=0x7f95fc1057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:38.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.596+0000 7f9603ff4700 1 --2- 172.21.15.102:0/2869088849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95fc101780 0x7f95fc1057d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009910 tx=0x7f95ec009c20 comp rx=0 tx=0).stop 2024-05-13T22:59:38.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.596+0000 7f9603ff4700 1 -- 172.21.15.102:0/2869088849 shutdown_connections 2024-05-13T22:59:38.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.596+0000 7f9603ff4700 1 --2- 172.21.15.102:0/2869088849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95fc101780 0x7f95fc1057d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:38.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.596+0000 7f9603ff4700 1 --2- 172.21.15.102:0/2869088849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95fc100dd0 0x7f95fc1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:38.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.596+0000 7f9603ff4700 1 -- 172.21.15.102:0/2869088849 >> 172.21.15.102:0/2869088849 conn(0x7f95fc0fc640 msgr2=0x7f95fc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:38.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.596+0000 7f9603ff4700 1 -- 172.21.15.102:0/2869088849 shutdown_connections 2024-05-13T22:59:38.572 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.596+0000 7f9603ff4700 1 -- 172.21.15.102:0/2869088849 wait complete. 2024-05-13T22:59:38.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9603ff4700 1 Processor -- start 2024-05-13T22:59:38.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9603ff4700 1 -- start start 2024-05-13T22:59:38.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9603ff4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95fc100dd0 0x7f95fc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:38.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9603ff4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95fc101780 0x7f95fc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:38.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9603ff4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc073b10 con 0x7f95fc100dd0 2024-05-13T22:59:38.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9603ff4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc073c50 con 0x7f95fc101780 2024-05-13T22:59:38.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9601d90700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95fc100dd0 0x7f95fc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:38.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9601d90700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95fc100dd0 0x7f95fc073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35232/0 (socket says 172.21.15.102:35232) 2024-05-13T22:59:38.574 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9601d90700 1 -- 172.21.15.102:0/1810679815 learned_addr learned my addr 172.21.15.102:0/1810679815 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:38.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f960158f700 1 --2- 172.21.15.102:0/1810679815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95fc101780 0x7f95fc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:38.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9601d90700 1 -- 172.21.15.102:0/1810679815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95fc101780 msgr2=0x7f95fc0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:38.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9601d90700 1 --2- 172.21.15.102:0/1810679815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95fc101780 0x7f95fc0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:38.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9601d90700 1 -- 172.21.15.102:0/1810679815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec0095f0 con 0x7f95fc100dd0 2024-05-13T22:59:38.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9601d90700 1 --2- 172.21.15.102:0/1810679815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95fc100dd0 0x7f95fc073090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f95f800d8d0 tx=0x7f95f800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:38.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f95f2ffd700 1 -- 172.21.15.102:0/1810679815 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f80098a0 con 0x7f95fc100dd0 2024-05-13T22:59:38.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.597+0000 7f9603ff4700 1 -- 172.21.15.102:0/1810679815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc1a2990 con 0x7f95fc100dd0 2024-05-13T22:59:38.575 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.598+0000 7f95f2ffd700 1 -- 172.21.15.102:0/1810679815 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95f8010460 con 0x7f95fc100dd0 2024-05-13T22:59:38.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.598+0000 7f95f2ffd700 1 -- 172.21.15.102:0/1810679815 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f800f5f0 con 0x7f95fc100dd0 2024-05-13T22:59:38.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.598+0000 7f9603ff4700 1 -- 172.21.15.102:0/1810679815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc1a2da0 con 0x7f95fc100dd0 2024-05-13T22:59:38.576 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.598+0000 7f9603ff4700 1 -- 172.21.15.102:0/1810679815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc1091d0 con 0x7f95fc100dd0 2024-05-13T22:59:38.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.599+0000 7f95f2ffd700 1 -- 172.21.15.102:0/1810679815 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f95f8009a00 con 0x7f95fc100dd0 2024-05-13T22:59:38.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.599+0000 7f95f2ffd700 1 --2- 172.21.15.102:0/1810679815 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f95e806e200 0x7f95e80706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:38.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.600+0000 7f95f2ffd700 1 -- 172.21.15.102:0/1810679815 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f95f808e190 con 0x7f95fc100dd0 2024-05-13T22:59:38.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.600+0000 7f960158f700 1 --2- 172.21.15.102:0/1810679815 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f95e806e200 0x7f95e80706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:38.577 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.600+0000 7f960158f700 1 --2- 172.21.15.102:0/1810679815 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f95e806e200 0x7f95e80706c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f95fc074880 tx=0x7f95ec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:38.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.604+0000 7f95f2ffd700 1 -- 172.21.15.102:0/1810679815 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f95f8057350 con 0x7f95fc100dd0 2024-05-13T22:59:38.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.821+0000 7f9603ff4700 1 -- 172.21.15.102:0/1810679815 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f95fc02ccc0 con 0x7f95e806e200 2024-05-13T22:59:38.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.822+0000 7f95f2ffd700 1 -- 172.21.15.102:0/1810679815 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f95fc02ccc0 con 0x7f95e806e200 2024-05-13T22:59:38.798 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:59:38.798 INFO:teuthology.orchestra.run.smithi102.stdout: "target_image": null, 2024-05-13T22:59:38.798 INFO:teuthology.orchestra.run.smithi102.stdout: "in_progress": false, 2024-05-13T22:59:38.798 INFO:teuthology.orchestra.run.smithi102.stdout: "which": "", 2024-05-13T22:59:38.798 INFO:teuthology.orchestra.run.smithi102.stdout: "services_complete": [], 2024-05-13T22:59:38.798 INFO:teuthology.orchestra.run.smithi102.stdout: "progress": null, 2024-05-13T22:59:38.798 INFO:teuthology.orchestra.run.smithi102.stdout: "message": "", 2024-05-13T22:59:38.799 INFO:teuthology.orchestra.run.smithi102.stdout: "is_paused": false 2024-05-13T22:59:38.799 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:59:38.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.825+0000 7f9603ff4700 1 -- 172.21.15.102:0/1810679815 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f95e806e200 msgr2=0x7f95e80706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:38.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.825+0000 7f9603ff4700 1 --2- 172.21.15.102:0/1810679815 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f95e806e200 0x7f95e80706c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f95fc074880 tx=0x7f95ec01a040 comp rx=0 tx=0).stop 2024-05-13T22:59:38.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.825+0000 7f9603ff4700 1 -- 172.21.15.102:0/1810679815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95fc100dd0 msgr2=0x7f95fc073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:38.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.825+0000 7f9603ff4700 1 --2- 172.21.15.102:0/1810679815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95fc100dd0 0x7f95fc073090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f95f800d8d0 tx=0x7f95f800dbe0 comp rx=0 tx=0).stop 2024-05-13T22:59:38.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.825+0000 7f9603ff4700 1 -- 172.21.15.102:0/1810679815 shutdown_connections 2024-05-13T22:59:38.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.825+0000 7f9603ff4700 1 --2- 172.21.15.102:0/1810679815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95fc100dd0 0x7f95fc073090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:38.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.825+0000 7f9603ff4700 1 --2- 172.21.15.102:0/1810679815 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f95e806e200 0x7f95e80706c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:38.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.825+0000 7f9603ff4700 1 --2- 172.21.15.102:0/1810679815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f95fc101780 0x7f95fc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:38.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.825+0000 7f9603ff4700 1 -- 172.21.15.102:0/1810679815 >> 172.21.15.102:0/1810679815 conn(0x7f95fc0fc640 msgr2=0x7f95fc0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:38.801 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.825+0000 7f9603ff4700 1 -- 172.21.15.102:0/1810679815 shutdown_connections 2024-05-13T22:59:38.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:38.825+0000 7f9603ff4700 1 -- 172.21.15.102:0/1810679815 wait complete. 2024-05-13T22:59:39.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:39 smithi102 ceph-mon[111694]: pgmap v255: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:39.501 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph health detail' 2024-05-13T22:59:39.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:39 smithi130 ceph-mon[84265]: pgmap v255: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:39.709 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:59:40.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:40 smithi102 ceph-mon[111694]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:40.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:40 smithi130 ceph-mon[84265]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T22:59:41.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.289+0000 7f1385a8b700 1 -- 172.21.15.102:0/1432534318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1380101c70 msgr2=0x7f1380102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:41.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.289+0000 7f1385a8b700 1 --2- 172.21.15.102:0/1432534318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1380101c70 0x7f1380102050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f137000b0a0 tx=0x7f137000b3b0 comp rx=0 tx=0).stop 2024-05-13T22:59:41.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.289+0000 7f1385a8b700 1 -- 172.21.15.102:0/1432534318 shutdown_connections 2024-05-13T22:59:41.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.289+0000 7f1385a8b700 1 --2- 172.21.15.102:0/1432534318 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1380102620 0x7f138010ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.289+0000 7f1385a8b700 1 --2- 172.21.15.102:0/1432534318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1380101c70 0x7f1380102050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.289+0000 7f1385a8b700 1 -- 172.21.15.102:0/1432534318 >> 172.21.15.102:0/1432534318 conn(0x7f13800fb550 msgr2=0x7f13800fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.289+0000 7f1385a8b700 1 -- 172.21.15.102:0/1432534318 shutdown_connections 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.289+0000 7f1385a8b700 1 -- 172.21.15.102:0/1432534318 wait complete. 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.289+0000 7f1385a8b700 1 Processor -- start 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f1385a8b700 1 -- start start 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f1385a8b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1380101c70 0x7f138019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f1385a8b700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1380102620 0x7f138019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f1385a8b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138019d0c0 con 0x7f1380101c70 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f1385a8b700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1380196570 con 0x7f1380102620 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f137f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1380101c70 0x7f138019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f137f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1380101c70 0x7f138019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35250/0 (socket says 172.21.15.102:35250) 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f137f7fe700 1 -- 172.21.15.102:0/851738284 learned_addr learned my addr 172.21.15.102:0/851738284 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:41.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f137effd700 1 --2- 172.21.15.102:0/851738284 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1380102620 0x7f138019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:41.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f137f7fe700 1 -- 172.21.15.102:0/851738284 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1380102620 msgr2=0x7f138019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:41.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f137f7fe700 1 --2- 172.21.15.102:0/851738284 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1380102620 0x7f138019ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:41.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f137f7fe700 1 -- 172.21.15.102:0/851738284 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1370009d00 con 0x7f1380101c70 2024-05-13T22:59:41.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.290+0000 7f137f7fe700 1 --2- 172.21.15.102:0/851738284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1380101c70 0x7f138019c4f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1370000f80 tx=0x7f137000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:41.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.291+0000 7f137cff9700 1 -- 172.21.15.102:0/851738284 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f137000e040 con 0x7f1380101c70 2024-05-13T22:59:41.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.291+0000 7f137cff9700 1 -- 172.21.15.102:0/851738284 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1370003e60 con 0x7f1380101c70 2024-05-13T22:59:41.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.291+0000 7f137cff9700 1 -- 172.21.15.102:0/851738284 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f137001d910 con 0x7f1380101c70 2024-05-13T22:59:41.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.291+0000 7f1385a8b700 1 -- 172.21.15.102:0/851738284 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13801967f0 con 0x7f1380101c70 2024-05-13T22:59:41.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.291+0000 7f1385a8b700 1 -- 172.21.15.102:0/851738284 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1380196ce0 con 0x7f1380101c70 2024-05-13T22:59:41.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.292+0000 7f1385a8b700 1 -- 172.21.15.102:0/851738284 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1380108210 con 0x7f1380101c70 2024-05-13T22:59:41.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.292+0000 7f137cff9700 1 -- 172.21.15.102:0/851738284 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1370004820 con 0x7f1380101c70 2024-05-13T22:59:41.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.292+0000 7f137cff9700 1 --2- 172.21.15.102:0/851738284 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f136806e200 0x7f13680706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:41.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.293+0000 7f137cff9700 1 -- 172.21.15.102:0/851738284 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f137008fab0 con 0x7f1380101c70 2024-05-13T22:59:41.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.293+0000 7f137effd700 1 --2- 172.21.15.102:0/851738284 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f136806e200 0x7f13680706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:41.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.293+0000 7f137effd700 1 --2- 172.21.15.102:0/851738284 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f136806e200 0x7f13680706c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f13800fcc30 tx=0x7f1374005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:41.270 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:41 smithi102 ceph-mon[111694]: pgmap v256: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:59:41.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.298+0000 7f137cff9700 1 -- 172.21.15.102:0/851738284 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f137005c820 con 0x7f1380101c70 2024-05-13T22:59:41.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.573+0000 7f1385a8b700 1 -- 172.21.15.102:0/851738284 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1380197010 con 0x7f1380101c70 2024-05-13T22:59:41.549 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.573+0000 7f137cff9700 1 -- 172.21.15.102:0/851738284 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f137005c3b0 con 0x7f1380101c70 2024-05-13T22:59:41.549 INFO:teuthology.orchestra.run.smithi102.stdout:HEALTH_OK 2024-05-13T22:59:41.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.575+0000 7f1385a8b700 1 -- 172.21.15.102:0/851738284 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f136806e200 msgr2=0x7f13680706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:41.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.575+0000 7f1385a8b700 1 --2- 172.21.15.102:0/851738284 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f136806e200 0x7f13680706c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f13800fcc30 tx=0x7f1374005cf0 comp rx=0 tx=0).stop 2024-05-13T22:59:41.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.575+0000 7f1385a8b700 1 -- 172.21.15.102:0/851738284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1380101c70 msgr2=0x7f138019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:41.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.575+0000 7f1385a8b700 1 --2- 172.21.15.102:0/851738284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1380101c70 0x7f138019c4f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1370000f80 tx=0x7f137000cdd0 comp rx=0 tx=0).stop 2024-05-13T22:59:41.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.575+0000 7f1385a8b700 1 -- 172.21.15.102:0/851738284 shutdown_connections 2024-05-13T22:59:41.551 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.575+0000 7f1385a8b700 1 --2- 172.21.15.102:0/851738284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1380101c70 0x7f138019c4f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:41.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.575+0000 7f1385a8b700 1 --2- 172.21.15.102:0/851738284 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f136806e200 0x7f13680706c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:41.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.575+0000 7f1385a8b700 1 --2- 172.21.15.102:0/851738284 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1380102620 0x7f138019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:41.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.575+0000 7f1385a8b700 1 -- 172.21.15.102:0/851738284 >> 172.21.15.102:0/851738284 conn(0x7f13800fb550 msgr2=0x7f1380105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:41.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.576+0000 7f1385a8b700 1 -- 172.21.15.102:0/851738284 shutdown_connections 2024-05-13T22:59:41.552 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:41.576+0000 7f1385a8b700 1 -- 172.21.15.102:0/851738284 wait complete. 2024-05-13T22:59:41.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:41 smithi130 ceph-mon[84265]: pgmap v256: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:59:42.290 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:42 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/851738284' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:59:42.354 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph versions' 2024-05-13T22:59:42.562 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:59:42.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:42 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/851738284' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-13T22:59:43.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:43 smithi102 ceph-mon[111694]: pgmap v257: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:43.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:43 smithi130 ceph-mon[84265]: pgmap v257: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.367+0000 7f0c4da93700 1 -- 172.21.15.102:0/2321186964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c48101c70 msgr2=0x7f0c48102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.367+0000 7f0c4da93700 1 --2- 172.21.15.102:0/2321186964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c48101c70 0x7f0c48102050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009910 tx=0x7f0c38009c20 comp rx=0 tx=0).stop 2024-05-13T22:59:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.368+0000 7f0c4da93700 1 -- 172.21.15.102:0/2321186964 shutdown_connections 2024-05-13T22:59:44.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.368+0000 7f0c4da93700 1 --2- 172.21.15.102:0/2321186964 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c48102620 0x7f0c4810ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.368+0000 7f0c4da93700 1 --2- 172.21.15.102:0/2321186964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c48101c70 0x7f0c48102050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.368+0000 7f0c4da93700 1 -- 172.21.15.102:0/2321186964 >> 172.21.15.102:0/2321186964 conn(0x7f0c480fb550 msgr2=0x7f0c480fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.368+0000 7f0c4da93700 1 -- 172.21.15.102:0/2321186964 shutdown_connections 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.368+0000 7f0c4da93700 1 -- 172.21.15.102:0/2321186964 wait complete. 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.368+0000 7f0c4da93700 1 Processor -- start 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.368+0000 7f0c4da93700 1 -- start start 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c4da93700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c48101c70 0x7f0c4819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c4da93700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c48102620 0x7f0c4819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c4da93700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4819d1f0 con 0x7f0c48102620 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c4da93700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c481966a0 con 0x7f0c48101c70 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c46ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c48102620 0x7f0c4819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c46ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c48102620 0x7f0c4819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:35262/0 (socket says 172.21.15.102:35262) 2024-05-13T22:59:44.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c46ffd700 1 -- 172.21.15.102:0/2785115246 learned_addr learned my addr 172.21.15.102:0/2785115246 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T22:59:44.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c477fe700 1 --2- 172.21.15.102:0/2785115246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c48101c70 0x7f0c4819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:44.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c46ffd700 1 -- 172.21.15.102:0/2785115246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c48101c70 msgr2=0x7f0c4819c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:44.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c46ffd700 1 --2- 172.21.15.102:0/2785115246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c48101c70 0x7f0c4819c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:44.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c46ffd700 1 -- 172.21.15.102:0/2785115246 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c380095f0 con 0x7f0c48102620 2024-05-13T22:59:44.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c46ffd700 1 --2- 172.21.15.102:0/2785115246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c48102620 0x7f0c4819cb60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00b480 tx=0x7f0c3c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:44.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.369+0000 7f0c44ff9700 1 -- 172.21.15.102:0/2785115246 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c3c011800 con 0x7f0c48102620 2024-05-13T22:59:44.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.370+0000 7f0c44ff9700 1 -- 172.21.15.102:0/2785115246 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c3c011e20 con 0x7f0c48102620 2024-05-13T22:59:44.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.370+0000 7f0c4da93700 1 -- 172.21.15.102:0/2785115246 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c48196980 con 0x7f0c48102620 2024-05-13T22:59:44.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.370+0000 7f0c44ff9700 1 -- 172.21.15.102:0/2785115246 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c3c010ea0 con 0x7f0c48102620 2024-05-13T22:59:44.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.370+0000 7f0c4da93700 1 -- 172.21.15.102:0/2785115246 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c48196ed0 con 0x7f0c48102620 2024-05-13T22:59:44.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.371+0000 7f0c4da93700 1 -- 172.21.15.102:0/2785115246 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c48108210 con 0x7f0c48102620 2024-05-13T22:59:44.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.371+0000 7f0c44ff9700 1 -- 172.21.15.102:0/2785115246 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0c3c010380 con 0x7f0c48102620 2024-05-13T22:59:44.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.371+0000 7f0c44ff9700 1 --2- 172.21.15.102:0/2785115246 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0c3006e200 0x7f0c300706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T22:59:44.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.372+0000 7f0c477fe700 1 --2- 172.21.15.102:0/2785115246 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0c3006e200 0x7f0c300706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T22:59:44.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.372+0000 7f0c44ff9700 1 -- 172.21.15.102:0/2785115246 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0c3c08ddb0 con 0x7f0c48102620 2024-05-13T22:59:44.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.372+0000 7f0c477fe700 1 --2- 172.21.15.102:0/2785115246 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0c3006e200 0x7f0c300706c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c38006010 tx=0x7f0c3801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T22:59:44.352 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.376+0000 7f0c44ff9700 1 -- 172.21.15.102:0/2785115246 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0c3c05aaa0 con 0x7f0c48102620 2024-05-13T22:59:44.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.655+0000 7f0c4da93700 1 -- 172.21.15.102:0/2785115246 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0c4804ea90 con 0x7f0c48102620 2024-05-13T22:59:44.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.656+0000 7f0c44ff9700 1 -- 172.21.15.102:0/2785115246 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0c3c05a630 con 0x7f0c48102620 2024-05-13T22:59:44.632 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T22:59:44.632 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T22:59:44.632 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:59:44.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:59:44.632 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T22:59:44.632 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T22:59:44.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:59:44.632 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T22:59:44.632 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 6 2024-05-13T22:59:44.632 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:59:44.633 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T22:59:44.633 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 4 2024-05-13T22:59:44.633 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T22:59:44.633 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T22:59:44.633 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 14 2024-05-13T22:59:44.633 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T22:59:44.633 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T22:59:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.658+0000 7f0c4da93700 1 -- 172.21.15.102:0/2785115246 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0c3006e200 msgr2=0x7f0c300706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.658+0000 7f0c4da93700 1 --2- 172.21.15.102:0/2785115246 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0c3006e200 0x7f0c300706c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0c38006010 tx=0x7f0c3801a040 comp rx=0 tx=0).stop 2024-05-13T22:59:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.658+0000 7f0c4da93700 1 -- 172.21.15.102:0/2785115246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c48102620 msgr2=0x7f0c4819cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T22:59:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.658+0000 7f0c4da93700 1 --2- 172.21.15.102:0/2785115246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c48102620 0x7f0c4819cb60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00b480 tx=0x7f0c3c00b790 comp rx=0 tx=0).stop 2024-05-13T22:59:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.658+0000 7f0c4da93700 1 -- 172.21.15.102:0/2785115246 shutdown_connections 2024-05-13T22:59:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.658+0000 7f0c4da93700 1 --2- 172.21.15.102:0/2785115246 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0c3006e200 0x7f0c300706c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.658+0000 7f0c4da93700 1 --2- 172.21.15.102:0/2785115246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c48101c70 0x7f0c4819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.658+0000 7f0c4da93700 1 --2- 172.21.15.102:0/2785115246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c48102620 0x7f0c4819cb60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T22:59:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.658+0000 7f0c4da93700 1 -- 172.21.15.102:0/2785115246 >> 172.21.15.102:0/2785115246 conn(0x7f0c480fb550 msgr2=0x7f0c48105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T22:59:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.658+0000 7f0c4da93700 1 -- 172.21.15.102:0/2785115246 shutdown_connections 2024-05-13T22:59:44.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T22:59:44.658+0000 7f0c4da93700 1 -- 172.21.15.102:0/2785115246 wait complete. 2024-05-13T22:59:45.339 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:45 smithi102 ceph-mon[111694]: pgmap v258: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:45.339 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:45 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2785115246' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:45.361 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-05-13T22:59:45.567 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:59:45.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:45 smithi130 ceph-mon[84265]: pgmap v258: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:45.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:45 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2785115246' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T22:59:46.363 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:59:46.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T22:59:47.250 INFO:teuthology.orchestra.run.smithi102.stdout:wait for servicemap items w/ changing names to refresh 2024-05-13T22:59:47.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:47 smithi102 ceph-mon[111694]: pgmap v259: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:47.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:47 smithi130 ceph-mon[84265]: pgmap v259: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:47.891 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'sleep 60' 2024-05-13T22:59:48.097 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T22:59:49.604 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:49 smithi102 ceph-mon[111694]: pgmap v260: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:49.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:49 smithi130 ceph-mon[84265]: pgmap v260: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:51.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:51 smithi130 ceph-mon[84265]: pgmap v261: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:59:51.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:51 smithi102 ceph-mon[111694]: pgmap v261: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T22:59:53.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:53 smithi130 ceph-mon[84265]: pgmap v262: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:53.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:53 smithi102 ceph-mon[111694]: pgmap v262: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:55.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:55 smithi102 ceph-mon[111694]: pgmap v263: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:55.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:55 smithi130 ceph-mon[84265]: pgmap v263: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:57.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:57 smithi130 ceph-mon[84265]: pgmap v264: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:57.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:57 smithi102 ceph-mon[111694]: pgmap v264: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T22:59:59.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 22:59:59 smithi130 ceph-mon[84265]: pgmap v265: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T22:59:59.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 22:59:59 smithi102 ceph-mon[111694]: pgmap v265: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:00.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:00 smithi102 ceph-mon[111694]: overall HEALTH_OK 2024-05-13T23:00:00.508 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:00 smithi130 ceph-mon[84265]: overall HEALTH_OK 2024-05-13T23:00:01.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:01 smithi130 ceph-mon[84265]: pgmap v266: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:01.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:00:01.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:01 smithi102 ceph-mon[111694]: pgmap v266: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:01.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:00:02.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:02 smithi130 ceph-mon[84265]: pgmap v267: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:02.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:02 smithi102 ceph-mon[111694]: pgmap v267: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:05.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:05 smithi130 ceph-mon[84265]: pgmap v268: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:05.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:05 smithi102 ceph-mon[111694]: pgmap v268: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:07.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:07 smithi130 ceph-mon[84265]: pgmap v269: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:07.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:07 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:00:07.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:07 smithi102 ceph-mon[111694]: pgmap v269: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:07.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:07 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:00:08.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:08 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:00:08.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:08 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:00:08.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:08 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:00:08.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:08 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:00:08.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:08 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:00:08.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:08 smithi130 ceph-mon[84265]: pgmap v270: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:08.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:08 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:00:08.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:08 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:00:08.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:08 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:00:08.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:08 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:00:08.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:08 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:00:08.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:08 smithi102 ceph-mon[111694]: pgmap v270: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:11.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:11 smithi130 ceph-mon[84265]: pgmap v271: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:11.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:11 smithi102 ceph-mon[111694]: pgmap v271: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:13.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:13 smithi130 ceph-mon[84265]: pgmap v272: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:13.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:13 smithi102 ceph-mon[111694]: pgmap v272: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:15.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:15 smithi130 ceph-mon[84265]: pgmap v273: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:15.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:15 smithi102 ceph-mon[111694]: pgmap v273: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:16.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:00:16.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:00:17.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:17 smithi102 ceph-mon[111694]: pgmap v274: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:17.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:17 smithi130 ceph-mon[84265]: pgmap v274: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:19.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:19 smithi102 ceph-mon[111694]: pgmap v275: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:19.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:19 smithi130 ceph-mon[84265]: pgmap v275: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:21.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:21 smithi102 ceph-mon[111694]: pgmap v276: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:21.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:21 smithi130 ceph-mon[84265]: pgmap v276: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:23.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:23 smithi102 ceph-mon[111694]: pgmap v277: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:23.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:23 smithi130 ceph-mon[84265]: pgmap v277: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:25.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:25 smithi102 ceph-mon[111694]: pgmap v278: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:25.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:25 smithi130 ceph-mon[84265]: pgmap v278: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:27.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:27 smithi102 ceph-mon[111694]: pgmap v279: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:27.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:27 smithi130 ceph-mon[84265]: pgmap v279: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:29.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:29 smithi130 ceph-mon[84265]: pgmap v280: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:29.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:29 smithi102 ceph-mon[111694]: pgmap v280: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:31.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:31 smithi130 ceph-mon[84265]: pgmap v281: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:31.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:00:31.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:31 smithi102 ceph-mon[111694]: pgmap v281: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:31.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:00:33.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:33 smithi130 ceph-mon[84265]: pgmap v282: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:33.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:33 smithi102 ceph-mon[111694]: pgmap v282: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:35.553 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:35 smithi102 ceph-mon[111694]: pgmap v283: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:35.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:35 smithi130 ceph-mon[84265]: pgmap v283: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:37.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:37 smithi130 ceph-mon[84265]: pgmap v284: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:37.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:37 smithi102 ceph-mon[111694]: pgmap v284: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:39.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:39 smithi130 ceph-mon[84265]: pgmap v285: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:39.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:39 smithi102 ceph-mon[111694]: pgmap v285: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:41.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:41 smithi130 ceph-mon[84265]: pgmap v286: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:41.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:41 smithi102 ceph-mon[111694]: pgmap v286: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:43.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:43 smithi130 ceph-mon[84265]: pgmap v287: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:43.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:43 smithi102 ceph-mon[111694]: pgmap v287: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:45.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:45 smithi130 ceph-mon[84265]: pgmap v288: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:45.666 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:45 smithi102 ceph-mon[111694]: pgmap v288: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:46.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:00:46.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:00:47.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:47 smithi130 ceph-mon[84265]: pgmap v289: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:47.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:47 smithi102 ceph-mon[111694]: pgmap v289: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:49.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:49 smithi130 ceph-mon[84265]: pgmap v290: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:49.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:49 smithi102 ceph-mon[111694]: pgmap v290: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:50.623 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph orch ps' 2024-05-13T23:00:50.843 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T23:00:51.281 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:51 smithi102 ceph-mon[111694]: pgmap v291: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:51.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:51 smithi130 ceph-mon[84265]: pgmap v291: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:00:52.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.407+0000 7f777c6e6700 1 -- 172.21.15.102:0/3428425806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7774100df0 msgr2=0x7f77741011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:52.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.407+0000 7f777c6e6700 1 --2- 172.21.15.102:0/3428425806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7774100df0 0x7f77741011d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7764009910 tx=0x7f7764009c20 comp rx=0 tx=0).stop 2024-05-13T23:00:52.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.407+0000 7f777c6e6700 1 -- 172.21.15.102:0/3428425806 shutdown_connections 2024-05-13T23:00:52.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.407+0000 7f777c6e6700 1 --2- 172.21.15.102:0/3428425806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77741017a0 0x7f77741057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:52.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.407+0000 7f777c6e6700 1 --2- 172.21.15.102:0/3428425806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7774100df0 0x7f77741011d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:52.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.407+0000 7f777c6e6700 1 -- 172.21.15.102:0/3428425806 >> 172.21.15.102:0/3428425806 conn(0x7f77740fc6a0 msgr2=0x7f77740feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:00:52.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.407+0000 7f777c6e6700 1 -- 172.21.15.102:0/3428425806 shutdown_connections 2024-05-13T23:00:52.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.407+0000 7f777c6e6700 1 -- 172.21.15.102:0/3428425806 wait complete. 2024-05-13T23:00:52.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.408+0000 7f777c6e6700 1 Processor -- start 2024-05-13T23:00:52.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.408+0000 7f777c6e6700 1 -- start start 2024-05-13T23:00:52.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.408+0000 7f777c6e6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7774100df0 0x7f777419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.408+0000 7f777c6e6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77741017a0 0x7f777419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.408+0000 7f777c6e6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777419d100 con 0x7f7774100df0 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.408+0000 7f777c6e6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77741965b0 con 0x7f77741017a0 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f777a482700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7774100df0 0x7f777419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f777a482700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7774100df0 0x7f777419c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57396/0 (socket says 172.21.15.102:57396) 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f777a482700 1 -- 172.21.15.102:0/1702233213 learned_addr learned my addr 172.21.15.102:0/1702233213 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f7779c81700 1 --2- 172.21.15.102:0/1702233213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77741017a0 0x7f777419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f777a482700 1 -- 172.21.15.102:0/1702233213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77741017a0 msgr2=0x7f777419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f777a482700 1 --2- 172.21.15.102:0/1702233213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77741017a0 0x7f777419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f777a482700 1 -- 172.21.15.102:0/1702233213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77640095f0 con 0x7f7774100df0 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f777a482700 1 --2- 172.21.15.102:0/1702233213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7774100df0 0x7f777419c530 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7764006010 tx=0x7f776400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f776b7fe700 1 -- 172.21.15.102:0/1702233213 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f776401d070 con 0x7f7774100df0 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f776b7fe700 1 -- 172.21.15.102:0/1702233213 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7764005dc0 con 0x7f7774100df0 2024-05-13T23:00:52.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f777c6e6700 1 -- 172.21.15.102:0/1702233213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7774196830 con 0x7f7774100df0 2024-05-13T23:00:52.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f776b7fe700 1 -- 172.21.15.102:0/1702233213 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7764017400 con 0x7f7774100df0 2024-05-13T23:00:52.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.409+0000 7f777c6e6700 1 -- 172.21.15.102:0/1702233213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7774196d20 con 0x7f7774100df0 2024-05-13T23:00:52.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.410+0000 7f777c6e6700 1 -- 172.21.15.102:0/1702233213 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7774109190 con 0x7f7774100df0 2024-05-13T23:00:52.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.411+0000 7f776b7fe700 1 -- 172.21.15.102:0/1702233213 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7764003a60 con 0x7f7774100df0 2024-05-13T23:00:52.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.411+0000 7f776b7fe700 1 --2- 172.21.15.102:0/1702233213 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f776006e200 0x7f77600706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:00:52.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.411+0000 7f776b7fe700 1 -- 172.21.15.102:0/1702233213 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f776408f670 con 0x7f7774100df0 2024-05-13T23:00:52.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.411+0000 7f7779c81700 1 --2- 172.21.15.102:0/1702233213 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f776006e200 0x7f77600706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:00:52.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.412+0000 7f7779c81700 1 --2- 172.21.15.102:0/1702233213 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f776006e200 0x7f77600706c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f77740fce50 tx=0x7f777000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:00:52.391 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.416+0000 7f776b7fe700 1 -- 172.21.15.102:0/1702233213 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f776405c3e0 con 0x7f7774100df0 2024-05-13T23:00:52.597 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.621+0000 7f777c6e6700 1 -- 172.21.15.102:0/1702233213 --> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7774197050 con 0x7f776006e200 2024-05-13T23:00:52.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.632+0000 7f776b7fe700 1 -- 172.21.15.102:0/1702233213 <== mgr.34104 v2:172.21.15.102:6800/2973074092 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f7774197050 con 0x7f776006e200 2024-05-13T23:00:52.607 INFO:teuthology.orchestra.run.smithi102.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-13T23:00:52.607 INFO:teuthology.orchestra.run.smithi102.stdout:alertmanager.smithi102 smithi102 *:9093,9094 running (10m) 112s ago 21m 26.0M - 0.25.0 c8568f914cd2 ec6713474e98 2024-05-13T23:00:52.607 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi102 smithi102 running (2m) 112s ago 21m 7936k - 19.0.0-2565-gc3e55145 b1b56bed7c91 bfcb10ba7c2a 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-exporter.smithi130 smithi130 running (118s) 115s ago 20m 7004k - 19.0.0-2565-gc3e55145 b1b56bed7c91 035d93e33eb8 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi102 smithi102 running (7m) 112s ago 21m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 b85ad27f9f53 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:crash.smithi130 smithi130 running (7m) 115s ago 20m 6656k - 19.0.0-2565-gc3e55145 b1b56bed7c91 1c56e8aae46e 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:grafana.smithi102 smithi102 *:3000 running (9m) 112s ago 20m 79.9M - 9.4.12 2bacad6d85d8 f584c1b9106e 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.cgkcwz smithi102 running (3m) 112s ago 15m 89.5M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1a34f8c7d234 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi102.sdylgv smithi102 running (3m) 112s ago 15m 110M - 19.0.0-2565-gc3e55145 b1b56bed7c91 2b50dedff13c 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.hzjanp smithi130 running (2m) 115s ago 15m 13.7M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1a45ebbee664 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:mds.cephfs.smithi130.ovlxvg smithi130 running (2m) 115s ago 15m 15.7M - 19.0.0-2565-gc3e55145 b1b56bed7c91 1d727b21600b 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi102.xcijrj smithi102 *:8443,9283,8765 running (12m) 112s ago 22m 513M - 19.0.0-2565-gc3e55145 b1b56bed7c91 3059b8f9f17e 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:mgr.smithi130.uoqoxl smithi130 *:8443,9283,8765 running (11m) 115s ago 20m 454M - 19.0.0-2565-gc3e55145 b1b56bed7c91 9f999099242a 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi102 smithi102 running (8m) 112s ago 22m 60.6M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 0b15816f3f6b 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:mon.smithi130 smithi130 running (7m) 115s ago 19m 49.2M 2048M 19.0.0-2565-gc3e55145 b1b56bed7c91 2c1e5923909a 2024-05-13T23:00:52.608 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi102 smithi102 *:9100 running (10m) 112s ago 21m 16.2M - 1.5.0 0da6a335fe13 4af9b9baf492 2024-05-13T23:00:52.609 INFO:teuthology.orchestra.run.smithi102.stdout:node-exporter.smithi130 smithi130 *:9100 running (10m) 115s ago 20m 17.1M - 1.5.0 0da6a335fe13 f7a2941ca8bd 2024-05-13T23:00:52.609 INFO:teuthology.orchestra.run.smithi102.stdout:osd.0 smithi102 running (6m) 112s ago 19m 142M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 0ada88c777a2 2024-05-13T23:00:52.609 INFO:teuthology.orchestra.run.smithi102.stdout:osd.1 smithi102 running (6m) 112s ago 18m 119M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 c3e9e68b06fd 2024-05-13T23:00:52.609 INFO:teuthology.orchestra.run.smithi102.stdout:osd.2 smithi102 running (5m) 112s ago 18m 106M 1216M 19.0.0-2565-gc3e55145 b1b56bed7c91 1b85728dccf5 2024-05-13T23:00:52.609 INFO:teuthology.orchestra.run.smithi102.stdout:osd.3 smithi130 running (4m) 115s ago 17m 110M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 66e804d4b801 2024-05-13T23:00:52.609 INFO:teuthology.orchestra.run.smithi102.stdout:osd.4 smithi130 running (4m) 115s ago 17m 101M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5bb5a8f94042 2024-05-13T23:00:52.609 INFO:teuthology.orchestra.run.smithi102.stdout:osd.5 smithi130 running (3m) 115s ago 17m 102M 2240M 19.0.0-2565-gc3e55145 b1b56bed7c91 5d983135967d 2024-05-13T23:00:52.609 INFO:teuthology.orchestra.run.smithi102.stdout:prometheus.smithi102 smithi102 *:9095 running (10m) 112s ago 20m 80.2M - 2.43.0 a07b618ecd1d b8236065e02b 2024-05-13T23:00:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.634+0000 7f777c6e6700 1 -- 172.21.15.102:0/1702233213 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f776006e200 msgr2=0x7f77600706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.634+0000 7f777c6e6700 1 --2- 172.21.15.102:0/1702233213 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f776006e200 0x7f77600706c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f77740fce50 tx=0x7f777000b410 comp rx=0 tx=0).stop 2024-05-13T23:00:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.634+0000 7f777c6e6700 1 -- 172.21.15.102:0/1702233213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7774100df0 msgr2=0x7f777419c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.634+0000 7f777c6e6700 1 --2- 172.21.15.102:0/1702233213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7774100df0 0x7f777419c530 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7764006010 tx=0x7f776400bd70 comp rx=0 tx=0).stop 2024-05-13T23:00:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.634+0000 7f777c6e6700 1 -- 172.21.15.102:0/1702233213 shutdown_connections 2024-05-13T23:00:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.634+0000 7f777c6e6700 1 --2- 172.21.15.102:0/1702233213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7774100df0 0x7f777419c530 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:52.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.634+0000 7f777c6e6700 1 --2- 172.21.15.102:0/1702233213 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f776006e200 0x7f77600706c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:52.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.634+0000 7f777c6e6700 1 --2- 172.21.15.102:0/1702233213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77741017a0 0x7f777419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:52.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.634+0000 7f777c6e6700 1 -- 172.21.15.102:0/1702233213 >> 172.21.15.102:0/1702233213 conn(0x7f77740fc6a0 msgr2=0x7f77740fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:00:52.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.634+0000 7f777c6e6700 1 -- 172.21.15.102:0/1702233213 shutdown_connections 2024-05-13T23:00:52.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:52.634+0000 7f777c6e6700 1 -- 172.21.15.102:0/1702233213 wait complete. 2024-05-13T23:00:53.224 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph versions' 2024-05-13T23:00:53.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:53 smithi102 ceph-mon[111694]: pgmap v292: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:53.437 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T23:00:53.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:53 smithi130 ceph-mon[84265]: pgmap v292: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:54.534 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:54 smithi102 ceph-mon[111694]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T23:00:54.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:54 smithi130 ceph-mon[84265]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-13T23:00:55.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.030+0000 7f5a1e933700 1 -- 172.21.15.102:0/3246643909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a180ffee0 msgr2=0x7f5a18105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:55.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.030+0000 7f5a1e933700 1 --2- 172.21.15.102:0/3246643909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a180ffee0 0x7f5a18105870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009910 tx=0x7f5a0c009c20 comp rx=0 tx=0).stop 2024-05-13T23:00:55.007 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.030+0000 7f5a1e933700 1 -- 172.21.15.102:0/3246643909 shutdown_connections 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.030+0000 7f5a1e933700 1 --2- 172.21.15.102:0/3246643909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a180ffee0 0x7f5a18105870 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.030+0000 7f5a1e933700 1 --2- 172.21.15.102:0/3246643909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a180ff530 0x7f5a180ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.030+0000 7f5a1e933700 1 -- 172.21.15.102:0/3246643909 >> 172.21.15.102:0/3246643909 conn(0x7f5a18074df0 msgr2=0x7f5a18075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.030+0000 7f5a1e933700 1 -- 172.21.15.102:0/3246643909 shutdown_connections 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.030+0000 7f5a1e933700 1 -- 172.21.15.102:0/3246643909 wait complete. 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.030+0000 7f5a1e933700 1 Processor -- start 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.030+0000 7f5a1e933700 1 -- start start 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a1e933700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a180ff530 0x7f5a1819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a1e933700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a180ffee0 0x7f5a1819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a1e933700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1819d2a0 con 0x7f5a180ff530 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a1e933700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a18196750 con 0x7f5a180ffee0 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a17fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a180ff530 0x7f5a1819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a17fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a180ff530 0x7f5a1819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57424/0 (socket says 172.21.15.102:57424) 2024-05-13T23:00:55.008 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a17fff700 1 -- 172.21.15.102:0/1766953804 learned_addr learned my addr 172.21.15.102:0/1766953804 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:00:55.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a177fe700 1 --2- 172.21.15.102:0/1766953804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a180ffee0 0x7f5a1819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:00:55.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a17fff700 1 -- 172.21.15.102:0/1766953804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a180ffee0 msgr2=0x7f5a1819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:55.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a17fff700 1 --2- 172.21.15.102:0/1766953804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a180ffee0 0x7f5a1819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:55.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a17fff700 1 -- 172.21.15.102:0/1766953804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c0095f0 con 0x7f5a180ff530 2024-05-13T23:00:55.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.031+0000 7f5a17fff700 1 --2- 172.21.15.102:0/1766953804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a180ff530 0x7f5a1819c6d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800b4b0 tx=0x7f5a0800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:00:55.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.032+0000 7f5a157fa700 1 -- 172.21.15.102:0/1766953804 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a08011800 con 0x7f5a180ff530 2024-05-13T23:00:55.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.032+0000 7f5a157fa700 1 -- 172.21.15.102:0/1766953804 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a08011e20 con 0x7f5a180ff530 2024-05-13T23:00:55.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.032+0000 7f5a1e933700 1 -- 172.21.15.102:0/1766953804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a18196a30 con 0x7f5a180ff530 2024-05-13T23:00:55.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.032+0000 7f5a157fa700 1 -- 172.21.15.102:0/1766953804 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a0800f2e0 con 0x7f5a180ff530 2024-05-13T23:00:55.009 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.032+0000 7f5a1e933700 1 -- 172.21.15.102:0/1766953804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a18196f80 con 0x7f5a180ff530 2024-05-13T23:00:55.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.033+0000 7f5a1e933700 1 -- 172.21.15.102:0/1766953804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a18109170 con 0x7f5a180ff530 2024-05-13T23:00:55.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.033+0000 7f5a157fa700 1 -- 172.21.15.102:0/1766953804 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5a08010380 con 0x7f5a180ff530 2024-05-13T23:00:55.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.033+0000 7f5a157fa700 1 --2- 172.21.15.102:0/1766953804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f5a0006e200 0x7f5a000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:00:55.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.034+0000 7f5a157fa700 1 -- 172.21.15.102:0/1766953804 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5a0808ddf0 con 0x7f5a180ff530 2024-05-13T23:00:55.010 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.034+0000 7f5a177fe700 1 --2- 172.21.15.102:0/1766953804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f5a0006e200 0x7f5a000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:00:55.011 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.034+0000 7f5a177fe700 1 --2- 172.21.15.102:0/1766953804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f5a0006e200 0x7f5a000706c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5a18197bc0 tx=0x7f5a0c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:00:55.013 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.038+0000 7f5a157fa700 1 -- 172.21.15.102:0/1766953804 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f5a0805aae0 con 0x7f5a180ff530 2024-05-13T23:00:55.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.322+0000 7f5a1e933700 1 -- 172.21.15.102:0/1766953804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5a18197250 con 0x7f5a180ff530 2024-05-13T23:00:55.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.323+0000 7f5a157fa700 1 -- 172.21.15.102:0/1766953804 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5a0805a670 con 0x7f5a180ff530 2024-05-13T23:00:55.298 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-05-13T23:00:55.298 INFO:teuthology.orchestra.run.smithi102.stdout: "mon": { 2024-05-13T23:00:55.298 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: "mgr": { 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 2 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: "osd": { 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 6 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 4 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: "overall": { 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)": 14 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-05-13T23:00:55.299 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-05-13T23:00:55.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.325+0000 7f5a1e933700 1 -- 172.21.15.102:0/1766953804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f5a0006e200 msgr2=0x7f5a000706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:55.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.325+0000 7f5a1e933700 1 --2- 172.21.15.102:0/1766953804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f5a0006e200 0x7f5a000706c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5a18197bc0 tx=0x7f5a0c01a040 comp rx=0 tx=0).stop 2024-05-13T23:00:55.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.325+0000 7f5a1e933700 1 -- 172.21.15.102:0/1766953804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a180ff530 msgr2=0x7f5a1819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:55.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.325+0000 7f5a1e933700 1 --2- 172.21.15.102:0/1766953804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a180ff530 0x7f5a1819c6d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800b4b0 tx=0x7f5a0800b7c0 comp rx=0 tx=0).stop 2024-05-13T23:00:55.300 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.325+0000 7f5a1e933700 1 -- 172.21.15.102:0/1766953804 shutdown_connections 2024-05-13T23:00:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.325+0000 7f5a1e933700 1 --2- 172.21.15.102:0/1766953804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a180ff530 0x7f5a1819c6d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.325+0000 7f5a1e933700 1 --2- 172.21.15.102:0/1766953804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f5a0006e200 0x7f5a000706c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.325+0000 7f5a1e933700 1 --2- 172.21.15.102:0/1766953804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a180ffee0 0x7f5a1819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.325+0000 7f5a1e933700 1 -- 172.21.15.102:0/1766953804 >> 172.21.15.102:0/1766953804 conn(0x7f5a18074df0 msgr2=0x7f5a180fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:00:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.325+0000 7f5a1e933700 1 -- 172.21.15.102:0/1766953804 shutdown_connections 2024-05-13T23:00:55.301 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:55.325+0000 7f5a1e933700 1 -- 172.21.15.102:0/1766953804 wait complete. 2024-05-13T23:00:55.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:55 smithi130 ceph-mon[84265]: pgmap v293: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:55.657 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:55 smithi102 ceph-mon[111694]: pgmap v293: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:55.983 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-05-13T23:00:56.194 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T23:00:56.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:56 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1766953804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T23:00:56.527 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:56 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1766953804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T23:00:57.557 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:57 smithi102 ceph-mon[111694]: pgmap v294: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:57.567 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:57 smithi130 ceph-mon[84265]: pgmap v294: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:00:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.818+0000 7f75be4da700 1 -- 172.21.15.102:0/473071905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b8105a40 msgr2=0x7f75b8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.818+0000 7f75be4da700 1 --2- 172.21.15.102:0/473071905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b8105a40 0x7f75b8105e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009910 tx=0x7f75a0009c20 comp rx=0 tx=0).stop 2024-05-13T23:00:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.818+0000 7f75be4da700 1 -- 172.21.15.102:0/473071905 shutdown_connections 2024-05-13T23:00:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.818+0000 7f75be4da700 1 --2- 172.21.15.102:0/473071905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b80ffa40 0x7f75b80ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:57.795 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.818+0000 7f75be4da700 1 --2- 172.21.15.102:0/473071905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b8105a40 0x7f75b8105e20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.818+0000 7f75be4da700 1 -- 172.21.15.102:0/473071905 >> 172.21.15.102:0/473071905 conn(0x7f75b80fb570 msgr2=0x7f75b80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.818+0000 7f75be4da700 1 -- 172.21.15.102:0/473071905 shutdown_connections 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.818+0000 7f75be4da700 1 -- 172.21.15.102:0/473071905 wait complete. 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.819+0000 7f75be4da700 1 Processor -- start 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.819+0000 7f75be4da700 1 -- start start 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.819+0000 7f75be4da700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b80ffa40 0x7f75b819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75be4da700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b8105a40 0x7f75b819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75be4da700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b819d110 con 0x7f75b80ffa40 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75be4da700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b81965c0 con 0x7f75b8105a40 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75b7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b80ffa40 0x7f75b819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75b7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b80ffa40 0x7f75b819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57442/0 (socket says 172.21.15.102:57442) 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75b7fff700 1 -- 172.21.15.102:0/1476894325 learned_addr learned my addr 172.21.15.102:0/1476894325 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:00:57.796 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75b77fe700 1 --2- 172.21.15.102:0/1476894325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b8105a40 0x7f75b819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:00:57.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75b7fff700 1 -- 172.21.15.102:0/1476894325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b8105a40 msgr2=0x7f75b819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:57.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75b7fff700 1 --2- 172.21.15.102:0/1476894325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b8105a40 0x7f75b819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:57.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75b7fff700 1 -- 172.21.15.102:0/1476894325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00095f0 con 0x7f75b80ffa40 2024-05-13T23:00:57.797 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75b7fff700 1 --2- 172.21.15.102:0/1476894325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b80ffa40 0x7f75b819c540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f75a0006010 tx=0x7f75a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:00:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75b57fa700 1 -- 172.21.15.102:0/1476894325 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75a001d070 con 0x7f75b80ffa40 2024-05-13T23:00:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75b57fa700 1 -- 172.21.15.102:0/1476894325 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75a0005dc0 con 0x7f75b80ffa40 2024-05-13T23:00:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75be4da700 1 -- 172.21.15.102:0/1476894325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b8196840 con 0x7f75b80ffa40 2024-05-13T23:00:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75b57fa700 1 -- 172.21.15.102:0/1476894325 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75a0017400 con 0x7f75b80ffa40 2024-05-13T23:00:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.820+0000 7f75be4da700 1 -- 172.21.15.102:0/1476894325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b8196d30 con 0x7f75b80ffa40 2024-05-13T23:00:57.798 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.821+0000 7f75be4da700 1 -- 172.21.15.102:0/1476894325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b8108040 con 0x7f75b80ffa40 2024-05-13T23:00:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.822+0000 7f75b57fa700 1 -- 172.21.15.102:0/1476894325 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f75a0003a60 con 0x7f75b80ffa40 2024-05-13T23:00:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.822+0000 7f75b57fa700 1 --2- 172.21.15.102:0/1476894325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f75a406e200 0x7f75a40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:00:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.822+0000 7f75b57fa700 1 -- 172.21.15.102:0/1476894325 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f75a008f720 con 0x7f75b80ffa40 2024-05-13T23:00:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.822+0000 7f75b77fe700 1 --2- 172.21.15.102:0/1476894325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f75a406e200 0x7f75a40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:00:57.799 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.823+0000 7f75b77fe700 1 --2- 172.21.15.102:0/1476894325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f75a406e200 0x7f75a40706c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f75b80fbb50 tx=0x7f75a80073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:00:57.802 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:57.827+0000 7f75b57fa700 1 -- 172.21.15.102:0/1476894325 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f75a005c410 con 0x7f75b80ffa40 2024-05-13T23:00:58.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.109+0000 7f75be4da700 1 -- 172.21.15.102:0/1476894325 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f75b804ea90 con 0x7f75b80ffa40 2024-05-13T23:00:58.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.109+0000 7f75b57fa700 1 -- 172.21.15.102:0/1476894325 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f75a005bfa0 con 0x7f75b80ffa40 2024-05-13T23:00:58.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.111+0000 7f75be4da700 1 -- 172.21.15.102:0/1476894325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f75a406e200 msgr2=0x7f75a40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:58.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.111+0000 7f75be4da700 1 --2- 172.21.15.102:0/1476894325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f75a406e200 0x7f75a40706c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f75b80fbb50 tx=0x7f75a80073d0 comp rx=0 tx=0).stop 2024-05-13T23:00:58.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.111+0000 7f75be4da700 1 -- 172.21.15.102:0/1476894325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b80ffa40 msgr2=0x7f75b819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:00:58.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.111+0000 7f75be4da700 1 --2- 172.21.15.102:0/1476894325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b80ffa40 0x7f75b819c540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f75a0006010 tx=0x7f75a000bd70 comp rx=0 tx=0).stop 2024-05-13T23:00:58.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.112+0000 7f75be4da700 1 -- 172.21.15.102:0/1476894325 shutdown_connections 2024-05-13T23:00:58.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.112+0000 7f75be4da700 1 --2- 172.21.15.102:0/1476894325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75b80ffa40 0x7f75b819c540 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:58.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.112+0000 7f75be4da700 1 --2- 172.21.15.102:0/1476894325 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f75a406e200 0x7f75a40706c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:58.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.112+0000 7f75be4da700 1 --2- 172.21.15.102:0/1476894325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75b8105a40 0x7f75b819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:00:58.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.112+0000 7f75be4da700 1 -- 172.21.15.102:0/1476894325 >> 172.21.15.102:0/1476894325 conn(0x7f75b80fb570 msgr2=0x7f75b80fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:00:58.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.112+0000 7f75be4da700 1 -- 172.21.15.102:0/1476894325 shutdown_connections 2024-05-13T23:00:58.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:00:58.112+0000 7f75be4da700 1 -- 172.21.15.102:0/1476894325 wait complete. 2024-05-13T23:00:58.097 INFO:teuthology.orchestra.run.smithi102.stdout:true 2024-05-13T23:00:58.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:58 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1476894325' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T23:00:58.489 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:58 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1476894325' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T23:00:58.719 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -e sha1=c3e55145f4f4b3a500bb66ee08c938a215bc231d -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-05-13T23:00:58.925 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T23:00:59.342 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:00:59 smithi102 ceph-mon[111694]: pgmap v295: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:00:59.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:00:59 smithi130 ceph-mon[84265]: pgmap v295: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:00.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.501+0000 7f715f27c700 1 -- 172.21.15.102:0/1818425720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71580ffeb0 msgr2=0x7f7158105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:00.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.501+0000 7f715f27c700 1 --2- 172.21.15.102:0/1818425720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71580ffeb0 0x7f7158105840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7148009910 tx=0x7f7148009c20 comp rx=0 tx=0).stop 2024-05-13T23:01:00.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.501+0000 7f715f27c700 1 -- 172.21.15.102:0/1818425720 shutdown_connections 2024-05-13T23:01:00.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.501+0000 7f715f27c700 1 --2- 172.21.15.102:0/1818425720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71580ffeb0 0x7f7158105840 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:00.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.501+0000 7f715f27c700 1 --2- 172.21.15.102:0/1818425720 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71580ff500 0x7f71580ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:00.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.501+0000 7f715f27c700 1 -- 172.21.15.102:0/1818425720 >> 172.21.15.102:0/1818425720 conn(0x7f7158074d80 msgr2=0x7f7158075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:00.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.501+0000 7f715f27c700 1 -- 172.21.15.102:0/1818425720 shutdown_connections 2024-05-13T23:01:00.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.501+0000 7f715f27c700 1 -- 172.21.15.102:0/1818425720 wait complete. 2024-05-13T23:01:00.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.501+0000 7f715f27c700 1 Processor -- start 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.501+0000 7f715f27c700 1 -- start start 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715f27c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71580ff500 0x7f7158100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715f27c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71580ffeb0 0x7f71581009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715f27c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7158100f00 con 0x7f71580ff500 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715f27c700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7158101040 con 0x7f71580ffeb0 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715d018700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71580ff500 0x7f7158100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715d018700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71580ff500 0x7f7158100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44236/0 (socket says 172.21.15.102:44236) 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715d018700 1 -- 172.21.15.102:0/1678299944 learned_addr learned my addr 172.21.15.102:0/1678299944 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715c817700 1 --2- 172.21.15.102:0/1678299944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71580ffeb0 0x7f71581009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715d018700 1 -- 172.21.15.102:0/1678299944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71580ffeb0 msgr2=0x7f71581009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715d018700 1 --2- 172.21.15.102:0/1678299944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71580ffeb0 0x7f71581009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715d018700 1 -- 172.21.15.102:0/1678299944 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71480095f0 con 0x7f71580ff500 2024-05-13T23:01:00.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715d018700 1 --2- 172.21.15.102:0/1678299944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71580ff500 0x7f7158100480 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f715400d8d0 tx=0x7f715400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:00.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f714e7fc700 1 -- 172.21.15.102:0/1678299944 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71540098a0 con 0x7f71580ff500 2024-05-13T23:01:00.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f714e7fc700 1 -- 172.21.15.102:0/1678299944 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7154010460 con 0x7f71580ff500 2024-05-13T23:01:00.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715f27c700 1 -- 172.21.15.102:0/1678299944 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71581a2930 con 0x7f71580ff500 2024-05-13T23:01:00.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f714e7fc700 1 -- 172.21.15.102:0/1678299944 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f715400f5f0 con 0x7f71580ff500 2024-05-13T23:01:00.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.502+0000 7f715f27c700 1 -- 172.21.15.102:0/1678299944 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71581a2d40 con 0x7f71580ff500 2024-05-13T23:01:00.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.503+0000 7f715f27c700 1 -- 172.21.15.102:0/1678299944 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71581090e0 con 0x7f71580ff500 2024-05-13T23:01:00.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.504+0000 7f714e7fc700 1 -- 172.21.15.102:0/1678299944 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7154009a00 con 0x7f71580ff500 2024-05-13T23:01:00.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.504+0000 7f714e7fc700 1 --2- 172.21.15.102:0/1678299944 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f714406e200 0x7f71440706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:00.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.504+0000 7f714e7fc700 1 -- 172.21.15.102:0/1678299944 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f715408e190 con 0x7f71580ff500 2024-05-13T23:01:00.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.504+0000 7f715c817700 1 --2- 172.21.15.102:0/1678299944 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f714406e200 0x7f71440706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:00.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.505+0000 7f715c817700 1 --2- 172.21.15.102:0/1678299944 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f714406e200 0x7f71440706c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7158101c70 tx=0x7f714801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:00.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.509+0000 7f714e7fc700 1 -- 172.21.15.102:0/1678299944 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7154057350 con 0x7f71580ff500 2024-05-13T23:01:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.789+0000 7f715f27c700 1 -- 172.21.15.102:0/1678299944 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f715804ea90 con 0x7f71580ff500 2024-05-13T23:01:00.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.789+0000 7f714e7fc700 1 -- 172.21.15.102:0/1678299944 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f715405aa10 con 0x7f71580ff500 2024-05-13T23:01:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.792+0000 7f715f27c700 1 -- 172.21.15.102:0/1678299944 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f714406e200 msgr2=0x7f71440706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.792+0000 7f715f27c700 1 --2- 172.21.15.102:0/1678299944 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f714406e200 0x7f71440706c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7158101c70 tx=0x7f714801a040 comp rx=0 tx=0).stop 2024-05-13T23:01:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.792+0000 7f715f27c700 1 -- 172.21.15.102:0/1678299944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71580ff500 msgr2=0x7f7158100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.792+0000 7f715f27c700 1 --2- 172.21.15.102:0/1678299944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71580ff500 0x7f7158100480 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f715400d8d0 tx=0x7f715400dbe0 comp rx=0 tx=0).stop 2024-05-13T23:01:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.792+0000 7f715f27c700 1 -- 172.21.15.102:0/1678299944 shutdown_connections 2024-05-13T23:01:00.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.792+0000 7f715f27c700 1 --2- 172.21.15.102:0/1678299944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71580ff500 0x7f7158100480 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.792+0000 7f715f27c700 1 --2- 172.21.15.102:0/1678299944 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f714406e200 0x7f71440706c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.792+0000 7f715f27c700 1 --2- 172.21.15.102:0/1678299944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71580ffeb0 0x7f71581009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.792+0000 7f715f27c700 1 -- 172.21.15.102:0/1678299944 >> 172.21.15.102:0/1678299944 conn(0x7f7158074d80 msgr2=0x7f71581050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.792+0000 7f715f27c700 1 -- 172.21.15.102:0/1678299944 shutdown_connections 2024-05-13T23:01:00.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:00.792+0000 7f715f27c700 1 -- 172.21.15.102:0/1678299944 wait complete. 2024-05-13T23:01:00.777 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph version 19.0.0-2565-gc3e55145 (c3e55145f4f4b3a500bb66ee08c938a215bc231d) squid (dev)" 2024-05-13T23:01:01.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:01 smithi130 ceph-mon[84265]: pgmap v296: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:01.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:01 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1678299944' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T23:01:01.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:01:01.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:01 smithi102 ceph-mon[111694]: pgmap v296: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:01.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:01 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1678299944' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-13T23:01:01.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:01:01.459 DEBUG:teuthology.parallel:result is None 2024-05-13T23:01:01.460 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-13T23:01:01.473 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T23:01:01.474 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 62275d5c-1179-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-13T23:01:01.681 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/mon.smithi102/config 2024-05-13T23:01:03.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.352+0000 7fe938ca9700 1 -- 172.21.15.102:0/2346032391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9340730b0 msgr2=0x7fe934073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:03.328 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.352+0000 7fe938ca9700 1 --2- 172.21.15.102:0/2346032391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9340730b0 0x7fe934073490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00b0a0 tx=0x7fe92c00b3b0 comp rx=0 tx=0).stop 2024-05-13T23:01:03.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.352+0000 7fe938ca9700 1 -- 172.21.15.102:0/2346032391 shutdown_connections 2024-05-13T23:01:03.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.352+0000 7fe938ca9700 1 --2- 172.21.15.102:0/2346032391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9340739d0 0x7fe93410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:03.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.352+0000 7fe938ca9700 1 --2- 172.21.15.102:0/2346032391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9340730b0 0x7fe934073490 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:03.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.352+0000 7fe938ca9700 1 -- 172.21.15.102:0/2346032391 >> 172.21.15.102:0/2346032391 conn(0x7fe9340fc6b0 msgr2=0x7fe9340fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:03.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.352+0000 7fe938ca9700 1 -- 172.21.15.102:0/2346032391 shutdown_connections 2024-05-13T23:01:03.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.352+0000 7fe938ca9700 1 -- 172.21.15.102:0/2346032391 wait complete. 2024-05-13T23:01:03.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.353+0000 7fe938ca9700 1 Processor -- start 2024-05-13T23:01:03.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.353+0000 7fe938ca9700 1 -- start start 2024-05-13T23:01:03.329 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:03 smithi102 ceph-mon[111694]: pgmap v297: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:03.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.353+0000 7fe938ca9700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9340730b0 0x7fe93419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:03.331 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.353+0000 7fe938ca9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9340739d0 0x7fe93419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.353+0000 7fe938ca9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93419f2d0 con 0x7fe9340739d0 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.353+0000 7fe938ca9700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe934198780 con 0x7fe9340730b0 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.353+0000 7fe931d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9340739d0 0x7fe93419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.353+0000 7fe931d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9340739d0 0x7fe93419ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44242/0 (socket says 172.21.15.102:44242) 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.353+0000 7fe931d9b700 1 -- 172.21.15.102:0/365777726 learned_addr learned my addr 172.21.15.102:0/365777726 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.354+0000 7fe93259c700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9340730b0 0x7fe93419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.354+0000 7fe931d9b700 1 -- 172.21.15.102:0/365777726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9340730b0 msgr2=0x7fe93419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.354+0000 7fe931d9b700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9340730b0 0x7fe93419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.354+0000 7fe931d9b700 1 -- 172.21.15.102:0/365777726 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c009d00 con 0x7fe9340739d0 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.354+0000 7fe93259c700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9340730b0 0x7fe93419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.354+0000 7fe931d9b700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9340739d0 0x7fe93419ec40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d8d0 tx=0x7fe91c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.354+0000 7fe92b7fe700 1 -- 172.21.15.102:0/365777726 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe91c009940 con 0x7fe9340739d0 2024-05-13T23:01:03.332 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.354+0000 7fe938ca9700 1 -- 172.21.15.102:0/365777726 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe934198a60 con 0x7fe9340739d0 2024-05-13T23:01:03.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.354+0000 7fe92b7fe700 1 -- 172.21.15.102:0/365777726 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe91c010460 con 0x7fe9340739d0 2024-05-13T23:01:03.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.354+0000 7fe92b7fe700 1 -- 172.21.15.102:0/365777726 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe91c00f5f0 con 0x7fe9340739d0 2024-05-13T23:01:03.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.354+0000 7fe938ca9700 1 -- 172.21.15.102:0/365777726 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe934198fb0 con 0x7fe9340739d0 2024-05-13T23:01:03.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.355+0000 7fe938ca9700 1 -- 172.21.15.102:0/365777726 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe93410a600 con 0x7fe9340739d0 2024-05-13T23:01:03.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.356+0000 7fe92b7fe700 1 -- 172.21.15.102:0/365777726 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe91c0105d0 con 0x7fe9340739d0 2024-05-13T23:01:03.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.356+0000 7fe92b7fe700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92006e200 0x7fe9200706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:03.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.356+0000 7fe92b7fe700 1 -- 172.21.15.102:0/365777726 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe91c08e2c0 con 0x7fe9340739d0 2024-05-13T23:01:03.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.356+0000 7fe93259c700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92006e200 0x7fe9200706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:03.333 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.357+0000 7fe93259c700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92006e200 0x7fe9200706c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0062a0 tx=0x7fe92c006210 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:03.336 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.361+0000 7fe92b7fe700 1 -- 172.21.15.102:0/365777726 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe91c05afb0 con 0x7fe9340739d0 2024-05-13T23:01:03.579 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.604+0000 7fe938ca9700 1 -- 172.21.15.102:0/365777726 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe93404ea90 con 0x7fe9340739d0 2024-05-13T23:01:03.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.604+0000 7fe92b7fe700 1 -- 172.21.15.102:0/365777726 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1971 (secure 0 0 0) 0x7fe91c05ab40 con 0x7fe9340739d0 2024-05-13T23:01:03.580 INFO:teuthology.orchestra.run.smithi102.stdout:e38 2024-05-13T23:01:03.580 INFO:teuthology.orchestra.run.smithi102.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-13T23:01:03.580 INFO:teuthology.orchestra.run.smithi102.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-13T23:01:03.580 INFO:teuthology.orchestra.run.smithi102.stdout:legacy client fscid: 1 2024-05-13T23:01:03.580 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:03.580 INFO:teuthology.orchestra.run.smithi102.stdout:Filesystem 'cephfs' (1) 2024-05-13T23:01:03.580 INFO:teuthology.orchestra.run.smithi102.stdout:fs_name cephfs 2024-05-13T23:01:03.580 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 38 2024-05-13T23:01:03.580 INFO:teuthology.orchestra.run.smithi102.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-13T23:01:03.580 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-05-13T22:45:10.143899+0000 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:modified 2024-05-13T22:59:05.497987+0000 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:tableserver 0 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:root 0 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:session_timeout 60 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:session_autoclose 300 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:max_file_size 1099511627776 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:max_xattr_size 65536 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:required_client_features {} 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure 0 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:last_failure_osd_epoch 96 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:max_mds 1 2024-05-13T23:01:03.581 INFO:teuthology.orchestra.run.smithi102.stdout:in 0 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:up {0=34410} 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:failed 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:damaged 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:stopped 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:data_pools [3] 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:metadata_pool 2 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:inline_data disabled 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:balancer 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:bal_rank_mask -1 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:standby_count_wanted 1 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:qdb_cluster leader: 34410 members: 34410 2024-05-13T23:01:03.582 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.cgkcwz{0:34410} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.102:6826/3479984123,v1:172.21.15.102:6827/3479984123] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T23:01:03.583 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi102.sdylgv{0:34414} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.102:6828/1256538594,v1:172.21.15.102:6829/1256538594] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T23:01:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:03.583 INFO:teuthology.orchestra.run.smithi102.stdout:Standby daemons: 2024-05-13T23:01:03.583 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:03.583 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.hzjanp{-1:44271} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6824/3836279348,v1:172.21.15.130:6825/3836279348] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T23:01:03.583 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.cephfs.smithi130.ovlxvg{-1:44297} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.130:6826/3806306097,v1:172.21.15.130:6827/3806306097] compat {c=[1],r=[1],i=[1fff]}] 2024-05-13T23:01:03.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.606+0000 7fe938ca9700 1 -- 172.21.15.102:0/365777726 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92006e200 msgr2=0x7fe9200706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:03.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.606+0000 7fe938ca9700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92006e200 0x7fe9200706c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0062a0 tx=0x7fe92c006210 comp rx=0 tx=0).stop 2024-05-13T23:01:03.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.606+0000 7fe938ca9700 1 -- 172.21.15.102:0/365777726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9340739d0 msgr2=0x7fe93419ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:03.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.606+0000 7fe938ca9700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9340739d0 0x7fe93419ec40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d8d0 tx=0x7fe91c00dc90 comp rx=0 tx=0).stop 2024-05-13T23:01:03.584 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.606+0000 7fe938ca9700 1 -- 172.21.15.102:0/365777726 shutdown_connections 2024-05-13T23:01:03.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.606+0000 7fe938ca9700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe92006e200 0x7fe9200706c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:03.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.606+0000 7fe938ca9700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9340730b0 0x7fe93419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:03.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.606+0000 7fe938ca9700 1 --2- 172.21.15.102:0/365777726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9340739d0 0x7fe93419ec40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:03.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.606+0000 7fe938ca9700 1 -- 172.21.15.102:0/365777726 >> 172.21.15.102:0/365777726 conn(0x7fe9340fc6b0 msgr2=0x7fe934107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:03.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.606+0000 7fe938ca9700 1 -- 172.21.15.102:0/365777726 shutdown_connections 2024-05-13T23:01:03.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:03.606+0000 7fe938ca9700 1 -- 172.21.15.102:0/365777726 wait complete. 2024-05-13T23:01:03.585 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 38 2024-05-13T23:01:03.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:03 smithi130 ceph-mon[84265]: pgmap v297: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:04.259 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-05-13T23:01:04.269 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-13T23:01:04.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.398+0000 7f74c992e640 1 -- 172.21.15.102:0/405426081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40fefe0 msgr2=0x7f74c40ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:04.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.398+0000 7f74c992e640 1 --2- 172.21.15.102:0/405426081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40fefe0 0x7f74c40ff440 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009880 tx=0x7f74ac02bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.398+0000 7f74c992e640 1 -- 172.21.15.102:0/405426081 shutdown_connections 2024-05-13T23:01:04.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.398+0000 7f74c992e640 1 --2- 172.21.15.102:0/405426081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40fefe0 0x7f74c40ff440 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.398+0000 7f74c992e640 1 --2- 172.21.15.102:0/405426081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74c4103f20 0x7f74c4104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.398+0000 7f74c992e640 1 -- 172.21.15.102:0/405426081 >> 172.21.15.102:0/405426081 conn(0x7f74c4075b40 msgr2=0x7f74c4075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:04.375 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.398+0000 7f74c992e640 1 -- 172.21.15.102:0/405426081 shutdown_connections 2024-05-13T23:01:04.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.398+0000 7f74c992e640 1 -- 172.21.15.102:0/405426081 wait complete. 2024-05-13T23:01:04.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.398+0000 7f74c992e640 1 Processor -- start 2024-05-13T23:01:04.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.398+0000 7f74c992e640 1 -- start start 2024-05-13T23:01:04.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c992e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40fefe0 0x7f74c4195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:04.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c992e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74c4103f20 0x7f74c41962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:04.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c992e640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4196950 con 0x7f74c40fefe0 2024-05-13T23:01:04.376 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c992e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c419a720 con 0x7f74c4103f20 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c2ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40fefe0 0x7f74c4195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c2ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40fefe0 0x7f74c4195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44252/0 (socket says 172.21.15.102:44252) 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c2ffd640 1 -- 172.21.15.102:0/1728014212 learned_addr learned my addr 172.21.15.102:0/1728014212 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74ba7fc640 1 --2- 172.21.15.102:0/1728014212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74c4103f20 0x7f74c41962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c2ffd640 1 -- 172.21.15.102:0/1728014212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74c4103f20 msgr2=0x7f74c41962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c2ffd640 1 --2- 172.21.15.102:0/1728014212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74c4103f20 0x7f74c41962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c2ffd640 1 -- 172.21.15.102:0/1728014212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac009560 con 0x7f74c40fefe0 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c2ffd640 1 --2- 172.21.15.102:0/1728014212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40fefe0 0x7f74c4195d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74b400c8f0 tx=0x7f74b400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c892c640 1 -- 172.21.15.102:0/1728014212 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74b4007b80 con 0x7f74c40fefe0 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c892c640 1 -- 172.21.15.102:0/1728014212 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74b40054e0 con 0x7f74c40fefe0 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c892c640 1 -- 172.21.15.102:0/1728014212 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74b400f450 con 0x7f74c40fefe0 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c992e640 1 -- 172.21.15.102:0/1728014212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c419aa00 con 0x7f74c40fefe0 2024-05-13T23:01:04.377 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.399+0000 7f74c992e640 1 -- 172.21.15.102:0/1728014212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c419af30 con 0x7f74c40fefe0 2024-05-13T23:01:04.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.400+0000 7f74c992e640 1 -- 172.21.15.102:0/1728014212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c404e040 con 0x7f74c40fefe0 2024-05-13T23:01:04.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.401+0000 7f74c892c640 1 -- 172.21.15.102:0/1728014212 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f74b4004f60 con 0x7f74c40fefe0 2024-05-13T23:01:04.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.401+0000 7f74c892c640 1 --2- 172.21.15.102:0/1728014212 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f74b006e230 0x7f74b00706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:04.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.401+0000 7f74c892c640 1 -- 172.21.15.102:0/1728014212 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f74b408e060 con 0x7f74c40fefe0 2024-05-13T23:01:04.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.401+0000 7f74ba7fc640 1 --2- 172.21.15.102:0/1728014212 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f74b006e230 0x7f74b00706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:04.379 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.402+0000 7f74ba7fc640 1 --2- 172.21.15.102:0/1728014212 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f74b006e230 0x7f74b00706f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f74ac006010 tx=0x7f74ac02cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:04.381 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.406+0000 7f74c892c640 1 -- 172.21.15.102:0/1728014212 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f74b40572d0 con 0x7f74c40fefe0 2024-05-13T23:01:04.511 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:04 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/365777726' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T23:01:04.564 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.589+0000 7f74c992e640 1 -- 172.21.15.102:0/1728014212 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74c41970d0 con 0x7f74c40fefe0 2024-05-13T23:01:04.566 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.591+0000 7f74c892c640 1 -- 172.21.15.102:0/1728014212 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17245 (secure 0 0 0) 0x7f74b405a990 con 0x7f74c40fefe0 2024-05-13T23:01:04.566 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:04.567 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":96,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","created":"2024-05-13T22:38:14.140940+0000","modified":"2024-05-13T22:58:22.091658+0000","last_up_change":"2024-05-13T22:57:16.054261+0000","last_in_change":"2024-05-13T22:43:35.225543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-13T22:42:39.969524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-13T22:45:08.878601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-13T22:45:09.369354+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d8cb2e3-52d6-45f5-94a1-3e8cc6535efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":90,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6803","nonce":307502084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6805","nonce":307502084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6809","nonce":307502084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6807","nonce":307502084}]},"public_addr":"172.21.15.102:6803/307502084","cluster_addr":"172.21.15.102:6805/307502084","heartbeat_back_addr":"172.21.15.102:6809/307502084","heartbeat_front_addr":"172.21.15.102:6807/307502084","state":["exists","up"]},{"osd":1,"uuid":"1d3eacb1-362c-4b6a-a959-7dea19c6521e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":66,"up_thru":90,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6811","nonce":272360893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6813","nonce":272360893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6817","nonce":272360893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6815","nonce":272360893}]},"public_addr":"172.21.15.102:6811/272360893","cluster_addr":"172.21.15.102:6813/272360893","heartbeat_back_addr":"172.21.15.102:6817/272360893","heartbeat_front_addr":"172.21.15.102:6815/272360893","state":["exists","up"]},{"osd":2,"uuid":"205b05a9-9d68-48ed-b87b-c4c677882eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6819","nonce":3530038065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6821","nonce":3530038065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6825","nonce":3530038065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6823","nonce":3530038065}]},"public_addr":"172.21.15.102:6819/3530038065","cluster_addr":"172.21.15.102:6821/3530038065","heartbeat_back_addr":"172.21.15.102:6825/3530038065","heartbeat_front_addr":"172.21.15.102:6823/3530038065","state":["exists","up"]},{"osd":3,"uuid":"d827066c-3201-479e-b142-27721c2761fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6801","nonce":3670165065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6803","nonce":3670165065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6807","nonce":3670165065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6805","nonce":3670165065}]},"public_addr":"172.21.15.130:6801/3670165065","cluster_addr":"172.21.15.130:6803/3670165065","heartbeat_back_addr":"172.21.15.130:6807/3670165065","heartbeat_front_addr":"172.21.15.130:6805/3670165065","state":["exists","up"]},{"osd":4,"uuid":"7d89edd4-8877-4cb8-a0af-019f9451bd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6809","nonce":3241450581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6811","nonce":3241450581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6815","nonce":3241450581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6813","nonce":3241450581}]},"public_addr":"172.21.15.130:6809/3241450581","cluster_addr":"172.21.15.130:6811/3241450581","heartbeat_back_addr":"172.21.15.130:6815/3241450581","heartbeat_front_addr":"172.21.15.130:6813/3241450581","state":["exists","up"]},{"osd":5,"uuid":"c0942077-8a53-4e3b-b411-f1763c592e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6817","nonce":3573323293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6819","nonce":3573323293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6823","nonce":3573323293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6821","nonce":3573323293}]},"public_addr":"172.21.15.130:6817/3573323293","cluster_addr":"172.21.15.130:6819/3573323293","heartbeat_back_addr":"172.21.15.130:6823/3573323293","heartbeat_front_addr":"172.21.15.130:6821/3573323293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:41:41.451627+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:09.327795+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:36.547203+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:00.627031+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:26.709919+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:52.576178+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":0,"to":1}]},{"pgid":"2.6","mappings":[{"from":3,"to":2}]},{"pgid":"2.e","mappings":[{"from":0,"to":1}]},{"pgid":"2.f","mappings":[{"from":0,"to":1}]},{"pgid":"2.15","mappings":[{"from":3,"to":2}]},{"pgid":"2.1d","mappings":[{"from":3,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":4,"to":5}]},{"pgid":"3.1b","mappings":[{"from":3,"to":1}]},{"pgid":"3.1d","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:6827/633282433":"2024-05-14T22:58:22.091628+0000","172.21.15.130:6826/633282433":"2024-05-14T22:58:22.091628+0000","172.21.15.102:6801/2015310067":"2024-05-14T22:52:45.995299+0000","172.21.15.130:0/1149656978":"2024-05-14T22:48:49.345907+0000","172.21.15.102:6827/3231884180":"2024-05-14T22:45:25.114218+0000","172.21.15.102:0/3978175820":"2024-05-14T22:52:45.995299+0000","172.21.15.102:6826/3231884180":"2024-05-14T22:45:25.114218+0000","172.21.15.130:0/2344917375":"2024-05-14T22:48:49.345907+0000","172.21.15.102:0/771792641":"2024-05-14T22:47:31.055945+0000","172.21.15.102:6801/2571859579":"2024-05-14T22:47:31.055945+0000","172.21.15.102:6828/2977397703":"2024-05-14T22:57:41.438543+0000","172.21.15.102:0/3812624194":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6800/2571859579":"2024-05-14T22:47:31.055945+0000","172.21.15.102:0/402887427":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/2015310067":"2024-05-14T22:52:45.995299+0000","172.21.15.102:6801/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.130:6829/3751205200":"2024-05-14T22:48:49.345907+0000","172.21.15.130:6828/3751205200":"2024-05-14T22:48:49.345907+0000","172.21.15.102:6800/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/3850189972":"2024-05-14T22:39:13.681560+0000","172.21.15.130:0/1881061148":"2024-05-14T22:48:49.345907+0000","172.21.15.102:0/2248276518":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/773571736":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/2675775894":"2024-05-14T22:47:31.055945+0000","172.21.15.102:6829/2977397703":"2024-05-14T22:57:41.438543+0000","172.21.15.102:0/457222314":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/1831496769":"2024-05-14T22:52:45.995299+0000","172.21.15.102:0/1238275992":"2024-05-14T22:47:31.055945+0000","172.21.15.130:0/3880089243":"2024-05-14T22:48:49.345907+0000","172.21.15.102:0/633075379":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/2096579260":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/233302973":"2024-05-14T22:47:31.055945+0000","172.21.15.130:0/495062226":"2024-05-14T22:48:49.345907+0000","172.21.15.130:0/1737286614":"2024-05-14T22:48:49.345907+0000","172.21.15.102:6801/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/17493483":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/1416077149":"2024-05-14T22:52:45.995299+0000","172.21.15.102:6801/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/1857247111":"2024-05-14T22:52:45.995299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-13T23:01:04.568 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.592+0000 7f74c992e640 1 -- 172.21.15.102:0/1728014212 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f74b006e230 msgr2=0x7f74b00706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.593+0000 7f74c992e640 1 --2- 172.21.15.102:0/1728014212 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f74b006e230 0x7f74b00706f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f74ac006010 tx=0x7f74ac02cf90 comp rx=0 tx=0).stop 2024-05-13T23:01:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.593+0000 7f74c992e640 1 -- 172.21.15.102:0/1728014212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40fefe0 msgr2=0x7f74c4195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.593+0000 7f74c992e640 1 --2- 172.21.15.102:0/1728014212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40fefe0 0x7f74c4195d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74b400c8f0 tx=0x7f74b400cdb0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.593+0000 7f74c992e640 1 -- 172.21.15.102:0/1728014212 shutdown_connections 2024-05-13T23:01:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.593+0000 7f74c992e640 1 --2- 172.21.15.102:0/1728014212 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f74b006e230 0x7f74b00706f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.593+0000 7f74c992e640 1 --2- 172.21.15.102:0/1728014212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74c4103f20 0x7f74c41962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.593+0000 7f74c992e640 1 --2- 172.21.15.102:0/1728014212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74c40fefe0 0x7f74c4195d80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.593+0000 7f74c992e640 1 -- 172.21.15.102:0/1728014212 >> 172.21.15.102:0/1728014212 conn(0x7f74c4075b40 msgr2=0x7f74c4067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.593+0000 7f74c992e640 1 -- 172.21.15.102:0/1728014212 shutdown_connections 2024-05-13T23:01:04.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.593+0000 7f74c992e640 1 -- 172.21.15.102:0/1728014212 wait complete. 2024-05-13T23:01:04.580 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-13T22:42:39.969524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-13T22:45:08.878601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-13T22:45:09.369354+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-13T23:01:04.580 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-13T23:01:04.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:04 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/365777726' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-13T23:01:04.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.708+0000 7f303dd3c640 1 -- 172.21.15.102:0/283780162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30380ffb50 msgr2=0x7f3038103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:04.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.708+0000 7f303dd3c640 1 --2- 172.21.15.102:0/283780162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30380ffb50 0x7f3038103b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3020009880 tx=0x7f302002bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.708+0000 7f303dd3c640 1 -- 172.21.15.102:0/283780162 shutdown_connections 2024-05-13T23:01:04.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.708+0000 7f303dd3c640 1 --2- 172.21.15.102:0/283780162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30380ffb50 0x7f3038103b90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.708+0000 7f303dd3c640 1 --2- 172.21.15.102:0/283780162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f30380ff1a0 0x7f30380ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.708+0000 7f303dd3c640 1 -- 172.21.15.102:0/283780162 >> 172.21.15.102:0/283780162 conn(0x7f30380faa60 msgr2=0x7f30380fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.708+0000 7f303dd3c640 1 -- 172.21.15.102:0/283780162 shutdown_connections 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.708+0000 7f303dd3c640 1 -- 172.21.15.102:0/283780162 wait complete. 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.708+0000 7f303dd3c640 1 Processor -- start 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.708+0000 7f303dd3c640 1 -- start start 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f303dd3c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30380ff1a0 0x7f3038196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f303dd3c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3038197100 0x7f3038197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f303dd3c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303819fc30 con 0x7f30380ff1a0 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f303dd3c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303819fd70 con 0x7f3038197100 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f30377fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30380ff1a0 0x7f3038196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f30377fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30380ff1a0 0x7f3038196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44272/0 (socket says 172.21.15.102:44272) 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f30377fe640 1 -- 172.21.15.102:0/2495578335 learned_addr learned my addr 172.21.15.102:0/2495578335 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f302ffff640 1 --2- 172.21.15.102:0/2495578335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3038197100 0x7f3038197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:04.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f30377fe640 1 -- 172.21.15.102:0/2495578335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3038197100 msgr2=0x7f3038197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:04.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f30377fe640 1 --2- 172.21.15.102:0/2495578335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3038197100 0x7f3038197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f30377fe640 1 -- 172.21.15.102:0/2495578335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3020009560 con 0x7f30380ff1a0 2024-05-13T23:01:04.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f30377fe640 1 --2- 172.21.15.102:0/2495578335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30380ff1a0 0x7f3038196bc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f302800d720 tx=0x7f302800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:04.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f30357fa640 1 -- 172.21.15.102:0/2495578335 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30280051f0 con 0x7f30380ff1a0 2024-05-13T23:01:04.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f30357fa640 1 -- 172.21.15.102:0/2495578335 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3028009ce0 con 0x7f30380ff1a0 2024-05-13T23:01:04.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f30357fa640 1 -- 172.21.15.102:0/2495578335 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3028010460 con 0x7f30380ff1a0 2024-05-13T23:01:04.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f303dd3c640 1 -- 172.21.15.102:0/2495578335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303819ffd0 con 0x7f30380ff1a0 2024-05-13T23:01:04.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.709+0000 7f303dd3c640 1 -- 172.21.15.102:0/2495578335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30381a0450 con 0x7f30380ff1a0 2024-05-13T23:01:04.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.710+0000 7f303dd3c640 1 -- 172.21.15.102:0/2495578335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303804e040 con 0x7f30380ff1a0 2024-05-13T23:01:04.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.711+0000 7f30357fa640 1 -- 172.21.15.102:0/2495578335 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3028005350 con 0x7f30380ff1a0 2024-05-13T23:01:04.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.711+0000 7f30357fa640 1 --2- 172.21.15.102:0/2495578335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f302406e170 0x7f3024070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:04.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.711+0000 7f302ffff640 1 --2- 172.21.15.102:0/2495578335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f302406e170 0x7f3024070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:04.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.711+0000 7f30357fa640 1 -- 172.21.15.102:0/2495578335 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f302808de30 con 0x7f30380ff1a0 2024-05-13T23:01:04.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.712+0000 7f302ffff640 1 --2- 172.21.15.102:0/2495578335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f302406e170 0x7f3024070630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3020006010 tx=0x7f3020000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:04.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.716+0000 7f30357fa640 1 -- 172.21.15.102:0/2495578335 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f3028056ff0 con 0x7f30380ff1a0 2024-05-13T23:01:04.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.894+0000 7f303dd3c640 1 -- 172.21.15.102:0/2495578335 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30380ffbf0 con 0x7f30380ff1a0 2024-05-13T23:01:04.869 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.894+0000 7f30357fa640 1 -- 172.21.15.102:0/2495578335 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30280170a0 con 0x7f30380ff1a0 2024-05-13T23:01:04.869 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-05-13T23:01:04.871 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.896+0000 7f303dd3c640 1 -- 172.21.15.102:0/2495578335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f302406e170 msgr2=0x7f3024070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:04.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.896+0000 7f303dd3c640 1 --2- 172.21.15.102:0/2495578335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f302406e170 0x7f3024070630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3020006010 tx=0x7f3020000bc0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.896+0000 7f303dd3c640 1 -- 172.21.15.102:0/2495578335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30380ff1a0 msgr2=0x7f3038196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:04.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.896+0000 7f303dd3c640 1 --2- 172.21.15.102:0/2495578335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30380ff1a0 0x7f3038196bc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f302800d720 tx=0x7f302800dbe0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.896+0000 7f303dd3c640 1 -- 172.21.15.102:0/2495578335 shutdown_connections 2024-05-13T23:01:04.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.896+0000 7f303dd3c640 1 --2- 172.21.15.102:0/2495578335 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f302406e170 0x7f3024070630 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.896+0000 7f303dd3c640 1 --2- 172.21.15.102:0/2495578335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3038197100 0x7f3038197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.896+0000 7f303dd3c640 1 --2- 172.21.15.102:0/2495578335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30380ff1a0 0x7f3038196bc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.896+0000 7f303dd3c640 1 -- 172.21.15.102:0/2495578335 >> 172.21.15.102:0/2495578335 conn(0x7f30380faa60 msgr2=0x7f30380fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:04.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.896+0000 7f303dd3c640 1 -- 172.21.15.102:0/2495578335 shutdown_connections 2024-05-13T23:01:04.872 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:04.896+0000 7f303dd3c640 1 -- 172.21.15.102:0/2495578335 wait complete. 2024-05-13T23:01:04.882 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-13T23:01:04.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.010+0000 7f2fe5807640 1 -- 172.21.15.102:0/23506643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00fe990 msgr2=0x7f2fe00fed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:04.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.010+0000 7f2fe5807640 1 --2- 172.21.15.102:0/23506643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00fe990 0x7f2fe00fed70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009880 tx=0x7f2fd802bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.010+0000 7f2fe5807640 1 -- 172.21.15.102:0/23506643 shutdown_connections 2024-05-13T23:01:04.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.010+0000 7f2fe5807640 1 --2- 172.21.15.102:0/23506643 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fe00ff340 0x7f2fe0104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.010+0000 7f2fe5807640 1 --2- 172.21.15.102:0/23506643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00fe990 0x7f2fe00fed70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.010+0000 7f2fe5807640 1 -- 172.21.15.102:0/23506643 >> 172.21.15.102:0/23506643 conn(0x7f2fe0075250 msgr2=0x7f2fe0075660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.010+0000 7f2fe5807640 1 -- 172.21.15.102:0/23506643 shutdown_connections 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.010+0000 7f2fe5807640 1 -- 172.21.15.102:0/23506643 wait complete. 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.010+0000 7f2fe5807640 1 Processor -- start 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.010+0000 7f2fe5807640 1 -- start start 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fe5807640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00fe990 0x7f2fe0195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fe5807640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fe0196300 0x7f2fe019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fe5807640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe0196900 con 0x7f2fe00fe990 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fe5807640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe0196a70 con 0x7f2fe0196300 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fdeffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00fe990 0x7f2fe0195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fdeffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00fe990 0x7f2fe0195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44280/0 (socket says 172.21.15.102:44280) 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fdeffd640 1 -- 172.21.15.102:0/2102953677 learned_addr learned my addr 172.21.15.102:0/2102953677 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fde7fc640 1 --2- 172.21.15.102:0/2102953677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fe0196300 0x7f2fe019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:04.988 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fdeffd640 1 -- 172.21.15.102:0/2102953677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fe0196300 msgr2=0x7f2fe019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:04.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fdeffd640 1 --2- 172.21.15.102:0/2102953677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fe0196300 0x7f2fe019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:04.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fdeffd640 1 -- 172.21.15.102:0/2102953677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd8009560 con 0x7f2fe00fe990 2024-05-13T23:01:04.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fdeffd640 1 --2- 172.21.15.102:0/2102953677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00fe990 0x7f2fe0195dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009430 tx=0x7f2fd8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:04.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fe4805640 1 -- 172.21.15.102:0/2102953677 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd80035f0 con 0x7f2fe00fe990 2024-05-13T23:01:04.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fe5807640 1 -- 172.21.15.102:0/2102953677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe019ad00 con 0x7f2fe00fe990 2024-05-13T23:01:04.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fe4805640 1 -- 172.21.15.102:0/2102953677 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fd8003c10 con 0x7f2fe00fe990 2024-05-13T23:01:04.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.011+0000 7f2fe4805640 1 -- 172.21.15.102:0/2102953677 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd8005670 con 0x7f2fe00fe990 2024-05-13T23:01:04.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.012+0000 7f2fe5807640 1 -- 172.21.15.102:0/2102953677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe019b260 con 0x7f2fe00fe990 2024-05-13T23:01:04.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.013+0000 7f2fe5807640 1 -- 172.21.15.102:0/2102953677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe004e040 con 0x7f2fe00fe990 2024-05-13T23:01:04.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.013+0000 7f2fe4805640 1 -- 172.21.15.102:0/2102953677 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2fd8004dd0 con 0x7f2fe00fe990 2024-05-13T23:01:04.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.013+0000 7f2fe4805640 1 --2- 172.21.15.102:0/2102953677 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2fc806e120 0x7f2fc80705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:04.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.014+0000 7f2fe4805640 1 -- 172.21.15.102:0/2102953677 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f2fd80b0660 con 0x7f2fe00fe990 2024-05-13T23:01:04.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.014+0000 7f2fde7fc640 1 --2- 172.21.15.102:0/2102953677 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2fc806e120 0x7f2fc80705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:04.990 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.014+0000 7f2fde7fc640 1 --2- 172.21.15.102:0/2102953677 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2fc806e120 0x7f2fc80705e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0072580 tx=0x7f2fd4009870 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:04.993 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.018+0000 7f2fe4805640 1 -- 172.21.15.102:0/2102953677 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2fd8079820 con 0x7f2fe00fe990 2024-05-13T23:01:05.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.196+0000 7f2fe5807640 1 -- 172.21.15.102:0/2102953677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f2fe01085c0 con 0x7f2fe00fe990 2024-05-13T23:01:05.172 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.196+0000 7f2fe4805640 1 -- 172.21.15.102:0/2102953677 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2fe01085c0 con 0x7f2fe00fe990 2024-05-13T23:01:05.172 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-05-13T23:01:05.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.199+0000 7f2fe5807640 1 -- 172.21.15.102:0/2102953677 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2fc806e120 msgr2=0x7f2fc80705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.174 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.199+0000 7f2fe5807640 1 --2- 172.21.15.102:0/2102953677 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2fc806e120 0x7f2fc80705e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0072580 tx=0x7f2fd4009870 comp rx=0 tx=0).stop 2024-05-13T23:01:05.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.199+0000 7f2fe5807640 1 -- 172.21.15.102:0/2102953677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00fe990 msgr2=0x7f2fe0195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.199+0000 7f2fe5807640 1 --2- 172.21.15.102:0/2102953677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00fe990 0x7f2fe0195dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009430 tx=0x7f2fd8004040 comp rx=0 tx=0).stop 2024-05-13T23:01:05.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.199+0000 7f2fe5807640 1 -- 172.21.15.102:0/2102953677 shutdown_connections 2024-05-13T23:01:05.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.199+0000 7f2fe5807640 1 --2- 172.21.15.102:0/2102953677 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2fc806e120 0x7f2fc80705e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.199+0000 7f2fe5807640 1 --2- 172.21.15.102:0/2102953677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2fe0196300 0x7f2fe019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.199+0000 7f2fe5807640 1 --2- 172.21.15.102:0/2102953677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fe00fe990 0x7f2fe0195dc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.199+0000 7f2fe5807640 1 -- 172.21.15.102:0/2102953677 >> 172.21.15.102:0/2102953677 conn(0x7f2fe0075250 msgr2=0x7f2fe00fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:05.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.199+0000 7f2fe5807640 1 -- 172.21.15.102:0/2102953677 shutdown_connections 2024-05-13T23:01:05.175 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.199+0000 7f2fe5807640 1 -- 172.21.15.102:0/2102953677 wait complete. 2024-05-13T23:01:05.185 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.313+0000 7f1cc711f640 1 -- 172.21.15.102:0/2927988882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cc00ff260 msgr2=0x7f1cc00ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.313+0000 7f1cc711f640 1 --2- 172.21.15.102:0/2927988882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cc00ff260 0x7f1cc00ff640 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009880 tx=0x7f1cb002bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.313+0000 7f1cc711f640 1 -- 172.21.15.102:0/2927988882 shutdown_connections 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cc711f640 1 --2- 172.21.15.102:0/2927988882 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cc00ffc10 0x7f1cc0103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cc711f640 1 --2- 172.21.15.102:0/2927988882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cc00ff260 0x7f1cc00ff640 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cc711f640 1 -- 172.21.15.102:0/2927988882 >> 172.21.15.102:0/2927988882 conn(0x7f1cc0074a50 msgr2=0x7f1cc0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cc711f640 1 -- 172.21.15.102:0/2927988882 shutdown_connections 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cc711f640 1 -- 172.21.15.102:0/2927988882 wait complete. 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cc711f640 1 Processor -- start 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cc711f640 1 -- start start 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cc711f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cc00ff260 0x7f1cc0194cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:05.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cc711f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cc01951f0 0x7f1cc01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cc711f640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc01957f0 con 0x7f1cc01951f0 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cc711f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc0195930 con 0x7f1cc00ff260 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cc01951f0 0x7f1cc01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cc01951f0 0x7f1cc01985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44290/0 (socket says 172.21.15.102:44290) 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.314+0000 7f1cbffff640 1 -- 172.21.15.102:0/2744157413 learned_addr learned my addr 172.21.15.102:0/2744157413 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.315+0000 7f1cc4ebb640 1 --2- 172.21.15.102:0/2744157413 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cc00ff260 0x7f1cc0194cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.315+0000 7f1cbffff640 1 -- 172.21.15.102:0/2744157413 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cc00ff260 msgr2=0x7f1cc0194cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.315+0000 7f1cbffff640 1 --2- 172.21.15.102:0/2744157413 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cc00ff260 0x7f1cc0194cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.315+0000 7f1cbffff640 1 -- 172.21.15.102:0/2744157413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb0009560 con 0x7f1cc01951f0 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.315+0000 7f1cbffff640 1 --2- 172.21.15.102:0/2744157413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cc01951f0 0x7f1cc01985a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400eb40 tx=0x7f1cb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.315+0000 7f1cbdffb640 1 -- 172.21.15.102:0/2744157413 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb400ce80 con 0x7f1cc01951f0 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.315+0000 7f1cbdffb640 1 -- 172.21.15.102:0/2744157413 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cb4005020 con 0x7f1cc01951f0 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.315+0000 7f1cc711f640 1 -- 172.21.15.102:0/2744157413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc0198ba0 con 0x7f1cc01951f0 2024-05-13T23:01:05.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.315+0000 7f1cbdffb640 1 -- 172.21.15.102:0/2744157413 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb40106f0 con 0x7f1cc01951f0 2024-05-13T23:01:05.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.315+0000 7f1cc711f640 1 -- 172.21.15.102:0/2744157413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc0199100 con 0x7f1cc01951f0 2024-05-13T23:01:05.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.316+0000 7f1cc711f640 1 -- 172.21.15.102:0/2744157413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc01993e0 con 0x7f1cc01951f0 2024-05-13T23:01:05.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.317+0000 7f1cbdffb640 1 -- 172.21.15.102:0/2744157413 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1cb4005190 con 0x7f1cc01951f0 2024-05-13T23:01:05.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.317+0000 7f1cbdffb640 1 --2- 172.21.15.102:0/2744157413 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1ca806e170 0x7f1ca8070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:05.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.317+0000 7f1cbdffb640 1 -- 172.21.15.102:0/2744157413 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1cb4014070 con 0x7f1cc01951f0 2024-05-13T23:01:05.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.317+0000 7f1cc4ebb640 1 --2- 172.21.15.102:0/2744157413 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1ca806e170 0x7f1ca8070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:05.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.317+0000 7f1cc4ebb640 1 --2- 172.21.15.102:0/2744157413 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1ca806e170 0x7f1ca8070630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006010 tx=0x7f1cb0005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:05.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.322+0000 7f1cbdffb640 1 -- 172.21.15.102:0/2744157413 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1cb4057d40 con 0x7f1cc01951f0 2024-05-13T23:01:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.498+0000 7f1cc711f640 1 -- 172.21.15.102:0/2744157413 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f1cc004e040 con 0x7f1cc01951f0 2024-05-13T23:01:05.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.498+0000 7f1cbdffb640 1 -- 172.21.15.102:0/2744157413 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1cb405b400 con 0x7f1cc01951f0 2024-05-13T23:01:05.474 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-05-13T23:01:05.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.500+0000 7f1cc711f640 1 -- 172.21.15.102:0/2744157413 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1ca806e170 msgr2=0x7f1ca8070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.500+0000 7f1cc711f640 1 --2- 172.21.15.102:0/2744157413 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1ca806e170 0x7f1ca8070630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006010 tx=0x7f1cb0005dd0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.500+0000 7f1cc711f640 1 -- 172.21.15.102:0/2744157413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cc01951f0 msgr2=0x7f1cc01985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.500+0000 7f1cc711f640 1 --2- 172.21.15.102:0/2744157413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cc01951f0 0x7f1cc01985a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400eb40 tx=0x7f1cb400c2d0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.500+0000 7f1cc711f640 1 -- 172.21.15.102:0/2744157413 shutdown_connections 2024-05-13T23:01:05.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.500+0000 7f1cc711f640 1 --2- 172.21.15.102:0/2744157413 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1ca806e170 0x7f1ca8070630 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.500+0000 7f1cc711f640 1 --2- 172.21.15.102:0/2744157413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cc01951f0 0x7f1cc01985a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.500+0000 7f1cc711f640 1 --2- 172.21.15.102:0/2744157413 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1cc00ff260 0x7f1cc0194cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.476 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.501+0000 7f1cc711f640 1 -- 172.21.15.102:0/2744157413 >> 172.21.15.102:0/2744157413 conn(0x7f1cc0074a50 msgr2=0x7f1cc00fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:05.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.501+0000 7f1cc711f640 1 -- 172.21.15.102:0/2744157413 shutdown_connections 2024-05-13T23:01:05.477 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.501+0000 7f1cc711f640 1 -- 172.21.15.102:0/2744157413 wait complete. 2024-05-13T23:01:05.486 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:05 smithi102 ceph-mon[111694]: pgmap v298: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:05.486 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:05 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1728014212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T23:01:05.486 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:05 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2495578335' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-13T23:01:05.486 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:05 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2102953677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-13T23:01:05.487 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-13T23:01:05.589 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.612+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3666269073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec580ff930 msgr2=0x7fec580ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.612+0000 7fec5e9a0640 1 --2- 172.21.15.102:0/3666269073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec580ff930 0x7fec580ffd10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009880 tx=0x7fec4c02bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.612+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3666269073 shutdown_connections 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.612+0000 7fec5e9a0640 1 --2- 172.21.15.102:0/3666269073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec581002e0 0x7fec581039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.612+0000 7fec5e9a0640 1 --2- 172.21.15.102:0/3666269073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec580ff930 0x7fec580ffd10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.612+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3666269073 >> 172.21.15.102:0/3666269073 conn(0x7fec58075100 msgr2=0x7fec58075510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.612+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3666269073 shutdown_connections 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.612+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3666269073 wait complete. 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5e9a0640 1 Processor -- start 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5e9a0640 1 -- start start 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5e9a0640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec580ff930 0x7fec58196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5e9a0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec58196ea0 0x7fec58197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5e9a0640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5819f9d0 con 0x7fec580ff930 2024-05-13T23:01:05.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5e9a0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5819fb10 con 0x7fec58196ea0 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5d99e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec580ff930 0x7fec58196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5d99e640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec580ff930 0x7fec58196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44312/0 (socket says 172.21.15.102:44312) 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5d99e640 1 -- 172.21.15.102:0/3391684745 learned_addr learned my addr 172.21.15.102:0/3391684745 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5d19d640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec58196ea0 0x7fec58197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5d99e640 1 -- 172.21.15.102:0/3391684745 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec58196ea0 msgr2=0x7fec58197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5d99e640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec58196ea0 0x7fec58197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5d99e640 1 -- 172.21.15.102:0/3391684745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c009560 con 0x7fec580ff930 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5d19d640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec58196ea0 0x7fec58197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.613+0000 7fec5d99e640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec580ff930 0x7fec58196960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009430 tx=0x7fec4c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.614+0000 7fec4affd640 1 -- 172.21.15.102:0/3391684745 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec4c0035f0 con 0x7fec580ff930 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.614+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3391684745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5819fd10 con 0x7fec580ff930 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.614+0000 7fec4affd640 1 -- 172.21.15.102:0/3391684745 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec4c003c10 con 0x7fec580ff930 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.614+0000 7fec4affd640 1 -- 172.21.15.102:0/3391684745 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec4c034440 con 0x7fec580ff930 2024-05-13T23:01:05.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.614+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3391684745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec581a01c0 con 0x7fec580ff930 2024-05-13T23:01:05.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.615+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3391684745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5804e890 con 0x7fec580ff930 2024-05-13T23:01:05.592 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.615+0000 7fec4affd640 1 -- 172.21.15.102:0/3391684745 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fec4c004fd0 con 0x7fec580ff930 2024-05-13T23:01:05.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.615+0000 7fec4affd640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fec4406e230 0x7fec440706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:05.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.616+0000 7fec4affd640 1 -- 172.21.15.102:0/3391684745 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fec4c0b0720 con 0x7fec580ff930 2024-05-13T23:01:05.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.616+0000 7fec5d19d640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fec4406e230 0x7fec440706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:05.593 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.616+0000 7fec5d19d640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fec4406e230 0x7fec440706f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fec540058c0 tx=0x7fec54005d30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:05.595 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.620+0000 7fec4affd640 1 -- 172.21.15.102:0/3391684745 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fec4c0798e0 con 0x7fec580ff930 2024-05-13T23:01:05.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:05 smithi130 ceph-mon[84265]: pgmap v298: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:05.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:05 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1728014212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T23:01:05.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:05 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2495578335' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-13T23:01:05.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:05 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2102953677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-13T23:01:05.806 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.831+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3391684745 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec581072f0 con 0x7fec580ff930 2024-05-13T23:01:05.807 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.831+0000 7fec4affd640 1 -- 172.21.15.102:0/3391684745 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5227 (secure 0 0 0) 0x7fec581072f0 con 0x7fec580ff930 2024-05-13T23:01:05.807 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:05.807 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44271,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/3836279348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3836279348},{"type":"v1","addr":"172.21.15.130:6825","nonce":3836279348}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44297,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6827/3806306097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3806306097},{"type":"v1","addr":"172.21.15.130:6827","nonce":3806306097}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:59:05.497987+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34410":{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":0,"incarnation":30,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34414":{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34410,"qdb_cluster":[34410]},"id":1}]} 2024-05-13T23:01:05.807 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 38 2024-05-13T23:01:05.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.833+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3391684745 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fec4406e230 msgr2=0x7fec440706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.833+0000 7fec5e9a0640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fec4406e230 0x7fec440706f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fec540058c0 tx=0x7fec54005d30 comp rx=0 tx=0).stop 2024-05-13T23:01:05.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.833+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3391684745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec580ff930 msgr2=0x7fec58196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.833+0000 7fec5e9a0640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec580ff930 0x7fec58196960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009430 tx=0x7fec4c004040 comp rx=0 tx=0).stop 2024-05-13T23:01:05.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.833+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3391684745 shutdown_connections 2024-05-13T23:01:05.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.833+0000 7fec5e9a0640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fec4406e230 0x7fec440706f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.833+0000 7fec5e9a0640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec58196ea0 0x7fec58197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.833+0000 7fec5e9a0640 1 --2- 172.21.15.102:0/3391684745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec580ff930 0x7fec58196960 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.833+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3391684745 >> 172.21.15.102:0/3391684745 conn(0x7fec58075100 msgr2=0x7fec580fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:05.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.834+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3391684745 shutdown_connections 2024-05-13T23:01:05.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.834+0000 7fec5e9a0640 1 -- 172.21.15.102:0/3391684745 wait complete. 2024-05-13T23:01:05.820 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 1, 'flags': 50} 2024-05-13T23:01:05.820 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-05-13T23:01:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.951+0000 7f83b8563640 1 -- 172.21.15.102:0/783393299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b00ffc10 msgr2=0x7f83b0103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.951+0000 7f83b8563640 1 --2- 172.21.15.102:0/783393299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b00ffc10 0x7f83b0103b70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009880 tx=0x7f83a002bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.951+0000 7f83b8563640 1 -- 172.21.15.102:0/783393299 shutdown_connections 2024-05-13T23:01:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.951+0000 7f83b8563640 1 --2- 172.21.15.102:0/783393299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b00ffc10 0x7f83b0103b70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.951+0000 7f83b8563640 1 --2- 172.21.15.102:0/783393299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83b00ff260 0x7f83b00ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.951+0000 7f83b8563640 1 -- 172.21.15.102:0/783393299 >> 172.21.15.102:0/783393299 conn(0x7f83b0074a50 msgr2=0x7f83b0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.951+0000 7f83b8563640 1 -- 172.21.15.102:0/783393299 shutdown_connections 2024-05-13T23:01:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.951+0000 7f83b8563640 1 -- 172.21.15.102:0/783393299 wait complete. 2024-05-13T23:01:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.951+0000 7f83b8563640 1 Processor -- start 2024-05-13T23:01:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.951+0000 7f83b8563640 1 -- start start 2024-05-13T23:01:05.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b8563640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b00ff260 0x7f83b0195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b8563640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83b01962f0 0x7f83b019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b8563640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01968f0 con 0x7f83b00ff260 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b8563640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b0196a60 con 0x7f83b01962f0 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b62ff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b00ff260 0x7f83b0195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b62ff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b00ff260 0x7f83b0195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44338/0 (socket says 172.21.15.102:44338) 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b62ff640 1 -- 172.21.15.102:0/2879173592 learned_addr learned my addr 172.21.15.102:0/2879173592 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b62ff640 1 -- 172.21.15.102:0/2879173592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83b01962f0 msgr2=0x7f83b019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b62ff640 1 --2- 172.21.15.102:0/2879173592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83b01962f0 0x7f83b019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b62ff640 1 -- 172.21.15.102:0/2879173592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0009560 con 0x7f83b00ff260 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b62ff640 1 --2- 172.21.15.102:0/2879173592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b00ff260 0x7f83b0195db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00d900 tx=0x7f83ac00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83a77fe640 1 -- 172.21.15.102:0/2879173592 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83ac009c00 con 0x7f83b00ff260 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b8563640 1 -- 172.21.15.102:0/2879173592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b019ad50 con 0x7f83b00ff260 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83a77fe640 1 -- 172.21.15.102:0/2879173592 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83ac009d60 con 0x7f83b00ff260 2024-05-13T23:01:05.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83a77fe640 1 -- 172.21.15.102:0/2879173592 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83ac00f710 con 0x7f83b00ff260 2024-05-13T23:01:05.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.952+0000 7f83b8563640 1 -- 172.21.15.102:0/2879173592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b019b2b0 con 0x7f83b00ff260 2024-05-13T23:01:05.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.953+0000 7f83b8563640 1 -- 172.21.15.102:0/2879173592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b004e040 con 0x7f83b00ff260 2024-05-13T23:01:05.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.954+0000 7f83a77fe640 1 -- 172.21.15.102:0/2879173592 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f83ac010460 con 0x7f83b00ff260 2024-05-13T23:01:05.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.954+0000 7f83a77fe640 1 --2- 172.21.15.102:0/2879173592 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f839c06e170 0x7f839c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:05.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.954+0000 7f83a77fe640 1 -- 172.21.15.102:0/2879173592 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f83ac08e0c0 con 0x7f83b00ff260 2024-05-13T23:01:05.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.954+0000 7f83b5afe640 1 --2- 172.21.15.102:0/2879173592 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f839c06e170 0x7f839c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:05.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.955+0000 7f83b5afe640 1 --2- 172.21.15.102:0/2879173592 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f839c06e170 0x7f839c070630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f83a002efd0 tx=0x7f83a0000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:05.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:05.959+0000 7f83a77fe640 1 -- 172.21.15.102:0/2879173592 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f83ac057280 con 0x7f83b00ff260 2024-05-13T23:01:06.148 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.173+0000 7f83b8563640 1 -- 172.21.15.102:0/2879173592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f83b019b530 con 0x7f83b00ff260 2024-05-13T23:01:06.149 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.173+0000 7f83a77fe640 1 -- 172.21.15.102:0/2879173592 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v38) v1 ==== 107+0+4091 (secure 0 0 0) 0x7f83ac05a940 con 0x7f83b00ff260 2024-05-13T23:01:06.149 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:06.149 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24281,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/244897459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":244897459},{"type":"v1","addr":"172.21.15.130:6825","nonce":244897459}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:57:18.071027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14498},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14498":{"gid":14498,"name":"cephfs.smithi102.sdylgv","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6829/2977397703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2977397703},{"type":"v1","addr":"172.21.15.102:6829","nonce":2977397703}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14498,"qdb_cluster":[14498]},"id":1}]} 2024-05-13T23:01:06.149 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 18 2024-05-13T23:01:06.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.175+0000 7f83b8563640 1 -- 172.21.15.102:0/2879173592 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f839c06e170 msgr2=0x7f839c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.175+0000 7f83b8563640 1 --2- 172.21.15.102:0/2879173592 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f839c06e170 0x7f839c070630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f83a002efd0 tx=0x7f83a0000bc0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.175+0000 7f83b8563640 1 -- 172.21.15.102:0/2879173592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b00ff260 msgr2=0x7f83b0195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.175+0000 7f83b8563640 1 --2- 172.21.15.102:0/2879173592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b00ff260 0x7f83b0195db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00d900 tx=0x7f83ac00ddc0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.175+0000 7f83b8563640 1 -- 172.21.15.102:0/2879173592 shutdown_connections 2024-05-13T23:01:06.151 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.175+0000 7f83b8563640 1 --2- 172.21.15.102:0/2879173592 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f839c06e170 0x7f839c070630 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.175+0000 7f83b8563640 1 --2- 172.21.15.102:0/2879173592 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83b01962f0 0x7f83b019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.175+0000 7f83b8563640 1 --2- 172.21.15.102:0/2879173592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f83b00ff260 0x7f83b0195db0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.175+0000 7f83b8563640 1 -- 172.21.15.102:0/2879173592 >> 172.21.15.102:0/2879173592 conn(0x7f83b0074a50 msgr2=0x7f83b0103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:06.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.176+0000 7f83b8563640 1 -- 172.21.15.102:0/2879173592 shutdown_connections 2024-05-13T23:01:06.152 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.176+0000 7f83b8563640 1 -- 172.21.15.102:0/2879173592 wait complete. 2024-05-13T23:01:06.162 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-05-13T23:01:06.162 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-05-13T23:01:06.266 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.289+0000 7f230d91a640 1 -- 172.21.15.102:0/979238648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23081002b0 msgr2=0x7f2308100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.289+0000 7f230d91a640 1 --2- 172.21.15.102:0/979238648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23081002b0 0x7f2308100690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009880 tx=0x7f22f802bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.289+0000 7f230d91a640 1 -- 172.21.15.102:0/979238648 shutdown_connections 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.289+0000 7f230d91a640 1 --2- 172.21.15.102:0/979238648 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2308100c60 0x7f2308104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.289+0000 7f230d91a640 1 --2- 172.21.15.102:0/979238648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23081002b0 0x7f2308100690 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.289+0000 7f230d91a640 1 -- 172.21.15.102:0/979238648 >> 172.21.15.102:0/979238648 conn(0x7f23080fbb30 msgr2=0x7f23080fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.289+0000 7f230d91a640 1 -- 172.21.15.102:0/979238648 shutdown_connections 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.289+0000 7f230d91a640 1 -- 172.21.15.102:0/979238648 wait complete. 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.290+0000 7f230d91a640 1 Processor -- start 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.290+0000 7f230d91a640 1 -- start start 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.290+0000 7f230d91a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23081002b0 0x7f2308195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.290+0000 7f230d91a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23081963e0 0x7f230819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:06.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.290+0000 7f230d91a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23081969e0 con 0x7f23081002b0 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.290+0000 7f230d91a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2308196b50 con 0x7f23081963e0 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.290+0000 7f2306ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23081002b0 0x7f2308195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.290+0000 7f2306ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23081002b0 0x7f2308195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44350/0 (socket says 172.21.15.102:44350) 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.290+0000 7f2306ffd640 1 -- 172.21.15.102:0/1324263160 learned_addr learned my addr 172.21.15.102:0/1324263160 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.291+0000 7f22fe7fc640 1 --2- 172.21.15.102:0/1324263160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23081963e0 0x7f230819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.291+0000 7f2306ffd640 1 -- 172.21.15.102:0/1324263160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23081963e0 msgr2=0x7f230819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.291+0000 7f2306ffd640 1 --2- 172.21.15.102:0/1324263160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23081963e0 0x7f230819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.291+0000 7f2306ffd640 1 -- 172.21.15.102:0/1324263160 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8009560 con 0x7f23081002b0 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.291+0000 7f2306ffd640 1 --2- 172.21.15.102:0/1324263160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23081002b0 0x7f2308195ea0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009430 tx=0x7f22f8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.291+0000 7f230c918640 1 -- 172.21.15.102:0/1324263160 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22f80035f0 con 0x7f23081002b0 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.291+0000 7f230c918640 1 -- 172.21.15.102:0/1324263160 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22f8003c10 con 0x7f23081002b0 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.291+0000 7f230d91a640 1 -- 172.21.15.102:0/1324263160 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230819ade0 con 0x7f23081002b0 2024-05-13T23:01:06.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.291+0000 7f230c918640 1 -- 172.21.15.102:0/1324263160 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22f8034440 con 0x7f23081002b0 2024-05-13T23:01:06.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.291+0000 7f230d91a640 1 -- 172.21.15.102:0/1324263160 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230819b340 con 0x7f23081002b0 2024-05-13T23:01:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.292+0000 7f230d91a640 1 -- 172.21.15.102:0/1324263160 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230804e040 con 0x7f23081002b0 2024-05-13T23:01:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.292+0000 7f230c918640 1 -- 172.21.15.102:0/1324263160 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f22f8003750 con 0x7f23081002b0 2024-05-13T23:01:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.293+0000 7f230c918640 1 --2- 172.21.15.102:0/1324263160 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f22f406e170 0x7f22f4070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.293+0000 7f230c918640 1 -- 172.21.15.102:0/1324263160 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f22f80b0700 con 0x7f23081002b0 2024-05-13T23:01:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.293+0000 7f22fe7fc640 1 --2- 172.21.15.102:0/1324263160 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f22f406e170 0x7f22f4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:06.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.293+0000 7f22fe7fc640 1 --2- 172.21.15.102:0/1324263160 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f22f406e170 0x7f22f4070630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f23080694a0 tx=0x7f22f0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:06.273 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.298+0000 7f230c918640 1 -- 172.21.15.102:0/1324263160 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f22f80798c0 con 0x7f23081002b0 2024-05-13T23:01:06.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:06 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2744157413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-13T23:01:06.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:06 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3391684745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-13T23:01:06.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:06 smithi130 ceph-mon[84265]: pgmap v299: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:06.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:06 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2879173592' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-05-13T23:01:06.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:06 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2744157413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-13T23:01:06.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:06 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3391684745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-13T23:01:06.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:06 smithi102 ceph-mon[111694]: pgmap v299: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:06.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:06 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2879173592' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-05-13T23:01:06.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.508+0000 7f230d91a640 1 -- 172.21.15.102:0/1324263160 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f23081085b0 con 0x7f23081002b0 2024-05-13T23:01:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.509+0000 7f230c918640 1 -- 172.21.15.102:0/1324263160 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v38) v1 ==== 107+0+4873 (secure 0 0 0) 0x7f22f807cf80 con 0x7f23081002b0 2024-05-13T23:01:06.484 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:06.484 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24281,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/244897459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":244897459},{"type":"v1","addr":"172.21.15.130:6825","nonce":244897459}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34378,"name":"cephfs.smithi102.cgkcwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6827/1720794729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":1720794729},{"type":"v1","addr":"172.21.15.102:6827","nonce":1720794729}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:57:18.071027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14498},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14498":{"gid":14498,"name":"cephfs.smithi102.sdylgv","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6829/2977397703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2977397703},{"type":"v1","addr":"172.21.15.102:6829","nonce":2977397703}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14498,"qdb_cluster":[14498]},"id":1}]} 2024-05-13T23:01:06.484 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 19 2024-05-13T23:01:06.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.511+0000 7f230d91a640 1 -- 172.21.15.102:0/1324263160 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f22f406e170 msgr2=0x7f22f4070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.511+0000 7f230d91a640 1 --2- 172.21.15.102:0/1324263160 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f22f406e170 0x7f22f4070630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f23080694a0 tx=0x7f22f0008040 comp rx=0 tx=0).stop 2024-05-13T23:01:06.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.511+0000 7f230d91a640 1 -- 172.21.15.102:0/1324263160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23081002b0 msgr2=0x7f2308195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.511+0000 7f230d91a640 1 --2- 172.21.15.102:0/1324263160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23081002b0 0x7f2308195ea0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009430 tx=0x7f22f8004040 comp rx=0 tx=0).stop 2024-05-13T23:01:06.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.511+0000 7f230d91a640 1 -- 172.21.15.102:0/1324263160 shutdown_connections 2024-05-13T23:01:06.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.511+0000 7f230d91a640 1 --2- 172.21.15.102:0/1324263160 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f22f406e170 0x7f22f4070630 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.511+0000 7f230d91a640 1 --2- 172.21.15.102:0/1324263160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f23081963e0 0x7f230819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.511+0000 7f230d91a640 1 --2- 172.21.15.102:0/1324263160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23081002b0 0x7f2308195ea0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.511+0000 7f230d91a640 1 -- 172.21.15.102:0/1324263160 >> 172.21.15.102:0/1324263160 conn(0x7f23080fbb30 msgr2=0x7f23080fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:06.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.511+0000 7f230d91a640 1 -- 172.21.15.102:0/1324263160 shutdown_connections 2024-05-13T23:01:06.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.511+0000 7f230d91a640 1 -- 172.21.15.102:0/1324263160 wait complete. 2024-05-13T23:01:06.498 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-05-13T23:01:06.498 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-05-13T23:01:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.623+0000 7fb29a563640 1 -- 172.21.15.102:0/215405627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2940ff1a0 msgr2=0x7fb2940ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.623+0000 7fb29a563640 1 --2- 172.21.15.102:0/215405627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2940ff1a0 0x7fb2940ff580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb284009880 tx=0x7fb28402bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.600 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.623+0000 7fb29a563640 1 -- 172.21.15.102:0/215405627 shutdown_connections 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.623+0000 7fb29a563640 1 --2- 172.21.15.102:0/215405627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2940ffb50 0x7fb294103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.623+0000 7fb29a563640 1 --2- 172.21.15.102:0/215405627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2940ff1a0 0x7fb2940ff580 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.623+0000 7fb29a563640 1 -- 172.21.15.102:0/215405627 >> 172.21.15.102:0/215405627 conn(0x7fb2940faa20 msgr2=0x7fb2940fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.623+0000 7fb29a563640 1 -- 172.21.15.102:0/215405627 shutdown_connections 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.623+0000 7fb29a563640 1 -- 172.21.15.102:0/215405627 wait complete. 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb29a563640 1 Processor -- start 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb29a563640 1 -- start start 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb29a563640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2940ff1a0 0x7fb294195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb29a563640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2941962b0 0x7fb29419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb29a563640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb294196820 con 0x7fb2941962b0 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb29a563640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb294196990 con 0x7fb2940ff1a0 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb2937fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2941962b0 0x7fb29419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb2937fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2941962b0 0x7fb29419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44372/0 (socket says 172.21.15.102:44372) 2024-05-13T23:01:06.601 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb2937fe640 1 -- 172.21.15.102:0/1413744784 learned_addr learned my addr 172.21.15.102:0/1413744784 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:06.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb2937fe640 1 -- 172.21.15.102:0/1413744784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2940ff1a0 msgr2=0x7fb294195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb2937fe640 1 --2- 172.21.15.102:0/1413744784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2940ff1a0 0x7fb294195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.624+0000 7fb2937fe640 1 -- 172.21.15.102:0/1413744784 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb284009560 con 0x7fb2941962b0 2024-05-13T23:01:06.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.625+0000 7fb2937fe640 1 --2- 172.21.15.102:0/1413744784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2941962b0 0x7fb29419a710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb28800c8f0 tx=0x7fb28800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:06.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.625+0000 7fb2917fa640 1 -- 172.21.15.102:0/1413744784 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb288004410 con 0x7fb2941962b0 2024-05-13T23:01:06.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.625+0000 7fb2917fa640 1 -- 172.21.15.102:0/1413744784 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb288007c60 con 0x7fb2941962b0 2024-05-13T23:01:06.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.625+0000 7fb29a563640 1 -- 172.21.15.102:0/1413744784 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29419acb0 con 0x7fb2941962b0 2024-05-13T23:01:06.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.625+0000 7fb2917fa640 1 -- 172.21.15.102:0/1413744784 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb28800f450 con 0x7fb2941962b0 2024-05-13T23:01:06.602 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.625+0000 7fb29a563640 1 -- 172.21.15.102:0/1413744784 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29419b1c0 con 0x7fb2941962b0 2024-05-13T23:01:06.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.626+0000 7fb29a563640 1 -- 172.21.15.102:0/1413744784 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29404e040 con 0x7fb2941962b0 2024-05-13T23:01:06.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.626+0000 7fb2917fa640 1 -- 172.21.15.102:0/1413744784 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb2880077a0 con 0x7fb2941962b0 2024-05-13T23:01:06.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.627+0000 7fb2917fa640 1 --2- 172.21.15.102:0/1413744784 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb27c06e170 0x7fb27c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:06.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.627+0000 7fb2917fa640 1 -- 172.21.15.102:0/1413744784 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb28808e2f0 con 0x7fb2941962b0 2024-05-13T23:01:06.603 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.627+0000 7fb293fff640 1 --2- 172.21.15.102:0/1413744784 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb27c06e170 0x7fb27c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:06.604 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.627+0000 7fb293fff640 1 --2- 172.21.15.102:0/1413744784 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb27c06e170 0x7fb27c070630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb284036040 tx=0x7fb284000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:06.606 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.631+0000 7fb2917fa640 1 -- 172.21.15.102:0/1413744784 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb2880574b0 con 0x7fb2941962b0 2024-05-13T23:01:06.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.843+0000 7fb29a563640 1 -- 172.21.15.102:0/1413744784 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fb29419b4a0 con 0x7fb2941962b0 2024-05-13T23:01:06.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.844+0000 7fb2917fa640 1 -- 172.21.15.102:0/1413744784 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v38) v1 ==== 107+0+4091 (secure 0 0 0) 0x7fb28805ab70 con 0x7fb2941962b0 2024-05-13T23:01:06.820 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:06.820 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24281,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/244897459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":244897459},{"type":"v1","addr":"172.21.15.130:6825","nonce":244897459}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:57:18.071027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14498},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14498":{"gid":14498,"name":"cephfs.smithi102.sdylgv","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6829/2977397703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2977397703},{"type":"v1","addr":"172.21.15.102:6829","nonce":2977397703}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14498,"qdb_cluster":[14498]},"id":1}]} 2024-05-13T23:01:06.820 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 20 2024-05-13T23:01:06.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.846+0000 7fb29a563640 1 -- 172.21.15.102:0/1413744784 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb27c06e170 msgr2=0x7fb27c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.846+0000 7fb29a563640 1 --2- 172.21.15.102:0/1413744784 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb27c06e170 0x7fb27c070630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb284036040 tx=0x7fb284000bc0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.846+0000 7fb29a563640 1 -- 172.21.15.102:0/1413744784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2941962b0 msgr2=0x7fb29419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.846+0000 7fb29a563640 1 --2- 172.21.15.102:0/1413744784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2941962b0 0x7fb29419a710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb28800c8f0 tx=0x7fb28800cdb0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.846+0000 7fb29a563640 1 -- 172.21.15.102:0/1413744784 shutdown_connections 2024-05-13T23:01:06.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.846+0000 7fb29a563640 1 --2- 172.21.15.102:0/1413744784 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb27c06e170 0x7fb27c070630 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.846+0000 7fb29a563640 1 --2- 172.21.15.102:0/1413744784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2941962b0 0x7fb29419a710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.846+0000 7fb29a563640 1 --2- 172.21.15.102:0/1413744784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb2940ff1a0 0x7fb294195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.846+0000 7fb29a563640 1 -- 172.21.15.102:0/1413744784 >> 172.21.15.102:0/1413744784 conn(0x7fb2940faa20 msgr2=0x7fb294103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:06.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.846+0000 7fb29a563640 1 -- 172.21.15.102:0/1413744784 shutdown_connections 2024-05-13T23:01:06.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.846+0000 7fb29a563640 1 -- 172.21.15.102:0/1413744784 wait complete. 2024-05-13T23:01:06.833 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-05-13T23:01:06.833 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-05-13T23:01:06.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.959+0000 7f08c235c640 1 -- 172.21.15.102:0/2144719747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08bc0ff1a0 msgr2=0x7f08bc0ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.959+0000 7f08c235c640 1 --2- 172.21.15.102:0/2144719747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08bc0ff1a0 0x7f08bc0ff580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009880 tx=0x7f08ac02bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.959+0000 7f08c235c640 1 -- 172.21.15.102:0/2144719747 shutdown_connections 2024-05-13T23:01:06.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.959+0000 7f08c235c640 1 --2- 172.21.15.102:0/2144719747 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08bc0ffb50 0x7f08bc103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.959+0000 7f08c235c640 1 --2- 172.21.15.102:0/2144719747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08bc0ff1a0 0x7f08bc0ff580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.959+0000 7f08c235c640 1 -- 172.21.15.102:0/2144719747 >> 172.21.15.102:0/2144719747 conn(0x7f08bc0faa20 msgr2=0x7f08bc0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:06.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.959+0000 7f08c235c640 1 -- 172.21.15.102:0/2144719747 shutdown_connections 2024-05-13T23:01:06.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.959+0000 7f08c235c640 1 -- 172.21.15.102:0/2144719747 wait complete. 2024-05-13T23:01:06.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.960+0000 7f08c235c640 1 Processor -- start 2024-05-13T23:01:06.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.960+0000 7f08c235c640 1 -- start start 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.960+0000 7f08c235c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08bc0ff1a0 0x7f08bc194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.960+0000 7f08c235c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08bc1952c0 0x7f08bc199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.960+0000 7f08c235c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1958c0 con 0x7f08bc0ff1a0 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.960+0000 7f08c235c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc195a30 con 0x7f08bc1952c0 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.960+0000 7f08bbfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08bc0ff1a0 0x7f08bc194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.960+0000 7f08bbfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08bc0ff1a0 0x7f08bc194d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44398/0 (socket says 172.21.15.102:44398) 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.960+0000 7f08bbfff640 1 -- 172.21.15.102:0/4115337198 learned_addr learned my addr 172.21.15.102:0/4115337198 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.961+0000 7f08bb7fe640 1 --2- 172.21.15.102:0/4115337198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08bc1952c0 0x7f08bc199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.961+0000 7f08bbfff640 1 -- 172.21.15.102:0/4115337198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08bc1952c0 msgr2=0x7f08bc199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.961+0000 7f08bbfff640 1 --2- 172.21.15.102:0/4115337198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08bc1952c0 0x7f08bc199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.961+0000 7f08bbfff640 1 -- 172.21.15.102:0/4115337198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac009560 con 0x7f08bc0ff1a0 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.961+0000 7f08bbfff640 1 --2- 172.21.15.102:0/4115337198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08bc0ff1a0 0x7f08bc194d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009430 tx=0x7f08ac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:06.938 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.961+0000 7f08b97fa640 1 -- 172.21.15.102:0/4115337198 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08ac0035f0 con 0x7f08bc0ff1a0 2024-05-13T23:01:06.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.961+0000 7f08b97fa640 1 -- 172.21.15.102:0/4115337198 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08ac003c10 con 0x7f08bc0ff1a0 2024-05-13T23:01:06.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.961+0000 7f08c235c640 1 -- 172.21.15.102:0/4115337198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc199cc0 con 0x7f08bc0ff1a0 2024-05-13T23:01:06.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.961+0000 7f08b97fa640 1 -- 172.21.15.102:0/4115337198 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08ac034440 con 0x7f08bc0ff1a0 2024-05-13T23:01:06.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.961+0000 7f08c235c640 1 -- 172.21.15.102:0/4115337198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc19a220 con 0x7f08bc0ff1a0 2024-05-13T23:01:06.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.962+0000 7f08c235c640 1 -- 172.21.15.102:0/4115337198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc04e040 con 0x7f08bc0ff1a0 2024-05-13T23:01:06.939 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.962+0000 7f08b97fa640 1 -- 172.21.15.102:0/4115337198 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f08ac004fd0 con 0x7f08bc0ff1a0 2024-05-13T23:01:06.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.963+0000 7f08b97fa640 1 --2- 172.21.15.102:0/4115337198 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f08a406e170 0x7f08a4070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:06.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.963+0000 7f08b97fa640 1 -- 172.21.15.102:0/4115337198 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f08ac0b0720 con 0x7f08bc0ff1a0 2024-05-13T23:01:06.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.963+0000 7f08bb7fe640 1 --2- 172.21.15.102:0/4115337198 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f08a406e170 0x7f08a4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:06.940 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.963+0000 7f08bb7fe640 1 --2- 172.21.15.102:0/4115337198 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f08a406e170 0x7f08a4070630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0697f0 tx=0x7f08b0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:06.942 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:06.967+0000 7f08b97fa640 1 -- 172.21.15.102:0/4115337198 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f08ac0798e0 con 0x7f08bc0ff1a0 2024-05-13T23:01:07.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.179+0000 7f08c235c640 1 -- 172.21.15.102:0/4115337198 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f08bc1074a0 con 0x7f08bc0ff1a0 2024-05-13T23:01:07.154 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.179+0000 7f08b97fa640 1 -- 172.21.15.102:0/4115337198 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v38) v1 ==== 107+0+4941 (secure 0 0 0) 0x7f08ac07cfa0 con 0x7f08bc0ff1a0 2024-05-13T23:01:07.155 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:07.155 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24281,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/244897459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":244897459},{"type":"v1","addr":"172.21.15.130:6825","nonce":244897459}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:57:18.071027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14498},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14498":{"gid":14498,"name":"cephfs.smithi102.sdylgv","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6829/2977397703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2977397703},{"type":"v1","addr":"172.21.15.102:6829","nonce":2977397703}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14498,"qdb_cluster":[14498]},"id":1}]} 2024-05-13T23:01:07.155 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 21 2024-05-13T23:01:07.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.181+0000 7f08c235c640 1 -- 172.21.15.102:0/4115337198 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f08a406e170 msgr2=0x7f08a4070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.181+0000 7f08c235c640 1 --2- 172.21.15.102:0/4115337198 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f08a406e170 0x7f08a4070630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0697f0 tx=0x7f08b0008040 comp rx=0 tx=0).stop 2024-05-13T23:01:07.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.181+0000 7f08c235c640 1 -- 172.21.15.102:0/4115337198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08bc0ff1a0 msgr2=0x7f08bc194d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.181+0000 7f08c235c640 1 --2- 172.21.15.102:0/4115337198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08bc0ff1a0 0x7f08bc194d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009430 tx=0x7f08ac004040 comp rx=0 tx=0).stop 2024-05-13T23:01:07.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.181+0000 7f08c235c640 1 -- 172.21.15.102:0/4115337198 shutdown_connections 2024-05-13T23:01:07.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.181+0000 7f08c235c640 1 --2- 172.21.15.102:0/4115337198 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f08a406e170 0x7f08a4070630 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.181+0000 7f08c235c640 1 --2- 172.21.15.102:0/4115337198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08bc1952c0 0x7f08bc199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.181+0000 7f08c235c640 1 --2- 172.21.15.102:0/4115337198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08bc0ff1a0 0x7f08bc194d80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.181+0000 7f08c235c640 1 -- 172.21.15.102:0/4115337198 >> 172.21.15.102:0/4115337198 conn(0x7f08bc0faa20 msgr2=0x7f08bc0fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:07.157 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.181+0000 7f08c235c640 1 -- 172.21.15.102:0/4115337198 shutdown_connections 2024-05-13T23:01:07.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.181+0000 7f08c235c640 1 -- 172.21.15.102:0/4115337198 wait complete. 2024-05-13T23:01:07.168 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-05-13T23:01:07.168 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-05-13T23:01:07.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.299+0000 7f9616d86640 1 -- 172.21.15.102:0/965204732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96100ff260 msgr2=0x7f96100ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.299+0000 7f9616d86640 1 --2- 172.21.15.102:0/965204732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96100ff260 0x7f96100ff640 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9604009880 tx=0x7f960402bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.299+0000 7f9616d86640 1 -- 172.21.15.102:0/965204732 shutdown_connections 2024-05-13T23:01:07.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.299+0000 7f9616d86640 1 --2- 172.21.15.102:0/965204732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96100ffc10 0x7f9610103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.299+0000 7f9616d86640 1 --2- 172.21.15.102:0/965204732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96100ff260 0x7f96100ff640 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.299+0000 7f9616d86640 1 -- 172.21.15.102:0/965204732 >> 172.21.15.102:0/965204732 conn(0x7f9610074a50 msgr2=0x7f9610074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:07.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.299+0000 7f9616d86640 1 -- 172.21.15.102:0/965204732 shutdown_connections 2024-05-13T23:01:07.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.299+0000 7f9616d86640 1 -- 172.21.15.102:0/965204732 wait complete. 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.299+0000 7f9616d86640 1 Processor -- start 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f9616d86640 1 -- start start 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f9616d86640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96100ff260 0x7f9610194cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f9616d86640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96101951f0 0x7f96101985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f9616d86640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96101957f0 con 0x7f96101951f0 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f9616d86640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9610195930 con 0x7f96100ff260 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f960ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96101951f0 0x7f96101985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f960ffff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96101951f0 0x7f96101985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44408/0 (socket says 172.21.15.102:44408) 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f960ffff640 1 -- 172.21.15.102:0/1994059230 learned_addr learned my addr 172.21.15.102:0/1994059230 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f9614b22640 1 --2- 172.21.15.102:0/1994059230 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96100ff260 0x7f9610194cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f960ffff640 1 -- 172.21.15.102:0/1994059230 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96100ff260 msgr2=0x7f9610194cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f960ffff640 1 --2- 172.21.15.102:0/1994059230 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96100ff260 0x7f9610194cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f960ffff640 1 -- 172.21.15.102:0/1994059230 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9604009560 con 0x7f96101951f0 2024-05-13T23:01:07.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.300+0000 7f960ffff640 1 --2- 172.21.15.102:0/1994059230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96101951f0 0x7f96101985a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00d6c0 tx=0x7f95fc00db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:07.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.301+0000 7f960dffb640 1 -- 172.21.15.102:0/1994059230 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95fc0051f0 con 0x7f96101951f0 2024-05-13T23:01:07.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.301+0000 7f960dffb640 1 -- 172.21.15.102:0/1994059230 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95fc009ce0 con 0x7f96101951f0 2024-05-13T23:01:07.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.301+0000 7f9616d86640 1 -- 172.21.15.102:0/1994059230 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9610198ba0 con 0x7f96101951f0 2024-05-13T23:01:07.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.301+0000 7f960dffb640 1 -- 172.21.15.102:0/1994059230 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95fc010460 con 0x7f96101951f0 2024-05-13T23:01:07.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.301+0000 7f9616d86640 1 -- 172.21.15.102:0/1994059230 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9610199100 con 0x7f96101951f0 2024-05-13T23:01:07.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.302+0000 7f9616d86640 1 -- 172.21.15.102:0/1994059230 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961004e040 con 0x7f96101951f0 2024-05-13T23:01:07.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.302+0000 7f960dffb640 1 -- 172.21.15.102:0/1994059230 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f95fc005350 con 0x7f96101951f0 2024-05-13T23:01:07.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.303+0000 7f960dffb640 1 --2- 172.21.15.102:0/1994059230 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f960006e170 0x7f9600070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:07.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.303+0000 7f960dffb640 1 -- 172.21.15.102:0/1994059230 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f95fc08dcf0 con 0x7f96101951f0 2024-05-13T23:01:07.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.303+0000 7f9614b22640 1 --2- 172.21.15.102:0/1994059230 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f960006e170 0x7f9600070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:07.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.303+0000 7f9614b22640 1 --2- 172.21.15.102:0/1994059230 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f960006e170 0x7f9600070630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9604006010 tx=0x7f9604005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:07.282 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.307+0000 7f960dffb640 1 -- 172.21.15.102:0/1994059230 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f95fc056f60 con 0x7f96101951f0 2024-05-13T23:01:07.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:07 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1324263160' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-13T23:01:07.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:07 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1413744784' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-13T23:01:07.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:07 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/4115337198' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-13T23:01:07.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.526+0000 7f9616d86640 1 -- 172.21.15.102:0/1994059230 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f9610195fc0 con 0x7f96101951f0 2024-05-13T23:01:07.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.526+0000 7f960dffb640 1 -- 172.21.15.102:0/1994059230 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v38) v1 ==== 107+0+4140 (secure 0 0 0) 0x7f95fc05a620 con 0x7f96101951f0 2024-05-13T23:01:07.502 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:07.502 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":24281,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/244897459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":244897459},{"type":"v1","addr":"172.21.15.130:6825","nonce":244897459}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:57:41.438793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-13T23:01:07.502 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 22 2024-05-13T23:01:07.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.528+0000 7f9616d86640 1 -- 172.21.15.102:0/1994059230 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f960006e170 msgr2=0x7f9600070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.528+0000 7f9616d86640 1 --2- 172.21.15.102:0/1994059230 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f960006e170 0x7f9600070630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9604006010 tx=0x7f9604005dd0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.528+0000 7f9616d86640 1 -- 172.21.15.102:0/1994059230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96101951f0 msgr2=0x7f96101985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.528+0000 7f9616d86640 1 --2- 172.21.15.102:0/1994059230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96101951f0 0x7f96101985a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00d6c0 tx=0x7f95fc00db80 comp rx=0 tx=0).stop 2024-05-13T23:01:07.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.528+0000 7f9616d86640 1 -- 172.21.15.102:0/1994059230 shutdown_connections 2024-05-13T23:01:07.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.528+0000 7f9616d86640 1 --2- 172.21.15.102:0/1994059230 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f960006e170 0x7f9600070630 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.528+0000 7f9616d86640 1 --2- 172.21.15.102:0/1994059230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f96101951f0 0x7f96101985a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.529+0000 7f9616d86640 1 --2- 172.21.15.102:0/1994059230 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f96100ff260 0x7f9610194cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.529+0000 7f9616d86640 1 -- 172.21.15.102:0/1994059230 >> 172.21.15.102:0/1994059230 conn(0x7f9610074a50 msgr2=0x7f96100fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:07.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.529+0000 7f9616d86640 1 -- 172.21.15.102:0/1994059230 shutdown_connections 2024-05-13T23:01:07.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.529+0000 7f9616d86640 1 -- 172.21.15.102:0/1994059230 wait complete. 2024-05-13T23:01:07.515 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-05-13T23:01:07.515 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-05-13T23:01:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.657+0000 7fbd28f7c640 1 -- 172.21.15.102:0/740135426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd240675c0 msgr2=0x7fbd240679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.657+0000 7fbd28f7c640 1 --2- 172.21.15.102:0/740135426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd240675c0 0x7fbd240679a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009880 tx=0x7fbd1402bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.657+0000 7fbd28f7c640 1 -- 172.21.15.102:0/740135426 shutdown_connections 2024-05-13T23:01:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.657+0000 7fbd28f7c640 1 --2- 172.21.15.102:0/740135426 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd24067f70 0x7fbd24104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.657+0000 7fbd28f7c640 1 --2- 172.21.15.102:0/740135426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd240675c0 0x7fbd240679a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.634 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.657+0000 7fbd28f7c640 1 -- 172.21.15.102:0/740135426 >> 172.21.15.102:0/740135426 conn(0x7fbd24074a50 msgr2=0x7fbd24074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.657+0000 7fbd28f7c640 1 -- 172.21.15.102:0/740135426 shutdown_connections 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.657+0000 7fbd28f7c640 1 -- 172.21.15.102:0/740135426 wait complete. 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd28f7c640 1 Processor -- start 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd28f7c640 1 -- start start 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd28f7c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd240675c0 0x7fbd24195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd28f7c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd241962d0 0x7fbd2419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd28f7c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd241968d0 con 0x7fbd240675c0 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd28f7c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd24196a40 con 0x7fbd241962d0 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd2259c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd240675c0 0x7fbd24195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd2259c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd240675c0 0x7fbd24195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44430/0 (socket says 172.21.15.102:44430) 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd2259c640 1 -- 172.21.15.102:0/3062298521 learned_addr learned my addr 172.21.15.102:0/3062298521 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd2259c640 1 -- 172.21.15.102:0/3062298521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd241962d0 msgr2=0x7fbd2419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd2259c640 1 --2- 172.21.15.102:0/3062298521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd241962d0 0x7fbd2419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd2259c640 1 -- 172.21.15.102:0/3062298521 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd14009560 con 0x7fbd240675c0 2024-05-13T23:01:07.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.658+0000 7fbd2259c640 1 --2- 172.21.15.102:0/3062298521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd240675c0 0x7fbd24195d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009430 tx=0x7fbd14004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:07.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.659+0000 7fbd137fe640 1 -- 172.21.15.102:0/3062298521 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd140035f0 con 0x7fbd240675c0 2024-05-13T23:01:07.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.659+0000 7fbd28f7c640 1 -- 172.21.15.102:0/3062298521 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2419acd0 con 0x7fbd240675c0 2024-05-13T23:01:07.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.659+0000 7fbd137fe640 1 -- 172.21.15.102:0/3062298521 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd14003c10 con 0x7fbd240675c0 2024-05-13T23:01:07.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.659+0000 7fbd137fe640 1 -- 172.21.15.102:0/3062298521 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd14005670 con 0x7fbd240675c0 2024-05-13T23:01:07.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.659+0000 7fbd28f7c640 1 -- 172.21.15.102:0/3062298521 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2419b230 con 0x7fbd240675c0 2024-05-13T23:01:07.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.660+0000 7fbd28f7c640 1 -- 172.21.15.102:0/3062298521 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd2404e040 con 0x7fbd240675c0 2024-05-13T23:01:07.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.660+0000 7fbd137fe640 1 -- 172.21.15.102:0/3062298521 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbd14004dd0 con 0x7fbd240675c0 2024-05-13T23:01:07.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.661+0000 7fbd137fe640 1 --2- 172.21.15.102:0/3062298521 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbd0c06e120 0x7fbd0c0705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:07.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.661+0000 7fbd137fe640 1 -- 172.21.15.102:0/3062298521 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fbd140b06e0 con 0x7fbd240675c0 2024-05-13T23:01:07.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.661+0000 7fbd21d9b640 1 --2- 172.21.15.102:0/3062298521 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbd0c06e120 0x7fbd0c0705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:07.637 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.661+0000 7fbd21d9b640 1 --2- 172.21.15.102:0/3062298521 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbd0c06e120 0x7fbd0c0705e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbd18005fd0 tx=0x7fbd18005c70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:07.640 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.665+0000 7fbd137fe640 1 -- 172.21.15.102:0/3062298521 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbd140798a0 con 0x7fbd240675c0 2024-05-13T23:01:07.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:07 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1324263160' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-13T23:01:07.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:07 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1413744784' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-13T23:01:07.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:07 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/4115337198' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-13T23:01:07.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.885+0000 7fbd28f7c640 1 -- 172.21.15.102:0/3062298521 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fbd2419b510 con 0x7fbd240675c0 2024-05-13T23:01:07.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.886+0000 7fbd137fe640 1 -- 172.21.15.102:0/3062298521 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v38) v1 ==== 107+0+4151 (secure 0 0 0) 0x7fbd14079030 con 0x7fbd240675c0 2024-05-13T23:01:07.861 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:07.862 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/244897459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":244897459},{"type":"v1","addr":"172.21.15.130:6825","nonce":244897459}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:57:41.443332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":0,"incarnation":23,"state":"up:replay","state_seq":2,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-13T23:01:07.862 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 23 2024-05-13T23:01:07.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.888+0000 7fbd28f7c640 1 -- 172.21.15.102:0/3062298521 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbd0c06e120 msgr2=0x7fbd0c0705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.888+0000 7fbd28f7c640 1 --2- 172.21.15.102:0/3062298521 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbd0c06e120 0x7fbd0c0705e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbd18005fd0 tx=0x7fbd18005c70 comp rx=0 tx=0).stop 2024-05-13T23:01:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.888+0000 7fbd28f7c640 1 -- 172.21.15.102:0/3062298521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd240675c0 msgr2=0x7fbd24195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.888+0000 7fbd28f7c640 1 --2- 172.21.15.102:0/3062298521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd240675c0 0x7fbd24195d90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009430 tx=0x7fbd14004040 comp rx=0 tx=0).stop 2024-05-13T23:01:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.888+0000 7fbd28f7c640 1 -- 172.21.15.102:0/3062298521 shutdown_connections 2024-05-13T23:01:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.888+0000 7fbd28f7c640 1 --2- 172.21.15.102:0/3062298521 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbd0c06e120 0x7fbd0c0705e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.888+0000 7fbd28f7c640 1 --2- 172.21.15.102:0/3062298521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd241962d0 0x7fbd2419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.888+0000 7fbd28f7c640 1 --2- 172.21.15.102:0/3062298521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd240675c0 0x7fbd24195d90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.888+0000 7fbd28f7c640 1 -- 172.21.15.102:0/3062298521 >> 172.21.15.102:0/3062298521 conn(0x7fbd24074a50 msgr2=0x7fbd240fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.888+0000 7fbd28f7c640 1 -- 172.21.15.102:0/3062298521 shutdown_connections 2024-05-13T23:01:07.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:07.888+0000 7fbd28f7c640 1 -- 172.21.15.102:0/3062298521 wait complete. 2024-05-13T23:01:07.875 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-05-13T23:01:07.875 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-05-13T23:01:07.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.001+0000 7f03fd5c3640 1 -- 172.21.15.102:0/933793433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f8067f70 msgr2=0x7f03f8104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.001+0000 7f03fd5c3640 1 --2- 172.21.15.102:0/933793433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f8067f70 0x7f03f8104c70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009880 tx=0x7f03e802bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.001+0000 7f03fd5c3640 1 -- 172.21.15.102:0/933793433 shutdown_connections 2024-05-13T23:01:07.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.001+0000 7f03fd5c3640 1 --2- 172.21.15.102:0/933793433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f8067f70 0x7f03f8104c70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.001+0000 7f03fd5c3640 1 --2- 172.21.15.102:0/933793433 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03f80675c0 0x7f03f80679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.979 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.001+0000 7f03fd5c3640 1 -- 172.21.15.102:0/933793433 >> 172.21.15.102:0/933793433 conn(0x7f03f8074a50 msgr2=0x7f03f8074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.002+0000 7f03fd5c3640 1 -- 172.21.15.102:0/933793433 shutdown_connections 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.002+0000 7f03fd5c3640 1 -- 172.21.15.102:0/933793433 wait complete. 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.002+0000 7f03fd5c3640 1 Processor -- start 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.002+0000 7f03fd5c3640 1 -- start start 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.002+0000 7f03fd5c3640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03f80675c0 0x7f03f8195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.002+0000 7f03fd5c3640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f81962c0 0x7f03f819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.002+0000 7f03fd5c3640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f81968c0 con 0x7f03f81962c0 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.002+0000 7f03fd5c3640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f8196a30 con 0x7f03f80675c0 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03f67fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f81962c0 0x7f03f819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03f67fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f81962c0 0x7f03f819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44450/0 (socket says 172.21.15.102:44450) 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03f67fc640 1 -- 172.21.15.102:0/2683319377 learned_addr learned my addr 172.21.15.102:0/2683319377 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03f6ffd640 1 --2- 172.21.15.102:0/2683319377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03f80675c0 0x7f03f8195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:07.980 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03f67fc640 1 -- 172.21.15.102:0/2683319377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03f80675c0 msgr2=0x7f03f8195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:07.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03f67fc640 1 --2- 172.21.15.102:0/2683319377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03f80675c0 0x7f03f8195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:07.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03f67fc640 1 -- 172.21.15.102:0/2683319377 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e8009560 con 0x7f03f81962c0 2024-05-13T23:01:07.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03f67fc640 1 --2- 172.21.15.102:0/2683319377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f81962c0 0x7f03f819a720 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f03e8036040 tx=0x7f03e802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:07.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03dffff640 1 -- 172.21.15.102:0/2683319377 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e80035f0 con 0x7f03f81962c0 2024-05-13T23:01:07.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03fd5c3640 1 -- 172.21.15.102:0/2683319377 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f819acc0 con 0x7f03f81962c0 2024-05-13T23:01:07.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03dffff640 1 -- 172.21.15.102:0/2683319377 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03e8003c10 con 0x7f03f81962c0 2024-05-13T23:01:07.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03dffff640 1 -- 172.21.15.102:0/2683319377 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e8034440 con 0x7f03f81962c0 2024-05-13T23:01:07.981 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.003+0000 7f03fd5c3640 1 -- 172.21.15.102:0/2683319377 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f819b220 con 0x7f03f81962c0 2024-05-13T23:01:07.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.004+0000 7f03fd5c3640 1 -- 172.21.15.102:0/2683319377 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f804e040 con 0x7f03f81962c0 2024-05-13T23:01:07.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.005+0000 7f03dffff640 1 -- 172.21.15.102:0/2683319377 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f03e8003750 con 0x7f03f81962c0 2024-05-13T23:01:07.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.005+0000 7f03dffff640 1 --2- 172.21.15.102:0/2683319377 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f03e006e170 0x7f03e0070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:07.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.005+0000 7f03dffff640 1 -- 172.21.15.102:0/2683319377 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f03e80b05c0 con 0x7f03f81962c0 2024-05-13T23:01:07.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.005+0000 7f03f6ffd640 1 --2- 172.21.15.102:0/2683319377 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f03e006e170 0x7f03e0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:07.982 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.006+0000 7f03f6ffd640 1 --2- 172.21.15.102:0/2683319377 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f03e006e170 0x7f03e0070630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f03ec005fd0 tx=0x7f03ec00b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:07.985 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.010+0000 7f03dffff640 1 -- 172.21.15.102:0/2683319377 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f03e8079830 con 0x7f03f81962c0 2024-05-13T23:01:08.224 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.249+0000 7f03fd5c3640 1 -- 172.21.15.102:0/2683319377 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f03f819b440 con 0x7f03f81962c0 2024-05-13T23:01:08.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.250+0000 7f03dffff640 1 -- 172.21.15.102:0/2683319377 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v38) v1 ==== 107+0+4156 (secure 0 0 0) 0x7f03e807cef0 con 0x7f03f81962c0 2024-05-13T23:01:08.225 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:08.225 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/244897459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":244897459},{"type":"v1","addr":"172.21.15.130:6825","nonce":244897459}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:57:50.234051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":0,"incarnation":23,"state":"up:reconnect","state_seq":188,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-13T23:01:08.225 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 24 2024-05-13T23:01:08.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.252+0000 7f03fd5c3640 1 -- 172.21.15.102:0/2683319377 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f03e006e170 msgr2=0x7f03e0070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:08.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.252+0000 7f03fd5c3640 1 --2- 172.21.15.102:0/2683319377 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f03e006e170 0x7f03e0070630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f03ec005fd0 tx=0x7f03ec00b390 comp rx=0 tx=0).stop 2024-05-13T23:01:08.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.252+0000 7f03fd5c3640 1 -- 172.21.15.102:0/2683319377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f81962c0 msgr2=0x7f03f819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:08.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.252+0000 7f03fd5c3640 1 --2- 172.21.15.102:0/2683319377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f81962c0 0x7f03f819a720 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f03e8036040 tx=0x7f03e802c850 comp rx=0 tx=0).stop 2024-05-13T23:01:08.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.252+0000 7f03fd5c3640 1 -- 172.21.15.102:0/2683319377 shutdown_connections 2024-05-13T23:01:08.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.252+0000 7f03fd5c3640 1 --2- 172.21.15.102:0/2683319377 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f03e006e170 0x7f03e0070630 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.252+0000 7f03fd5c3640 1 --2- 172.21.15.102:0/2683319377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f03f81962c0 0x7f03f819a720 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.252+0000 7f03fd5c3640 1 --2- 172.21.15.102:0/2683319377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03f80675c0 0x7f03f8195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.252+0000 7f03fd5c3640 1 -- 172.21.15.102:0/2683319377 >> 172.21.15.102:0/2683319377 conn(0x7f03f8074a50 msgr2=0x7f03f80fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:08.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.252+0000 7f03fd5c3640 1 -- 172.21.15.102:0/2683319377 shutdown_connections 2024-05-13T23:01:08.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.252+0000 7f03fd5c3640 1 -- 172.21.15.102:0/2683319377 wait complete. 2024-05-13T23:01:08.239 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-05-13T23:01:08.239 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-05-13T23:01:08.344 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.367+0000 7fb4a17c5640 1 -- 172.21.15.102:0/3235021120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb49c067350 msgr2=0x7fb49c067730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.367+0000 7fb4a17c5640 1 --2- 172.21.15.102:0/3235021120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb49c067350 0x7fb49c067730 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009880 tx=0x7fb48c02bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.367+0000 7fb4a17c5640 1 -- 172.21.15.102:0/3235021120 shutdown_connections 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.367+0000 7fb4a17c5640 1 --2- 172.21.15.102:0/3235021120 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb49c067c70 0x7fb49c10c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.367+0000 7fb4a17c5640 1 --2- 172.21.15.102:0/3235021120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb49c067350 0x7fb49c067730 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.367+0000 7fb4a17c5640 1 -- 172.21.15.102:0/3235021120 >> 172.21.15.102:0/3235021120 conn(0x7fb49c074a50 msgr2=0x7fb49c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.367+0000 7fb4a17c5640 1 -- 172.21.15.102:0/3235021120 shutdown_connections 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.367+0000 7fb4a17c5640 1 -- 172.21.15.102:0/3235021120 wait complete. 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.368+0000 7fb4a17c5640 1 Processor -- start 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.368+0000 7fb4a17c5640 1 -- start start 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.368+0000 7fb4a17c5640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb49c067350 0x7fb49c197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.368+0000 7fb4a17c5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb49c067c70 0x7fb49c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.368+0000 7fb4a17c5640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c198b80 con 0x7fb49c067350 2024-05-13T23:01:08.345 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.368+0000 7fb4a17c5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c19c950 con 0x7fb49c067c70 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.368+0000 7fb49affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb49c067350 0x7fb49c197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.368+0000 7fb49affd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb49c067350 0x7fb49c197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:44470/0 (socket says 172.21.15.102:44470) 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.368+0000 7fb49affd640 1 -- 172.21.15.102:0/754319570 learned_addr learned my addr 172.21.15.102:0/754319570 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.369+0000 7fb49a7fc640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb49c067c70 0x7fb49c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.369+0000 7fb49affd640 1 -- 172.21.15.102:0/754319570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb49c067c70 msgr2=0x7fb49c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.369+0000 7fb49affd640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb49c067c70 0x7fb49c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.369+0000 7fb49affd640 1 -- 172.21.15.102:0/754319570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c009560 con 0x7fb49c067350 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.369+0000 7fb49a7fc640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb49c067c70 0x7fb49c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.369+0000 7fb49affd640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb49c067350 0x7fb49c197fb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009230 tx=0x7fb48c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.369+0000 7fb483fff640 1 -- 172.21.15.102:0/754319570 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb48c0044a0 con 0x7fb49c067350 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.369+0000 7fb483fff640 1 -- 172.21.15.102:0/754319570 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb48c003ab0 con 0x7fb49c067350 2024-05-13T23:01:08.346 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.369+0000 7fb4a17c5640 1 -- 172.21.15.102:0/754319570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c19cbd0 con 0x7fb49c067350 2024-05-13T23:01:08.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.369+0000 7fb483fff640 1 -- 172.21.15.102:0/754319570 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb48c0055f0 con 0x7fb49c067350 2024-05-13T23:01:08.347 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.369+0000 7fb4a17c5640 1 -- 172.21.15.102:0/754319570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c19d0d0 con 0x7fb49c067350 2024-05-13T23:01:08.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.371+0000 7fb4a17c5640 1 -- 172.21.15.102:0/754319570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c04e040 con 0x7fb49c067350 2024-05-13T23:01:08.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.371+0000 7fb483fff640 1 -- 172.21.15.102:0/754319570 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb48c0035f0 con 0x7fb49c067350 2024-05-13T23:01:08.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.371+0000 7fb483fff640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb48406e170 0x7fb484070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:08.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.372+0000 7fb483fff640 1 -- 172.21.15.102:0/754319570 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb48c0b05d0 con 0x7fb49c067350 2024-05-13T23:01:08.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.372+0000 7fb49a7fc640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb48406e170 0x7fb484070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:08.348 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.372+0000 7fb49a7fc640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb48406e170 0x7fb484070630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb49c1995c0 tx=0x7fb490005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:08.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.376+0000 7fb483fff640 1 -- 172.21.15.102:0/754319570 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fb48c079840 con 0x7fb49c067350 2024-05-13T23:01:08.560 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:08 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1994059230' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-13T23:01:08.560 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:08 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3062298521' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-13T23:01:08.560 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:08 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:01:08.560 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:08 smithi130 ceph-mon[84265]: pgmap v300: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:08.561 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:08 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2683319377' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-13T23:01:08.585 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.610+0000 7fb4a17c5640 1 -- 172.21.15.102:0/754319570 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fb49c10f4d0 con 0x7fb49c067350 2024-05-13T23:01:08.587 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.612+0000 7fb483fff640 1 -- 172.21.15.102:0/754319570 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v38) v1 ==== 107+0+4153 (secure 0 0 0) 0x7fb48c07cf00 con 0x7fb49c067350 2024-05-13T23:01:08.587 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:08.588 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/244897459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":244897459},{"type":"v1","addr":"172.21.15.130:6825","nonce":244897459}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:57:51.093410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":0,"incarnation":23,"state":"up:rejoin","state_seq":189,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-13T23:01:08.588 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 25 2024-05-13T23:01:08.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.615+0000 7fb4a17c5640 1 -- 172.21.15.102:0/754319570 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb48406e170 msgr2=0x7fb484070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:08.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.615+0000 7fb4a17c5640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb48406e170 0x7fb484070630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb49c1995c0 tx=0x7fb490005f90 comp rx=0 tx=0).stop 2024-05-13T23:01:08.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.615+0000 7fb4a17c5640 1 -- 172.21.15.102:0/754319570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb49c067350 msgr2=0x7fb49c197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:08.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.615+0000 7fb4a17c5640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb49c067350 0x7fb49c197fb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009230 tx=0x7fb48c005e50 comp rx=0 tx=0).stop 2024-05-13T23:01:08.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.615+0000 7fb4a17c5640 1 -- 172.21.15.102:0/754319570 shutdown_connections 2024-05-13T23:01:08.590 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.615+0000 7fb4a17c5640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fb48406e170 0x7fb484070630 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.615+0000 7fb4a17c5640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb49c067c70 0x7fb49c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.615+0000 7fb4a17c5640 1 --2- 172.21.15.102:0/754319570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb49c067350 0x7fb49c197fb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.615+0000 7fb4a17c5640 1 -- 172.21.15.102:0/754319570 >> 172.21.15.102:0/754319570 conn(0x7fb49c074a50 msgr2=0x7fb49c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:08.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.615+0000 7fb4a17c5640 1 -- 172.21.15.102:0/754319570 shutdown_connections 2024-05-13T23:01:08.591 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.615+0000 7fb4a17c5640 1 -- 172.21.15.102:0/754319570 wait complete. 2024-05-13T23:01:08.600 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:08 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1994059230' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-13T23:01:08.600 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:08 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3062298521' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-13T23:01:08.600 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:08 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:01:08.601 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:08 smithi102 ceph-mon[111694]: pgmap v300: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:08.601 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:08 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2683319377' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-13T23:01:08.602 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-05-13T23:01:08.602 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-05-13T23:01:08.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.731+0000 7f8e626af640 1 -- 172.21.15.102:0/2826635700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e5c0feff0 msgr2=0x7f8e5c0ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:08.708 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.731+0000 7f8e626af640 1 --2- 172.21.15.102:0/2826635700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e5c0feff0 0x7f8e5c0ff3d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009880 tx=0x7f8e4c02bd70 comp rx=0 tx=0).stop 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.731+0000 7f8e626af640 1 -- 172.21.15.102:0/2826635700 shutdown_connections 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.731+0000 7f8e626af640 1 --2- 172.21.15.102:0/2826635700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e5c0ff910 0x7f8e5c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.731+0000 7f8e626af640 1 --2- 172.21.15.102:0/2826635700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e5c0feff0 0x7f8e5c0ff3d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.731+0000 7f8e626af640 1 -- 172.21.15.102:0/2826635700 >> 172.21.15.102:0/2826635700 conn(0x7f8e5c0faa40 msgr2=0x7f8e5c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.731+0000 7f8e626af640 1 -- 172.21.15.102:0/2826635700 shutdown_connections 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.731+0000 7f8e626af640 1 -- 172.21.15.102:0/2826635700 wait complete. 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.732+0000 7f8e626af640 1 Processor -- start 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.732+0000 7f8e626af640 1 -- start start 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.732+0000 7f8e626af640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e5c0feff0 0x7f8e5c19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.732+0000 7f8e626af640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e5c0ff910 0x7f8e5c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.732+0000 7f8e626af640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c19cee0 con 0x7f8e5c0feff0 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.732+0000 7f8e626af640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c1a0cb0 con 0x7f8e5c0ff910 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.732+0000 7f8e5bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e5c0feff0 0x7f8e5c19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:08.709 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.732+0000 7f8e5bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e5c0feff0 0x7f8e5c19c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47364/0 (socket says 172.21.15.102:47364) 2024-05-13T23:01:08.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.732+0000 7f8e5bfff640 1 -- 172.21.15.102:0/1678560646 learned_addr learned my addr 172.21.15.102:0/1678560646 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:08.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.732+0000 7f8e5b7fe640 1 --2- 172.21.15.102:0/1678560646 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e5c0ff910 0x7f8e5c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:08.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.733+0000 7f8e5bfff640 1 -- 172.21.15.102:0/1678560646 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e5c0ff910 msgr2=0x7f8e5c19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:08.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.733+0000 7f8e5bfff640 1 --2- 172.21.15.102:0/1678560646 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e5c0ff910 0x7f8e5c19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.733+0000 7f8e5bfff640 1 -- 172.21.15.102:0/1678560646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c009560 con 0x7f8e5c0feff0 2024-05-13T23:01:08.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.733+0000 7f8e5bfff640 1 --2- 172.21.15.102:0/1678560646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e5c0feff0 0x7f8e5c19c310 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009230 tx=0x7f8e4c02c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:08.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.733+0000 7f8e597fa640 1 -- 172.21.15.102:0/1678560646 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e4c0044a0 con 0x7f8e5c0feff0 2024-05-13T23:01:08.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.733+0000 7f8e597fa640 1 -- 172.21.15.102:0/1678560646 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e4c003ab0 con 0x7f8e5c0feff0 2024-05-13T23:01:08.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.733+0000 7f8e626af640 1 -- 172.21.15.102:0/1678560646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c1a0f30 con 0x7f8e5c0feff0 2024-05-13T23:01:08.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.733+0000 7f8e597fa640 1 -- 172.21.15.102:0/1678560646 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e4c005600 con 0x7f8e5c0feff0 2024-05-13T23:01:08.710 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.733+0000 7f8e626af640 1 -- 172.21.15.102:0/1678560646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c1a1490 con 0x7f8e5c0feff0 2024-05-13T23:01:08.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.734+0000 7f8e626af640 1 -- 172.21.15.102:0/1678560646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c04e040 con 0x7f8e5c0feff0 2024-05-13T23:01:08.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.734+0000 7f8e597fa640 1 -- 172.21.15.102:0/1678560646 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8e4c004dd0 con 0x7f8e5c0feff0 2024-05-13T23:01:08.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.735+0000 7f8e597fa640 1 --2- 172.21.15.102:0/1678560646 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8e4406e170 0x7f8e44070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:08.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.735+0000 7f8e597fa640 1 -- 172.21.15.102:0/1678560646 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f8e4c0b0d30 con 0x7f8e5c0feff0 2024-05-13T23:01:08.711 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.735+0000 7f8e5b7fe640 1 --2- 172.21.15.102:0/1678560646 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8e4406e170 0x7f8e44070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:08.712 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.735+0000 7f8e5b7fe640 1 --2- 172.21.15.102:0/1678560646 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8e4406e170 0x7f8e44070630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c19d920 tx=0x7f8e50005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:08.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.739+0000 7f8e597fa640 1 -- 172.21.15.102:0/1678560646 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f8e4c07a010 con 0x7f8e5c0feff0 2024-05-13T23:01:08.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.950+0000 7f8e626af640 1 -- 172.21.15.102:0/1678560646 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f8e5c101480 con 0x7f8e5c0feff0 2024-05-13T23:01:08.925 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.950+0000 7f8e597fa640 1 -- 172.21.15.102:0/1678560646 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v38) v1 ==== 107+0+5012 (secure 0 0 0) 0x7f8e4c043090 con 0x7f8e5c0feff0 2024-05-13T23:01:08.926 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:08.926 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24281,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/244897459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":244897459},{"type":"v1","addr":"172.21.15.130:6825","nonce":244897459}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:57:53.095951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":0,"incarnation":23,"state":"up:active","state_seq":190,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24275,"qdb_cluster":[24275]},"id":1}]} 2024-05-13T23:01:08.926 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 26 2024-05-13T23:01:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.952+0000 7f8e626af640 1 -- 172.21.15.102:0/1678560646 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8e4406e170 msgr2=0x7f8e44070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.952+0000 7f8e626af640 1 --2- 172.21.15.102:0/1678560646 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8e4406e170 0x7f8e44070630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c19d920 tx=0x7f8e50005f90 comp rx=0 tx=0).stop 2024-05-13T23:01:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.952+0000 7f8e626af640 1 -- 172.21.15.102:0/1678560646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e5c0feff0 msgr2=0x7f8e5c19c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.952+0000 7f8e626af640 1 --2- 172.21.15.102:0/1678560646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e5c0feff0 0x7f8e5c19c310 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009230 tx=0x7f8e4c02c7a0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.952+0000 7f8e626af640 1 -- 172.21.15.102:0/1678560646 shutdown_connections 2024-05-13T23:01:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.952+0000 7f8e626af640 1 --2- 172.21.15.102:0/1678560646 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f8e4406e170 0x7f8e44070630 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.952+0000 7f8e626af640 1 --2- 172.21.15.102:0/1678560646 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8e5c0ff910 0x7f8e5c19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.952+0000 7f8e626af640 1 --2- 172.21.15.102:0/1678560646 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e5c0feff0 0x7f8e5c19c310 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:08.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.952+0000 7f8e626af640 1 -- 172.21.15.102:0/1678560646 >> 172.21.15.102:0/1678560646 conn(0x7f8e5c0faa40 msgr2=0x7f8e5c110450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:08.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.953+0000 7f8e626af640 1 -- 172.21.15.102:0/1678560646 shutdown_connections 2024-05-13T23:01:08.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:08.953+0000 7f8e626af640 1 -- 172.21.15.102:0/1678560646 wait complete. 2024-05-13T23:01:08.939 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-05-13T23:01:08.939 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-05-13T23:01:09.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.065+0000 7f10fc3bc640 1 -- 172.21.15.102:0/773148957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10f40ff260 msgr2=0x7f10f40ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.065+0000 7f10fc3bc640 1 --2- 172.21.15.102:0/773148957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10f40ff260 0x7f10f40ff640 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009880 tx=0x7f10f002bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.065+0000 7f10fc3bc640 1 -- 172.21.15.102:0/773148957 shutdown_connections 2024-05-13T23:01:09.042 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.065+0000 7f10fc3bc640 1 --2- 172.21.15.102:0/773148957 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10f40ffc10 0x7f10f4103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.065+0000 7f10fc3bc640 1 --2- 172.21.15.102:0/773148957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10f40ff260 0x7f10f40ff640 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.065+0000 7f10fc3bc640 1 -- 172.21.15.102:0/773148957 >> 172.21.15.102:0/773148957 conn(0x7f10f4074a50 msgr2=0x7f10f4074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.065+0000 7f10fc3bc640 1 -- 172.21.15.102:0/773148957 shutdown_connections 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.065+0000 7f10fc3bc640 1 -- 172.21.15.102:0/773148957 wait complete. 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10fc3bc640 1 Processor -- start 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10fc3bc640 1 -- start start 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10fc3bc640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10f40ff260 0x7f10f4194cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10fc3bc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10f41951f0 0x7f10f41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10fc3bc640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f41957f0 con 0x7f10f41951f0 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10fc3bc640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f4195930 con 0x7f10f40ff260 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10f9957640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10f41951f0 0x7f10f41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10f9957640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10f41951f0 0x7f10f41985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47384/0 (socket says 172.21.15.102:47384) 2024-05-13T23:01:09.043 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10f9957640 1 -- 172.21.15.102:0/2534598468 learned_addr learned my addr 172.21.15.102:0/2534598468 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:09.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10fa158640 1 --2- 172.21.15.102:0/2534598468 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10f40ff260 0x7f10f4194cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:09.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10f9957640 1 -- 172.21.15.102:0/2534598468 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10f40ff260 msgr2=0x7f10f4194cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10f9957640 1 --2- 172.21.15.102:0/2534598468 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10f40ff260 0x7f10f4194cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10f9957640 1 -- 172.21.15.102:0/2534598468 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0009560 con 0x7f10f41951f0 2024-05-13T23:01:09.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10f9957640 1 --2- 172.21.15.102:0/2534598468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10f41951f0 0x7f10f41985a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f10e400e970 tx=0x7f10e400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:09.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10eb7fe640 1 -- 172.21.15.102:0/2534598468 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10e400cc90 con 0x7f10f41951f0 2024-05-13T23:01:09.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10eb7fe640 1 -- 172.21.15.102:0/2534598468 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10e400cdf0 con 0x7f10f41951f0 2024-05-13T23:01:09.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10eb7fe640 1 -- 172.21.15.102:0/2534598468 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10e4010430 con 0x7f10f41951f0 2024-05-13T23:01:09.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10fc3bc640 1 -- 172.21.15.102:0/2534598468 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f4198ba0 con 0x7f10f41951f0 2024-05-13T23:01:09.044 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.066+0000 7f10fc3bc640 1 -- 172.21.15.102:0/2534598468 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f4199100 con 0x7f10f41951f0 2024-05-13T23:01:09.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.067+0000 7f10fc3bc640 1 -- 172.21.15.102:0/2534598468 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f4199430 con 0x7f10f41951f0 2024-05-13T23:01:09.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.068+0000 7f10eb7fe640 1 -- 172.21.15.102:0/2534598468 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f10e40046c0 con 0x7f10f41951f0 2024-05-13T23:01:09.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.068+0000 7f10eb7fe640 1 --2- 172.21.15.102:0/2534598468 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f10e006e170 0x7f10e0070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:09.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.068+0000 7f10eb7fe640 1 -- 172.21.15.102:0/2534598468 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f10e4014070 con 0x7f10f41951f0 2024-05-13T23:01:09.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.068+0000 7f10fa158640 1 --2- 172.21.15.102:0/2534598468 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f10e006e170 0x7f10e0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:09.045 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.069+0000 7f10fa158640 1 --2- 172.21.15.102:0/2534598468 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f10e006e170 0x7f10e0070630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f10f0006010 tx=0x7f10f0005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:09.048 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.073+0000 7f10eb7fe640 1 -- 172.21.15.102:0/2534598468 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f10f4199430 con 0x7f10f41951f0 2024-05-13T23:01:09.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.320+0000 7f10fc3bc640 1 -- 172.21.15.102:0/2534598468 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f10f4199430 con 0x7f10f41951f0 2024-05-13T23:01:09.295 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.320+0000 7f10eb7fe640 1 -- 172.21.15.102:0/2534598468 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v38) v1 ==== 107+0+4233 (secure 0 0 0) 0x7f10e40579d0 con 0x7f10f41951f0 2024-05-13T23:01:09.296 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:09.296 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:57:53.095951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":0,"incarnation":23,"state":"up:active","state_seq":190,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24275,"qdb_cluster":[24275]},"id":1}]} 2024-05-13T23:01:09.296 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 27 2024-05-13T23:01:09.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.322+0000 7f10fc3bc640 1 -- 172.21.15.102:0/2534598468 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f10e006e170 msgr2=0x7f10e0070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.322+0000 7f10fc3bc640 1 --2- 172.21.15.102:0/2534598468 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f10e006e170 0x7f10e0070630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f10f0006010 tx=0x7f10f0005dd0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.322+0000 7f10fc3bc640 1 -- 172.21.15.102:0/2534598468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10f41951f0 msgr2=0x7f10f41985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.322+0000 7f10fc3bc640 1 --2- 172.21.15.102:0/2534598468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10f41951f0 0x7f10f41985a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f10e400e970 tx=0x7f10e400ee30 comp rx=0 tx=0).stop 2024-05-13T23:01:09.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.322+0000 7f10fc3bc640 1 -- 172.21.15.102:0/2534598468 shutdown_connections 2024-05-13T23:01:09.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.322+0000 7f10fc3bc640 1 --2- 172.21.15.102:0/2534598468 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f10e006e170 0x7f10e0070630 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.322+0000 7f10fc3bc640 1 --2- 172.21.15.102:0/2534598468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f10f41951f0 0x7f10f41985a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.322+0000 7f10fc3bc640 1 --2- 172.21.15.102:0/2534598468 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f10f40ff260 0x7f10f4194cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.322+0000 7f10fc3bc640 1 -- 172.21.15.102:0/2534598468 >> 172.21.15.102:0/2534598468 conn(0x7f10f4074a50 msgr2=0x7f10f40fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:09.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.322+0000 7f10fc3bc640 1 -- 172.21.15.102:0/2534598468 shutdown_connections 2024-05-13T23:01:09.298 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.322+0000 7f10fc3bc640 1 -- 172.21.15.102:0/2534598468 wait complete. 2024-05-13T23:01:09.309 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-05-13T23:01:09.310 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-05-13T23:01:09.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.436+0000 7fe94f91d640 1 -- 172.21.15.102:0/1643070532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9481002e0 msgr2=0x7fe9481006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.436+0000 7fe94f91d640 1 --2- 172.21.15.102:0/1643070532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9481002e0 0x7fe9481006c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe944009c20 tx=0x7fe94402c1b0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.436+0000 7fe94f91d640 1 -- 172.21.15.102:0/1643070532 shutdown_connections 2024-05-13T23:01:09.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.436+0000 7fe94f91d640 1 --2- 172.21.15.102:0/1643070532 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe948100c90 0x7fe948104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.436+0000 7fe94f91d640 1 --2- 172.21.15.102:0/1643070532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9481002e0 0x7fe9481006c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.436+0000 7fe94f91d640 1 -- 172.21.15.102:0/1643070532 >> 172.21.15.102:0/1643070532 conn(0x7fe9480fbb80 msgr2=0x7fe9480fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:09.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.436+0000 7fe94f91d640 1 -- 172.21.15.102:0/1643070532 shutdown_connections 2024-05-13T23:01:09.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.436+0000 7fe94f91d640 1 -- 172.21.15.102:0/1643070532 wait complete. 2024-05-13T23:01:09.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.436+0000 7fe94f91d640 1 Processor -- start 2024-05-13T23:01:09.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.436+0000 7fe94f91d640 1 -- start start 2024-05-13T23:01:09.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94f91d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9481002e0 0x7fe948195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:09.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94f91d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe948196300 0x7fe94819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:09.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94f91d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe948196900 con 0x7fe948196300 2024-05-13T23:01:09.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94f91d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe948196a70 con 0x7fe9481002e0 2024-05-13T23:01:09.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94ceb8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe948196300 0x7fe94819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:09.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94ceb8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe948196300 0x7fe94819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47402/0 (socket says 172.21.15.102:47402) 2024-05-13T23:01:09.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94ceb8640 1 -- 172.21.15.102:0/354273451 learned_addr learned my addr 172.21.15.102:0/354273451 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:09.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94d6b9640 1 --2- 172.21.15.102:0/354273451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9481002e0 0x7fe948195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:09.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94ceb8640 1 -- 172.21.15.102:0/354273451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9481002e0 msgr2=0x7fe948195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94ceb8640 1 --2- 172.21.15.102:0/354273451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9481002e0 0x7fe948195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94ceb8640 1 -- 172.21.15.102:0/354273451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe944009900 con 0x7fe948196300 2024-05-13T23:01:09.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94ceb8640 1 --2- 172.21.15.102:0/354273451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe948196300 0x7fe94819a760 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe93800e9a0 tx=0x7fe93800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:09.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe93e7fc640 1 -- 172.21.15.102:0/354273451 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe93800cc90 con 0x7fe948196300 2024-05-13T23:01:09.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94f91d640 1 -- 172.21.15.102:0/354273451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94819ad60 con 0x7fe948196300 2024-05-13T23:01:09.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe93e7fc640 1 -- 172.21.15.102:0/354273451 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe93800cdf0 con 0x7fe948196300 2024-05-13T23:01:09.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe93e7fc640 1 -- 172.21.15.102:0/354273451 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe938010430 con 0x7fe948196300 2024-05-13T23:01:09.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.437+0000 7fe94f91d640 1 -- 172.21.15.102:0/354273451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94819b2c0 con 0x7fe948196300 2024-05-13T23:01:09.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.438+0000 7fe94f91d640 1 -- 172.21.15.102:0/354273451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94819b5a0 con 0x7fe948196300 2024-05-13T23:01:09.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.439+0000 7fe93e7fc640 1 -- 172.21.15.102:0/354273451 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe938005020 con 0x7fe948196300 2024-05-13T23:01:09.416 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.440+0000 7fe93e7fc640 1 --2- 172.21.15.102:0/354273451 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe93406e170 0x7fe934070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:09.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.440+0000 7fe94d6b9640 1 --2- 172.21.15.102:0/354273451 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe93406e170 0x7fe934070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:09.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.440+0000 7fe93e7fc640 1 -- 172.21.15.102:0/354273451 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe938014070 con 0x7fe948196300 2024-05-13T23:01:09.417 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.440+0000 7fe94d6b9640 1 --2- 172.21.15.102:0/354273451 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe93406e170 0x7fe934070630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe944000f80 tx=0x7fe944009840 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:09.420 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.445+0000 7fe93e7fc640 1 -- 172.21.15.102:0/354273451 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe938057980 con 0x7fe948196300 2024-05-13T23:01:09.635 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.660+0000 7fe94f91d640 1 -- 172.21.15.102:0/354273451 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fe948197170 con 0x7fe948196300 2024-05-13T23:01:09.636 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.660+0000 7fe93e7fc640 1 -- 172.21.15.102:0/354273451 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v38) v1 ==== 107+0+5083 (secure 0 0 0) 0x7fe93805b040 con 0x7fe948196300 2024-05-13T23:01:09.636 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:09.636 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/3836279348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3836279348},{"type":"v1","addr":"172.21.15.130:6825","nonce":3836279348}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:57:53.095951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24275":{"gid":24275,"name":"cephfs.smithi130.ovlxvg","rank":0,"incarnation":23,"state":"up:active","state_seq":190,"addr":"172.21.15.130:6827/633282433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":633282433},{"type":"v1","addr":"172.21.15.130:6827","nonce":633282433}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24275,"qdb_cluster":[24275]},"id":1}]} 2024-05-13T23:01:09.636 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 28 2024-05-13T23:01:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.662+0000 7fe94f91d640 1 -- 172.21.15.102:0/354273451 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe93406e170 msgr2=0x7fe934070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.662+0000 7fe94f91d640 1 --2- 172.21.15.102:0/354273451 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe93406e170 0x7fe934070630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe944000f80 tx=0x7fe944009840 comp rx=0 tx=0).stop 2024-05-13T23:01:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.662+0000 7fe94f91d640 1 -- 172.21.15.102:0/354273451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe948196300 msgr2=0x7fe94819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.662+0000 7fe94f91d640 1 --2- 172.21.15.102:0/354273451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe948196300 0x7fe94819a760 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe93800e9a0 tx=0x7fe93800ee60 comp rx=0 tx=0).stop 2024-05-13T23:01:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.662+0000 7fe94f91d640 1 -- 172.21.15.102:0/354273451 shutdown_connections 2024-05-13T23:01:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.662+0000 7fe94f91d640 1 --2- 172.21.15.102:0/354273451 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe93406e170 0x7fe934070630 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.638 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.662+0000 7fe94f91d640 1 --2- 172.21.15.102:0/354273451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe948196300 0x7fe94819a760 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.662+0000 7fe94f91d640 1 --2- 172.21.15.102:0/354273451 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe9481002e0 0x7fe948195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.662+0000 7fe94f91d640 1 -- 172.21.15.102:0/354273451 >> 172.21.15.102:0/354273451 conn(0x7fe9480fbb80 msgr2=0x7fe9480fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:09.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.663+0000 7fe94f91d640 1 -- 172.21.15.102:0/354273451 shutdown_connections 2024-05-13T23:01:09.639 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.663+0000 7fe94f91d640 1 -- 172.21.15.102:0/354273451 wait complete. 2024-05-13T23:01:09.649 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-05-13T23:01:09.649 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-05-13T23:01:09.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.776+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/1941029910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dd40ff340 msgr2=0x7f0dd4104c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.776+0000 7f0ddb8bd640 1 --2- 172.21.15.102:0/1941029910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dd40ff340 0x7f0dd4104c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009880 tx=0x7f0dd002bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.753 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.776+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/1941029910 shutdown_connections 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.776+0000 7f0ddb8bd640 1 --2- 172.21.15.102:0/1941029910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dd40ff340 0x7f0dd4104c20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.776+0000 7f0ddb8bd640 1 --2- 172.21.15.102:0/1941029910 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd40fe990 0x7f0dd40fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.776+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/1941029910 >> 172.21.15.102:0/1941029910 conn(0x7f0dd4075250 msgr2=0x7f0dd4075660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.776+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/1941029910 shutdown_connections 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.776+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/1941029910 wait complete. 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.776+0000 7f0ddb8bd640 1 Processor -- start 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.776+0000 7f0ddb8bd640 1 -- start start 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0ddb8bd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dd40fe990 0x7f0dd40ff900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0ddb8bd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd40ffe40 0x7f0dd41002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0ddb8bd640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd41018e0 con 0x7f0dd40fe990 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0ddb8bd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd41a1fa0 con 0x7f0dd40ffe40 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dd9659640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dd40fe990 0x7f0dd40ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:09.754 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dd9659640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dd40fe990 0x7f0dd40ff900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47428/0 (socket says 172.21.15.102:47428) 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dd9659640 1 -- 172.21.15.102:0/2173473240 learned_addr learned my addr 172.21.15.102:0/2173473240 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dd8e58640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd40ffe40 0x7f0dd41002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dd9659640 1 -- 172.21.15.102:0/2173473240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd40ffe40 msgr2=0x7f0dd41002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dd9659640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd40ffe40 0x7f0dd41002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dd9659640 1 -- 172.21.15.102:0/2173473240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd0009560 con 0x7f0dd40fe990 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dd8e58640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd40ffe40 0x7f0dd41002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dd9659640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dd40fe990 0x7f0dd40ff900 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c8d0 tx=0x7f0dc400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dca7fc640 1 -- 172.21.15.102:0/2173473240 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dc4007b80 con 0x7f0dd40fe990 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dca7fc640 1 -- 172.21.15.102:0/2173473240 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0dc40054e0 con 0x7f0dd40fe990 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/2173473240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd41a2200 con 0x7f0dd40fe990 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0dca7fc640 1 -- 172.21.15.102:0/2173473240 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dc400f450 con 0x7f0dd40fe990 2024-05-13T23:01:09.755 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.777+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/2173473240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd41a2760 con 0x7f0dd40fe990 2024-05-13T23:01:09.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.778+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/2173473240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd404e040 con 0x7f0dd40fe990 2024-05-13T23:01:09.756 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.779+0000 7f0dca7fc640 1 -- 172.21.15.102:0/2173473240 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0dc4007ce0 con 0x7f0dd40fe990 2024-05-13T23:01:09.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.779+0000 7f0dca7fc640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0dc006e170 0x7f0dc0070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:09.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.779+0000 7f0dca7fc640 1 -- 172.21.15.102:0/2173473240 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0dc408e1c0 con 0x7f0dd40fe990 2024-05-13T23:01:09.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.780+0000 7f0dd8e58640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0dc006e170 0x7f0dc0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:09.757 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.780+0000 7f0dd8e58640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0dc006e170 0x7f0dc0070630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0005ec0 tx=0x7f0dd0005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:09.759 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.784+0000 7f0dca7fc640 1 -- 172.21.15.102:0/2173473240 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f0dc4057380 con 0x7f0dd40fe990 2024-05-13T23:01:09.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:01:09.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:09 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/754319570' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-13T23:01:09.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:01:09.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:09 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1678560646' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-13T23:01:09.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:01:09.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:01:09.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:09 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:01:09.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:09 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2534598468' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-13T23:01:09.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:01:09.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:09 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/754319570' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-13T23:01:09.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:01:09.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:09 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1678560646' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-13T23:01:09.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:01:09.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:01:09.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:09 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:01:09.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:09 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2534598468' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-13T23:01:09.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.997+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/2173473240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f0dd41085c0 con 0x7f0dd40fe990 2024-05-13T23:01:09.972 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.997+0000 7f0dca7fc640 1 -- 172.21.15.102:0/2173473240 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v38) v1 ==== 107+0+4282 (secure 0 0 0) 0x7f0dc405aa40 con 0x7f0dd40fe990 2024-05-13T23:01:09.973 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:09.973 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/3836279348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3836279348},{"type":"v1","addr":"172.21.15.130:6825","nonce":3836279348}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:58:22.091950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-13T23:01:09.973 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 29 2024-05-13T23:01:09.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.999+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/2173473240 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0dc006e170 msgr2=0x7f0dc0070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.999+0000 7f0ddb8bd640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0dc006e170 0x7f0dc0070630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0005ec0 tx=0x7f0dd0005fd0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.999+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/2173473240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dd40fe990 msgr2=0x7f0dd40ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:09.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.999+0000 7f0ddb8bd640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dd40fe990 0x7f0dd40ff900 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c8d0 tx=0x7f0dc400cd90 comp rx=0 tx=0).stop 2024-05-13T23:01:09.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.999+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/2173473240 shutdown_connections 2024-05-13T23:01:09.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.999+0000 7f0ddb8bd640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f0dc006e170 0x7f0dc0070630 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.999+0000 7f0ddb8bd640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0dd40ffe40 0x7f0dd41002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.999+0000 7f0ddb8bd640 1 --2- 172.21.15.102:0/2173473240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dd40fe990 0x7f0dd40ff900 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:09.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.999+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/2173473240 >> 172.21.15.102:0/2173473240 conn(0x7f0dd4075250 msgr2=0x7f0dd40fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:09.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.999+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/2173473240 shutdown_connections 2024-05-13T23:01:09.975 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:09.999+0000 7f0ddb8bd640 1 -- 172.21.15.102:0/2173473240 wait complete. 2024-05-13T23:01:09.986 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-05-13T23:01:09.986 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-05-13T23:01:10.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.113+0000 7f0937cce640 1 -- 172.21.15.102:0/2805035409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09301011d0 msgr2=0x7f09301015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.113+0000 7f0937cce640 1 --2- 172.21.15.102:0/2805035409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09301011d0 0x7f09301015b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f092c009880 tx=0x7f092c02bd70 comp rx=0 tx=0).stop 2024-05-13T23:01:10.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.113+0000 7f0937cce640 1 -- 172.21.15.102:0/2805035409 shutdown_connections 2024-05-13T23:01:10.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.113+0000 7f0937cce640 1 --2- 172.21.15.102:0/2805035409 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0930101af0 0x7f0930109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.113+0000 7f0937cce640 1 --2- 172.21.15.102:0/2805035409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09301011d0 0x7f09301015b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.113+0000 7f0937cce640 1 -- 172.21.15.102:0/2805035409 >> 172.21.15.102:0/2805035409 conn(0x7f09300fa9f0 msgr2=0x7f09300fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.113+0000 7f0937cce640 1 -- 172.21.15.102:0/2805035409 shutdown_connections 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.113+0000 7f0937cce640 1 -- 172.21.15.102:0/2805035409 wait complete. 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0937cce640 1 Processor -- start 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0937cce640 1 -- start start 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0937cce640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09301011d0 0x7f0930195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0937cce640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0930101af0 0x7f09301962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0937cce640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0930196960 con 0x7f09301011d0 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0937cce640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093019a6e0 con 0x7f0930101af0 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0935a6a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09301011d0 0x7f0930195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0935a6a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09301011d0 0x7f0930195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47442/0 (socket says 172.21.15.102:47442) 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0935a6a640 1 -- 172.21.15.102:0/923135769 learned_addr learned my addr 172.21.15.102:0/923135769 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:10.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0935a6a640 1 -- 172.21.15.102:0/923135769 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0930101af0 msgr2=0x7f09301962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0935a6a640 1 --2- 172.21.15.102:0/923135769 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0930101af0 0x7f09301962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0935a6a640 1 -- 172.21.15.102:0/923135769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c009560 con 0x7f09301011d0 2024-05-13T23:01:10.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.114+0000 7f0935a6a640 1 --2- 172.21.15.102:0/923135769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09301011d0 0x7f0930195d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f092c009230 tx=0x7f092c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:10.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.115+0000 7f0926ffd640 1 -- 172.21.15.102:0/923135769 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f092c0037c0 con 0x7f09301011d0 2024-05-13T23:01:10.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.115+0000 7f0926ffd640 1 -- 172.21.15.102:0/923135769 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f092c004aa0 con 0x7f09301011d0 2024-05-13T23:01:10.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.115+0000 7f0937cce640 1 -- 172.21.15.102:0/923135769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093019a960 con 0x7f09301011d0 2024-05-13T23:01:10.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.115+0000 7f0926ffd640 1 -- 172.21.15.102:0/923135769 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f092c03e450 con 0x7f09301011d0 2024-05-13T23:01:10.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.115+0000 7f0937cce640 1 -- 172.21.15.102:0/923135769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093019aec0 con 0x7f09301011d0 2024-05-13T23:01:10.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.116+0000 7f0937cce640 1 -- 172.21.15.102:0/923135769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093004e040 con 0x7f09301011d0 2024-05-13T23:01:10.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.116+0000 7f0926ffd640 1 -- 172.21.15.102:0/923135769 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f092c005060 con 0x7f09301011d0 2024-05-13T23:01:10.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.116+0000 7f0926ffd640 1 --2- 172.21.15.102:0/923135769 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f091c06e170 0x7f091c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:10.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.117+0000 7f0926ffd640 1 -- 172.21.15.102:0/923135769 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f092c0b1130 con 0x7f09301011d0 2024-05-13T23:01:10.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.117+0000 7f0935269640 1 --2- 172.21.15.102:0/923135769 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f091c06e170 0x7f091c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:10.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.117+0000 7f0935269640 1 --2- 172.21.15.102:0/923135769 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f091c06e170 0x7f091c070630 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0930197350 tx=0x7f0920005c90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:10.096 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.121+0000 7f0926ffd640 1 -- 172.21.15.102:0/923135769 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f092c07a2f0 con 0x7f09301011d0 2024-05-13T23:01:10.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.335+0000 7f0937cce640 1 -- 172.21.15.102:0/923135769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f093010d2b0 con 0x7f09301011d0 2024-05-13T23:01:10.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.335+0000 7f0926ffd640 1 -- 172.21.15.102:0/923135769 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v38) v1 ==== 107+0+4361 (secure 0 0 0) 0x7f092c07d9b0 con 0x7f09301011d0 2024-05-13T23:01:10.311 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:10.311 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/3836279348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3836279348},{"type":"v1","addr":"172.21.15.130:6825","nonce":3836279348}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:58:22.097012+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34410":{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":0,"incarnation":30,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-13T23:01:10.311 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 30 2024-05-13T23:01:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.337+0000 7f0937cce640 1 -- 172.21.15.102:0/923135769 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f091c06e170 msgr2=0x7f091c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.337+0000 7f0937cce640 1 --2- 172.21.15.102:0/923135769 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f091c06e170 0x7f091c070630 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0930197350 tx=0x7f0920005c90 comp rx=0 tx=0).stop 2024-05-13T23:01:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.337+0000 7f0937cce640 1 -- 172.21.15.102:0/923135769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09301011d0 msgr2=0x7f0930195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.337+0000 7f0937cce640 1 --2- 172.21.15.102:0/923135769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09301011d0 0x7f0930195d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f092c009230 tx=0x7f092c004270 comp rx=0 tx=0).stop 2024-05-13T23:01:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.337+0000 7f0937cce640 1 -- 172.21.15.102:0/923135769 shutdown_connections 2024-05-13T23:01:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.337+0000 7f0937cce640 1 --2- 172.21.15.102:0/923135769 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f091c06e170 0x7f091c070630 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.337+0000 7f0937cce640 1 --2- 172.21.15.102:0/923135769 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0930101af0 0x7f09301962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.313 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.337+0000 7f0937cce640 1 --2- 172.21.15.102:0/923135769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09301011d0 0x7f0930195d90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.337+0000 7f0937cce640 1 -- 172.21.15.102:0/923135769 >> 172.21.15.102:0/923135769 conn(0x7f09300fa9f0 msgr2=0x7f0930104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.337+0000 7f0937cce640 1 -- 172.21.15.102:0/923135769 shutdown_connections 2024-05-13T23:01:10.314 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.337+0000 7f0937cce640 1 -- 172.21.15.102:0/923135769 wait complete. 2024-05-13T23:01:10.323 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-05-13T23:01:10.324 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-05-13T23:01:10.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.449+0000 7f1a82b0a640 1 -- 172.21.15.102:0/652528314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a7c0ff180 msgr2=0x7f1a7c0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.449+0000 7f1a82b0a640 1 --2- 172.21.15.102:0/652528314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a7c0ff180 0x7f1a7c0ff560 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009880 tx=0x7f1a7002bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.449+0000 7f1a82b0a640 1 -- 172.21.15.102:0/652528314 shutdown_connections 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.449+0000 7f1a82b0a640 1 --2- 172.21.15.102:0/652528314 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a7c0ffb30 0x7f1a7c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.449+0000 7f1a82b0a640 1 --2- 172.21.15.102:0/652528314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a7c0ff180 0x7f1a7c0ff560 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.449+0000 7f1a82b0a640 1 -- 172.21.15.102:0/652528314 >> 172.21.15.102:0/652528314 conn(0x7f1a7c0faa00 msgr2=0x7f1a7c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.449+0000 7f1a82b0a640 1 -- 172.21.15.102:0/652528314 shutdown_connections 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.449+0000 7f1a82b0a640 1 -- 172.21.15.102:0/652528314 wait complete. 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a82b0a640 1 Processor -- start 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a82b0a640 1 -- start start 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a82b0a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a7c0ff180 0x7f1a7c194cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a82b0a640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a7c195200 0x7f1a7c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a82b0a640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c195800 con 0x7f1a7c195200 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a82b0a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c195940 con 0x7f1a7c0ff180 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a7bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a7c195200 0x7f1a7c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:10.427 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a7bfff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a7c195200 0x7f1a7c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47462/0 (socket says 172.21.15.102:47462) 2024-05-13T23:01:10.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a7bfff640 1 -- 172.21.15.102:0/2974906078 learned_addr learned my addr 172.21.15.102:0/2974906078 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:10.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a808a6640 1 --2- 172.21.15.102:0/2974906078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a7c0ff180 0x7f1a7c194cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:10.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a7bfff640 1 -- 172.21.15.102:0/2974906078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a7c0ff180 msgr2=0x7f1a7c194cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a7bfff640 1 --2- 172.21.15.102:0/2974906078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a7c0ff180 0x7f1a7c194cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a7bfff640 1 -- 172.21.15.102:0/2974906078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70009560 con 0x7f1a7c195200 2024-05-13T23:01:10.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a7bfff640 1 --2- 172.21.15.102:0/2974906078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a7c195200 0x7f1a7c1985b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800d6c0 tx=0x7f1a6800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:10.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a79ffb640 1 -- 172.21.15.102:0/2974906078 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a680051f0 con 0x7f1a7c195200 2024-05-13T23:01:10.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a79ffb640 1 -- 172.21.15.102:0/2974906078 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a68009ce0 con 0x7f1a7c195200 2024-05-13T23:01:10.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a82b0a640 1 -- 172.21.15.102:0/2974906078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7c198bb0 con 0x7f1a7c195200 2024-05-13T23:01:10.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.450+0000 7f1a79ffb640 1 -- 172.21.15.102:0/2974906078 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a68010460 con 0x7f1a7c195200 2024-05-13T23:01:10.428 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.451+0000 7f1a82b0a640 1 -- 172.21.15.102:0/2974906078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7c199110 con 0x7f1a7c195200 2024-05-13T23:01:10.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.452+0000 7f1a82b0a640 1 -- 172.21.15.102:0/2974906078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7c04e040 con 0x7f1a7c195200 2024-05-13T23:01:10.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.452+0000 7f1a79ffb640 1 -- 172.21.15.102:0/2974906078 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1a68005350 con 0x7f1a7c195200 2024-05-13T23:01:10.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.452+0000 7f1a79ffb640 1 --2- 172.21.15.102:0/2974906078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a6c06e170 0x7f1a6c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:10.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.453+0000 7f1a79ffb640 1 -- 172.21.15.102:0/2974906078 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1a6808dda0 con 0x7f1a7c195200 2024-05-13T23:01:10.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.453+0000 7f1a808a6640 1 --2- 172.21.15.102:0/2974906078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a6c06e170 0x7f1a6c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:10.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.453+0000 7f1a808a6640 1 --2- 172.21.15.102:0/2974906078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a6c06e170 0x7f1a6c070630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a70006010 tx=0x7f1a70000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:10.432 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.457+0000 7f1a79ffb640 1 -- 172.21.15.102:0/2974906078 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f1a68056f60 con 0x7f1a7c195200 2024-05-13T23:01:10.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.672+0000 7f1a82b0a640 1 -- 172.21.15.102:0/2974906078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f1a7c1960c0 con 0x7f1a7c195200 2024-05-13T23:01:10.647 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.672+0000 7f1a79ffb640 1 -- 172.21.15.102:0/2974906078 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v38) v1 ==== 107+0+5211 (secure 0 0 0) 0x7f1a6805a620 con 0x7f1a7c195200 2024-05-13T23:01:10.648 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:10.648 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/3836279348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3836279348},{"type":"v1","addr":"172.21.15.130:6825","nonce":3836279348}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44297,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6827/3806306097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3806306097},{"type":"v1","addr":"172.21.15.130:6827","nonce":3806306097}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:58:22.097012+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34410":{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":0,"incarnation":30,"state":"up:replay","state_seq":1,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-13T23:01:10.648 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 31 2024-05-13T23:01:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.674+0000 7f1a82b0a640 1 -- 172.21.15.102:0/2974906078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a6c06e170 msgr2=0x7f1a6c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.674+0000 7f1a82b0a640 1 --2- 172.21.15.102:0/2974906078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a6c06e170 0x7f1a6c070630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a70006010 tx=0x7f1a70000bc0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.674+0000 7f1a82b0a640 1 -- 172.21.15.102:0/2974906078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a7c195200 msgr2=0x7f1a7c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.674+0000 7f1a82b0a640 1 --2- 172.21.15.102:0/2974906078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a7c195200 0x7f1a7c1985b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800d6c0 tx=0x7f1a6800db80 comp rx=0 tx=0).stop 2024-05-13T23:01:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.674+0000 7f1a82b0a640 1 -- 172.21.15.102:0/2974906078 shutdown_connections 2024-05-13T23:01:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.674+0000 7f1a82b0a640 1 --2- 172.21.15.102:0/2974906078 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f1a6c06e170 0x7f1a6c070630 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.674+0000 7f1a82b0a640 1 --2- 172.21.15.102:0/2974906078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a7c195200 0x7f1a7c1985b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.674+0000 7f1a82b0a640 1 --2- 172.21.15.102:0/2974906078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a7c0ff180 0x7f1a7c194cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.650 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.674+0000 7f1a82b0a640 1 -- 172.21.15.102:0/2974906078 >> 172.21.15.102:0/2974906078 conn(0x7f1a7c0faa00 msgr2=0x7f1a7c0fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:10.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.674+0000 7f1a82b0a640 1 -- 172.21.15.102:0/2974906078 shutdown_connections 2024-05-13T23:01:10.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.674+0000 7f1a82b0a640 1 -- 172.21.15.102:0/2974906078 wait complete. 2024-05-13T23:01:10.661 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-05-13T23:01:10.661 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-05-13T23:01:10.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.786+0000 7f735f916640 1 -- 172.21.15.102:0/2069766728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73580675c0 msgr2=0x7f73580679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.786+0000 7f735f916640 1 --2- 172.21.15.102:0/2069766728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73580675c0 0x7f73580679a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7348009c20 tx=0x7f734802c1b0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 -- 172.21.15.102:0/2069766728 shutdown_connections 2024-05-13T23:01:10.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 --2- 172.21.15.102:0/2069766728 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7358067f70 0x7f7358104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 --2- 172.21.15.102:0/2069766728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73580675c0 0x7f73580679a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 -- 172.21.15.102:0/2069766728 >> 172.21.15.102:0/2069766728 conn(0x7f7358074a50 msgr2=0x7f7358074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:10.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 -- 172.21.15.102:0/2069766728 shutdown_connections 2024-05-13T23:01:10.764 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 -- 172.21.15.102:0/2069766728 wait complete. 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 Processor -- start 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 -- start start 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73580675c0 0x7f7358195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73581962f0 0x7f735819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73581968f0 con 0x7f73580675c0 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735f916640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7358196a60 con 0x7f73581962f0 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.787+0000 7f735d6b2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73580675c0 0x7f7358195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f735d6b2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73580675c0 0x7f7358195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47472/0 (socket says 172.21.15.102:47472) 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f735d6b2640 1 -- 172.21.15.102:0/3866598417 learned_addr learned my addr 172.21.15.102:0/3866598417 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f735ceb1640 1 --2- 172.21.15.102:0/3866598417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73581962f0 0x7f735819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f735d6b2640 1 -- 172.21.15.102:0/3866598417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73581962f0 msgr2=0x7f735819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f735d6b2640 1 --2- 172.21.15.102:0/3866598417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73581962f0 0x7f735819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f735d6b2640 1 -- 172.21.15.102:0/3866598417 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7348009900 con 0x7f73580675c0 2024-05-13T23:01:10.765 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f735d6b2640 1 --2- 172.21.15.102:0/3866598417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73580675c0 0x7f7358195db0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7348006010 tx=0x7f7348032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:10.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f73567fc640 1 -- 172.21.15.102:0/3866598417 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7348034820 con 0x7f73580675c0 2024-05-13T23:01:10.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f73567fc640 1 -- 172.21.15.102:0/3866598417 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7348003e30 con 0x7f73580675c0 2024-05-13T23:01:10.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f73567fc640 1 -- 172.21.15.102:0/3866598417 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7348004830 con 0x7f73580675c0 2024-05-13T23:01:10.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f735f916640 1 -- 172.21.15.102:0/3866598417 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735819acf0 con 0x7f73580675c0 2024-05-13T23:01:10.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.788+0000 7f735f916640 1 -- 172.21.15.102:0/3866598417 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735819b200 con 0x7f73580675c0 2024-05-13T23:01:10.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.789+0000 7f735f916640 1 -- 172.21.15.102:0/3866598417 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735804e040 con 0x7f73580675c0 2024-05-13T23:01:10.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.790+0000 7f73567fc640 1 -- 172.21.15.102:0/3866598417 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f734803b040 con 0x7f73580675c0 2024-05-13T23:01:10.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.790+0000 7f73567fc640 1 --2- 172.21.15.102:0/3866598417 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f734006e170 0x7f7340070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:10.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.790+0000 7f73567fc640 1 -- 172.21.15.102:0/3866598417 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f73480b1040 con 0x7f73580675c0 2024-05-13T23:01:10.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.790+0000 7f735ceb1640 1 --2- 172.21.15.102:0/3866598417 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f734006e170 0x7f7340070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:10.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.791+0000 7f735ceb1640 1 --2- 172.21.15.102:0/3866598417 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f734006e170 0x7f7340070630 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f734c005fd0 tx=0x7f734c005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:10.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:10.795+0000 7f73567fc640 1 -- 172.21.15.102:0/3866598417 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f734807a200 con 0x7f73580675c0 2024-05-13T23:01:10.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:10 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/354273451' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-13T23:01:10.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:10 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2173473240' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-13T23:01:10.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:10 smithi130 ceph-mon[84265]: pgmap v301: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:10.899 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:10 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/923135769' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-13T23:01:10.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:10 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/354273451' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-13T23:01:10.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:10 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2173473240' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-13T23:01:10.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:10 smithi102 ceph-mon[111694]: pgmap v301: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:10.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:10 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/923135769' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-13T23:01:10.983 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.008+0000 7f735f916640 1 -- 172.21.15.102:0/3866598417 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f735819b560 con 0x7f73580675c0 2024-05-13T23:01:10.984 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.009+0000 7f73567fc640 1 -- 172.21.15.102:0/3866598417 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v38) v1 ==== 107+0+5215 (secure 0 0 0) 0x7f734807d8c0 con 0x7f73580675c0 2024-05-13T23:01:10.984 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:10.984 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/3836279348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3836279348},{"type":"v1","addr":"172.21.15.130:6825","nonce":3836279348}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44297,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6827/3806306097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3806306097},{"type":"v1","addr":"172.21.15.130:6827","nonce":3806306097}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:58:30.976596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34410":{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":0,"incarnation":30,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-13T23:01:10.984 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 32 2024-05-13T23:01:10.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.011+0000 7f735f916640 1 -- 172.21.15.102:0/3866598417 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f734006e170 msgr2=0x7f7340070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.986 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.011+0000 7f735f916640 1 --2- 172.21.15.102:0/3866598417 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f734006e170 0x7f7340070630 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f734c005fd0 tx=0x7f734c005850 comp rx=0 tx=0).stop 2024-05-13T23:01:10.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.011+0000 7f735f916640 1 -- 172.21.15.102:0/3866598417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73580675c0 msgr2=0x7f7358195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:10.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.011+0000 7f735f916640 1 --2- 172.21.15.102:0/3866598417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73580675c0 0x7f7358195db0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7348006010 tx=0x7f7348032b60 comp rx=0 tx=0).stop 2024-05-13T23:01:10.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.011+0000 7f735f916640 1 -- 172.21.15.102:0/3866598417 shutdown_connections 2024-05-13T23:01:10.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.011+0000 7f735f916640 1 --2- 172.21.15.102:0/3866598417 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f734006e170 0x7f7340070630 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.011+0000 7f735f916640 1 --2- 172.21.15.102:0/3866598417 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73581962f0 0x7f735819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.011+0000 7f735f916640 1 --2- 172.21.15.102:0/3866598417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73580675c0 0x7f7358195db0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:10.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.011+0000 7f735f916640 1 -- 172.21.15.102:0/3866598417 >> 172.21.15.102:0/3866598417 conn(0x7f7358074a50 msgr2=0x7f73580fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:10.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.011+0000 7f735f916640 1 -- 172.21.15.102:0/3866598417 shutdown_connections 2024-05-13T23:01:10.987 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.011+0000 7f735f916640 1 -- 172.21.15.102:0/3866598417 wait complete. 2024-05-13T23:01:10.997 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-05-13T23:01:10.998 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-05-13T23:01:11.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.123+0000 7f2739cce640 1 -- 172.21.15.102:0/227269551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27340ff1c0 msgr2=0x7f27340ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.100 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.123+0000 7f2739cce640 1 --2- 172.21.15.102:0/227269551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27340ff1c0 0x7f27340ff5a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2724009880 tx=0x7f272402bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.123+0000 7f2739cce640 1 -- 172.21.15.102:0/227269551 shutdown_connections 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.123+0000 7f2739cce640 1 --2- 172.21.15.102:0/227269551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27340ffb70 0x7f2734103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.123+0000 7f2739cce640 1 --2- 172.21.15.102:0/227269551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27340ff1c0 0x7f27340ff5a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.123+0000 7f2739cce640 1 -- 172.21.15.102:0/227269551 >> 172.21.15.102:0/227269551 conn(0x7f27340faa20 msgr2=0x7f27340fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.123+0000 7f2739cce640 1 -- 172.21.15.102:0/227269551 shutdown_connections 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.123+0000 7f2739cce640 1 -- 172.21.15.102:0/227269551 wait complete. 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2739cce640 1 Processor -- start 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2739cce640 1 -- start start 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2739cce640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27340ff1c0 0x7f2734196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2739cce640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2734197270 0x7f27341976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2739cce640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273419fda0 con 0x7f2734197270 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2739cce640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273419fee0 con 0x7f27340ff1c0 2024-05-13T23:01:11.101 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2734197270 0x7f27341976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2734197270 0x7f27341976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47492/0 (socket says 172.21.15.102:47492) 2024-05-13T23:01:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2732ffd640 1 -- 172.21.15.102:0/1139130399 learned_addr learned my addr 172.21.15.102:0/1139130399 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2732ffd640 1 -- 172.21.15.102:0/1139130399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27340ff1c0 msgr2=0x7f2734196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2732ffd640 1 --2- 172.21.15.102:0/1139130399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27340ff1c0 0x7f2734196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2732ffd640 1 -- 172.21.15.102:0/1139130399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2724009560 con 0x7f2734197270 2024-05-13T23:01:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.124+0000 7f2732ffd640 1 --2- 172.21.15.102:0/1139130399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2734197270 0x7f27341976d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f272800b400 tx=0x7f272800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.125+0000 7f2730ff9640 1 -- 172.21.15.102:0/1139130399 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2728011800 con 0x7f2734197270 2024-05-13T23:01:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.125+0000 7f2739cce640 1 -- 172.21.15.102:0/1139130399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27341a0140 con 0x7f2734197270 2024-05-13T23:01:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.125+0000 7f2730ff9640 1 -- 172.21.15.102:0/1139130399 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2728011e20 con 0x7f2734197270 2024-05-13T23:01:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.125+0000 7f2730ff9640 1 -- 172.21.15.102:0/1139130399 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f272800f2e0 con 0x7f2734197270 2024-05-13T23:01:11.102 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.125+0000 7f2739cce640 1 -- 172.21.15.102:0/1139130399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27341a0650 con 0x7f2734197270 2024-05-13T23:01:11.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.126+0000 7f2739cce640 1 -- 172.21.15.102:0/1139130399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273404e040 con 0x7f2734197270 2024-05-13T23:01:11.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.126+0000 7f2730ff9640 1 -- 172.21.15.102:0/1139130399 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2728010380 con 0x7f2734197270 2024-05-13T23:01:11.103 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.127+0000 7f2730ff9640 1 --2- 172.21.15.102:0/1139130399 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f271c06e170 0x7f271c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:11.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.127+0000 7f2730ff9640 1 -- 172.21.15.102:0/1139130399 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f272808df60 con 0x7f2734197270 2024-05-13T23:01:11.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.127+0000 7f27337fe640 1 --2- 172.21.15.102:0/1139130399 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f271c06e170 0x7f271c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:11.104 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.127+0000 7f27337fe640 1 --2- 172.21.15.102:0/1139130399 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f271c06e170 0x7f271c070630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2724036040 tx=0x7f2724000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:11.106 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.131+0000 7f2730ff9640 1 -- 172.21.15.102:0/1139130399 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f27280c2850 con 0x7f2734197270 2024-05-13T23:01:11.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.344+0000 7f2739cce640 1 -- 172.21.15.102:0/1139130399 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f27341074c0 con 0x7f2734197270 2024-05-13T23:01:11.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.345+0000 7f2730ff9640 1 -- 172.21.15.102:0/1139130399 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v38) v1 ==== 107+0+5212 (secure 0 0 0) 0x7f272805ac50 con 0x7f2734197270 2024-05-13T23:01:11.320 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:11.320 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/3836279348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3836279348},{"type":"v1","addr":"172.21.15.130:6825","nonce":3836279348}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44297,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6827/3806306097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3806306097},{"type":"v1","addr":"172.21.15.130:6827","nonce":3806306097}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:58:31.824543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34410":{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":0,"incarnation":30,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-13T23:01:11.320 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 33 2024-05-13T23:01:11.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.346+0000 7f2739cce640 1 -- 172.21.15.102:0/1139130399 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f271c06e170 msgr2=0x7f271c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.346+0000 7f2739cce640 1 --2- 172.21.15.102:0/1139130399 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f271c06e170 0x7f271c070630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2724036040 tx=0x7f2724000bc0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.347+0000 7f2739cce640 1 -- 172.21.15.102:0/1139130399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2734197270 msgr2=0x7f27341976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.347+0000 7f2739cce640 1 --2- 172.21.15.102:0/1139130399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2734197270 0x7f27341976d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f272800b400 tx=0x7f272800b8c0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.347+0000 7f2739cce640 1 -- 172.21.15.102:0/1139130399 shutdown_connections 2024-05-13T23:01:11.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.347+0000 7f2739cce640 1 --2- 172.21.15.102:0/1139130399 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f271c06e170 0x7f271c070630 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.347+0000 7f2739cce640 1 --2- 172.21.15.102:0/1139130399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2734197270 0x7f27341976d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.347+0000 7f2739cce640 1 --2- 172.21.15.102:0/1139130399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27340ff1c0 0x7f2734196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.347+0000 7f2739cce640 1 -- 172.21.15.102:0/1139130399 >> 172.21.15.102:0/1139130399 conn(0x7f27340faa20 msgr2=0x7f27340fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:11.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.347+0000 7f2739cce640 1 -- 172.21.15.102:0/1139130399 shutdown_connections 2024-05-13T23:01:11.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.347+0000 7f2739cce640 1 -- 172.21.15.102:0/1139130399 wait complete. 2024-05-13T23:01:11.333 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-05-13T23:01:11.333 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-05-13T23:01:11.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.459+0000 7f5665496640 1 -- 172.21.15.102:0/1346773745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56600ff260 msgr2=0x7f56600ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.459+0000 7f5665496640 1 --2- 172.21.15.102:0/1346773745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56600ff260 0x7f56600ff640 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5650009880 tx=0x7f565002bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.460+0000 7f5665496640 1 -- 172.21.15.102:0/1346773745 shutdown_connections 2024-05-13T23:01:11.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.460+0000 7f5665496640 1 --2- 172.21.15.102:0/1346773745 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56600ffc10 0x7f5660103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.437 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.460+0000 7f5665496640 1 --2- 172.21.15.102:0/1346773745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56600ff260 0x7f56600ff640 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.460+0000 7f5665496640 1 -- 172.21.15.102:0/1346773745 >> 172.21.15.102:0/1346773745 conn(0x7f5660074a50 msgr2=0x7f5660074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.460+0000 7f5665496640 1 -- 172.21.15.102:0/1346773745 shutdown_connections 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.460+0000 7f5665496640 1 -- 172.21.15.102:0/1346773745 wait complete. 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.460+0000 7f5665496640 1 Processor -- start 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.460+0000 7f5665496640 1 -- start start 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f5665496640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56600ff260 0x7f5660100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f5665496640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5660101150 0x7f56601015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f5665496640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56601a2060 con 0x7f56600ff260 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f5665496640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56601a21a0 con 0x7f5660101150 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f565effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56600ff260 0x7f5660100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f565effd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56600ff260 0x7f5660100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47506/0 (socket says 172.21.15.102:47506) 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f565effd640 1 -- 172.21.15.102:0/2154625912 learned_addr learned my addr 172.21.15.102:0/2154625912 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f56567fc640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5660101150 0x7f56601015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:11.438 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f565effd640 1 -- 172.21.15.102:0/2154625912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5660101150 msgr2=0x7f56601015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f565effd640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5660101150 0x7f56601015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f565effd640 1 -- 172.21.15.102:0/2154625912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5650009560 con 0x7f56600ff260 2024-05-13T23:01:11.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f56567fc640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5660101150 0x7f56601015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T23:01:11.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f565effd640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56600ff260 0x7f5660100c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5650009430 tx=0x7f5650004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:11.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f565cff9640 1 -- 172.21.15.102:0/2154625912 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56500035f0 con 0x7f56600ff260 2024-05-13T23:01:11.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f5665496640 1 -- 172.21.15.102:0/2154625912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56601a22e0 con 0x7f56600ff260 2024-05-13T23:01:11.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f565cff9640 1 -- 172.21.15.102:0/2154625912 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5650003c10 con 0x7f56600ff260 2024-05-13T23:01:11.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f565cff9640 1 -- 172.21.15.102:0/2154625912 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5650034440 con 0x7f56600ff260 2024-05-13T23:01:11.439 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.461+0000 7f5665496640 1 -- 172.21.15.102:0/2154625912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56601a2790 con 0x7f56600ff260 2024-05-13T23:01:11.440 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.462+0000 7f5665496640 1 -- 172.21.15.102:0/2154625912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566004e040 con 0x7f56600ff260 2024-05-13T23:01:11.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.463+0000 7f565cff9640 1 -- 172.21.15.102:0/2154625912 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5650004fd0 con 0x7f56600ff260 2024-05-13T23:01:11.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.463+0000 7f565cff9640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f564c06e170 0x7f564c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:11.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.463+0000 7f565cff9640 1 -- 172.21.15.102:0/2154625912 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f56500b0720 con 0x7f56600ff260 2024-05-13T23:01:11.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.463+0000 7f56567fc640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f564c06e170 0x7f564c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:11.441 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.464+0000 7f56567fc640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f564c06e170 0x7f564c070630 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5648006fd0 tx=0x7f5648008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:11.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.468+0000 7f565cff9640 1 -- 172.21.15.102:0/2154625912 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f56500798e0 con 0x7f56600ff260 2024-05-13T23:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.682+0000 7f5665496640 1 -- 172.21.15.102:0/2154625912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f56600ffe40 con 0x7f56600ff260 2024-05-13T23:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.683+0000 7f565cff9640 1 -- 172.21.15.102:0/2154625912 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v38) v1 ==== 107+0+5221 (secure 0 0 0) 0x7f565007cfa0 con 0x7f56600ff260 2024-05-13T23:01:11.658 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:11.658 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/3836279348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3836279348},{"type":"v1","addr":"172.21.15.130:6825","nonce":3836279348}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44297,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6827/3806306097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3806306097},{"type":"v1","addr":"172.21.15.130:6827","nonce":3806306097}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:58:33.825746+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34410":{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":0,"incarnation":30,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34410,"qdb_cluster":[34410]},"id":1}]} 2024-05-13T23:01:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 34 2024-05-13T23:01:11.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.685+0000 7f5665496640 1 -- 172.21.15.102:0/2154625912 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f564c06e170 msgr2=0x7f564c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.685+0000 7f5665496640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f564c06e170 0x7f564c070630 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5648006fd0 tx=0x7f5648008040 comp rx=0 tx=0).stop 2024-05-13T23:01:11.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.685+0000 7f5665496640 1 -- 172.21.15.102:0/2154625912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56600ff260 msgr2=0x7f5660100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.685+0000 7f5665496640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56600ff260 0x7f5660100c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5650009430 tx=0x7f5650004040 comp rx=0 tx=0).stop 2024-05-13T23:01:11.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.685+0000 7f5665496640 1 -- 172.21.15.102:0/2154625912 shutdown_connections 2024-05-13T23:01:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.685+0000 7f5665496640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f564c06e170 0x7f564c070630 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.685+0000 7f5665496640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5660101150 0x7f56601015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.685+0000 7f5665496640 1 --2- 172.21.15.102:0/2154625912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56600ff260 0x7f5660100c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.685+0000 7f5665496640 1 -- 172.21.15.102:0/2154625912 >> 172.21.15.102:0/2154625912 conn(0x7f5660074a50 msgr2=0x7f5660103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.685+0000 7f5665496640 1 -- 172.21.15.102:0/2154625912 shutdown_connections 2024-05-13T23:01:11.661 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.685+0000 7f5665496640 1 -- 172.21.15.102:0/2154625912 wait complete. 2024-05-13T23:01:11.671 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-05-13T23:01:11.671 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-05-13T23:01:11.774 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.797+0000 7f6c6137c640 1 -- 172.21.15.102:0/3848399091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c5c0ff260 msgr2=0x7f6c5c0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.797+0000 7f6c6137c640 1 --2- 172.21.15.102:0/3848399091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c5c0ff260 0x7f6c5c0ff640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009880 tx=0x7f6c4c02bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.797+0000 7f6c6137c640 1 -- 172.21.15.102:0/3848399091 shutdown_connections 2024-05-13T23:01:11.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.797+0000 7f6c6137c640 1 --2- 172.21.15.102:0/3848399091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c5c0ffc10 0x7f6c5c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.797+0000 7f6c6137c640 1 --2- 172.21.15.102:0/3848399091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c5c0ff260 0x7f6c5c0ff640 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.797+0000 7f6c6137c640 1 -- 172.21.15.102:0/3848399091 >> 172.21.15.102:0/3848399091 conn(0x7f6c5c074a50 msgr2=0x7f6c5c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:11.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.797+0000 7f6c6137c640 1 -- 172.21.15.102:0/3848399091 shutdown_connections 2024-05-13T23:01:11.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.797+0000 7f6c6137c640 1 -- 172.21.15.102:0/3848399091 wait complete. 2024-05-13T23:01:11.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c6137c640 1 Processor -- start 2024-05-13T23:01:11.778 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c6137c640 1 -- start start 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c6137c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c5c0ff260 0x7f6c5c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c6137c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c5c197150 0x7f6c5c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c6137c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c19fc80 con 0x7f6c5c197150 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c6137c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c19fdc0 con 0x7f6c5c0ff260 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c5c197150 0x7f6c5c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c5c197150 0x7f6c5c1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47536/0 (socket says 172.21.15.102:47536) 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c53fff640 1 -- 172.21.15.102:0/145399049 learned_addr learned my addr 172.21.15.102:0/145399049 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c5affd640 1 --2- 172.21.15.102:0/145399049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c5c0ff260 0x7f6c5c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c53fff640 1 -- 172.21.15.102:0/145399049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c5c0ff260 msgr2=0x7f6c5c196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c53fff640 1 --2- 172.21.15.102:0/145399049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c5c0ff260 0x7f6c5c196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.798+0000 7f6c53fff640 1 -- 172.21.15.102:0/145399049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c009560 con 0x7f6c5c197150 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.799+0000 7f6c53fff640 1 --2- 172.21.15.102:0/145399049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c5c197150 0x7f6c5c1975b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400d6f0 tx=0x7f6c4400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.799+0000 7f6c58ff9640 1 -- 172.21.15.102:0/145399049 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c440051f0 con 0x7f6c5c197150 2024-05-13T23:01:11.779 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.799+0000 7f6c58ff9640 1 -- 172.21.15.102:0/145399049 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c44009ce0 con 0x7f6c5c197150 2024-05-13T23:01:11.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.799+0000 7f6c6137c640 1 -- 172.21.15.102:0/145399049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5c1a0020 con 0x7f6c5c197150 2024-05-13T23:01:11.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.799+0000 7f6c58ff9640 1 -- 172.21.15.102:0/145399049 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c44004e80 con 0x7f6c5c197150 2024-05-13T23:01:11.780 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.799+0000 7f6c6137c640 1 -- 172.21.15.102:0/145399049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5c1a0500 con 0x7f6c5c197150 2024-05-13T23:01:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.800+0000 7f6c6137c640 1 -- 172.21.15.102:0/145399049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5c04e040 con 0x7f6c5c197150 2024-05-13T23:01:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.800+0000 7f6c58ff9640 1 -- 172.21.15.102:0/145399049 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6c440046c0 con 0x7f6c5c197150 2024-05-13T23:01:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.801+0000 7f6c58ff9640 1 --2- 172.21.15.102:0/145399049 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6c4806e110 0x7f6c480705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.801+0000 7f6c58ff9640 1 -- 172.21.15.102:0/145399049 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6c4408dd30 con 0x7f6c5c197150 2024-05-13T23:01:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.801+0000 7f6c5affd640 1 --2- 172.21.15.102:0/145399049 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6c4806e110 0x7f6c480705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.801+0000 7f6c5affd640 1 --2- 172.21.15.102:0/145399049 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6c4806e110 0x7f6c480705d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c000c00 tx=0x7f6c4c02cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:11.781 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:11.805+0000 7f6c58ff9640 1 -- 172.21.15.102:0/145399049 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6c44056ef0 con 0x7f6c5c197150 2024-05-13T23:01:11.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:11 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2974906078' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-13T23:01:11.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:11 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3866598417' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-13T23:01:11.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:11 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1139130399' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-13T23:01:11.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:11 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2974906078' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-13T23:01:11.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:11 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3866598417' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-13T23:01:11.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:11 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1139130399' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-13T23:01:11.991 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.016+0000 7f6c6137c640 1 -- 172.21.15.102:0/145399049 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f6c5c107480 con 0x7f6c5c197150 2024-05-13T23:01:11.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.016+0000 7f6c58ff9640 1 -- 172.21.15.102:0/145399049 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v38) v1 ==== 107+0+5221 (secure 0 0 0) 0x7f6c44020090 con 0x7f6c5c197150 2024-05-13T23:01:11.992 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:11.992 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/3836279348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3836279348},{"type":"v1","addr":"172.21.15.130:6825","nonce":3836279348}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44297,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6827/3806306097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3806306097},{"type":"v1","addr":"172.21.15.130:6827","nonce":3806306097}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:58:37.433677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34410":{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":0,"incarnation":30,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34410,"qdb_cluster":[34410]},"id":1}]} 2024-05-13T23:01:11.992 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 35 2024-05-13T23:01:11.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.018+0000 7f6c6137c640 1 -- 172.21.15.102:0/145399049 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6c4806e110 msgr2=0x7f6c480705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.018+0000 7f6c6137c640 1 --2- 172.21.15.102:0/145399049 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6c4806e110 0x7f6c480705d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c000c00 tx=0x7f6c4c02cf90 comp rx=0 tx=0).stop 2024-05-13T23:01:11.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.018+0000 7f6c6137c640 1 -- 172.21.15.102:0/145399049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c5c197150 msgr2=0x7f6c5c1975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:11.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.018+0000 7f6c6137c640 1 --2- 172.21.15.102:0/145399049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c5c197150 0x7f6c5c1975b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400d6f0 tx=0x7f6c4400dbb0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.019+0000 7f6c6137c640 1 -- 172.21.15.102:0/145399049 shutdown_connections 2024-05-13T23:01:11.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.019+0000 7f6c6137c640 1 --2- 172.21.15.102:0/145399049 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6c4806e110 0x7f6c480705d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.019+0000 7f6c6137c640 1 --2- 172.21.15.102:0/145399049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c5c197150 0x7f6c5c1975b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.019+0000 7f6c6137c640 1 --2- 172.21.15.102:0/145399049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c5c0ff260 0x7f6c5c196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:11.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.019+0000 7f6c6137c640 1 -- 172.21.15.102:0/145399049 >> 172.21.15.102:0/145399049 conn(0x7f6c5c074a50 msgr2=0x7f6c5c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:11.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.019+0000 7f6c6137c640 1 -- 172.21.15.102:0/145399049 shutdown_connections 2024-05-13T23:01:11.995 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.019+0000 7f6c6137c640 1 -- 172.21.15.102:0/145399049 wait complete. 2024-05-13T23:01:12.005 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-05-13T23:01:12.005 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-05-13T23:01:12.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.131+0000 7f7ecab47640 1 -- 172.21.15.102:0/3787849559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ec40ff260 msgr2=0x7f7ec40ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:12.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.131+0000 7f7ecab47640 1 --2- 172.21.15.102:0/3787849559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ec40ff260 0x7f7ec40ff640 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009880 tx=0x7f7eb802bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.131+0000 7f7ecab47640 1 -- 172.21.15.102:0/3787849559 shutdown_connections 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.131+0000 7f7ecab47640 1 --2- 172.21.15.102:0/3787849559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ec40ffc10 0x7f7ec4103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.131+0000 7f7ecab47640 1 --2- 172.21.15.102:0/3787849559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ec40ff260 0x7f7ec40ff640 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.131+0000 7f7ecab47640 1 -- 172.21.15.102:0/3787849559 >> 172.21.15.102:0/3787849559 conn(0x7f7ec4074a50 msgr2=0x7f7ec4074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.131+0000 7f7ecab47640 1 -- 172.21.15.102:0/3787849559 shutdown_connections 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.131+0000 7f7ecab47640 1 -- 172.21.15.102:0/3787849559 wait complete. 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ecab47640 1 Processor -- start 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ecab47640 1 -- start start 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ecab47640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ec40ff260 0x7f7ec4196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ecab47640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ec41972c0 0x7f7ec4197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ecab47640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec419fdf0 con 0x7f7ec41972c0 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ecab47640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec419ff30 con 0x7f7ec40ff260 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ec3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ec41972c0 0x7f7ec4197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ec3fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ec41972c0 0x7f7ec4197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47558/0 (socket says 172.21.15.102:47558) 2024-05-13T23:01:12.109 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ec3fff640 1 -- 172.21.15.102:0/2979087374 learned_addr learned my addr 172.21.15.102:0/2979087374 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:12.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ec3fff640 1 -- 172.21.15.102:0/2979087374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ec40ff260 msgr2=0x7f7ec4196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:12.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ec3fff640 1 --2- 172.21.15.102:0/2979087374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ec40ff260 0x7f7ec4196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ec3fff640 1 -- 172.21.15.102:0/2979087374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb8009560 con 0x7f7ec41972c0 2024-05-13T23:01:12.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.132+0000 7f7ec3fff640 1 --2- 172.21.15.102:0/2979087374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ec41972c0 0x7f7ec4197720 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000b400 tx=0x7f7eb000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:12.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.133+0000 7f7ec1ffb640 1 -- 172.21.15.102:0/2979087374 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb0011800 con 0x7f7ec41972c0 2024-05-13T23:01:12.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.133+0000 7f7ecab47640 1 -- 172.21.15.102:0/2979087374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec41a0190 con 0x7f7ec41972c0 2024-05-13T23:01:12.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.133+0000 7f7ec1ffb640 1 -- 172.21.15.102:0/2979087374 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7eb0011e20 con 0x7f7ec41972c0 2024-05-13T23:01:12.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.133+0000 7f7ec1ffb640 1 -- 172.21.15.102:0/2979087374 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb000f2e0 con 0x7f7ec41972c0 2024-05-13T23:01:12.110 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.133+0000 7f7ecab47640 1 -- 172.21.15.102:0/2979087374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec41a06a0 con 0x7f7ec41972c0 2024-05-13T23:01:12.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.134+0000 7f7ecab47640 1 -- 172.21.15.102:0/2979087374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec404e040 con 0x7f7ec41972c0 2024-05-13T23:01:12.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.134+0000 7f7ec1ffb640 1 -- 172.21.15.102:0/2979087374 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7eb0010380 con 0x7f7ec41972c0 2024-05-13T23:01:12.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.135+0000 7f7ec1ffb640 1 --2- 172.21.15.102:0/2979087374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7eb406e170 0x7f7eb4070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:12.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.135+0000 7f7ec1ffb640 1 -- 172.21.15.102:0/2979087374 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f7eb008df60 con 0x7f7ec41972c0 2024-05-13T23:01:12.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.135+0000 7f7ec88e3640 1 --2- 172.21.15.102:0/2979087374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7eb406e170 0x7f7eb4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:12.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.135+0000 7f7ec88e3640 1 --2- 172.21.15.102:0/2979087374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7eb406e170 0x7f7eb4070630 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8036040 tx=0x7f7eb8000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:12.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.139+0000 7f7ec1ffb640 1 -- 172.21.15.102:0/2979087374 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7eb00c2850 con 0x7f7ec41972c0 2024-05-13T23:01:12.326 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.351+0000 7f7ecab47640 1 -- 172.21.15.102:0/2979087374 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f7ec4107480 con 0x7f7ec41972c0 2024-05-13T23:01:12.327 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.351+0000 7f7ec1ffb640 1 -- 172.21.15.102:0/2979087374 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v38) v1 ==== 107+0+5220 (secure 0 0 0) 0x7f7eb005ac50 con 0x7f7ec41972c0 2024-05-13T23:01:12.327 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:12.327 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44271,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/3836279348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3836279348},{"type":"v1","addr":"172.21.15.130:6825","nonce":3836279348}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44297,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6827/3806306097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3806306097},{"type":"v1","addr":"172.21.15.130:6827","nonce":3806306097}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:58:38.440863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34410":{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":0,"incarnation":30,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34410,"qdb_cluster":[34410]},"id":1}]} 2024-05-13T23:01:12.327 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 36 2024-05-13T23:01:12.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.353+0000 7f7ecab47640 1 -- 172.21.15.102:0/2979087374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7eb406e170 msgr2=0x7f7eb4070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:12.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.353+0000 7f7ecab47640 1 --2- 172.21.15.102:0/2979087374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7eb406e170 0x7f7eb4070630 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8036040 tx=0x7f7eb8000bc0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.353+0000 7f7ecab47640 1 -- 172.21.15.102:0/2979087374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ec41972c0 msgr2=0x7f7ec4197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:12.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.353+0000 7f7ecab47640 1 --2- 172.21.15.102:0/2979087374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ec41972c0 0x7f7ec4197720 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000b400 tx=0x7f7eb000b8c0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.354+0000 7f7ecab47640 1 -- 172.21.15.102:0/2979087374 shutdown_connections 2024-05-13T23:01:12.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.354+0000 7f7ecab47640 1 --2- 172.21.15.102:0/2979087374 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7eb406e170 0x7f7eb4070630 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.329 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.354+0000 7f7ecab47640 1 --2- 172.21.15.102:0/2979087374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ec41972c0 0x7f7ec4197720 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.354+0000 7f7ecab47640 1 --2- 172.21.15.102:0/2979087374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7ec40ff260 0x7f7ec4196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.354+0000 7f7ecab47640 1 -- 172.21.15.102:0/2979087374 >> 172.21.15.102:0/2979087374 conn(0x7f7ec4074a50 msgr2=0x7f7ec40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:12.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.354+0000 7f7ecab47640 1 -- 172.21.15.102:0/2979087374 shutdown_connections 2024-05-13T23:01:12.330 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.354+0000 7f7ecab47640 1 -- 172.21.15.102:0/2979087374 wait complete. 2024-05-13T23:01:12.340 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-05-13T23:01:12.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.465+0000 7fc69b35c640 1 -- 172.21.15.102:0/352014772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6940ff1a0 msgr2=0x7fc6940ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:12.442 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.465+0000 7fc69b35c640 1 --2- 172.21.15.102:0/352014772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6940ff1a0 0x7fc6940ff580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc690009880 tx=0x7fc69002bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.465+0000 7fc69b35c640 1 -- 172.21.15.102:0/352014772 shutdown_connections 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.465+0000 7fc69b35c640 1 --2- 172.21.15.102:0/352014772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6940ffb50 0x7fc694103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.465+0000 7fc69b35c640 1 --2- 172.21.15.102:0/352014772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6940ff1a0 0x7fc6940ff580 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.465+0000 7fc69b35c640 1 -- 172.21.15.102:0/352014772 >> 172.21.15.102:0/352014772 conn(0x7fc6940faa40 msgr2=0x7fc6940fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.465+0000 7fc69b35c640 1 -- 172.21.15.102:0/352014772 shutdown_connections 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.465+0000 7fc69b35c640 1 -- 172.21.15.102:0/352014772 wait complete. 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc69b35c640 1 Processor -- start 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc69b35c640 1 -- start start 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc69b35c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6940ff1a0 0x7fc694196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc69b35c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc694197250 0x7fc6941976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc69b35c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69419fd80 con 0x7fc6940ff1a0 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc69b35c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69419fec0 con 0x7fc694197250 2024-05-13T23:01:12.443 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc6990f8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6940ff1a0 0x7fc694196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:12.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc6990f8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6940ff1a0 0x7fc694196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47580/0 (socket says 172.21.15.102:47580) 2024-05-13T23:01:12.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc6990f8640 1 -- 172.21.15.102:0/3084741195 learned_addr learned my addr 172.21.15.102:0/3084741195 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:12.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc6990f8640 1 -- 172.21.15.102:0/3084741195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc694197250 msgr2=0x7fc6941976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:12.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc6990f8640 1 --2- 172.21.15.102:0/3084741195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc694197250 0x7fc6941976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.466+0000 7fc6990f8640 1 -- 172.21.15.102:0/3084741195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc690009560 con 0x7fc6940ff1a0 2024-05-13T23:01:12.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.467+0000 7fc6990f8640 1 --2- 172.21.15.102:0/3084741195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6940ff1a0 0x7fc694196d10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc690009430 tx=0x7fc690004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:12.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.467+0000 7fc68a7fc640 1 -- 172.21.15.102:0/3084741195 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6900035f0 con 0x7fc6940ff1a0 2024-05-13T23:01:12.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.467+0000 7fc69b35c640 1 -- 172.21.15.102:0/3084741195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6941a00c0 con 0x7fc6940ff1a0 2024-05-13T23:01:12.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.467+0000 7fc68a7fc640 1 -- 172.21.15.102:0/3084741195 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc690003c10 con 0x7fc6940ff1a0 2024-05-13T23:01:12.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.467+0000 7fc68a7fc640 1 -- 172.21.15.102:0/3084741195 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc690034440 con 0x7fc6940ff1a0 2024-05-13T23:01:12.444 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.467+0000 7fc69b35c640 1 -- 172.21.15.102:0/3084741195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6941a0570 con 0x7fc6940ff1a0 2024-05-13T23:01:12.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.468+0000 7fc69b35c640 1 -- 172.21.15.102:0/3084741195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69404e040 con 0x7fc6940ff1a0 2024-05-13T23:01:12.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.468+0000 7fc68a7fc640 1 -- 172.21.15.102:0/3084741195 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc690004fd0 con 0x7fc6940ff1a0 2024-05-13T23:01:12.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.469+0000 7fc68a7fc640 1 --2- 172.21.15.102:0/3084741195 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc68006e170 0x7fc680070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:12.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.469+0000 7fc68a7fc640 1 -- 172.21.15.102:0/3084741195 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fc6900b0720 con 0x7fc6940ff1a0 2024-05-13T23:01:12.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.469+0000 7fc6988f7640 1 --2- 172.21.15.102:0/3084741195 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc68006e170 0x7fc680070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:12.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.469+0000 7fc6988f7640 1 --2- 172.21.15.102:0/3084741195 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc68006e170 0x7fc680070630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc694069620 tx=0x7fc684006c90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:12.449 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.473+0000 7fc68a7fc640 1 -- 172.21.15.102:0/3084741195 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fc6900798e0 con 0x7fc6940ff1a0 2024-05-13T23:01:12.662 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.687+0000 7fc69b35c640 1 -- 172.21.15.102:0/3084741195 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7fc6940009c0 con 0x7fc6940ff1a0 2024-05-13T23:01:12.663 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.687+0000 7fc68a7fc640 1 -- 172.21.15.102:0/3084741195 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v38) v1 ==== 107+0+5227 (secure 0 0 0) 0x7fc6940009c0 con 0x7fc6940ff1a0 2024-05-13T23:01:12.663 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:12.663 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44271,"name":"cephfs.smithi130.hzjanp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6825/3836279348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3836279348},{"type":"v1","addr":"172.21.15.130:6825","nonce":3836279348}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44297,"name":"cephfs.smithi130.ovlxvg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6827/3806306097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3806306097},{"type":"v1","addr":"172.21.15.130:6827","nonce":3806306097}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-13T22:45:10.143899+0000","modified":"2024-05-13T22:58:39.442205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34410":{"gid":34410,"name":"cephfs.smithi102.cgkcwz","rank":0,"incarnation":30,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6827/3479984123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3479984123},{"type":"v1","addr":"172.21.15.102:6827","nonce":3479984123}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34414":{"gid":34414,"name":"cephfs.smithi102.sdylgv","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6829/1256538594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":1256538594},{"type":"v1","addr":"172.21.15.102:6829","nonce":1256538594}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34410,"qdb_cluster":[34410]},"id":1}]} 2024-05-13T23:01:12.663 INFO:teuthology.orchestra.run.smithi102.stderr:dumped fsmap epoch 37 2024-05-13T23:01:12.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.690+0000 7fc69b35c640 1 -- 172.21.15.102:0/3084741195 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc68006e170 msgr2=0x7fc680070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:12.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.690+0000 7fc69b35c640 1 --2- 172.21.15.102:0/3084741195 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc68006e170 0x7fc680070630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc694069620 tx=0x7fc684006c90 comp rx=0 tx=0).stop 2024-05-13T23:01:12.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.690+0000 7fc69b35c640 1 -- 172.21.15.102:0/3084741195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6940ff1a0 msgr2=0x7fc694196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:12.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.690+0000 7fc69b35c640 1 --2- 172.21.15.102:0/3084741195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6940ff1a0 0x7fc694196d10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc690009430 tx=0x7fc690004040 comp rx=0 tx=0).stop 2024-05-13T23:01:12.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.690+0000 7fc69b35c640 1 -- 172.21.15.102:0/3084741195 shutdown_connections 2024-05-13T23:01:12.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.690+0000 7fc69b35c640 1 --2- 172.21.15.102:0/3084741195 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fc68006e170 0x7fc680070630 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.690+0000 7fc69b35c640 1 --2- 172.21.15.102:0/3084741195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc694197250 0x7fc6941976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.690+0000 7fc69b35c640 1 --2- 172.21.15.102:0/3084741195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6940ff1a0 0x7fc694196d10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.690+0000 7fc69b35c640 1 -- 172.21.15.102:0/3084741195 >> 172.21.15.102:0/3084741195 conn(0x7fc6940faa40 msgr2=0x7fc6940fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:12.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.690+0000 7fc69b35c640 1 -- 172.21.15.102:0/3084741195 shutdown_connections 2024-05-13T23:01:12.666 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.690+0000 7fc69b35c640 1 -- 172.21.15.102:0/3084741195 wait complete. 2024-05-13T23:01:12.676 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-13T23:01:12.688 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-13T23:01:12.688 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T23:01:12.688 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-13T23:01:12.706 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T23:01:12.706 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-13T23:01:12.764 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-13T23:01:12.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:12 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2154625912' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-13T23:01:12.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:12 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/145399049' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-13T23:01:12.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:12 smithi130 ceph-mon[84265]: pgmap v302: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:12.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:12 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2979087374' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-05-13T23:01:12.905 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.930+0000 7fe540d92640 1 -- 172.21.15.102:0/2783354555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe53c0ff180 msgr2=0x7fe53c0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:12.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.930+0000 7fe540d92640 1 --2- 172.21.15.102:0/2783354555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe53c0ff180 0x7fe53c0ff560 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009880 tx=0x7fe52c02bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.930+0000 7fe540d92640 1 -- 172.21.15.102:0/2783354555 shutdown_connections 2024-05-13T23:01:12.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.930+0000 7fe540d92640 1 --2- 172.21.15.102:0/2783354555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe53c0ffb30 0x7fe53c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.930+0000 7fe540d92640 1 --2- 172.21.15.102:0/2783354555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe53c0ff180 0x7fe53c0ff560 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.930+0000 7fe540d92640 1 -- 172.21.15.102:0/2783354555 >> 172.21.15.102:0/2783354555 conn(0x7fe53c0fa9e0 msgr2=0x7fe53c0fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:12.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.930+0000 7fe540d92640 1 -- 172.21.15.102:0/2783354555 shutdown_connections 2024-05-13T23:01:12.906 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.930+0000 7fe540d92640 1 -- 172.21.15.102:0/2783354555 wait complete. 2024-05-13T23:01:12.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.931+0000 7fe540d92640 1 Processor -- start 2024-05-13T23:01:12.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.931+0000 7fe540d92640 1 -- start start 2024-05-13T23:01:12.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.931+0000 7fe540d92640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe53c0ff180 0x7fe53c194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:12.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.931+0000 7fe540d92640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe53c195210 0x7fe53c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:12.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.931+0000 7fe540d92640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c195810 con 0x7fe53c195210 2024-05-13T23:01:12.908 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.931+0000 7fe540d92640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c195950 con 0x7fe53c0ff180 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.931+0000 7fe533fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe53c195210 0x7fe53c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.931+0000 7fe533fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe53c195210 0x7fe53c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47598/0 (socket says 172.21.15.102:47598) 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.931+0000 7fe533fff640 1 -- 172.21.15.102:0/3871006119 learned_addr learned my addr 172.21.15.102:0/3871006119 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.931+0000 7fe53a59c640 1 --2- 172.21.15.102:0/3871006119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe53c0ff180 0x7fe53c194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.932+0000 7fe533fff640 1 -- 172.21.15.102:0/3871006119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe53c0ff180 msgr2=0x7fe53c194cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.932+0000 7fe533fff640 1 --2- 172.21.15.102:0/3871006119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe53c0ff180 0x7fe53c194cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.932+0000 7fe533fff640 1 -- 172.21.15.102:0/3871006119 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c009560 con 0x7fe53c195210 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.932+0000 7fe533fff640 1 --2- 172.21.15.102:0/3871006119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe53c195210 0x7fe53c1985c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe52400b3d0 tx=0x7fe52400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.932+0000 7fe5337fe640 1 -- 172.21.15.102:0/3871006119 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe524011800 con 0x7fe53c195210 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.932+0000 7fe540d92640 1 -- 172.21.15.102:0/3871006119 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53c198bc0 con 0x7fe53c195210 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.932+0000 7fe5337fe640 1 -- 172.21.15.102:0/3871006119 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe524011e20 con 0x7fe53c195210 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.932+0000 7fe5337fe640 1 -- 172.21.15.102:0/3871006119 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe52400f2e0 con 0x7fe53c195210 2024-05-13T23:01:12.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.932+0000 7fe540d92640 1 -- 172.21.15.102:0/3871006119 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53c199120 con 0x7fe53c195210 2024-05-13T23:01:12.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.933+0000 7fe540d92640 1 -- 172.21.15.102:0/3871006119 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53c04e040 con 0x7fe53c195210 2024-05-13T23:01:12.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.933+0000 7fe5337fe640 1 -- 172.21.15.102:0/3871006119 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe524010380 con 0x7fe53c195210 2024-05-13T23:01:12.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.934+0000 7fe5337fe640 1 --2- 172.21.15.102:0/3871006119 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe52806e230 0x7fe5280706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:12.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.934+0000 7fe5337fe640 1 -- 172.21.15.102:0/3871006119 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe52408df90 con 0x7fe53c195210 2024-05-13T23:01:12.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.934+0000 7fe53a59c640 1 --2- 172.21.15.102:0/3871006119 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe52806e230 0x7fe5280706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:12.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.934+0000 7fe53a59c640 1 --2- 172.21.15.102:0/3871006119 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe52806e230 0x7fe5280706f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe52c006010 tx=0x7fe52c005b50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:12.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:12.938+0000 7fe5337fe640 1 -- 172.21.15.102:0/3871006119 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fe524057150 con 0x7fe53c195210 2024-05-13T23:01:12.924 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:12 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2154625912' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-13T23:01:12.924 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:12 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/145399049' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-13T23:01:12.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:12 smithi102 ceph-mon[111694]: pgmap v302: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:12.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:12 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2979087374' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-05-13T23:01:13.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.115+0000 7fe540d92640 1 -- 172.21.15.102:0/3871006119 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe53c1960d0 con 0x7fe53c195210 2024-05-13T23:01:13.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.117+0000 7fe5337fe640 1 -- 172.21.15.102:0/3871006119 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17245 (secure 0 0 0) 0x7fe52405a810 con 0x7fe53c195210 2024-05-13T23:01:13.092 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:13.092 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":96,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","created":"2024-05-13T22:38:14.140940+0000","modified":"2024-05-13T22:58:22.091658+0000","last_up_change":"2024-05-13T22:57:16.054261+0000","last_in_change":"2024-05-13T22:43:35.225543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-13T22:42:39.969524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-13T22:45:08.878601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-13T22:45:09.369354+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d8cb2e3-52d6-45f5-94a1-3e8cc6535efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":90,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6803","nonce":307502084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6805","nonce":307502084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6809","nonce":307502084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6807","nonce":307502084}]},"public_addr":"172.21.15.102:6803/307502084","cluster_addr":"172.21.15.102:6805/307502084","heartbeat_back_addr":"172.21.15.102:6809/307502084","heartbeat_front_addr":"172.21.15.102:6807/307502084","state":["exists","up"]},{"osd":1,"uuid":"1d3eacb1-362c-4b6a-a959-7dea19c6521e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":66,"up_thru":90,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6811","nonce":272360893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6813","nonce":272360893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6817","nonce":272360893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6815","nonce":272360893}]},"public_addr":"172.21.15.102:6811/272360893","cluster_addr":"172.21.15.102:6813/272360893","heartbeat_back_addr":"172.21.15.102:6817/272360893","heartbeat_front_addr":"172.21.15.102:6815/272360893","state":["exists","up"]},{"osd":2,"uuid":"205b05a9-9d68-48ed-b87b-c4c677882eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6819","nonce":3530038065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6821","nonce":3530038065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6825","nonce":3530038065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6823","nonce":3530038065}]},"public_addr":"172.21.15.102:6819/3530038065","cluster_addr":"172.21.15.102:6821/3530038065","heartbeat_back_addr":"172.21.15.102:6825/3530038065","heartbeat_front_addr":"172.21.15.102:6823/3530038065","state":["exists","up"]},{"osd":3,"uuid":"d827066c-3201-479e-b142-27721c2761fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6801","nonce":3670165065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6803","nonce":3670165065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6807","nonce":3670165065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6805","nonce":3670165065}]},"public_addr":"172.21.15.130:6801/3670165065","cluster_addr":"172.21.15.130:6803/3670165065","heartbeat_back_addr":"172.21.15.130:6807/3670165065","heartbeat_front_addr":"172.21.15.130:6805/3670165065","state":["exists","up"]},{"osd":4,"uuid":"7d89edd4-8877-4cb8-a0af-019f9451bd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6809","nonce":3241450581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6811","nonce":3241450581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6815","nonce":3241450581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6813","nonce":3241450581}]},"public_addr":"172.21.15.130:6809/3241450581","cluster_addr":"172.21.15.130:6811/3241450581","heartbeat_back_addr":"172.21.15.130:6815/3241450581","heartbeat_front_addr":"172.21.15.130:6813/3241450581","state":["exists","up"]},{"osd":5,"uuid":"c0942077-8a53-4e3b-b411-f1763c592e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6817","nonce":3573323293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6819","nonce":3573323293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6823","nonce":3573323293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6821","nonce":3573323293}]},"public_addr":"172.21.15.130:6817/3573323293","cluster_addr":"172.21.15.130:6819/3573323293","heartbeat_back_addr":"172.21.15.130:6823/3573323293","heartbeat_front_addr":"172.21.15.130:6821/3573323293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:41:41.451627+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:09.327795+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:36.547203+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:00.627031+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:26.709919+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:52.576178+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":0,"to":1}]},{"pgid":"2.6","mappings":[{"from":3,"to":2}]},{"pgid":"2.e","mappings":[{"from":0,"to":1}]},{"pgid":"2.f","mappings":[{"from":0,"to":1}]},{"pgid":"2.15","mappings":[{"from":3,"to":2}]},{"pgid":"2.1d","mappings":[{"from":3,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":4,"to":5}]},{"pgid":"3.1b","mappings":[{"from":3,"to":1}]},{"pgid":"3.1d","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:6827/633282433":"2024-05-14T22:58:22.091628+0000","172.21.15.130:6826/633282433":"2024-05-14T22:58:22.091628+0000","172.21.15.102:6801/2015310067":"2024-05-14T22:52:45.995299+0000","172.21.15.130:0/1149656978":"2024-05-14T22:48:49.345907+0000","172.21.15.102:6827/3231884180":"2024-05-14T22:45:25.114218+0000","172.21.15.102:0/3978175820":"2024-05-14T22:52:45.995299+0000","172.21.15.102:6826/3231884180":"2024-05-14T22:45:25.114218+0000","172.21.15.130:0/2344917375":"2024-05-14T22:48:49.345907+0000","172.21.15.102:0/771792641":"2024-05-14T22:47:31.055945+0000","172.21.15.102:6801/2571859579":"2024-05-14T22:47:31.055945+0000","172.21.15.102:6828/2977397703":"2024-05-14T22:57:41.438543+0000","172.21.15.102:0/3812624194":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6800/2571859579":"2024-05-14T22:47:31.055945+0000","172.21.15.102:0/402887427":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/2015310067":"2024-05-14T22:52:45.995299+0000","172.21.15.102:6801/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.130:6829/3751205200":"2024-05-14T22:48:49.345907+0000","172.21.15.130:6828/3751205200":"2024-05-14T22:48:49.345907+0000","172.21.15.102:6800/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/3850189972":"2024-05-14T22:39:13.681560+0000","172.21.15.130:0/1881061148":"2024-05-14T22:48:49.345907+0000","172.21.15.102:0/2248276518":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/773571736":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/2675775894":"2024-05-14T22:47:31.055945+0000","172.21.15.102:6829/2977397703":"2024-05-14T22:57:41.438543+0000","172.21.15.102:0/457222314":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/1831496769":"2024-05-14T22:52:45.995299+0000","172.21.15.102:0/1238275992":"2024-05-14T22:47:31.055945+0000","172.21.15.130:0/3880089243":"2024-05-14T22:48:49.345907+0000","172.21.15.102:0/633075379":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/2096579260":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/233302973":"2024-05-14T22:47:31.055945+0000","172.21.15.130:0/495062226":"2024-05-14T22:48:49.345907+0000","172.21.15.130:0/1737286614":"2024-05-14T22:48:49.345907+0000","172.21.15.102:6801/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/17493483":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/1416077149":"2024-05-14T22:52:45.995299+0000","172.21.15.102:6801/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/1857247111":"2024-05-14T22:52:45.995299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-13T23:01:13.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.119+0000 7fe540d92640 1 -- 172.21.15.102:0/3871006119 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe52806e230 msgr2=0x7fe5280706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.119+0000 7fe540d92640 1 --2- 172.21.15.102:0/3871006119 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe52806e230 0x7fe5280706f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe52c006010 tx=0x7fe52c005b50 comp rx=0 tx=0).stop 2024-05-13T23:01:13.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.119+0000 7fe540d92640 1 -- 172.21.15.102:0/3871006119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe53c195210 msgr2=0x7fe53c1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.119+0000 7fe540d92640 1 --2- 172.21.15.102:0/3871006119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe53c195210 0x7fe53c1985c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe52400b3d0 tx=0x7fe52400b890 comp rx=0 tx=0).stop 2024-05-13T23:01:13.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.119+0000 7fe540d92640 1 -- 172.21.15.102:0/3871006119 shutdown_connections 2024-05-13T23:01:13.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.119+0000 7fe540d92640 1 --2- 172.21.15.102:0/3871006119 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fe52806e230 0x7fe5280706f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.119+0000 7fe540d92640 1 --2- 172.21.15.102:0/3871006119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe53c195210 0x7fe53c1985c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.119+0000 7fe540d92640 1 --2- 172.21.15.102:0/3871006119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe53c0ff180 0x7fe53c194cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.119+0000 7fe540d92640 1 -- 172.21.15.102:0/3871006119 >> 172.21.15.102:0/3871006119 conn(0x7fe53c0fa9e0 msgr2=0x7fe53c0fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:13.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.119+0000 7fe540d92640 1 -- 172.21.15.102:0/3871006119 shutdown_connections 2024-05-13T23:01:13.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.119+0000 7fe540d92640 1 -- 172.21.15.102:0/3871006119 wait complete. 2024-05-13T23:01:13.105 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-13T22:42:39.969524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-13T22:45:08.878601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-13T22:45:09.369354+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-13T23:01:13.106 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-13T23:01:13.212 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.234+0000 7f7ce2408640 1 -- 172.21.15.102:0/3992609647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cdc0675c0 msgr2=0x7f7cdc0679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.234+0000 7f7ce2408640 1 --2- 172.21.15.102:0/3992609647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cdc0675c0 0x7f7cdc0679a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009880 tx=0x7f7ccc02bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.234+0000 7f7ce2408640 1 -- 172.21.15.102:0/3992609647 shutdown_connections 2024-05-13T23:01:13.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.234+0000 7f7ce2408640 1 --2- 172.21.15.102:0/3992609647 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7cdc067f70 0x7f7cdc104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.234+0000 7f7ce2408640 1 --2- 172.21.15.102:0/3992609647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cdc0675c0 0x7f7cdc0679a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.234+0000 7f7ce2408640 1 -- 172.21.15.102:0/3992609647 >> 172.21.15.102:0/3992609647 conn(0x7f7cdc074a50 msgr2=0x7f7cdc074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:13.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.234+0000 7f7ce2408640 1 -- 172.21.15.102:0/3992609647 shutdown_connections 2024-05-13T23:01:13.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.234+0000 7f7ce2408640 1 -- 172.21.15.102:0/3992609647 wait complete. 2024-05-13T23:01:13.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.234+0000 7f7ce2408640 1 Processor -- start 2024-05-13T23:01:13.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.234+0000 7f7ce2408640 1 -- start start 2024-05-13T23:01:13.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7ce2408640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7cdc0675c0 0x7f7cdc195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:13.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7ce2408640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cdc1962f0 0x7f7cdc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:13.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7ce2408640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc1968f0 con 0x7f7cdc1962f0 2024-05-13T23:01:13.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7ce2408640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc196a60 con 0x7f7cdc0675c0 2024-05-13T23:01:13.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cdc1962f0 0x7f7cdc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:13.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cdc1962f0 0x7f7cdc19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47618/0 (socket says 172.21.15.102:47618) 2024-05-13T23:01:13.214 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7cdb7fe640 1 -- 172.21.15.102:0/3598302782 learned_addr learned my addr 172.21.15.102:0/3598302782 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:13.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7cdbfff640 1 --2- 172.21.15.102:0/3598302782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7cdc0675c0 0x7f7cdc195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:13.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7cdb7fe640 1 -- 172.21.15.102:0/3598302782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7cdc0675c0 msgr2=0x7f7cdc195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7cdb7fe640 1 --2- 172.21.15.102:0/3598302782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7cdc0675c0 0x7f7cdc195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7cdb7fe640 1 -- 172.21.15.102:0/3598302782 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc009560 con 0x7f7cdc1962f0 2024-05-13T23:01:13.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7cdb7fe640 1 --2- 172.21.15.102:0/3598302782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cdc1962f0 0x7f7cdc19a750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c920 tx=0x7f7cc400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:13.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7cd97fa640 1 -- 172.21.15.102:0/3598302782 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc4007b80 con 0x7f7cdc1962f0 2024-05-13T23:01:13.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7ce2408640 1 -- 172.21.15.102:0/3598302782 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cdc19ad50 con 0x7f7cdc1962f0 2024-05-13T23:01:13.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7cd97fa640 1 -- 172.21.15.102:0/3598302782 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cc4007ce0 con 0x7f7cdc1962f0 2024-05-13T23:01:13.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7cd97fa640 1 -- 172.21.15.102:0/3598302782 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc40187d0 con 0x7f7cdc1962f0 2024-05-13T23:01:13.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.235+0000 7f7ce2408640 1 -- 172.21.15.102:0/3598302782 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cdc19b2b0 con 0x7f7cdc1962f0 2024-05-13T23:01:13.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.236+0000 7f7ce2408640 1 -- 172.21.15.102:0/3598302782 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cdc19b590 con 0x7f7cdc1962f0 2024-05-13T23:01:13.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.237+0000 7f7cd97fa640 1 -- 172.21.15.102:0/3598302782 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7cc400f450 con 0x7f7cdc1962f0 2024-05-13T23:01:13.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.237+0000 7f7cd97fa640 1 --2- 172.21.15.102:0/3598302782 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7cc806e1c0 0x7f7cc8070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:13.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.238+0000 7f7cdbfff640 1 --2- 172.21.15.102:0/3598302782 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7cc806e1c0 0x7f7cc8070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:13.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.238+0000 7f7cd97fa640 1 -- 172.21.15.102:0/3598302782 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f7cc408e520 con 0x7f7cdc1962f0 2024-05-13T23:01:13.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.238+0000 7f7cdbfff640 1 --2- 172.21.15.102:0/3598302782 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7cc806e1c0 0x7f7cc8070680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006010 tx=0x7f7ccc000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:13.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.242+0000 7f7cd97fa640 1 -- 172.21.15.102:0/3598302782 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7cc4057640 con 0x7f7cdc1962f0 2024-05-13T23:01:13.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.418+0000 7f7ce2408640 1 -- 172.21.15.102:0/3598302782 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7cdc197100 con 0x7f7cdc1962f0 2024-05-13T23:01:13.393 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.418+0000 7f7cd97fa640 1 -- 172.21.15.102:0/3598302782 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cc405ad00 con 0x7f7cdc1962f0 2024-05-13T23:01:13.393 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-05-13T23:01:13.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.420+0000 7f7ce2408640 1 -- 172.21.15.102:0/3598302782 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7cc806e1c0 msgr2=0x7f7cc8070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.420+0000 7f7ce2408640 1 --2- 172.21.15.102:0/3598302782 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7cc806e1c0 0x7f7cc8070680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006010 tx=0x7f7ccc000bc0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.420+0000 7f7ce2408640 1 -- 172.21.15.102:0/3598302782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cdc1962f0 msgr2=0x7f7cdc19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.420+0000 7f7ce2408640 1 --2- 172.21.15.102:0/3598302782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cdc1962f0 0x7f7cdc19a750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c920 tx=0x7f7cc400cde0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.420+0000 7f7ce2408640 1 -- 172.21.15.102:0/3598302782 shutdown_connections 2024-05-13T23:01:13.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.420+0000 7f7ce2408640 1 --2- 172.21.15.102:0/3598302782 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7cc806e1c0 0x7f7cc8070680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.420+0000 7f7ce2408640 1 --2- 172.21.15.102:0/3598302782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7cdc1962f0 0x7f7cdc19a750 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.420+0000 7f7ce2408640 1 --2- 172.21.15.102:0/3598302782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7cdc0675c0 0x7f7cdc195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.420+0000 7f7ce2408640 1 -- 172.21.15.102:0/3598302782 >> 172.21.15.102:0/3598302782 conn(0x7f7cdc074a50 msgr2=0x7f7cdc0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:13.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.420+0000 7f7ce2408640 1 -- 172.21.15.102:0/3598302782 shutdown_connections 2024-05-13T23:01:13.396 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.420+0000 7f7ce2408640 1 -- 172.21.15.102:0/3598302782 wait complete. 2024-05-13T23:01:13.406 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.535+0000 7f7b9b380640 1 -- 172.21.15.102:0/914640971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b940fe720 msgr2=0x7f7b940feb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.535+0000 7f7b9b380640 1 --2- 172.21.15.102:0/914640971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b940fe720 0x7f7b940feb60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400a990 tx=0x7f7b8402d010 comp rx=0 tx=0).stop 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.535+0000 7f7b9b380640 1 -- 172.21.15.102:0/914640971 shutdown_connections 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9b380640 1 --2- 172.21.15.102:0/914640971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b940fe720 0x7f7b940feb60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9b380640 1 --2- 172.21.15.102:0/914640971 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b941046b0 0x7f7b94104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9b380640 1 -- 172.21.15.102:0/914640971 >> 172.21.15.102:0/914640971 conn(0x7f7b94075250 msgr2=0x7f7b94075660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9b380640 1 -- 172.21.15.102:0/914640971 shutdown_connections 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9b380640 1 -- 172.21.15.102:0/914640971 wait complete. 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9b380640 1 Processor -- start 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9b380640 1 -- start start 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9b380640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b940fe720 0x7f7b94197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:13.513 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9b380640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b941046b0 0x7f7b94198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9b380640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b94198ae0 con 0x7f7b941046b0 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9b380640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9419c8b0 con 0x7f7b940fe720 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9891b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b941046b0 0x7f7b94198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9891b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b941046b0 0x7f7b94198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47632/0 (socket says 172.21.15.102:47632) 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.536+0000 7f7b9891b640 1 -- 172.21.15.102:0/1637163914 learned_addr learned my addr 172.21.15.102:0/1637163914 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.537+0000 7f7b9911c640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b940fe720 0x7f7b94197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.537+0000 7f7b9891b640 1 -- 172.21.15.102:0/1637163914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b940fe720 msgr2=0x7f7b94197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.537+0000 7f7b9891b640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b940fe720 0x7f7b94197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.537+0000 7f7b9891b640 1 -- 172.21.15.102:0/1637163914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8400a670 con 0x7f7b941046b0 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.537+0000 7f7b9911c640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b940fe720 0x7f7b94197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.537+0000 7f7b9891b640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b941046b0 0x7f7b94198450 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7b84005ab0 tx=0x7f7b840051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.537+0000 7f7b8a7fc640 1 -- 172.21.15.102:0/1637163914 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b84004420 con 0x7f7b941046b0 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.537+0000 7f7b9b380640 1 -- 172.21.15.102:0/1637163914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9419cb30 con 0x7f7b941046b0 2024-05-13T23:01:13.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.537+0000 7f7b8a7fc640 1 -- 172.21.15.102:0/1637163914 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b84035e40 con 0x7f7b941046b0 2024-05-13T23:01:13.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.537+0000 7f7b8a7fc640 1 -- 172.21.15.102:0/1637163914 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b84003820 con 0x7f7b941046b0 2024-05-13T23:01:13.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.537+0000 7f7b9b380640 1 -- 172.21.15.102:0/1637163914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9419d090 con 0x7f7b941046b0 2024-05-13T23:01:13.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.538+0000 7f7b9b380640 1 -- 172.21.15.102:0/1637163914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9404e040 con 0x7f7b941046b0 2024-05-13T23:01:13.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.538+0000 7f7b8a7fc640 1 -- 172.21.15.102:0/1637163914 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7b84035980 con 0x7f7b941046b0 2024-05-13T23:01:13.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.539+0000 7f7b8a7fc640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7b8006e170 0x7f7b80070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:13.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.539+0000 7f7b8a7fc640 1 -- 172.21.15.102:0/1637163914 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f7b840b0fc0 con 0x7f7b941046b0 2024-05-13T23:01:13.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.539+0000 7f7b9911c640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7b8006e170 0x7f7b80070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:13.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.539+0000 7f7b9911c640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7b8006e170 0x7f7b80070630 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009b40 tx=0x7f7b90005e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:13.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.543+0000 7f7b8a7fc640 1 -- 172.21.15.102:0/1637163914 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7b8407a180 con 0x7f7b941046b0 2024-05-13T23:01:13.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:13 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3084741195' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-05-13T23:01:13.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:13 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3871006119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T23:01:13.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:13 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3598302782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-13T23:01:13.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.720+0000 7f7b9b380640 1 -- 172.21.15.102:0/1637163914 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f7b94106ba0 con 0x7f7b941046b0 2024-05-13T23:01:13.695 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.720+0000 7f7b8a7fc640 1 -- 172.21.15.102:0/1637163914 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7b8407d840 con 0x7f7b941046b0 2024-05-13T23:01:13.695 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-05-13T23:01:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.722+0000 7f7b9b380640 1 -- 172.21.15.102:0/1637163914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7b8006e170 msgr2=0x7f7b80070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.722+0000 7f7b9b380640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7b8006e170 0x7f7b80070630 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009b40 tx=0x7f7b90005e50 comp rx=0 tx=0).stop 2024-05-13T23:01:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.722+0000 7f7b9b380640 1 -- 172.21.15.102:0/1637163914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b941046b0 msgr2=0x7f7b94198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.722+0000 7f7b9b380640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b941046b0 0x7f7b94198450 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7b84005ab0 tx=0x7f7b840051e0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.722+0000 7f7b9b380640 1 -- 172.21.15.102:0/1637163914 shutdown_connections 2024-05-13T23:01:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.722+0000 7f7b9b380640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f7b8006e170 0x7f7b80070630 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.722+0000 7f7b9b380640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b941046b0 0x7f7b94198450 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.722+0000 7f7b9b380640 1 --2- 172.21.15.102:0/1637163914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7b940fe720 0x7f7b94197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.722+0000 7f7b9b380640 1 -- 172.21.15.102:0/1637163914 >> 172.21.15.102:0/1637163914 conn(0x7f7b94075250 msgr2=0x7f7b940fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.722+0000 7f7b9b380640 1 -- 172.21.15.102:0/1637163914 shutdown_connections 2024-05-13T23:01:13.698 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.722+0000 7f7b9b380640 1 -- 172.21.15.102:0/1637163914 wait complete. 2024-05-13T23:01:13.709 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-13T23:01:13.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.836+0000 7f6da8b00640 1 -- 172.21.15.102:0/956109192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da4100c50 msgr2=0x7f6da4104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.836+0000 7f6da8b00640 1 --2- 172.21.15.102:0/956109192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da4100c50 0x7f6da4104c90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009880 tx=0x7f6d9802bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.813 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.836+0000 7f6da8b00640 1 -- 172.21.15.102:0/956109192 shutdown_connections 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.836+0000 7f6da8b00640 1 --2- 172.21.15.102:0/956109192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da4100c50 0x7f6da4104c90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.836+0000 7f6da8b00640 1 --2- 172.21.15.102:0/956109192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6da41002a0 0x7f6da4100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.836+0000 7f6da8b00640 1 -- 172.21.15.102:0/956109192 >> 172.21.15.102:0/956109192 conn(0x7f6da40fbb20 msgr2=0x7f6da40fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.836+0000 7f6da8b00640 1 -- 172.21.15.102:0/956109192 shutdown_connections 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.836+0000 7f6da8b00640 1 -- 172.21.15.102:0/956109192 wait complete. 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da8b00640 1 Processor -- start 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da8b00640 1 -- start start 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da8b00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da41002a0 0x7f6da4195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da8b00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6da41963c0 0x7f6da419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da8b00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da41969c0 con 0x7f6da41002a0 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da8b00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da4196b30 con 0x7f6da41963c0 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da259c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da41002a0 0x7f6da4195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da259c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da41002a0 0x7f6da4195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47642/0 (socket says 172.21.15.102:47642) 2024-05-13T23:01:13.814 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da259c640 1 -- 172.21.15.102:0/1480943079 learned_addr learned my addr 172.21.15.102:0/1480943079 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:13.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da259c640 1 -- 172.21.15.102:0/1480943079 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6da41963c0 msgr2=0x7f6da419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da259c640 1 --2- 172.21.15.102:0/1480943079 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6da41963c0 0x7f6da419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.837+0000 7f6da259c640 1 -- 172.21.15.102:0/1480943079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d98009560 con 0x7f6da41002a0 2024-05-13T23:01:13.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.838+0000 7f6da259c640 1 --2- 172.21.15.102:0/1480943079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da41002a0 0x7f6da4195e80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400b450 tx=0x7f6d9400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:13.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.838+0000 7f6d937fe640 1 -- 172.21.15.102:0/1480943079 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d94011860 con 0x7f6da41002a0 2024-05-13T23:01:13.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.838+0000 7f6da8b00640 1 -- 172.21.15.102:0/1480943079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da419ae20 con 0x7f6da41002a0 2024-05-13T23:01:13.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.838+0000 7f6d937fe640 1 -- 172.21.15.102:0/1480943079 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d94011e80 con 0x7f6da41002a0 2024-05-13T23:01:13.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.838+0000 7f6d937fe640 1 -- 172.21.15.102:0/1480943079 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d9400f570 con 0x7f6da41002a0 2024-05-13T23:01:13.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.838+0000 7f6da8b00640 1 -- 172.21.15.102:0/1480943079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da419b380 con 0x7f6da41002a0 2024-05-13T23:01:13.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.839+0000 7f6da8b00640 1 -- 172.21.15.102:0/1480943079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da404e040 con 0x7f6da41002a0 2024-05-13T23:01:13.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.839+0000 7f6d937fe640 1 -- 172.21.15.102:0/1480943079 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6d940103e0 con 0x7f6da41002a0 2024-05-13T23:01:13.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.840+0000 7f6d937fe640 1 --2- 172.21.15.102:0/1480943079 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6d8c06e170 0x7f6d8c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:13.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.840+0000 7f6d937fe640 1 -- 172.21.15.102:0/1480943079 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6d9408e1c0 con 0x7f6da41002a0 2024-05-13T23:01:13.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.840+0000 7f6da1d9b640 1 --2- 172.21.15.102:0/1480943079 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6d8c06e170 0x7f6d8c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:13.816 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.840+0000 7f6da1d9b640 1 --2- 172.21.15.102:0/1480943079 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6d8c06e170 0x7f6d8c070630 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6d98036040 tx=0x7f6d98000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:13.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:13.844+0000 7f6d937fe640 1 -- 172.21.15.102:0/1480943079 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f6d94057380 con 0x7f6da41002a0 2024-05-13T23:01:13.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:13 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3084741195' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-05-13T23:01:13.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:13 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3871006119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T23:01:13.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:13 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3598302782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-13T23:01:13.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.018+0000 7f6da8b00640 1 -- 172.21.15.102:0/1480943079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f6da4197290 con 0x7f6da41002a0 2024-05-13T23:01:13.994 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.019+0000 7f6d937fe640 1 -- 172.21.15.102:0/1480943079 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7f6d9405aa40 con 0x7f6da41002a0 2024-05-13T23:01:13.994 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-05-13T23:01:13.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.021+0000 7f6da8b00640 1 -- 172.21.15.102:0/1480943079 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6d8c06e170 msgr2=0x7f6d8c070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.021+0000 7f6da8b00640 1 --2- 172.21.15.102:0/1480943079 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6d8c06e170 0x7f6d8c070630 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6d98036040 tx=0x7f6d98000bc0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.021+0000 7f6da8b00640 1 -- 172.21.15.102:0/1480943079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da41002a0 msgr2=0x7f6da4195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:13.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.021+0000 7f6da8b00640 1 --2- 172.21.15.102:0/1480943079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da41002a0 0x7f6da4195e80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400b450 tx=0x7f6d9400b910 comp rx=0 tx=0).stop 2024-05-13T23:01:13.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.021+0000 7f6da8b00640 1 -- 172.21.15.102:0/1480943079 shutdown_connections 2024-05-13T23:01:13.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.021+0000 7f6da8b00640 1 --2- 172.21.15.102:0/1480943079 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f6d8c06e170 0x7f6d8c070630 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.021+0000 7f6da8b00640 1 --2- 172.21.15.102:0/1480943079 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6da41963c0 0x7f6da419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.021+0000 7f6da8b00640 1 --2- 172.21.15.102:0/1480943079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6da41002a0 0x7f6da4195e80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:13.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.021+0000 7f6da8b00640 1 -- 172.21.15.102:0/1480943079 >> 172.21.15.102:0/1480943079 conn(0x7f6da40fbb20 msgr2=0x7f6da40fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:13.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.021+0000 7f6da8b00640 1 -- 172.21.15.102:0/1480943079 shutdown_connections 2024-05-13T23:01:13.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.021+0000 7f6da8b00640 1 -- 172.21.15.102:0/1480943079 wait complete. 2024-05-13T23:01:14.008 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-13T23:01:14.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.133+0000 7fbe0c75c640 1 -- 172.21.15.102:0/398570626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe04067350 msgr2=0x7fbe040677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.133+0000 7fbe0c75c640 1 --2- 172.21.15.102:0/398570626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe04067350 0x7fbe040677b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009880 tx=0x7fbdf802bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.133+0000 7fbe0c75c640 1 -- 172.21.15.102:0/398570626 shutdown_connections 2024-05-13T23:01:14.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0c75c640 1 --2- 172.21.15.102:0/398570626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe04067350 0x7fbe040677b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0c75c640 1 --2- 172.21.15.102:0/398570626 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe041061b0 0x7fbe04106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0c75c640 1 -- 172.21.15.102:0/398570626 >> 172.21.15.102:0/398570626 conn(0x7fbe04075b40 msgr2=0x7fbe04075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:14.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0c75c640 1 -- 172.21.15.102:0/398570626 shutdown_connections 2024-05-13T23:01:14.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0c75c640 1 -- 172.21.15.102:0/398570626 wait complete. 2024-05-13T23:01:14.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0c75c640 1 Processor -- start 2024-05-13T23:01:14.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0c75c640 1 -- start start 2024-05-13T23:01:14.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0c75c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe04067350 0x7fbe04195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0c75c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe041061b0 0x7fbe041961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0c75c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041968d0 con 0x7fbe04067350 2024-05-13T23:01:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0c75c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041995a0 con 0x7fbe041061b0 2024-05-13T23:01:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0a4f8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe04067350 0x7fbe04195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0a4f8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe04067350 0x7fbe04195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47658/0 (socket says 172.21.15.102:47658) 2024-05-13T23:01:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.134+0000 7fbe0a4f8640 1 -- 172.21.15.102:0/3169166583 learned_addr learned my addr 172.21.15.102:0/3169166583 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.135+0000 7fbe0a4f8640 1 -- 172.21.15.102:0/3169166583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe041061b0 msgr2=0x7fbe041961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.135+0000 7fbe0a4f8640 1 --2- 172.21.15.102:0/3169166583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe041061b0 0x7fbe041961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.135+0000 7fbe0a4f8640 1 -- 172.21.15.102:0/3169166583 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf8009560 con 0x7fbe04067350 2024-05-13T23:01:14.112 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.135+0000 7fbe0a4f8640 1 --2- 172.21.15.102:0/3169166583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe04067350 0x7fbe04195cb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400ea00 tx=0x7fbdf400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:14.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.135+0000 7fbe037fe640 1 -- 172.21.15.102:0/3169166583 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdf400cd50 con 0x7fbe04067350 2024-05-13T23:01:14.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.135+0000 7fbe037fe640 1 -- 172.21.15.102:0/3169166583 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdf4005020 con 0x7fbe04067350 2024-05-13T23:01:14.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.135+0000 7fbe0c75c640 1 -- 172.21.15.102:0/3169166583 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe04199880 con 0x7fbe04067350 2024-05-13T23:01:14.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.135+0000 7fbe037fe640 1 -- 172.21.15.102:0/3169166583 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdf40116a0 con 0x7fbe04067350 2024-05-13T23:01:14.113 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.135+0000 7fbe0c75c640 1 -- 172.21.15.102:0/3169166583 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe04199de0 con 0x7fbe04067350 2024-05-13T23:01:14.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.136+0000 7fbe0c75c640 1 -- 172.21.15.102:0/3169166583 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0404e040 con 0x7fbe04067350 2024-05-13T23:01:14.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.136+0000 7fbe037fe640 1 -- 172.21.15.102:0/3169166583 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbdf4005190 con 0x7fbe04067350 2024-05-13T23:01:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.137+0000 7fbe037fe640 1 --2- 172.21.15.102:0/3169166583 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbdec06e230 0x7fbdec0706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.137+0000 7fbe037fe640 1 -- 172.21.15.102:0/3169166583 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fbdf408e670 con 0x7fbe04067350 2024-05-13T23:01:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.137+0000 7fbe09cf7640 1 --2- 172.21.15.102:0/3169166583 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbdec06e230 0x7fbdec0706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:14.115 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.137+0000 7fbe09cf7640 1 --2- 172.21.15.102:0/3169166583 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbdec06e230 0x7fbdec0706f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8036040 tx=0x7fbdf8000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:14.116 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.141+0000 7fbe037fe640 1 -- 172.21.15.102:0/3169166583 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fbdf4057830 con 0x7fbe04067350 2024-05-13T23:01:14.305 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.330+0000 7fbe0c75c640 1 -- 172.21.15.102:0/3169166583 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbe04103390 con 0x7fbe04067350 2024-05-13T23:01:14.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.330+0000 7fbe037fe640 1 -- 172.21.15.102:0/3169166583 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v96) v1 ==== 81+0+2458 (secure 0 0 0) 0x7fbdf401f9b0 con 0x7fbe04067350 2024-05-13T23:01:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:6827/633282433 2024-05-14T22:58:22.091628+0000 2024-05-13T23:01:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:6826/633282433 2024-05-14T22:58:22.091628+0000 2024-05-13T23:01:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/2015310067 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/1149656978 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6827/3231884180 2024-05-14T22:45:25.114218+0000 2024-05-13T23:01:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/3978175820 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6826/3231884180 2024-05-14T22:45:25.114218+0000 2024-05-13T23:01:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/2344917375 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/771792641 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/2571859579 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6828/2977397703 2024-05-14T22:57:41.438543+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/3812624194 2024-05-14T22:38:41.659063+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/2571859579 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/402887427 2024-05-14T22:40:23.651424+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/263535069 2024-05-14T22:40:23.651424+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/2015310067 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/4002480395 2024-05-14T22:38:41.659063+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:6829/3751205200 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:6828/3751205200 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/4002480395 2024-05-14T22:38:41.659063+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/3850189972 2024-05-14T22:39:13.681560+0000 2024-05-13T23:01:14.307 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/1881061148 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/2248276518 2024-05-14T22:39:13.681560+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/773571736 2024-05-14T22:39:13.681560+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/2675775894 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6829/2977397703 2024-05-14T22:57:41.438543+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/457222314 2024-05-14T22:40:23.651424+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1831496769 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1238275992 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/3880089243 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/633075379 2024-05-14T22:38:41.659063+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/2096579260 2024-05-14T22:38:41.659063+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/233302973 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/495062226 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/1737286614 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/263535069 2024-05-14T22:40:23.651424+0000 2024-05-13T23:01:14.308 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/17493483 2024-05-14T22:40:23.651424+0000 2024-05-13T23:01:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/358342393 2024-05-14T22:39:13.681560+0000 2024-05-13T23:01:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1416077149 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/358342393 2024-05-14T22:39:13.681560+0000 2024-05-13T23:01:14.309 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1857247111 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:listed 41 entries 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.332+0000 7fbe0c75c640 1 -- 172.21.15.102:0/3169166583 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbdec06e230 msgr2=0x7fbdec0706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.332+0000 7fbe0c75c640 1 --2- 172.21.15.102:0/3169166583 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbdec06e230 0x7fbdec0706f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8036040 tx=0x7fbdf8000bc0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.332+0000 7fbe0c75c640 1 -- 172.21.15.102:0/3169166583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe04067350 msgr2=0x7fbe04195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.332+0000 7fbe0c75c640 1 --2- 172.21.15.102:0/3169166583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe04067350 0x7fbe04195cb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400ea00 tx=0x7fbdf400eec0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.332+0000 7fbe0c75c640 1 -- 172.21.15.102:0/3169166583 shutdown_connections 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.332+0000 7fbe0c75c640 1 --2- 172.21.15.102:0/3169166583 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fbdec06e230 0x7fbdec0706f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.332+0000 7fbe0c75c640 1 --2- 172.21.15.102:0/3169166583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe041061b0 0x7fbe041961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.332+0000 7fbe0c75c640 1 --2- 172.21.15.102:0/3169166583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe04067350 0x7fbe04195cb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.332+0000 7fbe0c75c640 1 -- 172.21.15.102:0/3169166583 >> 172.21.15.102:0/3169166583 conn(0x7fbe04075b40 msgr2=0x7fbe040fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.332+0000 7fbe0c75c640 1 -- 172.21.15.102:0/3169166583 shutdown_connections 2024-05-13T23:01:14.310 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.332+0000 7fbe0c75c640 1 -- 172.21.15.102:0/3169166583 wait complete. 2024-05-13T23:01:14.318 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-05-13T23:01:14.319 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-13T23:01:14.336 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-13T23:01:14.478 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.501+0000 7f2ce74dc640 1 -- 172.21.15.102:0/2371695678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce01011e0 msgr2=0x7f2ce01015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.501+0000 7f2ce74dc640 1 --2- 172.21.15.102:0/2371695678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce01011e0 0x7f2ce01015c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009880 tx=0x7f2cdc02bd70 comp rx=0 tx=0).stop 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.501+0000 7f2ce74dc640 1 -- 172.21.15.102:0/2371695678 shutdown_connections 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.501+0000 7f2ce74dc640 1 --2- 172.21.15.102:0/2371695678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ce0101b00 0x7f2ce0109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.501+0000 7f2ce74dc640 1 --2- 172.21.15.102:0/2371695678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce01011e0 0x7f2ce01015c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.501+0000 7f2ce74dc640 1 -- 172.21.15.102:0/2371695678 >> 172.21.15.102:0/2371695678 conn(0x7f2ce00faa20 msgr2=0x7f2ce00fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.501+0000 7f2ce74dc640 1 -- 172.21.15.102:0/2371695678 shutdown_connections 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.501+0000 7f2ce74dc640 1 -- 172.21.15.102:0/2371695678 wait complete. 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce74dc640 1 Processor -- start 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce74dc640 1 -- start start 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce74dc640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ce01011e0 0x7f2ce019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce74dc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0101b00 0x7f2ce019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce74dc640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0195b40 con 0x7f2ce0101b00 2024-05-13T23:01:14.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce74dc640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0195cb0 con 0x7f2ce01011e0 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce4a77640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0101b00 0x7f2ce019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce4a77640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0101b00 0x7f2ce019c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47682/0 (socket says 172.21.15.102:47682) 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce4a77640 1 -- 172.21.15.102:0/3510106099 learned_addr learned my addr 172.21.15.102:0/3510106099 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce5278640 1 --2- 172.21.15.102:0/3510106099 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ce01011e0 0x7f2ce019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce4a77640 1 -- 172.21.15.102:0/3510106099 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ce01011e0 msgr2=0x7f2ce019bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.502+0000 7f2ce4a77640 1 --2- 172.21.15.102:0/3510106099 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ce01011e0 0x7f2ce019bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.503+0000 7f2ce4a77640 1 -- 172.21.15.102:0/3510106099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc009560 con 0x7f2ce0101b00 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.503+0000 7f2ce4a77640 1 --2- 172.21.15.102:0/3510106099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0101b00 0x7f2ce019c010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000c8f0 tx=0x7f2cd000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.503+0000 7f2cd67fc640 1 -- 172.21.15.102:0/3510106099 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd0007b80 con 0x7f2ce0101b00 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.503+0000 7f2cd67fc640 1 -- 172.21.15.102:0/3510106099 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cd000ae50 con 0x7f2ce0101b00 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.503+0000 7f2cd67fc640 1 -- 172.21.15.102:0/3510106099 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd0004e50 con 0x7f2ce0101b00 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.503+0000 7f2ce74dc640 1 -- 172.21.15.102:0/3510106099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce0195f90 con 0x7f2ce0101b00 2024-05-13T23:01:14.480 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.503+0000 7f2ce74dc640 1 -- 172.21.15.102:0/3510106099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce01964a0 con 0x7f2ce0101b00 2024-05-13T23:01:14.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.504+0000 7f2ce74dc640 1 -- 172.21.15.102:0/3510106099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce004e040 con 0x7f2ce0101b00 2024-05-13T23:01:14.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.504+0000 7f2cd67fc640 1 -- 172.21.15.102:0/3510106099 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2cd0005020 con 0x7f2ce0101b00 2024-05-13T23:01:14.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.505+0000 7f2cd67fc640 1 --2- 172.21.15.102:0/3510106099 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2ccc06e1d0 0x7f2ccc070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:14.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.505+0000 7f2ce5278640 1 --2- 172.21.15.102:0/3510106099 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2ccc06e1d0 0x7f2ccc070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:14.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.505+0000 7f2cd67fc640 1 -- 172.21.15.102:0/3510106099 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f2cd008dfe0 con 0x7f2ce0101b00 2024-05-13T23:01:14.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.505+0000 7f2ce5278640 1 --2- 172.21.15.102:0/3510106099 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2ccc06e1d0 0x7f2ccc070690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc006010 tx=0x7f2cdc005b50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:14.485 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.510+0000 7f2cd67fc640 1 -- 172.21.15.102:0/3510106099 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f2cd0057250 con 0x7f2ce0101b00 2024-05-13T23:01:14.664 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.689+0000 7f2ce74dc640 1 -- 172.21.15.102:0/3510106099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ce0197070 con 0x7f2ce0101b00 2024-05-13T23:01:14.665 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.690+0000 7f2cd67fc640 1 -- 172.21.15.102:0/3510106099 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17245 (secure 0 0 0) 0x7f2cd005a910 con 0x7f2ce0101b00 2024-05-13T23:01:14.666 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-05-13T23:01:14.666 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":96,"fsid":"62275d5c-1179-11ef-bc98-c7b262605968","created":"2024-05-13T22:38:14.140940+0000","modified":"2024-05-13T22:58:22.091658+0000","last_up_change":"2024-05-13T22:57:16.054261+0000","last_in_change":"2024-05-13T22:43:35.225543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-13T22:42:39.969524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-13T22:45:08.878601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-13T22:45:09.369354+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8d8cb2e3-52d6-45f5-94a1-3e8cc6535efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":90,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6803","nonce":307502084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6805","nonce":307502084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6809","nonce":307502084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":307502084},{"type":"v1","addr":"172.21.15.102:6807","nonce":307502084}]},"public_addr":"172.21.15.102:6803/307502084","cluster_addr":"172.21.15.102:6805/307502084","heartbeat_back_addr":"172.21.15.102:6809/307502084","heartbeat_front_addr":"172.21.15.102:6807/307502084","state":["exists","up"]},{"osd":1,"uuid":"1d3eacb1-362c-4b6a-a959-7dea19c6521e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":66,"up_thru":90,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6811","nonce":272360893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6813","nonce":272360893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6817","nonce":272360893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":272360893},{"type":"v1","addr":"172.21.15.102:6815","nonce":272360893}]},"public_addr":"172.21.15.102:6811/272360893","cluster_addr":"172.21.15.102:6813/272360893","heartbeat_back_addr":"172.21.15.102:6817/272360893","heartbeat_front_addr":"172.21.15.102:6815/272360893","state":["exists","up"]},{"osd":2,"uuid":"205b05a9-9d68-48ed-b87b-c4c677882eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6819","nonce":3530038065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6821","nonce":3530038065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6825","nonce":3530038065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3530038065},{"type":"v1","addr":"172.21.15.102:6823","nonce":3530038065}]},"public_addr":"172.21.15.102:6819/3530038065","cluster_addr":"172.21.15.102:6821/3530038065","heartbeat_back_addr":"172.21.15.102:6825/3530038065","heartbeat_front_addr":"172.21.15.102:6823/3530038065","state":["exists","up"]},{"osd":3,"uuid":"d827066c-3201-479e-b142-27721c2761fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6801","nonce":3670165065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6803","nonce":3670165065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6807","nonce":3670165065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3670165065},{"type":"v1","addr":"172.21.15.130:6805","nonce":3670165065}]},"public_addr":"172.21.15.130:6801/3670165065","cluster_addr":"172.21.15.130:6803/3670165065","heartbeat_back_addr":"172.21.15.130:6807/3670165065","heartbeat_front_addr":"172.21.15.130:6805/3670165065","state":["exists","up"]},{"osd":4,"uuid":"7d89edd4-8877-4cb8-a0af-019f9451bd39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6809","nonce":3241450581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6811","nonce":3241450581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6815","nonce":3241450581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":3241450581},{"type":"v1","addr":"172.21.15.130:6813","nonce":3241450581}]},"public_addr":"172.21.15.130:6809/3241450581","cluster_addr":"172.21.15.130:6811/3241450581","heartbeat_back_addr":"172.21.15.130:6815/3241450581","heartbeat_front_addr":"172.21.15.130:6813/3241450581","state":["exists","up"]},{"osd":5,"uuid":"c0942077-8a53-4e3b-b411-f1763c592e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6817","nonce":3573323293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6819","nonce":3573323293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6823","nonce":3573323293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3573323293},{"type":"v1","addr":"172.21.15.130:6821","nonce":3573323293}]},"public_addr":"172.21.15.130:6817/3573323293","cluster_addr":"172.21.15.130:6819/3573323293","heartbeat_back_addr":"172.21.15.130:6823/3573323293","heartbeat_front_addr":"172.21.15.130:6821/3573323293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:41:41.451627+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:09.327795+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:42:36.547203+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:00.627031+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:26.709919+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-13T22:43:52.576178+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":0,"to":1}]},{"pgid":"2.6","mappings":[{"from":3,"to":2}]},{"pgid":"2.e","mappings":[{"from":0,"to":1}]},{"pgid":"2.f","mappings":[{"from":0,"to":1}]},{"pgid":"2.15","mappings":[{"from":3,"to":2}]},{"pgid":"2.1d","mappings":[{"from":3,"to":1}]},{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":4,"to":5}]},{"pgid":"3.1b","mappings":[{"from":3,"to":1}]},{"pgid":"3.1d","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.130:6827/633282433":"2024-05-14T22:58:22.091628+0000","172.21.15.130:6826/633282433":"2024-05-14T22:58:22.091628+0000","172.21.15.102:6801/2015310067":"2024-05-14T22:52:45.995299+0000","172.21.15.130:0/1149656978":"2024-05-14T22:48:49.345907+0000","172.21.15.102:6827/3231884180":"2024-05-14T22:45:25.114218+0000","172.21.15.102:0/3978175820":"2024-05-14T22:52:45.995299+0000","172.21.15.102:6826/3231884180":"2024-05-14T22:45:25.114218+0000","172.21.15.130:0/2344917375":"2024-05-14T22:48:49.345907+0000","172.21.15.102:0/771792641":"2024-05-14T22:47:31.055945+0000","172.21.15.102:6801/2571859579":"2024-05-14T22:47:31.055945+0000","172.21.15.102:6828/2977397703":"2024-05-14T22:57:41.438543+0000","172.21.15.102:0/3812624194":"2024-05-14T22:38:41.659063+0000","172.21.15.102:6800/2571859579":"2024-05-14T22:47:31.055945+0000","172.21.15.102:0/402887427":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/2015310067":"2024-05-14T22:52:45.995299+0000","172.21.15.102:6801/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.130:6829/3751205200":"2024-05-14T22:48:49.345907+0000","172.21.15.130:6828/3751205200":"2024-05-14T22:48:49.345907+0000","172.21.15.102:6800/4002480395":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/3850189972":"2024-05-14T22:39:13.681560+0000","172.21.15.130:0/1881061148":"2024-05-14T22:48:49.345907+0000","172.21.15.102:0/2248276518":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/773571736":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/2675775894":"2024-05-14T22:47:31.055945+0000","172.21.15.102:6829/2977397703":"2024-05-14T22:57:41.438543+0000","172.21.15.102:0/457222314":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/1831496769":"2024-05-14T22:52:45.995299+0000","172.21.15.102:0/1238275992":"2024-05-14T22:47:31.055945+0000","172.21.15.130:0/3880089243":"2024-05-14T22:48:49.345907+0000","172.21.15.102:0/633075379":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/2096579260":"2024-05-14T22:38:41.659063+0000","172.21.15.102:0/233302973":"2024-05-14T22:47:31.055945+0000","172.21.15.130:0/495062226":"2024-05-14T22:48:49.345907+0000","172.21.15.130:0/1737286614":"2024-05-14T22:48:49.345907+0000","172.21.15.102:6801/263535069":"2024-05-14T22:40:23.651424+0000","172.21.15.102:0/17493483":"2024-05-14T22:40:23.651424+0000","172.21.15.102:6800/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/1416077149":"2024-05-14T22:52:45.995299+0000","172.21.15.102:6801/358342393":"2024-05-14T22:39:13.681560+0000","172.21.15.102:0/1857247111":"2024-05-14T22:52:45.995299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-13T23:01:14.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.692+0000 7f2ce74dc640 1 -- 172.21.15.102:0/3510106099 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2ccc06e1d0 msgr2=0x7f2ccc070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.692+0000 7f2ce74dc640 1 --2- 172.21.15.102:0/3510106099 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2ccc06e1d0 0x7f2ccc070690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc006010 tx=0x7f2cdc005b50 comp rx=0 tx=0).stop 2024-05-13T23:01:14.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.692+0000 7f2ce74dc640 1 -- 172.21.15.102:0/3510106099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0101b00 msgr2=0x7f2ce019c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.692+0000 7f2ce74dc640 1 --2- 172.21.15.102:0/3510106099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0101b00 0x7f2ce019c010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000c8f0 tx=0x7f2cd000cdb0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.692+0000 7f2ce74dc640 1 -- 172.21.15.102:0/3510106099 shutdown_connections 2024-05-13T23:01:14.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.692+0000 7f2ce74dc640 1 --2- 172.21.15.102:0/3510106099 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f2ccc06e1d0 0x7f2ccc070690 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.692+0000 7f2ce74dc640 1 --2- 172.21.15.102:0/3510106099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0101b00 0x7f2ce019c010 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.692+0000 7f2ce74dc640 1 --2- 172.21.15.102:0/3510106099 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ce01011e0 0x7f2ce019bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.692+0000 7f2ce74dc640 1 -- 172.21.15.102:0/3510106099 >> 172.21.15.102:0/3510106099 conn(0x7f2ce00faa20 msgr2=0x7f2ce0104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:14.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.693+0000 7f2ce74dc640 1 -- 172.21.15.102:0/3510106099 shutdown_connections 2024-05-13T23:01:14.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.693+0000 7f2ce74dc640 1 -- 172.21.15.102:0/3510106099 wait complete. 2024-05-13T23:01:14.679 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-13T22:42:39.969524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-13T22:45:08.878601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-13T22:45:09.369354+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-13T23:01:14.679 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-13T23:01:14.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.807+0000 7f7960b1d640 1 -- 172.21.15.102:0/3862992830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f795c0ff260 msgr2=0x7f795c0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.807+0000 7f7960b1d640 1 --2- 172.21.15.102:0/3862992830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f795c0ff260 0x7f795c0ff640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f794c009880 tx=0x7f794c02bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.807+0000 7f7960b1d640 1 -- 172.21.15.102:0/3862992830 shutdown_connections 2024-05-13T23:01:14.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.807+0000 7f7960b1d640 1 --2- 172.21.15.102:0/3862992830 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c0ffc10 0x7f795c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.807+0000 7f7960b1d640 1 --2- 172.21.15.102:0/3862992830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f795c0ff260 0x7f795c0ff640 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.807+0000 7f7960b1d640 1 -- 172.21.15.102:0/3862992830 >> 172.21.15.102:0/3862992830 conn(0x7f795c074a50 msgr2=0x7f795c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:14.785 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.807+0000 7f7960b1d640 1 -- 172.21.15.102:0/3862992830 shutdown_connections 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.807+0000 7f7960b1d640 1 -- 172.21.15.102:0/3862992830 wait complete. 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7960b1d640 1 Processor -- start 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7960b1d640 1 -- start start 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7960b1d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c0ff260 0x7f795c198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7960b1d640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f795c198680 0x7f795c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7960b1d640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c198c80 con 0x7f795c198680 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7960b1d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c198df0 con 0x7f795c0ff260 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7959d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f795c198680 0x7f795c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7959d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f795c198680 0x7f795c19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47704/0 (socket says 172.21.15.102:47704) 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7959d9b640 1 -- 172.21.15.102:0/1369224142 learned_addr learned my addr 172.21.15.102:0/1369224142 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f795a59c640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c0ff260 0x7f795c198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7959d9b640 1 -- 172.21.15.102:0/1369224142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c0ff260 msgr2=0x7f795c198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.786 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7959d9b640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c0ff260 0x7f795c198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f7959d9b640 1 -- 172.21.15.102:0/1369224142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c009560 con 0x7f795c198680 2024-05-13T23:01:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.808+0000 7f795a59c640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c0ff260 0x7f795c198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T23:01:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.809+0000 7f7959d9b640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f795c198680 0x7f795c19cae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f795000b400 tx=0x7f795000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.809+0000 7f794b7fe640 1 -- 172.21.15.102:0/1369224142 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7950011800 con 0x7f795c198680 2024-05-13T23:01:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.809+0000 7f7960b1d640 1 -- 172.21.15.102:0/1369224142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795c19d0e0 con 0x7f795c198680 2024-05-13T23:01:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.809+0000 7f794b7fe640 1 -- 172.21.15.102:0/1369224142 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7950011e20 con 0x7f795c198680 2024-05-13T23:01:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.809+0000 7f794b7fe640 1 -- 172.21.15.102:0/1369224142 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f795000f2e0 con 0x7f795c198680 2024-05-13T23:01:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.809+0000 7f7960b1d640 1 -- 172.21.15.102:0/1369224142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795c19d640 con 0x7f795c198680 2024-05-13T23:01:14.787 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.810+0000 7f7960b1d640 1 -- 172.21.15.102:0/1369224142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795c04e040 con 0x7f795c198680 2024-05-13T23:01:14.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.810+0000 7f794b7fe640 1 -- 172.21.15.102:0/1369224142 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7950010380 con 0x7f795c198680 2024-05-13T23:01:14.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.811+0000 7f794b7fe640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f794406e170 0x7f7944070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:14.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.811+0000 7f794b7fe640 1 -- 172.21.15.102:0/1369224142 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f795008dfe0 con 0x7f795c198680 2024-05-13T23:01:14.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.811+0000 7f795a59c640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f794406e170 0x7f7944070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:14.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.811+0000 7f795a59c640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f794406e170 0x7f7944070630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f794c000c00 tx=0x7f794c02cc20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.816+0000 7f794b7fe640 1 -- 172.21.15.102:0/1369224142 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f79500571a0 con 0x7f795c198680 2024-05-13T23:01:14.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:14 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1637163914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-13T23:01:14.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:14 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1480943079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-13T23:01:14.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:14 smithi130 ceph-mon[84265]: pgmap v303: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:14.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:14 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3169166583' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-13T23:01:14.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:14 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1637163914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-13T23:01:14.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:14 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1480943079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-13T23:01:14.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:14 smithi102 ceph-mon[111694]: pgmap v303: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:14.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:14 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3169166583' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-13T23:01:14.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.991+0000 7f7960b1d640 1 -- 172.21.15.102:0/1369224142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f795c0ffc10 con 0x7f795c198680 2024-05-13T23:01:14.967 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.992+0000 7f794b7fe640 1 -- 172.21.15.102:0/1369224142 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f795005a860 con 0x7f795c198680 2024-05-13T23:01:14.967 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-05-13T23:01:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.994+0000 7f7960b1d640 1 -- 172.21.15.102:0/1369224142 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f794406e170 msgr2=0x7f7944070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.994+0000 7f7960b1d640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f794406e170 0x7f7944070630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f794c000c00 tx=0x7f794c02cc20 comp rx=0 tx=0).stop 2024-05-13T23:01:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.994+0000 7f7960b1d640 1 -- 172.21.15.102:0/1369224142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f795c198680 msgr2=0x7f795c19cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:14.969 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.994+0000 7f7960b1d640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f795c198680 0x7f795c19cae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f795000b400 tx=0x7f795000b8c0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.994+0000 7f7960b1d640 1 -- 172.21.15.102:0/1369224142 shutdown_connections 2024-05-13T23:01:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.994+0000 7f7960b1d640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f794406e170 0x7f7944070630 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.994+0000 7f7960b1d640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f795c198680 0x7f795c19cae0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.994+0000 7f7960b1d640 1 --2- 172.21.15.102:0/1369224142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f795c0ff260 0x7f795c198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.994+0000 7f7960b1d640 1 -- 172.21.15.102:0/1369224142 >> 172.21.15.102:0/1369224142 conn(0x7f795c074a50 msgr2=0x7f795c067c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.994+0000 7f7960b1d640 1 -- 172.21.15.102:0/1369224142 shutdown_connections 2024-05-13T23:01:14.970 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:14.994+0000 7f7960b1d640 1 -- 172.21.15.102:0/1369224142 wait complete. 2024-05-13T23:01:14.980 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-13T23:01:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 -- 172.21.15.102:0/399624629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7264100c50 msgr2=0x7f7264104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 --2- 172.21.15.102:0/399624629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7264100c50 0x7f7264104c90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7254009880 tx=0x7f725402bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 -- 172.21.15.102:0/399624629 shutdown_connections 2024-05-13T23:01:15.087 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 --2- 172.21.15.102:0/399624629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7264100c50 0x7f7264104c90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 --2- 172.21.15.102:0/399624629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72641002a0 0x7f7264100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 -- 172.21.15.102:0/399624629 >> 172.21.15.102:0/399624629 conn(0x7f72640fbb20 msgr2=0x7f72640fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:15.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 -- 172.21.15.102:0/399624629 shutdown_connections 2024-05-13T23:01:15.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 -- 172.21.15.102:0/399624629 wait complete. 2024-05-13T23:01:15.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 Processor -- start 2024-05-13T23:01:15.089 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 -- start start 2024-05-13T23:01:15.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72641002a0 0x7f72640725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:15.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7264072ae0 0x7f7264072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:15.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72641a1f20 con 0x7f7264072ae0 2024-05-13T23:01:15.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.108+0000 7f726b4c4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72641a2060 con 0x7f72641002a0 2024-05-13T23:01:15.090 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f7268a5f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7264072ae0 0x7f7264072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:15.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f7268a5f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7264072ae0 0x7f7264072f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47728/0 (socket says 172.21.15.102:47728) 2024-05-13T23:01:15.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f7268a5f640 1 -- 172.21.15.102:0/2188250804 learned_addr learned my addr 172.21.15.102:0/2188250804 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:15.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f7269260640 1 --2- 172.21.15.102:0/2188250804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72641002a0 0x7f72640725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:15.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f7268a5f640 1 -- 172.21.15.102:0/2188250804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72641002a0 msgr2=0x7f72640725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f7268a5f640 1 --2- 172.21.15.102:0/2188250804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72641002a0 0x7f72640725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f7268a5f640 1 -- 172.21.15.102:0/2188250804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254009560 con 0x7f7264072ae0 2024-05-13T23:01:15.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f7268a5f640 1 --2- 172.21.15.102:0/2188250804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7264072ae0 0x7f7264072f40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7254036040 tx=0x7f725402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:15.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f725a7fc640 1 -- 172.21.15.102:0/2188250804 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72540035f0 con 0x7f7264072ae0 2024-05-13T23:01:15.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f725a7fc640 1 -- 172.21.15.102:0/2188250804 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7254003c10 con 0x7f7264072ae0 2024-05-13T23:01:15.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f726b4c4640 1 -- 172.21.15.102:0/2188250804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72641a21a0 con 0x7f7264072ae0 2024-05-13T23:01:15.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f725a7fc640 1 -- 172.21.15.102:0/2188250804 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7254034440 con 0x7f7264072ae0 2024-05-13T23:01:15.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.109+0000 7f726b4c4640 1 -- 172.21.15.102:0/2188250804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72641a2650 con 0x7f7264072ae0 2024-05-13T23:01:15.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.110+0000 7f726b4c4640 1 -- 172.21.15.102:0/2188250804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726404e040 con 0x7f7264072ae0 2024-05-13T23:01:15.092 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.111+0000 7f725a7fc640 1 -- 172.21.15.102:0/2188250804 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7254003750 con 0x7f7264072ae0 2024-05-13T23:01:15.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.111+0000 7f725a7fc640 1 --2- 172.21.15.102:0/2188250804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f725006e170 0x7f7250070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:15.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.111+0000 7f7269260640 1 --2- 172.21.15.102:0/2188250804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f725006e170 0x7f7250070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:15.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.111+0000 7f725a7fc640 1 -- 172.21.15.102:0/2188250804 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f72540b06b0 con 0x7f7264072ae0 2024-05-13T23:01:15.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.112+0000 7f7269260640 1 --2- 172.21.15.102:0/2188250804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f725006e170 0x7f7250070630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7260005fd0 tx=0x7f7260005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:15.093 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.116+0000 7f725a7fc640 1 -- 172.21.15.102:0/2188250804 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f7254079870 con 0x7f7264072ae0 2024-05-13T23:01:15.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.291+0000 7f726b4c4640 1 -- 172.21.15.102:0/2188250804 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f72641085a0 con 0x7f7264072ae0 2024-05-13T23:01:15.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.291+0000 7f725a7fc640 1 -- 172.21.15.102:0/2188250804 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7f725407cf30 con 0x7f7264072ae0 2024-05-13T23:01:15.268 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-05-13T23:01:15.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.293+0000 7f726b4c4640 1 -- 172.21.15.102:0/2188250804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f725006e170 msgr2=0x7f7250070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.293+0000 7f726b4c4640 1 --2- 172.21.15.102:0/2188250804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f725006e170 0x7f7250070630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7260005fd0 tx=0x7f7260005850 comp rx=0 tx=0).stop 2024-05-13T23:01:15.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.293+0000 7f726b4c4640 1 -- 172.21.15.102:0/2188250804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7264072ae0 msgr2=0x7f7264072f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.293+0000 7f726b4c4640 1 --2- 172.21.15.102:0/2188250804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7264072ae0 0x7f7264072f40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7254036040 tx=0x7f725402c850 comp rx=0 tx=0).stop 2024-05-13T23:01:15.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.293+0000 7f726b4c4640 1 -- 172.21.15.102:0/2188250804 shutdown_connections 2024-05-13T23:01:15.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.293+0000 7f726b4c4640 1 --2- 172.21.15.102:0/2188250804 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f725006e170 0x7f7250070630 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.293+0000 7f726b4c4640 1 --2- 172.21.15.102:0/2188250804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7264072ae0 0x7f7264072f40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.293+0000 7f726b4c4640 1 --2- 172.21.15.102:0/2188250804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72641002a0 0x7f72640725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.293+0000 7f726b4c4640 1 -- 172.21.15.102:0/2188250804 >> 172.21.15.102:0/2188250804 conn(0x7f72640fbb20 msgr2=0x7f72640fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:15.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.293+0000 7f726b4c4640 1 -- 172.21.15.102:0/2188250804 shutdown_connections 2024-05-13T23:01:15.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.293+0000 7f726b4c4640 1 -- 172.21.15.102:0/2188250804 wait complete. 2024-05-13T23:01:15.280 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-13T23:01:15.384 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.409+0000 7fdeef469640 1 -- 172.21.15.102:0/2937325449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdee81002a0 msgr2=0x7fdee8100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.409+0000 7fdeef469640 1 --2- 172.21.15.102:0/2937325449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdee81002a0 0x7fdee8100680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009c20 tx=0x7fdee402c1b0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.409+0000 7fdeef469640 1 -- 172.21.15.102:0/2937325449 shutdown_connections 2024-05-13T23:01:15.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.409+0000 7fdeef469640 1 --2- 172.21.15.102:0/2937325449 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdee8100c50 0x7fdee8104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.409+0000 7fdeef469640 1 --2- 172.21.15.102:0/2937325449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdee81002a0 0x7fdee8100680 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.409+0000 7fdeef469640 1 -- 172.21.15.102:0/2937325449 >> 172.21.15.102:0/2937325449 conn(0x7fdee80fbb20 msgr2=0x7fdee80fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:15.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.409+0000 7fdeef469640 1 -- 172.21.15.102:0/2937325449 shutdown_connections 2024-05-13T23:01:15.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.409+0000 7fdeef469640 1 -- 172.21.15.102:0/2937325449 wait complete. 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeef469640 1 Processor -- start 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeef469640 1 -- start start 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeef469640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdee81002a0 0x7fdee8195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeef469640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdee8196340 0x7fdee819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeef469640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8196940 con 0x7fdee8196340 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeef469640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8196ab0 con 0x7fdee81002a0 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeeca04640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdee8196340 0x7fdee819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeeca04640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdee8196340 0x7fdee819a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47750/0 (socket says 172.21.15.102:47750) 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeeca04640 1 -- 172.21.15.102:0/2134767080 learned_addr learned my addr 172.21.15.102:0/2134767080 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeed205640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdee81002a0 0x7fdee8195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeeca04640 1 -- 172.21.15.102:0/2134767080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdee81002a0 msgr2=0x7fdee8195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeeca04640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdee81002a0 0x7fdee8195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeeca04640 1 -- 172.21.15.102:0/2134767080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee4009900 con 0x7fdee8196340 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeed205640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdee81002a0 0x7fdee8195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.410+0000 7fdeeca04640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdee8196340 0x7fdee819a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fded800c920 tx=0x7fded800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.411+0000 7fdede7fc640 1 -- 172.21.15.102:0/2134767080 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fded8007b80 con 0x7fdee8196340 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.411+0000 7fdeef469640 1 -- 172.21.15.102:0/2134767080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee819ada0 con 0x7fdee8196340 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.411+0000 7fdede7fc640 1 -- 172.21.15.102:0/2134767080 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fded8007ce0 con 0x7fdee8196340 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.411+0000 7fdede7fc640 1 -- 172.21.15.102:0/2134767080 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fded80187d0 con 0x7fdee8196340 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.411+0000 7fdeef469640 1 -- 172.21.15.102:0/2134767080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee819b300 con 0x7fdee8196340 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.412+0000 7fdeef469640 1 -- 172.21.15.102:0/2134767080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee819b5e0 con 0x7fdee8196340 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.412+0000 7fdede7fc640 1 -- 172.21.15.102:0/2134767080 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fded800f450 con 0x7fdee8196340 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.413+0000 7fdede7fc640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fded406e170 0x7fded4070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.413+0000 7fdede7fc640 1 -- 172.21.15.102:0/2134767080 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fded808e480 con 0x7fdee8196340 2024-05-13T23:01:15.389 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.413+0000 7fdeed205640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fded406e170 0x7fded4070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:15.390 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.413+0000 7fdeed205640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fded406e170 0x7fded4070630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009430 tx=0x7fdee4009360 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:15.394 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.417+0000 7fdede7fc640 1 -- 172.21.15.102:0/2134767080 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7fded8057640 con 0x7fdee8196340 2024-05-13T23:01:15.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.591+0000 7fdeef469640 1 -- 172.21.15.102:0/2134767080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fdee804e040 con 0x7fdee8196340 2024-05-13T23:01:15.567 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.591+0000 7fdede7fc640 1 -- 172.21.15.102:0/2134767080 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v96) v1 ==== 107+0+11 (secure 0 0 0) 0x7fded805ad00 con 0x7fdee8196340 2024-05-13T23:01:15.567 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 32 2024-05-13T23:01:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.593+0000 7fdeef469640 1 -- 172.21.15.102:0/2134767080 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fded406e170 msgr2=0x7fded4070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.593+0000 7fdeef469640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fded406e170 0x7fded4070630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009430 tx=0x7fdee4009360 comp rx=0 tx=0).stop 2024-05-13T23:01:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.593+0000 7fdeef469640 1 -- 172.21.15.102:0/2134767080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdee8196340 msgr2=0x7fdee819a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.593+0000 7fdeef469640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdee8196340 0x7fdee819a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fded800c920 tx=0x7fded800cde0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.593+0000 7fdeef469640 1 -- 172.21.15.102:0/2134767080 shutdown_connections 2024-05-13T23:01:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.593+0000 7fdeef469640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7fded406e170 0x7fded4070630 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.593+0000 7fdeef469640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdee8196340 0x7fdee819a7a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.593+0000 7fdeef469640 1 --2- 172.21.15.102:0/2134767080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdee81002a0 0x7fdee8195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.569 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.593+0000 7fdeef469640 1 -- 172.21.15.102:0/2134767080 >> 172.21.15.102:0/2134767080 conn(0x7fdee80fbb20 msgr2=0x7fdee80fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:15.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.594+0000 7fdeef469640 1 -- 172.21.15.102:0/2134767080 shutdown_connections 2024-05-13T23:01:15.570 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.594+0000 7fdeef469640 1 -- 172.21.15.102:0/2134767080 wait complete. 2024-05-13T23:01:15.580 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-13T23:01:15.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 -- 172.21.15.102:0/1635502402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864067350 msgr2=0x7f08640677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 --2- 172.21.15.102:0/1635502402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864067350 0x7f08640677b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0854009880 tx=0x7f085402bce0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 -- 172.21.15.102:0/1635502402 shutdown_connections 2024-05-13T23:01:15.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 --2- 172.21.15.102:0/1635502402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864067350 0x7f08640677b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 --2- 172.21.15.102:0/1635502402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08641061b0 0x7f0864106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 -- 172.21.15.102:0/1635502402 >> 172.21.15.102:0/1635502402 conn(0x7f0864075b40 msgr2=0x7f0864075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:15.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 -- 172.21.15.102:0/1635502402 shutdown_connections 2024-05-13T23:01:15.685 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 -- 172.21.15.102:0/1635502402 wait complete. 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 Processor -- start 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 -- start start 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864067350 0x7f0864195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08641061b0 0x7f08641961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641968d0 con 0x7f0864067350 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.708+0000 7f086ad0c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641995a0 con 0x7f08641061b0 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0868aa8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864067350 0x7f0864195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0868aa8640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864067350 0x7f0864195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47778/0 (socket says 172.21.15.102:47778) 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0868aa8640 1 -- 172.21.15.102:0/1890491753 learned_addr learned my addr 172.21.15.102:0/1890491753 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0863fff640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08641061b0 0x7f08641961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0868aa8640 1 -- 172.21.15.102:0/1890491753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08641061b0 msgr2=0x7f08641961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0868aa8640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08641061b0 0x7f08641961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0868aa8640 1 -- 172.21.15.102:0/1890491753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0854009560 con 0x7f0864067350 2024-05-13T23:01:15.686 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0863fff640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08641061b0 0x7f08641961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-13T23:01:15.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0868aa8640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864067350 0x7f0864195cb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f084c00cc60 tx=0x7f084c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:15.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0861ffb640 1 -- 172.21.15.102:0/1890491753 <== mon.0 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084c004b20 con 0x7f0864067350 2024-05-13T23:01:15.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0861ffb640 1 -- 172.21.15.102:0/1890491753 <== mon.0 v2:172.21.15.102:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f084c004c80 con 0x7f0864067350 2024-05-13T23:01:15.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f086ad0c640 1 -- 172.21.15.102:0/1890491753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0864199880 con 0x7f0864067350 2024-05-13T23:01:15.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f0861ffb640 1 -- 172.21.15.102:0/1890491753 <== mon.0 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f084c00f640 con 0x7f0864067350 2024-05-13T23:01:15.687 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.709+0000 7f086ad0c640 1 -- 172.21.15.102:0/1890491753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0864199db0 con 0x7f0864067350 2024-05-13T23:01:15.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.710+0000 7f086ad0c640 1 -- 172.21.15.102:0/1890491753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086404e040 con 0x7f0864067350 2024-05-13T23:01:15.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.711+0000 7f0861ffb640 1 -- 172.21.15.102:0/1890491753 <== mon.0 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 38) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f084c0055e0 con 0x7f0864067350 2024-05-13T23:01:15.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.711+0000 7f0861ffb640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f085006e230 0x7f08500706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-13T23:01:15.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.711+0000 7f0861ffb640 1 -- 172.21.15.102:0/1890491753 <== mon.0 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f084c08e420 con 0x7f0864067350 2024-05-13T23:01:15.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.711+0000 7f0863fff640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f085006e230 0x7f08500706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-13T23:01:15.688 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.712+0000 7f0863fff640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f085006e230 0x7f08500706f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0864197320 tx=0x7f085402cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-13T23:01:15.691 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.716+0000 7f0861ffb640 1 -- 172.21.15.102:0/1890491753 <== mon.0 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186418 (secure 0 0 0) 0x7f084c057660 con 0x7f0864067350 2024-05-13T23:01:15.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.906+0000 7f086ad0c640 1 -- 172.21.15.102:0/1890491753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0864103390 con 0x7f0864067350 2024-05-13T23:01:15.881 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.906+0000 7f0861ffb640 1 -- 172.21.15.102:0/1890491753 <== mon.0 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v96) v1 ==== 81+0+2458 (secure 0 0 0) 0x7f084c01d9b0 con 0x7f0864067350 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:6827/633282433 2024-05-14T22:58:22.091628+0000 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:6826/633282433 2024-05-14T22:58:22.091628+0000 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/2015310067 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/1149656978 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6827/3231884180 2024-05-14T22:45:25.114218+0000 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/3978175820 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6826/3231884180 2024-05-14T22:45:25.114218+0000 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/2344917375 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/771792641 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/2571859579 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6828/2977397703 2024-05-14T22:57:41.438543+0000 2024-05-13T23:01:15.882 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/3812624194 2024-05-14T22:38:41.659063+0000 2024-05-13T23:01:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/2571859579 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/402887427 2024-05-14T22:40:23.651424+0000 2024-05-13T23:01:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/263535069 2024-05-14T22:40:23.651424+0000 2024-05-13T23:01:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/2015310067 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/4002480395 2024-05-14T22:38:41.659063+0000 2024-05-13T23:01:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:6829/3751205200 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:6828/3751205200 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/4002480395 2024-05-14T22:38:41.659063+0000 2024-05-13T23:01:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/3850189972 2024-05-14T22:39:13.681560+0000 2024-05-13T23:01:15.883 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/1881061148 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/2248276518 2024-05-14T22:39:13.681560+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/773571736 2024-05-14T22:39:13.681560+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/2675775894 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6829/2977397703 2024-05-14T22:57:41.438543+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/457222314 2024-05-14T22:40:23.651424+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1831496769 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1238275992 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/3880089243 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/633075379 2024-05-14T22:38:41.659063+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/2096579260 2024-05-14T22:38:41.659063+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/233302973 2024-05-14T22:47:31.055945+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/495062226 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.130:0/1737286614 2024-05-14T22:48:49.345907+0000 2024-05-13T23:01:15.884 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/263535069 2024-05-14T22:40:23.651424+0000 2024-05-13T23:01:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/17493483 2024-05-14T22:40:23.651424+0000 2024-05-13T23:01:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6800/358342393 2024-05-14T22:39:13.681560+0000 2024-05-13T23:01:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1416077149 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:6801/358342393 2024-05-14T22:39:13.681560+0000 2024-05-13T23:01:15.885 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102:0/1857247111 2024-05-14T22:52:45.995299+0000 2024-05-13T23:01:15.886 INFO:teuthology.orchestra.run.smithi102.stderr:listed 41 entries 2024-05-13T23:01:15.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.908+0000 7f086ad0c640 1 -- 172.21.15.102:0/1890491753 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f085006e230 msgr2=0x7f08500706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.908+0000 7f086ad0c640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f085006e230 0x7f08500706f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0864197320 tx=0x7f085402cf90 comp rx=0 tx=0).stop 2024-05-13T23:01:15.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.908+0000 7f086ad0c640 1 -- 172.21.15.102:0/1890491753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864067350 msgr2=0x7f0864195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-13T23:01:15.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.908+0000 7f086ad0c640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864067350 0x7f0864195cb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f084c00cc60 tx=0x7f084c00a620 comp rx=0 tx=0).stop 2024-05-13T23:01:15.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.908+0000 7f086ad0c640 1 -- 172.21.15.102:0/1890491753 shutdown_connections 2024-05-13T23:01:15.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.908+0000 7f086ad0c640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.102:6800/2973074092,v1:172.21.15.102:6801/2973074092] conn(0x7f085006e230 0x7f08500706f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.908+0000 7f086ad0c640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f08641061b0 0x7f08641961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.908+0000 7f086ad0c640 1 --2- 172.21.15.102:0/1890491753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0864067350 0x7f0864195cb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-13T23:01:15.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.908+0000 7f086ad0c640 1 -- 172.21.15.102:0/1890491753 >> 172.21.15.102:0/1890491753 conn(0x7f0864075b40 msgr2=0x7f08640fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-13T23:01:15.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.908+0000 7f086ad0c640 1 -- 172.21.15.102:0/1890491753 shutdown_connections 2024-05-13T23:01:15.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-05-13T23:01:15.908+0000 7f086ad0c640 1 -- 172.21.15.102:0/1890491753 wait complete. 2024-05-13T23:01:15.893 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:15 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/3510106099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T23:01:15.894 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:15 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1369224142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-13T23:01:15.894 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:15 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2188250804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-13T23:01:15.894 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:15 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/2134767080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-13T23:01:15.895 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi102.front.sepia.ceph.com... 2024-05-13T23:01:15.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-13T23:01:15.895 DEBUG:teuthology.orchestra.run.smithi102:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-13T23:01:15.897 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:15 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/3510106099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-13T23:01:15.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:15 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1369224142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-13T23:01:15.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:15 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2188250804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-13T23:01:15.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:15 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/2134767080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-13T23:01:15.920 INFO:teuthology.orchestra.run:waiting for 300 2024-05-13T23:01:16.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:16 smithi130 ceph-mon[84265]: from='client.? 172.21.15.102:0/1890491753' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-13T23:01:16.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:01:16.898 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:16 smithi130 ceph-mon[84265]: pgmap v304: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:16.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:16 smithi102 ceph-mon[111694]: from='client.? 172.21.15.102:0/1890491753' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-13T23:01:16.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:01:16.925 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:16 smithi102 ceph-mon[111694]: pgmap v304: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:19.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:19 smithi102 ceph-mon[111694]: pgmap v305: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:19.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:19 smithi130 ceph-mon[84265]: pgmap v305: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:21.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:21 smithi102 ceph-mon[111694]: pgmap v306: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:21.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:21 smithi130 ceph-mon[84265]: pgmap v306: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:23.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:23 smithi102 ceph-mon[111694]: pgmap v307: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:23.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:23 smithi130 ceph-mon[84265]: pgmap v307: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:25.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:25 smithi102 ceph-mon[111694]: pgmap v308: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:25.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:25 smithi130 ceph-mon[84265]: pgmap v308: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:27.424 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:27 smithi102 ceph-mon[111694]: pgmap v309: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:27.549 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:27 smithi130 ceph-mon[84265]: pgmap v309: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:29.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:29 smithi102 ceph-mon[111694]: pgmap v310: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:29.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:29 smithi130 ceph-mon[84265]: pgmap v310: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:31.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:31 smithi102 ceph-mon[111694]: pgmap v311: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:31.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:01:31.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:31 smithi130 ceph-mon[84265]: pgmap v311: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:31.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:01:33.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:33 smithi130 ceph-mon[84265]: pgmap v312: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:33.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:33 smithi102 ceph-mon[111694]: pgmap v312: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:35.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:35 smithi130 ceph-mon[84265]: pgmap v313: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:35.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:35 smithi102 ceph-mon[111694]: pgmap v313: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:37.576 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:37 smithi102 ceph-mon[111694]: pgmap v314: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:37.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:37 smithi130 ceph-mon[84265]: pgmap v314: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:39.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:39 smithi130 ceph-mon[84265]: pgmap v315: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:39.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:39 smithi102 ceph-mon[111694]: pgmap v315: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:41.477 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:41 smithi130 ceph-mon[84265]: pgmap v316: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:41.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:41 smithi102 ceph-mon[111694]: pgmap v316: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:43.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:43 smithi130 ceph-mon[84265]: pgmap v317: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:43.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:43 smithi102 ceph-mon[111694]: pgmap v317: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:45.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:45 smithi130 ceph-mon[84265]: pgmap v318: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:45.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:45 smithi102 ceph-mon[111694]: pgmap v318: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:46.526 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:01:46.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:01:47.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:47 smithi130 ceph-mon[84265]: pgmap v319: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:47.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:47 smithi102 ceph-mon[111694]: pgmap v319: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:49.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:49 smithi130 ceph-mon[84265]: pgmap v320: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:49.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:49 smithi102 ceph-mon[111694]: pgmap v320: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:51.574 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:51 smithi130 ceph-mon[84265]: pgmap v321: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:51.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:51 smithi102 ceph-mon[111694]: pgmap v321: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:01:53.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:53 smithi130 ceph-mon[84265]: pgmap v322: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:53.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:53 smithi102 ceph-mon[111694]: pgmap v322: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:55.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:55 smithi130 ceph-mon[84265]: pgmap v323: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:55.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:55 smithi102 ceph-mon[111694]: pgmap v323: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:57.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:57 smithi130 ceph-mon[84265]: pgmap v324: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:57.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:57 smithi102 ceph-mon[111694]: pgmap v324: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:01:59.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:01:59 smithi130 ceph-mon[84265]: pgmap v325: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:01:59.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:01:59 smithi102 ceph-mon[111694]: pgmap v325: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:01.497 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:01 smithi102 ceph-mon[111694]: pgmap v326: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:01.497 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:02:01.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:01 smithi130 ceph-mon[84265]: pgmap v326: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:01.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:02:03.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:03 smithi130 ceph-mon[84265]: pgmap v327: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:03.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:03 smithi102 ceph-mon[111694]: pgmap v327: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:05.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:05 smithi130 ceph-mon[84265]: pgmap v328: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:05.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:05 smithi102 ceph-mon[111694]: pgmap v328: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:07.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:07 smithi130 ceph-mon[84265]: pgmap v329: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:07.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:07 smithi102 ceph-mon[111694]: pgmap v329: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:09.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:09 smithi130 ceph-mon[84265]: pgmap v330: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:09.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:09 smithi102 ceph-mon[111694]: pgmap v330: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:10.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:10 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:02:10.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:10 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:02:10.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:10 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:02:10.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:10 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:02:10.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:10 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:02:10.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:10 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:02:10.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:10 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:02:10.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:10 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:02:11.608 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:11 smithi102 ceph-mon[111694]: pgmap v331: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:11.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:11 smithi130 ceph-mon[84265]: pgmap v331: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:13.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:13 smithi130 ceph-mon[84265]: pgmap v332: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:13.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:13 smithi102 ceph-mon[111694]: pgmap v332: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:15.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:15 smithi130 ceph-mon[84265]: pgmap v333: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:15.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:15 smithi102 ceph-mon[111694]: pgmap v333: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:16.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:02:16.664 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:02:17.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:17 smithi130 ceph-mon[84265]: pgmap v334: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:17.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:17 smithi102 ceph-mon[111694]: pgmap v334: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:19.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:19 smithi130 ceph-mon[84265]: pgmap v335: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:19.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:19 smithi102 ceph-mon[111694]: pgmap v335: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:21.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:21 smithi130 ceph-mon[84265]: pgmap v336: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:21.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:21 smithi102 ceph-mon[111694]: pgmap v336: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:22.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:22 smithi130 ceph-mon[84265]: pgmap v337: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:22.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:22 smithi102 ceph-mon[111694]: pgmap v337: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:25.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:25 smithi102 ceph-mon[111694]: pgmap v338: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:25.647 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:25 smithi130 ceph-mon[84265]: pgmap v338: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:27.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:27 smithi130 ceph-mon[84265]: pgmap v339: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:27.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:27 smithi102 ceph-mon[111694]: pgmap v339: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:29.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:29 smithi130 ceph-mon[84265]: pgmap v340: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:29.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:29 smithi102 ceph-mon[111694]: pgmap v340: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:31.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:31 smithi130 ceph-mon[84265]: pgmap v341: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:31.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:02:31.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:31 smithi102 ceph-mon[111694]: pgmap v341: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:31.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:02:33.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:33 smithi130 ceph-mon[84265]: pgmap v342: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:33.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:33 smithi102 ceph-mon[111694]: pgmap v342: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:35.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:35 smithi130 ceph-mon[84265]: pgmap v343: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:35.674 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:35 smithi102 ceph-mon[111694]: pgmap v343: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:37.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:37 smithi130 ceph-mon[84265]: pgmap v344: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:37.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:37 smithi102 ceph-mon[111694]: pgmap v344: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:39.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:39 smithi130 ceph-mon[84265]: pgmap v345: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:39.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:39 smithi102 ceph-mon[111694]: pgmap v345: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:41.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:41 smithi130 ceph-mon[84265]: pgmap v346: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:41.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:41 smithi102 ceph-mon[111694]: pgmap v346: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:43.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:43 smithi130 ceph-mon[84265]: pgmap v347: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:43.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:43 smithi102 ceph-mon[111694]: pgmap v347: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:45.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:45 smithi130 ceph-mon[84265]: pgmap v348: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:45.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:45 smithi102 ceph-mon[111694]: pgmap v348: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:46.648 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:02:46.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:02:47.398 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:47 smithi130 ceph-mon[84265]: pgmap v349: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:47.425 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:47 smithi102 ceph-mon[111694]: pgmap v349: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:49.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:49 smithi130 ceph-mon[84265]: pgmap v350: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:49.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:49 smithi102 ceph-mon[111694]: pgmap v350: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:51.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:51 smithi130 ceph-mon[84265]: pgmap v351: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:51.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:51 smithi102 ceph-mon[111694]: pgmap v351: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:02:53.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:53 smithi130 ceph-mon[84265]: pgmap v352: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:53.675 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:53 smithi102 ceph-mon[111694]: pgmap v352: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:55.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:55 smithi130 ceph-mon[84265]: pgmap v353: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:55.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:55 smithi102 ceph-mon[111694]: pgmap v353: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:57.426 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:57 smithi102 ceph-mon[111694]: pgmap v354: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:57.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:57 smithi130 ceph-mon[84265]: pgmap v354: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:02:59.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:02:59 smithi130 ceph-mon[84265]: pgmap v355: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:02:59.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:02:59 smithi102 ceph-mon[111694]: pgmap v355: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:01.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:01 smithi130 ceph-mon[84265]: pgmap v356: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:01.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:03:01.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:01 smithi102 ceph-mon[111694]: pgmap v356: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:01.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:03:03.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:03 smithi130 ceph-mon[84265]: pgmap v357: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:03.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:03 smithi102 ceph-mon[111694]: pgmap v357: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:05.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:05 smithi130 ceph-mon[84265]: pgmap v358: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:05.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:05 smithi102 ceph-mon[111694]: pgmap v358: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:07.479 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:07 smithi102 ceph-mon[111694]: pgmap v359: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:07.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:07 smithi130 ceph-mon[84265]: pgmap v359: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:09.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:09 smithi130 ceph-mon[84265]: pgmap v360: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:09.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:09 smithi102 ceph-mon[111694]: pgmap v360: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:10.649 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:10 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:03:10.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:10 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:03:11.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:11 smithi130 ceph-mon[84265]: pgmap v361: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:11.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:11 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:03:11.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:11 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:03:11.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:11 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:03:11.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:11 smithi102 ceph-mon[111694]: pgmap v361: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:11.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:11 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:03:11.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:11 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:03:11.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:11 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:03:13.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:13 smithi130 ceph-mon[84265]: pgmap v362: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:13.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:13 smithi102 ceph-mon[111694]: pgmap v362: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:15.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:15 smithi130 ceph-mon[84265]: pgmap v363: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:15.676 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:15 smithi102 ceph-mon[111694]: pgmap v363: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:16.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:03:16.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:03:17.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:17 smithi130 ceph-mon[84265]: pgmap v364: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:17.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:17 smithi102 ceph-mon[111694]: pgmap v364: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:18.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:18 smithi130 ceph-mon[84265]: pgmap v365: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:18.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:18 smithi102 ceph-mon[111694]: pgmap v365: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:21.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:21 smithi130 ceph-mon[84265]: pgmap v366: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:21.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:21 smithi102 ceph-mon[111694]: pgmap v366: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:23.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:23 smithi130 ceph-mon[84265]: pgmap v367: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:23.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:23 smithi102 ceph-mon[111694]: pgmap v367: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:25.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:25 smithi130 ceph-mon[84265]: pgmap v368: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:25.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:25 smithi102 ceph-mon[111694]: pgmap v368: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:27.452 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:27 smithi130 ceph-mon[84265]: pgmap v369: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:27.453 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:27 smithi102 ceph-mon[111694]: pgmap v369: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:29.650 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:29 smithi130 ceph-mon[84265]: pgmap v370: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:29.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:29 smithi102 ceph-mon[111694]: pgmap v370: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:31.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:31 smithi130 ceph-mon[84265]: pgmap v371: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:31.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:03:31.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:31 smithi102 ceph-mon[111694]: pgmap v371: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:31.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:03:33.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:33 smithi130 ceph-mon[84265]: pgmap v372: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:33.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:33 smithi102 ceph-mon[111694]: pgmap v372: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:35.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:35 smithi130 ceph-mon[84265]: pgmap v373: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:35.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:35 smithi102 ceph-mon[111694]: pgmap v373: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:37.480 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:37 smithi102 ceph-mon[111694]: pgmap v374: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:37.592 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:37 smithi130 ceph-mon[84265]: pgmap v374: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:39.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:39 smithi130 ceph-mon[84265]: pgmap v375: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:39.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:39 smithi102 ceph-mon[111694]: pgmap v375: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:41.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:41 smithi130 ceph-mon[84265]: pgmap v376: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:41.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:41 smithi102 ceph-mon[111694]: pgmap v376: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:43.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:43 smithi130 ceph-mon[84265]: pgmap v377: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:43.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:43 smithi102 ceph-mon[111694]: pgmap v377: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:45.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:45 smithi130 ceph-mon[84265]: pgmap v378: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:45.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:45 smithi102 ceph-mon[111694]: pgmap v378: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:46.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:03:46.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:03:47.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:47 smithi130 ceph-mon[84265]: pgmap v379: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:47.677 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:47 smithi102 ceph-mon[111694]: pgmap v379: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:49.651 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:49 smithi130 ceph-mon[84265]: pgmap v380: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:49.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:49 smithi102 ceph-mon[111694]: pgmap v380: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:51.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:51 smithi130 ceph-mon[84265]: pgmap v381: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:51.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:51 smithi102 ceph-mon[111694]: pgmap v381: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:03:53.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:53 smithi130 ceph-mon[84265]: pgmap v382: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:53.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:53 smithi102 ceph-mon[111694]: pgmap v382: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:55.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:55 smithi130 ceph-mon[84265]: pgmap v383: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:55.678 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:55 smithi102 ceph-mon[111694]: pgmap v383: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:57.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:57 smithi130 ceph-mon[84265]: pgmap v384: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:57.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:57 smithi102 ceph-mon[111694]: pgmap v384: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:03:59.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:03:59 smithi130 ceph-mon[84265]: pgmap v385: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:03:59.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:03:59 smithi102 ceph-mon[111694]: pgmap v385: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:01.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:01 smithi130 ceph-mon[84265]: pgmap v386: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:01.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:04:01.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:01 smithi102 ceph-mon[111694]: pgmap v386: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:01.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:04:03.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:03 smithi130 ceph-mon[84265]: pgmap v387: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:03.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:03 smithi102 ceph-mon[111694]: pgmap v387: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:05.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:05 smithi130 ceph-mon[84265]: pgmap v388: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:05.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:05 smithi102 ceph-mon[111694]: pgmap v388: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:06.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:06 smithi130 ceph-mon[84265]: pgmap v389: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:06.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:06 smithi102 ceph-mon[111694]: pgmap v389: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:09.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:09 smithi130 ceph-mon[84265]: pgmap v390: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:09.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:09 smithi102 ceph-mon[111694]: pgmap v390: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:11.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:11 smithi130 ceph-mon[84265]: pgmap v391: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:11.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:11 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:04:11.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:11 smithi102 ceph-mon[111694]: pgmap v391: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:11.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:11 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:04:12.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:04:12.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:04:12.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:04:12.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:04:12.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:04:12.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:04:13.652 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:13 smithi130 ceph-mon[84265]: pgmap v392: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:13.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:13 smithi102 ceph-mon[111694]: pgmap v392: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:15.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:15 smithi130 ceph-mon[84265]: pgmap v393: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:15.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:15 smithi102 ceph-mon[111694]: pgmap v393: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:16.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:04:16.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:04:17.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:17 smithi130 ceph-mon[84265]: pgmap v394: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:17.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:17 smithi102 ceph-mon[111694]: pgmap v394: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:19.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:19 smithi130 ceph-mon[84265]: pgmap v395: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:19.679 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:19 smithi102 ceph-mon[111694]: pgmap v395: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:21.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:21 smithi130 ceph-mon[84265]: pgmap v396: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:21.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:21 smithi102 ceph-mon[111694]: pgmap v396: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:23.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:23 smithi130 ceph-mon[84265]: pgmap v397: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:23.430 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:23 smithi102 ceph-mon[111694]: pgmap v397: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:25.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:25 smithi130 ceph-mon[84265]: pgmap v398: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:25.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:25 smithi102 ceph-mon[111694]: pgmap v398: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:27.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:27 smithi130 ceph-mon[84265]: pgmap v399: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:27.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:27 smithi102 ceph-mon[111694]: pgmap v399: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:29.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:29 smithi130 ceph-mon[84265]: pgmap v400: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:29.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:29 smithi102 ceph-mon[111694]: pgmap v400: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:31.653 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:31 smithi130 ceph-mon[84265]: pgmap v401: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:31.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:04:31.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:31 smithi102 ceph-mon[111694]: pgmap v401: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:31.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:04:33.403 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:33 smithi130 ceph-mon[84265]: pgmap v402: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:33.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:33 smithi102 ceph-mon[111694]: pgmap v402: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:35.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:35 smithi130 ceph-mon[84265]: pgmap v403: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:35.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:35 smithi102 ceph-mon[111694]: pgmap v403: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:37.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:37 smithi130 ceph-mon[84265]: pgmap v404: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:37.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:37 smithi102 ceph-mon[111694]: pgmap v404: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:39.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:39 smithi130 ceph-mon[84265]: pgmap v405: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:39.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:39 smithi102 ceph-mon[111694]: pgmap v405: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:41.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:41 smithi130 ceph-mon[84265]: pgmap v406: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:41.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:41 smithi102 ceph-mon[111694]: pgmap v406: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:43.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:43 smithi130 ceph-mon[84265]: pgmap v407: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:43.680 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:43 smithi102 ceph-mon[111694]: pgmap v407: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:45.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:45 smithi130 ceph-mon[84265]: pgmap v408: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:45.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:45 smithi102 ceph-mon[111694]: pgmap v408: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:46.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:04:46.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:04:47.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:47 smithi130 ceph-mon[84265]: pgmap v409: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:47.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:47 smithi102 ceph-mon[111694]: pgmap v409: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:49.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:49 smithi130 ceph-mon[84265]: pgmap v410: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:49.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:49 smithi102 ceph-mon[111694]: pgmap v410: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:51.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:51 smithi130 ceph-mon[84265]: pgmap v411: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:51.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:51 smithi102 ceph-mon[111694]: pgmap v411: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:04:53.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:53 smithi130 ceph-mon[84265]: pgmap v412: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:53.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:53 smithi102 ceph-mon[111694]: pgmap v412: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:54.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:54 smithi130 ceph-mon[84265]: pgmap v413: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:54.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:54 smithi102 ceph-mon[111694]: pgmap v413: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:04:57.654 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:57 smithi130 ceph-mon[84265]: pgmap v414: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:57.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:57 smithi102 ceph-mon[111694]: pgmap v414: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:04:59.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:04:59 smithi130 ceph-mon[84265]: pgmap v415: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-13T23:04:59.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:04:59 smithi102 ceph-mon[111694]: pgmap v415: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-13T23:05:01.660 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:01 smithi130 ceph-mon[84265]: pgmap v416: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T23:05:01.660 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:05:01.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:01 smithi102 ceph-mon[111694]: pgmap v416: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T23:05:01.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:05:03.523 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:03 smithi102 ceph-mon[111694]: pgmap v417: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T23:05:03.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:03 smithi130 ceph-mon[84265]: pgmap v417: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T23:05:05.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:05 smithi130 ceph-mon[84265]: pgmap v418: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-13T23:05:05.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:05 smithi102 ceph-mon[111694]: pgmap v418: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-13T23:05:07.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:07 smithi130 ceph-mon[84265]: pgmap v419: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T23:05:07.681 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:07 smithi102 ceph-mon[111694]: pgmap v419: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-13T23:05:09.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:09 smithi130 ceph-mon[84265]: pgmap v420: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-13T23:05:09.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:09 smithi102 ceph-mon[111694]: pgmap v420: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-13T23:05:11.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:11 smithi130 ceph-mon[84265]: pgmap v421: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:05:11.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:11 smithi102 ceph-mon[111694]: pgmap v421: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:05:12.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:12 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:05:12.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:12 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:05:13.504 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:13 smithi130 ceph-mon[84265]: pgmap v422: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:13.504 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:13 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:05:13.505 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:13 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:05:13.505 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:13 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:05:13.637 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:13 smithi102 ceph-mon[111694]: pgmap v422: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:13.637 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:13 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:05:13.637 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:13 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:05:13.638 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:13 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:05:15.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:15 smithi130 ceph-mon[84265]: pgmap v423: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:15.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:15 smithi102 ceph-mon[111694]: pgmap v423: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:16.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:16 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:05:16.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:16 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:05:17.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:17 smithi130 ceph-mon[84265]: pgmap v424: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:17.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:17 smithi102 ceph-mon[111694]: pgmap v424: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:19.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:19 smithi130 ceph-mon[84265]: pgmap v425: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:19.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:19 smithi102 ceph-mon[111694]: pgmap v425: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:21.655 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:21 smithi130 ceph-mon[84265]: pgmap v426: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:05:21.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:21 smithi102 ceph-mon[111694]: pgmap v426: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:05:23.601 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:23 smithi130 ceph-mon[84265]: pgmap v427: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:23.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:23 smithi102 ceph-mon[111694]: pgmap v427: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:25.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:25 smithi130 ceph-mon[84265]: pgmap v428: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:25.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:25 smithi102 ceph-mon[111694]: pgmap v428: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:27.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:27 smithi130 ceph-mon[84265]: pgmap v429: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:27.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:27 smithi102 ceph-mon[111694]: pgmap v429: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:29.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:29 smithi130 ceph-mon[84265]: pgmap v430: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:29.682 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:29 smithi102 ceph-mon[111694]: pgmap v430: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:31.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:31 smithi130 ceph-mon[84265]: pgmap v431: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:05:31.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:31 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:05:31.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:31 smithi102 ceph-mon[111694]: pgmap v431: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:05:31.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:31 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:05:33.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:33 smithi130 ceph-mon[84265]: pgmap v432: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:33.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:33 smithi102 ceph-mon[111694]: pgmap v432: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:35.658 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:35 smithi130 ceph-mon[84265]: pgmap v433: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:35.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:35 smithi102 ceph-mon[111694]: pgmap v433: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:37.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:37 smithi130 ceph-mon[84265]: pgmap v434: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:37.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:37 smithi102 ceph-mon[111694]: pgmap v434: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:39.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:39 smithi130 ceph-mon[84265]: pgmap v435: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:39.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:39 smithi102 ceph-mon[111694]: pgmap v435: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:41.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:41 smithi130 ceph-mon[84265]: pgmap v436: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:05:41.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:41 smithi102 ceph-mon[111694]: pgmap v436: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:05:42.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:42 smithi130 ceph-mon[84265]: pgmap v437: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:42.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:42 smithi102 ceph-mon[111694]: pgmap v437: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:45.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:45 smithi130 ceph-mon[84265]: pgmap v438: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:45.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:45 smithi102 ceph-mon[111694]: pgmap v438: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:46.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:46 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:05:46.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:46 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:05:47.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:47 smithi130 ceph-mon[84265]: pgmap v439: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:47.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:47 smithi102 ceph-mon[111694]: pgmap v439: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:49.433 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:49 smithi102 ceph-mon[111694]: pgmap v440: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:49.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:49 smithi130 ceph-mon[84265]: pgmap v440: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:51.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:51 smithi130 ceph-mon[84265]: pgmap v441: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:05:51.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:51 smithi102 ceph-mon[111694]: pgmap v441: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:05:53.656 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:53 smithi130 ceph-mon[84265]: pgmap v442: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:53.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:53 smithi102 ceph-mon[111694]: pgmap v442: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:55.657 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:55 smithi130 ceph-mon[84265]: pgmap v443: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:55.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:55 smithi102 ceph-mon[111694]: pgmap v443: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:57.657 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:57 smithi130 ceph-mon[84265]: pgmap v444: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:57.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:57 smithi102 ceph-mon[111694]: pgmap v444: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:05:59.433 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:05:59 smithi102 ceph-mon[111694]: pgmap v445: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:05:59.657 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:05:59 smithi130 ceph-mon[84265]: pgmap v445: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:06:01.657 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:01 smithi130 ceph-mon[84265]: pgmap v446: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:06:01.657 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:01 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:06:01.683 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:01 smithi102 ceph-mon[111694]: pgmap v446: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:06:01.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:01 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-13T23:06:03.657 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:03 smithi130 ceph-mon[84265]: pgmap v447: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:06:03.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:03 smithi102 ceph-mon[111694]: pgmap v447: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:06:05.657 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:05 smithi130 ceph-mon[84265]: pgmap v448: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:06:05.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:05 smithi102 ceph-mon[111694]: pgmap v448: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:06:07.657 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:07 smithi130 ceph-mon[84265]: pgmap v449: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:06:07.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:07 smithi102 ceph-mon[111694]: pgmap v449: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:06:09.407 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:09 smithi130 ceph-mon[84265]: pgmap v450: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:06:09.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:09 smithi102 ceph-mon[111694]: pgmap v450: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:06:11.657 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:11 smithi130 ceph-mon[84265]: pgmap v451: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:06:11.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:11 smithi102 ceph-mon[111694]: pgmap v451: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-13T23:06:13.657 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:13 smithi130 ceph-mon[84265]: pgmap v452: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:06:13.657 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:13 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:06:13.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:13 smithi102 ceph-mon[111694]: pgmap v452: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-13T23:06:13.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:13 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-13T23:06:14.407 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:14 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:06:14.407 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:14 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:06:14.408 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:14 smithi130 ceph-mon[84265]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:06:14.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:14 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-13T23:06:14.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:14 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-13T23:06:14.684 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:14 smithi102 ceph-mon[111694]: from='mgr.34104 172.21.15.102:0/2083781032' entity='mgr.smithi102.xcijrj' 2024-05-13T23:06:14.992 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-05-13T23:06:14.992 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-13T23:06:14.995 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-13T23:06:15.007 INFO:tasks.cephadm:Teardown begin 2024-05-13T23:06:15.007 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa/tasks/cephadm.py", line 2240, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-13T23:06:15.008 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-13T23:06:15.042 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-13T23:06:15.075 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-13T23:06:15.076 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-13T23:06:15.101 DEBUG:teuthology.orchestra.run.smithi130:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-13T23:06:15.133 INFO:tasks.cephadm:Stopping all daemons... 2024-05-13T23:06:15.133 INFO:tasks.cephadm.mon.smithi102:Stopping mon.smithi102... 2024-05-13T23:06:15.134 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi102 2024-05-13T23:06:15.281 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:15 smithi102 systemd[1]: Stopping Ceph mon.smithi102 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T23:06:15.407 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:15 smithi130 ceph-mon[84265]: pgmap v453: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:06:15.559 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:15 smithi102 ceph-mon[111694]: pgmap v453: 65 pgs: 65 active+clean; 212 MiB data, 909 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-13T23:06:15.559 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:15 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102[111690]: 2024-05-13T23:06:15.452+0000 7fa33821a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi102 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T23:06:15.559 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:15 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102[111690]: 2024-05-13T23:06:15.453+0000 7fa33821a700 -1 mon.smithi102@0(leader) e3 *** Got Signal Terminated *** 2024-05-13T23:06:15.934 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:15 smithi102 podman[158527]: 2024-05-13 23:06:15.719094529 +0000 UTC m=+0.362348296 container died 0b15816f3f6bcd70bdc4a8681809b5598c8f715b1f4c5333e3bebdd3e1b25435 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-13T23:06:16.434 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:16 smithi102 podman[158527]: 2024-05-13 23:06:16.195782976 +0000 UTC m=+0.839036785 container cleanup 0b15816f3f6bcd70bdc4a8681809b5598c8f715b1f4c5333e3bebdd3e1b25435 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-c3e5514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-13T23:06:16.434 INFO:journalctl@ceph.mon.smithi102.smithi102.stdout:May 13 23:06:16 smithi102 bash[158527]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi102 2024-05-13T23:06:16.619 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi102.service' 2024-05-13T23:06:16.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-13T23:06:16.669 INFO:tasks.cephadm.mon.smithi102:Stopped mon.smithi102 2024-05-13T23:06:16.669 INFO:tasks.cephadm.mon.smithi130:Stopping mon.smithi130... 2024-05-13T23:06:16.669 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi130 2024-05-13T23:06:17.008 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:16 smithi130 systemd[1]: Stopping Ceph mon.smithi130 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T23:06:17.407 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:17 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130[84261]: 2024-05-13T23:06:17.023+0000 7f11f021a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi130 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T23:06:17.407 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:17 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130[84261]: 2024-05-13T23:06:17.023+0000 7f11f021a700 -1 mon.smithi130@1(peon) e3 *** Got Signal Terminated *** 2024-05-13T23:06:17.408 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:17 smithi130 podman[121127]: 2024-05-13 23:06:17.26946119 +0000 UTC m=+0.381677337 container died 2c1e5923909a1b2fa2ffd8d1fa02f8f5dbef15cb23246814ff80e64c4e3af358 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130, RELEASE=squid-c3e5514, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T23:06:17.908 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:17 smithi130 podman[121127]: 2024-05-13 23:06:17.749018383 +0000 UTC m=+0.861234529 container cleanup 2c1e5923909a1b2fa2ffd8d1fa02f8f5dbef15cb23246814ff80e64c4e3af358 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-13T23:06:17.908 INFO:journalctl@ceph.mon.smithi130.smithi130.stdout:May 13 23:06:17 smithi130 bash[121127]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-mon-smithi130 2024-05-13T23:06:18.164 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@mon.smithi130.service' 2024-05-13T23:06:18.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-13T23:06:18.232 INFO:tasks.cephadm.mon.smithi130:Stopped mon.smithi130 2024-05-13T23:06:18.232 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-13T23:06:18.232 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.0 2024-05-13T23:06:18.551 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:18 smithi102 systemd[1]: Stopping Ceph osd.0 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T23:06:18.934 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:18 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[123642]: 2024-05-13T23:06:18.566+0000 7f2e0a4bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T23:06:18.934 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:18 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[123642]: 2024-05-13T23:06:18.566+0000 7f2e0a4bf700 -1 osd.0 96 *** Got signal Terminated *** 2024-05-13T23:06:18.935 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:18 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0[123642]: 2024-05-13T23:06:18.566+0000 7f2e0a4bf700 -1 osd.0 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T23:06:24.311 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:24 smithi102 podman[158661]: 2024-05-13 23:06:24.03812763 +0000 UTC m=+5.596066180 container died 0ada88c777a224b3f5855974176832684e6a7be465b994196cc390d1df0cdde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-13T23:06:24.670 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:24 smithi102 podman[158661]: 2024-05-13 23:06:24.425549593 +0000 UTC m=+5.983488135 container cleanup 0ada88c777a224b3f5855974176832684e6a7be465b994196cc390d1df0cdde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-13T23:06:24.670 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:24 smithi102 bash[158661]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0 2024-05-13T23:06:24.934 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:24 smithi102 podman[158675]: 2024-05-13 23:06:24.685796497 +0000 UTC m=+0.638657865 container remove 0ada88c777a224b3f5855974176832684e6a7be465b994196cc390d1df0cdde7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-c3e5514, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-13T23:06:25.349 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:25 smithi102 podman[158769]: 2024-05-13 23:06:24.963514883 +0000 UTC m=+0.017860234 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T23:06:25.349 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:25 smithi102 podman[158769]: 2024-05-13 23:06:25.086951607 +0000 UTC m=+0.141296972 container create 444adf1f028be2774ff2ecf25690b2eec5fc020f015a39a91d86e497d36a14a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T23:06:25.349 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:25 smithi102 podman[158769]: 2024-05-13 23:06:25.289596665 +0000 UTC m=+0.343942010 container init 444adf1f028be2774ff2ecf25690b2eec5fc020f015a39a91d86e497d36a14a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-13T23:06:25.349 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:25 smithi102 podman[158769]: 2024-05-13 23:06:25.292999071 +0000 UTC m=+0.347344423 container start 444adf1f028be2774ff2ecf25690b2eec5fc020f015a39a91d86e497d36a14a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-13T23:06:25.349 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:25 smithi102 podman[158769]: 2024-05-13 23:06:25.364608224 +0000 UTC m=+0.418953560 container attach 444adf1f028be2774ff2ecf25690b2eec5fc020f015a39a91d86e497d36a14a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, ceph=True) 2024-05-13T23:06:25.684 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:25 smithi102 podman[158769]: 2024-05-13 23:06:25.593052691 +0000 UTC m=+0.647398035 container died 444adf1f028be2774ff2ecf25690b2eec5fc020f015a39a91d86e497d36a14a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, ceph=True, CEPH_POINT_RELEASE=) 2024-05-13T23:06:26.137 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.0.service' 2024-05-13T23:06:26.138 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:26 smithi102 podman[158795]: 2024-05-13 23:06:26.127044165 +0000 UTC m=+0.521623616 container remove 444adf1f028be2774ff2ecf25690b2eec5fc020f015a39a91d86e497d36a14a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T23:06:26.138 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:26 smithi102 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.0.service: Deactivated successfully. 2024-05-13T23:06:26.138 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:26 smithi102 systemd[1]: Stopped Ceph osd.0 for 62275d5c-1179-11ef-bc98-c7b262605968. 2024-05-13T23:06:26.138 INFO:journalctl@ceph.osd.0.smithi102.stdout:May 13 23:06:26 smithi102 systemd[1]: ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.0.service: Consumed 10.901s CPU time. 2024-05-13T23:06:26.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-13T23:06:26.181 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-13T23:06:26.181 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-13T23:06:26.181 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.1 2024-05-13T23:06:26.408 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:26 smithi102 systemd[1]: Stopping Ceph osd.1 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T23:06:26.684 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:26 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[129465]: 2024-05-13T23:06:26.554+0000 7f8a74a0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T23:06:26.685 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:26 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[129465]: 2024-05-13T23:06:26.554+0000 7f8a74a0d700 -1 osd.1 96 *** Got signal Terminated *** 2024-05-13T23:06:26.685 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:26 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1[129465]: 2024-05-13T23:06:26.554+0000 7f8a74a0d700 -1 osd.1 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T23:06:32.185 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:31 smithi102 podman[158876]: 2024-05-13 23:06:31.93627592 +0000 UTC m=+5.504156618 container died c3e9e68b06fd3826d5f699ef464e00e44c4540f0cf1d8c4dcd755e148579e7d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T23:06:32.566 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:32 smithi102 podman[158876]: 2024-05-13 23:06:32.288474607 +0000 UTC m=+5.856355345 container cleanup c3e9e68b06fd3826d5f699ef464e00e44c4540f0cf1d8c4dcd755e148579e7d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=squid-c3e5514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T23:06:32.566 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:32 smithi102 bash[158876]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1 2024-05-13T23:06:32.908 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:32 smithi102 podman[158904]: 2024-05-13 23:06:32.581413745 +0000 UTC m=+0.639014718 container remove c3e9e68b06fd3826d5f699ef464e00e44c4540f0cf1d8c4dcd755e148579e7d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True) 2024-05-13T23:06:33.185 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:32 smithi102 podman[158990]: 2024-05-13 23:06:32.826401078 +0000 UTC m=+0.017295615 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T23:06:33.185 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:32 smithi102 podman[158990]: 2024-05-13 23:06:32.932320856 +0000 UTC m=+0.123215369 container create 7fd657e9a11476049d6d316d02e9b0daeaf7020887c982f9da347f2b935f67e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-c3e5514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-13T23:06:33.185 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:33 smithi102 podman[158990]: 2024-05-13 23:06:33.160498276 +0000 UTC m=+0.351392801 container init 7fd657e9a11476049d6d316d02e9b0daeaf7020887c982f9da347f2b935f67e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-13T23:06:33.185 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:33 smithi102 podman[158990]: 2024-05-13 23:06:33.164342903 +0000 UTC m=+0.355237425 container start 7fd657e9a11476049d6d316d02e9b0daeaf7020887c982f9da347f2b935f67e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=squid-c3e5514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-13T23:06:33.685 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:33 smithi102 podman[158990]: 2024-05-13 23:06:33.252123343 +0000 UTC m=+0.443017874 container attach 7fd657e9a11476049d6d316d02e9b0daeaf7020887c982f9da347f2b935f67e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-13T23:06:33.685 INFO:journalctl@ceph.osd.1.smithi102.stdout:May 13 23:06:33 smithi102 podman[158990]: 2024-05-13 23:06:33.456082987 +0000 UTC m=+0.646977506 container died 7fd657e9a11476049d6d316d02e9b0daeaf7020887c982f9da347f2b935f67e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-13T23:06:34.056 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.1.service' 2024-05-13T23:06:34.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-13T23:06:34.097 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-13T23:06:34.098 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-13T23:06:34.098 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.2 2024-05-13T23:06:34.427 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:34 smithi102 systemd[1]: Stopping Ceph osd.2 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T23:06:34.685 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:34 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[134846]: 2024-05-13T23:06:34.441+0000 7fbd8066c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T23:06:34.685 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:34 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[134846]: 2024-05-13T23:06:34.441+0000 7fbd8066c700 -1 osd.2 96 *** Got signal Terminated *** 2024-05-13T23:06:34.685 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:34 smithi102 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2[134846]: 2024-05-13T23:06:34.441+0000 7fbd8066c700 -1 osd.2 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T23:06:40.073 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:40 smithi102 podman[159096]: 2024-05-13 23:06:40.008130005 +0000 UTC m=+5.669092414 container died 1b85728dccf5a1959cbab622b485fec7fd27d1341d8d84c032313d317121bfe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-c3e5514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-13T23:06:40.643 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:40 smithi102 podman[159096]: 2024-05-13 23:06:40.392551598 +0000 UTC m=+6.053514009 container cleanup 1b85728dccf5a1959cbab622b485fec7fd27d1341d8d84c032313d317121bfe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-c3e5514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T23:06:40.643 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:40 smithi102 bash[159096]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2 2024-05-13T23:06:40.935 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:40 smithi102 podman[159115]: 2024-05-13 23:06:40.658303858 +0000 UTC m=+0.650302888 container remove 1b85728dccf5a1959cbab622b485fec7fd27d1341d8d84c032313d317121bfe5 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2, GIT_CLEAN=True, RELEASE=squid-c3e5514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-13T23:06:41.324 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:41 smithi102 podman[159214]: 2024-05-13 23:06:41.053013233 +0000 UTC m=+0.111736707 container create 02f508dfbded5b9d0eb3133c100ad3ec8ca33294222821538f0505df74bf35df (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-c3e5514, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-13T23:06:41.324 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:41 smithi102 podman[159214]: 2024-05-13 23:06:40.959274024 +0000 UTC m=+0.017997511 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T23:06:41.325 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:41 smithi102 podman[159214]: 2024-05-13 23:06:41.26398723 +0000 UTC m=+0.322710706 container init 02f508dfbded5b9d0eb3133c100ad3ec8ca33294222821538f0505df74bf35df (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-c3e5514) 2024-05-13T23:06:41.325 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:41 smithi102 podman[159214]: 2024-05-13 23:06:41.267365724 +0000 UTC m=+0.326089199 container start 02f508dfbded5b9d0eb3133c100ad3ec8ca33294222821538f0505df74bf35df (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-13T23:06:41.325 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:41 smithi102 podman[159214]: 2024-05-13 23:06:41.338906546 +0000 UTC m=+0.397630020 container attach 02f508dfbded5b9d0eb3133c100ad3ec8ca33294222821538f0505df74bf35df (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-13T23:06:41.685 INFO:journalctl@ceph.osd.2.smithi102.stdout:May 13 23:06:41 smithi102 podman[159214]: 2024-05-13 23:06:41.558736334 +0000 UTC m=+0.617459808 container died 02f508dfbded5b9d0eb3133c100ad3ec8ca33294222821538f0505df74bf35df (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-13T23:06:42.050 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.2.service' 2024-05-13T23:06:42.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-13T23:06:42.091 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-13T23:06:42.091 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-13T23:06:42.091 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.3 2024-05-13T23:06:42.408 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:42 smithi130 systemd[1]: Stopping Ceph osd.3 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T23:06:42.908 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:42 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[93441]: 2024-05-13T23:06:42.435+0000 7f55c0774700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T23:06:42.909 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:42 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[93441]: 2024-05-13T23:06:42.435+0000 7f55c0774700 -1 osd.3 96 *** Got signal Terminated *** 2024-05-13T23:06:42.909 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:42 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[93441]: 2024-05-13T23:06:42.435+0000 7f55c0774700 -1 osd.3 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T23:06:46.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:46 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:46.092+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:47.090 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:46 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3[93441]: 2024-05-13T23:06:46.795+0000 7f55bd569700 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:21.666076+0000 front 2024-05-13T23:06:21.666157+0000 (oldest deadline 2024-05-13T23:06:46.366007+0000) 2024-05-13T23:06:47.091 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:46 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:46.737+0000 7ff4c5279700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:21.545150+0000 front 2024-05-13T23:06:21.545195+0000 (oldest deadline 2024-05-13T23:06:46.145624+0000) 2024-05-13T23:06:47.408 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:47 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:47.104+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:48.043 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:47 smithi130 podman[121262]: 2024-05-13 23:06:47.80746649 +0000 UTC m=+5.505042000 container died 66e804d4b801eac156e2356e42f8aad96012b9c00a0e0a87d4fe31d89693f480 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-c3e5514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-13T23:06:48.043 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:47 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:47.775+0000 7ff4c5279700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:21.545150+0000 front 2024-05-13T23:06:21.545195+0000 (oldest deadline 2024-05-13T23:06:46.145624+0000) 2024-05-13T23:06:48.043 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:48 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:48.056+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:48.408 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:48 smithi130 podman[121262]: 2024-05-13 23:06:48.168581196 +0000 UTC m=+5.866156706 container cleanup 66e804d4b801eac156e2356e42f8aad96012b9c00a0e0a87d4fe31d89693f480 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-13T23:06:48.409 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:48 smithi130 bash[121262]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3 2024-05-13T23:06:48.754 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:48 smithi130 podman[121278]: 2024-05-13 23:06:48.428100638 +0000 UTC m=+0.613107121 container remove 66e804d4b801eac156e2356e42f8aad96012b9c00a0e0a87d4fe31d89693f480 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-13T23:06:49.050 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:48 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:48.768+0000 7ff4c5279700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:21.545150+0000 front 2024-05-13T23:06:21.545195+0000 (oldest deadline 2024-05-13T23:06:46.145624+0000) 2024-05-13T23:06:49.051 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:48 smithi130 podman[121368]: 2024-05-13 23:06:48.787459931 +0000 UTC m=+0.101222143 container create a111ca6ddf276ca8ff545e87b4ee618a21759558b9ac16524028d4c9d0b31d71 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-c3e5514, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-13T23:06:49.051 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:48 smithi130 podman[121368]: 2024-05-13 23:06:48.703475133 +0000 UTC m=+0.017237360 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T23:06:49.051 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:48 smithi130 podman[121368]: 2024-05-13 23:06:48.990374949 +0000 UTC m=+0.304137173 container init a111ca6ddf276ca8ff545e87b4ee618a21759558b9ac16524028d4c9d0b31d71 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-13T23:06:49.051 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:48 smithi130 podman[121368]: 2024-05-13 23:06:48.994171096 +0000 UTC m=+0.307933308 container start a111ca6ddf276ca8ff545e87b4ee618a21759558b9ac16524028d4c9d0b31d71 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-deactivate, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-13T23:06:49.051 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:49 smithi130 podman[121368]: 2024-05-13 23:06:49.064981258 +0000 UTC m=+0.378743485 container attach a111ca6ddf276ca8ff545e87b4ee618a21759558b9ac16524028d4c9d0b31d71 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-c3e5514) 2024-05-13T23:06:49.328 INFO:journalctl@ceph.osd.3.smithi130.stdout:May 13 23:06:49 smithi130 podman[121368]: 2024-05-13 23:06:49.304019829 +0000 UTC m=+0.617782048 container died a111ca6ddf276ca8ff545e87b4ee618a21759558b9ac16524028d4c9d0b31d71 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-c3e5514) 2024-05-13T23:06:49.329 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:49 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:49.065+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:49.896 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:49 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:49.772+0000 7ff4c5279700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:21.545150+0000 front 2024-05-13T23:06:21.545195+0000 (oldest deadline 2024-05-13T23:06:46.145624+0000) 2024-05-13T23:06:49.922 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.3.service' 2024-05-13T23:06:49.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-13T23:06:49.962 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-13T23:06:49.962 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-13T23:06:49.962 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.4 2024-05-13T23:06:50.159 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:50 smithi130 systemd[1]: Stopping Ceph osd.4 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T23:06:50.159 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:50 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:50.098+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:50.658 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:50 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:50.330+0000 7ff4c8484700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T23:06:50.659 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:50 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:50.330+0000 7ff4c8484700 -1 osd.4 96 *** Got signal Terminated *** 2024-05-13T23:06:50.659 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:50 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:50.330+0000 7ff4c8484700 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T23:06:51.059 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:50 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:50.764+0000 7ff4c5279700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:21.545150+0000 front 2024-05-13T23:06:21.545195+0000 (oldest deadline 2024-05-13T23:06:46.145624+0000) 2024-05-13T23:06:51.408 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:51 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:51.073+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:52.102 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:51 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:51.768+0000 7ff4c5279700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:21.545150+0000 front 2024-05-13T23:06:21.545195+0000 (oldest deadline 2024-05-13T23:06:46.145624+0000) 2024-05-13T23:06:52.408 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:52 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:52.115+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:53.132 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:52 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:52.742+0000 7ff4c5279700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:21.545150+0000 front 2024-05-13T23:06:21.545195+0000 (oldest deadline 2024-05-13T23:06:46.145624+0000) 2024-05-13T23:06:53.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:53 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:53.146+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:53.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:53 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:53.146+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:30.656555+0000 front 2024-05-13T23:06:30.656538+0000 (oldest deadline 2024-05-13T23:06:52.356349+0000) 2024-05-13T23:06:54.102 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:53 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:53.784+0000 7ff4c5279700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:21.545150+0000 front 2024-05-13T23:06:21.545195+0000 (oldest deadline 2024-05-13T23:06:46.145624+0000) 2024-05-13T23:06:54.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:54 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:54.115+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:54.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:54 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:54.115+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:30.656555+0000 front 2024-05-13T23:06:30.656538+0000 (oldest deadline 2024-05-13T23:06:52.356349+0000) 2024-05-13T23:06:55.063 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:54 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:54.760+0000 7ff4c5279700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:21.545150+0000 front 2024-05-13T23:06:21.545195+0000 (oldest deadline 2024-05-13T23:06:46.145624+0000) 2024-05-13T23:06:55.063 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:54 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4[98607]: 2024-05-13T23:06:54.760+0000 7ff4c5279700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:31.446536+0000 front 2024-05-13T23:06:31.446551+0000 (oldest deadline 2024-05-13T23:06:54.346556+0000) 2024-05-13T23:06:55.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:55 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:55.076+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:55.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:55 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:55.076+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:30.656555+0000 front 2024-05-13T23:06:30.656538+0000 (oldest deadline 2024-05-13T23:06:52.356349+0000) 2024-05-13T23:06:55.967 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:55 smithi130 podman[121486]: 2024-05-13 23:06:55.717744232 +0000 UTC m=+5.489759384 container died 5bb5a8f94042c809b83d3e35fe89b3a5e23a860495bb291bb0210f93a8946e20 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T23:06:56.350 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:56 smithi130 podman[121486]: 2024-05-13 23:06:56.064064908 +0000 UTC m=+5.836080068 container cleanup 5bb5a8f94042c809b83d3e35fe89b3a5e23a860495bb291bb0210f93a8946e20 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=squid-c3e5514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-13T23:06:56.351 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:56 smithi130 bash[121486]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4 2024-05-13T23:06:56.351 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:56 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:56.056+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:56.351 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:56 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:56.056+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:30.656555+0000 front 2024-05-13T23:06:30.656538+0000 (oldest deadline 2024-05-13T23:06:52.356349+0000) 2024-05-13T23:06:56.659 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:56 smithi130 podman[121502]: 2024-05-13 23:06:56.365299074 +0000 UTC m=+0.648863721 container remove 5bb5a8f94042c809b83d3e35fe89b3a5e23a860495bb291bb0210f93a8946e20 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4, org.label-schema.schema-version=1.0, RELEASE=squid-c3e5514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-13T23:06:57.026 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:56 smithi130 podman[121595]: 2024-05-13 23:06:56.716296845 +0000 UTC m=+0.094162873 container create a446bee61909de2765355cd69e2d982a1abc9d219e6a4b55668ce08932ac316e (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-c3e5514, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-13T23:06:57.026 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:56 smithi130 podman[121595]: 2024-05-13 23:06:56.63961222 +0000 UTC m=+0.017478215 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T23:06:57.026 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:56 smithi130 podman[121595]: 2024-05-13 23:06:56.960203344 +0000 UTC m=+0.338069338 container init a446bee61909de2765355cd69e2d982a1abc9d219e6a4b55668ce08932ac316e (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-c3e5514) 2024-05-13T23:06:57.026 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:56 smithi130 podman[121595]: 2024-05-13 23:06:56.963668581 +0000 UTC m=+0.341534568 container start a446bee61909de2765355cd69e2d982a1abc9d219e6a4b55668ce08932ac316e (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T23:06:57.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:57 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:57.039+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:57.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:57 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:57.039+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:30.656555+0000 front 2024-05-13T23:06:30.656538+0000 (oldest deadline 2024-05-13T23:06:52.356349+0000) 2024-05-13T23:06:57.409 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:57 smithi130 podman[121595]: 2024-05-13 23:06:57.043437558 +0000 UTC m=+0.421303553 container attach a446bee61909de2765355cd69e2d982a1abc9d219e6a4b55668ce08932ac316e (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, RELEASE=squid-c3e5514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-13T23:06:57.409 INFO:journalctl@ceph.osd.4.smithi130.stdout:May 13 23:06:57 smithi130 podman[121595]: 2024-05-13 23:06:57.258374979 +0000 UTC m=+0.636240969 container died a446bee61909de2765355cd69e2d982a1abc9d219e6a4b55668ce08932ac316e (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-c3e5514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-13T23:06:57.833 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.4.service' 2024-05-13T23:06:57.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-13T23:06:57.873 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-13T23:06:57.873 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-13T23:06:57.873 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.5 2024-05-13T23:06:58.159 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:57 smithi130 systemd[1]: Stopping Ceph osd.5 for 62275d5c-1179-11ef-bc98-c7b262605968... 2024-05-13T23:06:58.159 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:58 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:58.030+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:58.159 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:58 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:58.030+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:30.656555+0000 front 2024-05-13T23:06:30.656538+0000 (oldest deadline 2024-05-13T23:06:52.356349+0000) 2024-05-13T23:06:58.659 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:58 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:58.205+0000 7f4ad6106700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-13T23:06:58.659 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:58 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:58.205+0000 7f4ad6106700 -1 osd.5 96 *** Got signal Terminated *** 2024-05-13T23:06:58.659 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:58 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:58.205+0000 7f4ad6106700 -1 osd.5 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-13T23:06:59.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:59 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:59.059+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:06:59.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:06:59 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:06:59.059+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:30.656555+0000 front 2024-05-13T23:06:30.656538+0000 (oldest deadline 2024-05-13T23:06:52.356349+0000) 2024-05-13T23:07:00.159 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:00 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:07:00.029+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:07:00.159 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:00 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:07:00.029+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:30.656555+0000 front 2024-05-13T23:06:30.656538+0000 (oldest deadline 2024-05-13T23:06:52.356349+0000) 2024-05-13T23:07:01.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:01 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:07:01.051+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:07:01.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:01 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:07:01.051+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:30.656555+0000 front 2024-05-13T23:06:30.656538+0000 (oldest deadline 2024-05-13T23:06:52.356349+0000) 2024-05-13T23:07:01.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:01 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:07:01.051+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6822 osd.2 since back 2024-05-13T23:06:37.657228+0000 front 2024-05-13T23:06:37.657209+0000 (oldest deadline 2024-05-13T23:07:00.557179+0000) 2024-05-13T23:07:02.410 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:02 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:07:02.097+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:07:02.410 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:02 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:07:02.097+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:30.656555+0000 front 2024-05-13T23:06:30.656538+0000 (oldest deadline 2024-05-13T23:06:52.356349+0000) 2024-05-13T23:07:02.410 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:02 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:07:02.097+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6822 osd.2 since back 2024-05-13T23:06:37.657228+0000 front 2024-05-13T23:06:37.657209+0000 (oldest deadline 2024-05-13T23:07:00.557179+0000) 2024-05-13T23:07:03.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:03 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:07:03.089+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6806 osd.0 since back 2024-05-13T23:06:20.055860+0000 front 2024-05-13T23:06:20.055872+0000 (oldest deadline 2024-05-13T23:06:45.955721+0000) 2024-05-13T23:07:03.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:03 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:07:03.089+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6814 osd.1 since back 2024-05-13T23:06:30.656555+0000 front 2024-05-13T23:06:30.656538+0000 (oldest deadline 2024-05-13T23:06:52.356349+0000) 2024-05-13T23:07:03.409 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:03 smithi130 ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5[103922]: 2024-05-13T23:07:03.089+0000 7f4aceef3700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.102:6822 osd.2 since back 2024-05-13T23:06:37.657228+0000 front 2024-05-13T23:06:37.657209+0000 (oldest deadline 2024-05-13T23:07:00.557179+0000) 2024-05-13T23:07:03.793 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:03 smithi130 podman[121699]: 2024-05-13 23:07:03.520288889 +0000 UTC m=+5.403057766 container died 5d983135967dfba3d8086f23b0b4d81fbfb968ed66316d0c7abf2c53810eb270 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514) 2024-05-13T23:07:03.793 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:03 smithi130 podman[121699]: 2024-05-13 23:07:03.807655928 +0000 UTC m=+5.690424784 container cleanup 5d983135967dfba3d8086f23b0b4d81fbfb968ed66316d0c7abf2c53810eb270 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-13T23:07:04.159 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:03 smithi130 bash[121699]: ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5 2024-05-13T23:07:04.159 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:04 smithi130 podman[121721]: 2024-05-13 23:07:04.042242754 +0000 UTC m=+0.517976378 container remove 5d983135967dfba3d8086f23b0b4d81fbfb968ed66316d0c7abf2c53810eb270 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-c3e5514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-13T23:07:04.659 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:04 smithi130 podman[121808]: 2024-05-13 23:07:04.288053891 +0000 UTC m=+0.018120790 image pull b1b56bed7c9161afc4ceb5e22f8d8cb1f63f2c9c4e5704fa5bbbc460745d969b quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64 2024-05-13T23:07:04.659 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:04 smithi130 podman[121808]: 2024-05-13 23:07:04.399025607 +0000 UTC m=+0.129092508 container create 437b76d55845d5a82ff07c00c49197b6d4aaec5f7afd2074012e7fddd0ab4405 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c3e5514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-13T23:07:04.659 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:04 smithi130 podman[121808]: 2024-05-13 23:07:04.612443386 +0000 UTC m=+0.342510288 container init 437b76d55845d5a82ff07c00c49197b6d4aaec5f7afd2074012e7fddd0ab4405 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-c3e5514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-13T23:07:04.659 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:04 smithi130 podman[121808]: 2024-05-13 23:07:04.61585931 +0000 UTC m=+0.345926214 container start 437b76d55845d5a82ff07c00c49197b6d4aaec5f7afd2074012e7fddd0ab4405 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-13T23:07:05.159 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:04 smithi130 podman[121808]: 2024-05-13 23:07:04.682345775 +0000 UTC m=+0.412412681 container attach 437b76d55845d5a82ff07c00c49197b6d4aaec5f7afd2074012e7fddd0ab4405 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=squid-c3e5514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-05-13T23:07:05.159 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:04 smithi130 podman[121808]: 2024-05-13 23:07:04.920598203 +0000 UTC m=+0.650665098 container died 437b76d55845d5a82ff07c00c49197b6d4aaec5f7afd2074012e7fddd0ab4405 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=squid-c3e5514, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-13T23:07:05.436 INFO:journalctl@ceph.osd.5.smithi130.stdout:May 13 23:07:05 smithi130 podman[121834]: 2024-05-13 23:07:05.432387849 +0000 UTC m=+0.505140695 container remove 437b76d55845d5a82ff07c00c49197b6d4aaec5f7afd2074012e7fddd0ab4405 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cd05a64e34795b62d7224161b7c229af8ae91396504d4169d8967902985ae64, name=ceph-62275d5c-1179-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-c3e5514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-13T23:07:05.439 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-62275d5c-1179-11ef-bc98-c7b262605968@osd.5.service' 2024-05-13T23:07:05.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-13T23:07:05.478 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-13T23:07:05.478 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 62275d5c-1179-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-13T23:07:05.640 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T23:07:09.879 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi102.stderr:ceph-fuse[76488]: fuse finished with error 0 and tester_r 0 2024-05-13T23:07:25.837 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 62275d5c-1179-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-13T23:07:25.994 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T23:07:39.040 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-13T23:07:39.074 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-13T23:07:39.101 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-13T23:07:39.103 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028/remote/smithi102/crash 2024-05-13T23:07:39.104 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/crash -- . 2024-05-13T23:07:39.142 INFO:teuthology.orchestra.run.smithi102.stderr:tar: /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-13T23:07:39.142 INFO:teuthology.orchestra.run.smithi102.stderr:tar: Error is not recoverable: exiting now 2024-05-13T23:07:39.144 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028/remote/smithi130/crash 2024-05-13T23:07:39.145 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/crash -- . 2024-05-13T23:07:39.169 INFO:teuthology.orchestra.run.smithi130.stderr:tar: /var/lib/ceph/62275d5c-1179-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-13T23:07:39.169 INFO:teuthology.orchestra.run.smithi130.stderr:tar: Error is not recoverable: exiting now 2024-05-13T23:07:39.170 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-13T23:07:39.170 DEBUG:teuthology.orchestra.run.smithi102:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-05-13T23:07:39.223 INFO:tasks.cephadm:Compressing logs... 2024-05-13T23:07:39.224 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-13T23:07:39.266 DEBUG:teuthology.orchestra.run.smithi130:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-13T23:07:39.288 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-13T23:07:39.288 INFO:teuthology.orchestra.run.smithi102.stderr:: No such file or directory 2024-05-13T23:07:39.289 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-13T23:07:39.290 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.log 2024-05-13T23:07:39.290 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-13T23:07:39.290 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi102.sdylgv.log 2024-05-13T23:07:39.290 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-13T23:07:39.297 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.2.log: /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi102.sdylgv.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mgr.smithi102.xcijrj.log 2024-05-13T23:07:39.297 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-13T23:07:39.297 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mgr.smithi102.xcijrj.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-13T23:07:39.298 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.1.log: 88.5% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-13T23:07:39.298 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi102.log 2024-05-13T23:07:39.298 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.0.log: 91.5% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-13T23:07:39.299 INFO:teuthology.orchestra.run.smithi130.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-05-13T23:07:39.299 INFO:teuthology.orchestra.run.smithi130.stderr:: No such file or directory 2024-05-13T23:07:39.299 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-13T23:07:39.299 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi130.log 2024-05-13T23:07:39.299 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.log 2024-05-13T23:07:39.299 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi130.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-13T23:07:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-13T23:07:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-13T23:07:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mon.smithi130.log 2024-05-13T23:07:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-13T23:07:39.300 INFO:teuthology.orchestra.run.smithi130.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-13T23:07:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mon.smithi130.log: 94.2% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi130.log.gz 2024-05-13T23:07:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi130.hzjanp.log 2024-05-13T23:07:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-13T23:07:39.300 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi130.hzjanp.log: 88.5% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-13T23:07:39.303 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi130.ovlxvg.log 2024-05-13T23:07:39.303 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.cephadm.log: 85.9% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-13T23:07:39.303 INFO:teuthology.orchestra.run.smithi130.stderr: 91.6% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-13T23:07:39.303 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-13T23:07:39.303 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi130.ovlxvg.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mgr.smithi130.uoqoxl.log 2024-05-13T23:07:39.304 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-volume.log: 92.6% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi130.hzjanp.log.gz 2024-05-13T23:07:39.307 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-13T23:07:39.309 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi102.log: 94.2% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-client.ceph-exporter.smithi102.log.gz 2024-05-13T23:07:39.314 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mgr.smithi130.uoqoxl.log: 91.0% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-13T23:07:39.316 INFO:teuthology.orchestra.run.smithi102.stderr: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-13T23:07:39.320 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi102.cgkcwz.log 2024-05-13T23:07:39.320 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-13T23:07:39.320 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi102.cgkcwz.log: 86.0% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-13T23:07:39.330 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mon.smithi102.log 2024-05-13T23:07:39.350 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-volume.log: /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mon.smithi102.log: 91.0% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-13T23:07:39.359 INFO:teuthology.orchestra.run.smithi130.stderr: 89.3% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mgr.smithi130.uoqoxl.log.gz 2024-05-13T23:07:39.736 INFO:teuthology.orchestra.run.smithi102.stderr: 88.7% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mgr.smithi102.xcijrj.log.gz 2024-05-13T23:07:39.782 INFO:teuthology.orchestra.run.smithi130.stderr: 92.5% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mon.smithi130.log.gz 2024-05-13T23:07:40.649 INFO:teuthology.orchestra.run.smithi102.stderr: 90.5% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mon.smithi102.log.gz 2024-05-13T23:07:44.432 INFO:teuthology.orchestra.run.smithi102.stderr: 93.6% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-13T23:07:44.692 INFO:teuthology.orchestra.run.smithi130.stderr: 93.5% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-13T23:07:45.420 INFO:teuthology.orchestra.run.smithi130.stderr: 93.2% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-13T23:07:46.126 INFO:teuthology.orchestra.run.smithi130.stderr: 93.4% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-13T23:07:46.260 INFO:teuthology.orchestra.run.smithi102.stderr: 93.3% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-13T23:07:46.490 INFO:teuthology.orchestra.run.smithi102.stderr: 93.5% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-13T23:07:46.954 INFO:teuthology.orchestra.run.smithi130.stderr: 94.8% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi130.ovlxvg.log.gz 2024-05-13T23:07:54.029 INFO:teuthology.orchestra.run.smithi130.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-05-13T23:07:54.029 INFO:teuthology.orchestra.run.smithi130.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-05-13T23:07:54.030 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-05-13T23:07:54.030 INFO:teuthology.orchestra.run.smithi130.stderr:real 0m14.750s 2024-05-13T23:07:54.030 INFO:teuthology.orchestra.run.smithi130.stderr:user 0m40.047s 2024-05-13T23:07:54.030 INFO:teuthology.orchestra.run.smithi130.stderr:sys 0m1.241s 2024-05-13T23:07:54.567 INFO:teuthology.orchestra.run.smithi102.stderr: 94.8% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi102.cgkcwz.log.gz 2024-05-13T23:07:55.786 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /var/log/ceph/ceph-client.0.log: file size changed while zipping 2024-05-13T23:07:55.982 INFO:teuthology.orchestra.run.smithi102.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-13T23:09:10.004 INFO:teuthology.orchestra.run.smithi102.stderr: 92.8% -- replaced with /var/log/ceph/62275d5c-1179-11ef-bc98-c7b262605968/ceph-mds.cephfs.smithi102.sdylgv.log.gz 2024-05-13T23:09:10.006 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-05-13T23:09:10.006 INFO:teuthology.orchestra.run.smithi102.stderr:real 1m30.723s 2024-05-13T23:09:10.006 INFO:teuthology.orchestra.run.smithi102.stderr:user 2m18.916s 2024-05-13T23:09:10.006 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m4.391s 2024-05-13T23:09:10.006 INFO:tasks.cephadm:Archiving logs... 2024-05-13T23:09:10.007 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028/remote/smithi102/log 2024-05-13T23:09:10.008 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-13T23:09:30.959 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028/remote/smithi130/log 2024-05-13T23:09:30.960 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-13T23:09:37.093 INFO:tasks.cephadm:Removing cluster... 2024-05-13T23:09:37.093 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 62275d5c-1179-11ef-bc98-c7b262605968 --force 2024-05-13T23:09:37.246 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T23:09:37.665 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 62275d5c-1179-11ef-bc98-c7b262605968 --force 2024-05-13T23:09:37.822 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: 62275d5c-1179-11ef-bc98-c7b262605968 2024-05-13T23:09:38.114 INFO:tasks.cephadm:Removing cephadm ... 2024-05-13T23:09:38.115 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-13T23:09:38.132 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-13T23:09:38.148 INFO:tasks.cephadm:Teardown complete 2024-05-13T23:09:38.148 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-13T23:09:38.160 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-13T23:09:38.161 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-13T23:09:38.161 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-13T23:09:38.175 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-13T23:09:38.267 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-13T23:09:38.279 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-13T23:09:38.279 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-13T23:09:38.282 DEBUG:teuthology.orchestra.run.smithi130:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-13T23:09:38.294 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-05-13T23:09:38.296 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-05-13T23:09:38.299 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-13T23:09:38.299 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-05-13T23:09:38.299 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 +340us[ +341us] +/- 46ms 2024-05-13T23:09:38.299 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 -739us[ -737us] +/- 68ms 2024-05-13T23:09:38.299 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +304us[ +305us] +/- 82ms 2024-05-13T23:09:38.299 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-13T23:09:38.300 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-13T23:09:38.300 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-05-13T23:09:38.300 INFO:teuthology.orchestra.run.smithi130.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 +325us[ +323us] +/- 46ms 2024-05-13T23:09:38.300 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -734us[ -734us] +/- 68ms 2024-05-13T23:09:38.300 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +295us[ +295us] +/- 82ms 2024-05-13T23:09:38.301 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-13T23:09:38.301 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-13T23:09:38.312 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-13T23:09:38.313 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-13T23:09:38.324 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-13T23:09:38.419 DEBUG:teuthology.orchestra.run.smithi130:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-13T23:09:38.493 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-13T23:09:38.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-13T23:09:38.522 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-13T23:09:38.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-13T23:09:38.551 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-13T23:09:38.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-13T23:09:38.573 INFO:teuthology.task.internal:Duration was 2374.849052 seconds 2024-05-13T23:09:38.573 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-13T23:09:38.583 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-13T23:09:38.583 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-13T23:09:38.586 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-13T23:09:38.622 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-13T23:09:38.630 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-13T23:09:38.983 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-13T23:09:38.983 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-05-13T23:09:38.983 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-13T23:09:39.044 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi130.front.sepia.ceph.com 2024-05-13T23:09:39.045 DEBUG:teuthology.orchestra.run.smithi130:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-13T23:09:39.067 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-13T23:09:39.068 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-13T23:09:39.087 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-13T23:09:39.133 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-13T23:09:39.133 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-13T23:09:39.154 DEBUG:teuthology.orchestra.run.smithi130:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-13T23:09:39.822 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-13T23:09:39.835 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-13T23:09:39.835 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-13T23:09:39.859 DEBUG:teuthology.orchestra.run.smithi130:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-13T23:09:39.886 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-13T23:09:39.897 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-13T23:09:39.905 DEBUG:teuthology.orchestra.run.smithi130:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-13T23:09:39.926 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-05-13T23:09:39.950 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = core 2024-05-13T23:09:39.995 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-13T23:09:40.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-13T23:09:40.011 DEBUG:teuthology.orchestra.run.smithi130:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-13T23:09:40.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-13T23:09:40.051 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-13T23:09:40.063 INFO:teuthology.task.internal:Transferring archived files... 2024-05-13T23:09:40.064 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028/remote/smithi102 2024-05-13T23:09:40.064 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-13T23:09:40.116 DEBUG:teuthology.misc:Transferring archived files from smithi130:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028/remote/smithi130 2024-05-13T23:09:40.116 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-13T23:09:40.154 INFO:teuthology.task.internal:Removing archive directory... 2024-05-13T23:09:40.155 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-13T23:09:40.157 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-13T23:09:40.203 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-13T23:09:40.216 INFO:teuthology.task.internal:Not uploading archives. 2024-05-13T23:09:40.216 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-13T23:09:40.228 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-13T23:09:40.228 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-13T23:09:40.231 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-13T23:09:40.243 INFO:teuthology.orchestra.run.smithi102.stdout: 262165 4 drwxr-xr-x 3 ubuntu ubuntu 4096 May 13 23:09 /home/ubuntu/cephtest 2024-05-13T23:09:40.244 INFO:teuthology.orchestra.run.smithi102.stdout: 556296 4 d--------- 2 ubuntu ubuntu 4096 May 13 22:45 /home/ubuntu/cephtest/mnt.0 2024-05-13T23:09:40.245 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-05-13T23:09:40.245 INFO:teuthology.orchestra.run.smithi102.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-05-13T23:09:40.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-13T23:09:40.248 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_c3e55145f4f4b3a500bb66ee08c938a215bc231d/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi102 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-05-13T23:09:40.248 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-13T23:09:40.262 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-13T23:09:40.275 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-13T23:09:40.276 INFO:teuthology.nuke:Checking targets against current locks 2024-05-13T23:09:40.302 DEBUG:teuthology.nuke:shortname: smithi102 2024-05-13T23:09:40.303 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-13T23:09:40.326 DEBUG:teuthology.nuke:shortname: smithi130 2024-05-13T23:09:40.326 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-13T23:09:40.329 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-13 22:21:08.553861', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFHHykhW0QJ1+PifbqX+AGbMudNyzBH9SP5TnsP/vfMbeIdaBUKSUoXTD/3T8est43rde8+dlA8rxIGSrWzST3I='} 2024-05-13T23:09:40.350 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-13_21:08:02-orch-squid-distro-default-smithi/7705028', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-13 22:21:08.555620', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOMHgsGBASIxa/qjEYlPJwBIJ6EfSpSn/WcPJ3g1i3lk9XMcKXpOsjyBYt4f8UobW6tsqTC55zpw+MljJwAtL5M='} 2024-05-13T23:09:40.379 INFO:teuthology.orchestra.console.smithi102:Power off 2024-05-13T23:09:40.380 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-13T23:09:40.403 DEBUG:teuthology.orchestra.console.smithi102:power off output: Chassis Power Control: Down/Off 2024-05-13T23:09:40.403 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-13T23:09:40.418 INFO:teuthology.orchestra.console.smithi130:Power off 2024-05-13T23:09:40.418 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-13T23:09:40.428 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-05-13T23:09:40.451 DEBUG:teuthology.orchestra.console.smithi130:power off output: Chassis Power Control: Down/Off 2024-05-13T23:09:40.451 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-13T23:09:40.474 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-05-13T23:09:44.429 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-13T23:09:44.475 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-13T23:09:44.540 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2024-05-13T23:09:44.586 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2024-05-13T23:09:48.544 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-13T23:09:48.587 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-13T23:09:48.655 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is off 2024-05-13T23:09:48.697 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is off 2024-05-13T23:09:48.755 INFO:teuthology.orchestra.console.smithi102:Power off completed 2024-05-13T23:09:48.798 INFO:teuthology.orchestra.console.smithi130:Power off completed 2024-05-13T23:09:48.903 INFO:teuthology.run:Summary data: description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2374.8490517139435 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-05-13T23:09:48.903 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-13T23:09:48.986 INFO:teuthology.run:FAIL